I wonder if one can formally model protocols (and thus improve them) with tools from distributed systems and reliability engineering?

This curve goes the other way! The more skewed the distribution, the better the performance, over something of a broad range. This is what makes it so important that workload mix in load tests and benchmarks matches real production traffic: throughput can vary by multiple orders of magnitude, in both directions, depending on things as subtle as key skew in the benchmark workload. This is especially true in relational database-shaped things, but applies to systems of all kinds.

Traffic mix matters - Marc's Blog ↗