Your average latency is lying to you

A dashboard that shows a single “average response time” line is comforting and almost useless. Latency distributions are long-tailed: most requests are fast, a few are very slow, and the mean lands somewhere that describes neither.

The tail is what people feel

A page that makes twenty backend calls is only as fast as the slowest one. If each call has a 1% chance of hitting your p99, then roughly one page load in five does. The p99 is not an edge case for that page — it is the common case.

What to measure instead

• Percentiles, not means. p50 for the typical experience, p99 and p99.9 for the one people complain about.
• Histograms, not pre-aggregated percentiles. You cannot average two p99s; you can merge two histograms.
• Per-endpoint, per-tenant. A healthy global number can hide one customer having a terrible day.

Where tails come from

Garbage-collection pauses, cold caches, lock contention, a noisy neighbour on the same host, a retry that waited out a full timeout. None of these move the average much. All of them show up immediately at p99.

Set your alerts on the tail, and keep the average for capacity planning, where it belongs.