Splitting an application into microservices turns function calls into network calls, and network calls fail in ways that function calls do not. They are slow, they time out, and they sometimes succeed after the caller has given up. The standard response is a toolkit of resilience patterns, including timeouts, retries, circuit breakers, bulkheads and fallbacks. These patterns are good advice, and most production systems need them. This commentary makes a narrower point. Every one of those mechanisms costs something, either in added delay or in extra load on a struggling dependency, and they interact in ways that are easy to miss. Resilience is therefore a design problem with trade-offs that have to be managed, not a feature to switch on.
Why does the slowest call dominate?
A single user request often fans out to many services, and the response cannot be complete until the slowest relevant call returns. Even if each service is fast most of the time, the chance that at least one is slow rises with the number of calls, so the typical user experience is shaped by the tail of the latency distribution, not by the average. This is why engineers watch high percentiles rather than means. It also explains why a small increase in delay in one deep dependency can show up as a large increase at the edge. Any mechanism that adds to the tail, such as waiting out a long timeout before trying again, has an amplified effect on what users see.
Retries and timeouts can make things worse
A retry is a bet that a failure was transient, and when the bet is wrong it adds load to a service that is already struggling. If several layers each retry a few times, one failed user request can become many requests to the bottom service, a pattern known as retry amplification that can turn a partial slowdown into a full outage. Safer designs retry at one layer, use exponential backoff with jitter so clients do not synchronize, and limit retries with a budget. Timeouts need the same discipline. A timeout that is too long ties up threads and connections while the user waits, and one that is too short fails requests that would have succeeded. Propagating a deadline from the edge down the call chain lets each service know how much time is left.
Circuit breakers, bulkheads and fallbacks
A circuit breaker stops calls to a failing dependency for a period, which protects both the caller and the dependency and lets the caller fail fast. Its parameters decide how well it works. A breaker that trips too readily rejects healthy traffic, and one that resets too eagerly sends a burst of requests at a service that has not recovered. Bulkheads limit the resources any one dependency can consume, so a slow service cannot exhaust the shared pool. Fallbacks, such as serving cached or default data, keep the user experience going, but they must be tested, because a fallback path that is rarely exercised often fails the first time it is needed. Load shedding and graceful degradation belong in the same toolbox.
Why it matters
The practical conclusion is to design resilience against an explicit latency and load budget. Decide how long a request may take, divide that budget across the call chain, and choose retry, timeout and breaker settings that fit within it. Then test them with fault injection and load, observing the behaviour of the whole system rather than each service in isolation. Resilience that has never been exercised under failure is a hypothesis, and in distributed systems the hypothesis is often wrong.
Every safety mechanism in a distributed system spends something, and the bill arrives during the incident.
