Give retries a time budget, not just an attempt count

"Retry five times" leaves an important question unanswered: how long is someone supposed to wait?

Five fast failures can finish almost immediately. Five attempts that each reach a socket timeout can keep a screen stuck for most of a minute. Add backoff and the difference gets larger.

I prefer to calculate the worst-case wait before deciding how many retries a task should get.

A synthetic schedule with one-second initial backoff, doubling up to sixteen seconds. Dots show one seeded jitter example.

Count the original request

Take an initial request followed by five retries. That is six requests in total.

If each request can take three seconds, request time alone can reach eighteen seconds. Backoff ceilings of 1, 2, 4, 8 and 16 seconds add another thirty-one seconds.

maximum wait = 6 * 3 + 1 + 2 + 4 + 8 + 16
             = 49 seconds

This assumes each timeout covers the entire attempt. Some clients have separate connection and read timeouts. Name resolution or a TLS handshake may also sit outside the limit you thought was controlling the call. Check the actual boundaries.

An overall deadline is useful because it limits the complete operation, including waits between attempts.

Add jitter without hiding the ceiling

With full jitter, each delay is sampled uniformly between zero and the current backoff ceiling. The delay is still bounded, while several clients are less likely to retry together.

import random

def retry_delays(retries, base=1.0, cap=16.0):
    for index in range(retries):
        ceiling = min(cap, base * 2 ** index)
        yield random.uniform(0.0, ceiling)

delays = list(retry_delays(5))
assert all(0 <= delay <= ceiling for delay, ceiling in
           zip(delays, [1, 2, 4, 8, 16]))

The expected total backoff in this example is half the sum of the ceilings: 15.5 seconds. That average does not replace the 31-second worst-case bound.

Use a fixed random seed in a repeatable test. Avoid giving every deployed client the same seed, which would recreate the synchronisation jitter was meant to reduce.

Decide which failures deserve another attempt

A brief connection loss might recover. A malformed request usually needs different input. Repeating an authentication failure with the same credentials is rarely useful.

The classification belongs beside the retry policy. Otherwise a generic wrapper can quietly turn a clear error into a long wait.

Writes need another check: could the first attempt have succeeded even though the response was lost? Retrying a read and retrying a payment, counter increment or record creation have different consequences. A stable operation identifier lets a server recognise a repeated write. A new identifier on every retry defeats that protection.

Keep the wait outside the work loop

On a small device, sleeping inside a retry function can stop buttons, DNS responses and a setup page from being served. Store a next-attempt time instead, then return control to the main loop.

A retry state can be small: current attempt, attempt deadline, next-attempt time and last failure. Each loop checks whether work is due. Most iterations do nothing and return promptly.

Use a monotonic clock for these deadlines. If the clock can wrap, use the platform's wrap-safe comparison helpers instead of subtracting raw counters without considering overflow.

I test a failed first attempt, a stalled attempt, a later successful attempt and a deadline that expires during backoff. The interface should remain usable throughout. A retry policy has done its job when recovery is bounded and understandable, not merely when the request eventually succeeds.

Reference: AWS on timeouts, retries and jitter.