Works locally, 504s in staging

(a) The timeout you set is per chunk, not a deadline

The timeouts people reach for in the two dominant Python HTTP libraries are read timeouts that reset every time bytes arrive. The requests documentation defines the read timeout as "the number of seconds that the client will wait between bytes sent from the server" and states plainly that "neither the connect nor read timeouts are wall clock". The httpx documentation defines its read timeout as "the maximum duration to wait for a chunk of data to be received" and offers four knobs - connect, read, write and pool.

The failure: a response that trickles, or a proxy that dribbles keep-alive bytes, blocks your call indefinitely under a setting that reads like a hard deadline. Neither library documents a total-wall-clock option at all, so there is nothing to switch on.

Fix: keep a monotonic clock at the loop level and bail out yourself, or wrap the whole call in an async timeout.

import time
import httpx

DEADLINE_SECONDS = 600.0   # your wall-clock budget, enforced by you

def stream_with_deadline(url, payload):
    started = time.monotonic()
    # 30.0 here is a PER-CHUNK read timeout. It is not a deadline.
    with httpx.stream("POST", url, json=payload,
                      timeout=httpx.Timeout(30.0)) as response:
        for chunk in response.iter_bytes():
            elapsed = time.monotonic() - started
            if elapsed > DEADLINE_SECONDS:
                raise TimeoutError(
                    "wall-clock budget exceeded after "
                    + str(round(elapsed, 1)) + "s"
                )
            yield chunk

# async equivalent:
#   await asyncio.wait_for(do_request(), timeout=DEADLINE_SECONDS)

(b) The unit is not the same in every language

Every one of those SDKs documents a 10 minute default, and each expresses it differently. (Go scopes its 10 minutes to non-streaming Messages requests and documents no default timeout at all for anything else.) Python takes a float in seconds (or an httpx.Timeout). Ruby takes seconds. TypeScript takes milliseconds - its own example reads timeout: 20 * 1000 for twenty seconds. Go takes a time.Duration through option.WithRequestTimeout, and that one is a per-retry timeout rather than a whole-call one. Java takes a Duration, C# a TimeSpan.

The failure: a value copied from a Python service into a TypeScript one is off by a factor of a thousand in the direction that hurts. 600 meant as ten minutes becomes six hundred milliseconds.

Fix: the converter beside the stack table, and the warning this tool raises when a millisecond field holds a number that looks like somebody's seconds value.

(c) The SDK quietly moves its own default

The effective deadline is often not the one you configured. The TypeScript SDK documents that for a large max_tokens on a non-streaming request the default timeout is computed as (60 * 60 * maxTokens) / 128000 seconds, floored at ten minutes and reaching up to sixty. The Java SDK documents the mirror image: the same shape of formula applies to streaming requests, while its non-streaming default scales from a thirty second minimum to a ten minute maximum.

Meanwhile the Python SDK refuses rather than hangs: it "will throw a ValueError if a non-streaming request is expected to take longer than approximately 10 minutes", disabled by passing stream=True or by overriding the timeout. The TypeScript, Go and Java SDKs document the same guard.

Fix: set the timeout explicitly so the default never applies, and tick the rescaling option in section 02 to see what your configured value actually becomes.

(d) Timeouts are retried, so multiply

All six SDKs document the same policy: connection errors, 408, 409, 429 and 5xx are retried automatically, two times by default. The Python and TypeScript docs both add the sentence that catches people out - "requests that time out are retried twice by default". The Ruby docs list timeouts explicitly among the retried conditions.

The failure: your worst-case wall clock is not the timeout, it is the timeout times three. That product is usually what crosses the serverless ceiling or the gateway limit, long after the per-attempt number looked comfortable.

Fix: set the retry count deliberately, and budget the whole loop rather than one attempt. Feed the worst-case backoff wait from retry-backoff-calculator into the backoff field in section 02.