Classify the failure
Back off between attempts
For a retryable failure, wait progressively longer between attempts, add jitter so concurrent clients do not retry together, and bound both the attempt count and total time. HonorRetry-After when an endpoint returns it.
Check the selected SDK’s retry settings before adding another retry loop. Nested retry policies can multiply the number of requests.
Avoid duplicate writes
A timeout does not prove the server rejected the request. Before repeating a create operation, determine whether the resource was created. Where an endpoint supports an idempotency key, reuse the same key only for an identical retry. Generate a new key for a new operation. Do not assume that an idempotency header is supported by every API. For Compute, use the examples in Run a fine-tune job and Deploy a model server.Handle streaming and background work
A streaming connection can end after delivering partial output. Avoid treating the partial output as a completed result. Follow the status and retrieval mechanisms described in Background responses.Separate refusals from transport errors
A guardrail can return a refusal in a successful HTTP response. Inspect the output; retrying the same blocked request is not a transport-recovery strategy.Investigate a failure
- Confirm the gateway URL and caller identity.
- Record the request identifier, resource ID, status, and error body without logging credentials.
- Check Rate limits, Budgets, and Resource permissions as appropriate.
- Use CLI debugging to inspect request construction.
- Correlate the request with Usage and auditing.