Handle errors and retries

async function understand(body: unknown, tries = 3) {
  // one key for every try: a retry after a lost response gets the first answer, not a second charge
  const idempotencyKey = crypto.randomUUID();
  for (let i = 0; ; i++) {
    const res = await fetch(`${base}/v1/understand`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${key}`,
        "Content-Type": "application/json",
        "Idempotency-Key": idempotencyKey,
      },
      body: JSON.stringify(body),
      signal: AbortSignal.timeout(10_000),
    });
    const data = await res.json();
    if (res.ok) return data;
    if (![409, 429, 500, 502, 503].includes(res.status) || i + 1 >= tries) {
      throw Object.assign(new Error(data.error.message), {
        status: res.status,
        field: data.error.field,
      });
    }
    // a 409 or 429 says how long to wait; otherwise 0.5 s, then 1 s, plus up to 0.25 s of jitter
    const wait = Number(res.headers.get("retry-after")) * 1000 || 2 ** i * 500;
    await new Promise((r) => setTimeout(r, wait + Math.random() * 250));
  }
}

import random
import time
import uuid

def understand(body, tries=3):
    # one key for every try: a retry after a lost response gets the first answer, not a second charge
    idempotency_key = str(uuid.uuid4())
    for i in range(tries):
        res = requests.post(
            f"{base}/v1/understand",
            headers={"Authorization": f"Bearer {key}", "Idempotency-Key": idempotency_key},
            json=body,
            timeout=10,
        )
        data = res.json()
        if res.ok:
            return data
        if res.status_code not in (409, 429, 500, 502, 503) or i + 1 == tries:
            err = data["error"]
            raise RuntimeError(f"{res.status_code} {err.get('field', '')}: {err['message']}")
        # a 409 or 429 says how long to wait; otherwise 0.5 s, then 1 s, plus up to 0.25 s of jitter
        wait = float(res.headers.get("retry-after") or 2 ** i * 0.5)
        time.sleep(wait + random.random() * 0.25)

# Handle errors and retries

Retry a 409, 429, 500, 502 or 503 with the same `Idempotency-Key`; fix everything else.

Built and tested, not yet deployed. Early access.

## Which errors to retry

- 500, 502 and 503: retry with exponential backoff and jitter, up to three tries. That’s our recommendation; none sends a `Retry-After` header.
- 429: wait for the `Retry-After` header (10 seconds), then retry.
- 409: the first request with your `Idempotency-Key` is still running. Wait for the `Retry-After` header (1 second), then retry with the same key.
- 400, 401 and 413: never retry unchanged. Fix the request.
- 422: the `Idempotency-Key` was used for a different request. Use a new key.
- 402: the account is out of credit. Retrying won’t help.

[Every status and its exact message](/api/docs/errors)

## Make retries safe

Send an `Idempotency-Key` header, the same on every try of one request. If the first try succeeded and only its response was lost, the retry gets that response back, with `idempotent-replayed: true`, and no second model call or charge.

- The key is 1 to 255 printable characters; a UUID works.
- It holds for 24 hours, per account.
- Only a success is kept. After an error, a retry with the same key runs again.

## Timeouts

Set a client timeout. The examples use 10 seconds: an example value, not a measured latency. If your client disconnects, the API stops waiting for the model.

## Code

The retry in TypeScript:

```typescript
async function understand(body: unknown, tries = 3) {
  // one key for every try: a retry after a lost response gets the first answer, not a second charge
  const idempotencyKey = crypto.randomUUID();
  for (let i = 0; ; i++) {
    const res = await fetch(`${base}/v1/understand`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${key}`,
        "Content-Type": "application/json",
        "Idempotency-Key": idempotencyKey,
      },
      body: JSON.stringify(body),
      signal: AbortSignal.timeout(10_000),
    });
    const data = await res.json();
    if (res.ok) return data;
    if (![409, 429, 500, 502, 503].includes(res.status) || i + 1 >= tries) {
      throw Object.assign(new Error(data.error.message), {
        status: res.status,
        field: data.error.field,
      });
    }
    // a 409 or 429 says how long to wait; otherwise 0.5 s, then 1 s, plus up to 0.25 s of jitter
    const wait = Number(res.headers.get("retry-after")) * 1000 || 2 ** i * 500;
    await new Promise((r) => setTimeout(r, wait + Math.random() * 250));
  }
}
```

The retry in Python:

```python
import random
import time
import uuid

def understand(body, tries=3):
    # one key for every try: a retry after a lost response gets the first answer, not a second charge
    idempotency_key = str(uuid.uuid4())
    for i in range(tries):
        res = requests.post(
            f"{base}/v1/understand",
            headers={"Authorization": f"Bearer {key}", "Idempotency-Key": idempotency_key},
            json=body,
            timeout=10,
        )
        data = res.json()
        if res.ok:
            return data
        if res.status_code not in (409, 429, 500, 502, 503) or i + 1 == tries:
            err = data["error"]
            raise RuntimeError(f"{res.status_code} {err.get('field', '')}: {err['message']}")
        # a 409 or 429 says how long to wait; otherwise 0.5 s, then 1 s, plus up to 0.25 s of jitter
        wait = float(res.headers.get("retry-after") or 2 ** i * 0.5)
        time.sleep(wait + random.random() * 0.25)
```

Both read the body as JSON before checking the status, so a reply that isn’t JSON (a proxy’s error page, say) throws instead of retrying, and a client timeout isn’t retried. Adapt them to your stack.

Source: /api/docs/guides/errors-and-retries