Platform
Batches
Send many chat completion requests at once. They run in the background, cost 50% of the list price, and finish within 24 hours.
POST /v1/batches
GET /v1/batches
GET /v1/batches/{id}
POST /v1/batches/{id}/cancel
GET /v1/batches/{id}/resultsCreate a batch
The requests go in the JSON body. There is no file upload step.
curl https://api.utmost.site/v1/batches \
-H "Authorization: Bearer $UTMOST_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"requests": [
{"custom_id": "doc-1", "body": {"model": "auto", "messages": [{"role": "user", "content": "Summarize: ..."}]}},
{"custom_id": "doc-2", "body": {"model": "utmost/llama-4-scout", "messages": [{"role": "user", "content": "Summarize: ..."}]}}
],
"metadata": {"job": "nightly-summaries"}
}'| Field | Notes |
|---|---|
requests |
1 to 50,000 items. |
requests[].custom_id |
1 to 128 characters, unique within the batch. Results are matched by it. |
requests[].body |
A chat completion request. stream must be absent or false. stream_options is ignored. |
completion_window |
Optional. Only "24h". |
metadata |
Optional. String values up to 512 characters. |
The whole request body is limited to 8 MB, which caps large batches before the 50,000-item limit. Split big jobs into several batches.
Each body is validated when you create the batch. One invalid body rejects the whole batch with invalid_request, and param points at it. A duplicate custom_id does the same.
Creating a batch needs a positive balance and room under your spend caps. It counts as one request against your rate limit.
The response is 201 with the batch:
{
"id": "batch_6HsowVhheD3l7E8JGI4V",
"object": "batch",
"endpoint": "/v1/chat/completions",
"status": "in_progress",
"created_at": 1795066929,
"completed_at": null,
"expires_at": 1795153329,
"results_expire_at": null,
"request_counts": { "total": 2, "completed": 0, "failed": 0 },
"cost_usd": 0,
"metadata": { "job": "nightly-summaries" },
"results_url": null
}Status
status |
Meaning |
|---|---|
in_progress |
Requests are queued or running. |
completed |
Every request succeeded or failed. |
cancelled |
You cancelled it. |
expired |
24 hours passed. Unfinished requests failed with expired. |
expires_at is the 24-hour deadline. cost_usd grows as requests finish. results_url is set once the batch is finished.
curl https://api.utmost.site/v1/batches/batch_6HsowVhheD3l7E8JGI4V -H "Authorization: Bearer $UTMOST_API_KEY"GET /v1/batches returns your organization's 20 most recent batches. There is no pagination.
How requests run
- Each request runs through the normal gateway: routing, refusal retry, fallback, hard-limit checks and metering. A batch request can use an auto alias or a pinned model.
x-utmost-variantis not available in batches. Pin an-ablmodel id instead.- Requests run in order of batch creation, then submission order, and are not rate limited.
- Requests that fail with
model_unavailable,upstream_error,upstream_timeout,safety_unavailableorinternal_errorare retried, up to 3 attempts in total. Other errors fail the request at once. - The balance, spend caps and key are checked again for each request. If credit runs out or a cap is reached part-way, the remaining requests fail with
insufficient_creditsorspend_cap_reached. If the key is revoked, they fail withkey_revoked.
Results
curl https://api.utmost.site/v1/batches/batch_6HsowVhheD3l7E8JGI4V/results \
-H "Authorization: Bearer $UTMOST_API_KEY"JSON Lines (application/jsonl), one line per request, in submission order:
{"custom_id":"doc-1","response":{"status_code":200,"body":{"id":"chatcmpl-req_...","object":"chat.completion","model":"utmost/qwen3-30b-a3b","choices":[...],"usage":{...,"cost_usd":0.000004}}},"error":null}
{"custom_id":"doc-2","response":null,"error":{"code":"cancelled","message":"Batch cancelled."}}Each line has response or error, not both. error.code is an API error code, or one of:
| Code | Meaning |
|---|---|
cancelled |
The batch was cancelled before this request ran. |
expired |
The 24-hour window ended before this request ran. |
not_processed |
The request never ran. |
Results are available once the batch is no longer in_progress. Before that, the endpoint returns invalid_request.
Cancel
curl -X POST https://api.utmost.site/v1/batches/batch_6HsowVhheD3l7E8JGI4V/cancel \
-H "Authorization: Bearer $UTMOST_API_KEY"Requests that have not started fail with cancelled. Requests already running finish and are billed. Finished requests keep their results. Cancelling a batch that is not in progress returns it unchanged.
Pricing
Each request is billed at 50% of the served model's list price, including cached-token pricing. You pay only for requests that succeed. See Billing.
Retention
Batch inputs are stored only so they can run. Each request's input is deleted once that request succeeds or fails. Inputs of requests that never ran (cancelled or expired) are kept until the results expire. Results are kept for 7 days after the batch finishes (results_expire_at). After that the results endpoint returns not_found. The batch object itself, with its counts and cost, stays listed.
Webhook
When a batch finishes, whether it completed, was cancelled or expired, we send batch.completed:
{
"type": "batch.completed",
"timestamp": "2026-11-19T05:42:27.000Z",
"data": { "batch_id": "batch_6HsowVhheD3l7E8JGI4V", "status": "completed", "completed": 2, "failed": 0, "total": 2, "cost_usd": 0.000009 }
}status is completed, cancelled or expired. See Webhooks.