Batch processing

The Batch API lets you send groups of requests as a single job that processes asynchronously. This is useful when you need to run many requests and do not need immediate results — for example, running evaluations, generating embeddings for a large dataset, or classifying content in bulk. Batch requests run within a 24-hour completion window and have separate, higher rate limits than synchronous API calls.

Supported endpoints

Endpoint	Description
`/v1/chat/completions`	Chat completion requests
`/v1/embeddings`	Embedding generation
`/v1/images/generations`	Image generation

All requests in a single batch must target the same endpoint.

Lifecycle

A batch moves through these statuses:

validating → in_progress → finalizing → completed
     ↓            ↓
   failed     cancelling → cancelled

Status	Description
`validating`	The input file is being checked for format and content errors.
`failed`	Validation failed — the input file contains errors. Check `batch.errors` for details.
`in_progress`	Requests are being processed.
`finalizing`	All requests have been processed and the output files are being generated.
`completed`	The batch finished. Download results from `output_file_id`.
`cancelling`	A cancel was requested. In-flight requests are finishing.
`cancelled`	The batch was cancelled. Partial results may be available.
`expired`	The batch did not complete within the 24-hour window.

Step 1 — Prepare the input file

Create a JSONL file where each line is one request. Every line has four fields:

Field	Type	Description
`custom_id`	string	Your identifier for this request. Used to match input to output. Must be unique within the file.
`method`	string	`"POST"` — the only supported method.
`url`	string	The endpoint path — must match the `endpoint` you declare when creating the batch.
`body`	object	The request body — the same parameters you would send to the synchronous endpoint.

{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "auto", "messages": [{"role": "user", "content": "Summarize the benefits of batch processing in one sentence."}], "max_tokens": 100}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "auto", "messages": [{"role": "user", "content": "What is the capital of France?"}], "max_tokens": 100}}
{"custom_id": "request-3", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "auto", "messages": [{"role": "user", "content": "Explain embeddings in one paragraph."}], "max_tokens": 100}}

A single batch can contain up to 10,000 requests.

Step 2 — Upload the input file

Upload the JSONL file using the Files API with purpose: "batch".

mka1 llm files upload \
  --file ./batch_input.jsonl \
  --purpose batch \
  -H 'X-On-Behalf-Of: <end-user-id>'

import { SDK } from '@meetkai/mka1';

const mka1 = new SDK({
  bearerAuth: `Bearer ${YOUR_API_KEY}`,
});

const file = await mka1.llm.files.upload({
  requestBody: {
    file: new File([jsonlContent], 'batch_input.jsonl', { type: 'application/jsonl' }),
    purpose: 'batch',
  },
});

console.log(file.id);     // "file_abc123"
console.log(file.status); // "processed"

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: '<mka1-api-key>',
  baseURL: 'https://apigw.mka1.com/api/v1/llm/',
  defaultHeaders: { 'X-On-Behalf-Of': '<end-user-id>' },
});

const file = await openai.files.create({
  file: new File([jsonlContent], 'batch_input.jsonl', { type: 'application/jsonl' }),
  purpose: 'batch',
});

console.log(file.id);     // "file_abc123"
console.log(file.status); // "processed"

using System.Text;
using MeetKai.MKA1;
using MeetKai.MKA1.Types.Components;
using MeetKai.MKA1.Types.Requests;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

var file = await sdk.Llm.Files.UploadAsync(new UploadFileRequestBody()
{
    File = new UploadFileFile()
    {
        FileName = "batch_input.jsonl",
        Content = Encoding.UTF8.GetBytes(jsonlContent),
    },
    Purpose = UploadFilePurpose.Batch,
});

Console.WriteLine(file.File!.Id);     // "file_abc123"

from meetkai_mka1 import SDK

sdk = SDK(bearer_auth="Bearer YOUR_API_KEY")

file = sdk.llm.files.upload(
    file={"file_name": "batch_input.jsonl", "content": open("batch_input.jsonl", "rb")},
    purpose="batch",
)

print(file.id)      # "file_abc123"
print(file.status)  # "processed"

curl https://apigw.mka1.com/api/v1/llm/files \
  --request POST \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --header 'X-On-Behalf-Of: <end-user-id>' \
  --form 'file=@batch_input.jsonl;type=application/jsonl' \
  --form 'purpose=batch'

Step 3 — Create the batch

Pass the uploaded file ID, the target endpoint, and the completion window.

mka1 llm batches create --body '{
  "input_file_id": "file_abc123",
  "endpoint": "/v1/chat/completions",
  "completion_window": "24h"
}'

const batch = await mka1.llm.batches.create({
  createBatchRequest: {
    inputFileId: file.id,
    endpoint: '/v1/chat/completions',
    completionWindow: '24h',
  },
});

console.log(batch.id);            // "batch_abc123"
console.log(batch.status);        // "validating" or "in_progress"
console.log(batch.requestCounts); // { total: 3, completed: 0, failed: 0 }

const batch = await openai.batches.create({
  input_file_id: file.id,
  endpoint: '/v1/chat/completions',
  completion_window: '24h',
});

console.log(batch.id);              // "batch_abc123"
console.log(batch.status);          // "validating" or "in_progress"
console.log(batch.request_counts);  // { total: 3, completed: 0, failed: 0 }

using MeetKai.MKA1;
using MeetKai.MKA1.Types.Components;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

var batch = await sdk.Llm.Batches.CreateAsync(new CreateBatchRequest()
{
    InputFileId = file.File!.Id,
    Endpoint = BatchEndpoint.RootV1ChatCompletions,
});

Console.WriteLine(batch.BatchObject!.Id);            // "batch_abc123"
Console.WriteLine(batch.BatchObject!.Status);        // "validating" or "in_progress"
Console.WriteLine(batch.BatchObject!.RequestCounts); // { Total: 3, Completed: 0, Failed: 0 }

batch = sdk.llm.batches.create(
    input_file_id=file.id,
    endpoint="/v1/chat/completions",
)

print(batch.id)              # "batch_abc123"
print(batch.status)          # "validating" or "in_progress"
print(batch.request_counts)  # { total: 3, completed: 0, failed: 0 }

curl https://apigw.mka1.com/api/v1/llm/batches \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --header 'X-On-Behalf-Of: <end-user-id>' \
  --data '{
    "input_file_id": "file_abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

You can also attach metadata for your own tracking:

mka1 llm batches create --body '{
  "input_file_id": "file_abc123",
  "endpoint": "/v1/chat/completions",
  "completion_window": "24h",
  "metadata": {
    "description": "nightly evaluation run",
    "run_id": "eval-2026-03-31"
  }
}'

const batch = await mka1.llm.batches.create({
  createBatchRequest: {
    inputFileId: file.id,
    endpoint: '/v1/chat/completions',
    completionWindow: '24h',
    metadata: {
      description: 'nightly evaluation run',
      run_id: 'eval-2026-03-31',
    },
  },
});

const batch = await openai.batches.create({
  input_file_id: file.id,
  endpoint: '/v1/chat/completions',
  completion_window: '24h',
  metadata: {
    description: 'nightly evaluation run',
    run_id: 'eval-2026-03-31',
  },
});

using MeetKai.MKA1;
using MeetKai.MKA1.Types.Components;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

var batch = await sdk.Llm.Batches.CreateAsync(new CreateBatchRequest()
{
    InputFileId = file.File!.Id,
    Endpoint = BatchEndpoint.RootV1ChatCompletions,
    Metadata = new Dictionary<string, string>
    {
        { "description", "nightly evaluation run" },
        { "run_id", "eval-2026-03-31" },
    },
});

batch = sdk.llm.batches.create(
    input_file_id=file.id,
    endpoint="/v1/chat/completions",
    metadata={
        "description": "nightly evaluation run",
        "run_id": "eval-2026-03-31",
    },
)

Step 4 — Check batch status

Poll the batch until it reaches a terminal status.

mka1 llm batches get --batch-id batch_abc123

const batch = await mka1.llm.batches.get({ batchId: 'batch_abc123' });

console.log(batch.status);                // "completed"
console.log(batch.requestCounts.completed); // 3
console.log(batch.requestCounts.failed);    // 0
console.log(batch.outputFileId);           // "file_xyz789"

const batch = await openai.batches.retrieve('batch_abc123');

console.log(batch.status);                  // "completed"
console.log(batch.request_counts.completed); // 3
console.log(batch.request_counts.failed);    // 0
console.log(batch.output_file_id);           // "file_xyz789"

using MeetKai.MKA1;
using MeetKai.MKA1.Types.Components;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

var batch = await sdk.Llm.Batches.GetAsync("batch_abc123");

Console.WriteLine(batch.BatchObject!.Status);                // "completed"
Console.WriteLine(batch.BatchObject!.RequestCounts);         // { Total: 3, Completed: 3, Failed: 0 }
Console.WriteLine(batch.BatchObject!.OutputFileId);          // "file_xyz789"

batch = sdk.llm.batches.get(batch_id="batch_abc123")

print(batch.status)                  # "completed"
print(batch.request_counts.completed)  # 3
print(batch.request_counts.failed)     # 0
print(batch.output_file_id)           # "file_xyz789"

curl https://apigw.mka1.com/api/v1/llm/batches/batch_abc123 \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --header 'X-On-Behalf-Of: <end-user-id>'

Here is a polling helper that waits for the batch to finish:

# Poll a batch until it reaches a terminal status using --jq and a shell loop.
BATCH_ID=batch_abc123
while :; do
  STATUS=$(mka1 llm batches get --batch-id "$BATCH_ID" --jq '.status' --output-format json)
  echo "status: $STATUS"
  case "$STATUS" in
    completed|failed|cancelled|expired) break ;;
  esac
  sleep 2
done

async function waitForBatch(batchId: string, timeoutMs = 120_000) {
  const terminal = ['completed', 'failed', 'cancelled', 'expired'];
  const start = Date.now();

  while (Date.now() - start < timeoutMs) {
    const batch = await mka1.llm.batches.get({ batchId });
    if (terminal.includes(batch.status)) return batch;
    await new Promise((r) => setTimeout(r, 2000));
  }

  throw new Error(`Batch ${batchId} did not complete within ${timeoutMs}ms`);
}

const completed = await waitForBatch(batch.id);

async function waitForBatch(batchId: string, timeoutMs = 120_000) {
  const terminal = ['completed', 'failed', 'cancelled', 'expired'];
  const start = Date.now();

  while (Date.now() - start < timeoutMs) {
    const batch = await openai.batches.retrieve(batchId);
    if (terminal.includes(batch.status)) return batch;
    await new Promise((r) => setTimeout(r, 2000));
  }

  throw new Error(`Batch ${batchId} did not complete within ${timeoutMs}ms`);
}

const completed = await waitForBatch(batch.id);

using MeetKai.MKA1;
using MeetKai.MKA1.Types.Components;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

async Task<BatchObject> WaitForBatch(SDK sdk, string batchId, int timeoutMs = 300_000)
{
    var terminal = new HashSet<BatchObjectStatus>
    {
        BatchObjectStatus.Completed,
        BatchObjectStatus.Failed,
        BatchObjectStatus.Cancelled,
        BatchObjectStatus.Expired,
    };
    var start = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();

    while (DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() - start < timeoutMs)
    {
        var batch = await sdk.Llm.Batches.GetAsync(batchId);
        if (terminal.Contains(batch.BatchObject!.Status))
            return batch.BatchObject;
        await Task.Delay(2000);
    }

    throw new TimeoutException($"Batch {batchId} did not complete within {timeoutMs}ms");
}

var completed = await WaitForBatch(sdk, batch.BatchObject!.Id);
Console.WriteLine(completed.Status); // BatchObjectStatus.Completed

import time

def wait_for_batch(sdk, batch_id, timeout_ms=120_000):
    terminal = {"completed", "failed", "cancelled", "expired"}
    start = time.time() * 1000

    while (time.time() * 1000) - start < timeout_ms:
        batch = sdk.llm.batches.get(batch_id=batch_id)
        if batch.status in terminal:
            return batch
        time.sleep(2)

    raise TimeoutError(f"Batch {batch_id} did not complete within {timeout_ms}ms")

completed = wait_for_batch(sdk, batch.id)

Step 5 — Download the results

Once the batch is completed, download the output file. It is a JSONL file where each line contains the custom_id you provided, the response, and any error.

# Download the JSONL output file
mka1 llm files content \
  --file-id file_xyz789 \
  --output-file ./batch_output.jsonl

# Inspect the results inline with jq
mka1 llm files content --file-id file_xyz789 \
  --jq '"\(.custom_id): status=\(.response.status_code)"'

const stream = await mka1.llm.files.content({ fileId: completed.outputFileId! });
const reader = stream.getReader();
const chunks: Uint8Array[] = [];
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  chunks.push(value);
}
const text = new TextDecoder().decode(Buffer.concat(chunks));

const results = text
  .split('\n')
  .filter((line) => line.trim())
  .map((line) => JSON.parse(line));

for (const result of results) {
  console.log(`${result.custom_id}: status=${result.response.status_code}`);
  console.log(`  body:`, result.response.body);
}

const content = await openai.files.content(completed.output_file_id!);
const text = await content.text();

const results = text
  .split('\n')
  .filter((line) => line.trim())
  .map((line) => JSON.parse(line));

for (const result of results) {
  console.log(`${result.custom_id}: status=${result.response.status_code}`);
  console.log(`  body:`, result.response.body);
}

using System.Text;
using MeetKai.MKA1;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

var content = await sdk.Llm.Files.ContentAsync(completed.OutputFileId!);

var bytes = content.TwoHundredTextPlainBytes
    ?? content.TwoHundredApplicationJsonlBytes
    ?? content.TwoHundredApplicationJsonBytes;
var text = Encoding.UTF8.GetString(bytes!);

Console.WriteLine(text); // JSONL with one line per request

import json

content = sdk.llm.files.content(file_id=completed.output_file_id)
text = content.decode("utf-8")

results = [json.loads(line) for line in text.strip().split("\n") if line.strip()]

for result in results:
    print(f"{result['custom_id']}: status={result['response']['status_code']}")
    print(f"  body: {result['response']['body']}")

curl https://apigw.mka1.com/api/v1/llm/files/file_xyz789/content \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --header 'X-On-Behalf-Of: <end-user-id>'

Each line in the output file has this structure:

{
  "id": "response_abc123",
  "custom_id": "request-1",
  "response": {
    "status_code": 200,
    "request_id": "req_abc123",
    "body": { "...": "same shape as the synchronous endpoint response" }
  },
  "error": null
}

If a request failed, response is null and error contains the details:

{
  "id": "response_def456",
  "custom_id": "request-2",
  "response": null,
  "error": {
    "code": "processing_error",
    "message": "The request could not be processed."
  }
}

If any requests failed, the batch also provides an error_file_id containing only the failed entries.

Cancel a batch

Cancel a batch that is still in progress. Requests that have already completed remain in the output.

mka1 llm batches cancel --batch-id batch_abc123

const cancelled = await mka1.llm.batches.cancel({ batchId: 'batch_abc123' });
console.log(cancelled.status); // "cancelling"

const cancelled = await openai.batches.cancel('batch_abc123');
console.log(cancelled.status); // "cancelling"

using MeetKai.MKA1;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

var cancelled = await sdk.Llm.Batches.CancelAsync("batch_abc123");
Console.WriteLine(cancelled.BatchObject!.Status); // "cancelling"

cancelled = sdk.llm.batches.cancel(batch_id="batch_abc123")
print(cancelled.status)  # "cancelling"

curl https://apigw.mka1.com/api/v1/llm/batches/batch_abc123/cancel \
  --request POST \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --header 'X-On-Behalf-Of: <end-user-id>'

The batch transitions to cancelling while in-flight requests finish, then to cancelled.

List batches

Retrieve all batches for the current account, newest first. Supports pagination.

mka1 llm batches list --limit 20

const page = await mka1.llm.batches.list({ limit: 20 });

for (const batch of page.data) {
  console.log(`${batch.id}: ${batch.status} (${batch.requestCounts?.completed}/${batch.requestCounts?.total})`);
}

const page = await openai.batches.list({ limit: 20 });

for (const batch of page.data) {
  console.log(`${batch.id}: ${batch.status} (${batch.request_counts?.completed}/${batch.request_counts?.total})`);
}

using MeetKai.MKA1;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

var page = await sdk.Llm.Batches.ListAsync(limit: 20);

foreach (var batch in page.ListBatchesResponseValue!.Data!)
{
    Console.WriteLine($"{batch.Id}: {batch.Status} ({batch.RequestCounts?.Completed}/{batch.RequestCounts?.Total})");
}

page = sdk.llm.batches.list(limit=20)

for batch in page.data:
    print(f"{batch.id}: {batch.status} ({batch.request_counts.completed}/{batch.request_counts.total})")

curl 'https://apigw.mka1.com/api/v1/llm/batches?limit=20' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --header 'X-On-Behalf-Of: <end-user-id>'

Use the after parameter with a batch ID to page through results.

Example: batch embeddings

The same flow works for embeddings. Change the url in each JSONL line and the endpoint when creating the batch.

{"custom_id": "embed-1", "method": "POST", "url": "/v1/embeddings", "body": {"model": "auto", "input": "The quick brown fox"}}
{"custom_id": "embed-2", "method": "POST", "url": "/v1/embeddings", "body": {"model": "auto", "input": "jumps over the lazy dog"}}

# Upload the embeddings JSONL input
FILE_ID=$(mka1 llm files upload \
  --file ./embed_batch.jsonl \
  --purpose batch \
  --jq '.id' --output-format json | tr -d '"')

# Create the batch against the embeddings endpoint
mka1 llm batches create --body "{
  \"input_file_id\": \"$FILE_ID\",
  \"endpoint\": \"/v1/embeddings\",
  \"completion_window\": \"24h\"
}"

# Poll, then download the results — see Steps 4 and 5

const file = await mka1.llm.files.upload({
  requestBody: {
    file: new File([jsonlContent], 'embed_batch.jsonl', { type: 'application/jsonl' }),
    purpose: 'batch',
  },
});

const batch = await mka1.llm.batches.create({
  createBatchRequest: {
    inputFileId: file.id,
    endpoint: '/v1/embeddings',
    completionWindow: '24h',
  },
});

const completed = await waitForBatch(batch.id);
const stream = await mka1.llm.files.content({ fileId: completed.outputFileId! });
// ... read stream as shown in Step 5

const file = await openai.files.create({
  file: new File([jsonlContent], 'embed_batch.jsonl', { type: 'application/jsonl' }),
  purpose: 'batch',
});

const batch = await openai.batches.create({
  input_file_id: file.id,
  endpoint: '/v1/embeddings',
  completion_window: '24h',
});

const completed = await waitForBatch(batch.id);
const content = await openai.files.content(completed.output_file_id!);
const results = (await content.text()).split('\n').filter(Boolean).map(JSON.parse);

for (const r of results) {
  console.log(`${r.custom_id}: ${r.response.body.data[0].embedding.length} dimensions`);
}

using System.Text;
using MeetKai.MKA1;
using MeetKai.MKA1.Types.Components;
using MeetKai.MKA1.Types.Requests;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

var file = await sdk.Llm.Files.UploadAsync(new UploadFileRequestBody()
{
    File = new UploadFileFile()
    {
        FileName = "embed_batch.jsonl",
        Content = Encoding.UTF8.GetBytes(jsonlContent),
    },
    Purpose = UploadFilePurpose.Batch,
});

var batch = await sdk.Llm.Batches.CreateAsync(new CreateBatchRequest()
{
    InputFileId = file.File!.Id,
    Endpoint = BatchEndpoint.RootV1Embeddings,
});

var completed = await WaitForBatch(sdk, batch.BatchObject!.Id);
// Download and parse results as shown in Step 5

file = sdk.llm.files.upload(
    file={"file_name": "embed_batch.jsonl", "content": open("embed_batch.jsonl", "rb")},
    purpose="batch",
)

batch = sdk.llm.batches.create(
    input_file_id=file.id,
    endpoint="/v1/embeddings",
)

completed = wait_for_batch(sdk, batch.id)
# Download and parse results as shown in Step 5

Validation errors

If the input file has formatting issues, the batch moves to failed immediately. Common causes:

Invalid JSON — a line is not valid JSON.
Missing fields — a line is missing custom_id, method, url, or body.
Wrong method — method must be "POST".
URL mismatch — the url in a line does not match the endpoint declared when creating the batch.
Duplicate custom_id — each custom_id must be unique within the file.

Check batch.errors.data for the specific error messages and line numbers.

mka1 llm batches get --batch-id batch_abc123 \
  --jq '.errors.data[] | "Line \(.line): [\(.code)] \(.message)"'

const batch = await mka1.llm.batches.get({ batchId: 'batch_abc123' });

if (batch.status === 'failed' && batch.errors) {
  for (const err of batch.errors.data ?? []) {
    console.log(`Line ${err.line}: [${err.code}] ${err.message}`);
  }
}

const batch = await openai.batches.retrieve('batch_abc123');

if (batch.status === 'failed' && batch.errors) {
  for (const err of batch.errors.data ?? []) {
    console.log(`Line ${err.line}: [${err.code}] ${err.message}`);
  }
}

using MeetKai.MKA1;
using MeetKai.MKA1.Types.Components;

var sdk = new SDK(bearerAuth: "Bearer YOUR_API_KEY");

var batch = await sdk.Llm.Batches.GetAsync("batch_abc123");

if (batch.BatchObject!.Status == BatchObjectStatus.Failed && batch.BatchObject.Errors != null)
{
    foreach (var err in batch.BatchObject.Errors.Data ?? new List<BatchObjectErrorsData>())
    {
        Console.WriteLine($"Line {err.Line}: [{err.Code}] {err.Message}");
    }
}

batch = sdk.llm.batches.get(batch_id="batch_abc123")

if batch.status == "failed" and batch.errors:
    for err in batch.errors.data or []:
        print(f"Line {err.line}: [{err.code}] {err.message}")

Getting started

Responses

Features

Compute

CLI

Recipes

Benchmarks

Infrastructure

Batch processing

Supported endpoints

Lifecycle

Step 1 — Prepare the input file

Step 2 — Upload the input file

Step 3 — Create the batch

Step 4 — Check batch status

Step 5 — Download the results

Cancel a batch

List batches

Example: batch embeddings

Validation errors

See also

​Supported endpoints

​Lifecycle

​Step 1 — Prepare the input file

​Step 2 — Upload the input file

​Step 3 — Create the batch

​Step 4 — Check batch status

​Step 5 — Download the results

​Cancel a batch

​List batches

​Example: batch embeddings

​Validation errors

​See also

Supported endpoints

Lifecycle

Step 1 — Prepare the input file

Step 2 — Upload the input file

Step 3 — Create the batch

Step 4 — Check batch status

Step 5 — Download the results

Cancel a batch

List batches

Example: batch embeddings

Validation errors

See also