> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mka1.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Auto routing

> Understand and validate how MKA1 selects a model when a request uses model auto.

When a request uses <code>"model": "auto"</code>, MKA1 applies the
organization's auto policy for that endpoint.

By default, <code>auto</code> is a direct alias to a model in the model
registry. Organizations can also configure an LLM judge to select from an
approved set of models. The judge is given signals about the request, the
available models, and the existing conversation so it can choose the model
best suited to the work.

<span id="validate-auto-routing" />

## Supported endpoints

Judge-based selection is available for several types of AI request.

| Endpoint | Candidate type |
| - | - |
| Responses and Chat Completions | LLM |
| Audio Transcriptions | Speech-to-text |
| Text-to-Speech | Text-to-speech |

Only models suited to the request type are considered. For example, a TTS
model cannot be selected for a Responses request.

## How model selection works

The organization defines the models that may be selected. For each request,
MKA1 excludes candidates that are no longer active in the organization's
registry or that cannot handle the request, such as a model without image
input for a request that contains an image. If only one suitable model
remains, MKA1 selects it directly; when several are available, the configured
judge chooses among them.

Each judged request makes one extra model call to the judge. The call is
billed at the judge model's price and counts toward your budgets, but it is
not included in the response's <code>usage</code>. It also adds a full judge
round trip before the selected model starts, and a streamed response opens
only after the judge has chosen.

A judge policy does not fail over. If the selected model fails, the request
fails: MKA1 does not try another candidate, and the candidates'
[reroute rules](/docs/model-registry#reroute-rules) do not apply. Provider
health is not a selection signal, so a selected model whose provider is
unavailable also fails the request.

If no suitable model remains, or if the configured judge is unavailable or
fails, MKA1 returns a transparent error instead of silently changing the
policy. See [Errors](#errors).

## Signals used for model selection

MKA1 gives the judge a consistent view of the factors that matter for the
current request:

* **Request and tools:** the nature of the task, available tools, requested
  output, and whether images, audio, or files are present. Models that can work
  with the original input directly are preferred.
* **Model capabilities:** supported modalities and file types, available
  context, and other requirements needed to complete the request.
* **Price:** each candidate's per-unit rates at your organization's effective
  price. A model without a price counts as free.
* **Latency:** rolling latency and failure-rate estimates that the gateway
  measures on live traffic.
* **Evals:** the most recent completed [eval runs](/docs/evals) from your own
  team that score two or more of the candidates. Platform benchmarks are not
  used.
* **Conversation continuity:** for a Responses request that continues a
  conversation, which candidate served the previous response and how many
  tokens it processed, so the judge can weigh staying on that model to keep
  cost and latency down.

The selected policy tells the judge which of these signals matters most:

* <code>balanced</code> considers quality, price, speed, capabilities, and
  conversation continuity together.
* <code>price</code> prioritizes the lowest expected cost among models suited
  to the request.
* <code>quality</code> prioritizes task fit and evaluation results, and ignores
  conversation continuity.
* <code>speed</code> prioritizes responsiveness using rolling latency and
  reliability signals.

Capability requirements always apply, regardless of policy. The best choice
can change as the request, model availability, performance, or conversation
evolves.

## Privacy and safety

The judge receives a bounded, sanitized summary of the request and its tool
requirements. Credentials and raw binary content are not forwarded. The
organization's normal safety policies still apply, and the judge can select
only from models the organization has approved for the policy.

## Inspect the current policy

The auto endpoints are organization admin operations: the API key needs the
<code>read:registry</code> scope to list overrides and
<code>write:registry</code> to change them. Each override's
<code>mode</code> is either <code>direct</code> or the judge policy name, so
read that field first to know which other fields are present.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  import { SDK } from '@meetkai/mka1';
  import type { ResponseObject } from '@meetkai/mka1/models/components';

  const sdk = new SDK({ bearerAuth: 'Bearer <mka1-api-key>' });

  const overrides = await sdk.llm.models.listOrgAutoModels();
  for (const override of overrides.data) {
    switch (override.mode) {
      case 'direct':
        console.log(override.endpoint, '->', override.modelId, 'resolvable:', override.resolvable);
        break;
      case 'balanced':
      case 'price':
      case 'quality':
      case 'speed':
        console.log(
          override.endpoint, '->', override.mode,
          'judge:', override.judgeModelId,
          'candidates:', override.resolvableCandidateModelIds,
        );
        break;
    }
  }
  ```

  ```python Python SDK theme={null}
  from meetkai_mka1 import SDK

  sdk = SDK(bearer_auth="Bearer <mka1-api-key>")

  overrides = sdk.llm.models.list_org_auto_models()
  for override in overrides.data:
      if override.mode == "direct":
          print(override.endpoint, "->", override.model_id, "resolvable:", override.resolvable)
      else:
          print(
              override.endpoint, "->", override.mode,
              "judge:", override.judge_model_id,
              "candidates:", override.resolvable_candidate_model_ids,
          )
  ```

  ```csharp C# SDK theme={null}
  using MeetKai.MKA1;
  using MeetKai.MKA1.Types.Components;
  using System.Collections.Generic;

  var sdk = new SDK(bearerAuth: "Bearer <mka1-api-key>");

  var overrides = await sdk.Llm.Models.ListOrgAutoModelsAsync();
  foreach (var o in overrides.ListOrgAutoModelsResponseValue!.Data)
  {
      if (o.DirectOrgAutoModel is { } direct)
      {
          Console.WriteLine($"{direct.Endpoint} -> {direct.ModelId} (resolvable: {direct.Resolvable})");
      }
      else if (o.BalancedOrgAutoModel is { } judged)
      {
          Console.WriteLine($"{judged.Endpoint} -> balanced, judge {judged.JudgeModelId}, " +
              $"candidates {string.Join(", ", judged.ResolvableCandidateModelIds)}");
      }
      // price, quality and speed overrides arrive on o.PriceOrgAutoModel,
      // o.QualityOrgAutoModel and o.SpeedOrgAutoModel with the same fields.
  }
  ```

  ```bash CLI theme={null}
  mka1 llm models get \
    --model-id auto
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/llm/models/auto \
    --header 'Authorization: Bearer <mka1-api-key>'
  ```
</CodeGroup>

A direct override resembles:

```json theme={null}
{
  "object": "list",
  "data": [
    {
      "endpoint": "responses",
      "mode": "direct",
      "model_id": "meetkai:functionary-pt",
      "resolvable": true,
      "set_by": "user-org-admin-1",
      "set_at": "2026-09-01T12:00:00.000Z"
    }
  ]
}
```

## Configure an LLM judge

The following example enables judge-based model selection for LLM requests
using the <code>balanced</code> policy. Set <code>policy</code> to
<code>price</code>, <code>quality</code>, or <code>speed</code> to emphasize a
different goal. The judge and every candidate must be valid LLM model
identifiers available to the organization. Choose a fast, non-reasoning judge
that supports structured outputs, such as <code>openai:gpt-4.1-nano</code>:
the judge runs before every judged request and must return its choice as
structured JSON. Model availability is organization-specific;
replace the example IDs with IDs returned by <code>GET /models</code> for your
organization.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const override = await sdk.llm.models.putOrgAutoModel({
    endpoint: 'responses',
    putOrgAutoModelRequest: {
      policy: 'balanced',
      judgeModelId: 'openai:gpt-4.1-nano',
      candidateModelIds: ['meetkai:functionary-pt', 'openai:gpt-4.1-mini'],
    },
  });
  if (override.mode === 'balanced') {
    console.log(override.judgeResolvable);              // true
    console.log(override.resolvableCandidateModelIds);  // both candidates
  }
  ```

  ```python Python SDK theme={null}
  override = sdk.llm.models.put_org_auto_model(
      endpoint="responses",
      body={
          "policy": "balanced",
          "judge_model_id": "openai:gpt-4.1-nano",
          "candidate_model_ids": ["meetkai:functionary-pt", "openai:gpt-4.1-mini"],
      },
  )
  print(override.mode)                            # "balanced"
  print(override.judge_resolvable)                # True
  print(override.resolvable_candidate_model_ids)  # both candidates
  ```

  ```csharp C# SDK theme={null}
  var judged = await sdk.Llm.Models.PutOrgAutoModelAsync(
      endpoint: AutoEndpoint.Responses,
      body: PutOrgAutoModelRequestUnion.CreatePutOrgAutoModelRequest2(new PutOrgAutoModelRequest2()
      {
          Policy = Policy.Balanced,
          JudgeModelId = "openai:gpt-4.1-nano",
          CandidateModelIds = new List<string> { "meetkai:functionary-pt", "openai:gpt-4.1-mini" },
      })
  );
  var balanced = judged.OneOf!.BalancedOrgAutoModel!;
  Console.WriteLine(balanced.JudgeResolvable);                               // True
  Console.WriteLine(string.Join(", ", balanced.ResolvableCandidateModelIds)); // both candidates
  ```

  ```bash CLI theme={null}
  mka1 llm models put-org-auto-model \
    --body '{
      "policy": "balanced",
      "judge_model_id": "openai:gpt-4.1-nano",
      "candidate_model_ids": [
        "meetkai:functionary-pt",
        "openai:gpt-4.1-mini"
      ]
    }' \
    --endpoint responses
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \
    --request PUT \
    --header 'Content-Type: application/json' \
    --header 'Authorization: Bearer <mka1-api-key>' \
    --data '{
      "policy": "balanced",
      "judge_model_id": "openai:gpt-4.1-nano",
      "candidate_model_ids": [
        "meetkai:functionary-pt",
        "openai:gpt-4.1-mini"
      ]
    }'
  ```
</CodeGroup>

The stored override reports the policy as its <code>mode</code>, plus which of
the configured models currently resolve:

```json theme={null}
{
  "endpoint": "responses",
  "mode": "balanced",
  "judge_model_id": "openai:gpt-4.1-nano",
  "candidate_model_ids": ["meetkai:functionary-pt", "openai:gpt-4.1-mini"],
  "judge_resolvable": true,
  "resolvable_candidate_model_ids": ["meetkai:functionary-pt", "openai:gpt-4.1-mini"],
  "set_by": "user-org-admin-1",
  "set_at": "2026-09-01T12:00:00.000Z"
}
```

This candidate set pairs a Portuguese-tuned model with a general one: a
request in Portuguese can route to <code>meetkai:functionary-pt</code>, while
other requests can route to <code>openai:gpt-4.1-mini</code>.

You can configure <code>responses</code>, <code>completions</code>,
<code>transcriptions</code>, and <code>tts</code> independently. Endpoints
without an organization override retain direct model-registry alias routing.

## Validate the selected model

Make a normal request using <code>"model": "auto"</code>:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const response = await sdk.llm.responses.create({
    xOnBehalfOf: '<end-user-id>', // optional — attribute the request to one of your end users
    responsesCreateRequest: {
      model: 'auto',
      input: 'Resuma este relatório de incidente e liste os três próximos passos de maior risco.',
      maxOutputTokens: 300,
    },
  }) as ResponseObject;
  console.log(response.model);         // "auto"
  console.log(response.resolvedModel); // e.g. "meetkai:functionary-pt"
  ```

  ```ts OpenAI SDK theme={null}
  import OpenAI from 'openai';

  const openai = new OpenAI({
    apiKey: '<mka1-api-key>',
    baseURL: 'https://apigw.mka1.com/api/v1/llm/',
    defaultHeaders: { 'X-On-Behalf-Of': '<end-user-id>' },
  });

  const response = await openai.responses.create({
    model: 'auto',
    input: 'Resuma este relatório de incidente e liste os três próximos passos de maior risco.',
    max_output_tokens: 300,
    stream: false,
  });
  console.log(response.model); // "auto"
  // resolved_model is an MKA1 extension, so it is not on the OpenAI SDK's response type.
  console.log((response as { resolved_model?: string }).resolved_model); // e.g. "meetkai:functionary-pt"
  ```

  ```python Python SDK theme={null}
  res = sdk.llm.responses.create(
      model="auto",
      input="Resuma este relatório de incidente e liste os três próximos passos de maior risco.",
      max_output_tokens=300,
      x_on_behalf_of="<end-user-id>",  # optional — attribute the request to one of your end users
  )
  print(res.model)           # "auto"
  print(res.resolved_model)  # e.g. "meetkai:functionary-pt"
  ```

  ```csharp C# SDK theme={null}
  var res = await sdk.Llm.Responses.CreateAsync(new ResponsesCreateRequest()
  {
      Model = "auto",
      Input = ResponsesCreateRequestInput.CreateStr(
          "Resuma este relatório de incidente e liste os três próximos passos de maior risco."),
      MaxOutputTokens = 300,
  });
  Console.WriteLine(res.ResponseObject!.Model);         // auto
  Console.WriteLine(res.ResponseObject!.ResolvedModel); // e.g. meetkai:functionary-pt
  ```

  ```bash CLI theme={null}
  mka1 llm responses create \
    --model auto \
    --input '"Resuma este relatório de incidente e liste os três próximos passos de maior risco."' \
    -H 'X-On-Behalf-Of: <end-user-id>'
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/llm/responses \
    --request POST \
    --header 'Content-Type: application/json' \
    --header 'Authorization: Bearer <mka1-api-key>' \
    --data '{
      "model": "auto",
      "input": "Resuma este relatório de incidente e liste os três próximos passos de maior risco.",
      "max_output_tokens": 300
    }'
  ```
</CodeGroup>

The response echoes the alias you sent in its <code>model</code> field and
names the model that actually served the request in
<code>resolved\_model</code>. With a direct policy, that is the registry alias
target, or the model a [reroute rule](/docs/model-registry#reroute-rules) sent
the request to. With an LLM judge configured, it is the approved candidate
selected for the request.

<code>resolved\_model</code> is a Responses API field. Chat Completions returns
only the requested name in <code>model</code> and does not report the model
that served the request.

When you stream a response, read <code>resolved\_model</code> from the final
event, such as <code>response.completed</code>. The
<code>response.created</code> event carries only the first resolution, which a
reroute can change later in the request. A background request with a judge
policy reports no <code>resolved\_model</code> until the final event.

## Errors

When auto routing cannot select a model, the request fails with HTTP
<code>503</code> and one of these codes in <code>error.code</code>:

| Code | Cause |
| - | - |
| <code>auto\_judge\_unavailable</code> | The judge model is not active in the organization's registry, is hidden, or is not an LLM. |
| <code>auto\_no\_eligible\_models</code> | No candidate can handle the request, or the selected candidate stopped being available before the request was sent to it. |
| <code>auto\_judge\_failed</code> | The judge call failed or took longer than 30 seconds, or the judge did not return one of the candidates. |

A streamed request receives the error as a normal HTTP response, before the
stream opens. A background request that fails selection after it was queued
ends with <code>status: "failed"</code> and the same code in
<code>error.code</code>. In <code>GET /models/auto</code>,
<code>judge\_resolvable: false</code> points to
<code>auto\_judge\_unavailable</code>, and an empty
<code>resolvable\_candidate\_model\_ids</code> to
<code>auto\_no\_eligible\_models</code>.

## Restore direct routing

Set a direct policy explicitly:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const override = await sdk.llm.models.putOrgAutoModel({
    endpoint: 'responses',
    putOrgAutoModelRequest: { modelId: 'meetkai:functionary-pt' },
  });
  if (override.mode === 'direct') {
    console.log(override.modelId, override.resolvable); // "meetkai:functionary-pt" true
  }
  ```

  ```python Python SDK theme={null}
  override = sdk.llm.models.put_org_auto_model(
      endpoint="responses",
      body={"model_id": "meetkai:functionary-pt"},
  )
  print(override.mode, override.model_id, override.resolvable)  # direct meetkai:functionary-pt True
  ```

  ```csharp C# SDK theme={null}
  var direct = await sdk.Llm.Models.PutOrgAutoModelAsync(
      endpoint: AutoEndpoint.Responses,
      body: PutOrgAutoModelRequestUnion.CreatePutOrgAutoModelRequest1(
          new PutOrgAutoModelRequest1() { ModelId = "meetkai:functionary-pt" })
  );
  Console.WriteLine(direct.OneOf!.DirectOrgAutoModel!.ModelId); // meetkai:functionary-pt
  ```

  ```bash CLI theme={null}
  mka1 llm models put-org-auto-model \
    --body '{ "model_id": "meetkai:functionary-pt" }' \
    --endpoint responses
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \
    --request PUT \
    --header 'Content-Type: application/json' \
    --header 'Authorization: Bearer <mka1-api-key>' \
    --data '{ "model_id": "meetkai:functionary-pt" }'
  ```
</CodeGroup>

Or, clear your org's override so <code>"model": "auto"</code> falls through to
the cluster default again:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const cleared = await sdk.llm.models.deleteOrgAutoModel({ endpoint: 'responses' });
  console.log(cleared.cleared); // true
  ```

  ```python Python SDK theme={null}
  cleared = sdk.llm.models.delete_org_auto_model(endpoint="responses")
  print(cleared.cleared)  # True
  ```

  ```csharp C# SDK theme={null}
  var cleared = await sdk.Llm.Models.DeleteOrgAutoModelAsync(endpoint: AutoEndpoint.Responses);
  Console.WriteLine(cleared.DeleteOrgAutoModelResponseValue!.Cleared); // True
  ```

  ```bash CLI theme={null}
  mka1 llm models delete-org-auto-model \
    --endpoint responses
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \
    --request DELETE \
    --header 'Authorization: Bearer <mka1-api-key>'
  ```
</CodeGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.