Auto routing

When a request uses “model”: “auto”, MKA1 applies the organization’s auto policy for that endpoint. By default, auto is a direct alias to a model in the model registry. Organizations can also configure an LLM judge to select from an approved set of models. The judge is given signals about the request, the available models, and the existing conversation so it can choose the model best suited to the work.

Supported endpoints

Judge-based selection is available for several types of AI request.

Endpoint	Candidate type
Responses and Chat Completions	LLM
Audio Transcriptions	Speech-to-text
Text-to-Speech	Text-to-speech

Only models suited to the request type are considered. For example, a TTS model cannot be selected for a Responses request.

How model selection works

The organization defines the models that may be selected. For each request, MKA1 excludes candidates that are no longer active in the organization’s registry or that cannot handle the request, such as a model without image input for a request that contains an image. If only one suitable model remains, MKA1 selects it directly; when several are available, the configured judge chooses among them. Each judged request makes one extra model call to the judge. The call is billed at the judge model’s price and counts toward your budgets, but it is not included in the response’s usage. It also adds a full judge round trip before the selected model starts, and a streamed response opens only after the judge has chosen. A judge policy does not fail over. If the selected model fails, the request fails: MKA1 does not try another candidate, and the candidates’ reroute rules do not apply. Provider health is not a selection signal, so a selected model whose provider is unavailable also fails the request. If no suitable model remains, or if the configured judge is unavailable or fails, MKA1 returns a transparent error instead of silently changing the policy. See Errors.

Signals used for model selection

MKA1 gives the judge a consistent view of the factors that matter for the current request:

Request and tools: the nature of the task, available tools, requested output, and whether images, audio, or files are present. Models that can work with the original input directly are preferred.
Model capabilities: supported modalities and file types, available context, and other requirements needed to complete the request.
Price: each candidate’s per-unit rates at your organization’s effective price. A model without a price counts as free.
Latency: rolling latency and failure-rate estimates that the gateway measures on live traffic.
Evals: the most recent completed eval runs from your own team that score two or more of the candidates. Platform benchmarks are not used.
Conversation continuity: for a Responses request that continues a conversation, which candidate served the previous response and how many tokens it processed, so the judge can weigh staying on that model to keep cost and latency down.

The selected policy tells the judge which of these signals matters most:

balanced considers quality, price, speed, capabilities, and conversation continuity together.
price prioritizes the lowest expected cost among models suited to the request.
quality prioritizes task fit and evaluation results, and ignores conversation continuity.
speed prioritizes responsiveness using rolling latency and reliability signals.

Capability requirements always apply, regardless of policy. The best choice can change as the request, model availability, performance, or conversation evolves.

Privacy and safety

The judge receives a bounded, sanitized summary of the request and its tool requirements. Credentials and raw binary content are not forwarded. The organization’s normal safety policies still apply, and the judge can select only from models the organization has approved for the policy.

Inspect the current policy

The auto endpoints are organization admin operations: the API key needs the read:registry scope to list overrides and write:registry to change them. Each override’s mode is either direct or the judge policy name, so read that field first to know which other fields are present.

import { SDK } from '@meetkai/mka1';
import type { ResponseObject } from '@meetkai/mka1/models/components';

const sdk = new SDK({ bearerAuth: 'Bearer <mka1-api-key>' });

const overrides = await sdk.llm.models.listOrgAutoModels();
for (const override of overrides.data) {
  switch (override.mode) {
    case 'direct':
      console.log(override.endpoint, '->', override.modelId, 'resolvable:', override.resolvable);
      break;
    case 'balanced':
    case 'price':
    case 'quality':
    case 'speed':
      console.log(
        override.endpoint, '->', override.mode,
        'judge:', override.judgeModelId,
        'candidates:', override.resolvableCandidateModelIds,
      );
      break;
  }
}

from meetkai_mka1 import SDK

sdk = SDK(bearer_auth="Bearer <mka1-api-key>")

overrides = sdk.llm.models.list_org_auto_models()
for override in overrides.data:
    if override.mode == "direct":
        print(override.endpoint, "->", override.model_id, "resolvable:", override.resolvable)
    else:
        print(
            override.endpoint, "->", override.mode,
            "judge:", override.judge_model_id,
            "candidates:", override.resolvable_candidate_model_ids,
        )

using MeetKai.MKA1;
using MeetKai.MKA1.Types.Components;
using System.Collections.Generic;

var sdk = new SDK(bearerAuth: "Bearer <mka1-api-key>");

var overrides = await sdk.Llm.Models.ListOrgAutoModelsAsync();
foreach (var o in overrides.ListOrgAutoModelsResponseValue!.Data)
{
    if (o.DirectOrgAutoModel is { } direct)
    {
        Console.WriteLine($"{direct.Endpoint} -> {direct.ModelId} (resolvable: {direct.Resolvable})");
    }
    else if (o.BalancedOrgAutoModel is { } judged)
    {
        Console.WriteLine($"{judged.Endpoint} -> balanced, judge {judged.JudgeModelId}, " +
            $"candidates {string.Join(", ", judged.ResolvableCandidateModelIds)}");
    }
    // price, quality and speed overrides arrive on o.PriceOrgAutoModel,
    // o.QualityOrgAutoModel and o.SpeedOrgAutoModel with the same fields.
}

mka1 llm models get \
  --model-id auto

curl https://apigw.mka1.com/api/v1/llm/models/auto \
  --header 'Authorization: Bearer <mka1-api-key>'

A direct override resembles:

{
  "object": "list",
  "data": [
    {
      "endpoint": "responses",
      "mode": "direct",
      "model_id": "meetkai:functionary-pt",
      "resolvable": true,
      "set_by": "user-org-admin-1",
      "set_at": "2026-09-01T12:00:00.000Z"
    }
  ]
}

Configure an LLM judge

The following example enables judge-based model selection for LLM requests using the balanced policy. Set policy to price, quality, or speed to emphasize a different goal. The judge and every candidate must be valid LLM model identifiers available to the organization. Choose a fast, non-reasoning judge that supports structured outputs, such as openai:gpt-4.1-nano: the judge runs before every judged request and must return its choice as structured JSON. Model availability is organization-specific; replace the example IDs with IDs returned by GET /models for your organization.

const override = await sdk.llm.models.putOrgAutoModel({
  endpoint: 'responses',
  putOrgAutoModelRequest: {
    policy: 'balanced',
    judgeModelId: 'openai:gpt-4.1-nano',
    candidateModelIds: ['meetkai:functionary-pt', 'openai:gpt-4.1-mini'],
  },
});
if (override.mode === 'balanced') {
  console.log(override.judgeResolvable);              // true
  console.log(override.resolvableCandidateModelIds);  // both candidates
}

override = sdk.llm.models.put_org_auto_model(
    endpoint="responses",
    body={
        "policy": "balanced",
        "judge_model_id": "openai:gpt-4.1-nano",
        "candidate_model_ids": ["meetkai:functionary-pt", "openai:gpt-4.1-mini"],
    },
)
print(override.mode)                            # "balanced"
print(override.judge_resolvable)                # True
print(override.resolvable_candidate_model_ids)  # both candidates

var judged = await sdk.Llm.Models.PutOrgAutoModelAsync(
    endpoint: AutoEndpoint.Responses,
    body: PutOrgAutoModelRequestUnion.CreatePutOrgAutoModelRequest2(new PutOrgAutoModelRequest2()
    {
        Policy = Policy.Balanced,
        JudgeModelId = "openai:gpt-4.1-nano",
        CandidateModelIds = new List<string> { "meetkai:functionary-pt", "openai:gpt-4.1-mini" },
    })
);
var balanced = judged.OneOf!.BalancedOrgAutoModel!;
Console.WriteLine(balanced.JudgeResolvable);                               // True
Console.WriteLine(string.Join(", ", balanced.ResolvableCandidateModelIds)); // both candidates

mka1 llm models put-org-auto-model \
  --body '{
    "policy": "balanced",
    "judge_model_id": "openai:gpt-4.1-nano",
    "candidate_model_ids": [
      "meetkai:functionary-pt",
      "openai:gpt-4.1-mini"
    ]
  }' \
  --endpoint responses

curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \
  --request PUT \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --data '{
    "policy": "balanced",
    "judge_model_id": "openai:gpt-4.1-nano",
    "candidate_model_ids": [
      "meetkai:functionary-pt",
      "openai:gpt-4.1-mini"
    ]
  }'

The stored override reports the policy as its mode, plus which of the configured models currently resolve:

{
  "endpoint": "responses",
  "mode": "balanced",
  "judge_model_id": "openai:gpt-4.1-nano",
  "candidate_model_ids": ["meetkai:functionary-pt", "openai:gpt-4.1-mini"],
  "judge_resolvable": true,
  "resolvable_candidate_model_ids": ["meetkai:functionary-pt", "openai:gpt-4.1-mini"],
  "set_by": "user-org-admin-1",
  "set_at": "2026-09-01T12:00:00.000Z"
}

This candidate set pairs a Portuguese-tuned model with a general one: a request in Portuguese can route to meetkai:functionary-pt, while other requests can route to openai:gpt-4.1-mini. You can configure responses, completions, transcriptions, and tts independently. Endpoints without an organization override retain direct model-registry alias routing.

Validate the selected model

Make a normal request using “model”: “auto”:

const response = await sdk.llm.responses.create({
  xOnBehalfOf: '<end-user-id>', // optional — attribute the request to one of your end users
  responsesCreateRequest: {
    model: 'auto',
    input: 'Resuma este relatório de incidente e liste os três próximos passos de maior risco.',
    maxOutputTokens: 300,
  },
}) as ResponseObject;
console.log(response.model);         // "auto"
console.log(response.resolvedModel); // e.g. "meetkai:functionary-pt"

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: '<mka1-api-key>',
  baseURL: 'https://apigw.mka1.com/api/v1/llm/',
  defaultHeaders: { 'X-On-Behalf-Of': '<end-user-id>' },
});

const response = await openai.responses.create({
  model: 'auto',
  input: 'Resuma este relatório de incidente e liste os três próximos passos de maior risco.',
  max_output_tokens: 300,
  stream: false,
});
console.log(response.model); // "auto"
// resolved_model is an MKA1 extension, so it is not on the OpenAI SDK's response type.
console.log((response as { resolved_model?: string }).resolved_model); // e.g. "meetkai:functionary-pt"

res = sdk.llm.responses.create(
    model="auto",
    input="Resuma este relatório de incidente e liste os três próximos passos de maior risco.",
    max_output_tokens=300,
    x_on_behalf_of="<end-user-id>",  # optional — attribute the request to one of your end users
)
print(res.model)           # "auto"
print(res.resolved_model)  # e.g. "meetkai:functionary-pt"

var res = await sdk.Llm.Responses.CreateAsync(new ResponsesCreateRequest()
{
    Model = "auto",
    Input = ResponsesCreateRequestInput.CreateStr(
        "Resuma este relatório de incidente e liste os três próximos passos de maior risco."),
    MaxOutputTokens = 300,
});
Console.WriteLine(res.ResponseObject!.Model);         // auto
Console.WriteLine(res.ResponseObject!.ResolvedModel); // e.g. meetkai:functionary-pt

mka1 llm responses create \
  --model auto \
  --input '"Resuma este relatório de incidente e liste os três próximos passos de maior risco."' \
  -H 'X-On-Behalf-Of: <end-user-id>'

curl https://apigw.mka1.com/api/v1/llm/responses \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --data '{
    "model": "auto",
    "input": "Resuma este relatório de incidente e liste os três próximos passos de maior risco.",
    "max_output_tokens": 300
  }'

The response echoes the alias you sent in its model field and names the model that actually served the request in resolved_model. With a direct policy, that is the registry alias target, or the model a reroute rule sent the request to. With an LLM judge configured, it is the approved candidate selected for the request. resolved_model is a Responses API field. Chat Completions returns only the requested name in model and does not report the model that served the request. When you stream a response, read resolved_model from the final event, such as response.completed. The response.created event carries only the first resolution, which a reroute can change later in the request. A background request with a judge policy reports no resolved_model until the final event.

Errors

When auto routing cannot select a model, the request fails with HTTP 503 and one of these codes in error.code:

Code	Cause
`auto_judge_unavailable`	The judge model is not active in the organization’s registry, is hidden, or is not an LLM.
`auto_no_eligible_models`	No candidate can handle the request, or the selected candidate stopped being available before the request was sent to it.
`auto_judge_failed`	The judge call failed or took longer than 30 seconds, or the judge did not return one of the candidates.

A streamed request receives the error as a normal HTTP response, before the stream opens. A background request that fails selection after it was queued ends with status: “failed” and the same code in error.code. In GET /models/auto, judge_resolvable: false points to auto_judge_unavailable, and an empty resolvable_candidate_model_ids to auto_no_eligible_models.

Restore direct routing

Set a direct policy explicitly:

const override = await sdk.llm.models.putOrgAutoModel({
  endpoint: 'responses',
  putOrgAutoModelRequest: { modelId: 'meetkai:functionary-pt' },
});
if (override.mode === 'direct') {
  console.log(override.modelId, override.resolvable); // "meetkai:functionary-pt" true
}

override = sdk.llm.models.put_org_auto_model(
    endpoint="responses",
    body={"model_id": "meetkai:functionary-pt"},
)
print(override.mode, override.model_id, override.resolvable)  # direct meetkai:functionary-pt True

var direct = await sdk.Llm.Models.PutOrgAutoModelAsync(
    endpoint: AutoEndpoint.Responses,
    body: PutOrgAutoModelRequestUnion.CreatePutOrgAutoModelRequest1(
        new PutOrgAutoModelRequest1() { ModelId = "meetkai:functionary-pt" })
);
Console.WriteLine(direct.OneOf!.DirectOrgAutoModel!.ModelId); // meetkai:functionary-pt

mka1 llm models put-org-auto-model \
  --body '{ "model_id": "meetkai:functionary-pt" }' \
  --endpoint responses

curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \
  --request PUT \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --data '{ "model_id": "meetkai:functionary-pt" }'

Or, clear your org’s override so “model”: “auto” falls through to the cluster default again:

const cleared = await sdk.llm.models.deleteOrgAutoModel({ endpoint: 'responses' });
console.log(cleared.cleared); // true

cleared = sdk.llm.models.delete_org_auto_model(endpoint="responses")
print(cleared.cleared)  # True

var cleared = await sdk.Llm.Models.DeleteOrgAutoModelAsync(endpoint: AutoEndpoint.Responses);
Console.WriteLine(cleared.DeleteOrgAutoModelResponseValue!.Cleared); // True

mka1 llm models delete-org-auto-model \
  --endpoint responses

curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \
  --request DELETE \
  --header 'Authorization: Bearer <mka1-api-key>'

Overview

Setup

Model management

Operations

Access control

Security and compliance

Benchmarks

Auto routing

Supported endpoints

How model selection works

Signals used for model selection

Privacy and safety

Inspect the current policy

Configure an LLM judge

Validate the selected model

Errors

Restore direct routing

​Supported endpoints

​How model selection works

​Signals used for model selection

​Privacy and safety

​Inspect the current policy

​Configure an LLM judge

​Validate the selected model

​Errors

​Restore direct routing

Supported endpoints

How model selection works

Signals used for model selection

Privacy and safety

Inspect the current policy

Configure an LLM judge

Validate the selected model

Errors

Restore direct routing