Auto routing

When a request uses “model”: “auto”, MKA1 applies the organization’s auto policy for that endpoint. By default, auto is a direct alias to a model in the model registry. Organizations can also configure an LLM judge to select from an approved set of models. The judge is given signals about the request, the available models, and the existing conversation so it can choose the model best suited to the work.

Supported endpoints

Judge-based selection is available for several types of AI request.

Endpoint	Candidate type
Responses and Chat Completions	LLM
Audio Transcriptions	Speech-to-text
Text-to-Speech	Text-to-speech

Only models suited to the request type are considered. For example, a TTS model cannot be selected for a Responses request.

How model selection works

The organization defines the models that may be selected. Models that are no longer available are excluded automatically. If only one suitable model remains, MKA1 selects it directly; when several are available, the configured judge chooses among them. If no suitable model remains, or if the configured judge is unavailable, MKA1 returns a transparent error instead of silently changing the policy.

Signals used for model selection

MKA1 gives the judge a consistent view of the factors that matter for the current request:

Request and tools: the nature of the task, available tools, requested output, and whether images, audio, or files are present. Models that can work with the original input directly are preferred.
Model capabilities: supported modalities and file types, available context, and other requirements needed to complete the request.
Price: comparative model pricing.
Latency: rolling latency estimates measured by the gateway.
Evals: relevant evaluations and benchmark results for the available models.
Conversation continuity: whether staying with the previously selected model can preserve context and reduce cost and latency, including through KV-cache awareness.

The selected policy tells the judge which of these signals matters most:

balanced considers quality, price, speed, capabilities, and conversation continuity together.
price prioritizes the lowest expected cost among models suited to the request.
quality prioritizes task fit and relevant evaluation results.
speed prioritizes responsiveness using rolling latency and reliability signals.

Capability requirements always apply, regardless of policy. The best choice can change as the request, model availability, performance, or conversation evolves.

Privacy and safety

The judge receives a bounded, sanitized summary of the request and its tool requirements. Credentials and raw binary content are not forwarded. The organization’s normal safety policies still apply, and the judge can select only from models the organization has approved for the policy.

Inspect the current policy

The auto endpoints are organization admin operations: the API key needs the read:registry scope to list overrides and write:registry to change them. Each override’s mode is either direct or the judge policy name, so read that field first to know which other fields are present.

import { SDK } from '@meetkai/mka1';

const mka1 = new SDK({
  bearerAuth: 'Bearer <mka1-api-key>',
});

const overrides = await mka1.llm.models.listOrgAutoModels();
for (const override of overrides.data) {
  switch (override.mode) {
    case 'direct':
      console.log(override.endpoint, '->', override.modelId, 'resolvable:', override.resolvable);
      break;
    case 'balanced':
    case 'price':
    case 'quality':
    case 'speed':
      console.log(
        override.endpoint, '->', override.mode,
        'judge:', override.judgeModelId,
        'candidates:', override.resolvableCandidateModelIds,
      );
      break;
  }
}

using MeetKai.MKA1;
using MeetKai.MKA1.Types.Components;

var sdk = new SDK(bearerAuth: "Bearer <mka1-api-key>");

var overrides = await sdk.Llm.Models.ListOrgAutoModelsAsync();
foreach (var o in overrides.ListOrgAutoModelsResponseValue!.Data)
{
    if (o.DirectOrgAutoModel is { } direct)
    {
        Console.WriteLine($"{direct.Endpoint} -> {direct.ModelId} (resolvable: {direct.Resolvable})");
    }
    else if (o.BalancedOrgAutoModel is { } judged)
    {
        Console.WriteLine($"{judged.Endpoint} -> balanced, judge {judged.JudgeModelId}, " +
            $"candidates {string.Join(", ", judged.ResolvableCandidateModelIds)}");
    }
    // price, quality and speed overrides arrive on o.PriceOrgAutoModel,
    // o.QualityOrgAutoModel and o.SpeedOrgAutoModel with the same fields.
}

from meetkai_mka1 import SDK

sdk = SDK(bearer_auth="Bearer <mka1-api-key>")

overrides = sdk.llm.models.list_org_auto_models()
for override in overrides.data:
    if override.mode == "direct":
        print(override.endpoint, "->", override.model_id, "resolvable:", override.resolvable)
    else:
        print(
            override.endpoint, "->", override.mode,
            "judge:", override.judge_model_id,
            "candidates:", override.resolvable_candidate_model_ids,
        )

curl https://apigw.mka1.com/api/v1/llm/models/auto \
  --header 'Authorization: Bearer <mka1-api-key>'

A direct override resembles:

{
  "object": "list",
  "data": [
    {
      "endpoint": "responses",
      "mode": "direct",
      "model_id": "meetkai:functionary-pt",
      "resolvable": true,
      "set_by": "user-org-admin-1",
      "set_at": "2026-09-01T12:00:00.000Z"
    }
  ]
}

Configure an LLM judge

The following example enables judge-based model selection for LLM requests using the balanced policy. Set policy to price, quality, or speed to emphasize a different goal. The judge and every candidate must be valid LLM model identifiers available to the organization. Model availability is organization-specific; replace the example IDs with IDs returned by GET /models for your organization.

const override = await mka1.llm.models.putOrgAutoModel({
  endpoint: 'responses',
  putOrgAutoModelRequest: {
    policy: 'balanced',
    judgeModelId: 'meetkai:functionary-pt',
    candidateModelIds: ['meetkai:functionary-pt', 'meetkai:functionary-es-419'],
  },
});
if (override.mode === 'balanced') {
  console.log(override.judgeResolvable);              // true
  console.log(override.resolvableCandidateModelIds);  // both candidates
}

using System.Collections.Generic;

var judged = await sdk.Llm.Models.PutOrgAutoModelAsync(
    endpoint: AutoEndpoint.Responses,
    body: PutOrgAutoModelRequestUnion.CreatePutOrgAutoModelRequest2(new PutOrgAutoModelRequest2()
    {
        Policy = Policy.Balanced,
        JudgeModelId = "meetkai:functionary-pt",
        CandidateModelIds = new List<string> { "meetkai:functionary-pt", "meetkai:functionary-es-419" },
    })
);
var balanced = judged.OneOf!.BalancedOrgAutoModel!;
Console.WriteLine(balanced.JudgeResolvable);                               // True
Console.WriteLine(string.Join(", ", balanced.ResolvableCandidateModelIds)); // both candidates

override = sdk.llm.models.put_org_auto_model(
    endpoint="responses",
    body={
        "policy": "balanced",
        "judge_model_id": "meetkai:functionary-pt",
        "candidate_model_ids": ["meetkai:functionary-pt", "meetkai:functionary-es-419"],
    },
)
print(override.mode)                            # "balanced"
print(override.judge_resolvable)                # True
print(override.resolvable_candidate_model_ids)  # both candidates

curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \
  --request PUT \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --data '{
    "policy": "balanced",
    "judge_model_id": "meetkai:functionary-pt",
    "candidate_model_ids": [
      "meetkai:functionary-pt",
      "meetkai:functionary-es-419"
    ]
  }'

The stored override reports the policy as its mode, plus which of the configured models currently resolve:

{
  "endpoint": "responses",
  "mode": "balanced",
  "judge_model_id": "meetkai:functionary-pt",
  "candidate_model_ids": ["meetkai:functionary-pt", "meetkai:functionary-es-419"],
  "judge_resolvable": true,
  "resolvable_candidate_model_ids": ["meetkai:functionary-pt", "meetkai:functionary-es-419"],
  "set_by": "user-org-admin-1",
  "set_at": "2026-09-01T12:00:00.000Z"
}

This candidate set illustrates language-aware selection: a request in Portuguese can route to meetkai:functionary-pt, while a request in Latin American Spanish can route to meetkai:functionary-es-419. You can configure responses, completions, transcriptions, and tts independently. Endpoints without an organization override retain direct model-registry alias routing.

Validate the selected model

Make a normal request using “model”: “auto”:

mka1 llm responses create \
  --model auto \
  --input '"Resuma este relatório de incidente e liste os três próximos passos de maior risco."' \
  -H 'X-On-Behalf-Of: <end-user-id>'

import type { ResponseObject } from '@meetkai/mka1/models/components';

const response = await mka1.llm.responses.create({
  xOnBehalfOf: '<end-user-id>', // optional — attribute the request to one of your end users
  responsesCreateRequest: {
    model: 'auto',
    input: 'Resuma este relatório de incidente e liste os três próximos passos de maior risco.',
    maxOutputTokens: 300,
  },
}) as ResponseObject;
console.log(response.model);         // "auto"
console.log(response.resolvedModel); // e.g. "meetkai:functionary-pt"

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: '<mka1-api-key>',
  baseURL: 'https://apigw.mka1.com/api/v1/llm/',
  defaultHeaders: { 'X-On-Behalf-Of': '<end-user-id>' },
});

const response = await openai.responses.create({
  model: 'auto',
  input: 'Resuma este relatório de incidente e liste os três próximos passos de maior risco.',
  max_output_tokens: 300,
  stream: false,
});
console.log(response.model); // "auto"
// resolved_model is an MKA1 extension, so it is not on the OpenAI SDK's response type.
console.log((response as { resolved_model?: string }).resolved_model); // e.g. "meetkai:functionary-pt"

var res = await sdk.Llm.Responses.CreateAsync(new ResponsesCreateRequest()
{
    Model = "auto",
    Input = ResponsesCreateRequestInput.CreateStr(
        "Resuma este relatório de incidente e liste os três próximos passos de maior risco."),
    MaxOutputTokens = 300,
});
Console.WriteLine(res.ResponseObject!.Model);         // auto
Console.WriteLine(res.ResponseObject!.ResolvedModel); // e.g. meetkai:functionary-pt

res = sdk.llm.responses.create(
    model="auto",
    input="Resuma este relatório de incidente e liste os três próximos passos de maior risco.",
    max_output_tokens=300,
    x_on_behalf_of="<end-user-id>",  # optional — attribute the request to one of your end users
)
print(res.model)           # "auto"
print(res.resolved_model)  # e.g. "meetkai:functionary-pt"

curl https://apigw.mka1.com/api/v1/llm/responses \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --data '{
    "model": "auto",
    "input": "Resuma este relatório de incidente e liste os três próximos passos de maior risco.",
    "max_output_tokens": 300
  }'

The response echoes the alias you sent in its model field and names the model that actually served the request in resolved_model. With a direct policy, that is the registry alias target. With an LLM judge configured, it is the approved candidate selected for the request.

Restore direct routing

Set a direct policy explicitly:

const override = await mka1.llm.models.putOrgAutoModel({
  endpoint: 'responses',
  putOrgAutoModelRequest: { modelId: 'meetkai:functionary-pt' },
});
if (override.mode === 'direct') {
  console.log(override.modelId, override.resolvable); // "meetkai:functionary-pt" true
}

var direct = await sdk.Llm.Models.PutOrgAutoModelAsync(
    endpoint: AutoEndpoint.Responses,
    body: PutOrgAutoModelRequestUnion.CreatePutOrgAutoModelRequest1(
        new PutOrgAutoModelRequest1() { ModelId = "meetkai:functionary-pt" })
);
Console.WriteLine(direct.OneOf!.DirectOrgAutoModel!.ModelId); // meetkai:functionary-pt

override = sdk.llm.models.put_org_auto_model(
    endpoint="responses",
    body={"model_id": "meetkai:functionary-pt"},
)
print(override.mode, override.model_id, override.resolvable)  # direct meetkai:functionary-pt True

curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \
  --request PUT \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --data '{ "model_id": "meetkai:functionary-pt" }'

Or, clear your org’s override so “model”: “auto” falls through to the cluster default again:

const cleared = await mka1.llm.models.deleteOrgAutoModel({ endpoint: 'responses' });
console.log(cleared.cleared); // true

var cleared = await sdk.Llm.Models.DeleteOrgAutoModelAsync(endpoint: AutoEndpoint.Responses);
Console.WriteLine(cleared.DeleteOrgAutoModelResponseValue!.Cleared); // True

cleared = sdk.llm.models.delete_org_auto_model(endpoint="responses")
print(cleared.cleared)  # True

curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \
  --request DELETE \
  --header 'Authorization: Bearer <mka1-api-key>'

Getting started

Responses

Features

Compute

CLI

Recipes

Benchmarks

Infrastructure

Auto routing

Supported endpoints

How model selection works

Signals used for model selection

Privacy and safety

Inspect the current policy

Configure an LLM judge

Validate the selected model

Restore direct routing

​Supported endpoints

​How model selection works

​Signals used for model selection

​Privacy and safety

​Inspect the current policy

​Configure an LLM judge

​Validate the selected model

​Restore direct routing

Supported endpoints

How model selection works

Signals used for model selection

Privacy and safety

Inspect the current policy

Configure an LLM judge

Validate the selected model

Restore direct routing