Skip to main content
When a request uses “model”: “auto”, MKA1 applies the organization’s auto policy for that endpoint. By default, auto is a direct alias to a model in the model registry. Organizations can also configure an LLM judge to select from an approved set of models. The judge is given signals about the request, the available models, and the existing conversation so it can choose the model best suited to the work.

​
Supported endpoints

Judge-based selection is available for several types of AI request. Only models suited to the request type are considered. For example, a TTS model cannot be selected for a Responses request.

​
How model selection works

The organization defines the models that may be selected. Models that are no longer available are excluded automatically. If only one suitable model remains, MKA1 selects it directly; when several are available, the configured judge chooses among them. If no suitable model remains, or if the configured judge is unavailable, MKA1 returns a transparent error instead of silently changing the policy.

​
Signals used for model selection

MKA1 gives the judge a consistent view of the factors that matter for the current request:
  • Request and tools: the nature of the task, available tools, requested output, and whether images, audio, or files are present. Models that can work with the original input directly are preferred.
  • Model capabilities: supported modalities and file types, available context, and other requirements needed to complete the request.
  • Price: comparative model pricing.
  • Latency: rolling latency estimates measured by the gateway.
  • Evals: relevant evaluations and benchmark results for the available models.
  • Conversation continuity: whether staying with the previously selected model can preserve context and reduce cost and latency, including through KV-cache awareness.
The selected policy tells the judge which of these signals matters most:
  • balanced considers quality, price, speed, capabilities, and conversation continuity together.
  • price prioritizes the lowest expected cost among models suited to the request.
  • quality prioritizes task fit and relevant evaluation results.
  • speed prioritizes responsiveness using rolling latency and reliability signals.
Capability requirements always apply, regardless of policy. The best choice can change as the request, model availability, performance, or conversation evolves.

​
Privacy and safety

The judge receives a bounded, sanitized summary of the request and its tool requirements. Credentials and raw binary content are not forwarded. The organization’s normal safety policies still apply, and the judge can select only from models the organization has approved for the policy.

​
Inspect the current policy

bash
A direct override resembles:

​
Configure an LLM judge

The following example enables judge-based model selection for LLM requests using the balanced policy. Set policy to price, quality, or speed to emphasize a different goal. The judge and every candidate must be valid LLM model identifiers available to the organization.
bash
This candidate set illustrates language-aware selection: a request in Portuguese can route to meetkai:functionary-pt, while a request in Latin American Spanish can route to meetkai:functionary-es. You can configure responses, completions, transcriptions, and tts independently. Endpoints without an organization override retain direct model-registry alias routing.

​
Validate the selected model

Make a normal request using “model”: “auto”:
bash
The response’s model field identifies the model selected by the policy. With a direct policy, it is the registry alias target. With an LLM judge configured, it is the approved candidate selected for the request.

​
Restore direct routing

Set a direct policy explicitly:
bash
Or, clear your org’s override so “model”: “auto” falls through to the cluster default again:
bash