> ## Documentation Index > Fetch the complete documentation index at: https://docs.mka1.com/llms.txt > Use this file to discover all available pages before exploring further. # Manage models > See which models your organization can call, activate catalog entries in its registry, and choose what `model: "auto"` resolves to. Every request that names a model, including `model: "auto"`, resolves through your organization's model registry. This guide covers the two layers behind that lookup and the operations that change them. * The **catalog** is what your organization could activate: its own bring-your-own (BYO) definitions, models it serves on Compute, and the cluster models a cluster admin has granted it access to. * The **registry** is what it has activated. A model name resolves only while the registry holds it. Access to a cluster model does nothing by itself. Until someone activates the entry, `GET /models` omits it, a request that names it fails to resolve, and `auto` cannot land on it either. ## Before you start | Requirement | Notes | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | API key | Send it as `Authorization: Bearer ` on every request. | | Owner or admin role | Catalog, registry, and auto-model operations reject a bearer whose organization role is `member` with `403 Organization admin access required`. Listing models and effective prices needs no role. | | `read:registry` scope | Lets a key list the catalog, the registry, and auto-model overrides. Any member can mint it, but the role check above still applies. | | `write:registry` scope | Lets a key activate and deactivate entries, define and health-check BYO entries, and set auto-model overrides. Only owners and admins can mint it. Console sessions carry no scopes and pass on role alone. | The console has the same views under **Admin → Model Registry**, on the **Models**, **Auto models**, **Add model**, and **Raw JSON** tabs. **LLM → Models** shows what your requests can use right now. ## Step 1 - List the models your requests can use `GET /models` returns the active entries of your registry, minus any whose definition sets `hidden: true` (see [Troubleshooting](#troubleshooting)). Each item carries `model_type` and a `capabilities` object with `supports_temperature` and `supports_top_p`. Both are `false` for anything that is not an LLM, so read them before sending sampling parameters. The `mka1` CLI covers only `llm models list` and `llm models get`, so the remaining steps show the SDKs and curl. ```bash CLI theme={null} mka1 llm models list -H 'X-On-Behalf-Of: ' mka1 llm models get --model-id meetkai:functionary ``` ```ts MKA1 SDK theme={null} import { SDK } from '@meetkai/mka1'; const mka1 = new SDK({ bearerAuth: 'Bearer ' }); const models = await mka1.llm.models.list({ xOnBehalfOf: '', // optional — attribute the request to one of your end users }); console.log(models.data.map((m) => `${m.id} (${m.modelType})`)); const model = await mka1.llm.models.get({ modelId: 'meetkai:functionary' }); console.log(model.capabilities.supportsTemperature, model.capabilities.supportsTopP); ``` ```csharp C# SDK theme={null} using MeetKai.MKA1; using MeetKai.MKA1.Types.Components; var sdk = new SDK(bearerAuth: "Bearer "); var models = await sdk.Llm.Models.ListAsync( xOnBehalfOf: "" // optional — attribute the request to one of your end users ); foreach (var m in models.Object!.Data) { Console.WriteLine($"{m.Id} ({m.ModelType})"); } var model = await sdk.Llm.Models.GetAsync(modelId: "meetkai:functionary"); Console.WriteLine(model.Object!.Capabilities.SupportsTemperature); ``` ```python Python SDK theme={null} from meetkai_mka1 import SDK sdk = SDK(bearer_auth="Bearer ") models = sdk.llm.models.list( x_on_behalf_of="", # optional — attribute the request to one of your end users ) print([f"{m.id} ({m.model_type})" for m in models.data]) model = sdk.llm.models.get(model_id="meetkai:functionary") print(model.capabilities.supports_temperature, model.capabilities.supports_top_p) ``` ```bash bash theme={null} curl https://apigw.mka1.com/api/v1/llm/models \ --header 'Authorization: Bearer ' \ --header 'X-On-Behalf-Of: ' curl https://apigw.mka1.com/api/v1/llm/models/meetkai:functionary \ --header 'Authorization: Bearer ' ``` A list item looks like this: ```json theme={null} { "id": "meetkai:functionary", "object": "model", "created": 1704067200, "owned_by": "meetkai", "model_type": "llm", "capabilities": { "supports_temperature": true, "supports_top_p": true } } ``` `GET /models/{model_id}` returns `404` for a name that is not in the catalog, for one that is in the catalog but inactive, and for one whose definition is hidden. The gateway does not say which, so a response cannot reveal what other organizations have. ## Step 2 - Read the catalog `GET /models/catalog` lists every definition your organization could activate, with its source, activation state, provider health, and a price preview. Cluster entries appear only after a cluster admin grants access to them. ```ts MKA1 SDK theme={null} const catalog = await mka1.llm.models.listCatalog(); for (const entry of catalog.data) { console.log(entry.modelId, entry.source, entry.activationState, entry.blockerSource); } ``` ```csharp C# SDK theme={null} var catalog = await sdk.Llm.Models.ListCatalogAsync(); foreach (var entry in catalog.CatalogListResponse!.Data) { Console.WriteLine($"{entry.ModelId} {entry.Source} {entry.ActivationState}"); } ``` ```python Python SDK theme={null} catalog = sdk.llm.models.list_catalog() for entry in catalog.data: print(entry.model_id, entry.source, entry.activation_state, entry.blocker_source) ``` ```bash bash theme={null} curl https://apigw.mka1.com/api/v1/llm/models/catalog \ --header 'Authorization: Bearer ' ``` Field notes: | Field | Meaning | | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `source` | Where the definition lives. `byo` is your organization's own entry, `serving` is a model served on Compute, and `cluster` is the operator's catalog, meaning the cluster admins who run your cluster; a cluster entry is visible only through an access grant. | | `activation_state` | `available`, `active`, or `available_name_blocked`. See [Activation states](#activation-states). | | `blocker_source` | For `available_name_blocked`, the source that holds the name. | | `health` | The last probe of the entry's provider. Health is per provider, not per activation. | | `effective_price` | `rates` plus a `source` of `org_override`, `cluster_default`, or `unpriced`. Cluster entries preview their default rate before activation. | The same model id can appear more than once, once per source. Only one of them can hold the name at a time. ## Step 3 - Activate a catalog entry Activation claims a model name for one source. Pass the id and the source you are activating from. ```ts MKA1 SDK theme={null} const entry = await mka1.llm.models.activateRegistryEntry({ modelId: 'meetkai:functionary', source: 'cluster', }); console.log(entry.activatedBy, entry.activatedAt); ``` ```csharp C# SDK theme={null} var entry = await sdk.Llm.Models.ActivateRegistryEntryAsync( request: new ActivateRegistryEntryRequest() { ModelId = "meetkai:functionary", Source = RegistrySource.Cluster, } ); Console.WriteLine(entry.RegistryEntryResponse!.ActivatedAt); ``` ```python Python SDK theme={null} entry = sdk.llm.models.activate_registry_entry( model_id="meetkai:functionary", source="cluster", ) print(entry.activated_by, entry.activated_at) ``` ```bash bash theme={null} curl https://apigw.mka1.com/api/v1/llm/models/registry \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --data '{"model_id": "meetkai:functionary", "source": "cluster"}' ``` The gateway checks two things, in order: 1. The entry exists for your organization from that source. A cluster id you have no access to, or a BYO id nobody defined, returns `422`. 2. Nobody else holds the name. If the same source already holds it, the call returns the existing activation. If a different source holds it, the call returns `409`. Neither case changes anything. The response carries `model_id`, `source`, `activated_by`, and `activated_at`. The activation applies to the next request. In the console, the **Activate** button on **Admin → Model Registry → Models** makes the same call. The setup wizard's first step activates every available cluster entry the same way, but only while nothing in the registry is active yet. ## Step 4 - Review the registry and deactivate `GET /models/registry` is the catalog filtered to `active` entries, so each row has the same shape as in Step 2. `DELETE /models/registry?id=...` releases a name. ```ts MKA1 SDK theme={null} const registry = await mka1.llm.models.listRegistry(); console.log(registry.data.map((entry) => `${entry.modelId} from ${entry.source}`)); const result = await mka1.llm.models.deactivateRegistryEntry({ id: 'meetkai:functionary' }); console.log(result.deactivated); ``` ```csharp C# SDK theme={null} var registry = await sdk.Llm.Models.ListRegistryAsync(); foreach (var entry in registry.RegistryManagementListResponse!.Data) { Console.WriteLine($"{entry.ModelId} from {entry.Source}"); } var result = await sdk.Llm.Models.DeactivateRegistryEntryAsync(id: "meetkai:functionary"); Console.WriteLine(result.DeactivateRegistryEntryResponse!.Deactivated); ``` ```python Python SDK theme={null} registry = sdk.llm.models.list_registry() print([f"{entry.model_id} from {entry.source}" for entry in registry.data]) result = sdk.llm.models.deactivate_registry_entry(id="meetkai:functionary") print(result.deactivated) ``` ```bash bash theme={null} curl https://apigw.mka1.com/api/v1/llm/models/registry \ --header 'Authorization: Bearer ' curl 'https://apigw.mka1.com/api/v1/llm/models/registry?id=meetkai:functionary' \ --request DELETE \ --header 'Authorization: Bearer ' ``` Deactivation removes the name from the registry and closes any organization price override for it (the `org_override` rate in Step 6). It does not touch the catalog definition, so you can activate the entry again later, from the same source or a different one. The call is idempotent: `deactivated` is `false` when nothing was active. Requests already in flight finish. The next request that names the model fails to resolve, and if an auto-model override pointed at it, `auto` fails for that endpoint too (see Step 5). ## Step 5 - Choose what `auto` resolves to The gateway resolves `model: "auto"` per endpoint. It first checks your organization's override for that endpoint, then falls back to the operator's default from the cluster's `model-registry.yaml`. Either target still has to be active in your registry. If neither resolves, the request fails with the same not-found error as any unknown model. The endpoint key is the model's API format: | Endpoint key | Requests it covers | | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `responses` | `POST /responses` and the agents built on it. `/extract` and `/classify` honor this override too; with no override they use the YAML's `autoModels.extract` and `autoModels.classify`, or `autoModels.responses` when those are not defined. A `completions`-format target is accepted here. | | `completions` | `POST /chat/completions` | | `embeddings` | `POST /embeddings` | | `images` | Image generation | | `transcriptions` | Speech to text | | `tts` | Text to speech | ```ts MKA1 SDK theme={null} const overrides = await mka1.llm.models.listOrgAutoModels(); console.log(overrides.data.map((o) => `${o.endpoint} -> ${o.modelId} (resolvable: ${o.resolvable})`)); const override = await mka1.llm.models.putOrgAutoModel({ endpoint: 'responses', putOrgAutoModelRequest: { modelId: 'meetkai:functionary' }, }); console.log(override.modelId, override.setBy); await mka1.llm.models.deleteOrgAutoModel({ endpoint: 'responses' }); ``` ```csharp C# SDK theme={null} var overrides = await sdk.Llm.Models.ListOrgAutoModelsAsync(); foreach (var o in overrides.ListOrgAutoModelsResponseValue!.Data) { Console.WriteLine($"{o.Endpoint} -> {o.ModelId} (resolvable: {o.Resolvable})"); } var over = await sdk.Llm.Models.PutOrgAutoModelAsync( endpoint: AutoEndpoint.Responses, body: new PutOrgAutoModelRequest() { ModelId = "meetkai:functionary" } ); Console.WriteLine(over.OrgAutoModel!.ModelId); await sdk.Llm.Models.DeleteOrgAutoModelAsync(endpoint: AutoEndpoint.Responses); ``` ```python Python SDK theme={null} overrides = sdk.llm.models.list_org_auto_models() print([f"{o.endpoint} -> {o.model_id} (resolvable: {o.resolvable})" for o in overrides.data]) override = sdk.llm.models.put_org_auto_model( endpoint="responses", model_id="meetkai:functionary", ) print(override.model_id, override.set_by) sdk.llm.models.delete_org_auto_model(endpoint="responses") ``` ```bash bash theme={null} curl https://apigw.mka1.com/api/v1/llm/models/auto \ --header 'Authorization: Bearer ' curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \ --request PUT \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --data '{"modelId": "meetkai:functionary"}' curl https://apigw.mka1.com/api/v1/llm/models/auto/responses \ --request DELETE \ --header 'Authorization: Bearer ' ``` `PUT` returns `422` when the target is the literal `auto`, is not active in your registry, or has a different API format than the endpoint. The gateway stores the canonical id and later looks up that exact name. If you deactivate that model, the override's `resolvable` flag turns `false` and `auto` fails on that endpoint rather than moving to the operator default on its own. Clear the override or activate the model again. `DELETE` is idempotent. The `PUT` body key is `modelId`, not the `model_id` the registry call takes. The console shows the same overrides on **Admin → Model Registry → Auto models**. ## Step 6 - Check effective prices `GET /models/pricing/effective` returns the rate your organization pays for every model it can call. Any bearer can read it, with no role or scope, because anyone whose requests are billed can see the rate they are billed at. Audit fields such as who set a price stay on the admin pricing routes. ```ts MKA1 SDK theme={null} const prices = await mka1.llm.models.listEffectivePrices({}); console.log(prices.currency); for (const row of prices.data) { console.log(row.model, row.source, row.rates); } ``` ```csharp C# SDK theme={null} var prices = await sdk.Llm.Models.ListEffectivePricesAsync(); Console.WriteLine(prices.EffectivePricesResponse!.Currency); foreach (var row in prices.EffectivePricesResponse!.Data) { Console.WriteLine($"{row.Model} {row.Source}"); } ``` ```python Python SDK theme={null} prices = sdk.llm.models.list_effective_prices() print(prices.currency) for row in prices.data: print(row.model, row.source, row.rates) ``` ```bash bash theme={null} curl https://apigw.mka1.com/api/v1/llm/models/pricing/effective \ --header 'Authorization: Bearer ' ``` `source` is `org_override` when a price override was set for your organization through `PUT /models/pricing/org/{orgId}`, `cluster_default` when a cluster entry uses the operator's rate card, and `unpriced` (billed as 0) otherwise. An organization admin can set that override for their own organization's active `byo` and `serving` entries with `setOrgPrice`, read it back with `listOrgPrices`, and remove it with `clearOrgPrice`; an override on a `cluster` entry is cluster-admin only. `currency` is the cluster-wide ISO 4217 code, or `null` until the operator sets one. ## Bring your own model An organization admin can define a BYO entry in the catalog and then activate it with `source: "byo"`. Defining does not activate. This is the flow the [Deploy a model server](/docs/compute-deployment#step-5---register-with-the-llm-gateway) guide uses to register a vLLM endpoint. Only curl is shown; the SDKs' `addCatalogEntry` takes the same fields. ```bash bash theme={null} curl https://apigw.mka1.com/api/v1/llm/models/catalog \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --data '{ "id": "meetkai:custom-chat-v1", "provider": "openai-compatible", "modelId": "custom-chat-v1", "displayName": "Custom Chat v1", "baseUrl": "https://models.example.com/v1", "apiFormat": "responses", "apiProviderType": "openai", "created": 1704067200, "auth": {"type": "api-key", "value": ""}, "capabilities": {"modalities": {"input": ["text"], "output": ["text"]}, "reasoning": false} }' ``` `id` is the name your requests send as `model`; `modelId` is the name the gateway forwards to the provider at `baseUrl`. The response is a catalog entry with the auth value redacted and `activation_state` set to `available`, or `available_name_blocked` if another source already holds the name. Activate it as in Step 3 with `source: "byo"`. `provider` is a free-form label, and entries that share one share a health probe. `apiProviderType` is the wire protocol the gateway speaks to `baseUrl`; `openai` covers any OpenAI-compatible server. All four BYO operations need `write:registry`, the health check included. | SDK operation | Route | Effect | | -------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------- | | `addCatalogEntry` | `POST /models/catalog` | Defines the entry, as above. | | `updateCatalogEntry` | `PATCH /models/catalog` | Changes an existing entry. Send the `id` plus the fields to change. | | `deleteCatalogEntry` | `DELETE /models/catalog` | Withdraws the definition, and deactivates the name only if this BYO entry was the one holding it. | | `checkCatalogHealth` | `POST /models/catalog/health` | Probes the provider whether or not the entry is active. | Entries the operator manages in `model-registry.yaml` cannot be changed through the API. ## For cluster admins Everything in this section needs a cluster admin bearer. The four access operations, the two reads included, also need that bearer bound to the cluster organization (`org_cluster`); cluster prices and the currency need only the cluster-admin flag, from any organization. Organization admins get `403` here. Cluster admins decide which cluster models each organization may activate. A grant adds an entry to the organization's catalog as `available`; it activates nothing. In the console this is **Access → Organizations → Cluster**, then the organization's **Available models** tab, where you stage toggles, and **Save** sends the whole set through `replaceCatalogOrgAccess`. ```ts MKA1 SDK theme={null} const access = await mka1.llm.models.listCatalogOrgAccess({ orgId: 'org_acme' }); console.log(access.data.map((row) => `${row.modelId}: ${row.activationState}`)); await mka1.llm.models.addCatalogOrgAccess({ orgId: 'org_acme', accessMutationRequest: { modelIds: ['meetkai:functionary'] }, }); ``` ```csharp C# SDK theme={null} var access = await sdk.Llm.Models.ListCatalogOrgAccessAsync(orgId: "org_acme"); foreach (var row in access.AccessListResponse!.Data) { Console.WriteLine($"{row.ModelId}: {row.ActivationState}"); } await sdk.Llm.Models.AddCatalogOrgAccessAsync( orgId: "org_acme", body: new AccessMutationRequest() { ModelIds = new List { "meetkai:functionary" } } ); ``` ```python Python SDK theme={null} access = sdk.llm.models.list_catalog_org_access(org_id="org_acme") print([f"{row.model_id}: {row.activation_state}" for row in access.data]) sdk.llm.models.add_catalog_org_access(org_id="org_acme", model_ids=["meetkai:functionary"]) ``` ```bash bash theme={null} curl https://apigw.mka1.com/api/v1/llm/models/catalog/orgs/org_acme/access \ --header 'Authorization: Bearer ' curl https://apigw.mka1.com/api/v1/llm/models/catalog/orgs/org_acme/access \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --data '{"model_ids": ["meetkai:functionary"]}' ``` The operations, and what each one touches: | SDK operation | Route | Effect | | ----------------------------------------------------------- | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `addCatalogOrgAccess` | `POST /models/catalog/orgs/{orgId}/access` | Adds ids to the organization's access set. Idempotent; never activates or revokes. | | `replaceCatalogOrgAccess` | `PUT /models/catalog/orgs/{orgId}/access` | Replaces the whole set (up to 1000 ids). The gateway deactivates any active cluster name that drops out of the set and closes its organization price override. The console warns before you revoke a model that another model falls back to. | | `listCatalogOrgAccess` | `GET /models/catalog/orgs/{orgId}/access` | One organization's cluster entries with their activation state. | | `listCatalogModelAccess` | `GET /models/catalog/access` | The reverse lookup: every organization with access to one cluster model id, and whether each has activated it. | | `listClusterPrices`, `setClusterPrice`, `clearClusterPrice` | `GET`, `PUT`, `DELETE /models/pricing` | The cluster-default rate card that `cluster_default` prices come from. | | `getClusterCurrency`, `setClusterCurrency` | `GET`, `PUT /models/pricing/currency` | The single cluster-wide currency. Changing it does not convert existing prices. | The two reads require `read:registry` and the two mutations `write:registry`, as at the organization level. Both mutations reject an unknown cluster catalog id with `422 Unknown cluster catalog model id(s): ...`. The cluster organization activates its own catalog with `source: "byo"` or `"serving"`; `"cluster"` is for tenant organizations only, meaning every organization other than `org_cluster`. ## Activation states | State | Meaning | What to do | | ------------------------ | ------------------------------------------------------------------------------------------------------ | ------------------------------------------------ | | `available` | No source holds this name. | Activate it. | | `active` | This entry holds the name. Requests resolve to it. | Deactivate it to release the name. | | `available_name_blocked` | Another source holds the name; `blocker_source` says which. The console shows this as **name in use**. | Deactivate the holder, then activate this entry. | ## Troubleshooting | Symptom | Cause | Fix | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `404 model_not_found` with `Model 'x' not found. Check the model ID and try again.` from `/responses` (`/chat/completions` and agents say `Model 'x' not found or not supported`), or `404` from `GET /models/{model_id}` | The name is not active in your registry, its definition is hidden, or the key belongs to a different organization. | Find the entry in the catalog and activate it. If it is already `active`, check that the key belongs to the same organization and that the definition is not hidden (see the `hidden: true` row). | | `auto` fails while named models work (`/extract` and `/classify` report `Model 'auto' is not available to your organization for extract.`) | No override is set and the operator's default is not active in your registry; the override's target was deactivated (`resolvable: false`); or the cluster's YAML defines no `autoModels` entry for that endpoint. | Activate the default target, set an override to an active model, or clear the stale override. With no YAML default, only an override makes `auto` work on that endpoint. | | `409 Model 'x' is active from source 'byo'` on activation | Another source holds the name. | Deactivate that entry first, or keep it and leave this one unactivated. | | `422 Catalog entry 'x' from source 'cluster' is not available` | No access grant for that id, the wrong `source`, or a BYO id nobody defined. | Check the catalog (Step 2) for the id and its source; ask a cluster admin for access. | | `422 The cluster org must activate its own catalog with source 'byo' or 'serving'` | You are acting in `org_cluster`. | Use the entry's real source, or switch to the tenant organization. | | `403 Organization admin access required` | The bearer's organization role is `member`. | Use an owner or admin key, or a console session with that role. | | `403 API key is missing required scope(s): write:registry` | The key was minted without the scope. | Mint a key with `write:registry` (owners and admins only). | | An entry is `active` in the catalog but missing from `GET /models`, and requests naming it return `404` | Its definition sets `hidden: true`. A hidden definition never resolves for a user request; it is reachable only as the fallback target of a reroute rule (`rerouteRules` on the definition). | Expected. Point requests at a non-hidden entry. | | A cluster model disappeared from the catalog and requests to it fail | A cluster admin revoked access, which deactivates the name. | Ask for the access grant back, then activate again. | ## API reference For the full request and response schemas, open the Models group in the [API Reference](/api-reference/models/list-available-models). ## See also * [Getting started](/docs/platform-getting-started#enable-your-models) - the setup wizard's activation step. * [Deploy a model server](/docs/compute-deployment) - serve your own weights and register them as a BYO entry. * [Generate a response](/docs/generate-a-response) - call an activated model.