> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mka1.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy a model server

> <Badge color="blue" shape="pill">Early access</Badge>

Serve a model behind an authenticated HTTP endpoint with a Compute service, pull weights from mka1-repos, and register the endpoint with the LLM gateway.


Use a Compute service when you need GPUs for a persistent workload such as model serving.
A service is the same generic container run as a [job](/docs/compute-fine-tune-job), plus named ports, an optional readiness probe, and a public endpoint — Compute has no deployment or model schema, so this guide serves with [vLLM](https://docs.vllm.ai) purely as workload payload.

GPU services run on external GPU providers, and CPU services run in the MKA1 cluster.
This guide's example requests a GPU, so it runs on an external GPU provider.
The two placements protect endpoints differently; see [Endpoint access](#endpoint-access).

This example creates a single persistent model server. Keep its process in the foreground and terminate the service when finished.
Some placements also support the public `service.scaling` and volume fields; support depends on the selected backend. Do not assume every external GPU provider supports shared storage or dynamic replicas.

The CLI examples assume an [authenticated MKA1 CLI](/docs/cli/authentication).

## Before you start

You need:

| Requirement | Notes |
| - | - |
| API key | Send it as `Authorization: Bearer <mka1-api-key>` on every request. |
| Compute enabled for your organization | Compute is a fail-closed allowlist. If requests return `403 tenant_disabled`, ask a cluster administrator to enable your organization. |
| A serving image | Any Linux GPU image the provider can pull without interactive authentication, keeping the server process in the foreground. |
| Weights to serve | A public model identifier, or a repository in mka1-repos — for example one published by a [fine-tune job](/docs/compute-fine-tune-job). |
| An API key for the workload itself | Only for a service on an external GPU provider, whose serving runtime must authenticate requests itself. Generate a strong random string for this. MKA1 cluster endpoints need no workload key; see [Endpoint access](#endpoint-access). |

Services move through `requested` → `allocating` → `provisioning` → `ready`, and stay there until something ends the run:

* A failure in any phase, including the workload exiting after `ready`, tears down through `terminating` into `failed`.
* Explicit termination, a binding limit, or an exhausted organization budget ends in `terminated` instead.

With a readiness probe, `ready` means the probe passed.
Without one, it only means the workload launched; it makes no application-level health claim, so declare a probe whenever the image offers a health route.

<Warning>
  **Billing starts at allocation, not readiness.**

  Spend accrues from allocation until the service reaches a terminal state, including all provisioning and model-loading time.
  A service never completes on its own — terminate it when you are done, and set `limits` as a backstop.
</Warning>

## Endpoint access

A service's placement decides who authenticates requests to its endpoints.

**MKA1 cluster endpoints** are always authenticated by the MKA1 API gateway.
Send an MKA1 API key of the organization that owns the service, holding the admin-only `read:compute` [scope](/docs/authentication#create-a-key-in-the-console).
A request without a valid key gets `401`.
A key that lacks `read:compute`, or that belongs to another organization, gets `403`.
The API gateway strips the `Authorization` header and cookies before forwarding a request, so your container never receives a caller's credential.
Do not configure the serving runtime to require a key of its own.

**External GPU provider endpoints** are reachable at the provider's URL, with no MKA1 authentication in front of them.
Protect them with the serving runtime's own key, as this guide does with `VLLM_API_KEY`.

## Step 1 - Store the endpoint key as a secret

Pass the workload API key through `secret_env`, not inline in the command, so it never appears in provider pod specs.
Skip this step for a CPU service: the MKA1 API gateway authenticates its endpoint.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  import { SDK } from '@meetkai/mka1';

  const sdk = new SDK({ bearerAuth: 'Bearer <mka1-api-key>' });

  const result = await sdk.computeSecrets.createComputeSecret({
    "computeSecretCreate": {
      "name": "inference-endpoint-key",
      "data": {
        "VLLM_API_KEY": "<generated-endpoint-key>"
      }
    }
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.id);
  ```

  ```python Python SDK theme={null}
  from meetkai_mka1 import models
  import os
  from openai import OpenAI
  from meetkai_mka1 import SDK

  sdk = SDK(bearer_auth="Bearer <mka1-api-key>")

  result = sdk.compute_secrets.create_compute_secret(
      name="inference-endpoint-key",
      data={"VLLM_API_KEY": "<generated-endpoint-key>"},
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.id)
  ```

  ```csharp C# SDK theme={null}
  using MeetKai.MKA1.Types.Components;
  using MeetKai.MKA1.Types.Requests;
  using Newtonsoft.Json;
  using OpenAI;
  using OpenAI.Chat;
  using System.ClientModel;
  using MeetKai.MKA1;

  var sdk = new SDK(bearerAuth: "Bearer <mka1-api-key>");

  // Deserialize the wire-format payload into the SDK request model.
  var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.ComputeSecretCreate> ("""
  {
    "name": "inference-endpoint-key",
    "data": {
      "VLLM_API_KEY": "<generated-endpoint-key>"
    }
  }
  """)!;

  var response = await sdk.ComputeSecrets.CreateComputeSecretAsync(body);
  var result = response.ComputeSecret
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Id));
  ```

  ```bash CLI theme={null}
  mka1 compute-secrets create \
    --output-format json \
    --body @- <<'JSON'
  {
    "data": {
      "VLLM_API_KEY": "<generated-endpoint-key>"
    },
    "name": "inference-endpoint-key"
  }
  JSON
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/compute/secrets \
    --request POST \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $MKA1_API_KEY" \
    --data '{
      "name": "inference-endpoint-key",
      "data": {"VLLM_API_KEY": "<generated-endpoint-key>"}
    }'
  ```
</CodeGroup>

Secret values are write-only.
Keep the returned `sec_...` id for the create body, and the key value itself for your clients.

## Step 2 - Create the service

Create the service with a required `Idempotency-Key` header, exactly as for jobs.
This example serves Qwen2.5-0.5B-Instruct with vLLM's OpenAI-compatible server, matching the platform smoke test:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  // Keep this value unchanged when retrying this request.
  const idempotencyKey = process.env.MKA1_REQUEST_ID;
  if (!idempotencyKey) throw new Error('Set MKA1_REQUEST_ID');

  const result = await sdk.computeServices.createService({
    "idempotencyKey": idempotencyKey,
    "computeResourceCreate": {
      "name": "qwen-inference",
      "compute": {
        "accelerator": "nvidia-rtx-4090-24gb",
        "gpuCount": 1,
        "ephemeralDiskGb": 60
      },
      "container": {
        "image": "vllm/vllm-openai:v0.26.0-cu129",
        "command": [
          "vllm",
          "serve",
          "Qwen/Qwen2.5-0.5B-Instruct",
          "--host",
          "0.0.0.0",
          "--port",
          "8000",
          "--max-model-len",
          "4096",
          "--gpu-memory-utilization",
          "0.85",
          "--enable-auto-tool-choice",
          "--tool-call-parser",
          "hermes"
        ],
        "secretEnv": {
          "VLLM_API_KEY": {
            "secretId": "sec_Qw8pLm2vTn5xRc7J",
            "key": "VLLM_API_KEY"
          }
        },
        "ports": [
          {
            "name": "api",
            "containerPort": 8000,
            "protocol": "http"
          }
        ]
      },
      "service": {
        "readiness": {
          "type": "http",
          "port": "api",
          "path": "/health",
          "successStatus": 200
        }
      },
      "limits": {
        "maxRuntimeHours": 2,
        "maxCostUsd": 5
      }
    }
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.id);
  ```

  ```python Python SDK theme={null}
  idempotency_key = os.environ["MKA1_REQUEST_ID"]  # Reuse for retries.

  result = sdk.compute_services.create_service(
      idempotency_key=idempotency_key,
      name="qwen-inference",
      compute={
          "accelerator": "nvidia-rtx-4090-24gb",
          "gpu_count": 1,
          "ephemeral_disk_gb": 60,
      },
      container={
          "image": "vllm/vllm-openai:v0.26.0-cu129",
          "command": [
              "vllm",
              "serve",
              "Qwen/Qwen2.5-0.5B-Instruct",
              "--host",
              "0.0.0.0",
              "--port",
              "8000",
              "--max-model-len",
              "4096",
              "--gpu-memory-utilization",
              "0.85",
              "--enable-auto-tool-choice",
              "--tool-call-parser",
              "hermes",
          ],
          "secret_env": {
              "VLLM_API_KEY": {"secret_id": "sec_Qw8pLm2vTn5xRc7J", "key": "VLLM_API_KEY"}
          },
          "ports": [{"name": "api", "container_port": 8000, "protocol": "http"}],
      },
      service={
          "readiness": {
              "type": "http",
              "port": "api",
              "path": "/health",
              "success_status": 200,
          }
      },
      limits={"max_runtime_hours": 2, "max_cost_usd": 5},
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.id)
  ```

  ```csharp C# SDK theme={null}
  var idempotencyKey = System.Environment.GetEnvironmentVariable("MKA1_REQUEST_ID")
      ?? throw new InvalidOperationException("Set MKA1_REQUEST_ID");

  // Deserialize the wire-format payload into the SDK request model.
  var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.ComputeResourceCreate> ("""
  {
    "name": "qwen-inference",
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "gpu_count": 1,
      "ephemeral_disk_gb": 60
    },
    "container": {
      "image": "vllm/vllm-openai:v0.26.0-cu129",
      "command": [
        "vllm",
        "serve",
        "Qwen/Qwen2.5-0.5B-Instruct",
        "--host",
        "0.0.0.0",
        "--port",
        "8000",
        "--max-model-len",
        "4096",
        "--gpu-memory-utilization",
        "0.85",
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "hermes"
      ],
      "secret_env": {
        "VLLM_API_KEY": {
          "secret_id": "sec_Qw8pLm2vTn5xRc7J",
          "key": "VLLM_API_KEY"
        }
      },
      "ports": [
        {
          "name": "api",
          "container_port": 8000,
          "protocol": "http"
        }
      ]
    },
    "service": {
      "readiness": {
        "type": "http",
        "port": "api",
        "path": "/health",
        "success_status": 200
      }
    },
    "limits": {
      "max_runtime_hours": 2,
      "max_cost_usd": 5
    }
  }
  """)!;

  var response = await sdk.ComputeServices.CreateServiceAsync(idempotencyKey, body);
  var result = response.ComputeService
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Id));
  ```

  ```bash CLI theme={null}
  mka1 compute-services create-service \
    --idempotency-key "$MKA1_REQUEST_ID" \
    --output-format json \
    --body @- <<'JSON'
  {
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "ephemeral_disk_gb": 60,
      "gpu_count": 1
    },
    "container": {
      "command": [
        "vllm",
        "serve",
        "Qwen/Qwen2.5-0.5B-Instruct",
        "--host",
        "0.0.0.0",
        "--port",
        "8000",
        "--max-model-len",
        "4096",
        "--gpu-memory-utilization",
        "0.85",
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "hermes"
      ],
      "image": "vllm/vllm-openai:v0.26.0-cu129",
      "ports": [
        {
          "container_port": 8000,
          "name": "api",
          "protocol": "http"
        }
      ],
      "secret_env": {
        "VLLM_API_KEY": {
          "key": "VLLM_API_KEY",
          "secret_id": "sec_Qw8pLm2vTn5xRc7J"
        }
      }
    },
    "limits": {
      "max_cost_usd": 5,
      "max_runtime_hours": 2
    },
    "name": "qwen-inference",
    "service": {
      "readiness": {
        "path": "/health",
        "port": "api",
        "success_status": 200,
        "type": "http"
      }
    }
  }
  JSON
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/compute/services \
    --request POST \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $MKA1_API_KEY" \
    --header "Idempotency-Key: $MKA1_REQUEST_ID" \
    --data '{
    "name": "qwen-inference",
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "gpu_count": 1,
      "ephemeral_disk_gb": 60
    },
    "container": {
      "image": "vllm/vllm-openai:v0.26.0-cu129",
      "command": [
        "vllm",
        "serve",
        "Qwen/Qwen2.5-0.5B-Instruct",
        "--host",
        "0.0.0.0",
        "--port",
        "8000",
        "--max-model-len",
        "4096",
        "--gpu-memory-utilization",
        "0.85",
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "hermes"
      ],
      "secret_env": {
        "VLLM_API_KEY": {
          "secret_id": "sec_Qw8pLm2vTn5xRc7J",
          "key": "VLLM_API_KEY"
        }
      },
      "ports": [
        {
          "name": "api",
          "container_port": 8000,
          "protocol": "http"
        }
      ]
    },
    "service": {
      "readiness": {
        "type": "http",
        "port": "api",
        "path": "/health",
        "success_status": 200
      }
    },
    "limits": {
      "max_runtime_hours": 2,
      "max_cost_usd": 5
    }
  }'
  ```
</CodeGroup>

Field notes:

| Field | Notes |
| - | - |
| `container.ports` | Each port needs a unique name, a container port, and a protocol of `http` or `tcp`. Every named port is exposed on the service's public endpoint. |
| `service.readiness` | References a declared port by name, never by number. An `http` probe requires that port's protocol to be `http`; a `tcp` probe takes only the port name. |
| `secret_env.VLLM_API_KEY` | vLLM reads `VLLM_API_KEY` and requires it on inference requests; the health route is separate. Set it only on an external GPU provider. |
| `--enable-auto-tool-choice --tool-call-parser hermes` | Required if the [LLM gateway](#step-5---register-with-the-llm-gateway) will call this endpoint: the gateway sends `tool_choice: "auto"` by default, and an unflagged vLLM rejects that with a `400`. |
| `placement` | Optional, same as jobs, and chooses among external GPU providers: `strategy` is `capacity_optimized` (default) or `lowest_cost`, and `providers` is an unordered filter — see the [fine-tune guide's field notes](/docs/compute-fine-tune-job#step-3---create-the-job). |

As with jobs, `201` means the request was accepted; capacity and provisioning problems surface afterwards through `state`, `reason`, logs, and events.

## Step 3 - Wait for ready

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeServices.getService({
    "id": "service_2mVx7cKq9dRw4bTn"
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.endpoints);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_services.get_service(
      id="service_2mVx7cKq9dRw4bTn",
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.endpoints)
  ```

  ```csharp C# SDK theme={null}
  var response = await sdk.ComputeServices.GetServiceAsync("service_2mVx7cKq9dRw4bTn");
  var result = response.ComputeService
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Endpoints));
  ```

  ```bash CLI theme={null}
  mka1 compute-services get-service \
    --id service_2mVx7cKq9dRw4bTn \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/compute/services/service_2mVx7cKq9dRw4bTn \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

Model download and load happen inside the workload, so expect minutes of `provisioning` before the readiness probe passes.
A ready service carries its public endpoints:

```json theme={null}
{
  "id": "service_2mVx7cKq9dRw4bTn",
  "name": "qwen-inference",
  "state": "ready",
  "ready": true,
  "reason": null,
  "compute": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "ephemeral_disk_gb": 60},
  "hardware": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "gpu_memory_gb": 24},
  "price_usd_hr": 0.46,
  "accrued_usd": 0.12,
  "allocated_at": "2026-07-29T13:02:41Z",
  "limits": {},
  "endpoints": [
    {
      "name": "api",
      "protocol": "http",
      "host": "svc-2mvx7ckq.endpoints.example.net",
      "port": 443,
      "url": "https://svc-2mvx7ckq.endpoints.example.net"
    }
  ],
  "created_at": "2026-07-29T13:01:12Z",
  "started_at": "2026-07-29T13:06:30Z",
  "terminal_at": null
}
```

If the service lands in `failed` instead, read `reason`, then `GET .../logs?source=user` for the server's own output and `source=system` for allocation and bootstrap problems — the same logs and events surface as [jobs](/docs/compute-fine-tune-job#step-5---read-logs-and-events).

## Step 4 - Call your endpoint

The endpoint is plain vLLM: an OpenAI-compatible API.

The calls below use the official OpenAI-compatible client libraries directly against your vLLM endpoint, rather than the MKA1 control-plane SDK.
Install `openai@5.0.0` for TypeScript, `OpenAI` 2.4.0 for C#, or `openai==1.109.1` for Python.
Replace `https://<model-server-host>` with the endpoint's `url`, and `<model-server-api-key>` with the credential its placement accepts: the endpoint key from Step 1 on an external GPU provider, or an MKA1 API key holding `read:compute` in the MKA1 cluster.
Use an HTTPS endpoint for these credentialed requests.

<CodeGroup>
  ```ts OpenAI SDK theme={null}
  import OpenAI from 'openai';

  const openai = new OpenAI({
    apiKey: '<model-server-api-key>',
    baseURL: 'https://<model-server-host>/v1',
  });

  const result = await openai.chat.completions.create({
    model: 'Qwen/Qwen2.5-0.5B-Instruct',
    messages: [{ role: 'user', content: 'Say hello.' }],
  });
  console.log(result.choices[0]?.message.content);
  ```

  ```python Python SDK theme={null}
  client = OpenAI(
      api_key="<model-server-api-key>",
      base_url="https://<model-server-host>/v1",
  )
  result = client.chat.completions.create(
      model="Qwen/Qwen2.5-0.5B-Instruct",
      messages=[{"role": "user", "content": "Say hello."}],
  )
  print(result.choices[0].message.content)
  ```

  ```csharp C# SDK theme={null}
  var key = "<model-server-api-key>";
  var endpoint = "https://<model-server-host>/v1";
  var client = new ChatClient(
      model: "Qwen/Qwen2.5-0.5B-Instruct",
      credential: new ApiKeyCredential(key),
      options: new OpenAIClientOptions { Endpoint = new Uri(endpoint) });
  ChatCompletion result = await client.CompleteChatAsync("Say hello.");
  Console.WriteLine(result.Content[0].Text);
  ```

  ```bash Bash theme={null}
  curl "https://<model-server-host>/v1/chat/completions" \
    --request POST \
    --header 'Content-Type: application/json' \
    --header 'Authorization: Bearer <model-server-api-key>' \
    --data '{"model":"Qwen/Qwen2.5-0.5B-Instruct","messages":[{"role":"user","content":"Say hello."}]}'
  ```
</CodeGroup>

For the external GPU endpoint illustrated here, vLLM authenticates inference requests using the endpoint key. Readiness uses its health route, which does not require that key.

<Warning>
  Check the endpoint's `url` scheme before sending anything sensitive.
  MKA1 cluster endpoints always use `https`, but an external GPU provider may serve an endpoint over plain `http`, in which case bearer tokens and payloads cross the internet unencrypted.
</Warning>

## Step 5 - Register with the LLM gateway

Optionally register the endpoint as a bring-your-own model in the MKA1 LLM gateway, so it is callable through the platform's `/responses` API with normal platform keys.

Add the endpoint to your model catalog using an organization admin or owner key. The catalog `id` names your MKA1 model; `modelId` must match the upstream model name served by vLLM.

`auth.value` is the credential the LLM gateway sends to your endpoint, so it depends on the placement:

* **External GPU provider**: the serving runtime's key. The example below registers this guide's endpoint with the endpoint key from Step 1.
* **MKA1 cluster**: an MKA1 API key of the organization that owns the service, holding `read:compute`. Prefer a dedicated key with only that scope. The MKA1 API gateway rejects a runtime key with `401`.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.llm.models.addCatalogEntry({
    "id": "qwen-compute-demo",
    "provider": "openai-compatible",
    "modelId": "Qwen/Qwen2.5-0.5B-Instruct",
    "displayName": "Qwen Compute demo",
    "created": 1789488000,
    "capabilities": {
      "modalities": {
        "input": [
          "text"
        ],
        "output": [
          "text"
        ]
      },
      "reasoning": false
    },
    "apiFormat": "completions",
    "apiProviderType": "openai",
    "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1",
    "auth": {
      "type": "api-key",
      "value": "<generated-endpoint-key>"
    }
  });
  console.log(result.modelId);
  ```

  ```python Python SDK theme={null}
  result = sdk.llm.models.add_catalog_entry(
      id="qwen-compute-demo",
      provider="openai-compatible",
      model_id="Qwen/Qwen2.5-0.5B-Instruct",
      display_name="Qwen Compute demo",
      created=1789488000,
      capabilities={
          "modalities": {"input": ["text"], "output": ["text"]},
          "reasoning": False,
      },
      api_format="completions",
      api_provider_type="openai",
      base_url="https://svc-2mvx7ckq.endpoints.example.net/v1",
      auth={"type": "api-key", "value": "<generated-endpoint-key>"},
  )
  print(result.model_id)
  ```

  ```csharp C# SDK theme={null}
  // Deserialize the wire-format payload into the SDK request model.
  var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.AddCatalogModelRequest> ("""
  {
    "id": "qwen-compute-demo",
    "provider": "openai-compatible",
    "modelId": "Qwen/Qwen2.5-0.5B-Instruct",
    "displayName": "Qwen Compute demo",
    "created": 1789488000,
    "capabilities": {
      "modalities": {
        "input": [
          "text"
        ],
        "output": [
          "text"
        ]
      },
      "reasoning": false
    },
    "apiFormat": "completions",
    "apiProviderType": "openai",
    "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1",
    "auth": {
      "type": "api-key",
      "value": "<generated-endpoint-key>"
    }
  }
  """)!;

  var response = await sdk.Llm.Models.AddCatalogEntryAsync(body);
  var result = response.CatalogEntry
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.ModelId));
  ```

  ```bash CLI theme={null}
  mka1 llm models add-catalog-entry \
    --output-format json \
    --body @- <<'JSON'
  {
    "id": "qwen-compute-demo",
    "provider": "openai-compatible",
    "modelId": "Qwen/Qwen2.5-0.5B-Instruct",
    "displayName": "Qwen Compute demo",
    "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1",
    "apiFormat": "completions",
    "apiProviderType": "openai",
    "created": 1789488000,
    "auth": {
      "type": "api-key",
      "value": "<generated-endpoint-key>"
    },
    "capabilities": {
      "modalities": {
        "input": [
          "text"
        ],
        "output": [
          "text"
        ]
      },
      "reasoning": false
    }
  }
  JSON
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/llm/models/catalog \
    --request POST \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $MKA1_API_KEY" \
    --data '{
      "id": "qwen-compute-demo",
      "provider": "openai-compatible",
      "modelId": "Qwen/Qwen2.5-0.5B-Instruct",
      "displayName": "Qwen Compute demo",
      "created": 1789488000,
      "capabilities": {"modalities": {"input": ["text"], "output": ["text"]}, "reasoning": false},
      "apiFormat": "completions",
      "apiProviderType": "openai",
      "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1",
      "auth": {"type": "api-key", "value": "<generated-endpoint-key>"}
    }'
  ```
</CodeGroup>

Then activate the catalog model using its returned `model_id` (the `id` you supplied above):

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.llm.models.activateRegistryEntry({
    "source": "byo",
    "modelId": "qwen-compute-demo"
  });
  console.log(result.modelId);
  ```

  ```python Python SDK theme={null}
  result = sdk.llm.models.activate_registry_entry(
      source="byo",
      model_id="qwen-compute-demo",
  )
  print(result.model_id)
  ```

  ```csharp C# SDK theme={null}
  // Deserialize the wire-format payload into the SDK request model.
  var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.ActivateRegistryEntryRequest> ("""
  {
    "source": "byo",
    "model_id": "qwen-compute-demo"
  }
  """)!;

  var response = await sdk.Llm.Models.ActivateRegistryEntryAsync(body);
  var result = response.RegistryEntryResponse
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.ModelId));
  ```

  ```bash CLI theme={null}
  mka1 llm models activate-registry-entry \
    --output-format json \
    --body @- <<'JSON'
  {
    "source": "byo",
    "model_id": "qwen-compute-demo"
  }
  JSON
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/llm/models/registry \
    --request POST \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $MKA1_API_KEY" \
    --data '{
      "source": "byo",
      "model_id": "qwen-compute-demo"
    }'
  ```
</CodeGroup>

`completions`-format models are accepted at `/responses`; the gateway drives the chat-completions upstream for you.
This is why the create body in Step 2 passed the tool-choice flags to vLLM.

## Step 6 - Terminate

A service runs, and bills, until you stop it.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeServices.terminateService({
    "id": "service_2mVx7cKq9dRw4bTn"
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.state);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_services.terminate_service(
      id="service_2mVx7cKq9dRw4bTn",
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.state)
  ```

  ```csharp C# SDK theme={null}
  var response = await sdk.ComputeServices.TerminateServiceAsync("service_2mVx7cKq9dRw4bTn");
  var result = response.ComputeService
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.State));
  ```

  ```bash CLI theme={null}
  mka1 compute-services terminate-service \
    --id service_2mVx7cKq9dRw4bTn \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/compute/services/service_2mVx7cKq9dRw4bTn/terminate \
    --request POST \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

Termination is idempotent, deallocates the provider resource, and stops spend; the record stays readable for audit and usage.

Terminating does not remove a catalog entry you added in Step 5.
The model stays active, and requests to it return `404`; remove it in Step 7.

## Step 7 - Remove the model from the LLM gateway

If you registered the endpoint in Step 5, deactivate the model and delete its catalog entry, using the same organization admin or owner key:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const deactivation = await sdk.llm.models.deactivateRegistryEntry({
    "id": "qwen-compute-demo"
  });
  console.log(deactivation.deactivated);

  const deletion = await sdk.llm.models.deleteCatalogEntry({
    "id": "qwen-compute-demo"
  });
  console.log(deletion.deleted);
  ```

  ```python Python SDK theme={null}
  deactivation = sdk.llm.models.deactivate_registry_entry(
      id="qwen-compute-demo",
  )
  print(deactivation.deactivated)

  deletion = sdk.llm.models.delete_catalog_entry(
      id="qwen-compute-demo",
  )
  print(deletion.deleted)
  ```

  ```csharp C# SDK theme={null}
  var deactivateResponse = await sdk.Llm.Models.DeactivateRegistryEntryAsync(id: "qwen-compute-demo");
  var deactivation = deactivateResponse.DeactivateRegistryEntryResponse
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(deactivateResponse));
  Console.WriteLine(JsonConvert.SerializeObject(deactivation.Deactivated));

  var deleteResponse = await sdk.Llm.Models.DeleteCatalogEntryAsync(id: "qwen-compute-demo");
  var deletion = deleteResponse.DeleteCatalogModelResponse
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(deleteResponse));
  Console.WriteLine(JsonConvert.SerializeObject(deletion.Deleted));
  ```

  ```bash CLI theme={null}
  mka1 llm models deactivate-registry-entry \
    --id qwen-compute-demo \
    --output-format json

  mka1 llm models delete-catalog-entry \
    --id qwen-compute-demo \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl 'https://apigw.mka1.com/api/v1/llm/models/registry?id=qwen-compute-demo' \
    --request DELETE \
    --header "Authorization: Bearer $MKA1_API_KEY"

  curl 'https://apigw.mka1.com/api/v1/llm/models/catalog?id=qwen-compute-demo' \
    --request DELETE \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

Deactivating releases the model name, so requests to `qwen-compute-demo` stop resolving; deleting withdraws its catalog definition.

## Serve a model from mka1-repos

To serve weights from [mka1-repos](/docs/repositories) — such as the merged output of a [fine-tune job](/docs/compute-fine-tune-job#use-mka1-repos-for-datasets-and-weights), or weights you [pushed over git](/docs/repositories#push-and-pull-with-git) — change only the model identifier and the credentials.

Set `HF_ENDPOINT` on the service, pointing at the artifact endpoint for your cluster, and vLLM resolves model identifiers against it — the same Hugging Face access described in [Manage repositories](/docs/repositories#push-and-pull-with-the-hugging-face-cli), so serving a repository is just a model identifier and two environment variables. The console pre-fills `HF_ENDPOINT` when you create a workload; set it yourself when you call the API directly.
mka1-repos authenticates with your MKA1 API key, injected as `HF_TOKEN` through `secret_env`:

```json theme={null}
{
  "container": {
    "image": "vllm/vllm-openai:v0.26.0-cu129",
    "command": ["vllm", "serve", "meetkai/functionary", "--host", "0.0.0.0", "--port", "8000", "--max-model-len", "4096", "--gpu-memory-utilization", "0.85", "--enable-auto-tool-choice", "--tool-call-parser", "hermes"],
    "env": {
      "HF_ENDPOINT": "https://hf.mka1.com"
    },
    "secret_env": {
      "VLLM_API_KEY": {"secret_id": "sec_Qw8pLm2vTn5xRc7J", "key": "VLLM_API_KEY"},
      "HF_TOKEN": {"secret_id": "sec_Vb3nRk8sQw1xYz2M", "key": "HF_TOKEN"}
    },
    "ports": [
      {"name": "api", "container_port": 8000, "protocol": "http"}
    ]
  }
}
```

The service downloads the weights from your organization's repository at startup and serves them under the same identifier, completing the loop: fine-tune as a job, publish to mka1-repos, serve as a service.

## Track spend

Service spend accrues per whole minute from allocation and appears alongside jobs in the usage endpoint:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeUsage.getComputeUsage({
    "resourceType": "service"
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.summary);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_usage.get_compute_usage(
      resource_type="service",
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.summary)
  ```

  ```csharp C# SDK theme={null}
  var request = new GetComputeUsageRequest
  {
      ResourceType = ComputeUsageResourceType.Service,
      Limit = 50,
  };

  var response = await sdk.ComputeUsage.GetComputeUsageAsync(request);
  var result = response.ComputeUsage
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Summary));
  ```

  ```bash CLI theme={null}
  mka1 compute-usage get \
    --limit 50 \
    --resource-type service \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl "https://apigw.mka1.com/api/v1/compute/usage?resource_type=service" \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

Compute spend draws from the same organization budgets as every other MKA1 service, and a service that hits an organization budget or its own `limits` is terminated with no grace period.

## API reference

For the full request and response schemas, open the Compute groups in the [API Reference](/api-reference/introduction).

## See also

* [Manage resources](/docs/compute-resources) - inspect capacity, workloads, secrets, and volumes.

* [Run a fine-tune job](/docs/compute-fine-tune-job) - produce the weights this service serves.

* [Manage repositories](/docs/repositories) - create and manage the repositories this service pulls weights from.

* [Generate a response](/docs/generate-a-response) - call your registered model through the platform `/responses` API.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.