Deploy a model server

Use a Compute service when you need GPUs for a persistent workload such as model serving. A service is the same generic container run as a job, plus named ports, an optional readiness probe, and a public endpoint — Compute has no deployment or model schema, so this guide serves with vLLM purely as workload payload. A service runs until you terminate it, and you own the lifecycle end to end: Compute never restarts, rescales, or swaps revisions behind your back, so what you deployed is exactly what is running. That predictability comes with two rules: keep the server process in the foreground for the life of the service — a workload that exits, even with code 0, ends the service as failed — and replace a failed or outdated service by creating a new one.

Before you start

You need:

Requirement	Notes
API key	Send it as `Authorization: Bearer <mka1-api-key>` on every request.
Compute enabled for your organization	Compute is a fail-closed allowlist. If requests return `403 tenant_disabled`, ask a cluster administrator to enable your organization.
A serving image	Any Linux GPU image the provider can pull without interactive authentication, keeping the server process in the foreground.
Weights to serve	A public model identifier, or a repository in mka1-repos — for example one published by a fine-tune job.
An API key for the workload itself	Compute exposes your port to the internet and does not authenticate calls to it; your server must. Generate a strong random string for this.

Services move through requested → allocating → provisioning → ready, and stay there until something ends the run:

A failure in any phase, including the workload exiting after ready, tears down through terminating into failed.
Explicit termination, a binding limit, or an exhausted organization budget ends in terminated instead.

With a readiness probe, ready means the probe passed. Without one, it only means the workload launched; it makes no application-level health claim, so declare a probe whenever the image offers a health route.

Billing starts at allocation, not readiness.Spend accrues from allocation until the service reaches a terminal state, including all provisioning and model-loading time. A service never completes on its own — terminate it when you are done, and set limits as a backstop.

Step 1 - Store the endpoint key as a secret

Pass the workload API key through secret_env, not inline in the command, so it never appears in provider pod specs.

import { SDK } from '@meetkai/mka1';

const sdk = new SDK({
  bearerAuth: 'Bearer <mka1-api-key>',
});

const secret = await sdk.computeSecrets.createComputeSecret({
  computeSecretCreate: {
    name: 'inference-endpoint-key',
    data: { VLLM_API_KEY: '<generated-endpoint-key>' },
  },
});
console.log(secret.id);

curl https://apigw.mka1.com/api/v1/compute/secrets \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --data '{
    "name": "inference-endpoint-key",
    "data": {"VLLM_API_KEY": "<generated-endpoint-key>"}
  }'

Secret values are write-only. Keep the returned sec_... id for the create body, and the key value itself for your clients.

Step 2 - Create the service

Create the service with a required Idempotency-Key header, exactly as for jobs. This example serves Qwen2.5-0.5B-Instruct with vLLM’s OpenAI-compatible server, matching the platform smoke test:

const service = await sdk.computeServices.createService({
  idempotencyKey: '<unique-request-id>',
  computeResourceCreate: {
    name: 'qwen-inference',
    compute: {
      accelerator: 'nvidia-rtx-4090-24gb',
      gpuCount: 1,
      ephemeralDiskGb: 60,
    },
    container: {
      image: 'vllm/vllm-openai:v0.26.0-cu129',
      command: [
        'vllm', 'serve', 'Qwen/Qwen2.5-0.5B-Instruct',
        '--host', '0.0.0.0', '--port', '8000',
        '--max-model-len', '4096', '--gpu-memory-utilization', '0.85',
        '--enable-auto-tool-choice', '--tool-call-parser', 'hermes',
      ],
      secretEnv: {
        VLLM_API_KEY: { secretId: 'sec_Qw8pLm2vTn5xRc7J', key: 'VLLM_API_KEY' },
      },
      ports: [{ name: 'api', containerPort: 8000, protocol: 'http' }],
    },
    service: {
      readiness: { type: 'http', port: 'api', path: '/health', successStatus: 200 },
    },
  },
});
console.log(service.id, service.state);

curl https://apigw.mka1.com/api/v1/compute/services \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --header 'Idempotency-Key: <unique-request-id>' \
  --data '{
    "name": "qwen-inference",
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "gpu_count": 1,
      "ephemeral_disk_gb": 60
    },
    "container": {
      "image": "vllm/vllm-openai:v0.26.0-cu129",
      "command": ["vllm", "serve", "Qwen/Qwen2.5-0.5B-Instruct", "--host", "0.0.0.0", "--port", "8000", "--max-model-len", "4096", "--gpu-memory-utilization", "0.85", "--enable-auto-tool-choice", "--tool-call-parser", "hermes"],
      "secret_env": {
        "VLLM_API_KEY": {"secret_id": "sec_Qw8pLm2vTn5xRc7J", "key": "VLLM_API_KEY"}
      },
      "ports": [
        {"name": "api", "container_port": 8000, "protocol": "http"}
      ]
    },
    "service": {
      "readiness": {
        "type": "http",
        "port": "api",
        "path": "/health",
        "success_status": 200
      }
    }
  }'

Field notes:

Field	Notes
`container.ports`	Each port needs a unique name, a container port, and a protocol of `http` or `tcp`. Every named port is exposed on the service’s public endpoint.
`service.readiness`	References a declared port by name, never by number. An `http` probe requires that port’s protocol to be `http`; a `tcp` probe takes only the port name.
`secret_env.VLLM_API_KEY`	vLLM reads `VLLM_API_KEY` natively and then requires it as a bearer token on every request.
`--enable-auto-tool-choice --tool-call-parser hermes`	Required if the LLM gateway will call this endpoint: the gateway sends `tool_choice: "auto"` by default, and an unflagged vLLM rejects that with a `400`.

As with jobs, 201 means the request was accepted; capacity and provisioning problems surface afterwards through state, reason, logs, and events.

Step 3 - Wait for ready

const service = await sdk.computeServices.getService({ id: 'service_2mVx7cKq9dRw4bTn' });
console.log(service.state, service.ready, service.endpoints);

curl https://apigw.mka1.com/api/v1/compute/services/service_2mVx7cKq9dRw4bTn \
  --header 'Authorization: Bearer <mka1-api-key>'

Model download and load happen inside the workload, so expect minutes of provisioning before the readiness probe passes. A ready service carries its public endpoints:

{
  "id": "service_2mVx7cKq9dRw4bTn",
  "name": "qwen-inference",
  "state": "ready",
  "ready": true,
  "reason": null,
  "compute": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "ephemeral_disk_gb": 60},
  "hardware": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "gpu_memory_gb": 24},
  "price_usd_hr": 0.46,
  "accrued_usd": 0.12,
  "allocated_at": "2026-07-29T13:02:41Z",
  "limits": {},
  "endpoints": [
    {
      "name": "api",
      "protocol": "http",
      "host": "svc-2mvx7ckq.endpoints.example.net",
      "port": 443,
      "url": "https://svc-2mvx7ckq.endpoints.example.net"
    }
  ],
  "created_at": "2026-07-29T13:01:12Z",
  "started_at": "2026-07-29T13:06:30Z",
  "terminal_at": null
}

If the service lands in failed instead, read reason, then GET .../logs?source=user for the server’s own output and source=system for allocation and bootstrap problems — the same logs and events surface as jobs.

Step 4 - Call your endpoint

The endpoint is plain vLLM: an OpenAI-compatible API authenticated with the key you stored in Step 1.

curl

curl https://svc-2mvx7ckq.endpoints.example.net/v1/chat/completions \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <generated-endpoint-key>' \
  --data '{
    "model": "Qwen/Qwen2.5-0.5B-Instruct",
    "messages": [{"role": "user", "content": "Say hello."}]
  }'

A request without the key is rejected by vLLM itself — Compute does not sit in front of your endpoint.

Check the endpoint’s url scheme before sending anything sensitive. Depending on where capacity was allocated, an endpoint may be served over plain http, in which case bearer tokens and payloads cross the internet unencrypted.

Step 5 - Register with the LLM gateway

Optionally register the endpoint as a bring-your-own model in the MKA1 LLM gateway, so it is callable through the platform’s /responses API with normal platform keys. Add the endpoint to your model catalog:

curl

curl https://apigw.mka1.com/api/v1/llm/models/catalog \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --data '{
    "apiFormat": "completions",
    "apiProviderType": "openai",
    "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1",
    "auth": {"type": "api-key", "value": "<generated-endpoint-key>"}
  }'

Then register the returned model id:

curl

curl https://apigw.mka1.com/api/v1/llm/models/registry \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --data '{
    "source": "byo",
    "model_id": "<model-id-from-catalog>"
  }'

completions-format models are accepted at /responses; the gateway drives the chat-completions upstream for you. This is why the create body in Step 2 passed the tool-choice flags to vLLM.

Step 6 - Terminate

A service runs, and bills, until you stop it.

const service = await sdk.computeServices.terminateService({ id: 'service_2mVx7cKq9dRw4bTn' });
console.log(service.state);

curl https://apigw.mka1.com/api/v1/compute/services/service_2mVx7cKq9dRw4bTn/terminate \
  --request POST \
  --header 'Authorization: Bearer <mka1-api-key>'

Termination is idempotent, deallocates the provider resource, and stops spend; the record stays readable for audit and usage.

Serve a model from mka1-repos

To serve weights from mka1-repos — such as the merged output of a fine-tune job — change only the model identifier and the credentials. Compute always injects HF_ENDPOINT into every workload, pointing at the artifact endpoint configured for your cluster, and vLLM resolves model identifiers against it. mka1-repos authenticates with your MKA1 API key, injected as HF_TOKEN through secret_env:

{
  "container": {
    "image": "vllm/vllm-openai:v0.26.0-cu129",
    "command": ["vllm", "serve", "acme/qwen2.5-0.5b-support", "--host", "0.0.0.0", "--port", "8000", "--max-model-len", "4096", "--gpu-memory-utilization", "0.85", "--enable-auto-tool-choice", "--tool-call-parser", "hermes"],
    "secret_env": {
      "VLLM_API_KEY": {"secret_id": "sec_Qw8pLm2vTn5xRc7J", "key": "VLLM_API_KEY"},
      "HF_TOKEN": {"secret_id": "sec_Vb3nRk8sQw1xYz2M", "key": "HF_TOKEN"}
    },
    "ports": [
      {"name": "api", "container_port": 8000, "protocol": "http"}
    ]
  }
}

The service downloads the weights from your organization’s repository at startup and serves them under the same identifier, completing the loop: fine-tune as a job, publish to mka1-repos, serve as a service.

Track spend

Service spend accrues per whole minute from allocation and appears alongside jobs in the usage endpoint:

const usage = await sdk.computeUsage.getComputeUsage({ resourceType: 'service' });
console.log(usage.summary);

curl "https://apigw.mka1.com/api/v1/compute/usage?resource_type=service" \
  --header 'Authorization: Bearer <mka1-api-key>'

Compute spend draws from the same organization budgets as every other MKA1 service, and a service that hits an organization budget or its own limits is terminated with no grace period.

API reference

For the full request and response schemas, open the Compute groups in the API Reference.

Getting started

Responses

Features

Compute

CLI

Recipes

Benchmarks

Infrastructure

Deploy a model server

Before you start

Step 1 - Store the endpoint key as a secret

Step 2 - Create the service

Step 3 - Wait for ready

Step 4 - Call your endpoint

Step 5 - Register with the LLM gateway

Step 6 - Terminate

Serve a model from mka1-repos

Track spend

API reference

See also

​Before you start

​Step 1 - Store the endpoint key as a secret

​Step 2 - Create the service

​Step 3 - Wait for ready

​Step 4 - Call your endpoint

​Step 5 - Register with the LLM gateway

​Step 6 - Terminate

​Serve a model from mka1-repos

​Track spend

​API reference

​See also

Before you start

Step 1 - Store the endpoint key as a secret

Step 2 - Create the service

Step 3 - Wait for ready

Step 4 - Call your endpoint

Step 5 - Register with the LLM gateway

Step 6 - Terminate

Serve a model from mka1-repos

Track spend

API reference

See also