Deploy a model server

Use a Compute service when you need GPUs for a persistent workload such as model serving. A service is the same generic container run as a job, plus named ports, an optional readiness probe, and a public endpoint — Compute has no deployment or model schema, so this guide serves with vLLM purely as workload payload. GPU services run on external GPU providers, and CPU services run in the MKA1 cluster. This guide’s example requests a GPU, so it runs on an external GPU provider. The two placements protect endpoints differently; see Endpoint access. This example creates a single persistent model server. Keep its process in the foreground and terminate the service when finished. Some placements also support the public service.scaling and volume fields; support depends on the selected backend. Do not assume every external GPU provider supports shared storage or dynamic replicas. The CLI examples assume an authenticated MKA1 CLI.

Before you start

You need:

Requirement	Notes
API key	Send it as `Authorization: Bearer <mka1-api-key>` on every request.
Compute enabled for your organization	Compute is a fail-closed allowlist. If requests return `403 tenant_disabled`, ask a cluster administrator to enable your organization.
A serving image	Any Linux GPU image the provider can pull without interactive authentication, keeping the server process in the foreground.
Weights to serve	A public model identifier, or a repository in mka1-repos — for example one published by a fine-tune job.
An API key for the workload itself	Only for a service on an external GPU provider, whose serving runtime must authenticate requests itself. Generate a strong random string for this. MKA1 cluster endpoints need no workload key; see Endpoint access.

Services move through requested → allocating → provisioning → ready, and stay there until something ends the run:

A failure in any phase, including the workload exiting after ready, tears down through terminating into failed.
Explicit termination, a binding limit, or an exhausted organization budget ends in terminated instead.

With a readiness probe, ready means the probe passed. Without one, it only means the workload launched; it makes no application-level health claim, so declare a probe whenever the image offers a health route.

Billing starts at allocation, not readiness.Spend accrues from allocation until the service reaches a terminal state, including all provisioning and model-loading time. A service never completes on its own — terminate it when you are done, and set limits as a backstop.

Endpoint access

A service’s placement decides who authenticates requests to its endpoints. MKA1 cluster endpoints are always authenticated by the MKA1 API gateway. Send an MKA1 API key of the organization that owns the service, holding the admin-only read:compute scope. A request without a valid key gets 401. A key that lacks read:compute, or that belongs to another organization, gets 403. The API gateway strips the Authorization header and cookies before forwarding a request, so your container never receives a caller’s credential. Do not configure the serving runtime to require a key of its own. External GPU provider endpoints are reachable at the provider’s URL, with no MKA1 authentication in front of them. Protect them with the serving runtime’s own key, as this guide does with VLLM_API_KEY.

Step 1 - Store the endpoint key as a secret

Pass the workload API key through secret_env, not inline in the command, so it never appears in provider pod specs. Skip this step for a CPU service: the MKA1 API gateway authenticates its endpoint.

import { SDK } from '@meetkai/mka1';

const sdk = new SDK({ bearerAuth: 'Bearer <mka1-api-key>' });

const result = await sdk.computeSecrets.createComputeSecret({
  "computeSecretCreate": {
    "name": "inference-endpoint-key",
    "data": {
      "VLLM_API_KEY": "<generated-endpoint-key>"
    }
  }
});
if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
console.log(result.id);

from meetkai_mka1 import models
import os
from openai import OpenAI
from meetkai_mka1 import SDK

sdk = SDK(bearer_auth="Bearer <mka1-api-key>")

result = sdk.compute_secrets.create_compute_secret(
    name="inference-endpoint-key",
    data={"VLLM_API_KEY": "<generated-endpoint-key>"},
)
if isinstance(result, models.ComputeErrorEnvelope):
    raise RuntimeError(result.error)
print(result.id)

using MeetKai.MKA1.Types.Components;
using MeetKai.MKA1.Types.Requests;
using Newtonsoft.Json;
using OpenAI;
using OpenAI.Chat;
using System.ClientModel;
using MeetKai.MKA1;

var sdk = new SDK(bearerAuth: "Bearer <mka1-api-key>");

// Deserialize the wire-format payload into the SDK request model.
var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.ComputeSecretCreate> ("""
{
  "name": "inference-endpoint-key",
  "data": {
    "VLLM_API_KEY": "<generated-endpoint-key>"
  }
}
""")!;

var response = await sdk.ComputeSecrets.CreateComputeSecretAsync(body);
var result = response.ComputeSecret
    ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
Console.WriteLine(JsonConvert.SerializeObject(result.Id));

mka1 compute-secrets create \
  --output-format json \
  --body @- <<'JSON'
{
  "data": {
    "VLLM_API_KEY": "<generated-endpoint-key>"
  },
  "name": "inference-endpoint-key"
}
JSON

curl https://apigw.mka1.com/api/v1/compute/secrets \
  --request POST \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer $MKA1_API_KEY" \
  --data '{
    "name": "inference-endpoint-key",
    "data": {"VLLM_API_KEY": "<generated-endpoint-key>"}
  }'

Secret values are write-only. Keep the returned sec_... id for the create body, and the key value itself for your clients.

Step 2 - Create the service

Create the service with a required Idempotency-Key header, exactly as for jobs. This example serves Qwen2.5-0.5B-Instruct with vLLM’s OpenAI-compatible server, matching the platform smoke test:

// Keep this value unchanged when retrying this request.
const idempotencyKey = process.env.MKA1_REQUEST_ID;
if (!idempotencyKey) throw new Error('Set MKA1_REQUEST_ID');

const result = await sdk.computeServices.createService({
  "idempotencyKey": idempotencyKey,
  "computeResourceCreate": {
    "name": "qwen-inference",
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "gpuCount": 1,
      "ephemeralDiskGb": 60
    },
    "container": {
      "image": "vllm/vllm-openai:v0.26.0-cu129",
      "command": [
        "vllm",
        "serve",
        "Qwen/Qwen2.5-0.5B-Instruct",
        "--host",
        "0.0.0.0",
        "--port",
        "8000",
        "--max-model-len",
        "4096",
        "--gpu-memory-utilization",
        "0.85",
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "hermes"
      ],
      "secretEnv": {
        "VLLM_API_KEY": {
          "secretId": "sec_Qw8pLm2vTn5xRc7J",
          "key": "VLLM_API_KEY"
        }
      },
      "ports": [
        {
          "name": "api",
          "containerPort": 8000,
          "protocol": "http"
        }
      ]
    },
    "service": {
      "readiness": {
        "type": "http",
        "port": "api",
        "path": "/health",
        "successStatus": 200
      }
    },
    "limits": {
      "maxRuntimeHours": 2,
      "maxCostUsd": 5
    }
  }
});
if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
console.log(result.id);

idempotency_key = os.environ["MKA1_REQUEST_ID"]  # Reuse for retries.

result = sdk.compute_services.create_service(
    idempotency_key=idempotency_key,
    name="qwen-inference",
    compute={
        "accelerator": "nvidia-rtx-4090-24gb",
        "gpu_count": 1,
        "ephemeral_disk_gb": 60,
    },
    container={
        "image": "vllm/vllm-openai:v0.26.0-cu129",
        "command": [
            "vllm",
            "serve",
            "Qwen/Qwen2.5-0.5B-Instruct",
            "--host",
            "0.0.0.0",
            "--port",
            "8000",
            "--max-model-len",
            "4096",
            "--gpu-memory-utilization",
            "0.85",
            "--enable-auto-tool-choice",
            "--tool-call-parser",
            "hermes",
        ],
        "secret_env": {
            "VLLM_API_KEY": {"secret_id": "sec_Qw8pLm2vTn5xRc7J", "key": "VLLM_API_KEY"}
        },
        "ports": [{"name": "api", "container_port": 8000, "protocol": "http"}],
    },
    service={
        "readiness": {
            "type": "http",
            "port": "api",
            "path": "/health",
            "success_status": 200,
        }
    },
    limits={"max_runtime_hours": 2, "max_cost_usd": 5},
)
if isinstance(result, models.ComputeErrorEnvelope):
    raise RuntimeError(result.error)
print(result.id)

var idempotencyKey = System.Environment.GetEnvironmentVariable("MKA1_REQUEST_ID")
    ?? throw new InvalidOperationException("Set MKA1_REQUEST_ID");

// Deserialize the wire-format payload into the SDK request model.
var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.ComputeResourceCreate> ("""
{
  "name": "qwen-inference",
  "compute": {
    "accelerator": "nvidia-rtx-4090-24gb",
    "gpu_count": 1,
    "ephemeral_disk_gb": 60
  },
  "container": {
    "image": "vllm/vllm-openai:v0.26.0-cu129",
    "command": [
      "vllm",
      "serve",
      "Qwen/Qwen2.5-0.5B-Instruct",
      "--host",
      "0.0.0.0",
      "--port",
      "8000",
      "--max-model-len",
      "4096",
      "--gpu-memory-utilization",
      "0.85",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "hermes"
    ],
    "secret_env": {
      "VLLM_API_KEY": {
        "secret_id": "sec_Qw8pLm2vTn5xRc7J",
        "key": "VLLM_API_KEY"
      }
    },
    "ports": [
      {
        "name": "api",
        "container_port": 8000,
        "protocol": "http"
      }
    ]
  },
  "service": {
    "readiness": {
      "type": "http",
      "port": "api",
      "path": "/health",
      "success_status": 200
    }
  },
  "limits": {
    "max_runtime_hours": 2,
    "max_cost_usd": 5
  }
}
""")!;

var response = await sdk.ComputeServices.CreateServiceAsync(idempotencyKey, body);
var result = response.ComputeService
    ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
Console.WriteLine(JsonConvert.SerializeObject(result.Id));

mka1 compute-services create-service \
  --idempotency-key "$MKA1_REQUEST_ID" \
  --output-format json \
  --body @- <<'JSON'
{
  "compute": {
    "accelerator": "nvidia-rtx-4090-24gb",
    "ephemeral_disk_gb": 60,
    "gpu_count": 1
  },
  "container": {
    "command": [
      "vllm",
      "serve",
      "Qwen/Qwen2.5-0.5B-Instruct",
      "--host",
      "0.0.0.0",
      "--port",
      "8000",
      "--max-model-len",
      "4096",
      "--gpu-memory-utilization",
      "0.85",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "hermes"
    ],
    "image": "vllm/vllm-openai:v0.26.0-cu129",
    "ports": [
      {
        "container_port": 8000,
        "name": "api",
        "protocol": "http"
      }
    ],
    "secret_env": {
      "VLLM_API_KEY": {
        "key": "VLLM_API_KEY",
        "secret_id": "sec_Qw8pLm2vTn5xRc7J"
      }
    }
  },
  "limits": {
    "max_cost_usd": 5,
    "max_runtime_hours": 2
  },
  "name": "qwen-inference",
  "service": {
    "readiness": {
      "path": "/health",
      "port": "api",
      "success_status": 200,
      "type": "http"
    }
  }
}
JSON

curl https://apigw.mka1.com/api/v1/compute/services \
  --request POST \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer $MKA1_API_KEY" \
  --header "Idempotency-Key: $MKA1_REQUEST_ID" \
  --data '{
  "name": "qwen-inference",
  "compute": {
    "accelerator": "nvidia-rtx-4090-24gb",
    "gpu_count": 1,
    "ephemeral_disk_gb": 60
  },
  "container": {
    "image": "vllm/vllm-openai:v0.26.0-cu129",
    "command": [
      "vllm",
      "serve",
      "Qwen/Qwen2.5-0.5B-Instruct",
      "--host",
      "0.0.0.0",
      "--port",
      "8000",
      "--max-model-len",
      "4096",
      "--gpu-memory-utilization",
      "0.85",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "hermes"
    ],
    "secret_env": {
      "VLLM_API_KEY": {
        "secret_id": "sec_Qw8pLm2vTn5xRc7J",
        "key": "VLLM_API_KEY"
      }
    },
    "ports": [
      {
        "name": "api",
        "container_port": 8000,
        "protocol": "http"
      }
    ]
  },
  "service": {
    "readiness": {
      "type": "http",
      "port": "api",
      "path": "/health",
      "success_status": 200
    }
  },
  "limits": {
    "max_runtime_hours": 2,
    "max_cost_usd": 5
  }
}'

Field notes:

Field	Notes
`container.ports`	Each port needs a unique name, a container port, and a protocol of `http` or `tcp`. Every named port is exposed on the service’s public endpoint.
`service.readiness`	References a declared port by name, never by number. An `http` probe requires that port’s protocol to be `http`; a `tcp` probe takes only the port name.
`secret_env.VLLM_API_KEY`	vLLM reads `VLLM_API_KEY` and requires it on inference requests; the health route is separate. Set it only on an external GPU provider.
`--enable-auto-tool-choice --tool-call-parser hermes`	Required if the LLM gateway will call this endpoint: the gateway sends `tool_choice: "auto"` by default, and an unflagged vLLM rejects that with a `400`.
`placement`	Optional, same as jobs, and chooses among external GPU providers: `strategy` is `capacity_optimized` (default) or `lowest_cost`, and `providers` is an unordered filter — see the fine-tune guide’s field notes.

As with jobs, 201 means the request was accepted; capacity and provisioning problems surface afterwards through state, reason, logs, and events.

Step 3 - Wait for ready

const result = await sdk.computeServices.getService({
  "id": "service_2mVx7cKq9dRw4bTn"
});
if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
console.log(result.endpoints);

result = sdk.compute_services.get_service(
    id="service_2mVx7cKq9dRw4bTn",
)
if isinstance(result, models.ComputeErrorEnvelope):
    raise RuntimeError(result.error)
print(result.endpoints)

var response = await sdk.ComputeServices.GetServiceAsync("service_2mVx7cKq9dRw4bTn");
var result = response.ComputeService
    ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
Console.WriteLine(JsonConvert.SerializeObject(result.Endpoints));

mka1 compute-services get-service \
  --id service_2mVx7cKq9dRw4bTn \
  --output-format json

curl https://apigw.mka1.com/api/v1/compute/services/service_2mVx7cKq9dRw4bTn \
  --header "Authorization: Bearer $MKA1_API_KEY"

Model download and load happen inside the workload, so expect minutes of provisioning before the readiness probe passes. A ready service carries its public endpoints:

{
  "id": "service_2mVx7cKq9dRw4bTn",
  "name": "qwen-inference",
  "state": "ready",
  "ready": true,
  "reason": null,
  "compute": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "ephemeral_disk_gb": 60},
  "hardware": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "gpu_memory_gb": 24},
  "price_usd_hr": 0.46,
  "accrued_usd": 0.12,
  "allocated_at": "2026-07-29T13:02:41Z",
  "limits": {},
  "endpoints": [
    {
      "name": "api",
      "protocol": "http",
      "host": "svc-2mvx7ckq.endpoints.example.net",
      "port": 443,
      "url": "https://svc-2mvx7ckq.endpoints.example.net"
    }
  ],
  "created_at": "2026-07-29T13:01:12Z",
  "started_at": "2026-07-29T13:06:30Z",
  "terminal_at": null
}

If the service lands in failed instead, read reason, then GET .../logs?source=user for the server’s own output and source=system for allocation and bootstrap problems — the same logs and events surface as jobs.

Step 4 - Call your endpoint

The endpoint is plain vLLM: an OpenAI-compatible API. The calls below use the official OpenAI-compatible client libraries directly against your vLLM endpoint, rather than the MKA1 control-plane SDK. Install openai@5.0.0 for TypeScript, OpenAI 2.4.0 for C#, or openai==1.109.1 for Python. Replace https://<model-server-host> with the endpoint’s url, and <model-server-api-key> with the credential its placement accepts: the endpoint key from Step 1 on an external GPU provider, or an MKA1 API key holding read:compute in the MKA1 cluster. Use an HTTPS endpoint for these credentialed requests.

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: '<model-server-api-key>',
  baseURL: 'https://<model-server-host>/v1',
});

const result = await openai.chat.completions.create({
  model: 'Qwen/Qwen2.5-0.5B-Instruct',
  messages: [{ role: 'user', content: 'Say hello.' }],
});
console.log(result.choices[0]?.message.content);

client = OpenAI(
    api_key="<model-server-api-key>",
    base_url="https://<model-server-host>/v1",
)
result = client.chat.completions.create(
    model="Qwen/Qwen2.5-0.5B-Instruct",
    messages=[{"role": "user", "content": "Say hello."}],
)
print(result.choices[0].message.content)

var key = "<model-server-api-key>";
var endpoint = "https://<model-server-host>/v1";
var client = new ChatClient(
    model: "Qwen/Qwen2.5-0.5B-Instruct",
    credential: new ApiKeyCredential(key),
    options: new OpenAIClientOptions { Endpoint = new Uri(endpoint) });
ChatCompletion result = await client.CompleteChatAsync("Say hello.");
Console.WriteLine(result.Content[0].Text);

curl "https://<model-server-host>/v1/chat/completions" \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <model-server-api-key>' \
  --data '{"model":"Qwen/Qwen2.5-0.5B-Instruct","messages":[{"role":"user","content":"Say hello."}]}'

For the external GPU endpoint illustrated here, vLLM authenticates inference requests using the endpoint key. Readiness uses its health route, which does not require that key.

Check the endpoint’s url scheme before sending anything sensitive. MKA1 cluster endpoints always use https, but an external GPU provider may serve an endpoint over plain http, in which case bearer tokens and payloads cross the internet unencrypted.

Step 5 - Register with the LLM gateway

Optionally register the endpoint as a bring-your-own model in the MKA1 LLM gateway, so it is callable through the platform’s /responses API with normal platform keys. Add the endpoint to your model catalog using an organization admin or owner key. The catalog id names your MKA1 model; modelId must match the upstream model name served by vLLM. auth.value is the credential the LLM gateway sends to your endpoint, so it depends on the placement:

External GPU provider: the serving runtime’s key. The example below registers this guide’s endpoint with the endpoint key from Step 1.
MKA1 cluster: an MKA1 API key of the organization that owns the service, holding read:compute. Prefer a dedicated key with only that scope. The MKA1 API gateway rejects a runtime key with 401.

const result = await sdk.llm.models.addCatalogEntry({
  "id": "qwen-compute-demo",
  "provider": "openai-compatible",
  "modelId": "Qwen/Qwen2.5-0.5B-Instruct",
  "displayName": "Qwen Compute demo",
  "created": 1789488000,
  "capabilities": {
    "modalities": {
      "input": [
        "text"
      ],
      "output": [
        "text"
      ]
    },
    "reasoning": false
  },
  "apiFormat": "completions",
  "apiProviderType": "openai",
  "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1",
  "auth": {
    "type": "api-key",
    "value": "<generated-endpoint-key>"
  }
});
console.log(result.modelId);

result = sdk.llm.models.add_catalog_entry(
    id="qwen-compute-demo",
    provider="openai-compatible",
    model_id="Qwen/Qwen2.5-0.5B-Instruct",
    display_name="Qwen Compute demo",
    created=1789488000,
    capabilities={
        "modalities": {"input": ["text"], "output": ["text"]},
        "reasoning": False,
    },
    api_format="completions",
    api_provider_type="openai",
    base_url="https://svc-2mvx7ckq.endpoints.example.net/v1",
    auth={"type": "api-key", "value": "<generated-endpoint-key>"},
)
print(result.model_id)

// Deserialize the wire-format payload into the SDK request model.
var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.AddCatalogModelRequest> ("""
{
  "id": "qwen-compute-demo",
  "provider": "openai-compatible",
  "modelId": "Qwen/Qwen2.5-0.5B-Instruct",
  "displayName": "Qwen Compute demo",
  "created": 1789488000,
  "capabilities": {
    "modalities": {
      "input": [
        "text"
      ],
      "output": [
        "text"
      ]
    },
    "reasoning": false
  },
  "apiFormat": "completions",
  "apiProviderType": "openai",
  "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1",
  "auth": {
    "type": "api-key",
    "value": "<generated-endpoint-key>"
  }
}
""")!;

var response = await sdk.Llm.Models.AddCatalogEntryAsync(body);
var result = response.CatalogEntry
    ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
Console.WriteLine(JsonConvert.SerializeObject(result.ModelId));

mka1 llm models add-catalog-entry \
  --output-format json \
  --body @- <<'JSON'
{
  "id": "qwen-compute-demo",
  "provider": "openai-compatible",
  "modelId": "Qwen/Qwen2.5-0.5B-Instruct",
  "displayName": "Qwen Compute demo",
  "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1",
  "apiFormat": "completions",
  "apiProviderType": "openai",
  "created": 1789488000,
  "auth": {
    "type": "api-key",
    "value": "<generated-endpoint-key>"
  },
  "capabilities": {
    "modalities": {
      "input": [
        "text"
      ],
      "output": [
        "text"
      ]
    },
    "reasoning": false
  }
}
JSON

curl https://apigw.mka1.com/api/v1/llm/models/catalog \
  --request POST \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer $MKA1_API_KEY" \
  --data '{
    "id": "qwen-compute-demo",
    "provider": "openai-compatible",
    "modelId": "Qwen/Qwen2.5-0.5B-Instruct",
    "displayName": "Qwen Compute demo",
    "created": 1789488000,
    "capabilities": {"modalities": {"input": ["text"], "output": ["text"]}, "reasoning": false},
    "apiFormat": "completions",
    "apiProviderType": "openai",
    "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1",
    "auth": {"type": "api-key", "value": "<generated-endpoint-key>"}
  }'

Then activate the catalog model using its returned model_id (the id you supplied above):

const result = await sdk.llm.models.activateRegistryEntry({
  "source": "byo",
  "modelId": "qwen-compute-demo"
});
console.log(result.modelId);

result = sdk.llm.models.activate_registry_entry(
    source="byo",
    model_id="qwen-compute-demo",
)
print(result.model_id)

// Deserialize the wire-format payload into the SDK request model.
var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.ActivateRegistryEntryRequest> ("""
{
  "source": "byo",
  "model_id": "qwen-compute-demo"
}
""")!;

var response = await sdk.Llm.Models.ActivateRegistryEntryAsync(body);
var result = response.RegistryEntryResponse
    ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
Console.WriteLine(JsonConvert.SerializeObject(result.ModelId));

mka1 llm models activate-registry-entry \
  --output-format json \
  --body @- <<'JSON'
{
  "source": "byo",
  "model_id": "qwen-compute-demo"
}
JSON

curl https://apigw.mka1.com/api/v1/llm/models/registry \
  --request POST \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer $MKA1_API_KEY" \
  --data '{
    "source": "byo",
    "model_id": "qwen-compute-demo"
  }'

completions-format models are accepted at /responses; the gateway drives the chat-completions upstream for you. This is why the create body in Step 2 passed the tool-choice flags to vLLM.

Step 6 - Terminate

A service runs, and bills, until you stop it.

const result = await sdk.computeServices.terminateService({
  "id": "service_2mVx7cKq9dRw4bTn"
});
if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
console.log(result.state);

result = sdk.compute_services.terminate_service(
    id="service_2mVx7cKq9dRw4bTn",
)
if isinstance(result, models.ComputeErrorEnvelope):
    raise RuntimeError(result.error)
print(result.state)

var response = await sdk.ComputeServices.TerminateServiceAsync("service_2mVx7cKq9dRw4bTn");
var result = response.ComputeService
    ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
Console.WriteLine(JsonConvert.SerializeObject(result.State));

mka1 compute-services terminate-service \
  --id service_2mVx7cKq9dRw4bTn \
  --output-format json

curl https://apigw.mka1.com/api/v1/compute/services/service_2mVx7cKq9dRw4bTn/terminate \
  --request POST \
  --header "Authorization: Bearer $MKA1_API_KEY"

Termination is idempotent, deallocates the provider resource, and stops spend; the record stays readable for audit and usage. Terminating does not remove a catalog entry you added in Step 5. The model stays active, and requests to it return 404; remove it in Step 7.

Step 7 - Remove the model from the LLM gateway

If you registered the endpoint in Step 5, deactivate the model and delete its catalog entry, using the same organization admin or owner key:

const deactivation = await sdk.llm.models.deactivateRegistryEntry({
  "id": "qwen-compute-demo"
});
console.log(deactivation.deactivated);

const deletion = await sdk.llm.models.deleteCatalogEntry({
  "id": "qwen-compute-demo"
});
console.log(deletion.deleted);

deactivation = sdk.llm.models.deactivate_registry_entry(
    id="qwen-compute-demo",
)
print(deactivation.deactivated)

deletion = sdk.llm.models.delete_catalog_entry(
    id="qwen-compute-demo",
)
print(deletion.deleted)

var deactivateResponse = await sdk.Llm.Models.DeactivateRegistryEntryAsync(id: "qwen-compute-demo");
var deactivation = deactivateResponse.DeactivateRegistryEntryResponse
    ?? throw new InvalidOperationException(JsonConvert.SerializeObject(deactivateResponse));
Console.WriteLine(JsonConvert.SerializeObject(deactivation.Deactivated));

var deleteResponse = await sdk.Llm.Models.DeleteCatalogEntryAsync(id: "qwen-compute-demo");
var deletion = deleteResponse.DeleteCatalogModelResponse
    ?? throw new InvalidOperationException(JsonConvert.SerializeObject(deleteResponse));
Console.WriteLine(JsonConvert.SerializeObject(deletion.Deleted));

mka1 llm models deactivate-registry-entry \
  --id qwen-compute-demo \
  --output-format json

mka1 llm models delete-catalog-entry \
  --id qwen-compute-demo \
  --output-format json

curl 'https://apigw.mka1.com/api/v1/llm/models/registry?id=qwen-compute-demo' \
  --request DELETE \
  --header "Authorization: Bearer $MKA1_API_KEY"

curl 'https://apigw.mka1.com/api/v1/llm/models/catalog?id=qwen-compute-demo' \
  --request DELETE \
  --header "Authorization: Bearer $MKA1_API_KEY"

Deactivating releases the model name, so requests to qwen-compute-demo stop resolving; deleting withdraws its catalog definition.

Serve a model from mka1-repos

To serve weights from mka1-repos — such as the merged output of a fine-tune job, or weights you pushed over git — change only the model identifier and the credentials. Set HF_ENDPOINT on the service, pointing at the artifact endpoint for your cluster, and vLLM resolves model identifiers against it — the same Hugging Face access described in Manage repositories, so serving a repository is just a model identifier and two environment variables. The console pre-fills HF_ENDPOINT when you create a workload; set it yourself when you call the API directly. mka1-repos authenticates with your MKA1 API key, injected as HF_TOKEN through secret_env:

{
  "container": {
    "image": "vllm/vllm-openai:v0.26.0-cu129",
    "command": ["vllm", "serve", "meetkai/functionary", "--host", "0.0.0.0", "--port", "8000", "--max-model-len", "4096", "--gpu-memory-utilization", "0.85", "--enable-auto-tool-choice", "--tool-call-parser", "hermes"],
    "env": {
      "HF_ENDPOINT": "https://hf.mka1.com"
    },
    "secret_env": {
      "VLLM_API_KEY": {"secret_id": "sec_Qw8pLm2vTn5xRc7J", "key": "VLLM_API_KEY"},
      "HF_TOKEN": {"secret_id": "sec_Vb3nRk8sQw1xYz2M", "key": "HF_TOKEN"}
    },
    "ports": [
      {"name": "api", "container_port": 8000, "protocol": "http"}
    ]
  }
}

The service downloads the weights from your organization’s repository at startup and serves them under the same identifier, completing the loop: fine-tune as a job, publish to mka1-repos, serve as a service.

Track spend

Service spend accrues per whole minute from allocation and appears alongside jobs in the usage endpoint:

const result = await sdk.computeUsage.getComputeUsage({
  "resourceType": "service"
});
if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
console.log(result.summary);

result = sdk.compute_usage.get_compute_usage(
    resource_type="service",
)
if isinstance(result, models.ComputeErrorEnvelope):
    raise RuntimeError(result.error)
print(result.summary)

var request = new GetComputeUsageRequest
{
    ResourceType = ComputeUsageResourceType.Service,
    Limit = 50,
};

var response = await sdk.ComputeUsage.GetComputeUsageAsync(request);
var result = response.ComputeUsage
    ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
Console.WriteLine(JsonConvert.SerializeObject(result.Summary));

mka1 compute-usage get \
  --limit 50 \
  --resource-type service \
  --output-format json

curl "https://apigw.mka1.com/api/v1/compute/usage?resource_type=service" \
  --header "Authorization: Bearer $MKA1_API_KEY"

Compute spend draws from the same organization budgets as every other MKA1 service, and a service that hits an organization budget or its own limits is terminated with no grace period.

API reference

For the full request and response schemas, open the Compute groups in the API Reference.

Get started

SDKs and CLI

Responses

Standalone APIs

Agents

Evals

Compute

Authentication

Production

Deploy a model server

Before you start

Endpoint access

Step 1 - Store the endpoint key as a secret

Step 2 - Create the service

Step 3 - Wait for ready

Step 4 - Call your endpoint

Step 5 - Register with the LLM gateway

Step 6 - Terminate

Step 7 - Remove the model from the LLM gateway

Serve a model from mka1-repos

Track spend

API reference

See also

​Before you start

​Endpoint access

​Step 1 - Store the endpoint key as a secret

​Step 2 - Create the service

​Step 3 - Wait for ready

​Step 4 - Call your endpoint

​Step 5 - Register with the LLM gateway

​Step 6 - Terminate

​Step 7 - Remove the model from the LLM gateway

​Serve a model from mka1-repos

​Track spend

​API reference

​See also

Before you start

Endpoint access

Step 1 - Store the endpoint key as a secret

Step 2 - Create the service

Step 3 - Wait for ready

Step 4 - Call your endpoint

Step 5 - Register with the LLM gateway

Step 6 - Terminate

Step 7 - Remove the model from the LLM gateway

Serve a model from mka1-repos

Track spend

API reference

See also