> ## Documentation Index > Fetch the complete documentation index at: https://docs.mka1.com/llms.txt > Use this file to discover all available pages before exploring further. # Deploy a model server > Serve a model behind an authenticated HTTP endpoint with a Compute service, pull weights from mka1-repos, and register the endpoint with the LLM gateway. Use a Compute service when you need GPUs for a persistent workload such as model serving. A service is the same generic container run as a [job](/docs/compute-fine-tune-job), plus named ports, an optional readiness probe, and a public endpoint — Compute has no deployment or model schema, so this guide serves with [vLLM](https://docs.vllm.ai) purely as workload payload. A service runs until you terminate it, and you own the lifecycle end to end: Compute never restarts, rescales, or swaps revisions behind your back, so what you deployed is exactly what is running. That predictability comes with two rules: keep the server process in the foreground for the life of the service — a workload that exits, even with code `0`, ends the service as `failed` — and replace a failed or outdated service by creating a new one. ## Before you start You need: | Requirement | Notes | | ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | API key | Send it as `Authorization: Bearer ` on every request. | | Compute enabled for your organization | Compute is a fail-closed allowlist. If requests return `403 tenant_disabled`, ask a cluster administrator to enable your organization. | | A serving image | Any Linux GPU image the provider can pull without interactive authentication, keeping the server process in the foreground. | | Weights to serve | A public model identifier, or a repository in mka1-repos — for example one published by a [fine-tune job](/docs/compute-fine-tune-job). | | An API key for the workload itself | Compute exposes your port to the internet and does not authenticate calls to it; your server must. Generate a strong random string for this. | Services move through `requested` → `allocating` → `provisioning` → `ready`, and stay there until something ends the run: * A failure in any phase, including the workload exiting after `ready`, tears down through `terminating` into `failed`. * Explicit termination, a binding limit, or an exhausted organization budget ends in `terminated` instead. With a readiness probe, `ready` means the probe passed. Without one, it only means the workload launched; it makes no application-level health claim, so declare a probe whenever the image offers a health route. **Billing starts at allocation, not readiness.** Spend accrues from allocation until the service reaches a terminal state, including all provisioning and model-loading time. A service never completes on its own — terminate it when you are done, and set `limits` as a backstop. ## Step 1 - Store the endpoint key as a secret Pass the workload API key through `secret_env`, not inline in the command, so it never appears in provider pod specs. ```ts MKA1 SDK theme={null} import { SDK } from '@meetkai/mka1'; const sdk = new SDK({ bearerAuth: 'Bearer ', }); const secret = await sdk.computeSecrets.createComputeSecret({ computeSecretCreate: { name: 'inference-endpoint-key', data: { VLLM_API_KEY: '' }, }, }); console.log(secret.id); ``` ```bash curl theme={null} curl https://apigw.mka1.com/api/v1/compute/secrets \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --data '{ "name": "inference-endpoint-key", "data": {"VLLM_API_KEY": ""} }' ``` Secret values are write-only. Keep the returned `sec_...` id for the create body, and the key value itself for your clients. ## Step 2 - Create the service Create the service with a required `Idempotency-Key` header, exactly as for jobs. This example serves Qwen2.5-0.5B-Instruct with vLLM's OpenAI-compatible server, matching the platform smoke test: ```ts MKA1 SDK theme={null} const service = await sdk.computeServices.createService({ idempotencyKey: '', computeResourceCreate: { name: 'qwen-inference', compute: { accelerator: 'nvidia-rtx-4090-24gb', gpuCount: 1, ephemeralDiskGb: 60, }, container: { image: 'vllm/vllm-openai:v0.26.0-cu129', command: [ 'vllm', 'serve', 'Qwen/Qwen2.5-0.5B-Instruct', '--host', '0.0.0.0', '--port', '8000', '--max-model-len', '4096', '--gpu-memory-utilization', '0.85', '--enable-auto-tool-choice', '--tool-call-parser', 'hermes', ], secretEnv: { VLLM_API_KEY: { secretId: 'sec_Qw8pLm2vTn5xRc7J', key: 'VLLM_API_KEY' }, }, ports: [{ name: 'api', containerPort: 8000, protocol: 'http' }], }, service: { readiness: { type: 'http', port: 'api', path: '/health', successStatus: 200 }, }, }, }); console.log(service.id, service.state); ``` ```bash curl theme={null} curl https://apigw.mka1.com/api/v1/compute/services \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --header 'Idempotency-Key: ' \ --data '{ "name": "qwen-inference", "compute": { "accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "ephemeral_disk_gb": 60 }, "container": { "image": "vllm/vllm-openai:v0.26.0-cu129", "command": ["vllm", "serve", "Qwen/Qwen2.5-0.5B-Instruct", "--host", "0.0.0.0", "--port", "8000", "--max-model-len", "4096", "--gpu-memory-utilization", "0.85", "--enable-auto-tool-choice", "--tool-call-parser", "hermes"], "secret_env": { "VLLM_API_KEY": {"secret_id": "sec_Qw8pLm2vTn5xRc7J", "key": "VLLM_API_KEY"} }, "ports": [ {"name": "api", "container_port": 8000, "protocol": "http"} ] }, "service": { "readiness": { "type": "http", "port": "api", "path": "/health", "success_status": 200 } } }' ``` Field notes: | Field | Notes | | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `container.ports` | Each port needs a unique name, a container port, and a protocol of `http` or `tcp`. Every named port is exposed on the service's public endpoint. | | `service.readiness` | References a declared port by name, never by number. An `http` probe requires that port's protocol to be `http`; a `tcp` probe takes only the port name. | | `secret_env.VLLM_API_KEY` | vLLM reads `VLLM_API_KEY` natively and then requires it as a bearer token on every request. | | `--enable-auto-tool-choice --tool-call-parser hermes` | Required if the [LLM gateway](#step-5---register-with-the-llm-gateway) will call this endpoint: the gateway sends `tool_choice: "auto"` by default, and an unflagged vLLM rejects that with a `400`. | As with jobs, `201` means the request was accepted; capacity and provisioning problems surface afterwards through `state`, `reason`, logs, and events. ## Step 3 - Wait for ready ```ts MKA1 SDK theme={null} const service = await sdk.computeServices.getService({ id: 'service_2mVx7cKq9dRw4bTn' }); console.log(service.state, service.ready, service.endpoints); ``` ```bash curl theme={null} curl https://apigw.mka1.com/api/v1/compute/services/service_2mVx7cKq9dRw4bTn \ --header 'Authorization: Bearer ' ``` Model download and load happen inside the workload, so expect minutes of `provisioning` before the readiness probe passes. A ready service carries its public endpoints: ```json theme={null} { "id": "service_2mVx7cKq9dRw4bTn", "name": "qwen-inference", "state": "ready", "ready": true, "reason": null, "compute": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "ephemeral_disk_gb": 60}, "hardware": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "gpu_memory_gb": 24}, "price_usd_hr": 0.46, "accrued_usd": 0.12, "allocated_at": "2026-07-29T13:02:41Z", "limits": {}, "endpoints": [ { "name": "api", "protocol": "http", "host": "svc-2mvx7ckq.endpoints.example.net", "port": 443, "url": "https://svc-2mvx7ckq.endpoints.example.net" } ], "created_at": "2026-07-29T13:01:12Z", "started_at": "2026-07-29T13:06:30Z", "terminal_at": null } ``` If the service lands in `failed` instead, read `reason`, then `GET .../logs?source=user` for the server's own output and `source=system` for allocation and bootstrap problems — the same logs and events surface as [jobs](/docs/compute-fine-tune-job#step-5---read-logs-and-events). ## Step 4 - Call your endpoint The endpoint is plain vLLM: an OpenAI-compatible API authenticated with the key you stored in Step 1. ```bash curl theme={null} curl https://svc-2mvx7ckq.endpoints.example.net/v1/chat/completions \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --data '{ "model": "Qwen/Qwen2.5-0.5B-Instruct", "messages": [{"role": "user", "content": "Say hello."}] }' ``` A request without the key is rejected by vLLM itself — Compute does not sit in front of your endpoint. Check the endpoint's `url` scheme before sending anything sensitive. Depending on where capacity was allocated, an endpoint may be served over plain `http`, in which case bearer tokens and payloads cross the internet unencrypted. ## Step 5 - Register with the LLM gateway Optionally register the endpoint as a bring-your-own model in the MKA1 LLM gateway, so it is callable through the platform's `/responses` API with normal platform keys. Add the endpoint to your model catalog: ```bash curl theme={null} curl https://apigw.mka1.com/api/v1/llm/models/catalog \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --data '{ "apiFormat": "completions", "apiProviderType": "openai", "baseUrl": "https://svc-2mvx7ckq.endpoints.example.net/v1", "auth": {"type": "api-key", "value": ""} }' ``` Then register the returned model id: ```bash curl theme={null} curl https://apigw.mka1.com/api/v1/llm/models/registry \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --data '{ "source": "byo", "model_id": "" }' ``` `completions`-format models are accepted at `/responses`; the gateway drives the chat-completions upstream for you. This is why the create body in Step 2 passed the tool-choice flags to vLLM. ## Step 6 - Terminate A service runs, and bills, until you stop it. ```ts MKA1 SDK theme={null} const service = await sdk.computeServices.terminateService({ id: 'service_2mVx7cKq9dRw4bTn' }); console.log(service.state); ``` ```bash curl theme={null} curl https://apigw.mka1.com/api/v1/compute/services/service_2mVx7cKq9dRw4bTn/terminate \ --request POST \ --header 'Authorization: Bearer ' ``` Termination is idempotent, deallocates the provider resource, and stops spend; the record stays readable for audit and usage. ## Serve a model from mka1-repos To serve weights from [mka1-repos](/docs/repositories) — such as the merged output of a [fine-tune job](/docs/compute-fine-tune-job#use-mka1-repos-for-datasets-and-weights), or weights you [pushed over git](/docs/repositories#push-and-pull-with-git) — change only the model identifier and the credentials. Set `HF_ENDPOINT` on the service, pointing at the artifact endpoint for your cluster, and vLLM resolves model identifiers against it — the same Hugging Face access described in [Manage repositories](/docs/repositories#push-and-pull-with-the-hugging-face-cli), so serving a repository is just a model identifier and two environment variables. The console pre-fills `HF_ENDPOINT` when you create a workload; set it yourself when you call the API directly. mka1-repos authenticates with your MKA1 API key, injected as `HF_TOKEN` through `secret_env`: ```json theme={null} { "container": { "image": "vllm/vllm-openai:v0.26.0-cu129", "command": ["vllm", "serve", "meetkai/functionary", "--host", "0.0.0.0", "--port", "8000", "--max-model-len", "4096", "--gpu-memory-utilization", "0.85", "--enable-auto-tool-choice", "--tool-call-parser", "hermes"], "env": { "HF_ENDPOINT": "https://hf.mka1.com" }, "secret_env": { "VLLM_API_KEY": {"secret_id": "sec_Qw8pLm2vTn5xRc7J", "key": "VLLM_API_KEY"}, "HF_TOKEN": {"secret_id": "sec_Vb3nRk8sQw1xYz2M", "key": "HF_TOKEN"} }, "ports": [ {"name": "api", "container_port": 8000, "protocol": "http"} ] } } ``` The service downloads the weights from your organization's repository at startup and serves them under the same identifier, completing the loop: fine-tune as a job, publish to mka1-repos, serve as a service. ## Track spend Service spend accrues per whole minute from allocation and appears alongside jobs in the usage endpoint: ```ts MKA1 SDK theme={null} const usage = await sdk.computeUsage.getComputeUsage({ resourceType: 'service' }); console.log(usage.summary); ``` ```bash curl theme={null} curl "https://apigw.mka1.com/api/v1/compute/usage?resource_type=service" \ --header 'Authorization: Bearer ' ``` Compute spend draws from the same organization budgets as every other MKA1 service, and a service that hits an organization budget or its own `limits` is terminated with no grace period. ## API reference For the full request and response schemas, open the Compute groups in the [API Reference](/api-reference/introduction). ## See also * [Run a fine-tune job](/docs/compute-fine-tune-job) - produce the weights this service serves. * [Manage repositories](/docs/repositories) - create and manage the repositories this service pulls weights from. * [Manage repositories](/docs/repositories) - create a repository and get weights into it in the first place. * [Generate a response](/docs/generate-a-response) - call your registered model through the platform `/responses` API.