> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mka1.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run a fine-tune job

> <Badge color="blue" shape="pill">Early access</Badge>

Fine-tune a model on rented GPUs with a Compute job, monitor training from logs and SSH, and publish the merged weights to mka1-repos.


Use a Compute job when you need GPUs for a finite workload such as fine-tuning.
You bring an ordinary container image and a command; Compute allocates GPU capacity, runs the workload to completion, streams durable logs, and deallocates the hardware automatically.

Compute has no fine-tuning schema.
A job is a generic container run, so this guide fine-tunes with [ms-swift](https://github.com/modelscope/ms-swift) purely as workload payload — any training stack works the same way.

<Note>
  This guide rents GPUs and runs your own training container.
</Note>

The CLI examples assume an [authenticated MKA1 CLI](/docs/cli/authentication).

## Before you start

You need:

| Requirement | Notes |
| - | - |
| API key | Send it as `Authorization: Bearer <mka1-api-key>` on every request. |
| Compute enabled for your organization | Compute is a fail-closed allowlist because real money is at stake. If requests return `403 tenant_disabled`, ask a cluster administrator to enable your organization. |
| A container image with your training stack | Any Linux GPU image the provider can pull without interactive authentication. It must run as root with `bash`, `curl`, `base64`, and GNU coreutils available, allow outbound HTTPS, and keep the workload in the foreground. |
| SSH public key | Optional. Supply one at create time if you want to debug the run interactively. Compute never generates or holds private keys. |

Jobs move through `requested` → `allocating` → `provisioning` → `running` → `finalizing`, then land in a terminal state:

* A workload that exits `0` ends as `succeeded`; a failure in any phase ends as `failed`.
* Explicit termination, a binding limit, or an exhausted organization budget moves the job through `terminating` to `terminated`.

<Warning>
  **Billing starts at allocation, not readiness.**

  Spend accrues from the moment hardware is allocated, so provisioning time and failed boots cost money.
  Set `limits.max_runtime_hours` and `limits.max_cost_usd` on every job; a job that hits a binding limit is terminated and deallocated immediately.
</Warning>

## Step 1 - Pick an accelerator

List the curated accelerator catalog and choose one by its stable `name`.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  import { SDK } from '@meetkai/mka1';
  import { readFileSync } from 'node:fs';

  const sdk = new SDK({ bearerAuth: 'Bearer <mka1-api-key>' });

  const result = await sdk.computeCatalog.listAccelerators({});
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.data);
  ```

  ```python Python SDK theme={null}
  from meetkai_mka1 import models
  import base64
  from pathlib import Path
  import os
  from meetkai_mka1 import SDK

  sdk = SDK(bearer_auth="Bearer <mka1-api-key>")

  result = sdk.compute_catalog.list_accelerators()
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.data)
  ```

  ```csharp C# SDK theme={null}
  using MeetKai.MKA1.Types.Components;
  using MeetKai.MKA1.Types.Requests;
  using Newtonsoft.Json;
  using MeetKai.MKA1;

  var sdk = new SDK(bearerAuth: "Bearer <mka1-api-key>");

  var response = await sdk.ComputeCatalog.ListAcceleratorsAsync();
  var result = response.ComputeAcceleratorList
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Data));
  ```

  ```bash CLI theme={null}
  mka1 compute-catalog list-accelerators \
    --limit 50 \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl "https://apigw.mka1.com/api/v1/compute/catalog/accelerators" \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

Each entry describes the hardware, the GPU counts you can request, and the interconnects it supports:

```json theme={null}
{
  "name": "nvidia-rtx-4090-24gb",
  "display_name": "NVIDIA GeForce RTX 4090 24GB",
  "vendor": "nvidia",
  "memory_gb": 24,
  "gpu_counts": [1, 2, 3, 4, 5, 6, 7, 8],
  "interconnects": ["pcie"]
}
```

## Step 2 - Check price and availability

Optionally request a quote before creating anything.
A quote is a live observation of the market: it returns whether the configuration is currently obtainable and the hourly price range, and it reserves nothing.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeCatalog.createComputeQuote({
    "computeQuoteRequest": {
      "resourceType": "job",
      "compute": {
        "accelerator": "nvidia-rtx-4090-24gb",
        "gpuCount": 1,
        "ephemeralDiskGb": 100
      },
      "container": {
        "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3"
      },
      "access": {
        "ssh": true
      }
    }
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.available);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_catalog.create_compute_quote(
      resource_type="job",
      compute={
          "accelerator": "nvidia-rtx-4090-24gb",
          "gpu_count": 1,
          "ephemeral_disk_gb": 100,
      },
      container={
          "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3"
      },
      access={"ssh": True},
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.available)
  ```

  ```csharp C# SDK theme={null}
  // Deserialize the wire-format payload into the SDK request model.
  var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.ComputeQuoteRequest> ("""
  {
    "resource_type": "job",
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "gpu_count": 1,
      "ephemeral_disk_gb": 100
    },
    "container": {
      "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3"
    },
    "access": {
      "ssh": true
    }
  }
  """)!;

  var response = await sdk.ComputeCatalog.CreateComputeQuoteAsync(body);
  var result = response.ComputeQuote
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Available));
  ```

  ```bash CLI theme={null}
  mka1 compute-catalog create-compute-quote \
    --output-format json \
    --body @- <<'JSON'
  {
    "access": {
      "ssh": true
    },
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "ephemeral_disk_gb": 100,
      "gpu_count": 1
    },
    "container": {
      "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3"
    },
    "resource_type": "job"
  }
  JSON
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/compute/catalog/quotes \
    --request POST \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $MKA1_API_KEY" \
    --data '{
      "resource_type": "job",
      "compute": {
        "accelerator": "nvidia-rtx-4090-24gb",
        "gpu_count": 1,
        "ephemeral_disk_gb": 100
      },
      "container": {
        "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3"
      },
      "access": {"ssh": true}
    }'
  ```
</CodeGroup>

```json theme={null}
{
  "available": true,
  "price_usd_hr": {"min": 0.38, "max": 0.69},
  "observed_at": "2026-07-29T12:00:00Z",
  "expires_at": null
}
```

An unavailable configuration is still a `200` response, with `available: false` and null price bounds.

## Step 3 - Create the job

Create the job with a required `Idempotency-Key` header.
Retrying with the same key and the same body returns the same job with a `200` instead of creating a duplicate; the same key with a different body returns `409 idempotency_conflict`.

This example runs a four-step LoRA fine-tune of Qwen2.5-0.5B-Instruct, merges the adapter, and publishes the weights before the job exits.
Create the output repository and artifact secret using [Prepare output storage](#prepare-output-storage) first.
Save this complete workload as `train-and-publish.sh` beside the SDK or CLI example. The Bash and CLI tabs also require `jq`.
The CLI runs this same script through `container.command`.

```bash train-and-publish.sh theme={null}
#!/usr/bin/env bash
set -euo pipefail
: "${OUTPUT_MODEL_REPO:?Set OUTPUT_MODEL_REPO to org/name}"
: "${ARTIFACT_TOKEN:?Inject ARTIFACT_TOKEN through a Compute secret}"
export ARTIFACT_ENDPOINT="${ARTIFACT_ENDPOINT:-https://hf.mka1.com}"
export USE_HF=1 HF_HUB_DISABLE_XET=1
# Download the base model and dataset from the public Hub.
unset HF_ENDPOINT HF_TOKEN
python - <<'PY'
import os
from huggingface_hub import HfApi
HfApi(endpoint=os.environ['ARTIFACT_ENDPOINT'], token=os.environ['ARTIFACT_TOKEN']).repo_info(repo_id=os.environ['OUTPUT_MODEL_REPO'])
PY
swift sft \
  --model Qwen/Qwen2.5-0.5B-Instruct \
  --dataset 'tatsu-lab/alpaca#64' \
  --max_steps 4 --per_device_train_batch_size 1 --max_length 256 \
  --lora_rank 4 --save_steps 4 --logging_steps 1 \
  --output_dir /workspace/training-output
checkpoint=$(python - <<'PY'
from pathlib import Path
checkpoints = list(Path('/workspace/training-output').rglob('checkpoint-*'))
checkpoints = [p for p in checkpoints if p.is_dir() and p.name.removeprefix('checkpoint-').isdigit()]
if not checkpoints:
    raise RuntimeError('Training produced no checkpoint')
print(max(checkpoints, key=lambda p: int(p.name.removeprefix('checkpoint-'))))
PY
)
swift export --adapters "$checkpoint" --merge_lora true --output_dir /workspace/merged
python - <<'PY'
import os
from huggingface_hub import HfApi
api = HfApi(endpoint=os.environ['ARTIFACT_ENDPOINT'], token=os.environ['ARTIFACT_TOKEN'])
api.upload_folder(repo_id=os.environ['OUTPUT_MODEL_REPO'], folder_path='/workspace/merged')
print('Published merged weights to', os.environ['OUTPUT_MODEL_REPO'], flush=True)
PY
```

Replace `YOUR_ORG/qwen-fine-tune` and `sec_REPLACE_WITH_ARTIFACT_SECRET` with your repository and returned secret ID.
The SDK tabs encode the local script. For Bash, encode it once and use `jq` to build the request body as shown below.
This is a short workflow check, not a quality benchmark.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  // Keep this value unchanged when retrying this request.
  const idempotencyKey = process.env.MKA1_REQUEST_ID;
  if (!idempotencyKey) throw new Error('Set MKA1_REQUEST_ID');

  const result = await sdk.computeJobs.createJob({
    "idempotencyKey": idempotencyKey,
    "computeJobCreate": {
      "name": "qwen-fine-tune",
      "compute": {
        "accelerator": "nvidia-rtx-4090-24gb",
        "gpuCount": 1,
        "ephemeralDiskGb": 100
      },
      "container": {
        "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3",
        "env": {
          "USE_HF": "1",
          "HF_HUB_DISABLE_XET": "1",
          "ARTIFACT_ENDPOINT": "https://hf.mka1.com",
          "OUTPUT_MODEL_REPO": "YOUR_ORG/qwen-fine-tune"
        },
        "scriptB64": readFileSync('train-and-publish.sh').toString('base64'),
        "secretEnv": {
          "ARTIFACT_TOKEN": {
            "secretId": "sec_REPLACE_WITH_ARTIFACT_SECRET",
            "key": "ARTIFACT_TOKEN"
          }
        }
      },
      "limits": {
        "maxRuntimeHours": 2,
        "maxCostUsd": 5
      }
    }
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.id);
  ```

  ```python Python SDK theme={null}
  idempotency_key = os.environ["MKA1_REQUEST_ID"]  # Reuse for retries.

  result = sdk.compute_jobs.create_job(
      idempotency_key=idempotency_key,
      name="qwen-fine-tune",
      compute={
          "accelerator": "nvidia-rtx-4090-24gb",
          "gpu_count": 1,
          "ephemeral_disk_gb": 100,
      },
      container={
          "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3",
          "env": {
              "USE_HF": "1",
              "HF_HUB_DISABLE_XET": "1",
              "ARTIFACT_ENDPOINT": "https://hf.mka1.com",
              "OUTPUT_MODEL_REPO": "YOUR_ORG/qwen-fine-tune",
          },
          "script_b64": base64.b64encode(
              Path("train-and-publish.sh").read_bytes()
          ).decode("ascii"),
          "secret_env": {
              "ARTIFACT_TOKEN": {
                  "secret_id": "sec_REPLACE_WITH_ARTIFACT_SECRET",
                  "key": "ARTIFACT_TOKEN",
              }
          },
      },
      limits={"max_runtime_hours": 2, "max_cost_usd": 5},
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.id)
  ```

  ```csharp C# SDK theme={null}
  var idempotencyKey = System.Environment.GetEnvironmentVariable("MKA1_REQUEST_ID")
      ?? throw new InvalidOperationException("Set MKA1_REQUEST_ID");

  // Deserialize the wire-format payload into the SDK request model.
  var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.ComputeJobCreate> ("""
  {
    "name": "qwen-fine-tune",
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "gpu_count": 1,
      "ephemeral_disk_gb": 100
    },
    "container": {
      "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3",
      "env": {
        "USE_HF": "1",
        "HF_HUB_DISABLE_XET": "1",
        "ARTIFACT_ENDPOINT": "https://hf.mka1.com",
        "OUTPUT_MODEL_REPO": "YOUR_ORG/qwen-fine-tune"
      },
      "script_b64": "SCRIPT_BASE64",
      "secret_env": {
        "ARTIFACT_TOKEN": {
          "secret_id": "sec_REPLACE_WITH_ARTIFACT_SECRET",
          "key": "ARTIFACT_TOKEN"
        }
      }
    },
    "limits": {
      "max_runtime_hours": 2,
      "max_cost_usd": 5
    }
  }
  """.Replace("SCRIPT_BASE64", Convert.ToBase64String(System.IO.File.ReadAllBytes("train-and-publish.sh"))))!;

  var response = await sdk.ComputeJobs.CreateJobAsync(idempotencyKey, body);
  var result = response.ComputeJob
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Id));
  ```

  ```bash CLI theme={null}
  jq --arg script "$(base64 < train-and-publish.sh | tr -d '\n')" \
    '.container.command = ["bash", "-lc", ("set -o pipefail; printf %s " + $script + " | base64 -d | bash")]' <<'JSON' > job.json
  {
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "ephemeral_disk_gb": 100,
      "gpu_count": 1
    },
    "container": {
      "env": {
        "USE_HF": "1",
        "HF_HUB_DISABLE_XET": "1",
        "ARTIFACT_ENDPOINT": "https://hf.mka1.com",
        "OUTPUT_MODEL_REPO": "YOUR_ORG/qwen-fine-tune"
      },
      "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3",
      "secret_env": {
        "ARTIFACT_TOKEN": {
          "key": "ARTIFACT_TOKEN",
          "secret_id": "sec_REPLACE_WITH_ARTIFACT_SECRET"
        }
      }
    },
    "limits": {
      "max_cost_usd": 5,
      "max_runtime_hours": 2
    },
    "name": "qwen-fine-tune"
  }
  JSON

  mka1 compute-jobs create-job \
    --idempotency-key "$MKA1_REQUEST_ID" \
    --output-format json \
    --body @job.json
  ```

  ```bash Bash theme={null}
  jq --arg script "$(base64 < train-and-publish.sh | tr -d '\n')" '.container.script_b64 = $script' <<'JSON' > job.json
  {
    "name": "qwen-fine-tune",
    "compute": {
      "accelerator": "nvidia-rtx-4090-24gb",
      "gpu_count": 1,
      "ephemeral_disk_gb": 100
    },
    "container": {
      "image": "modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.9.1-py312-torch2.10.0-vllm0.19.1-modelscope1.35.4-swift4.1.3",
      "env": {
        "USE_HF": "1",
        "HF_HUB_DISABLE_XET": "1",
        "ARTIFACT_ENDPOINT": "https://hf.mka1.com",
        "OUTPUT_MODEL_REPO": "YOUR_ORG/qwen-fine-tune"
      },
      "script_b64": "SCRIPT_BASE64",
      "secret_env": {
        "ARTIFACT_TOKEN": {
          "secret_id": "sec_REPLACE_WITH_ARTIFACT_SECRET",
          "key": "ARTIFACT_TOKEN"
        }
      }
    },
    "limits": {
      "max_runtime_hours": 2,
      "max_cost_usd": 5
    }
  }
  JSON

  curl https://apigw.mka1.com/api/v1/compute/jobs \
    --request POST \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $MKA1_API_KEY" \
    --header "Idempotency-Key: $MKA1_REQUEST_ID" \
    --data @job.json
  ```
</CodeGroup>

Field notes:

| Field | Notes |
| - | - |
| `compute.accelerator`, `compute.gpu_count` | Required. A catalog accelerator name and 1-8 GPUs on a single node. |
| `compute.ephemeral_disk_gb` | Required, with no default. Size it for the image plus every download; the ms-swift image above alone needs most of 60 GB. |
| `container.command` or `container.script_b64` | Exactly one is required. `command` is an argv; `script_b64` is a base64-encoded bash script, which is easier for multi-step workloads. |
| `container.env` | Plain environment variables. Only `MKA1_*` names are reserved; every other name, including `HF_ENDPOINT`, is yours to set. |
| `container.secret_env` | Environment variables resolved from stored secrets at launch. Use this for tokens so they never appear in provider pod specs — see the [mka1-repos section](#use-mka1-repos-for-datasets-and-weights). |
| `placement.strategy` | Optional. `capacity_optimized` (the default) prefers the provider most likely to reach a running workload, then price within each provider; `lowest_cost` takes the cheapest offer wherever it is. |
| `placement.providers` | Optional. Restricts placement to the named providers. A filter, not an ordering — the strategy decides the order, so listing providers in a different order changes nothing. |
| `limits` | Optional but recommended. Runtime and cost ceilings are independent, and either one terminates the job when hit. |

Why `capacity_optimized` is the default: billing starts at allocation, so the
cheapest offer is not the cheapest outcome when the machine is slow to start
or never reaches the workload. Choose `lowest_cost` when the quoted rate
matters more than time-to-start.

The response is `201` with the durable job in state `requested`:

```json theme={null}
{
  "id": "job_9f2kQxWv3bT8mLpZ",
  "name": "qwen-fine-tune",
  "state": "requested",
  "reason": null,
  "compute": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "ephemeral_disk_gb": 100},
  "accrued_usd": 0,
  "limits": {"max_runtime_hours": 2, "max_cost_usd": 5},
  "endpoints": [],
  "created_at": "2026-07-29T12:01:00Z",
  "started_at": null,
  "terminal_at": null
}
```

Creation succeeding means the request was accepted, not that capacity exists.
Capacity exhaustion, provider errors, and bootstrap failures surface later through `state`, `reason`, logs, and events — never as an HTTP error on a create that already returned `201`.

## Step 4 - Poll until it finishes

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeJobs.getJob({
    "id": "job_9f2kQxWv3bT8mLpZ"
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.state);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_jobs.get_job(
      id="job_9f2kQxWv3bT8mLpZ",
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.state)
  ```

  ```csharp C# SDK theme={null}
  var response = await sdk.ComputeJobs.GetJobAsync("job_9f2kQxWv3bT8mLpZ");
  var result = response.ComputeJob
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.State));
  ```

  ```bash CLI theme={null}
  mka1 compute-jobs get-job \
    --id job_9f2kQxWv3bT8mLpZ \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/compute/jobs/job_9f2kQxWv3bT8mLpZ \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

While running, the job reports the hardware it actually obtained, the hourly price captured at allocation, spend so far, and a ready-to-use SSH command if you supplied a key:

```json theme={null}
{
  "id": "job_9f2kQxWv3bT8mLpZ",
  "name": "qwen-fine-tune",
  "state": "running",
  "reason": null,
  "compute": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "ephemeral_disk_gb": 100},
  "hardware": {"accelerator": "nvidia-rtx-4090-24gb", "gpu_count": 1, "gpu_memory_gb": 24},
  "price_usd_hr": 0.44,
  "accrued_usd": 0.07,
  "allocated_at": "2026-07-29T12:03:12Z",
  "ssh": {"host": "203.0.113.7", "port": 22022, "user": "root", "command": "ssh -p 22022 root@203.0.113.7"},
  "limits": {"max_runtime_hours": 2, "max_cost_usd": 5},
  "endpoints": [],
  "created_at": "2026-07-29T12:01:00Z",
  "started_at": "2026-07-29T12:04:40Z",
  "terminal_at": null
}
```

When the workload exits `0`, the job moves through `finalizing` to `succeeded`, `exit_code` is recorded, the provider resource is deallocated automatically, and spend stops accruing.
A non-zero exit ends in `failed` with a `reason` explaining why.
Terminal jobs stay readable for audit and usage.

## Step 5 - Read logs and events

Logs are durable and cursor-paginated, and they distinguish your workload's output from system output.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeJobs.listJobLogs({
    "source": "user",
    "order": "desc",
    "limit": 50,
    "id": "job_9f2kQxWv3bT8mLpZ"
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.data);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_jobs.list_job_logs(
      source="user",
      order="desc",
      limit=50,
      id="job_9f2kQxWv3bT8mLpZ",
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.data)
  ```

  ```csharp C# SDK theme={null}
  var request = JsonConvert.DeserializeObject<ListJobLogsRequest> ("""
  {
    "source": "user",
    "order": "desc",
    "limit": 50,
    "id": "job_9f2kQxWv3bT8mLpZ"
  }
  """)!;

  var response = await sdk.ComputeJobs.ListJobLogsAsync(request);
  var result = response.ComputeLogList
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Data));
  ```

  ```bash CLI theme={null}
  mka1 compute-jobs list-job-logs \
    --id job_9f2kQxWv3bT8mLpZ \
    --limit 50 \
    --order desc \
    --source user \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl "https://apigw.mka1.com/api/v1/compute/jobs/job_9f2kQxWv3bT8mLpZ/logs?source=user&order=desc&limit=50" \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

```json theme={null}
{
  "data": [
    {
      "id": "log_01J1",
      "timestamp": "2026-07-29T12:06:02Z",
      "source": "user",
      "stream": "stdout",
      "message": "{'loss': 2.31, 'epoch': 0.25, 'step': 1}"
    }
  ],
  "next_cursor": null,
  "total": 412
}
```

Use `source=system` for allocation and bootstrap output, and `order=asc` (the default) to page oldest-first.
Lifecycle transitions such as `provider_allocated`, `workload_started`, and `workload_exited` are also available as structured events:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeJobs.listJobEvents({
    "id": "job_9f2kQxWv3bT8mLpZ"
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.data);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_jobs.list_job_events(
      id="job_9f2kQxWv3bT8mLpZ",
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.data)
  ```

  ```csharp C# SDK theme={null}
  var request = JsonConvert.DeserializeObject<ListJobEventsRequest> ("""
  {
    "id": "job_9f2kQxWv3bT8mLpZ"
  }
  """)!;

  var response = await sdk.ComputeJobs.ListJobEventsAsync(request);
  var result = response.ComputeEventList
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Data));
  ```

  ```bash CLI theme={null}
  mka1 compute-jobs list-job-events \
    --id job_9f2kQxWv3bT8mLpZ \
    --limit 50 \
    --order asc \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl "https://apigw.mka1.com/api/v1/compute/jobs/job_9f2kQxWv3bT8mLpZ/events" \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

## Step 6 - Stop a job early

Termination is explicit, idempotent, and returns the current resource.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeJobs.terminateJob({
    "id": "job_9f2kQxWv3bT8mLpZ"
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.state);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_jobs.terminate_job(
      id="job_9f2kQxWv3bT8mLpZ",
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.state)
  ```

  ```csharp C# SDK theme={null}
  var response = await sdk.ComputeJobs.TerminateJobAsync("job_9f2kQxWv3bT8mLpZ");
  var result = response.ComputeJob
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.State));
  ```

  ```bash CLI theme={null}
  mka1 compute-jobs terminate-job \
    --id job_9f2kQxWv3bT8mLpZ \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/compute/jobs/job_9f2kQxWv3bT8mLpZ/terminate \
    --request POST \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

There is no `DELETE`: the record is retained for audit, usage, and reconciliation.
A job that hits `limits.max_runtime_hours`, `limits.max_cost_usd`, or an organization budget is terminated the same way, with no grace period and no partial-result guarantee, so publish artifacts from inside the workload before it exits.

## Prepare output storage

Create a repository named `qwen-fine-tune` with [Manage repositories](/docs/repositories#create-a-repository).
Use the returned organization slug in the job's `OUTPUT_MODEL_REPO`.
Store an API key with access to that repository as a Compute secret:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeSecrets.createComputeSecret({
    "computeSecretCreate": {
      "name": "training-artifacts",
      "data": {
        "ARTIFACT_TOKEN": "<mka1-api-key>"
      }
    }
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.id);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_secrets.create_compute_secret(
      name="training-artifacts",
      data={"ARTIFACT_TOKEN": "<mka1-api-key>"},
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.id)
  ```

  ```csharp C# SDK theme={null}
  // Deserialize the wire-format payload into the SDK request model.
  var body = JsonConvert.DeserializeObject<MeetKai.MKA1.Types.Components.ComputeSecretCreate> ("""
  {
    "name": "training-artifacts",
    "data": {
      "ARTIFACT_TOKEN": "<mka1-api-key>"
    }
  }
  """)!;

  var response = await sdk.ComputeSecrets.CreateComputeSecretAsync(body);
  var result = response.ComputeSecret
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Id));
  ```

  ```bash CLI theme={null}
  mka1 compute-secrets create \
    --output-format json \
    --body @- <<'JSON'
  {
    "data": {
      "ARTIFACT_TOKEN": "<mka1-api-key>"
    },
    "name": "training-artifacts"
  }
  JSON
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/compute/secrets \
    --request POST \
    --header "Authorization: Bearer $MKA1_API_KEY" \
    --header 'Content-Type: application/json' \
    --data '{"name":"training-artifacts","data":{"ARTIFACT_TOKEN":"<mka1-api-key>"}}'
  ```
</CodeGroup>

Use the returned secret ID in `container.secret_env.ARTIFACT_TOKEN`.
The script uses a separate artifact client so public base-model and dataset downloads continue to use the public Hub.
An artifact upload error fails the job.

## Use mka1-repos for datasets and weights

Compute's durable logs do not preserve the job's ephemeral filesystem.
The script above publishes merged weights to MKA1 repositories before returning success.
Use [the Hugging Face artifact tools](/docs/repositories#push-and-pull-with-the-hugging-face-cli) to download them, or [serve the repository](/docs/compute-deployment#serve-a-model-from-mka1-repos).

For private training inputs, download them with an explicit `HfApi` or `snapshot_download` client configured for your MKA1 artifact endpoint and token, then pass the local paths to your trainer.
When ms-swift cannot infer a mirrored model's family, set its model type and template explicitly for the model you use.

This short script publishes at completion. If it is terminated earlier, its local checkpoint is lost.
For longer runs, add periodic checkpoint publishing; the [reinforcement learning example](/docs/compute-reinforcement-learning) includes a working checkpoint callback.

## Track spend

Every job response carries `accrued_usd`, and the usage endpoint aggregates spend across resources for a time window:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.computeUsage.getComputeUsage({
    "resourceType": "job"
  });
  if (result && 'error' in result) throw new Error(JSON.stringify(result.error));
  console.log(result.summary);
  ```

  ```python Python SDK theme={null}
  result = sdk.compute_usage.get_compute_usage(
      resource_type="job",
  )
  if isinstance(result, models.ComputeErrorEnvelope):
      raise RuntimeError(result.error)
  print(result.summary)
  ```

  ```csharp C# SDK theme={null}
  var request = new GetComputeUsageRequest
  {
      ResourceType = ComputeUsageResourceType.Job,
      Limit = 50,
  };

  var response = await sdk.ComputeUsage.GetComputeUsageAsync(request);
  var result = response.ComputeUsage
      ?? throw new InvalidOperationException(JsonConvert.SerializeObject(response));
  Console.WriteLine(JsonConvert.SerializeObject(result.Summary));
  ```

  ```bash CLI theme={null}
  mka1 compute-usage get \
    --limit 50 \
    --resource-type job \
    --output-format json
  ```

  ```bash Bash theme={null}
  curl "https://apigw.mka1.com/api/v1/compute/usage?resource_type=job" \
    --header "Authorization: Bearer $MKA1_API_KEY"
  ```
</CodeGroup>

```json theme={null}
{
  "window": {"from": "2026-06-29T12:10:00Z", "to": "2026-07-29T12:10:00Z"},
  "summary": {"total_usd": 12.41, "job_usd": 4.12, "service_usd": 8.29},
  "data": [
    {
      "resource_type": "job",
      "resource_id": "job_9f2kQxWv3bT8mLpZ",
      "name": "qwen-fine-tune",
      "billable_seconds": 1560,
      "accrued_usd": 0.19
    }
  ],
  "next_cursor": null,
  "total": 6
}
```

Billing is metered in whole minutes per resource, and Compute spend draws from the same organization budgets as every other MKA1 service.

## API reference

For the full request and response schemas, open the Compute groups in the [API Reference](/api-reference/introduction).

## See also

* [Manage resources](/docs/compute-resources) - inspect capacity, workloads, secrets, and volumes.

* [Reinforcement learning](/docs/compute-reinforcement-learning) - Early Access reward-driven training workflow.

* [Deploy a model server](/docs/compute-deployment) - serve the weights this job produced behind an authenticated endpoint.

* [Manage repositories](/docs/repositories) - create and manage the repositories this job reads from and publishes to.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.