> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mka1.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Results and artifacts

> Inspect eval samples, scores, and downloadable artifacts.

First [run an eval](/docs/evals) and keep its run ID. Results describe the suite version, model, inputs, and graders used for that run.

## Inspect samples

List samples when you need per-row debugging.
You can filter by `task_id`, `model`, or `status`.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  import { SDK } from '@meetkai/mka1';

  const sdk = new SDK({ bearerAuth: 'Bearer <mka1-api-key>' });

  const result = await sdk.llm.evals.listSamples({ runId: 'eval_run_abc123', limit: 10, taskId: 'repeat_exactly' });
  console.log(result);
  ```

  ```python Python SDK theme={null}
  from meetkai_mka1 import SDK

  sdk = SDK(bearer_auth="Bearer <mka1-api-key>")

  result = sdk.llm.evals.list_samples(run_id="eval_run_abc123", limit=10, task_id="repeat_exactly")
  print(result)
  ```

  ```csharp C# SDK theme={null}
  using MeetKai.MKA1;
  using MeetKai.MKA1.Types.Requests;

  var sdk = new SDK(bearerAuth: "Bearer <mka1-api-key>");

  var result = await sdk.Llm.Evals.ListSamplesAsync(new ListEvalSamplesRequest { RunId = "eval_run_abc123", Limit = 10, TaskId = "repeat_exactly" });
  Console.WriteLine(result);
  ```

  ```bash CLI theme={null}
  mka1 llm evals list-samples \
    --run-id eval_run_abc123 \
    --limit 10 \
    --task-id repeat_exactly
  ```

  ```bash Bash theme={null}
  curl 'https://apigw.mka1.com/api/v1/llm/evals/runs/eval_run_abc123/samples?limit=10&task_id=repeat_exactly' \
    --header 'Authorization: Bearer <mka1-api-key>'
  ```
</CodeGroup>

You can also:

* Filter by a numeric score band using `score_metric` + `score_min`/`score_max`.
* Skip large rows (like inlined audio) by setting `include_dataset_row=false` (samples will return `dataset_row: null`).
* Fetch only specific sample indices using `sample_index` (comma-separated indices).

Each sample includes the source row, rendered prompt, target, stored Responses `response_id`, raw model output, extracted output, scores, judge details, and error details.
It also carries the tokens and time the sample spent, described in [Token and time fields](#token-and-time-fields).

```json theme={null}
{
  "object": "eval.sample",
  "task_id": "repeat_exactly",
  "model": "openai:gpt-4.1-mini",
  "status": "completed",
  "dataset_row": {
    "question": "Repeat exactly: MKA1_EVAL_SMOKE_OK",
    "answer": "MKA1_EVAL_SMOKE_OK"
  },
  "prompt": "Repeat exactly: MKA1_EVAL_SMOKE_OK",
  "target": "MKA1_EVAL_SMOKE_OK",
  "response_id": "resp_...",
  "output_text": "MKA1_EVAL_SMOKE_OK",
  "extracted_output": "MKA1_EVAL_SMOKE_OK",
  "scores": {
    "exact_match": 1
  },
  "judge": {
    "output": "MKA1_EVAL_SMOKE_OK",
    "target": "MKA1_EVAL_SMOKE_OK"
  },
  "error": null
}
```

### Fetch sample audio (transcription tasks)

For transcription evals, sample lists may redact inline `data:` audio blobs in `dataset_row`. To fetch a single sample's clip reference, call:

`GET /api/v1/llm/evals/runs/{run_id}/samples/{sample_index}/audio`

`sample_index` is unique per task (and model), not per run. If a run includes more than one transcription task and the same `sample_index` could match multiple tasks, pass `task_id` or the API returns 400.

Optionally pass `model` to choose which model row to read the clip from (the clip is identical across models within a task, so this does not change the audio).

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.llm.evals.getSampleAudio({ runId: 'eval_run_abc123', sampleIndex: 0, taskId: 'Common_voice' });
  console.log(result);
  ```

  ```python Python SDK theme={null}
  result = sdk.llm.evals.get_sample_audio(run_id="eval_run_abc123", sample_index=0, task_id="Common_voice")
  print(result)
  ```

  ```csharp C# SDK theme={null}
  var result = await sdk.Llm.Evals.GetSampleAudioAsync(new GetEvalSampleAudioRequest { RunId = "eval_run_abc123", SampleIndex = 0, TaskId = "Common_voice" });
  Console.WriteLine(result);
  ```

  ```bash CLI theme={null}
  mka1 llm evals get-sample-audio \
    --run-id eval_run_abc123 \
    --sample-index 0 \
    --task-id Common_voice
  ```

  ```bash Bash theme={null}
  curl 'https://apigw.mka1.com/api/v1/llm/evals/runs/eval_run_abc123/samples/0/audio?task_id=Common_voice' \
    --header 'Authorization: Bearer <mka1-api-key>'
  ```
</CodeGroup>

The response includes `{ object, audio, sample_index, task_id, model }` where `audio` is a base64 `data:` URI or a URL.

## Fetch artifacts

Completed runs create result files with `purpose=evals`.
Use the artifacts endpoint to find the result and sample artifact file IDs.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.llm.evals.getArtifacts({ runId: 'eval_run_abc123' });
  console.log(result);
  ```

  ```python Python SDK theme={null}
  result = sdk.llm.evals.get_artifacts(run_id="eval_run_abc123")
  print(result)
  ```

  ```csharp C# SDK theme={null}
  var result = await sdk.Llm.Evals.GetArtifactsAsync(runId: "eval_run_abc123");
  Console.WriteLine(result);
  ```

  ```bash CLI theme={null}
  mka1 llm evals get-artifacts \
    --run-id eval_run_abc123
  ```

  ```bash Bash theme={null}
  curl https://apigw.mka1.com/api/v1/llm/evals/runs/eval_run_abc123/artifacts \
    --header 'Authorization: Bearer <mka1-api-key>'
  ```
</CodeGroup>

Then download the files through the Files API:

```bash Bash theme={null}
curl https://apigw.mka1.com/api/v1/llm/files/file_result123/content \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --output eval-result.json
```

The result artifact summarizes run metadata and final metrics.
The samples artifact preserves per-sample details for offline analysis.

## Compare models

### Open the leaderboard

In the console, open **Evaluations** and select an evaluation.
Its **Leaderboard** lists every model with a completed run on the active version, scored task by task, with **Token Usage** and **Cost** charts above the table.
To compare two models answer by answer, select both on the leaderboard and click **Compare selected**.
The compare page shows both models' answers to each sample of a task, and can narrow them to the samples only one model got right.

Through the API, `GET /api/v1/llm/evals/suites/{suite_id}/leaderboard-runs` returns the runs behind the leaderboard.
Pass `suite_version`, because scores are only comparable within one version of a suite.

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.llm.evals.listSuiteLeaderboardRuns({ suiteId: 'eval_suite_abc123', suiteVersion: 1 });
  console.log(result);
  ```

  ```python Python SDK theme={null}
  result = sdk.llm.evals.list_suite_leaderboard_runs(suite_id="eval_suite_abc123", suite_version=1)
  print(result)
  ```

  ```csharp C# SDK theme={null}
  var result = await sdk.Llm.Evals.ListSuiteLeaderboardRunsAsync(new ListEvalSuiteLeaderboardRunsRequest { SuiteId = "eval_suite_abc123", SuiteVersion = 1 });
  Console.WriteLine(result);
  ```

  ```bash CLI theme={null}
  mka1 llm evals list-suite-leaderboard-runs \
    --suite-id eval_suite_abc123 \
    --suite-version 1
  ```

  ```bash Bash theme={null}
  curl 'https://apigw.mka1.com/api/v1/llm/evals/suites/eval_suite_abc123/leaderboard-runs?suite_version=1' \
    --header 'Authorization: Bearer <mka1-api-key>'
  ```
</CodeGroup>

`data` holds the full run objects, and `groups` places them on the board.
Each group entry names a `model`, a `task_id` (`null` when the run covered every task), a `rank` (`1` is the newest completed run), and the `run_id`.
Build the board from `groups`, not from each run's `models`.
If `truncated` is `true`, request again with `cursor` set to `next_cursor` and combine the responses.
The [API reference](/api-reference/evals/list-leaderboard-runs-for-an-eval-suite) lists the other parameters.

### Token and time fields

Completed runs report what each model spent.
Each `metrics.by_task.<task>.<model>` entry carries `tokens` and `timing` next to its scores when they were measured, and the run's `tokens` field holds the run totals:

```json theme={null}
{
  "metrics": {
    "by_task": {
      "repeat_exactly": {
        "openai:gpt-4.1-mini": {
          "sample_count": 1,
          "failed_count": 0,
          "metrics": {
            "exact_match": 1
          },
          "tokens": {
            "input": 17,
            "output": 7,
            "reasoning": 0,
            "reasoning_estimated": 0
          },
          "timing": {
            "generation_seconds": 0.784,
            "timed_samples": 1
          }
        }
      }
    }
  },
  "tokens": {
    "input": 17,
    "output": 7,
    "cached": null,
    "reasoning": null
  }
}
```

| Field | What it holds |
| - | - |
| `metrics.by_task.<task>.<model>.tokens` | Tokens one model spent on one task, failed samples included. `input` includes `cached`, and `output` includes `reasoning`. Where the provider reports no reasoning split, `reasoning` is estimated from the stored text, and `reasoning_estimated` counts the estimated samples. |
| `metrics.by_task.<task>.<model>.timing` | `generation_seconds` is the wall-clock time of the model calls, summed over `timed_samples` samples. Divide one by the other for the average time per answer. |
| `tokens` | The run totals over every sample: `input`, `output`, `cached`, and `reasoning`. `reasoning` counts only splits the provider reported, so it is `null` when none did. |

Each sample carries its own figures in `input_tokens`, `output_tokens`, and `phase_timings`.
The `generation` phase records when the model call started and finished:

```json theme={null}
{
  "object": "eval.sample",
  "task_id": "repeat_exactly",
  "model": "openai:gpt-4.1-mini",
  "input_tokens": 17,
  "output_tokens": 7,
  "phase_timings": {
    "generation": {
      "started_at": "2026-10-07T09:12:03.118Z",
      "finished_at": "2026-10-07T09:12:03.902Z"
    }
  }
}
```

A figure that was never measured is `null` or absent, not zero.
When the provider reports no token counts, the tokens are counted from the stored text and the sample's `tokens_estimated` is `true`.
Use those counts to compare how verbose models are, not as billing figures.

### Cost per answer

The console prices answers at your organization's effective rates, which `GET /api/v1/llm/models/pricing/effective` returns ([Check effective prices](/docs/models#step-6-check-effective-prices)).
It applies those rates to the per-task token totals and divides by the number of answers.
A model without a price has no cost per answer, so the **Cost** charts leave it out instead of showing it as free.
Cost stays empty while your models are unpriced.

### Compare more than 20 models

A run takes up to 20 models.
The leaderboard is not limited to one run: it combines every completed run of a suite version and uses each model's most recent result.
To compare more than 20 models, split them across several runs of the same suite version.

### Import external results

To put a coding-agent run you executed outside MKA1 on the leaderboard, import it with `POST /api/v1/llm/evals/imports/agent-run`.
The import records a completed run against the suite version you name, or the active version by default, without executing anything.
Every `task_id` in its trials must already exist in that version's manifest.
Pass an `idempotency_key` so a retried upload does not import the run twice.
In the console, an evaluation of coding agents shows **Import run** in place of **Run evaluation**.
The [API reference](/api-reference/evals/import-a-finished-coding-agent-run) describes the request body.

## Pagination and filtering

List endpoints use cursor pagination.

| Endpoint | Filters |
| - | - |
| `GET /evals/suites` | `after`, `limit` |
| `GET /evals/suites/{suite_id}/versions` | `after`, `limit` |
| `GET /evals/runs` | `after`, `limit`, `suite_id`, `suite_version`, `status` (Note: `suite_version` must be paired with `suite_id`.) |
| `GET /evals/schedules` | `after`, `limit`, `suite_id`, `enabled` (string) |
| `GET /evals/schedules/{schedule_id}/runs` | `after`, `limit`, `suite_id`, `suite_version`, `status` (Note: `suite_version` may be used without `suite_id` since the schedule already pins the suite; supplying a different `suite_id` returns 400.) |
| `GET /evals/runs/{run_id}/samples` | `after`, `limit`, `task_id`, `model`, `status`, `score_metric`, `score_min`, `score_max`, `include_dataset_row`, `include_reasoning`, `sample_index` |
| `GET /files` | `after`, `limit`, `order`, `purpose` |

Example:

<CodeGroup>
  ```ts TypeScript SDK theme={null}
  const result = await sdk.llm.evals.listRuns({ status: 'completed', limit: 20 });
  console.log(result);
  ```

  ```python Python SDK theme={null}
  result = sdk.llm.evals.list_runs(status="completed", limit=20)
  print(result)
  ```

  ```csharp C# SDK theme={null}
  var result = await sdk.Llm.Evals.ListRunsAsync(new ListEvalRunsRequest { Status = ListEvalRunsStatus.Completed, Limit = 20 });
  Console.WriteLine(result);
  ```

  ```bash CLI theme={null}
  mka1 llm evals list-runs \
    --status completed \
    --limit 20
  ```

  ```bash Bash theme={null}
  curl 'https://apigw.mka1.com/api/v1/llm/evals/runs?status=completed&limit=20' \
    --header 'Authorization: Bearer <mka1-api-key>'
  ```
</CodeGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.