Results and artifacts

First run an eval and keep its run ID. Results describe the suite version, model, inputs, and graders used for that run.

Inspect samples

List samples when you need per-row debugging. You can filter by task_id, model, or status.

import { SDK } from '@meetkai/mka1';

const sdk = new SDK({ bearerAuth: 'Bearer <mka1-api-key>' });

const result = await sdk.llm.evals.listSamples({ runId: 'eval_run_abc123', limit: 10, taskId: 'repeat_exactly' });
console.log(result);

from meetkai_mka1 import SDK

sdk = SDK(bearer_auth="Bearer <mka1-api-key>")

result = sdk.llm.evals.list_samples(run_id="eval_run_abc123", limit=10, task_id="repeat_exactly")
print(result)

using MeetKai.MKA1;
using MeetKai.MKA1.Types.Requests;

var sdk = new SDK(bearerAuth: "Bearer <mka1-api-key>");

var result = await sdk.Llm.Evals.ListSamplesAsync(new ListEvalSamplesRequest { RunId = "eval_run_abc123", Limit = 10, TaskId = "repeat_exactly" });
Console.WriteLine(result);

mka1 llm evals list-samples \
  --run-id eval_run_abc123 \
  --limit 10 \
  --task-id repeat_exactly

curl 'https://apigw.mka1.com/api/v1/llm/evals/runs/eval_run_abc123/samples?limit=10&task_id=repeat_exactly' \
  --header 'Authorization: Bearer <mka1-api-key>'

You can also:

Filter by a numeric score band using score_metric + score_min/score_max.
Skip large rows (like inlined audio) by setting include_dataset_row=false (samples will return dataset_row: null).
Fetch only specific sample indices using sample_index (comma-separated indices).

Each sample includes the source row, rendered prompt, target, stored Responses response_id, raw model output, extracted output, scores, judge details, and error details.

{
  "object": "eval.sample",
  "task_id": "repeat_exactly",
  "model": "openai:gpt-4.1-mini",
  "status": "completed",
  "dataset_row": {
    "question": "Repeat exactly: MKA1_EVAL_SMOKE_OK",
    "answer": "MKA1_EVAL_SMOKE_OK"
  },
  "prompt": "Repeat exactly: MKA1_EVAL_SMOKE_OK",
  "target": "MKA1_EVAL_SMOKE_OK",
  "response_id": "resp_...",
  "output_text": "MKA1_EVAL_SMOKE_OK",
  "extracted_output": "MKA1_EVAL_SMOKE_OK",
  "scores": {
    "exact_match": 1
  },
  "judge": {
    "output": "MKA1_EVAL_SMOKE_OK",
    "target": "MKA1_EVAL_SMOKE_OK"
  },
  "error": null
}

Fetch sample audio (transcription tasks)

For transcription evals, sample lists may redact inline data: audio blobs in dataset_row. To fetch a single sample’s clip reference, call: GET /api/v1/llm/evals/runs/{run_id}/samples/{sample_index}/audio sample_index is unique per task (and model), not per run. If a run includes more than one transcription task and the same sample_index could match multiple tasks, pass task_id or the API returns 400. Optionally pass model to choose which model row to read the clip from (the clip is identical across models within a task, so this does not change the audio).

const result = await sdk.llm.evals.getSampleAudio({ runId: 'eval_run_abc123', sampleIndex: 0, taskId: 'Common_voice' });
console.log(result);

result = sdk.llm.evals.get_sample_audio(run_id="eval_run_abc123", sample_index=0, task_id="Common_voice")
print(result)

var result = await sdk.Llm.Evals.GetSampleAudioAsync(new GetEvalSampleAudioRequest { RunId = "eval_run_abc123", SampleIndex = 0, TaskId = "Common_voice" });
Console.WriteLine(result);

mka1 llm evals get-sample-audio \
  --run-id eval_run_abc123 \
  --sample-index 0 \
  --task-id Common_voice

curl 'https://apigw.mka1.com/api/v1/llm/evals/runs/eval_run_abc123/samples/0/audio?task_id=Common_voice' \
  --header 'Authorization: Bearer <mka1-api-key>'

The response includes { object, audio, sample_index, task_id, model } where audio is a base64 data: URI or a URL.

Fetch artifacts

Completed runs create result files with purpose=evals. Use the artifacts endpoint to find the result and sample artifact file IDs.

const result = await sdk.llm.evals.getArtifacts({ runId: 'eval_run_abc123' });
console.log(result);

result = sdk.llm.evals.get_artifacts(run_id="eval_run_abc123")
print(result)

var result = await sdk.Llm.Evals.GetArtifactsAsync(runId: "eval_run_abc123");
Console.WriteLine(result);

mka1 llm evals get-artifacts \
  --run-id eval_run_abc123

curl https://apigw.mka1.com/api/v1/llm/evals/runs/eval_run_abc123/artifacts \
  --header 'Authorization: Bearer <mka1-api-key>'

Then download the files through the Files API:

Bash

curl https://apigw.mka1.com/api/v1/llm/files/file_result123/content \
  --header 'Authorization: Bearer <mka1-api-key>' \
  --output eval-result.json

The result artifact summarizes run metadata and final metrics. The samples artifact preserves per-sample details for offline analysis.

Pagination and filtering

List endpoints use cursor pagination.

Endpoint	Filters
`GET /evals/suites`	`after`, `limit`
`GET /evals/suites/{suite_id}/versions`	`after`, `limit`
`GET /evals/runs`	`after`, `limit`, `suite_id`, `suite_version`, `status` (Note: `suite_version` must be paired with `suite_id`.)
`GET /evals/schedules`	`after`, `limit`, `suite_id`, `enabled` (string)
`GET /evals/schedules/{schedule_id}/runs`	`after`, `limit`, `suite_id`, `suite_version`, `status` (Note: `suite_version` may be used without `suite_id` since the schedule already pins the suite; supplying a different `suite_id` returns 400.)
`GET /evals/runs/{run_id}/samples`	`after`, `limit`, `task_id`, `model`, `status`, `score_metric`, `score_min`, `score_max`, `include_dataset_row`, `include_reasoning`, `sample_index`
`GET /files`	`after`, `limit`, `order`, `purpose`

Example:

const result = await sdk.llm.evals.listRuns({ status: 'completed', limit: 20 });
console.log(result);

result = sdk.llm.evals.list_runs(status="completed", limit=20)
print(result)

var result = await sdk.Llm.Evals.ListRunsAsync(new ListEvalRunsRequest { Status = ListEvalRunsStatus.Completed, Limit = 20 });
Console.WriteLine(result);

mka1 llm evals list-runs \
  --status completed \
  --limit 20

curl 'https://apigw.mka1.com/api/v1/llm/evals/runs?status=completed&limit=20' \
  --header 'Authorization: Bearer <mka1-api-key>'

Get started

SDKs and CLI

Responses

Standalone APIs

Agents

Evals

Compute

Authentication

Production

Results and artifacts

Inspect samples

Fetch sample audio (transcription tasks)

Fetch artifacts

​Inspect samples

​Fetch sample audio (transcription tasks)

​Fetch artifacts

​Pagination and filtering

Inspect samples

Fetch sample audio (transcription tasks)

Fetch artifacts

Pagination and filtering