> ## Documentation Index > Fetch the complete documentation index at: https://docs.mka1.com/llms.txt > Use this file to discover all available pages before exploring further. # Start an error analysis of a leaderboard row > Picks the failing samples of one model on one suite version (100 by default, `sample_size` up to 200), sends them (with their audio on speech boards) to the analyzer in two stages and stores a report of error categories, weak tasks, judge and benchmark issues and actionable items. The job runs in the background: poll the returned id until `status` is `done` or `failed`. A finished analysis of the same samples, metric and prompt version is returned as is unless `force` is true; so is an analysis that is still running. Owner org only; shared orgs read reports but cannot start one. Returns 400 for embedding and coding-agent boards, for a row with no failures, and when the analyzer is not configured on the cluster. ## OpenAPI ````yaml https://apigw.mka1.com/speakeasy.json post /api/v1/llm/evals/suites/{suite_id}/error-analyses openapi: 3.1.1 info: title: MKA1 API version: 1.1.0 description: >- The MKA1 API is a RESTful API that provides access to the MKA1 platform. Learn how to get started with the API and the TypeScript SDK [here](https://mka1.apidocumentation.com/guides/getting-started). license: name: Proprietary servers: - url: https://apigw.mka1.com description: MKA1 API Gateway - url: / description: Relative server URL (configurable via SDK constructor) security: [] tags: - name: Resource Authorization description: >- Manage permissions for LLM resources. Create resources, grant/revoke permissions, and delete resources. Only resource owners can grant, revoke, or delete permissions. x-displayName: Resource Authorization - name: Audit description: >- Audit API for org owners and admins. Trace one request end to end by its X-Request-ID: API gateway timing, the responses it created, and their model invocations and tool calls. x-displayName: Audit - name: Embeddings description: >- Text embedding API endpoints for generating vector representations of text. Create semantic embeddings for search, clustering, and similarity matching using various embedding models. x-displayName: Embeddings - name: Feedback description: >- User feedback API for rating and commenting on chat completions. Collect thumbs up/down ratings and detailed feedback to improve model responses and track user satisfaction. x-displayName: Feedback - name: Images description: >- Image generation API endpoints for creating images from text descriptions. Generate images with control over size, quality, and style. x-displayName: Images - name: MCP Vault description: >- MCP vault API for storing user-owned MCP server configurations and encrypted credentials. Agents reference vault IDs so secrets are resolved only at tool execution time. x-displayName: MCP Vault - name: Speech description: >- Speech API endpoints for audio processing. Convert text to natural-sounding speech (TTS) or transcribe speech to text (STT) in different languages. x-displayName: Speech - name: Usage description: >- Usage tracking and analytics API for monitoring token consumption, request counts, and cost analysis. View detailed statistics per user, model, and time period. x-displayName: Usage - name: Extract description: >- Structured data extraction API for extracting information from files. Define JSON schemas to extract structured data from images, PDFs, and documents. Supports reusable schema templates. x-displayName: Extract - name: Text Classification description: >- Text classification API for categorizing text into predefined labels. Use AI models to classify text content for sentiment analysis, topic categorization, and content moderation. x-displayName: Text Classification - name: Responses description: >- Agent-powered responses API for creating AI agents with autonomous tool usage. Build conversational assistants that can use web search, file operations, image generation, code execution, and MCP integrations. Supports background processing, streaming, and real-time status tracking. x-displayName: Responses - name: Files description: >- File management API for uploading, storing, and managing files with automatic expiration and S3 integration. Upload files that can be used with Assistants, Vector Stores, and other features. Files are stored in S3 with metadata tracked in PostgreSQL. Supports automatic cleanup of expired files. x-displayName: Files - name: Vector Stores description: >- Vector store API for storing and searching documents using embeddings. Create vector stores, upload files with automatic chunking and embedding generation, and perform semantic search. Files are processed asynchronously using Temporal workflows for durability. Supports automatic cleanup of expired stores and LanceDB for efficient vector storage. x-displayName: Vector Stores - name: Conversations description: >- Conversation management API for storing and retrieving conversation state across Response API calls. Create conversations, add items (user messages, assistant messages, system messages), and maintain conversation history. Supports metadata tracking and multi-turn dialogue state management. x-displayName: Conversations - name: Guardrails description: >- AI safety guardrails API for configuring content moderation and security policies. Set up ban word lists, prompt injection detection, and system prompt leakage prevention. Guardrails apply to all requests from an account and can be tested before deployment. x-displayName: Guardrails - name: Models description: >- Model listing API for discovering available models. Returns model IDs, ownership, and metadata for all registered models in the gateway. x-displayName: Models - name: Skills description: >- Skills API for managing versioned bundles of instructions and files following the Agent Skills standard. Create, version, and download reusable skill packages that include SKILL.md manifests for agent environments. x-displayName: Skills - name: Chat Completions description: >- **Deprecated: Use the Responses API (`/api/v1/llm/responses`) instead.** Chat completion endpoints with support for streaming, tool calls, and multiple providers. x-deprecated: true x-displayName: Chat Completions - name: Batches x-displayName: Batches - name: Evals x-displayName: Evals - name: Fine-Tuning x-displayName: Fine-Tuning - name: Memory Stores x-displayName: Memory Stores - name: Prompts x-displayName: Prompts - name: API Key x-displayName: API Key - name: OAuth x-displayName: OAuth - name: Session x-displayName: Session - name: Organization x-displayName: Organization - name: Cluster Admin x-displayName: Cluster Admin - name: Sessions description: Create, inspect, access, and terminate sandbox sessions. x-displayName: Sessions - name: Browser description: >- Connect to browser sessions through the gateway port proxy. Browser sessions expose a Chrome DevTools Protocol endpoint on port 9222. x-displayName: Browser - name: Execution description: Run shell commands and code inside an existing sandbox session. x-displayName: Execution - name: Workspace description: >- Inspect the workspace manifest, transfer files or archives, and download generated artifacts. x-displayName: Workspace - name: Sandbox Usage description: >- Aggregate sandbox usage statistics across sessions, execution, and workspace operations. x-displayName: Sandbox Usage - name: Sandbox Pricing description: >- Cluster-admin management of the sandbox compute rate card used for budgeted spend. x-displayName: Sandbox Pricing - name: Sandbox GitHub description: >- Connect a GitHub account, clone its repositories into sandbox sessions, and push branches and open pull requests from a session. The GitHub token stays in the sandbox service and never enters a session. x-displayName: Sandbox GitHub - name: Sandbox Environments description: >- Save reusable session configuration (repositories, environment variables, network access, and a setup script) and run the setup script in a session. x-displayName: Sandbox Environments - name: Sandbox Connectors description: >- Connect remote MCP servers (Linear, Notion, Sentry and others) to your account so Coding Harness sessions started from an environment that lists them can call their tools. The sandbox service keeps each connector's credential and never puts it in a session. x-displayName: Sandbox Connectors - name: Agents description: Create and manage reusable agent definitions. x-displayName: Agents - name: Agent Versions description: Inspect an agent's configuration history and roll back to a prior version. x-displayName: Agent Versions - name: Agent Runs description: Execute saved agents and inspect persisted run results. x-displayName: Agent Runs - name: Agent Connectors description: >- Connect saved agents to external messaging channels such as Telegram, including text, photo, and supported document exchange. x-displayName: Agent Connectors - name: Agent Schedules description: Create and manage scheduled or recurring saved agent runs. x-displayName: Agent Schedules - name: schema-4_other x-displayName: other - name: Billing x-displayName: Billing - name: Budgets x-displayName: Budgets - name: Settings x-displayName: Settings - name: Compute Jobs x-displayName: Compute Jobs - name: Compute Services x-displayName: Compute Services - name: Compute Catalog x-displayName: Compute Catalog - name: Compute Volumes x-displayName: Compute Volumes - name: Compute Pricing x-displayName: Compute Pricing - name: Compute Secrets x-displayName: Compute Secrets - name: Compute Usage x-displayName: Compute Usage - name: Compute Tenants x-displayName: Compute Tenants - description: Create, list, read, update, and delete repositories. name: Repositories x-displayName: Repositories - name: Tracking Experiments description: Create and query tenant experiments. x-displayName: Tracking Experiments - name: Tracking Runs description: Record and retrieve run parameters, metrics, and tags. x-displayName: Tracking Runs - name: analytics.projects x-displayName: Projects description: Create, list, rename, and archive analytics projects. - name: analytics.keys x-displayName: Capture keys description: Create and revoke project keys for sending events. - name: analytics x-displayName: Events description: Send events and query event counts or unique visitors. - name: analytics.events x-displayName: Event search description: List individual events with their properties, newest first. - name: analytics.visitors x-displayName: Visitor search description: List visitors by their latest activity, with first and last seen times. - name: generators x-displayName: generators - name: generations x-displayName: generations - name: formats x-displayName: formats - description: >- Flags, environments, evaluation, and the audit log. Administrative operations require a cluster administrator; evaluation uses an environment-specific evaluation key. name: Feature Flags x-displayName: Feature Flags - name: Service Status description: >- Current status of each service and the incident history, for this deployment's region. x-displayName: Service Status - name: Support Cases description: >- Open and follow support cases. Organization owners and admins see all of the organization's cases; other members see the cases they opened. x-displayName: Support Cases paths: /api/v1/llm/evals/suites/{suite_id}/error-analyses: post: tags: - Evals summary: Start an error analysis of a leaderboard row description: >- Picks the failing samples of one model on one suite version (100 by default, `sample_size` up to 200), sends them (with their audio on speech boards) to the analyzer in two stages and stores a report of error categories, weak tasks, judge and benchmark issues and actionable items. The job runs in the background: poll the returned id until `status` is `done` or `failed`. A finished analysis of the same samples, metric and prompt version is returned as is unless `force` is true; so is an analysis that is still running. Owner org only; shared orgs read reports but cannot start one. Returns 400 for embedding and coding-agent boards, for a row with no failures, and when the analyzer is not configured on the cluster. operationId: createEvalErrorAnalysis parameters: - name: suite_id in: path required: true schema: type: string example: eval_suite_aa87e2b1112a455b8deabed784372198 - name: X-On-Behalf-Of in: header required: false schema: type: string description: Optional external end-user identifier forwarded by the API gateway. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateEvalErrorAnalysisRequest' example: model: meetkai:functionary-urdu-mini-pak metric_id: score responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/EvalErrorAnalysisObject' example: id: eval_ea_aa87e2b1112a455b8deabed784372198 object: eval.error_analysis suite_id: eval_suite_aa87e2b1112a455b8deabed784372198 suite_version: 3 model: meetkai:functionary-urdu-mini-pak metric_id: score kind: text status: queued analyzer_model: gemini-4-argon prompt_version: 2026-11-04.1 cache_key: 3b2c… selection: failures_total: 146 selected: 100 tiered: true samples_analysed: 100 usage: null error: null created_by: user_123 created_at: 1762243200 started_at: null finished_at: null sample_ids: [] tags: [] report: null report_markdown: null security: - bearerAuth: [] x-codeSamples: - lang: csharp label: CSharp (SDK) source: |- using MeetKai.MKA1; using MeetKai.MKA1.Types.Components; var sdk = new SDK(bearerAuth: ""); var res = await sdk.Llm.Evals.CreateErrorAnalysisAsync( suiteId: "eval_suite_aa87e2b1112a455b8deabed784372198", body: new MeetKai.MKA1.Types.Components.CreateEvalErrorAnalysisRequest() { Model = "meetkai:functionary-urdu-mini-pak", MetricId = "score", } ); // handle response - lang: python label: Python (SDK) source: |- from meetkai_mka1 import SDK with SDK( bearer_auth="", ) as sdk: res = sdk.llm.evals.create_error_analysis(suite_id="eval_suite_aa87e2b1112a455b8deabed784372198", model="meetkai:functionary-urdu-mini-pak", metric_id="score") # Handle response print(res) - lang: typescript label: Typescript (SDK) source: |- import { SDK } from "@meetkai/mka1"; const sdk = new SDK({ bearerAuth: "", }); async function run() { const result = await sdk.llm.evals.createErrorAnalysis({ suiteId: "eval_suite_aa87e2b1112a455b8deabed784372198", createEvalErrorAnalysisRequest: { model: "meetkai:functionary-urdu-mini-pak", metricId: "score", }, }); console.log(result); } run(); - lang: go label: Go (SDK) source: "package main\n\nimport(\n\t\"context\"\n\tmka1 \"github.com/MeetKai/mka1-sdks/go\"\n\t\"github.com/MeetKai/mka1-sdks/go/models/components\"\n\t\"log\"\n)\n\nfunc main() {\n ctx := context.Background()\n\n s := mka1.New(\n mka1.WithSecurity(\"\"),\n )\n\n res, err := s.Llm.Evals.CreateErrorAnalysis(ctx, \"eval_suite_aa87e2b1112a455b8deabed784372198\", components.CreateEvalErrorAnalysisRequest{\n Model: \"meetkai:functionary-urdu-mini-pak\",\n MetricID: mka1.Pointer(\"score\"),\n }, nil)\n if err != nil {\n log.Fatal(err)\n }\n if res.EvalErrorAnalysisObject != nil {\n // handle response\n }\n}" components: schemas: CreateEvalErrorAnalysisRequest: type: object properties: model: type: string minLength: 1 maxLength: 255 description: >- The leaderboard row to analyse: a model id exactly as it appears on the board. suite_version: type: integer minimum: 1 maximum: 9007199254740991 description: >- Suite version whose runs are analysed. Defaults to the suite's active version. metric_id: type: string minLength: 1 maxLength: 128 description: >- Headline metric that defines a failure (e.g. `wer`, `scribe_v2_wer`, `score`). Defaults to the board's default metric. failure_threshold: type: number description: >- Overrides the failure threshold: a sample fails when a lower-is-better metric is at or above it, or a higher-is-better metric is below it. Defaults: 0.1 and 0.5. sample_size: type: integer minimum: 10 maximum: 200 description: How many failures to analyse. Default 100. force: type: boolean description: >- Start a new analysis even when a finished one exists for the same samples, metric and prompt version. analyzer_model: type: string minLength: 1 maxLength: 128 description: >- Analyzer model to use, one of `GET /evals/error-analysis/analyzers`. Defaults to the cluster's default analyzer. required: - model additionalProperties: false EvalErrorAnalysisObject: type: object properties: id: type: string object: const: eval.error_analysis suite_id: type: string suite_version: type: integer minimum: -9007199254740991 maximum: 9007199254740991 model: type: string metric_id: type: string kind: $ref: '#/components/schemas/EvalErrorAnalysisKind' status: $ref: '#/components/schemas/EvalErrorAnalysisStatus' analyzer_model: type: string prompt_version: type: string cache_key: type: string selection: type: object propertyNames: type: string additionalProperties: {} description: >- Counts behind the sample: failures in total, per task and per tier, the threshold, the board context. samples_analysed: type: integer minimum: -9007199254740991 maximum: 9007199254740991 usage: anyOf: - $ref: '#/components/schemas/EvalErrorAnalysisUsage' - type: 'null' error: anyOf: - type: string - type: 'null' created_by: anyOf: - type: string - type: 'null' description: >- The user id that started it; null for a reader in a shared org, which does not see the owner org's users. created_at: type: integer minimum: -9007199254740991 maximum: 9007199254740991 started_at: anyOf: - type: integer minimum: -9007199254740991 maximum: 9007199254740991 - type: 'null' finished_at: anyOf: - type: integer minimum: -9007199254740991 maximum: 9007199254740991 - type: 'null' sample_ids: type: array items: $ref: '#/components/schemas/EvalErrorAnalysisSampleRef' tags: type: array items: $ref: '#/components/schemas/EvalErrorAnalysisTag' report: anyOf: - $ref: '#/components/schemas/EvalErrorAnalysisReport' - type: 'null' report_markdown: anyOf: - type: string - type: 'null' description: The report rendered as Markdown; null until the analysis is done. required: - id - object - suite_id - suite_version - model - metric_id - kind - status - analyzer_model - prompt_version - cache_key - selection - samples_analysed - usage - error - created_by - created_at - started_at - finished_at - sample_ids - tags - report - report_markdown EvalErrorAnalysisKind: enum: - tts - asr - text type: string EvalErrorAnalysisStatus: enum: - queued - running - done - failed type: string EvalErrorAnalysisUsage: type: object properties: analyzer_model: type: string rate_card_usd_per_1m: type: object properties: input: type: number output: type: number required: - input - output calls: type: integer minimum: -9007199254740991 maximum: 9007199254740991 prompt_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 prompt_text_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 prompt_audio_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 output_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 thinking_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 seconds: type: number usd_input: type: number usd_output: type: number usd_total: type: number per_stage: type: object propertyNames: type: string additionalProperties: type: object properties: calls: type: integer minimum: -9007199254740991 maximum: 9007199254740991 prompt_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 output_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 thinking_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 seconds: type: number usd: type: number required: - calls - prompt_tokens - output_tokens - thinking_tokens - seconds - usd calls_detail: type: array items: $ref: '#/components/schemas/EvalErrorAnalysisUsageCall' dropped_ids: type: integer minimum: -9007199254740991 maximum: 9007199254740991 description: Cited sample ids the analyzer invented; removed before storing. report_attempts: type: integer minimum: -9007199254740991 maximum: 9007199254740991 description: Stage-2 calls made; more than 1 means a leaked answer was retried. samples_without_audio: type: integer minimum: -9007199254740991 maximum: 9007199254740991 description: >- Audio-board samples sent as text only because their clip could not be read. required: - analyzer_model - rate_card_usd_per_1m - calls - prompt_tokens - prompt_text_tokens - prompt_audio_tokens - output_tokens - thinking_tokens - seconds - usd_input - usd_output - usd_total - per_stage - calls_detail - dropped_ids - report_attempts - samples_without_audio EvalErrorAnalysisSampleRef: type: object properties: id: type: string description: Citation id used in the tags and the report. run_id: type: string task_id: type: string sample_index: type: integer minimum: -9007199254740991 maximum: 9007199254740991 tier: enum: - A - B - '-' type: string metric_value: type: number secondary_value: anyOf: - type: number - type: 'null' board_median: anyOf: - type: number - type: 'null' others_failing: type: integer minimum: -9007199254740991 maximum: 9007199254740991 others_total: type: integer minimum: -9007199254740991 maximum: 9007199254740991 excess_over_board: anyOf: - type: number - type: 'null' both_judges_fail: type: boolean required: - id - run_id - task_id - sample_index - tier - metric_value - secondary_value - board_median - others_failing - others_total - excess_over_board - both_judges_fail EvalErrorAnalysisTag: type: object properties: id: type: string description: Citation id used in the tags and the report. run_id: type: string task_id: type: string sample_index: type: integer minimum: -9007199254740991 maximum: 9007199254740991 tier: enum: - A - B - '-' type: string metric_value: type: number secondary_value: anyOf: - type: number - type: 'null' board_median: anyOf: - type: number - type: 'null' others_failing: type: integer minimum: -9007199254740991 maximum: 9007199254740991 others_total: type: integer minimum: -9007199254740991 maximum: 9007199254740991 excess_over_board: anyOf: - type: number - type: 'null' both_judges_fail: type: boolean heard_text: type: string description: >- What the analyzer heard (audio boards) or a one-line restatement of what the model did (text boards). verdict: enum: - model_error - judge_error - reference_issue - mixed - untagged type: string categories: type: array items: type: string primary_category: type: string severity: enum: - minor - major - severe type: string affected_span: type: string explanation: type: string batch: type: string required: - id - run_id - task_id - sample_index - tier - metric_value - secondary_value - board_median - others_failing - others_total - excess_over_board - both_judges_fail - heard_text - verdict - categories - primary_category - severity - affected_span - explanation - batch EvalErrorAnalysisReport: type: object properties: headline: type: string overall_assessment: type: string error_categories: type: array items: type: object properties: category: type: string count: type: integer minimum: -9007199254740991 maximum: 9007199254740991 share_of_analysed_failures: type: string severity: enum: - minor - major - severe type: string what_happens: type: string example_ids: type: array items: type: string required: - category - count - share_of_analysed_failures - severity - what_happens - example_ids weak_tasks: type: array items: type: object properties: task: type: string why: type: string example_ids: type: array items: type: string required: - task - why - example_ids judge_and_benchmark_issues: type: object properties: judge_error_count: type: integer minimum: -9007199254740991 maximum: 9007199254740991 reference_issue_count: type: integer minimum: -9007199254740991 maximum: 9007199254740991 note: type: string example_ids: type: array items: type: string required: - judge_error_count - reference_issue_count - note - example_ids actionable_items: type: array items: type: object properties: priority: type: integer minimum: -9007199254740991 maximum: 9007199254740991 title: type: string action_type: enum: - text_frontend - serving_config - decoding - training_data - model_architecture - judge_or_metric - benchmark_fix type: string what_to_do: type: string why_it_helps: type: string addresses_categories: type: array items: type: string estimated_share_of_failures_fixed: type: string evidence_ids: type: array items: type: string confidence: enum: - low - medium - high type: string effort: enum: - hours - days - weeks type: string required: - priority - title - action_type - what_to_do - why_it_helps - addresses_categories - estimated_share_of_failures_fixed - evidence_ids - confidence - effort strengths: type: array items: type: string limits_of_this_analysis: type: array items: type: string required: - headline - overall_assessment - error_categories - weak_tasks - judge_and_benchmark_issues - actionable_items - strengths - limits_of_this_analysis EvalErrorAnalysisUsageCall: type: object properties: stage: enum: - tag - report type: string batch: anyOf: - type: string - type: 'null' prompt_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 prompt_text_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 prompt_audio_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 output_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 thinking_tokens: type: integer minimum: -9007199254740991 maximum: 9007199254740991 seconds: type: number required: - stage - batch - prompt_tokens - prompt_text_tokens - prompt_audio_tokens - output_tokens - thinking_tokens - seconds securitySchemes: bearerAuth: type: http scheme: bearer bearerFormat: API Key description: >- Send Authorization: Bearer , using an MKA1 API key for automation or an OAuth API access token for a signed-in application. Use a dynamic security callback to refresh OAuth access tokens. Browser session-only operations still require their declared cookie. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.