> ## Documentation Index > Fetch the complete documentation index at: https://docs.mka1.com/llms.txt > Use this file to discover all available pages before exploring further. # Create an eval schedule > Schedules an eval run to fire on a cron or interval cadence, with optional webhook notifications of run status. ## OpenAPI ````yaml https://apigw.mka1.com/speakeasy.json post /api/v1/llm/evals/schedules openapi: 3.1.1 info: title: MKA1 API version: 1.1.0 description: >- The MKA1 API is a RESTful API that provides access to the MKA1 platform. Learn how to get started with the API and the TypeScript SDK [here](https://mka1.apidocumentation.com/guides/getting-started). license: name: Proprietary servers: - url: https://apigw.mka1.com description: MKA1 API Gateway - url: / description: Relative server URL (configurable via SDK constructor) security: [] tags: - name: Resource Authorization description: >- Manage permissions for LLM resources. Create resources, grant/revoke permissions, and delete resources. Only resource owners can grant, revoke, or delete permissions. x-displayName: Resource Authorization - name: Embeddings description: >- Text embedding API endpoints for generating vector representations of text. Create semantic embeddings for search, clustering, and similarity matching using various embedding models. x-displayName: Embeddings - name: Feedback description: >- User feedback API for rating and commenting on chat completions. Collect thumbs up/down ratings and detailed feedback to improve model responses and track user satisfaction. x-displayName: Feedback - name: Images description: >- Image generation API endpoints for creating images from text descriptions. Generate images with control over size, quality, and style. x-displayName: Images - name: MCP Vault description: >- MCP vault API for storing user-owned MCP server configurations and encrypted credentials. Agents reference vault IDs so secrets are resolved only at tool execution time. x-displayName: MCP Vault - name: Speech description: >- Speech API endpoints for audio processing. Convert text to natural-sounding speech (TTS) or transcribe speech to text (STT) in different languages. x-displayName: Speech - name: Usage description: >- Usage tracking and analytics API for monitoring token consumption, request counts, and cost analysis. View detailed statistics per user, model, and time period. x-displayName: Usage - name: Extract description: >- Structured data extraction API for extracting information from files. Define JSON schemas to extract structured data from images, PDFs, and documents. Supports reusable schema templates. x-displayName: Extract - name: Text Classification description: >- Text classification API for categorizing text into predefined labels. Use AI models to classify text content for sentiment analysis, topic categorization, and content moderation. x-displayName: Text Classification - name: Responses description: >- Agent-powered responses API for creating AI agents with autonomous tool usage. Build conversational assistants that can use web search, file operations, image generation, code execution, computer use simulation, and MCP integrations. Supports background processing, streaming, and real-time status tracking. x-displayName: Responses - name: Files description: >- File management API for uploading, storing, and managing files with automatic expiration and S3 integration. Upload files that can be used with Assistants, Vector Stores, and other features. Files are stored in S3 with metadata tracked in PostgreSQL. Supports automatic cleanup of expired files. x-displayName: Files - name: Vector Stores description: >- Vector store API for storing and searching documents using embeddings. Create vector stores, upload files with automatic chunking and embedding generation, and perform semantic search. Files are processed asynchronously using Temporal workflows for durability. Supports automatic cleanup of expired stores and LanceDB for efficient vector storage. x-displayName: Vector Stores - name: Conversations description: >- Conversation management API for storing and retrieving conversation state across Response API calls. Create conversations, add items (user messages, assistant messages, system messages), and maintain conversation history. Supports metadata tracking and multi-turn dialogue state management. x-displayName: Conversations - name: Guardrails description: >- AI safety guardrails API for configuring content moderation and security policies. Set up ban word lists, prompt injection detection, and system prompt leakage prevention. Guardrails apply to all requests from an account and can be tested before deployment. x-displayName: Guardrails - name: Models description: >- Model listing API for discovering available models. Returns model IDs, ownership, and metadata for all registered models in the gateway. x-displayName: Models - name: Skills description: >- Skills API for managing versioned bundles of instructions and files following the Agent Skills standard. Create, version, and download reusable skill packages that include SKILL.md manifests for agent environments. x-displayName: Skills - name: Chat Completions description: >- **Deprecated: Use the Responses API (`/api/v1/llm/responses`) instead.** Chat completion endpoints with support for streaming, tool calls, and multiple providers. x-deprecated: true x-displayName: Chat Completions - name: Batches x-displayName: Batches - name: Evals x-displayName: Evals - name: Fine-Tuning x-displayName: Fine-Tuning - name: Memory Stores x-displayName: Memory Stores - name: Prompts x-displayName: Prompts - name: Tables description: Manage table schemas, data operations, search, and indices. x-displayName: Tables - name: Text Store description: >- Manage text stores with hybrid (vector + full-text) search and grouped text sets. x-displayName: Text Store - name: GraphRAG description: >- Construct and query lightweight knowledge graphs backed by Redis and LanceDB. x-displayName: GraphRAG - name: API Key x-displayName: API Key - name: Sessions description: Create, inspect, access, and terminate sandbox sessions. x-displayName: Sessions - name: Browser description: >- Connect to browser sessions through the gateway port proxy. Browser sessions expose a Chrome DevTools Protocol endpoint on port 9222. x-displayName: Browser - name: Execution description: Run shell commands and code inside an existing sandbox session. x-displayName: Execution - name: Workspace description: >- Inspect the workspace manifest, transfer files or archives, and download generated artifacts. x-displayName: Workspace - name: Sandbox Usage x-displayName: Sandbox Usage - name: Agents description: Create and manage reusable agent definitions. x-displayName: Agents - name: Agent Versions description: Inspect an agent's configuration history and roll back to a prior version. x-displayName: Agent Versions - name: Agent Runs description: Execute saved agents and inspect persisted run results. x-displayName: Agent Runs - name: Agent Schedules description: Create and manage scheduled or recurring saved agent runs. x-displayName: Agent Schedules - name: schema-5_other x-displayName: other paths: /api/v1/llm/evals/schedules: post: tags: - Evals summary: Create an eval schedule description: >- Schedules an eval run to fire on a cron or interval cadence, with optional webhook notifications of run status. operationId: createEvalSchedule parameters: - name: X-On-Behalf-Of in: header required: false schema: type: string description: Optional external end-user identifier forwarded by the API gateway. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateEvalScheduleRequest' example: name: Nightly MMLU regression suite_id: eval_suite_aa87e2b1112a455b8deabed784372198 cron: 0 3 * * * timezone: US/Pacific webhook_url: https://example.com/hooks/evals webhook_secret: whsec_example_signing_secret overlap_policy: skip run: models: - meetkai/model-a metadata: team: eval-ops responses: '200': description: OK content: application/json: schema: type: object properties: id: type: string object: const: eval.schedule name: type: string description: anyOf: - type: string - type: 'null' suite_id: type: string suite_version: anyOf: - type: integer minimum: -9007199254740991 maximum: 9007199254740991 - type: 'null' cron: anyOf: - type: string - type: 'null' interval_seconds: anyOf: - type: integer minimum: -9007199254740991 maximum: 9007199254740991 - type: 'null' timezone: type: string overlap_policy: $ref: '#/components/schemas/EvalScheduleOverlapPolicy' enabled: type: boolean webhook_url: anyOf: - type: string - type: 'null' webhook_configured: type: boolean description: >- True when a signing secret is stored. The secret itself is never returned. run: type: object properties: models: type: array minItems: 1 maxItems: 20 items: type: string minLength: 1 task_ids: type: array minItems: 1 items: type: string judge_model: type: string embedding_model: type: string generation: type: object properties: instructions: type: string temperature: type: number minimum: 0 maximum: 2 top_p: type: number minimum: 0 maximum: 1 max_output_tokens: type: integer minimum: 1 maximum: 9007199254740991 max_gen_toks: type: integer minimum: 1 maximum: 9007199254740991 description: lm-eval alias for max_output_tokens. stop: anyOf: - type: string - type: array minItems: 1 items: type: string until: type: array minItems: 1 items: type: string description: lm-eval generate_until stop sequences. max_tool_calls: type: integer minimum: 1 maximum: 9007199254740991 reasoning: {} text: {} tools: type: array items: anyOf: - {} - type: 'null' tool_choice: {} parallel_tool_calls: type: boolean truncation: enum: - auto - disabled service_tier: enum: - auto - default - flex - priority presence_penalty: type: number minimum: -2 maximum: 2 frequency_penalty: type: number minimum: -2 maximum: 2 top_k: type: integer minimum: 0 maximum: 9007199254740991 min_p: type: number minimum: 0 maximum: 1 repetition_penalty: type: number minimum: 0 do_sample: type: boolean extra_body: type: object propertyNames: type: string additionalProperties: {} chat_template_kwargs: type: object propertyNames: type: string additionalProperties: {} prefill_think: anyOf: - type: boolean - type: string use_cache: type: boolean timeout_seconds: type: integer minimum: 1 maximum: 3600 max_retries: type: integer minimum: 0 maximum: 10 max_empty_retries: type: integer minimum: 0 maximum: 10 additionalProperties: {} default: {} generation_concurrency: type: integer minimum: 1 maximum: 256 concurrency: type: integer minimum: 1 maximum: 256 grader_concurrency: type: integer minimum: 1 maximum: 256 max_samples_per_task: type: integer minimum: 1 maximum: 9007199254740991 max_workflow_sample_activities: type: integer minimum: 100 maximum: 50000 required: - models metadata: type: object propertyNames: type: string maxLength: 64 additionalProperties: type: string maxLength: 512 last_run_id: anyOf: - type: string - type: 'null' last_run_at: anyOf: - type: integer minimum: -9007199254740991 maximum: 9007199254740991 - type: 'null' last_run_status: anyOf: - type: string - type: 'null' next_run_at: anyOf: - type: integer minimum: -9007199254740991 maximum: 9007199254740991 - type: 'null' description: Next scheduled fire time from Temporal, when resolvable. created_at: type: integer minimum: -9007199254740991 maximum: 9007199254740991 updated_at: type: integer minimum: -9007199254740991 maximum: 9007199254740991 required: - id - object - name - description - suite_id - suite_version - cron - interval_seconds - timezone - overlap_policy - enabled - webhook_url - webhook_configured - run - metadata - last_run_id - last_run_at - last_run_status - next_run_at - created_at - updated_at example: id: eval_sched_aa87e2b1112a455b8deabed784372198 object: eval.schedule name: Nightly MMLU regression description: null suite_id: eval_suite_aa87e2b1112a455b8deabed784372198 suite_version: null cron: 0 3 * * * interval_seconds: null timezone: US/Pacific overlap_policy: skip enabled: true webhook_url: https://example.com/hooks/evals webhook_configured: true run: models: - meetkai/model-a generation: {} metadata: team: eval-ops last_run_id: null last_run_at: null last_run_status: null next_run_at: 1751328000 created_at: 1751241600 updated_at: 1751241600 security: - bearerAuth: [] x-codeSamples: - lang: python label: Python (SDK) source: |- from meetkai_mka1 import SDK, models with SDK( bearer_auth="", ) as sdk: res = sdk.llm.evals.create_schedule(name="Nightly MMLU regression", suite_id="eval_suite_aa87e2b1112a455b8deabed784372198", run=models.EvalScheduleRunConfig( models=[ "meetkai/model-a", ], ), cron="0 3 * * *", timezone="US/Pacific", overlap_policy="skip", enabled=True, webhook_url="https://example.com/hooks/evals", webhook_secret="whsec_example_signing_secret", metadata={ "team": "eval-ops", }) # Handle response print(res) - lang: typescript label: Typescript (SDK) source: |- import { SDK } from "@meetkai/mka1"; const sdk = new SDK({ bearerAuth: "", }); async function run() { const result = await sdk.llm.evals.createSchedule({ createEvalScheduleRequest: { name: "Nightly MMLU regression", suiteId: "eval_suite_aa87e2b1112a455b8deabed784372198", cron: "0 3 * * *", timezone: "US/Pacific", overlapPolicy: "skip", webhookUrl: "https://example.com/hooks/evals", webhookSecret: "whsec_example_signing_secret", run: { models: [ "meetkai/model-a", ], }, metadata: { "team": "eval-ops", }, }, }); console.log(result); } run(); - lang: csharp label: CSharp (SDK) source: >- using MeetKai.MKA1; using MeetKai.MKA1.Types.Components; using System.Collections.Generic; var sdk = new SDK(bearerAuth: ""); var res = await sdk.Llm.Evals.CreateScheduleAsync(body: new MeetKai.MKA1.Types.Components.CreateEvalScheduleRequest() { Name = "Nightly MMLU regression", SuiteId = "eval_suite_aa87e2b1112a455b8deabed784372198", Cron = "0 3 * * *", Timezone = "US/Pacific", OverlapPolicy = EvalScheduleOverlapPolicy.Skip, WebhookUrl = "https://example.com/hooks/evals", WebhookSecret = "whsec_example_signing_secret", Run = new EvalScheduleRunConfig() { Models = new List() { "meetkai/model-a", }, }, Metadata = new Dictionary() { { "team", "eval-ops" }, }, }); // handle response components: schemas: CreateEvalScheduleRequest: type: object properties: name: type: string minLength: 1 maxLength: 255 description: type: string maxLength: 10000 suite_id: type: string suite_version: type: integer minimum: 1 maximum: 9007199254740991 description: >- Pin a suite version. Omit to resolve the active version on each fire. cron: type: string minLength: 1 maxLength: 255 description: >- 5-field cron expression. Provide exactly one of cron or interval_seconds. interval_seconds: type: integer minimum: 60 maximum: 31536000 description: >- Fixed interval between fires, in seconds. Provide exactly one of cron or interval_seconds. timezone: type: string minLength: 1 maxLength: 64 default: UTC description: IANA timezone applied to cron expressions, for example US/Pacific. overlap_policy: $ref: '#/components/schemas/EvalScheduleOverlapPolicy' default: skip enabled: type: boolean default: true description: When false, the schedule is created paused. webhook_url: type: string maxLength: 2048 format: uri description: HTTPS endpoint notified of run status transitions. webhook_secret: type: string minLength: 16 maxLength: 256 description: Secret used to HMAC-SHA256 sign webhook payloads. run: $ref: '#/components/schemas/EvalScheduleRunConfig' metadata: type: object propertyNames: type: string maxLength: 64 additionalProperties: type: string maxLength: 512 default: {} required: - name - suite_id - run EvalScheduleOverlapPolicy: enum: - skip - buffer_one - allow_all EvalScheduleRunConfig: type: object properties: models: type: array minItems: 1 maxItems: 20 items: type: string minLength: 1 task_ids: type: array minItems: 1 items: type: string judge_model: type: string embedding_model: type: string generation: $ref: '#/components/schemas/EvalGenerationConfig' default: {} generation_concurrency: type: integer minimum: 1 maximum: 256 concurrency: type: integer minimum: 1 maximum: 256 grader_concurrency: type: integer minimum: 1 maximum: 256 max_samples_per_task: type: integer minimum: 1 maximum: 9007199254740991 max_workflow_sample_activities: type: integer minimum: 100 maximum: 50000 required: - models EvalGenerationConfig: type: object properties: instructions: type: string temperature: type: number minimum: 0 maximum: 2 top_p: type: number minimum: 0 maximum: 1 max_output_tokens: type: integer minimum: 1 maximum: 9007199254740991 max_gen_toks: type: integer minimum: 1 maximum: 9007199254740991 description: lm-eval alias for max_output_tokens. stop: anyOf: - type: string - type: array minItems: 1 items: type: string until: type: array minItems: 1 items: type: string description: lm-eval generate_until stop sequences. max_tool_calls: type: integer minimum: 1 maximum: 9007199254740991 reasoning: {} text: {} tools: type: array items: {} tool_choice: {} parallel_tool_calls: type: boolean truncation: enum: - auto - disabled service_tier: enum: - auto - default - flex - priority presence_penalty: type: number minimum: -2 maximum: 2 frequency_penalty: type: number minimum: -2 maximum: 2 top_k: type: integer minimum: 0 maximum: 9007199254740991 min_p: type: number minimum: 0 maximum: 1 repetition_penalty: type: number minimum: 0 do_sample: type: boolean extra_body: type: object propertyNames: type: string additionalProperties: {} chat_template_kwargs: type: object propertyNames: type: string additionalProperties: {} prefill_think: anyOf: - type: boolean - type: string use_cache: type: boolean timeout_seconds: type: integer minimum: 1 maximum: 3600 max_retries: type: integer minimum: 0 maximum: 10 max_empty_retries: type: integer minimum: 0 maximum: 10 additionalProperties: {} securitySchemes: bearerAuth: type: http scheme: bearer bearerFormat: API Key description: >- Gateway auth: send `Authorization: Bearer `. For multi-user server-side integrations, you can also send `X-On-Behalf-Of: `. ````