Every generation runs as a Compute job under your organization and publishes under a short-lived credential the platform mints for that run, so the hardware and any LLM calls the generator makes bill to you like any other platform usage.
Before you start
You need:
Everything you create belongs to your organization.
A generation from another organization is
404, never 403.
The examples export the key once:
MKA1_BEARER_AUTH="Bearer <mka1-api-key>" is exported; see authenticate the CLI.
Dataset formats
A dataset is a repository withtrain.jsonl at its root and, optionally, test.jsonl and validation.jsonl: UTF-8, one JSON object per line.
Alongside the rows, the platform writes a README.md with an mka1: front-matter block and a provenance.json that records the generator, its version, the exact parameters, row counts, and LLM usage.
Every row in a split has the same format, and the format is what a trainer declares it accepts.
All three formats share the OpenAI-style message object:
reasoning is an MKA1 extension for assistant turns that carries a thinking trace apart from content; trainers that do not understand it ignore it.
The catalog and each format’s row schema are served by the API, so a trainer can validate a dataset before spending GPU time:
row_schema (JSON Schema for one line), layout (required, optional, and platform-written files), semantics (the checks beyond schema, each with a code and a severity), and converters:
Generate a dataset
The walkthrough usesconstitution-prompts, which turns a written constitution into a prompt-only dataset for reinforcement learning: it extracts the rules, samples topics for each rule, and generates user prompts that target each rule, along with the judge templates that score an answer against them.
Every generator follows the same five steps; only the parameters change.
1. Discover a generator
List the available generators, then read one to get its parameter schema.params_schema (JSON Schema, draft 2020-12) and example_params, a small set of parameters the author guarantees to be valid:
version is a label the platform records on every generation and in the dataset’s provenance; you do not choose it.
default_limits are the ceilings a generation of this generator runs under, and inputs names any dataset repositories the generator can take as input.
A generator that calls an LLM exposes the model as a parameter.
model is the id you would send to /responses, and auto takes your organization’s default; the per-stage rules_model, topics_model, and prompts_model override it for one stage.
The walkthrough runs the author’s example_params, which is a twenty-prompt run; the defaults produce a 5,500-prompt dataset.
2. Quote the run
A quote is an advisory estimate of the LLM spend for a set of parameters. It reserves nothing and runs the same parameter validation as create, so it is also the cheapest way to check a request body.llm entry per model the generator will call, with the estimated calls and tokens, and the cost at your organization’s current prices.
Every money figure on this API is in currency.
A model that has no price for your organization gets estimated_cost: null and is listed in unpriced, as auto is above; when nothing in the estimate is priced, the total is null too, never 0.
An unpriced model does not block the create: POST /generations admits it.
The quote is where you see what was not priced; if you want a figure for your model, ask your organization admin to price it (see Manage models).
The priced figure is the generator author’s point estimate, not a bound.
What bounds a run is limits.max_llm_calls and limits.max_runtime_hours on the create, and your organization’s budgets, which cap the spend of every billable service.
Parameters that fail the schema return 400 invalid_params with the offending paths in details.errors:
3. Create the generation
Creating a generation needs anIdempotency-Key header: any string up to 255 characters that is unique within your organization.
Replaying the same key with the same body returns the original generation; the same key with a different body is 409 idempotency_conflict.
The response is
201 and the generation is pending:
created_by.api_key_id is the key you presented (null from a console session), delegation_id identifies the credential minted for the run, and external_user_id is the end user the request was made for when it carried X-On-Behalf-Of, so a run stays attributable to that user.
Creation is accepted, not started.
Everything that can go wrong once the job runs shows up as state, reason, and events on the generation, never as an HTTP error on the create.
Two responses are worth planning for:
duplicate_of. If a generation with the same parameters, generator version, and output repository has already succeeded, the201carries"duplicate_of": "gen_…". The new run still starts; whether to cancel it and reuse the earlier dataset is your call.409 repo_in_use. One repository holds at most one live generation. A second create against the sameoutput.repois refused until the first reachessucceeded,failed, orcancelled.
4. Watch it run
Poll the generation until its state is terminal.
On success the record carries what the run produced.
The example run took a minute and forty seconds from create to
succeeded:
params are exactly what the request sent; this run named its model explicitly, and a run created with auto records auto in params, llm.models, and usage.by_model.
counts reports every split the generator wrote plus dropped, the rows it generated and then discarded.
usage is the LLM traffic the run made, grouped by_model; the spend it represents is on your organization’s usage reports, rated at the price of each model.
When a run fails, reason.code is one of a fixed vocabulary.
The ones a caller can act on:
Two more endpoints show what happened inside the run.
Events are the structured timeline, in the order they were recorded, including every state change and each
stage the generator announced; logs are the container’s captured stdout and stderr, retained after the container is gone and capped at the last 20,000 lines.
stage, container_started) and events from the platform (state_changed) are recorded by different observers, so their timestamps can interleave.
Each log line has a timestamp, a stream, and a message:
cursor and limit (up to 1,000 per page). Use order=asc or order=desc to choose oldest-first (default) or newest-first ordering; stream=stdout or stream=stderr filters to one stream.
They are empty until the container has started.
5. Download the dataset
output.repo and output.commit on the succeeded generation name the exact revision that was published and verified.
Download it with the Hugging Face CLI pointed at the platform’s hub endpoint, pinning that commit:
train.jsonl and test.jsonl, a judge/ directory with the extracted rules and the system and user templates a reward model scores answers with, and an artifacts/ directory with the intermediate rules, topics, and full prompt set:
<org>/<name> reference is what you hand to a training workload: see Run a fine-tune job and Reinforcement learning.
provenance.json in the repository records the generator, version, parameters, usage, and row counts, so a dataset stays explainable after the generation record is out of sight.
List generations
Generations are retained forever. Filter withstate, generator_id, output_repo, created_after, and created_before; page with limit (default 20, at most 100) and cursor. Use order=desc (default) or order=asc to choose newest-first or oldest-first.
GET /generations/{id} returns it.
Cancel a generation
Cancel is idempotent and returns the generation, nowcancelling.
The container receives SIGTERM, the run’s credentials are revoked, and output.commit is never set.
A cancel that arrives once the run is already publishing is too late: the container has exited and published, and the run finishes on its read-back.
Cancelling a generation that is already terminal returns the record unchanged.
POST without a Content-Length header is 411 length_required, which --data '' avoids. The CLI and SDKs send it for you.
There is no delete.
Generations are records; the datasets they produced live in mka1-repos, and removing one is a repository delete.
Generator catalog
GET /api/v1/datasets/generators is the source of truth for what you can run.
New generators are added over time, and one that is still being rolled out answers 503 generator_unavailable on create until it is ready.
A generator that calls an LLM takes the model as a parameter, usually model with auto as the documented example; there is no model parameter on the API itself.
Each entry’s fields:
Errors
Every error is{"error": "<code>", "message": "<text>", "correlation_id": "<request id>"}, with details where noted.
Request bodies reject unknown properties.