Skip to main content
POST
/
api
/
v1
/
llm
/
evals
/
imports
/
huggingface-results
Python (SDK)
from meetkai_mka1 import SDK


with SDK(
    bearer_auth="<YOUR_BEARER_TOKEN_HERE>",
) as sdk:

    res = sdk.llm.evals.import_historical_results(x_hugging_face_token="hf_your_read_token", source={
        "path_prefix": "qwen/ur/",
    }, suite_name="MKA1 historical eval results", create_missing_tasks=True, dry_run=False, metadata={
        "owner": "eval-team",
    })

    # Handle response
    print(res)
{
  "object": "eval.import",
  "source": {
    "type": "huggingface",
    "repo": "meetkai/lm-eval-harness-result",
    "revision": "main"
  },
  "suite_id": "eval_suite_aa87e2b1112a455b8deabed784372198",
  "suite_version": 1,
  "created_suite": true,
  "created_suite_version": true,
  "dry_run": false,
  "discovered_files": 30,
  "imported_runs": 30,
  "skipped_runs": 0,
  "imported_samples": 8870
}

Authorizations

Authorization
string
header
required

Gateway auth: send Authorization: Bearer <mka1-api-key>. For multi-user server-side integrations, you can also send X-On-Behalf-Of: <external-user-id>.

Headers

X-HuggingFace-Token
string
required

Caller-owned Hugging Face token with read access to the source dataset repository. Gateway does not use its server-side Hugging Face token for historical imports.

X-On-Behalf-Of
string

Optional external end-user identifier forwarded by the API gateway.

Body

application/json
source
object
suite_id
string

Existing suite to import into. Missing tasks are appended as a new version by default.

suite_name
string
default:Historical lm-eval harness results
Required string length: 1 - 255
suite_description
string
Maximum string length: 10000
create_missing_tasks
boolean
default:true
dry_run
boolean
default:false
limit_files
integer

Maximum number of result files to import from Hugging Face, capped at 100. Repository listing imports default to 50; explicit source.files imports are not capped unless this is set.

Required range: 1 <= x <= 100
metadata
object

Response

200 - application/json

OK

object
any
required
source
object
required
suite_id
string
required
suite_version
integer
required
Required range: -9007199254740991 <= x <= 9007199254740991
created_suite
boolean
required
created_suite_version
boolean
required
dry_run
boolean
required
discovered_files
integer
required
Required range: 0 <= x <= 9007199254740991
imported_runs
integer
required
Required range: 0 <= x <= 9007199254740991
skipped_runs
integer
required
Required range: 0 <= x <= 9007199254740991
imported_samples
integer
required
Required range: 0 <= x <= 9007199254740991