Method

Workload library

Fixed, versioned workloads with frozen corpora, hashes and acceptance checks. A qualification result is only meaningful relative to a specific workload revision — a material change creates a new revision, never a silent update.

All v1 workloads are drafts: identity fields are defined, but corpora and acceptance suites are not yet frozen. Draft workloads are not used for public headline claims.

interactive-assistant-v1 Draft

Is single-user streaming chat responsive on this system?

Client-side TTFT, inter-token latency, end-to-end time at concurrency 1/2/4; format, language and repetition gates; cache-disabled baseline; 3 repeats.
team-serving-v1 Draft

What request rate does the system sustain within a stated SLO?

Concurrency sweep (closed-loop) plus frozen-trace open-loop mode; goodput vs throughput; failed requests stay in the denominator; output is an operating envelope.
long-context-v1 Draft

How deep is the useful context on this system — not the configurable one?

Depth ladder 8K/32K/64K/128K; RULER-derived tasks plus LongBench-style subset with answerable/unanswerable controls; distinguishes configured / loadable / completed / quality-preserving / recommended.
structured-agent-v1 Draft

Does JSON/function calling stay reliable under realistic agent traffic?

BFCL-style calls including relevance rejection, parallel and executable calls, loop detection; deterministic sandbox; parsing success alone is not task success.
endurance-30m-v1 Draft

Does a pre-qualified operating point survive 30 minutes of sustained load?

Time-series throughput, TTFT, memory growth, thermals and throttling at one operating cell; preregistered pass criteria; no universal pass.
rag-pipeline-ru-v0 Internal

Where does a Russian-language RAG pipeline actually lose quality, component by component?

Per-component evaluation: OCR CER/WER → splitter boundary F1 → Recall@k/nDCG → reranker delta → generation support/abstention/citations. Internal until 6 release gates pass.