Skip to main content
Eval Labs data is designed to preserve prompt context, run source, Lucia response, layered review judgment, lifecycle state, tester identity, and—on newly executed runs—captured Lucia runtime/model provenance.

​
Session metadata

A session export includes metadata such as:
The important field for the custom launcher is:
Custom and Auto-generated launchers both create run sessions that flow into the same Review Queue. The session mode remains the run mechanics; runSource distinguishes whether the run came from the Custom launcher or the Auto-generated launcher. Controlled batch readiness runs also rely on the same run/session lifecycle, but their product meaning is different: they are platform readiness evidence, not normal evaluator-facing tests. Registry Diagnostics reads existing session/run/review evidence and derives dataset membership and review-lane suggestions from it. Those suggestions are diagnostic output, not saved labels. Behavioral Observatory labels are separate persisted records when saved through the Behavioral Observatory flow.

​
Cases

Each prompt becomes a case. A case contains:
Order matters. The exported orderIndex must remain stable.

​
Prompt results

Prompt results include:
  • draft review state, including reviewer input and suggestions
  • saved review state
  • saved timestamp
  • saved-by tester identity
  • completion/dirty state derived from saved vs draft review
A generated-but-unreviewed item may have null ratings and savedBy: null. That is expected. A saved review should include savedBy.

​
Runtime and model provenance

Eval Labs production targets Lucia Engine Development at:
The Engine returns runtime evidence in the same /admin/operator-focus response as the evaluated answer. Eval Labs captures that evidence as runtimeProvenance for new runs, persists it on the case, and mirrors the latest captured value in the session metadata. The JSON session export contains both records, so the full captured evidence travels with JSON exports. Current CSV and Markdown exports do not include the full provenance object. The current stored object uses these exact fields:
Each model_tasks item may contain:
These are persisted schema keys, not merely suggested export labels. The owner-only Lucia Runtime Verification dashboard presents the corresponding resolved endpoint, classified and Engine environments, Engine commit SHA, configured model, provider-resolved model, model-invoked state, deterministic-path state, fallback state, model-task evidence, and verification timestamp. configured_model records the Engine configuration. resolved_model records the provider-resolved model captured from the Engine response. A configured model alone does not prove that a model ran. model_invoked and deterministic_path are complements: they record whether any model task was attempted. fallback_used is independent and may be true with either state. Task evidence separately proves whether a provider response arrived, parsing succeeded, and a resolved model was captured. Live verification on 2026-07-10 produced this evidence:
Historical records are not backfilled. When a run predates provenance capture or otherwise has no stored runtimeProvenance, Eval Labs states: “Runtime provenance was not captured for this run.” Do not infer an endpoint, environment, Engine commit, model, or invocation path for that run.

​
Confirmed routing drift

Configuration and deployment evidence establishes that the confirmed uninterrupted Eval Labs production routing drift toward Staging began at 2026-05-04T16:21:35Z. Routing was corrected to Development and live-verified on 2026-07-10. That interval is a routing-history fact, not proof that every individual historical run used Staging. Only per-run captured provenance can establish the runtime used by a specific run. Runs without it remain explicitly unverified rather than being rewritten.

​
exportedBy vs savedBy

exportedBy identifies who exported the file. savedBy identifies who reviewed/saved the individual prompt. This distinction matters because one person may export a run that another person reviewed.

​
Tester identity fields

Eval Labs stores only limited identity fields:
No unnecessary Clerk metadata should be stored. Role gating reads Clerk public metadata through eval_labs_role, but role metadata is product access state, not review authorship.

​
Nulls in exports

Some nulls are normal. Expected nulls include:
Do not treat every null as a bug. Treat nulls as suspicious only when the workflow step should have populated them.

​
Export Options and Example

export-controls Export controls for easy usability with multiple data formats. The compact example below focuses on the core session/review shape. Current JSON session exports carry runtimeProvenance at session and case level when it was captured; the exact current keys are defined above. Current CSV and Markdown exports omit the full object, and historical JSON exports may omit it.

​
Review-layer fields

Prompt reviews now support these additional fields:
The suggested fields are product suggestions, not final reviewer judgment. They are generated from prompt/response/run-status heuristics and remain separate from the reviewer-saved values. Suggested fields may feed derived context in Registry Diagnostics or Behavioral Observatory, but they do not become persisted Behavioral Observatory labels unless a reviewer saves a label in the Behavioral Observatory surface.

​
Employee Review object

Employee Review captures guided non-expert signal:
These fields are intentionally simple and should remain employee-friendly.

​
Human Guidance Evaluation object

Human Guidance Evaluation captures a 1-5 review layer:
Warmth and intelligence are not separate export fields. They are expressed through the current scoring dimensions and guidance fields: tone, calming, naturalness, trust, usefulness, cognitiveUnderstanding, actionability, and authenticity.

​
Adjudication object

Adjudication captures final senior-review meaning when it exists in the review record:
Final labels may include:

​
Review lifecycle object

Run lifecycle finalization is stored at the session level:
Finalization does not replace per-prompt review data. It marks the run lifecycle after all prompts are reviewed.

​
Supabase persistence contract

The Supabase persistence layer stores the run and item contract in three places:
eval_run_items.payload.promptRecord embeds the full prompt review record, including saved/draft state, suggested review, employee review, Human Guidance Evaluation, adjudication metadata, canon candidate signal, tester identity, and dirty/completion state. eval_item_reviews is still written for review rows, but hydration prefers the embedded eval_run_items.payload.promptRecord instead of relying on fragile review-table reads. Behavioral Observatory labels are stored separately:
This table stores first-class Behavioral Observatory labels:
The key distinction:
One saved Behavioral Observatory label exists per reviewer per run item. Current persisted run evidence is scoped by the signed-in Clerk user and the role claim available to Supabase RLS. Owner/admin can inspect shared persisted evidence where privileged RLS allows it. Evaluator and tester data remains scoped to their own work except where owner/admin oversight applies. Current readiness verification checks counts across:
For the 60-run readiness gate, the final verified result was:
Meaning:
  • 60 ready runs
  • 3,000 expected prompts
  • 3,000 run items
  • 3,000 non-empty Lucia responses
  • 3,000 reviews for the tested reviewer id

​
localStorage compaction contract

Completed cloud-backed runs should not leave full item-level payloads persisted in localStorage. The platform-readiness diagnostic target is:
The final verified 60-run diagnostic was:
This supports platform readiness and client compactness. It does not prove backend authorization is complete.

​
Export rule

Exports should preserve the full review contract:
Do not collapse these into one field.