Skip to main content
Eval Labs is a separate role-based human evaluation platform that tests Lucia through the deployed Engine, stores review evidence durably, and exposes scoped testing, run-history, review, Team Review, and Analysis surfaces.

​
High-level architecture

Employee / ReviewerEval Labs ProductionClerk authentication and authorized data scopeEngine Development or Guest Agent Production targetTarget-owned response and runtime evidenceEval Labs Review QueueSuggested selections plus human reviewLifecycle finalizationDurable or specialized evidence persistenceRun History / Team Review / Analysis / Exports
Release identity verified 2026-08-12: Eval Labs Production at https://evaluationlabs.ai serves commit 6cbfd15f75330c414bfe79e0d2fab51ef8115102, version 0.1.0, with same-deploy runtime identity evidence.

​
Runtime responsibility split

Eval Labs owns:
  • top app shell and route identity
  • test launcher UX
  • custom prompt suite UX
  • auto-generated prompt tester UX
  • guest-facing verification check and results UX
  • controlled batch runner UX
  • Fieldwork Communication Baseline orchestration
  • controlled Fieldwork Conversation v1 and v2 interfaces
  • run orchestration
  • owner-only Lucia Runtime Verification panel
  • owner-only Platform Runtime Registry
  • extraction, assessment, and persistence of Engine runtime/model provenance
  • Run History
  • Team Review
  • Analysis
  • Human Eval Research snapshots
  • Single Run Analysis
  • copy Session ID / copy Deep Link controls
  • role-gated product access
  • Clerk authentication and role-aware behavior
  • durable server/data authorization requirements
  • Review Queue
  • suggested review generation
  • human ratings
  • semantic scoring sliders
  • Quick Review
  • Human Guidance Evaluation
  • review lifecycle and finalization
  • dirty / completion state
  • tester identity capture
  • exports
  • durable persistence for eval data
  • staged hydration from run summaries to recent/deep evidence
  • compact browser state for completed cloud-backed runs
  • specialized Runtime Registry and Fieldwork claims, seals, receipts, review attestations, and recovery lineage
Lucia Engine owns:
  • actual Lucia behavior
  • intent/routing
  • response generation
  • emotional containment
  • operational prioritization
  • model gateway behavior
  • runtime identity and model-execution provenance emitted in /admin/operator-focus response metadata
Guest Agent owns:
  • booked-guest verification behavior at https://guest.hellolucia.ai
  • its direct model gateway and provider-returned model evidence
  • guest-facing production response behavior

​
Direct versus delegated model ownership

The target services own OpenAI Responses API calls. Eval Labs calls target service routes, reads their same-response or signed evidence, and persists the evaluation record. It does not own a separate direct OpenAI model boundary. Eval Labs does not decide Lucia’s response quality. It records and evaluates the response Lucia produced. AI-reviewed platform evidence proves that the Eval Labs lifecycle works. Human reviewers still decide Lucia behavioral quality.

​
Current Engine target

Eval Labs endpoint selection is environment-configured through VITE_LUCIA_EVAL_ENDPOINT. The current operator-facing validation target is:
Development is where active Engine iteration happens. Guest verification separately targets Guest Agent Production at https://guest.hellolucia.ai. Fieldwork evaluation relays remain Development-only. Staging is for promoted validation only when intentionally configured. Routing was corrected and live-verified against Development on 2026-07-10 after an uninterrupted Staging-target configuration that began at 2026-05-04T16:21:35Z. That is dated routing history, not current proof and not the endpoint used by every historical run.

​
Runtime provenance responsibility split

The Engine emits a stable response contract with same-response runtime and model evidence. Eval Labs reads that evidence, classifies the target environment, assesses model execution, renders owner-only verification, and persists captured provenance for new run cases and sessions. The record separates:
Historical runs without the record remain historical. The review surface says provenance was not captured; it does not reconstruct endpoint or model use from later configuration.

​
Source of truth hierarchy

When debugging Eval Labs platform behavior:
  1. Browser Network request URL
  2. Same-response or signed target runtime/model evidence
  3. Current deployed target and Eval Labs identities
  4. Current route and role state
  5. Durable records, receipts, and counts
  6. Run History / Analysis UI truth
  7. Exported run metadata
  8. Browser-state diagnostics
  9. Human memory
Human memory is useful. It is not the source of truth.

​
Current route architecture

The current route map is documented in Product Surfaces and Route Map. Core canonical paths:
Legacy aliases:

​
Current role architecture

Role gating is documented in Role and Access Model. Current recognized application roles:
Owner/admin are privileged roles with broad platform access, Team Review, Analysis, shared persisted evidence, and all Launcher test surfaces. Platform Runtime Registry is owner-only, so owner and admin are not identical. Evaluator is the full evaluator workbench role. Evaluator can use evaluator-safe test surfaces and own run/review/history routes, but cannot use Team Review or Analysis. Tester is the entry-level prompt-testing lane. Tester can use Custom Prompt Test, including saved-suite deep links, Auto-generated Prompt Test, keyboard review, history, and eligible owned runs. Tester cannot use verification, controlled batch, Team Review, Analysis, Registry Diagnostics, Behavioral Observatory, or privileged tools. Missing or unknown roles should fail closed. Clerk authenticates the participant and supplies an application role. Durable server/data authorization independently enforces privileged access and evaluator/tester ownership scope.

​
Important design decision

The custom prompt feature did not require a separate database model because Eval Labs already had a general structure:
SessionRun itemsLucia responsesHuman reviews
Custom prompts are a new run source, not a new evaluation universe. That is good architecture.

​
Ordinary and specialized evidence are different graphs

Ordinary evaluation uses:
Prompt suiteEvaluation runRun casesHuman reviewsReview positions and optional behavioral labels
Runtime Registry and Fieldwork evaluation add separate append-only claims, seals, cohort records, receipts, review attestations, and recovery lineage. Do not flatten those records into generic sessions or treat browser-local history as durable evidence.

​
Current run source strategy

Custom runs reuse the ordinary run lifecycle while retaining an explicit custom-run classification. This preserves compatibility with the existing run engine while clearly distinguishing custom runs from generated automated runs. Controlled batch runs use the same platform lifecycle to create, execute, review, finalize, persist, and verify runs. They are operational readiness evidence, not a separate human-review standard. Guest Facing Agent Verification Check is a separate app surface for booked-guest verification behavior and results. It is evaluator-safe but not tester-facing. The 72-case Fieldwork Communication Baseline is owner/admin-only. Fieldwork Conversation v1 freezes 40 cases across three runs; v2 requires an explicitly minted 120-entry cohort with durable receipt and recovery-lineage rules. Neither controlled interface runs on page load.

​
Review-layer architecture

Eval Labs now separates review responsibility into layers:
Review Queue UISuggested review valuesEmployee Review fieldsHuman Guidance EvaluationReview State / Escalation flagsLifecycle / dirty / completion stateAdjudication metadataExports / Analysis
The schema supports high-resolution analysis while the employee UI remains simple. This is intentional. The user-facing review experience should remain calm and guided even when the exported data is detailed.
This architecture snapshot did not make a model-bearing request, inspect authenticated production counts, or execute a Fieldwork cohort. The model table records current owning-source policy; provider execution still requires per-request evidence.