Skip to main content
Eval Labs improves Lucia by turning subjective impressions into repeated, inspectable behavioral evidence and by recording dated platform-readiness evidence for Lucia intelligence work.

​
The improvement mechanism

Eval Labs helps Lucia improve by creating this loop:
Behavior observedPattern identifiedSuggested review and human judgment comparedRun History / Analysis evidence inspectedOwner file inspectedSmallest correct patch madeDev deployedSame suite re-runHuman review confirms or rejects improvement

​
What Eval Labs catches

Eval Labs can catch:
  • wrong intent routing
  • tone drift
  • weak containment
  • generic language
  • overclaiming
  • missing next moves
  • payment-risk prioritization errors
  • arrival-readiness misses
  • concierge readiness gaps
  • multilingual regressions
  • model upgrade regressions
It also protects the evaluation platform itself by validating run creation, persistence, finalization, Run History, Analysis, and compact client state before employees depend on the system.

​
Why repeated suites matter

If we only test new prompts every time, we cannot tell whether Lucia improved. Custom suites let us compare before/after behavior. That turns product feel into product evidence.

​
Lucia source-of-truth behavior

Eval Labs does not patch Lucia. Eval Labs reveals where Lucia needs patching. Route a confirmed finding to the smallest responsible product layer: intent classification, response refinement, runtime model configuration, interface behavior, persistence, or another evidenced owner. Do not infer an implementation owner from the symptom alone. Preserve the run’s endpoint, environment, runtime provenance, review state, deployed commit, and smallest reproducible example so engineering can locate the correct source seam privately.

​
The real-world milestone

Verified 2026-08-12: https://evaluationlabs.ai serves Eval Labs Production commit 6cbfd15f75330c414bfe79e0d2fab51ef8115102, and the documented production source contract targets Lucia Engine Development at https://api-dev.hellolucia.ai/admin/operator-focus. That verification did not make a model-bearing request. See Current System State for the complete evidence boundary.
Eval Labs is implemented as part of Lucia’s development evaluation loop, not merely planned infrastructure. In May 2026, the AI-reviewed platform-readiness gate recorded 60 completed runs and 3,000 prompts, run items, Lucia responses, and reviews. Those dated counts describe that gate, not the current database, and do not constitute human approval of Lucia quality.

​
Improvement mechanism — from employee signal to canon signal

The improvement loop separates signal quality:
Lucia behavior observedapp suggestions provide initial signalemployee quick review captures reactionHuman Guidance Evaluation captures structured judgmentreviewer-owned final judgment is savedsenior reviewer adjudicates important casesexported lifecycle evidence preserves the trailreusable learning becomes canon candidateengineering patches smallest correct layersame suite is re-runevidence confirms or rejects improvement
This prevents non-expert review from directly becoming Lucia doctrine while still letting the whole team contribute useful signal. The app may suggest, but the reviewer must decide. Reviewed exports should preserve the signal chain: suggested review, employee review, Human Guidance Evaluation, adjudication metadata, lifecycle state, tester identity, and dirty / completion state.

​
Intelligence stack role

Eval Labs is part of Lucia’s intelligence stack. It helps harden:
  • truthfulness
  • emotional containment
  • operational usefulness
  • intent routing
  • trust-state discipline
  • evaluator feedback loops
  • platform evidence recovery for future threads
The Canon should therefore treat Eval Labs as product infrastructure, not as a side tool.