Skip to main content
This is the simple operator guide for the major Eval Labs surfaces. Use it to avoid confusing diagnostic pages, review pages, analysis pages, and persistence truth.

​
Status labels

Use these words exactly:

​
A. Home

Use Home when you need the owner/admin overview. Home shows:
  • platform status
  • recent activity
  • quick access to major surfaces
  • product state
  • readiness or evidence summaries when available
Do not treat Home as a human approval page. Home can show platform progress. Human Lucia-quality approval still comes from human review.

​
B. Registry Diagnostics

Use Registry Diagnostics when you need to inspect derived classification behavior. Route:
Why it exists:
How to read the top summary:
  1. Check how many datasets exist.
  2. Check how many runs are included.
  3. Check how many suggested memberships exist.
  4. Check confidence breakdowns.
  5. Treat the whole page as diagnostic.
How to inspect dataset cards:
  1. Read the dataset name.
  2. Read the suggested membership count.
  3. Check confidence.
  4. Check source fields.
  5. Ask whether the evidence is real or thin.
How to inspect queue lanes:
  1. Read the suggested lane.
  2. Check which items triggered it.
  3. Check confidence.
  4. Look for broad or weak matches.
How to use Noise / Watch:
  1. Look for overmatching.
  2. Look for weak low-confidence matches.
  3. Look for items suggested for too many datasets.
  4. Write down issues for product or engineering.
What not to assume:
  • Do not assume dataset membership is final.
  • Do not assume queue routing is final.
  • Do not assume a human saved a label.
  • Do not use this page as Behavioral Observatory.

​
C. Behavioral Observatory

Use Behavioral Observatory when you need to review conversations and save behavioral labels. Route:
Before using it:
  1. Confirm your role and assignment allow access.
  2. Confirm the data shown is from the scoped run context you intend to review.
  3. Notice whether the selected conversation is showing a derived suggestion or a saved label.
Select a conversation:
  1. Open the labeling queue.
  2. Pick one conversation.
  3. Read the Human message first.
  4. Read Lucia’s response second.
Set Intent:
  1. Choose what the human was trying to do.
  2. Use Other only when the listed categories do not fit.
Set Guest Affect:
  1. Choose the smallest truthful emotional read.
  2. Do not dramatize the guest’s state.
Set Response Strategy:
  1. Choose Lucia’s main response move.
  2. Pick the dominant strategy, not every strategy present.
Set Humanness:
Do not use humanness as a substitute for truth, usefulness, or safety. Add notes:
  1. Add a note only when it preserves useful behavioral evidence.
  2. Keep it short.
  3. Name the behavior and why it matters.
Save label:
  1. Click Save label or Save updates.
  2. Wait for the saved state.
  3. If the state is error, do not count the label as persisted.
Refresh/check saved status if needed:
  1. Refresh the page.
  2. Confirm the label reloads.
  3. If it does not reload, treat the label as not verified.

​
D. Guest Facing Agent Verification

Use Guest Facing Agent Verification when you need to run or inspect booked-guest verification behavior. Routes:
Verification is for:
  • running the scenario pack from the app surface
  • inspecting pass/fail results
  • reviewing failure details
  • exporting or copying verification summaries
Verification is not:
  • a tester lane
  • Team Review
  • Global Analysis
  • proof that Lucia is human-approved

​
E. Team Review

Use Team Review when owner/admin needs oversight of evaluator activity and review quality. Route:
Team Review is for:
  • inspecting evaluator activity
  • finding missing checks
  • spotting flags and failures
  • reviewing recent human-evaluation signal
  • deciding what needs owner/admin attention
Team Review is not:
  • evaluator productivity tracking for its own sake
  • tester onboarding
  • a human approval page

​
F. Global Analysis

Use Global Analysis when you need read-only behavioral and analytics evidence. Route:
Global Analysis is for:
  • inspecting completed run evidence
  • reading behavioral summaries
  • comparing patterns
  • opening Single Run Analysis when available
Global Analysis is not:
  • a human approval page
  • a Behavioral Observatory label-save workflow
  • Registry Diagnostics

​
G. Run History

Use Run History when you need the run ledger. Route:
Run History is for:
  • finding completed/finalized runs
  • checking run lifecycle truth
  • copying run/session identifiers
  • opening review or analysis routes when allowed
Run History is not:
  • proof that a response was good
  • proof that Behavioral Observatory labels exist
  • proof that Lucia is human-approved

​
Operator rule

Use the right surface for the right truth: