Skip to main content
Eval Labs now separates fast employee judgment from senior adjudication so Lucia can learn from structured signal without letting non-expert reviewer language become training truth.

​
The three-layer review model

Eval Labs review is not one flat annotation step. It is a layered judgment system:
Layer 1: Employee ReviewLayer 2: Escalation RoutingLayer 3: Senior Adjudication
Each layer has a different job.

​
Behavioral Observatory position

Behavioral Observatory adds a saved behavioral label layer beside the Review Queue workflow. It does not erase the three-layer review model. Use this distinction: 
Review Queue = score and review the prompt/response item.
Behavioral Observatory = save structured behavioral labels for the conversation.
Registry Diagnostics = inspect derived classification suggestions.
Behavioral Observatory labels can become useful behavioral evidence, but they should not be confused with senior adjudication or Gold Standard approval.

​
Layer 1 — Employee Review

Employee Review captures fast human reaction. It is designed for reviewers who may not know AI, prompting, ontologies, or model training. Employees answer guided questions such as: 
Did Lucia understand what was needed?
Did Lucia give the right next move?
Did Lucia make the situation feel calmer?
Did anything feel risky, confusing, or wrong?
Should a senior reviewer look at this?
Could this teach Lucia something reusable?
This layer should be:
  • simple
  • fast
  • structured
  • low-friction
  • non-technical
  • psychologically clear
Employee reviewers should not be asked to invent labels, taxonomies, intent categories, action classes, or training language.

​
Layer 2 — Escalation Routing

Escalation Routing converts guided review answers into workflow state. Examples: 
seniorReview = true → reviewState: needs_adjudication
reusableLearning = true → canonCandidate: true
riskOrConfusion = slightly_off → reviewState: needs_review
riskOrConfusion = definitely_wrong → reviewState: needs_review
The important doctrine: 
The employee reports what they experienced.
The system routes the case.
The senior reviewer interprets meaning.

​
Layer 3 — Senior Adjudication

Adjudication is the senior-review layer where canonical meaning is assigned. Adjudication may include:
  • final human labels
  • final intent interpretation
  • follow-through decision
  • final action type
  • emotional read
  • owner pressure level
  • reason for the final call
  • reusable canon/training signal
This layer exists to prevent ontology drift. Employees should not train Lucia directly with improvised language.

​
Why this architecture matters

Without this separation, Eval Labs risks collecting inconsistent reviewer opinions as if they were stable training truth. That creates: 
ontology drift
label noise
inconsistent training signal
reviewer fatigue
low-confidence exports
With this separation, Eval Labs captures simple human judgment while preserving a high-quality senior interpretation layer.

​
Canon rule

Non-expert reviewers provide signal. Senior adjudication provides meaning.
This is the core protection layer for scalable human evaluation.