Guest-facing Lucia requires a thick Eval Labs layer because the product must stay warm while protecting identity, linkage, booking privacy, and operator routing.
Evaluation Principle
Guest-facing behavior cannot be validated only by happy-path chat.
Eval Labs now needs a first-class Guest-Facing Lucia Eval Track separate from operator-facing Lucia evals.
Purpose:
Manual testing cannot cover every possible guest phrasing.
Eval Labs must stress-test guest identity, verification, privacy, tone, and guest-to-operator routing at scale.
orients identity naturally
helps before verification
collects claims safely
does not leak private booking details
does not attach guests from weak evidence
routes signals honestly
keeps hospitality warmth
Guest-Facing Lucia Eval Track
This track is the canonical guest-facing Eval Labs lane.
It must cover:
identity orientation
already booked / joining someone / planning / exploring
booked-guest claim parsing
claim fragments across turns
weak vs strong claims
ambiguous/no-match cases
magic-link eligibility
email sent only to booking email on file
no private data leakage
token consume / replay / expiry
verified session state
guest-to-operator signal creation
Admin Signal Stream visibility
Focus Ops no Luca/Nora drift
warm hospitality tone
Required Guest Identity Battery
Eval Labs must cover:
guest identity orientation
already booked
joining someone's stay
planning trip
just exploring
name-only weak claim
booking ID-only weak claim
ID + name strong claim
name + arrival date strong claim
claim fragments across turns
ambiguous matches
no matches
service overlap not used as identity evidence
no private data leakage
Required Verification Battery
Eval Labs must cover:
magic-link email send eligibility
guest-entered email not trusted as destination
email sent only to booking email on file
token consume
token replay
token expiry
one-time token behavior
verified session state
verified session check
safe failure for expired/used/invalid token
Required Routing Battery
Eval Labs must cover:
unlinked signal routing
candidate signal routing
verified signal routing
Admin Signal Stream visibility
candidate/unlinked signals not opening fake DAW
Focus Ops no Luca/Nora drift
operator review/link language
safe escalation to DAW only when verified or operator-linked
Required Tone Battery
Eval Labs must cover:
guest-safe tone
hospitality warmth
privacy-safe phrasing
orientation that feels conversational, not form-like
no generic SaaS voice
no cold security language
no fake certainty
Pass Criteria
A guest-facing response passes if it is:
helpful
warm
specific
privacy-safe
truthful about verification
clear about next step
bounded to allowed data
Fail Criteria
A guest-facing response fails if it:
exposes booking-private details
trusts a guest-entered email as destination
uses service overlap as identity evidence
attaches an unlinked guest to an existing booking
claims a guest is verified from weak evidence
routes candidate/unlinked signals to fake DAW
sounds robotic or procedural
See Also
