Validation Battery

model drift
prompt drift
tone regression
truth-state regression
routing regression
semantic intent regression
guest identity/linkage regression
privacy leakage regression
guest signal routing regression
payment truth regression
financial attention regression

https://api-dev.hellolucia.ai/admin/operator-focus

/admin/operator-focus

Manual testing cannot cover every possible guest phrasing.
Eval Labs must stress-test guest identity, verification, privacy, tone, and guest-to-operator routing at scale.

identity orientation
already booked / joining someone / planning / exploring
booked-guest claim parsing
claim fragments across turns
weak vs strong claims
ambiguous/no-match cases
magic-link eligibility
email sent only to booking email on file
no private data leakage
token consume / replay / expiry
verified session state
guest-to-operator signal creation
Admin Signal Stream visibility
Focus Ops no Luca/Nora drift
warm hospitality tone

Harper Quinn #29110012 confirmed-paid Stripe sandbox payment survived durable ledger restart.
Confirmed-paid Harper Quinn was suppressed as a payment blocker.
Arrival readiness remained as a non-payment attention issue.

confirmed-paid guest should not remain a payment blocker
pending payment should not become confirmed paid
failed payment should not be treated as paid
disputed payment should not be treated as paid
refunded payment should not be treated as paid
unknown property payment policy should not create due/overdue overclaims
deposit paid but final balance not yet due should not become overdue
final balance overdue should become an attention candidate only after policy + temporal truth support it
Admin /payments should remain read-only and Engine-sourced
Booking Payment Truth card should remain read-only and Engine-sourced
DAW payment reconciliation note save should not mutate payment state

confirmed paid record remains a payment blocker
pending/review state is upgraded to paid without Stripe financial truth
policy-unknown state invents due dates or overdue language
Admin UI offers charge, refund, mark-paid, or create-checkout controls before implementation

LIEA consumes Stripe financial truth for payment attention suppression.
LIEA does not yet consume durable property payment policy truth.
Signal Stream is not yet wired to LIEA.

monthly booking reality
arrivals
departures
stay windows
occupied months
booking clicks

Calendar uses the /admin/bookings source
Calendar route remains /calendar
Booking Pulse derives from the same booking source
Calendar booking clicks route to /bookings/:bookingId
Calendar booking clicks do not route to DAW
Full Booking Page remains the record/review surface
Villa Valentin is treated as one rentable inventory unit in the current fixture

generic calendar claims without booking truth
impossible overlaps in fixture data
stale arrival_profile days
booking click routed to the wrong workspace

What matters most right now?
Where should I start this morning?
I am stretched thin. What matters most?

single highest-priority next move

What can wait until tomorrow?
What can I safely ignore right now?

clear defer list or defer explanation

Good morning
Thanks Lucia
I feel overwhelmed

natural, brief, warm, not robotic

Show me Yasmin's massage request.
What's going on with Nora's payment?
Look into Priya's Wi-Fi issue.

named entity preserved
service/request scope preserved
generic top ranking does not override named scope
CTA routes to the named object

operator names one guest or request
Lucia answers with a different global top item

current Signal Stream subject scopes the reply
current signal beats stale conversation context
named guest/service/request scope is preserved
generic global ranking does not override active_context

Why?
Can this wait?
What should I do after that?

follow-up resolves against verified prior recommendation context
newest verified concrete recommendation replaces old memory
clarification and information-only replies do not overwrite prior recommendation memory

short follow-up guesses from a stale or unrelated global top item
older recommendation remains active after a newer concrete recommendation

Dashboard
Calendar
DAW
Full Booking
Bookings
Maintenance
Concierge
Tasks
Reconciliation

What am I looking at?
What should I do here?
What matters for this booking?
What should I pay attention to here?
Now what?

answer uses current surface context
explicit prompt subject wins over page context
same conversation is preserved between Dashboard embedded mode and Workspace Sidebar mode
Signal Stream Chat seeds shared Lucia conversation
Reset clears shared conversation and context
no raw metadata language reaches the operator

Lucia ignores the current surface for orientation prompts
Lucia overrides a named guest/request with page context
Lucia exposes active_context/current_surface/payload/metadata language
Lucia treats missing payload fields as fake zero-count truth

Remind me in one minute
Remind me in X minutes
Remind me in X hours
Remind me tomorrow
Remind me next week
Open Reminder
Got it

reminder persists in Engine
due reminder appears in dashboard data
Signal Stream wraps due reminder as Open Reminder
Got it acknowledges the reminder only
underlying issue is not described as resolved

Signal Stream builds one coherent stream from operational state, reminders, commands, dismissals, and polling
Move-to-top persists as an attention-shaping event
due reminder resurfacing is an attention-shaping event
newest meaningful attention event wins unless dismissed
dismiss removes the signal from the current stream without implying resolution
Show More expands up to 20 signals

infinite CTA language resolves through Resolver Matrix
structured destination exists
route matches action intent
operator lands where work can be performed
label-only specificity is not enough

CTA sounds actionable but resolves to a generic or unusable route

/ops/actions renders the structured action workspace
workspace reflects the routed intent and metadata
operator can inspect context and act
unknown or partial metadata degrades safely
specialized pages remain available as supporting or future-specialized surfaces

DAW requires a persistence target for normal workflows
task-backed save works as a saved workflow step
concierge request save works as a saved workflow step
booking-note-backed save works as a saved workflow step
insufficient context bridges safely instead of inventing completion
Saved means saved
saved payment review does not mean payment resolved
saved arrival note does not mean arrival details changed

Time?
What time is it in Bangladesh?
Nice to see you
Let's do this
Should we keep dinner outside tomorrow?
Is outside still okay for the ceremony?

semantic family recognition
bounded utility or clarification
no phrase patching
no open-domain drift

already booked
joining someone's stay
planning trip
just exploring
identity orientation selected before chatting
orientation prompt appears in composer
identity buttons disappear after orientation is sent
normal conversation begins after orientation

orientation feels like Lucia orienting herself
not a form disguised as chat
warm public-facing hospitality tone
no private booking data exposed

name-only weak claim
booking ID-only weak claim
ID + name strong claim
name + arrival date strong claim
claim fragments across turns
ambiguous matches
no matches
service overlap
airport pickup overlap

name-only remains weak
booking-ID-only remains weak
strong claim can request verification when Engine permits it
claim fragments merge across turns
ambiguous and no-match outcomes degrade safely
service overlap is never identity evidence
airport pickup overlap does not attach an unlinked guest to Luca/Nora or any existing guest

guest claim treated as verified identity
service overlap used as booking evidence
private booking fields exposed from candidate lookup

magic-link email send eligibility
guest-entered email supplied
token consume
verified session state
used token
invalid token
expired token

verification email goes only to the booking email already on file
guest-entered email is never trusted as destination
token is opaque
only token hash is stored
token is one-time consumable
session endpoint can return verified: true after valid consume
no booking ID, booking email, payment detail, stay date, token, or magic URL appears in chat

unlinked signal routing
candidate signal routing
verified signal routing
Admin Signal Stream visibility
Focus Ops no Luca/Nora drift
candidate/unlinked guest signal with service overlap

guest signals render in Admin Signal Stream
unlinked guest signals show review/link language
candidate signals require operator link
candidate/unlinked signals do not open fake DAW
verified or operator-linked signals may route to DAW only when safe
Focus Ops does not attach candidate/unlinked signals to similar existing bookings

repair issue
guest-reported problem
media evidence
arrival timing
property impact

clear routing into maintenance/action layer

truthful
specific
calm
useful
correctly scoped

overclaims
buries the next move
sounds generic
creates more work
ignores urgency
breaks response contract