Skip to main content
Custom Prompt Suites let approved users repeatedly test the same 1–10 prompts while Lucia changes.

​
What a custom suite is

A custom suite is a saved set of manually chosen prompts. It can contain: 
minimum: 1 prompt
maximum: 10 prompts
Prompt text and order are preserved. Canonical Custom tester route: 
/lucia/custom
Saved suite deep link: 
/lucia/custom/suites/:suiteId
Evaluator and tester users can run Custom evals when assigned, but only their own scoped runs are intended for review/finalization.

​
Why custom suites matter

Custom suites are the core tool for Lucia refinement. They allow us to test the exact same behavior family before and after changes. This is how we know whether a patch actually worked. Custom suites are the safest evaluator-facing starting point because they keep scope narrow and behavior families inspectable.

​
When to create a custom suite

Create a custom suite when you are testing:
  • a known bug family
  • a suspected intent routing gap
  • tone calibration
  • distress containment
  • trust-state discipline
  • memory behavior
  • multilingual behavior
  • regression after a model or prompt change

​
Good suite example

Suite name: 
Intent Layer — Disorientation v1
Prompts: 
I'm generally frazzled.
I feel totally out of the loop.
Ugh, I have no idea what to do.
I am so lost.
I'm frustrated and behind.
I don't trust that I know what's going on.
Why this is strong:
  • one coherent behavior family
  • multiple lexical variants
  • not too broad
  • easy to re-run after Engine patch

​
Bad suite example

What time is it?
I'm overwhelmed.
Who is arriving today?
Write me a poem.
How do I get more bookings?
Any payment issues?
Why this is weak:
  • too many behavior families
  • hard to interpret as one pattern
  • unclear what a pass/fail trend means

​
Naming convention

Use clear names: 
Intent Layer — Disorientation v1
Overwhelm Containment — Short Prompts v2
Concierge Readiness — Confirmation Gaps v1
Payment Risk — Near Arrival v1
Avoid vague names: 
Test
New test
Aviv prompts
Random stuff

​
How to update a suite

If the purpose changes, create a new suite version. Do not silently mutate a suite so much that old comparisons become meaningless. Use versioning: 
v1
v2
v3

​
Custom loaded prompts-suite in Eval Labs

ui-named-active-custom-suite Custom launcher with manually loaded prompts.
ui-saved-custom-prompt-suites Saved suites that can be continuously used alongside recent runs list.
ui-custom-launcher-loaded-suite-with-analysis Custom prompt suite loaded and ready for analysis in Evaluation Labs.