Skip to main content
Custom Prompt Suites let approved users repeatedly test the same 1–10 prompts while Lucia changes.

​
What a custom suite is

A custom suite is a saved set of manually chosen prompts. It can contain:
Prompt text and order are preserved. Canonical Custom tester route:
Saved suite deep link:
Evaluator and tester users can run Custom evals when assigned, but only their own scoped runs are intended for review/finalization.

​
Why custom suites matter

Custom suites are the core tool for Lucia refinement. They allow us to test the exact same behavior family before and after changes. This is how we know whether a patch actually worked. Custom suites are the safest evaluator-facing starting point because they keep scope narrow and behavior families inspectable.

​
When to create a custom suite

Create a custom suite when you are testing:
  • a known bug family
  • a suspected intent routing gap
  • tone calibration
  • distress containment
  • trust-state discipline
  • memory behavior
  • multilingual behavior
  • regression after a model or prompt change

​
Good suite example

Suite name:
Prompts:
Why this is strong:
  • one coherent behavior family
  • multiple lexical variants
  • not too broad
  • easy to re-run after Engine patch

​
Bad suite example

Why this is weak:
  • too many behavior families
  • hard to interpret as one pattern
  • unclear what a pass/fail trend means

​
Naming convention

Use clear names:
Avoid vague names:

​
How to update a suite

If the purpose changes, create a new suite version. Do not silently mutate a suite so much that old comparisons become meaningless. Use versioning:

​
Custom loaded prompts-suite in Eval Labs

ui-named-active-custom-suite Custom launcher with manually loaded prompts.
ui-saved-custom-prompt-suites Saved suites that can be continuously used alongside recent runs list.
ui-custom-launcher-loaded-suite-with-analysis Custom prompt suite loaded and ready for analysis in Evaluation Labs.