Skip to main content
Test sets are the API form of the dashboard Test Suite. A test set groups existing conversations, attaches one or more grading criteria, and queues asynchronous runs that evaluate those conversations against the current agent behavior. Use these routes when you want to seed regression checks from conversations already in Fini, manage their criteria from your backend, or trigger a run after changing prompts, knowledge, rules, or actions.
Test set runs evaluate existing conversation IDs. Create or import the conversations first, then pass their IDs in conversationIds when you create or update a test set.

​
Endpoints

List test sets

Return every test set in the workspace, newest first.

Create a test set

Create a set from one to 200 existing conversation IDs.

Get fields context

Read default criteria and deterministic-condition fields.

Get a test set

Fetch a test set with its resolved criteria.

Update a test set

Update the set name, description, or conversation list.

Delete a test set

Delete a set after active runs finish.

Add criteria

Attach default or custom criteria to a set.

Update a criterion

Change one criterion on a set.

Delete a criterion

Remove one criterion from a set.

List runs

Page through run summaries for one set.

Start a run

Queue an asynchronous evaluation run.

Get a run

Fetch one run and its detailed result.

​
Authentication

​
string
required
Bearer token containing your Fini workspace API key. Format: Bearer fini_...
Write routes also require Content-Type: application/json when they send a body.

​
TestSet object

​
string
Test set ID.
​
string
Workspace ID that owns the test set. This field is omitted from list summaries.
​
string
Test set name.
​
string | null
Optional description.
​
string[]
Existing Fini conversation IDs included in the set. The API accepts 1 to 200 UUIDs.
​
string | null
User ID that created the set. API-created records can be null.
​
string
ISO 8601 creation timestamp.
​
string
ISO 8601 last-update timestamp.
List responses return conversationCount and criteriaCount instead of the full conversationIds array.

​
Criterion object

​
string
Criterion ID.
​
string | null
Test set that owns the criterion. Default criteria return null.
​
string | null
Workspace that owns the criterion. Fini-provided defaults can return null.
​
string | null
Default criterion copied into this test set, if this criterion was created from a default.
​
string
Criterion label.
​
string
One of deterministic, basic_judge, or complex_judge.
​
string | null
Judge prompt for LLM-graded criteria.
​
string | null
What should count as a pass for LLM-graded criteria.
​
string | null
What should count as a fail for LLM-graded criteria.
​
object | null
Deterministic condition. Allowed root paths are knowledgeSearchUsed, usedArticles, usedKnowledgeFolders, intentRules, tagGroups, and replyTypes.
​
boolean
Whether failing this criterion should make the conversation fail overall.
​
boolean
Whether the criterion is active.

​
Run object

​
string
Test run ID.
​
string
Test set evaluated by the run.
​
string
running, completed, or failed.
​
object | null
Full result for Get a run. While the run is queued or processing, this is null.
​
string | null
User ID that started the run. API-started runs can be null.

​
Result shape

When a run completes, result.summary contains aggregate counts and result.conversations contains per-conversation results.
​
string
Overall verdict, pass or fail.
​
number
Number of conversations evaluated.
​
number
Conversations with a passing result.
​
number
Conversations with a failing result.
​
number
Conversations that errored during evaluation.
​
array
Criterion-level verdicts, reasoning, and evidence for each evaluated conversation.

​
Errors

The request body or query parameters failed validation. Common cases: empty names, invalid UUIDs, more than 200 conversation IDs, no criteria before starting a run, or invalid criterion definitions.
The API key is missing, malformed, revoked, or invalid.
The API key does not include the required read or write scope.
The test set, criterion, run, default criterion, or conversation does not exist in your workspace.
The test set has an active run and cannot be deleted until the run finishes.