> ## Documentation Index
> Fetch the complete documentation index at: https://docs.usefini.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Create Test Suite regression sets, attach criteria, start runs, and read run results through Fini's public API.

Test sets are the API form of the dashboard [Test Suite](/en/testing/test-suite). A test set groups existing conversations, attaches one or more grading criteria, and queues asynchronous runs that evaluate those conversations against the current agent behavior.

Use these routes when you want to seed regression checks from conversations already in Fini, manage their criteria from your backend, or trigger a run after changing prompts, knowledge, rules, or actions.

<Info>
  Test set runs evaluate existing conversation IDs. Create or import the conversations first, then pass their IDs in `conversationIds` when you create or update a test set.
</Info>

## Endpoints

<CardGroup cols={2}>
  <Card title="List test sets" icon="list" href="/en/api-reference/list-test-sets">
    Return every test set in the workspace, newest first.
  </Card>

  <Card title="Create a test set" icon="plus" href="/en/api-reference/create-test-set">
    Create a set from one to 200 existing conversation IDs.
  </Card>

  <Card title="Get fields context" icon="table-list" href="/en/api-reference/get-test-set-fields-context">
    Read default criteria and deterministic-condition fields.
  </Card>

  <Card title="Get a test set" icon="eye" href="/en/api-reference/get-test-set">
    Fetch a test set with its resolved criteria.
  </Card>

  <Card title="Update a test set" icon="pen" href="/en/api-reference/update-test-set">
    Update the set name, description, or conversation list.
  </Card>

  <Card title="Delete a test set" icon="trash" href="/en/api-reference/delete-test-set">
    Delete a set after active runs finish.
  </Card>

  <Card title="Add criteria" icon="circle-plus" href="/en/api-reference/add-test-set-criteria">
    Attach default or custom criteria to a set.
  </Card>

  <Card title="Update a criterion" icon="pen-to-square" href="/en/api-reference/update-test-set-criterion">
    Change one criterion on a set.
  </Card>

  <Card title="Delete a criterion" icon="circle-minus" href="/en/api-reference/delete-test-set-criterion">
    Remove one criterion from a set.
  </Card>

  <Card title="List runs" icon="bars-progress" href="/en/api-reference/list-test-set-runs">
    Page through run summaries for one set.
  </Card>

  <Card title="Start a run" icon="play" href="/en/api-reference/start-test-set-run">
    Queue an asynchronous evaluation run.
  </Card>

  <Card title="Get a run" icon="chart-simple" href="/en/api-reference/get-test-set-run">
    Fetch one run and its detailed result.
  </Card>
</CardGroup>

## Authentication

<ParamField header="Authorization" type="string" required>
  Bearer token containing your Fini workspace API key. Format: `Bearer fini_...`
</ParamField>

Write routes also require `Content-Type: application/json` when they send a body.

## TestSet object

<ResponseField name="id" type="string">
  Test set ID.
</ResponseField>

<ResponseField name="companyId" type="string">
  Workspace ID that owns the test set. This field is omitted from list summaries.
</ResponseField>

<ResponseField name="name" type="string">
  Test set name.
</ResponseField>

<ResponseField name="description" type="string | null">
  Optional description.
</ResponseField>

<ResponseField name="conversationIds" type="string[]">
  Existing Fini conversation IDs included in the set. The API accepts 1 to 200 UUIDs.
</ResponseField>

<ResponseField name="createdBy" type="string | null">
  User ID that created the set. API-created records can be `null`.
</ResponseField>

<ResponseField name="createdAt" type="string">
  ISO 8601 creation timestamp.
</ResponseField>

<ResponseField name="updatedAt" type="string">
  ISO 8601 last-update timestamp.
</ResponseField>

List responses return `conversationCount` and `criteriaCount` instead of the full `conversationIds` array.

## Criterion object

<ResponseField name="id" type="string">
  Criterion ID.
</ResponseField>

<ResponseField name="testSetId" type="string | null">
  Test set that owns the criterion. Default criteria return `null`.
</ResponseField>

<ResponseField name="companyId" type="string | null">
  Workspace that owns the criterion. Fini-provided defaults can return `null`.
</ResponseField>

<ResponseField name="defaultCriterionId" type="string | null">
  Default criterion copied into this test set, if this criterion was created from a default.
</ResponseField>

<ResponseField name="name" type="string">
  Criterion label.
</ResponseField>

<ResponseField name="type" type="string">
  One of `deterministic`, `basic_judge`, or `complex_judge`.
</ResponseField>

<ResponseField name="judgePrompt" type="string | null">
  Judge prompt for LLM-graded criteria.
</ResponseField>

<ResponseField name="passPrompt" type="string | null">
  What should count as a pass for LLM-graded criteria.
</ResponseField>

<ResponseField name="failPrompt" type="string | null">
  What should count as a fail for LLM-graded criteria.
</ResponseField>

<ResponseField name="condition" type="object | null">
  Deterministic condition. Allowed root paths are `knowledgeSearchUsed`, `usedArticles`, `usedKnowledgeFolders`, `intentRules`, `tagGroups`, and `replyTypes`.
</ResponseField>

<ResponseField name="blocking" type="boolean">
  Whether failing this criterion should make the conversation fail overall.
</ResponseField>

<ResponseField name="isActive" type="boolean">
  Whether the criterion is active.
</ResponseField>

## Run object

<ResponseField name="id" type="string">
  Test run ID.
</ResponseField>

<ResponseField name="testSetId" type="string">
  Test set evaluated by the run.
</ResponseField>

<ResponseField name="status" type="string">
  `running`, `completed`, or `failed`.
</ResponseField>

<ResponseField name="result" type="object | null">
  Full result for [Get a run](/en/api-reference/get-test-set-run). While the run is queued or processing, this is `null`.
</ResponseField>

<ResponseField name="createdBy" type="string | null">
  User ID that started the run. API-started runs can be `null`.
</ResponseField>

### Result shape

When a run completes, `result.summary` contains aggregate counts and `result.conversations` contains per-conversation results.

<ResponseField name="result.summary.testSetResult" type="string">
  Overall verdict, `pass` or `fail`.
</ResponseField>

<ResponseField name="result.summary.totalConversations" type="number">
  Number of conversations evaluated.
</ResponseField>

<ResponseField name="result.summary.passedConversations" type="number">
  Conversations with a passing result.
</ResponseField>

<ResponseField name="result.summary.failedConversations" type="number">
  Conversations with a failing result.
</ResponseField>

<ResponseField name="result.summary.errorConversations" type="number">
  Conversations that errored during evaluation.
</ResponseField>

<ResponseField name="result.conversations[].criteriaResults[]" type="array">
  Criterion-level verdicts, reasoning, and evidence for each evaluated conversation.
</ResponseField>

## Errors

<AccordionGroup>
  <Accordion title="400 Bad Request" icon="circle-exclamation">
    The request body or query parameters failed validation. Common cases: empty names, invalid UUIDs, more than 200 conversation IDs, no criteria before starting a run, or invalid criterion definitions.
  </Accordion>

  <Accordion title="401 Unauthorized" icon="lock">
    The API key is missing, malformed, revoked, or invalid.
  </Accordion>

  <Accordion title="403 Forbidden" icon="shield-halved">
    The API key does not include the required `read` or `write` scope.
  </Accordion>

  <Accordion title="404 Not Found" icon="circle-question">
    The test set, criterion, run, default criterion, or conversation does not exist in your workspace.
  </Accordion>

  <Accordion title="409 Conflict" icon="triangle-exclamation">
    The test set has an active run and cannot be deleted until the run finishes.
  </Accordion>
</AccordionGroup>
