> ## Documentation Index
> Fetch the complete documentation index at: https://docs.usefini.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Save regression cases, share criteria, organize collections and evaluate agent changes through the Test Suite API.

Test Suite stores reusable cases for one agent, combines local and shared criteria, and runs those cases against a captured configuration. Cases can come from Inbox conversations or scripted customer turns. See [Test Suite](/en/testing/test-suite) for the dashboard workflow.

<Warning>
  The previous Test Sets API has been removed. Use the `/v2/test-suite/.../public` routes below. Old test-set IDs and criterion IDs are not interchangeable with case or criteria-group IDs.
</Warning>

## Workflow

<Steps>
  <Step title="Save cases and criteria">
    [Create a case](/en/api-reference/create-test-suite-case) from a conversation or scenario. Add local criteria or link a [criteria group](/en/api-reference/create-test-suite-criteria-group). Use [fields context](/en/api-reference/get-test-suite-fields-context) for library criteria and condition metadata.
  </Step>

  <Step title="Organize and review setup">
    [Collections](/en/api-reference/list-test-suite-collections) include cases directly or through linked groups. Review captured [conversation input](/en/api-reference/get-test-suite-conversation-input), or use [Action fixtures](/en/api-reference/list-test-suite-action-fixtures) to script mock responses.
  </Step>

  <Step title="Check readiness and run">
    [Preview readiness](/en/api-reference/preview-test-suite-run) before [starting a run](/en/api-reference/start-test-suite-run). Choose draft configuration from [run configuration](/en/api-reference/get-test-suite-run-configuration). Poll [Get run](/en/api-reference/get-test-suite-run) for results.
  </Step>
</Steps>

Runs use recorded or mock Action responses and never execute live Actions. Draft Behavior, rules and articles affect the run without being published. A missing Action response can produce an execution error; [save a missing mock](/en/api-reference/save-test-suite-missing-action-mock) into the original conversation case and start another run.

## Shared objects

<ResponseField name="case" type="object">
  A named conversation or scripted scenario for one `botId`. Detail responses expose messages, setup, artifacts and history rather than the stored raw input. `criteria` contains effective checks; `localCriteria` contains case-specific checks. `criteriaGroups` and `criterionOrigins` identify shared checks.
</ResponseField>

<ResponseField name="criteria group" type="object">
  Shared checks for one agent. Detail responses include `linkedCases`, `caseCount`, `revision` and `impactToken`. Updating the group or removing members requires the latest impact token. A stale token returns `409`; reload and review the changed impact before retrying.
</ResponseField>

<ResponseField name="collection" type="object">
  A named grouping of cases and criteria groups. The API calls these `folders`. Effective case membership includes direct cases and cases from linked groups, without duplicates. Removing a direct case link does not remove membership through a group.
</ResponseField>

<ResponseField name="run" type="object">
  A configuration snapshot and case-run `items`. Item statuses are `queued`, `running`, `completed` and `failed`. A completed evaluation returns a pass/fail verdict; an execution failure can return diagnostics and an `error`. Saved snapshots remain readable after a case is deleted.
</ResponseField>

## Criterion definitions

A case can have up to 50 effective criteria across its local checks and all shared groups. Cases without criteria can be saved, but cannot run. Groups require at least one criterion.

Use one of these definition shapes:

* **Library criterion:** `defaultCriterionId` from fields context, with an optional `blocking` override. Do not include a name, type, condition or judge prompts with a library reference.
* **Deterministic criterion:** nonblank `name`, `type: "deterministic"` and a valid `condition`. Judge prompts must be absent or null.
* **AI judgement:** nonblank `name`, `type: "basic_judge"` or `"complex_judge"`, and all three nonblank fields: `judgePrompt`, `passPrompt`, `failPrompt`. `condition` must be absent or null.

For a new custom criterion, `blocking` defaults to `true`. A library reference inherits the saved value unless overridden. Only blocking failures make the overall case fail. [Create library criterion](/en/api-reference/create-test-suite-criterion) saves a new custom definition; it does not accept a library reference.

A Test Suite condition uses one `predicate` object, rather than the predicate array used by Rulebook flows:

```json theme={null}
{
  "name": "No handoff",
  "type": "deterministic",
  "blocking": true,
  "condition": {
    "scope": "SCALAR",
    "predicate": {
      "left": { "path": "escalated", "dataType": "boolean" },
      "operator": "==",
      "right": { "value": false }
    }
  }
}
```

Allowed context roots are `calledActions`, `escalated`, `knowledgeSearchUsed`, `usedArticles`, `usedKnowledgeFolders`, `intentRules`, `tagGroups` and `replyTypes`. Array conditions also use `array` and a `quantifier` of `ANY`, `ALL` or `NONE`. Get supported field types, operators and reply-type values from [fields context](/en/api-reference/get-test-suite-fields-context).

## Case input

For `sourceType: "conversation"`, provide `conversationId` and omit `scenario`. The conversation must belong to the same agent, contain customer messages, have at most 200 events, and come from Inbox rather than a replay or Test Suite run. Original agent replies are preview-only; execution generates fresh replies.

For `sourceType: "scenario"`, provide `scenario` and omit `conversationId`. A scenario has one to 50 `turns` and optional `userAttributes`. Each turn has a nonblank `message` of at most 10,000 characters and up to 50 `actions`. Each Action includes an assigned `ruleId`, a nonblank `nodeId` of at most 100 characters, a `toolId` and JSON-object `output`; `success` defaults to `true`. Use [Action fixtures](/en/api-reference/list-test-suite-action-fixtures) for valid node identifiers.

Both source types support `attributeOverrides`, shared `criteriaGroupIds` and manual `artifacts`. Captured input is limited to 1 MB. Criterion definitions and attribute overrides are each limited to 100 KB. Updating a case always requires `name`; omit optional fields to preserve them.

## Recorded Action overrides

Captured-conversation cases expose `actionResponses`. To change a recorded response, send its `fingerprint`, `eventId` and zero-based `resultIndex`, plus `success` and JSON-object `output`. These identity fields must match the saved recording. Up to 200 overrides can be sent; the resulting saved test input must fit within the 1 MB limit.

To restore the original response, send `reset: true` with the identity fields and omit `success` and `output`. A recovered mock has no original recording to reset to. Scripted cases use scenario `actions` instead of `actionResponses`.

## Run selection and configuration

A run selects one to 200 effective cases. `caseIds` selects individual cases and `folderIds` selects collections. With no selectors, all active cases for the agent are selected, subject to the limit. `criteriaGroupId` selects a group and evaluates only that group's shared checks; it cannot be combined with nonempty `folderIds`. `collectionId` records a collection origin and restricts selected cases to that collection.

Readiness preview checks selection and missing criteria. It does not validate or apply draft-version fields. Starting a run validates Behavior ownership, draft availability, and at most one selected draft per rule or original article. Run responses include the captured public `environment`; private execution artifacts are not exposed.

## Endpoints

| Endpoint | Method | Scope |
| - | - | - |
| [List criteria groups](/en/api-reference/list-test-suite-criteria-groups) | `GET` | `read` |
| [Create criteria group](/en/api-reference/create-test-suite-criteria-group) | `POST` | `write` |
| [Get criteria group](/en/api-reference/get-test-suite-criteria-group) | `GET` | `read` |
| [Update criteria group](/en/api-reference/update-test-suite-criteria-group) | `PATCH` | `write` |
| [Add cases to criteria group](/en/api-reference/add-test-suite-group-cases) | `POST` | `write` |
| [Remove cases from criteria group](/en/api-reference/remove-test-suite-group-cases) | `DELETE` | `write` |
| [Add conversations to criteria group](/en/api-reference/add-test-suite-group-conversations) | `POST` | `write` |
| [Get Test Suite fields context](/en/api-reference/get-test-suite-fields-context) | `GET` | `read` |
| [Create library criterion](/en/api-reference/create-test-suite-criterion) | `POST` | `write` |
| [Get run configuration](/en/api-reference/get-test-suite-run-configuration) | `GET` | `read` |
| [List Action fixtures](/en/api-reference/list-test-suite-action-fixtures) | `GET` | `read` |
| [List test cases](/en/api-reference/list-test-suite-cases) | `GET` | `read` |
| [Get conversation input](/en/api-reference/get-test-suite-conversation-input) | `GET` | `read` |
| [Create test case](/en/api-reference/create-test-suite-case) | `POST` | `write` |
| [Get test case](/en/api-reference/get-test-suite-case) | `GET` | `read` |
| [Update test case](/en/api-reference/update-test-suite-case) | `PATCH` | `write` |
| [Delete test case](/en/api-reference/delete-test-suite-case) | `DELETE` | `write` |
| [List collections](/en/api-reference/list-test-suite-collections) | `GET` | `read` |
| [Create collection](/en/api-reference/create-test-suite-collection) | `POST` | `write` |
| [Delete collection](/en/api-reference/delete-test-suite-collection) | `DELETE` | `write` |
| [Add cases to collection](/en/api-reference/add-test-suite-collection-cases) | `POST` | `write` |
| [Add criteria groups to collection](/en/api-reference/add-test-suite-collection-groups) | `POST` | `write` |
| [Remove cases from collection](/en/api-reference/remove-test-suite-collection-cases) | `DELETE` | `write` |
| [Remove criteria groups from collection](/en/api-reference/remove-test-suite-collection-groups) | `DELETE` | `write` |
| [List runs](/en/api-reference/list-test-suite-runs) | `GET` | `read` |
| [Get Test Suite summary](/en/api-reference/get-test-suite-summary) | `GET` | `read` |
| [Preview run readiness](/en/api-reference/preview-test-suite-run) | `POST` | `read` |
| [Start run](/en/api-reference/start-test-suite-run) | `POST` | `write` |
| [Get run](/en/api-reference/get-test-suite-run) | `GET` | `read` |
| [Save missing Action mock](/en/api-reference/save-test-suite-missing-action-mock) | `POST` | `write` |


## Related topics

- [Overview](/en/api-reference/tags.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.