> ## Documentation Index > Fetch the complete documentation index at: https://docs.usefini.com/llms.txt > Use this file to discover all available pages before exploring further. # Test Suite > Build conversation-based test cases, share criteria, and check agent changes with recorded or mock Action responses. Test Suite turns conversations into repeatable regression checks. Each **test case** captures a conversation, its test setup, and the criteria that define a passing result. Fini replays the customer's messages and generates new agent replies, then evaluates those replies against your criteria. Select the agent you want to test, then open **Test Suite**. The workspace has **Test cases**, **Criteria groups**, and **Runs** tabs. Use **All tests** or a collection to narrow the workspace. Test Suite uses recorded or mock responses for external Actions. It does not offer live Action execution. Running a test with draft Behavior, rules, or articles does not publish those drafts. ## Create a test case Click **Create test case**, then **Select conversation**. Use **Paste a link** to load an Inbox conversation and preview it before selecting it, or click **Browse Inbox with filters** to find a conversation for the selected agent. If you need a new example, use **Create new conversation** to chat with the agent in [Inbox](/en/testing/inbox), then click **Use this conversation** to return to the test case. You can also start from the flask icon in Inbox: select one eligible conversation and click **Select**. Name the case and open **Criteria**. Use **Add criterion** for case-specific checks and **Link group** for shared criteria groups, or combine both. Criteria are optional when saving a new case, but at least one must be present before it can run. **Add criterion** opens a dialog with three tabs: **Suggested checks** for reusable criteria, **Exact checks** for deterministic checks, and **Custom expectation** for an AI judgement of a written expectation. In **Suggested checks**, search and select a check, review its preview, then click **Add criterion**. In **Exact checks**, use **What do you want to check?** to choose an Action, rule, article, reply, handoff, or **Advanced condition**. Mark checks **Required to pass** when they should determine the case verdict; this option also applies to suggested checks. Open **Test setup** to inspect recorded Action responses and conversation attributes. Override responses or attributes when you want to test a different outcome. These changes belong to the test case, not the source conversation. Click **Create test case**. The saved case has **Conversation**, **Criteria**, **Test setup**, and **Run history** tabs. Save later edits with **Save changes** before running. Customer messages come from the captured conversation. Test Suite does not generate a new customer scenario from a CSV, knowledge article, or judge prompt. To test another customer path, create or select another conversation. Creating a new conversation in Inbox may call live integrations. Test Suite's recorded/mock Action behavior applies to test runs, not to the Inbox conversation you create as an input. ## Criteria and shared groups Use an AI judgement for an outcome that needs interpretation, such as whether the reply explains a refund policy without inventing an exception. Use exact checks for observable events, such as whether the agent called a particular Action or handed off to a human. All required checks must pass for the case to pass. Non-required checks still appear in the results but do not turn the case verdict into a failure. **Criteria groups** share checks across linked cases. A case can belong to multiple groups and also have its own criteria. Editing a group's criteria changes what its linked cases use on their next run; saved run results do not change. Review the affected cases before confirming a group update. A run started from a criteria group evaluates that group's criteria only, using each member's individual conversation setup. A normal case run includes its case-specific and linked group criteria. Removing a case from a group keeps the case, its setup, other group links, and run history. If no criteria remain, add a criterion or link another group before running it again. ## Organize cases into collections Collections organize cases and criteria groups for a particular feature or release. Select a collection to see its cases, groups, and run history, then use **Run collection** to test that scope. Removing a case from a collection or deleting the collection does not delete the case. Use **Duplicate test case** from a saved case's menu to try a different setup. The copy's setup and case-specific criteria are independent; linked group criteria remain shared. ## Delete test cases Use **Delete test case** from a case's menu, or select cases in the workspace and click **Delete selected**. Review the confirmation before deleting. Deletion is permanent. It removes the case, its setup, and its links to all collections and criteria groups. This is different from removing a case from one collection or group. Original Inbox conversations and past run results are kept. Queued or running tests finish using their saved setup. A historical result marks a deleted case as deleted; you cannot save a missing Action mock back to that case. If a bulk deletion only partly succeeds, the dialog keeps the failed cases selected for retry. Retrying deletes only those remaining cases. ## Run tests without publishing Use **Run test case** for one case, **Run all** from the workspace, or **Run collection** for a collection. Runs support up to 200 cases; select a smaller batch if the workspace exceeds that limit. The run dialog checks that cases have criteria. If some are incomplete, add criteria first or explicitly choose **Run only the ready test cases** using the count shown in the dialog. If none are ready, the run cannot start. Under **Behaviour**, use **Latest published behaviour** or a saved version. You can also select available rule and article drafts. Selected drafts replace their published versions for this run; new drafts are added for the run. Nothing is published. For recorded Inbox conversations with linked agent replies, the run preserves those reply boundaries instead of generating a reply after every customer message. Consecutive customer messages remain context, and an unanswered final message still gets a reply. External Actions use recorded or mock responses. If a required response is missing, the case reports an execution error instead of calling the live service. This differs from the separate [Replay](/en/api-reference/replays) workflow, which has its own execution modes. ## Read run results Open **Runs**, or a case's **Run history**, then click the run name to open its dedicated results page. **Copy link** gives you a URL to share with teammates who have workspace access. Distinguish **Failed**, where a required criterion did not pass, from **Execution error**, where the run could not complete normally. Expand a result to inspect criterion-level reasoning, the generated conversation, and Action responses. Compare the recorded run configuration before attributing a changed verdict to an agent regression: the criteria, setup, or selected draft versions may also have changed. For a completed result marked **Failed**, start with **What happened** and **Main issue**. A suggested cause is labeled **Suggested · Not confirmed**: review its evidence before changing the agent. **Needs investigation** means the analysis could not establish a supported cause; it does not change the original verdict. Open **More details** to read the explanation and **Criteria results**. Failed results keep this section collapsed initially. Under **Supporting evidence**, message and Action links jump to the corresponding saved reply or Action result when available. Other references identify the supporting context or version. **Open saved execution trace** opens the replay in Inbox when that link is available. Older saved runs may show **Not analysed** and retain their original check results without a failure analysis. If analysis is unavailable for a new failed result, the check verdict is still preserved. AI judgement separates customer-facing replies from private context such as Action results, attributes, and internal notes. When available, it uses the historical article version referenced by the run, not today’s article text. A disclosure check should identify content in a customer-facing reply, not merely a value present in private context. For a missing Action response, click **Add mock response** in the execution error. Choose **Success** or **Failure**, enter the **Mock response JSON**, and click **Save mock response**. The response is saved to this case's **Test setup → Action responses**; nothing is sent to the external service. Use **Rerun test case** to try again. ## Troubleshooting Add at least one case-specific criterion or link a criteria group. Finish saving or canceling pending edits, then reopen the run dialog. Select a conversation belonging to the current agent with customer messages to replay. The Inbox picker excludes Replay and Test Suite conversations, as well as conversations without a non-empty customer message. Other eligibility checks still apply when selecting a conversation. Create a new example in Inbox if needed. Inspect the requested inputs in the execution error, save an appropriate mock response, and rerun. The suite does not fall back to a live external Action call. Reopen the run configuration and choose an available version. Review the warning before rerunning; a deleted or unavailable draft cannot be reused automatically. Read the failed criterion's reasoning. Refine the expectation if it tests the wrong behavior, or fix the agent's prompt, knowledge, or rule if the failure is valid. Then run the case again. ## Related topics - [API overview](/en/api-reference/overview.md) - [Home](/en/configuration/agent-home.md) - [Get Test Suite summary](/en/api-reference/get-test-suite-summary.md) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.