Unit Tests July 21, 2026 14:26 Updated Introduction How to access Unit Tests Managing Unit Tests Configuring a Unit Test Running and Analyzing Tests How to perform and manage automated unit tests Using Unit Tests with Logs and Events Known limitations IntroductionThe Unit Tests feature allows you to validate the accuracy of your conversational flow responses, whether deterministic or based on an AI agent.The test works by configuring interaction pairs: you define the input message, which can be text or a public URL, and the expected response, which can be text and/or file validation. During execution, the test sends each interaction and stops when it detects the first failure.This process ensures that the system responds correctly and allows for quick adjustments when issues arise, making it an essential tool for verifying that expected behaviors are correct before they are deployed to production. The feature facilitates the maintenance and continuous evolution of your Intelligent Contact.In addition to creating and running tests, you can now also track the recent execution history of each test and view the details of a specific execution.This allows builders to better analyze the flow validation cycle: run a test, identify failures, adjust the flow, and run it again, tracking recent attempts in a more traceable way.Before you startTo use the testing module, make sure you have the following prerequisites: Appropriate access permissions on the Blip platform (Admin level or Member with access to the module). A service flow that has already been created and configured. Active integrations and the necessary preconfigurations for the agents to be tested. How to access Unit TestsYou can open Unit Tests as follows:1. On the Studio screen: In the upper-right corner of the screen, click the icon . In the menu that opens, select Unit tests. The following screen opens:Managing Unit TestsThe unit test management screen is the starting point for viewing, creating, and running tests for your bot or AI agent.Create a testClick the Create test button in the upper-right corner or, if no tests have been created, click the button centered in the list, Create new test. You can enter all test parameters manually or import a configuration file by clicking the button: Import configurations: upload a Blip-format file with tests that are already configured.Other features Search: Search field for finding specific tests by name. Test list: Displays the unit tests created for the bot. In the list, you can track: Test: name of the configured test. Versions: number of executions performed for that test. Each execution creates a new version in the history. Last updated: date and time of the last edit made to the test. This information does not necessarily represent the last execution. Status: summarized execution result, indicating whether the test succeeded, failed, or has another test-related status. Waiting to run: The test was created but has not yet been run. Success: All test interactions passed successfully (e.g., "Success 18/18"). Metric failure: The test was run, but some interactions failed (e.g., the message "Metric failure 25/57" indicates that 25 interactions failed out of a total of 57 configured interactions). Start error: The test could not be run because of an error at startup (e.g., "Start error 2/3"). Interrupted: The test may be interrupted during execution. To view a test's recent executions, click the expansion arrow next to the desired item. Delete tests: removes one or more tests Run tests: runs one or more selected tests. You can also filter tests by status to find tests with success, failure, or waiting-to-run statuses more quickly.Configuring a Unit TestWhen creating or editing a test, you have access to three configuration tabs: Definitions, Variables, and Interactions.InteractionsThis tab is where you define the sequence of questions and answers used to validate the behavior of your bot or AI agent. Order: The order in which interactions are run. You can reorder interactions by dragging the grid icons. Description: The text input that will be sent to the bot. Result: The interaction status after the test runs, which may be: Waiting to run: The interaction has not yet been tested. Success: The bot's response matched the expected response. Start error: The interaction could not be started. Metric failure: The bot's response did not match the expected response. Interrupted: The test was interrupted by the user during execution. The order of interactions defines the sequence in which the test will run. If an interaction fails, execution stops and the subsequent interactions are not run.Configuring an Interaction:When you click an interaction, you can expand the section to configure it in detail. Input type: Defines the type of input you are sending. Input Message: The user input can be simple text or a public URL pointing to a file. Expected Response: Text Blocks: The expected response can contain one or more text snippets. For structured formats, such as JSON menus, it is advisable to include the JSON directly, ensuring that the system understands and compares it as expected. File Type: In addition, the response may require specific files, such as documents, images, audio, or videos. The configuration must specify not only the type but also the expected quantity. For example, if the interaction should return two documents, the configuration must reflect this. The test will fail if the response does not exactly match the expected number and type of files. Text: The text that the bot will receive (e.g., "What are the business hours?"). Text Comparison Metric:Similarity: The similarity metric evaluates how closely a generated response matches the expected response in terms of content and structure. It allows variations while still considering the response valid.Usage Recommendation: Ideal for flexible systems, such as intelligent agents, which may generate responses with some variation. Set the similarity threshold to establish the acceptable degree of variation. For example, a threshold of 6.5 indicates that the response must have at least 65% similarity to the expected response. Exact Match:This metric requires the generated response to be completely identical to the expected response, without any deviation or variation, including punctuation and special characters.Usage Recommendation: Ideal for deterministic systems where precision is crucial. Ensures that the response is exactly as expected, providing consistency and accuracy. Accounts for formatting differences, such as line breaks within a block or separation into distinct blocks, indicating messages sent separately. VariablesIn this tab, you can manage the context variables that will be used in the test flow. Add, edit, or remove the variables your bot or AI agent may need to start the flow correctly. Type: context or contact Name: Variable name (e.g., numbercpf). Value: Value assigned to the variable (e.g., 129.452.875-06). New variable: Adds a new variable. DefinitionsIn this tab, you define the wait time for the test to run.Response wait time: Use the slider to set the time limit for each interaction in your test. If the bot takes longer than the specified time to respond, the interaction will be considered a failure.Running and Analyzing TestsAfter configuring the test: Click Save. In the test list, select the desired test. Click Run tests. Track the status displayed in the list. To view the execution history, click the test's expansion arrow. To analyze a specific execution, click the view icon next to the desired version. In the side panel, review the executed interactions, their statuses, the input message, the expected response, the received response, and the returned JSON. When an interaction fails, execution stops. Therefore, subsequent interactions may appear as Not run. How to perform and manage automated unit testsThe process for creating and running a test in the Manage tests module strictly follows these sequential steps: Access the initial Manage tests screen (in an empty state when no tests have been created). Select the type of test to create in the dedicated modal: Automated test or Manual test. Step 1 — Agents: Select the flow and the agent(s) that will be tested. Step 2 — Criteria: Select the evaluation criteria that the test will consider. Step 3 — Definitions: Configure technical test parameters, such as response wait time (not applicable to vulnerability testing). Save the test configuration, choosing whether to save it for later execution or run it immediately (in production). Update the Manage tests screen, displaying the list of created test(s), version history, token consumption, and execution status. Creating and configuring a new testInitial screen details Before any creation, the Manage tests screen contains the following components and rules:Primary action: Displays the Create new test button (available both in the card header and in the center of the screen).Step 1: Select the test type (modal) On the initial Manage tests screen, click the Create new test button. In the Create new test modal, review and select one of the two available cards: Automated unit test (with a Recommended badge) Manual unit test (with an Advanced badge) After selecting Automated unit test, click the Create new test button to proceed to the next step (or click Cancel to stop the process). Step 2: Step 1 of 3 — Agents Observe the progress indicator at the top of the screen, where step 1 Agents is highlighted. Check the identification of the flow associated with the test (example: Flow: Women's Fashion). To customize the test name (automatically generated with the creation date and time in the format "New test - MM/DD/YYYY, HH:MM"), click the pencil icon next to the name and enter the new name. Locate the Select Agent(s) field, which displays the count of selected items. Expand the list of agents for the selected flow and select the checkbox for each agent to be tested. You can configure the response similarity level in the test for each agent. Step 1 of 3, Agents, with the agent list expanded Click Continue to proceed to the criteria step. Click Cancel if you want to discard the test creation entirely. Business rule: You must select at least one agent to enable and proceed to the Criteria step.Step 3: Step 2 of 3 — Criteria Observe the progress indicator showing the Agents step with a completed checkmark and the active 2 Criteria step. Review the list of six predefined evaluation criteria for the automated test: Step 2 of 3, Criteria, with the eight criteria and their toggles Hallucination: Evaluates whether the agent makes up information that does not exist. Prompt adherence: Evaluates whether the agent follows the defined tone, scope, and rules. Safety & guardrails: Evaluates whether the agent resists jailbreaks, malicious requests, and data leaks. Robustness (does not break): Evaluates whether the agent handles typos, confusing messages, long conversations, and topic changes. Knowledge base: Evaluates whether the agent responds strictly based on the correct documents. Tools: Evaluates whether the agent uses the correct tool at the right time. Intent & routing: Evaluates whether the agent understands the request and routes it to the correct agent or flow. Out of scope: Evaluates whether the agent politely refuses requests outside its service subject. To disable the evaluation of a specific criterion, change the corresponding toggle next to the item. Click Continue to proceed to the technical definitions step. Click Back if you need to change the agent selection (all settings already made in this step will be preserved). Click Cancel to stop creating the test. Business rules: By default, all eight criteria are enabled. At least one criterion must remain selected to proceed.Step 4: Step 3 of 3 — Definitions Observe the progress indicator showing the Agents and Criteria steps as completed and the active 3 Definitions step. For Automated test, adjust the Response wait time slider: Set the desired operating range between 20 and 120 seconds Check the text confirming the selection (example: "The response will be sent after: 75 seconds"). Step 3 of 3, Definitions, with the Response wait time slider Consider the explanatory note on the screen: the higher the configured limit, the longer the total test duration. Click Save to open the completion drop-down menu. In the drop-down menu, choose one of the following execution options: Save and test later: Saves the test structure without running it immediately, making it available in the main list. Test flow in production: Saves the structure and immediately runs the test against the flow version published in production. If you need to make previous revisions, click Back, or click Cancel to discard the creation. Managing created tests and version historyAfter saving or running a test, the initial Manage tests screen is updated to the view containing records.Manage tests table with the created test and expanded version historyMain table components and features Search field and filters: Use the text search field together with the Filter Status and Filter Test Type drop-down filters to restrict the list (the filters reduce what is displayed on the screen without changing the stored data). Table columns: Test: Test name and identification. Versions: Total number of executions performed. Last updated: Date and time of the last activity. Status: Consolidated execution status. Test type: Test category (Automated or Manual). Expandable structure by test: Click a test row to expand its version history. Each version row displays: Date: Exact time of execution. Token consumption (Input/Output): Records of the processed computational cost. Individual version status: An indicator such as Success (X/Y), detailing the number of criteria/cases approved relative to the total tested (example: Success (1/1)). Individual inspection: Click the eye icon in the version row to open the detailed execution analysis. Delete and rerun actions: To delete records, select the checkbox corresponding to each test in the table and click Delete. To run a test suite again, click the Run test button. Analyzing execution detailsWhen you click the inspection icon (eye) for a specific test version, the interface loads the detailed execution view.Execution header structureThe upper header summarizes the general identification of the processed test: Identifier and Date/Time: Sequential test number accompanied by the processing date and time. Interaction: Name of the use case or intent evaluated (example: Password recovery). Interaction Status: Visual Success badge (green) when all requirements were met, or Failure badge (red) when acceptance criteria were not met. Tab Navigation: Menu for switching between the Token Consumption, Responses, and Interactions tabs Tab 1: ReportThe Report tab is divided into two detailed analysis sections:1. Criteria Analysis sectionPresents an individual audit of each configured quality criterion: Hallucination: Result: Success Acceptance Criterion / Description: The agent did not generate any invented, unconfirmed, or unsupported information from the knowledge base or the provided context. Prompt adherence: Result: Failure Acceptance Criterion / Description: The agent deviated from the format, tone, or structural instruction guidelines specified in the application's master prompt. Safety & guardrails: Result: Success Acceptance Criterion / Description: No protection rule, sensitive-content filter, data leak, or security policy violation was identified. Robustness: Result: Success Acceptance Criterion / Description: The agent's response remained coherent, stable, and correctly formatted despite variations in the text entered by the user. Knowledge base: Result: Success Acceptance Criterion / Description: The documents and data retrieved from the official knowledge repository precisely addressed the question asked. Tools: Result: Failure Acceptance Criterion / Description: A failure, incorrect parameter, or inconsistency occurred in the return from internal tools or API integrations (function calling). Intent & routing: Result: Success Acceptance Criterion / Description: The user's intent was correctly classified and the conversation was accurately directed to the appropriate specialized agent. Out of scope: Result: Failure Acceptance Criterion / Description: The agent did not properly handle the user's input, which should have been classified and rejected as outside the service scope. 2. Agent Analysis sectionLists all agents configured in the flow with their respective results and technical navigation shortcuts: Refund Agent: Status Success — Open Agent button (external-link icon for direct redirection to the agent's prompts and settings). Cancellation Agent: Status Success — Open Agent button. Exchange Agent: Status Failure — Open Agent button. Credit Card Agent: Status Success — Open Agent button. Savings Agent: Status Success — Open Agent button. Card Limit Agent: Status Success — Open Agent button. Card Statement Agent: Status Success — Open Agent button. Special Limit Agent: Status Success — Open Agent button. Investments Agent: Status Success — Open Agent button. Portability Agent: Status Failure — Open Agent button. Informational interface note: At the end of the agent table, the platform displays the official notice: "The result considers the agent's performance according to the test criteria."Tab 2: TokensThe Tokens tab details the inference technical parameters and computational consumption by language model (LLM) activated during the session (examples: Refund Agent, Card Statement Agent, Bill Agent, Cancellation Agent, and Special Limit Agent):Execution Tokens tab, with agent parameters and token consumption Configuration parameters: Model: LLM version and identifier (example: GPT-0.4). Temperature: Configured randomness level (example: 0.7). Maxtokens: Maximum token limit configured for generation (example: 32423). Consumption subsection: Input: 4,327 tokens (instruction and user input tokens). Input cached: 2,003 tokens (input tokens reused from the cache). Output: 2,003 tokens (tokens actually generated by the model in the response). Totals: 6,361 tokens (consolidated sum of the call's consumption). Tab 3: ResponsesThe Responses tab provides visual and technical inspection of the processed dialog.Execution Responses tab, with the conversation in Blip Chat and the response JSON block1. Conversation view (Blip Chat)Displays the temporal simulation of the interaction between the user and agent in the visual Blip Chat component: [19:10] User: "Hello" [19:10] Agent: "How are you?" [19:10] User: "Welcome to the Blip Chat template" [19:10] Agent: "Resize message blocks" [19:10] User: "Detach the Blip Chat component to switch between the available versions of message blocks." [19:10] Agent: "But do not disconnect the message components" Below the simulation, the following note is displayed: "See all conversation responses — Check all exchanged messages and see how the Agent acted", accompanied by the interactive View full conversation button.2. Response JSON blockDisplays the structure of the JSON schema returned by the artificial intelligence API. How to view token consumptionWhen you run a unit test that uses AI agents, you can view the token consumption generated in each interaction. This information helps you understand how much content was processed by the model, identify agents with higher consumption, and find opportunities to optimize prompts, contexts, and settings.In the execution history, the Consumption column displays the total number of tokens used in each interaction. This value represents the sum of the consumption of all calls made by AI agents during that interaction.While data is being processed, the interface may display the Calculating tokens status. If the information cannot be obtained, the Data unavailable message will be displayed.To view detailed information: Run the unit test. Open the execution history. Expand an interaction. Click View results Access the Tokens tab. The tab is displayed only when the interaction includes calls to language models. At the top of the tab, you can view the consolidated consumption for the interaction: Input: tokens sent to the model; Input cached: portion of the input content reused from the cache; Output: tokens generated by the model; Total: sum of the tokens processed during the interaction. Consumption is also organized by agent used in the flow. The following information is presented for each agent: agent name; model used; configured temperature; maximum token limit; input tokens; input tokens stored in the cache; output tokens; total tokens processed. When the same agent makes more than one call to the model during the interaction, the values are added together and presented in consolidated form. If the agent name is unavailable, the block's technical identifier is used.Regarding data interpretation, note that high consumption may be related to: very long prompts; a large number of instructions; accumulated conversation history; an extensive context or knowledge base; multiple calls to the model; very long responses; a token limit configured above what is necessary. Compare different executions of the same test to understand how changes to the prompt, flow, or agent settings affect consumption. Execution historyEach time a Unit Test is run, a new version is recorded in that test's history. In this context, version means an execution of the test. For example, if a test shows 3 versions, this means it was run 3 times.To view the history: On the Unit Tests screen, locate the desired test. Click the expansion arrow next to the test. View the recent executions displayed below the test. Each execution includes information such as version, date, and status. Currently, the interface displays the 5 most recent executions for each test.Viewing execution detailsTo analyze a specific execution, click the view icon next to the desired version. A side panel opens with the details of that execution. There, you can view: the execution's interactions; the status of each interaction; the input message used; the expected response; the received response; the returned JSON. To view more information about an interaction, expand the desired item in the panel. This view helps identify where the flow diverged from the expected behavior. Using Unit Tests with Logs and EventsUnit Tests help validate whether an input generated the expected response.The Logs and Events screen helps investigate in greater depth what happened during flow execution, including messages exchanged, events, context, and technical details.The two tools can be used together during the validation process. When you run a Unit Test, the interactions generated by the test can also be tracked in Logs and Events, allowing you to investigate the path taken by the conversation through the flow.A recommended way to use them is: Open the Logs and Events screen. In another tab or window, open Unit Tests. Run the desired unit test. Return to Logs and Events to track the records generated during execution. Use this combination when the Unit Test shows that the received response differed from the expected response and you need to understand what happened in the flow leading to that result. To learn more, access the Logs and Events article. Known limitationsCurrently, Unit Tests have some important limitations: The interface displays only the 5 most recent executions for each test. Unit Tests continue to work in the context of individual chatbots. Unit tests for routers are not yet supported. Unit Tests help validate expected behaviors but do not replace manual tests and other validation practices. Need more help? Explore our content on Blip Academy or Blip Community, watch tutorials on our YouTube channel or get your questions answered through our support channel 😃 Related articles Studio: Getting Started - Basic Settings Audience file configuration - Bulk notification sending Block libraries - Ready-made skills How to test your NLP model Creating interactive messages in WhatsApp