› Benchmarking › Tests › Create benchmark tests

Create benchmark tests

Use the following procedure to create benchmark Tests.

Before you create a benchmark test, ensure at least one prompt is created.

Procedure
  1. Navigate to Benchmarking > Tests.
    The Tests page appears.
  2. Select New.

    The Create benchmark test dialog box is displayed.

  3. In the Test name box, enter a test name that clearly identifies the purpose of this benchmark test.
  4. Provide a Description for your benchmark test.
  5. Select the Category to organize your tests.

    By default, the category is set to your working category. You can select a different category from the Category list.

  6. The list of prompts that are already created appear. Select prompts for your test. You can select individual prompts, select all prompts at once, search the prompts or filter by category or tags to quickly add groups of related prompts.

    You must select at least one prompt.

  7. In the AI Knowledge Base list, select the configured knowledge base. See Integrate TotalAgility with AI Knowledge Base.

    The filter and content fields that are added in the selected AI knowledge base integration become available.

  8. In the Index list, select the index to search within the selected knowledge base.
  9. In the Filter operator list, select the logical operator for your filters.
  10. In the Filter field list, define filters to narrow down your search results.
  11. In the Content field list, select one or more content fields to include in the search results.
  12. To override the maximum number of matching results to return, select Override max number of matches and specify the count. (Default: 10)
  13. By default, the query type set in the AI knowledge base integration appears on the Override query type list. You can select a different query type to override the one set at the integration level.

    The selected query type is used in the benchmark test. Based on the selected query type, the search engine retrieves results from your index.

  14. Select Use response profile and enter the text, such as "Summarize the response in 200 words or less."
  15. Select Use seed and select the seed value.

    The seed value helps make results more consistent across repeated requests.

  16. Under Execution options, select an option to execute the benchmark test.
  17. Select Save and run or Save depending on the selected execution option.

    The benchmark test appears on the Tests page with the following details:

    • A summary card with the following metrics.

      • Total tests: Number of tests in the system.

      • Running: Tests that are currently in progress.

      • Completed: Tests that have finished successfully.

      • Scheduled: Tests that are scheduled to run later

    • A table that displays all tests by default, regardless of the status.

    • A Search box where you can search for a specific test by entering the name and then selecting . You can filter the tests by category and status (Running, Completed, Scheduled, Canceled, and Suspended.

More info