Create AI benchmark tests
Use the following procedure to create benchmark AI Tests.
Before you create a benchmark AI test, ensure at least one prompt is created.
-
Navigate to
.
The AI Benchmark Tests page appears.
-
Select
New.
The Create benchmark test dialog box is displayed.
- In the Test name box, enter a test name that clearly identifies the purpose of this benchmark test.
- Provide a Description for your benchmark test.
-
Select the
Category to organize your tests.
By default, the category is set to your working category. You can select a different category from the Category list.
-
The list of
AI prompts that are already created
appear.
Select prompts for your test. You can select individual prompts, select all
prompts at once, search the prompts or filter by category or tags to quickly add groups of related prompts.
You must select at least one prompt.
- In the Generative AI provider list, select the configured Generative AI provider to use for generating the expecetd results. See Integrate TotalAgility with Generative AI provider.
-
Select
Use seed and select the seed value.
The seed value helps make results more consistent across repeated requests.
- Under Execution options, select an option to execute the benchmark test.
-
Select
Save and run or
Save depending on the selected execution option.
The benchmark test appears on the Tests page with the following details:
-
A summary card with the following metrics.
-
Total tests: Number of tests in the system.
-
Pending: The number of tests pending.
-
Scheduled: Tests that are scheduled to run later.
-
Running: Tests that are currently in progress.
-
Completed: Tests that have finished successfully.
-
-
A table that displays all tests by default, regardless of the status.
-
A Search box where you can search for a specific test by entering the name and then selecting . You can filter the tests by category and status (Running, Completed, Scheduled, Canceled, and Suspended).
-