Create AI prompts

Create and manage prompts with expected answers.

Procedure

Navigate to Benchmarking > AI Prompts.
The AI Prompts page appears.
Select New.

The New prompt dialog box is displayed.
Enter a unique Name for to identify your prompt.
In the Prompt box, enter your prompt question in detail that will be used for benchmarking. Prompts should be comprehensive, specific, and significant in length.
Select the Category to organize your prompts.

By default, the category is set to your working category. You can select a different category from the Category list.
In the Tags box, enter a tag for your prompt and press Enter to add tag.
Add multiple tags as needed.

For Expected results, do either of the following:

Option	Description
Manually enter expected answer.	In the Expected answer box, enter the expected answer that should be returned for your prompt. Specify Expected tokens for executing this prompt.
Automatically generate the answers.	Select Auto populate. The AI Auto Populate Configuration dialog box appears. In the Generative AI provider list, select the Generative AI provider configuration to use for generating expected results. If you select Tungsten provider, on the model list, select a Tungsten AI model to use for benchmarking. This model overrides the system-configured model without affecting production. Select Use seed and select the seed value. Set the Temperature. Select Run Auto populate. The system does the following: Runs your prompt against the selected Generative AI provider. It uses the AI response to automatically populate the expected answer and reference documents, saving you time in creating comprehensive benchmark data; however, you can still edit or remove them manually afterward. The results are auto-populated in the following boxes: Expected answer Generated from retrieved content. This information is used as the reference answer for comparison during benchmarking. If the documents are not available, a message appears stating "The information you requested is not available in the retrieved documents." Expected references: Retrieved from ranked retrieval results. Expected tokens: Estimated from auto-generated expected answer and not pulled from documents.

Option

Description

Manually enter expected answer.

In the Expected answer box, enter the expected answer that should be returned for your prompt.
Specify Expected tokens for executing this prompt.

Automatically generate the answers.

Select Auto populate.

The AI Auto Populate Configuration dialog box appears.
In the Generative AI provider list, select the Generative AI provider configuration to use for generating expected results. If you select Tungsten provider, on the model list, select a Tungsten AI model to use for benchmarking. This model overrides the system-configured model without affecting production.
Select Use seed and select the seed value.
Set the Temperature.
Select Run Auto populate.

The system does the following:
- Runs your prompt against the selected Generative AI provider. It uses the AI response to automatically populate the expected answer and reference documents, saving you time in creating comprehensive benchmark data; however, you can still edit or remove them manually afterward. The results are auto-populated in the following boxes:
  - Expected answer Generated from retrieved content.
    
    This information is used as the reference answer for comparison during benchmarking.
    
    If the documents are not available, a message appears stating "The information you requested is not available in the retrieved documents."
  - Expected references: Retrieved from ranked retrieval results.
  - Expected tokens: Estimated from auto-generated expected answer and not pulled from documents.

Select either of the following:

Save	Saves the prompt and displays it on the Prompts page.
Save and add another	Saves the prompt and the New prompt dialog box is displayed for creating another prompt.

The prompt appears on the Prompts page with the following details:

A summary card with the following metrics.
- Total prompts: Number of prompts in the system.
- Tagged: Prompts that have metadata tags assigned.
A Search box where you can search for a specific prompt by entering the form name and then selecting . You can filter the prompts by category and tags.
A table that displays the prompt name, prompt text, the tags assigned to the prompt, the user who last modified the tag, the date it was last updated, and actions to edit and delete the prompt.

More info

Maintain prompts