Benchmarking

Benchmarking is a robust, standardized methodology for evaluating system performance through AI benchmarking and AI Knowledge Base (KB) benchmarking, using user-provided prompts, corresponding reference answers, and automated scoring. AI benchmarking evaluates the performance of the generative AI model, while AI Knowledge Base benchmarking evaluates how effectively the AI retrieves and uses information from a specific knowledge base to generate accurate and relevant responses. This approach ensures that evaluation is aligned with defined information needs and expectations, providing a measurable assessment of system effectiveness. Dashboards display the results, enabling users to analyze performance and set a generated answer as a reference for subsequent benchmark runs. For AI Knowledge Base benchmarking, the knowledge base is pre-populated using an appropriate chunking strategy, as the choice of chunking strategy directly impacts the relevance and quality of retrieved information. You can create benchmark tests, run them against an AI model or knowledge base, and compare the results.

• Create KB prompts

• Maintain KB prompts

• Create KB benchmark tests

• Execute KB benchmark tests

• View KB benchmark test results

• Maintain KB benchmark tests

• Create AI prompts

• Maintain AI prompts

• Create AI benchmark tests

• Execute AI benchmark tests

• View AI benchmark test results

• Maintain AI benchmark tests

• Compare AI benchmark tests

• Import a benchmark test

• Export a benchmark test