Benchmarking
Benchmarking is a robust, standardized methodology for Knowledge Discovery that evaluates system performance using user-provided prompts and corresponding reference answers, along with automated scoring. This approach ensures that the evaluation is tailored to the defined information needs and expectations, providing a more relevant and accurate measure of system effectiveness. Dashboards display results, providing the user with the opportunity to set the new answer as a reference for further benchmark runs. A key prerequisite for effective benchmarking is that the knowledge base must be pre-populated using an appropriate chunking strategy. The choice of chunking strategy directly impacts the relevance and quality of retrieved information, making it a foundational aspect for any benchmarking process. You can create benchmark tests and run them against a knowledge base and compare the results.