Extraction benchmark comparison
Once you save two or more extraction benchmarks, compare them to see if your modifications improve or degrade the extraction quality. The comparison tab is separated into a comparison of the summary statistics as well as a comparison of the detailed statistics.
This enables you to look at the differences between the benchmarks and clearly see where you gained or lost extraction quality.
If you only have two benchmarks saved, these are automatically loaded into the comparison tab. If one is set to be the baseline, it is displayed on the left while the other is displayed on the right. If you have multiple benchmarks saved, the baseline is loaded automatically on the left and you select the other benchmark to view from the list.
The Summary table shows the overall results for each field and each extraction result displayed in the legend. Each cell contains the result from the non-baseline extraction, and the difference in percentage between baselines is displayed in brackets.
The following color scheme explains the results:
-
If the number of correct valid fields (A) decreases, the field has red highlighting.
-
If the number of incorrect valid fields (D) increases, the field has red highlighting.
-
If the number of correct valid fields (A) increases, the field has green highlighting.
-
If the number of incorrect valid fields (D) decreases, the field has green highlighting.
-
If the values between the benchmark results are the same, the field has no highlighting.
-
Changes in correct invalid fields (B) and incorrect invalid fields (C) are displayed in the cell, but the field has no highlighting.
The Details Statistics Comparison table compares information about each field and document. In each summary column cells (A, B, C, and D), the non-baseline extraction result is displayed and the difference in percentage between it and the baseline is displayed in brackets.
The following information is provided:
- Legend:
-
This group explains the four different result types.
-
Correct valid fields
-
Correct invalid fields
-
Incorrect invalid fields
-
Incorrect valid fields
-
- Benchmark:
-
This group has the following settings:
- Select Benchmark:
-
Select the desired benchmark file from the list.
- Date:
-
The date and time the benchmark file was saved. This field is read-only.
- Precision/Recall:
-
The precision and recall values saved in the benchmark file. This field is read-only.
- Comment:
-
The comment added when the benchmark is saved. This field is read-only.
- Baseline:
-
If selected, all other benchmarks are compared to the results in this benchmark file. Clear it to set another file as the baseline.
- Delete:
-
Deletes the benchmark file from the project.
- Summary Statistics Comparison:
-
Displays a table with the differences between the summary information in the two benchmarks.
- Details Statistics Comparison:
-
Displays the differences between the detailed information in the two benchmarks.