Four frontier models re-ran 5 unpublished ack3 audits in one harness, recovering 30, 23, 16, and 11 of the reports' 32 critical and high findings.
Glossary definition
Recall
Recall is the number of issues a method discovered out of the total number of issues. A recall of 30 of 32 means two issues were missed.
Recall is the number of issues a method discovered out of the total number of issues. A recall of 30 of 32 means the method discovered 30 issues and missed two.
Recall ignores noise: reporting incorrect findings costs nothing on this measure. That cost is measured by precision. Our private benchmark measures recall on unpublished audits, so no evaluated model has trained on the code or the reports.
Used in context
Articles using this term.