Four frontier models re-ran 5 unpublished ack3 audits in one harness, recovering 30, 23, 16, and 11 of the reports' 32 critical and high findings.
Glossary definition
Precision
Precision is the number of correct findings out of everything a method reported; incorrect findings lower it and cost reviewer time to reject.
Precision is the number of correct findings out of everything a method reported. Every incorrect finding lowers it and consumes reviewer time to check and reject.
Precision and recall pull against each other: aggressive filtering raises precision and risks recall. A benchmark result is complete only when it states which of the two it measures; our private benchmark measures recall and says so.
Used in context
Articles using this term.
Learn how Balancer’s StableSwap math failed and how differential fuzzing in Python detects precision bugs in DeFi swap logic for safer smart contract design.