Surely the developers must know what they're doing? Surely the smartest of the smart are working on it, and reviewing it. Surely they must have checks in place, to double and triple check every line of code added. The answers to all of those are YES. New features regularly take years to be rolled out, where people are running the code in testbeds for long long times before public release.
And yet, last week a bug so bad was found, it could have brought down the whole system.
https://twitter.com/jfnewbery
Of course, this developer wasn't REALLY responsible. He didn't write the original code. The original code was written by one of the main developers:I am responsible for the CVE-2018-17144 bug.
I spend my days reading the Bitcoin Core codebase. There's no chance I haven't read CheckTransaction(). When I read it, the "...so we skip it in CheckBlock" comment should have jumped out at me.
That comment and the fCheckDuplicateInputs flag don't just smell, they stink. I should have followed my nose. At the very least I should have looked up Bitcoin Core PR #9049. I didn't.
Instead of verifying for myself, I trusted that people smarter and wiser than I am had it covered. I took it for granted that someone else had done the work.
Last week I was found short in my knowledge and in my judgement. I'm embarrassed and sorry
https://github.com/bitcoin/bitcoin/pull/9049
Nor was his duty to be a reviewer. That was done by other smart people. But he IS a regular contributor. He overlooked his intuition because he assumed the original author, and the reviewers, and everyone else, already had it covered.