Making sense of EOS stops producing blocks

Info is a bit scarse so i’ve tried to make sense of what happened yesterday with EOS block production halting (please add/ correct if i’m wrong). Yesterday the EOS network stopped producing blocks, freezing on block #1027926.

This is a major fault and an interesting case study. Why did EOS stop producing blocks? it seems its all the cause of a small commit to the EOS code pushed 8 days ago (June 9th), check it here.

lines 809–813 in controller.cpp are in fault

It seems the commit had the goal of throwing an exception if a scheduled (deferred) transaction had failed to be pushed into a queue for processing. A normal transaction that fails to be pushed should cause the block to except but, as EOS stated in documentation, a deferred transaction is not guaranteed to execute yet alone successfully.

The fix that made EOS continue producing blocks is as simple as the fix that caused it. Actualy there are 2 seperate fix commits.

Bottom line is, new blocks were failing because the check that determines if a block is valid or not was broken. Any block producer who upgraded couldn’t produce any blocks because they were marked as in-valid.

CEO @ and Developing trustless staking products for eth2.0.

CEO @ and Developing trustless staking products for eth2.0.