The DAO soft-fork try was troublesome. Not solely did it prove that we underestimated the unintended effects on the consensus protocol (i.e. DoS vulnerability), however we additionally managed to introduce a knowledge race into the rushed implementation that was a ticking time bomb. It was not ultimate, and though averted on the final occasion, the quick approaching hard-fork deadline appeared eerily bleak to say the least. We wanted a new technique…
The stepping stone in the direction of this was an concept borrowed from Google (courtesy of Nick Johnson): writing up a detailed postmortem of the occasion, aiming to evaluate the foundation causes of the difficulty, focusing solely on the technical points and acceptable measures to forestall recurrence.
Technical options scale and persist; blaming folks doesn’t. ~ Nick
From the postmortem, one attention-grabbing discovery from the attitude of this weblog publish was made. The soft-fork code inside [go-ethereum]( strong from all views: a) it was totally lined by unit checks with a 3:1 test-to-code ratio; b) it was totally reviewed by six basis builders; and c) it was even manually reside examined on a non-public community… But nonetheless, a deadly knowledge race remained, which might have doubtlessly brought on extreme community disruption.
It transpired that the flaw might solely ever happen in a community consisting of a number of nodes, a number of miners and a number of blocks being minted concurrently. Even when all of these eventualities held true, there was solely a slight likelihood for the bug to floor. Unit checks can not catch it, code reviewers could or could not catch it, and handbook testing catching it could be unlikely. Our conclusion was that the event groups wanted extra instruments to carry out reproducible checks that may cowl the intricate interaction of a number of nodes in a concurrent networked situation. With out such a software, manually checking the assorted edge circumstances is unwieldy; and with out doing these checks constantly as a part of the event workflow, uncommon errors would grow to be not possible to find in time.
And thus, hive was born…
What’s hive?
Ethereum grew massive to the purpose the place testing implementations turned a enormous burden. Unit checks are high-quality for checking varied implementation quirks, however validating that a consumer conforms to some baseline high quality, or validating that shoppers can play properly collectively in a multi consumer setting, is all however easy.
Hive is supposed to function an simply expandable take a look at harness the place anybody can add checks (be these easy validations or community simulations) in any programming language that they’re snug with, and hive ought to concurrently be capable of run these checks towards all potential shoppers. As such, the harness is supposed to do black field testing the place no consumer particular inner particulars/state might be examined and/or inspected, somewhat emphasis could be placed on adherence to official specs or behaviors underneath totally different circumstances.
Most significantly, hive was designed from the bottom as much as run as a part of any shoppers’ CI workflow!
How does hive work?
Hive’s physique and soul is [docker]( Each consumer implementation is a docker picture; each validation suite is a docker picture; and each community simulation is a docker picture. Hive itself is an all encompassing docker picture. That is a very highly effective abstraction…
Since Ethereum clients are docker photos in hive, builders of the shoppers can assemble the absolute best setting for their shoppers to run in (dependency, tooling and configuration sensible). Hive will spin up as many cases as wanted, all of them operating in their very own Linux programs.
Equally, as test suites validating Ethereum shoppers are docker photos, the author of the checks can use any programing setting he’s most acquainted with. Hive will guarantee a consumer is operating when it begins the tester, which may then validate if the actual consumer conforms to some desired habits.
Lastly, network simulations are but once more outlined by docker photos, however in comparison with easy checks, simulators not solely execute code towards a operating consumer, however can truly begin and terminate shoppers at will. These shoppers run in the identical digital community and might freely (or as dictated by the simulator container) join to one another, forming an on-demand non-public Ethereum community.
How did hive help the fork?
Hive is neither a substitute for unit testing nor for thorough reviewing. All present employed practices are important to get a clean implementation of any characteristic. Hive can present validation past what’s possible from a mean developer’s perspective: operating in depth checks that may require advanced execution environments; and checking networking nook circumstances that may take hours to arrange.
Within the case of the DAO hard-fork, past all of the consensus and unit checks, we wanted to make sure most significantly that nodes partition cleanly into two subsets on the networking stage: one supporting and one opposing the fork. This was important because it’s not possible to foretell what antagonistic results operating two competing chains in a single community might need, particularly from the minority’s perspective.
As such we’ve carried out three particular community simulations in hive:
-
The first to examine that miners operating the complete Ethash DAGs generate right block extra-data fields for each pro-forkers and no-forkers, even when attempting to naively spoof.
-
The second to confirm that a community consisting of combined pro-fork and no-fork nodes/miners appropriately splits into two when the fork block arrives, additionally sustaining the break up afterwards.
-
The third to examine that given an already forked community, newly becoming a member of nodes can sync, quick sync and light-weight sync to the chain of their selection.
The attention-grabbing query although is: did hive truly catch any errors, or did is simply act as an additional affirmation that all the things’s all proper? And the reply is, each. Hive caught three fork-unrelated bugs in Geth, however additionally closely aided Geth’s hard-fork growth by constantly offering suggestions on how adjustments affected community habits.
There was some criticism of the go-ethereum group for taking their time on the hard-fork implementation. Hopefully folks will now see what we have been as much as, whereas concurrently implementing the fork itself. All in all, I consider hive turned out to play fairly an vital function within the cleanness of this transition.
What’s hive’s future?
The Ethereum GitHub group options [4 test tools already]( with at the very least one EVM benchmark software cooking in some exterior repository. They aren’t being utilised to their full extent. They’ve a ton of dependencies, generate a ton of junk and are very sophisticated to make use of.
With hive, we’re aiming to combination all the assorted scattered checks underneath one common consumer validator that has minimal dependencies, might be prolonged by anybody, and might run as a part of the day by day CI workflow of consumer builders.
We welcome anybody to make a contribution to the mission, be that including new shoppers to validate, validators to check with, or simulators to search out attention-grabbing networking points. Within the meantime, we’ll attempt to additional polish hive itself, including help for operating benchmarks in addition to mixed-client simulations.
With a bit or work, possibly we’ll even have help for operating hive within the cloud, permitting it to run community simulations at a way more attention-grabbing scale.
