Solidity Optimizer and ABIEncoderV2 Bug

Solidity Optimizer and ABIEncoderV2 Bug Announcement

By the Ethereum bug bounty program, we obtained a report a couple of flaw inside the new experimental ABI encoder (known as ABIEncoderV2). Upon investigation, it was discovered that the part suffers from a couple of completely different variations of the identical sort. The primary a part of this announcement explains this bug intimately. The brand new ABI encoder remains to be marked as experimental, however we however assume that this deserves a outstanding announcement since it’s already used on mainnet.

Moreover, two low-impact bugs within the optimizer have been recognized over the previous two weeks, one among which was mounted with Solidity v0.5.6. Each have been launched with model 0.5.5. See the second a part of this announcement for particulars.

The 0.5.7 release incorporates the fixes to all bugs defined on this weblog submit.

All of the bugs talked about right here ought to be simply seen in checks that contact the related code paths, a minimum of when run with all combos of zero and nonzero values.

Credit to Melonport staff (Travis Jacobs & Jenna Zenk) and the Melon Council (Nick Munoz-McDonald, Martin Lundfall, Matt di Ferrante & Adam Kolar), who reported this by way of the Ethereum bug bounty program!

Who ought to be involved

When you’ve got deployed contracts which use the experimental ABI encoder V2, then these could be affected. Which means that solely contracts which use the next directive inside the supply code could be affected:

pragma experimental ABIEncoderV2;

Moreover, there are a variety of necessities for the bug to set off. See technical particulars additional beneath for extra info.

So far as we will inform, there are about 2500 contracts reside on mainnet that use the experimental ABIEncoderV2. It’s not clear what number of of them comprise the bug.

Easy methods to verify if contract is weak

The bug solely manifests itself when the entire following circumstances are met:

• Storage knowledge involving arrays or structs is distributed on to an exterior operate name, to abi.encode or to occasion knowledge with out prior project to an area (reminiscence) variable AND
• there may be an array that incorporates parts with measurement lower than 32 bytes or a struct that has parts that share a storage slot or members of sort bytesNN shorter than 32 bytes.

Along with that, within the following conditions, your code is NOT affected:

• if all of your structs or arrays solely use uint256 or int256 varieties
• for those who solely use integer varieties (which may be shorter) and solely encode at most one array at a time
• for those who solely return such knowledge and don’t use it in abi.encode, exterior calls or occasion knowledge.

When you’ve got a contract that meets these circumstances, and wish to confirm whether or not the contract is certainly weak, you may attain out to us by way of safety@ethereum.org.

Easy methods to stop these kinds of flaws sooner or later

With a view to be conservative about adjustments, the experimental ABI encoder has been obtainable solely when explicitly enabled, to permit individuals to work together with it and take a look at it with out placing an excessive amount of belief in it earlier than it’s thought of steady.

We do our greatest to make sure top quality, and have not too long ago began engaged on ‘semantic’ fuzzing of sure elements on OSS-Fuzz (we’ve got beforehand crash-fuzzed the compiler, however that didn’t take a look at compiler correctness).

For builders — bugs inside the Solidity compiler are troublesome to detect with instruments like vulnerability detectors, since instruments which function on supply code or AST-representations don’t detect flaws which are launched solely into the compiled bytecode.

One of the simplest ways to guard in opposition to these kinds of flaws is to have a rigorous set of end-to-end checks to your contracts (verifying all code paths), since bugs in a compiler very seemingly usually are not “silent” and as an alternative manifest in invalid knowledge.

Doable penalties

Naturally, any bug can have wildly various penalties relying on this system management circulate, however we anticipate that that is extra prone to result in malfunction than exploitability.

The bug, when triggered, will below sure circumstances ship corrupt parameters on technique invocations to different contracts.

Timeline

2019-03-16:

• Report by way of bug bounty, about corruption precipitated when studying from arrays of booleans immediately from storage into ABI encoder.

2019-03-16 to 2019-03-21:

• Investigation of root trigger, evaluation of affected contracts. An unexpectedly excessive rely of contracts compiled with the experimental encoder have been discovered deployed on mainnet, many with out verified source-code.
• Investigation of bug discovered extra methods to set off the bug, e.g. utilizing structs. Moreover, an array overflow bug was present in the identical routine.
• A handful of contracts discovered on Github have been checked, and none have been discovered to be affected.
• A bugfix to the ABI encoder was made.

2019-03-20:

• Determination to make info public.
• Reasoning: It might not be possible to detect all weak contracts and attain out to all authors in a well timed method, and it might be good to stop additional proliferation of weak contracts on mainnet.

2019-03-26:

• New compiler launch, model 0.5.7.
• This submit launched.

Technical particulars

Background

The Contract ABI is a specification how knowledge could be exchanged with contracts from the skin (a Dapp) or when interacting between contracts. It helps a wide range of sorts of knowledge, together with easy values like numbers, bytes and strings, in addition to extra advanced knowledge varieties, together with arrays and structs.

When a contract receives enter knowledge, it should decode that (that is finished by the “ABI decoder”) and previous to returning knowledge or sending knowledge to a different contract, it should encode it (that is finished by the “ABI encoder”). The Solidity compiler generates these two items of code for every outlined operate in a contract (and additionally for abi.encode and abi.decode). Within the Solidity compiler the subsystem producing the encoder and decoder is named the “ABI encoder”.

In mid-2017 the Solidity staff began to work on a recent implementation named “ABI encoder V2” with the objective of getting a extra versatile, secure, performant and auditable code generator. This experimental code generator, when explicitly enabled, has been provided to customers because the finish of 2017 with the 0.4.19 launch.

The flaw

The experimental ABI encoder doesn’t deal with non-integer values shorter than 32 bytes correctly. This is applicable to bytesNN varieties, bool, enum and different varieties when they’re a part of an array or a struct and encoded immediately from storage. This implies these storage references have for use immediately inside abi.encode(…), as arguments in exterior operate calls or in occasion knowledge with out prior project to an area variable. Utilizing return doesn’t set off the bug. The kinds bytesNN and bool will lead to corrupted knowledge whereas enum may result in an invalid revert.

Moreover, arrays with parts shorter than 32 bytes is probably not dealt with accurately even when the bottom sort is an integer sort. Encoding such arrays in the way in which described above can result in different knowledge within the encoding being overwritten if the variety of parts encoded just isn’t a a number of of the variety of parts that match a single slot. If nothing follows the array within the encoding (word that dynamically-sized arrays are all the time encoded after statically-sized arrays with statically-sized content material), or if solely a single array is encoded, no different knowledge is overwritten.

Unrelated to the ABI encoder problem defined above, two bugs have been discovered within the optimiser. Each have been launched with 0.5.5 (launched on fifth of March). They’re unlikely to happen in code generated by the compiler, except inline meeting is used.

These two bugs have been recognized by the latest addition of Solidity to OSS-Fuzz – a safety toolkit for locating discrepancies or points in a wide range of tasks. For Solidity we’ve got included a number of completely different fuzzers testing completely different elements of the compiler.

1. The optimizer turns opcode sequences like ((x << a) << b)), the place a and b are compile-time constants, into (x << (a + b)) whereas not dealing with overflow within the addition correctly.
2. The optimizer incorrectly handles the byte opcode if the fixed 31 is used as second argument. This could occur when performing index entry on bytesNN varieties with a compile-time fixed worth (not index) of 31 or when utilizing the byte opcode in inline meeting.

This submit was collectively composed by @axic, @chriseth, @holiman