Conference PaperPDF Available

Echidna: effective, usable, and fast fuzzing for smart contracts

Authors:
  • Trail of Bits

Figures

Content may be subject to copyright.
Echidna: Eective, Usable, and Fast Fuzzing for Smart Contracts
Gustavo Grieco
Trail of Bits, USA
Will Song
Trail of Bits, USA
Artur Cygan
Trail of Bits, USA
Josselin Feist
Trail of Bits, USA
Alex Groce
Northern Arizona University, USA
ABSTRACT
Ethereum smart contracts—autonomous programs that run on a
blockchain—often control transactions of nancial and intellectual
property. Because of the critical role they play, smart contracts need
complete, comprehensive, and eective test generation. This paper
introduces an open-source smart contract fuzzer called Echidna that
makes it easy to automatically generate tests to detect violations in
assertions and custom properties. Echidna is easy to install and does
not require a complex conguration or deployment of contracts
to a local blockchain. It oers responsive feedback, captures many
property violations, and its default settings are calibrated based on
experimental data. To date, Echidna has been used in more than
10 large paid security audits, and feedback from those audits has
driven the features and user experience of Echidna, both in terms of
practical usability (e.g., smart contract frameworks like True and
Embark) and test generation strategies. Echidna aims to be good at
nding real bugs in smart contracts, with minimal user eort and
maximal speed.
CCS CONCEPTS
Software and its engineering Dynamic analysis
;
Soft-
ware testing and debugging.
KEYWORDS
smart contracts, fuzzing, test generation
ACM Reference Format:
Gustavo Grieco, Will Song, Artur Cygan, Josselin Feist, and Alex Groce.
2020. Echidna: Eective, Usable, and Fast Fuzzing for Smart Contracts. In
Proceedings of the 29th ACM SIGSOFT International Symposium on Software
Testing and Analysis (ISSTA ’20), July 18–22, 2020, Virtual Event, USA. ACM,
New York, NY, USA, 4pages. https://doi.org/10.1145/3395363.3404366
1 INTRODUCTION
Smart contracts for the Ethereum blockchain [
5
], usually written
in the Solidity language [
25
], facilitate and verify high-value nan-
cial transactions, as well as track physical goods and intellectual
property. Thus, it is essential that these programs be correct and
secure, which is not always the case [
4
]. Recent work surveying
and categorizing aws in critical contracts [
11
] established that
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specic permission
and/or a fee. Request permissions from permissions@acm.org.
ISSTA ’20, July 18–22, 2020, Virtual Event, USA
©2020 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 978-1-4503-8008-9/20/07. . . $15.00
https://doi.org/10.1145/3395363.3404366
fuzzing using custom user-dened properties might detect up to
63% of the most severe and exploitable aws in contracts. This sug-
gests an important need for high-quality, easy-to-use fuzzing for
smart contract developers and security auditors. Echidna [
23
] is an
open-source Ethereum smart contract fuzzer. Rather than relying
on a xed set of pre-dened bug oracles to detect vulnerabilities
during fuzzing campaigns, Echidna supports three types of proper-
ties: (1) user-dened properties (for property-based testing [
7
]), (2)
assertion checking, and (3) gas use estimation. Currently, Echidna
can test both Solidity and Vyper smart contracts, and supports most
contract development frameworks, including True and Embark.
Echidna has been used by Trail of Bits for numerous code audits
[
11
,
24
]. The use of Echidna in internal audits is a key driver in
three primary design goals for Echidna. Echidna must (1) be easy
to use and congure; (2) produce good coverage of the contract or
blockchain state; and (3) be fast and produce results quickly.
The third design goal is essential for supporting the rst two
design goals. Tools that are easy to run and produce quick results
are more likely to be integrated by engineers during the develop-
ment process. This is why most property-based testing tools have
a small default run-time or number of tests. Speed also makes use
of a tool in continuous integration (CI) (e.g., https://crytic.io) more
practical. Finally, a fast fuzzer is more amenable to experimental
techniques like mutation testing [
21
] or using a large set of bench-
mark contracts. The size of the statistical basis for decision-making
and parameter choices explored is directly limited by the speed of
the tool. Of course, Echidna supports lengthy testing campaigns as
well: there is no upper bound on how long Echidna can run, and
with coverage-based feedback there is a long-term improvement
in test generation quality. Nonetheless, the goal of Echidna is to
reveal issues to the user in less than 5 minutes.
Fuzzing smart contracts introduces some challenges that are un-
usual for fuzzer development. First, a large amount of engineering
eort is required to represent the semantics of blockchain execution;
this is a dierent challenge than executing instrumented native
binaries. Second, since Ethereum smart contracts compute using
transactions rather than arbitrary byte buers, the core problem is
one of transaction sequence generation, more akin to the problem
of unit test generation [
20
] than traditional fuzzing. This makes it
important to choose parameters such as the length of sequences
[
3
] that are not normally as important in fuzzing or in single-value-
generation as in Haskell’s QuickCheck [
7
]. Finally, nding smart
contract inputs that cause pathological execution times is not an
exotic, unusual concern, as in traditional fuzzing [
17
]. Execution on
the Ethereum blockchain requires use of gas, which has a price in
cryptocurrency. Ineciency can be costly, and malicious inputs can
lock a contract by making all transactions require more gas than
a transaction is allowed to use. Therefore producing quantitative
ISSTA ’20, July 18–22, 2020, Virtual Event, USA Gustavo Grieco, Will Song, Artur Cygan, Josselin Feist, and Alex Groce
Figure 1: The Echidna architecture.
output of maximum gas usage is an important fuzzer feature, along-
side more traditional correctness checks. Echidna incorporates a
worst-case gas estimator into a general-purpose fuzzer, rather than
forcing users to add a special-purpose gas-estimation tool [
2
,
18
]
to their workow.
2 ARCHITECTURE AND DESIGN
2.1 Echidna Architecture
Figure 1shows the Echidna architecture divided into two steps:
pre-processing and the fuzzing campaign. Our tool starts with a
set of provided contracts, plus properties integrated into one of the
contracts. As a rst step, Echidna leverages Slither [
10
], our smart
contract static analysis framework, to compile the contracts and
analyze them to identify useful constants and functions that handle
Ether (ETH) directly. In the second step, the fuzzing campaign starts.
This iterative process generates random transactions using the ap-
plication binary interface (ABI) provided by the contract, important
constants dened in the contract, and any previously collected sets
of transactions from the corpus. When a property violation is de-
tected, a counterexample is automatically minimized to report the
smallest and simplest sequence of transactions that triggers the
failure. Optionally, Echidna can output a set of transactions that
maximize coverage over all the contracts.
2.2 Continuous Improvement
A key element of Echidna’s design is to make continuous improve-
ment of functionality sustainable. Echidna has an extensive test
suite (that checks detection of seeded faults, not just proper execu-
tion) to ensure that existing features are not degraded by improve-
ments, and uses Haskell language features to maximize abstraction
and applicability of code to new testing approaches.
Tuning Parameters. Echidna provides a large number of cong-
urable parameters that control various aspects of testing. There are
currently more than 30 settings, controlled by providing Echidna
with a YAML conguration le. However, to avoid overwhelming
users with complexity and to make the out-of-the-box experience
as smooth as possible, these settings are assigned carefully chosen
default values. Default settings with a signicant impact on test
generation are occasionally re-checked via mutation testing [
12
] or
benchmark examples to maintain acceptable performance. This, like
other maintenance, is required because other functionality changes
may impact defaults. For example, changes in which functions are
called (e.g., removing view/pure functions with no assertions) may
necessitate using a dierent default sequence length. Parameter
tuning can produce some surprises with major impact on users:
e.g., the dictionary of mined constants was initially only used infre-
quently in transaction generation, but we found that mean coverage
on benchmarks could be improved signicantly by using constants
a full 40% of the time.
3 USAGE
Before starting Echidna, the smart contract to test should have
either explicit
echidna_
properties (public methods that return a
Boolean have no arguments) or use Solidity’s
assert
to express
properties. For instance, Figure 2a shows a contract with a property
that tests a simple invariant. After dening the properties to test,
running Echidna is often as simple as installing it or using the
provided Docker image and then typing:
$ ec hi dn a t e s t C o n t r a c t . s o l −− c o n t r a c t TEST
An optional YAML conguration le overriding default settings
can be provided using the
–config
option. Additionally, if a path
to a directory is used instead of a le, Echidna will auto-detect the
framework used (e.g. True) and start the fuzzing campaign.
By default, Echidna uses a dashboard output similar to AFL’s as
shown in Figure 2b. However, a command line option or a cong
le can change this to output plaintext or JSON. The cong le
also controls various properties of test generation, such as the
maximum length of generated transaction sequences, the frequency
with which mined constants are used, whether coverage driven
feedback is applied, whether maximum gas usage is computed, and
any functions to blacklist from the fuzzing campaign.
4 EXPERIMENTAL EVALUATION
4.1 Setup
We compared Echidna’s performance to the MythX platform [
9
],
accessed via the
solfuzz
[
19
] interface, on a set of reachability tar-
gets. Our experiments are produced by insertion of
assert(false)
statements, on a set of benchmark contracts [
1
] produced for the
VeriSmart safety-checker [
22
]. To our knowledge, MythX is the only
comparable fuzzer that supports arbitrary reachability targets (via
supporting assertion-checking). Comparing with a fuzzer that only
supports a custom set of built-in detectors, such as ContractFuzzer
[
16
], which does not support arbitrary assertions in Solidity code,
is dicult to do objectively, as any dierences are likely to be due
to specication semantics, not exploration capability. MythX is a
commercial SaaS platform for analyzing smart contracts. It oers a
free tier of access (limited to 5 runs/month, however) and can easily
run assertion checking on contracts via
solfuzz
, which provides
an interface similiar to Echidna’s. MythX analyzes the submitted
contracts using the Mythril symbolic execution tool [
8
] and the Har-
vey fuzzer [
26
]. Harvey is a state-of-the-art closed-source tool, with
a research paper describing its design and implementation in ICSE
2020 [
26
]. We also attempted to compare to the ChainFuzz [
6
] tool;
unfortunately, it is not maintained, and failed to analyze contracts,
producing an error reported in a GitHub issue submitted in April
of 2019 (https://github.com/ChainSecurity/ChainFuzz/issues/2).
4.2 Datasets
VeriSmart. To compare MythX and Echidna, we rst analyzed
the contracts in the VeriSmart benchmark [
1
] and identied all con-
tracts such that 1) both tools ran on the contract and 2) neither tool
Echidna: Eective, Usable, and Fast Fuzzing for Smart Contracts ISSTA ’20, July 18–22, 2020, Virtual Event, USA
co n t ra c t T E ST {
bo ol fl a g0 ; boo l fla g 1 ;
fu n ct i o n set 0 ( int va l ) public ret u rn s (b oo l ) {
if ( v a l % 10 0 = = 23 ) { fl a g 0 = true;}}
fu n ct i o n s et 1 ( int v al ) public r e tu r ns (b o ol ) {
if ( v a l % 10 == 5 & & f la g 0 ) { f la g 1 = true ;}}
fu n ct io n e ch id n a_ fl a g () public r et u rn s ( b o ol ) {
return( ! fl ag 1 ); }
}
(a) A contract with an echidna property. (b) A screenshot of the UI with the result of a fuzzing campaign
Figure 2: Using Echidna to test a smart contract
reported any issues with the contract. This left us with 12 clean con-
tracts to compare the tools’ ability to explore behavior. We inserted
assert(false)
statements into each of these contracts, after ev-
ery statement, resulting in 459 contracts representing reachability
targets. We discarded 44 of these, as the assert was unconditionally
executed in the contract’s constructor, so no behavior exploration
was required to reach it.
Tether. For a larger, more realistic example, we modied the
actual blockchain code for the TetherToken contract
1
, and again
inserted
assert(false)
targets to investigate reachability of the
code. Tether is one of the most famous “stablecoins”, a cryptocur-
rency pegged to a real-world currency, in this case the US dollar, and
has a market cap of approximately 6 billion dollars. The contract
has been involved in more than 23 million blockchain transactions.
4.3 Results
We then ran
solfuzz
’s default quick check and Echidna with a
2-minute timeout on 40 randomly selected targets. Echidna was
able to produce a transaction sequence reaching the assert for 19 of
the 40 targets, and solfuzz/MythX generated a reaching sequence
for 15 of the 40, all of which were also reached by Echidna. While
the time to reach the assertion was usually close to 2 minutes with
solfuzz, Echidna needed a maximum of only 52 seconds to hit the
hardest target; the mean time required was 13.9 seconds, and the
median time was only 6.9 seconds. We manually examined the
targets not detected by either tool, and believe them all to repre-
sent unreachable targets, usually due to being inserted after an
unavoidable return statement, or being inserted in the SafeMath
contract, which redenes assert. Of the reachable targets, Echidna
was able to produce sequences for 100%, and solfuzz/MythX for
78.9%. For Echidna, we repeated each experiment 10 more times,
and Echidna always reached each target. Due to the limit on MythX
runs, even under a developer license (500 executions/month), we
were unable to statistically determine the stability of its results to
the same degree, but can conrm that for two of the targets, a sec-
ond run succeeded, and for two of the targets three additional runs
still failed to reach the assert. Running
solfuzz
with the
–mode
standard
argument (not available to free accounts) did detect all
four, but it required at least 15 minutes of analysis time in each
1https://etherscan.io/address/0xdac17f958d2ee523a2206206994597c13d831ec7#code
case. Figure 4shows the key part of the code for one of the four
targets Echidna, but not solfuzz/MythX (even with additional runs),
was able to reach. The assert can only be executed when a call
has been made to the
approve
function, allowing the sender of
the
transferFrom
call to send an amount greater than or equal to
_amount
, and when the contract from which transfer is to be made
has a token balance greater than
_amount
. Generating a sequence
with the proper set of functions called and the proper relationships
between variables is a dicult problem, but Echidna’s heuristic
use of small numeric values in arguments and heuristic repetition
of addresses in arguments and as message senders is able to navi-
gate the set of constraints easily. A similar set of constraints over
allowances and/or senders and function arguments is involved in
two of the other four targets where Echidna performs better.
When using the Tether contract, we again randomly selected 40
targets, and ran two minutes of testing on each with
solfuzz
and
Echidna. Echidna was able to reach 28 of the 40 targets, with mean
and median runtimes of 24 and 15 seconds, respectively. The longest
run required 103 seconds. On the other hand, solfuzz/MythX was
unable to reach any of the targets using the default search. MythX/-
solfuzz was able to all detect the targets Echidna detected using
the standard search, and detected one additional target. The mean
time required for detection, however, was almost 16 minutes. The
additional target reached by
solfuzz
involves adding an address
to a blacklist, then destroying the funds of that address. Because
an address can also be removed from a blacklist, there is no simple
coverage-feedback to encourage avoiding this, and there are many
functions to test, Echidna has trouble generating such a sequence.
However, using a prototype of a swarm testing [
13
] mode not yet
included in the public release of Echidna, but briey discussed in
the conclusion below, we were able to produce such a sequence
in less than ve minutes. Even without swarm testing, we were
able to detect the problem in between 10 and 12 minutes, using a
branch (to be merged in the near future) that incorporates more
information from Slither, and uses some novel mutation strategies.
Of the 11 targets hit by neither tool, we manually determined that
all but two are clearly actually unreachable.
As a separate set of experiments, we measured the average cov-
erage obtained on the VeriSmart and Tether token contracts, with
various settings for the length of transaction sequences, ranging
from very short (length 10) to very long (length 500) for runs of 2,
ISSTA ’20, July 18–22, 2020, Virtual Event, USA Gustavo Grieco, Will Song, Artur Cygan, Josselin Feist, and Alex Groce
(a) TetherToken (b) VeriSmart (avg)
Figure 3: Coverage obtained given short runs (2, 5 and 10
minutes) with dierent transaction sequence lengths.
if ( b a la n ce s [ _ fr om ] >= _ am o un t
&& a l l ow e d [ _f r om ][ ms g . s en d er ] >= _ am o un t
&& _ a mo u n t > 0
&& b a l an c es [_t o ] + _ am o u nt > ba l an c es [ _ t o ]) {
ba l an c es [ _ f ro m ] - = _a m ou n t ;
al l ow e d [_ f ro m ][ msg . s en de r ] - = _a m ou n t ;
as s er t ( f al se );
Figure 4: Code for a dicult reachabilility target.
5, and 10 minutes each. Figures 3a and 3b show that the current
default value used by Echidna (100) is a reasonable compromise
to maximize coverage in short fuzzing campaigns. Each run was
repeated 10 times to reduce the variability of such short campaigns.
5 RELATED WORK
Echidna inherits concepts from property-based fuzzing, rst pop-
ularized by the QuickCheck tool [
7
] and from coverage-driven
fuzzing, perhaps best known via the American Fuzzy Lop tool [
27
].
Other fuzzers for Ethereum smart contracts include Harvey [
26
],
ContractFuzzer [
16
], and ChainFuzz [
6
]. We were unable to get Con-
tractFuzzer to produce useful output within a four hour timeout,
and ChainFuzz no longer appears to work. Harvey is closed-source,
but is usable via the MythX [
9
] CI platform and the
solfuzz
[
19
]
tool. Echidna uses information from the Slither static analysis tool
[10] to improve the creation of Ethereum transactions.
6 CONCLUSIONS AND FUTURE WORK
Echidna is an eective, easy-to-use, and fast fuzzer for Ethereum
blockchain smart contracts. Echidna provides a potent out-of-the-
box fuzzing experience with little setup or preparation, but allows
for considerable customization. Echidna supports assertion check-
ing, custom property-checking, and estimation of maximum gas
usage—a core feature set based on experience with security audits
of contracts. The default test generation parameters of Echidna
have been calibrated using real-world experience in commercial
audits and via benchmark experiments and mutation analysis. In
our experiments, Echidna outperformed a comparable fuzzer using
sophisticated techniques: Echidna detected, in less than 2 minutes,
many reachability targets that required 15 or more minutes with
solfuzz
, on both benchmark contracts and the real-world Tether
token. Echidna is under heavy active development. Recently added
or in-progress features include gas estimation, test corpus collec-
tion, integration of Slither static analysis information, and improved
mutation for feedback-driven fuzzing. One future work will add
a driver mode, similar to the swarm tool [
14
] for the SPIN model
checker [
15
], to make better use of conguration diversity, includ-
ing swarm testing [
13
], in order to fully exploit multicore machines.
In particular, this mode will enable Echidna to produce even more
accurate maximum gas usage estimates.
REFERENCES
[1] VeriSmart benchmark. https://github.com/kupl/VeriSmart-benchmarks.
[2]
Elvira Albert, Jesús Correas, Pablo Gordillo, Guillermo Román-Díez, and Albert
Rubio. Gasol: Gas analysis and optimization for ethereum smart contracts, 2019.
[3]
James H. Andrews, Alex Groce, Melissa Weston, and Ru-Gang Xu. Random test
run length and eectiveness. In Automated Software Engineering, pages 19–28,
2008.
[4]
Nicola Atzei, Massimo Bartoletti, and Tiziana Cimoli. A survey of attacks on
Ethereum smart contracts SoK. In International Conference on Principles of Security
and Trust, pages 164–186, 2017.
[5]
Vitalik Buterin. Ethereum: A next-generation smart contract and decentralized
application platform. https://github.com/ethereum/wiki/wiki/White-Paper, 2013.
[6] Chain Security. https://github.com/ChainSecurity/ChainFuzz.
[7]
Koen Claessen and John Hughes. QuickCheck: a lightweight tool for random test-
ing of Haskell programs. In International Conference on Functional Programming
(ICFP), pages 268–279, 2000.
[8]
ConsenSys. Mythril: a security analysis tool for ethereum smart contracts.
https://github.com/ConsenSys/mythril-classic, 2017.
[9] Consensys Diligence. https://mythx.io/.
[10]
Josselin Feist, Gustavo Grieco, and Alex Groce. Slither: A static analysis frame-
work for smart contracts. In International Workshop on Emerging Trends in
Software Engineering for Blockchain, 2019.
[11]
Alex Groce, Josselin Feist, Gustavo Grieco, and Michael Colburn. What are
the actual aws in important smart contracts (and how can we nd them)? In
International Conference on Financial Cryptography and Data Security, 2020.
[12]
Alex Groce, Josie Holmes, Darko Marinov, August Shi, and Lingming Zhang.
An extensible, regular-expression-based tool for multi-language mutant genera-
tion. In Proceedings of the 40th International Conference on Software Engineering:
Companion Proceeedings, ICSE ’18, pages 25–28, New York, NY, USA, 2018. ACM.
[13]
Alex Groce, Chaoqiang Zhang, Eric Eide, Yang Chen, and John Regehr. Swarm
testing. In International Symposium on Software Testing and Analysis, pages 78–88,
2012.
[14]
Gerard Holzmann, Rajeev Joshi, and Alex Groce. Swarm verication techniques.
IEEE Transactions on Software Engineering, 37(6):845–857, 2011.
[15]
Gerard J. Holzmann. The SPIN Model Checker: Primer and Reference Manual.
Addison-Wesley Professional, 2003.
[16]
Bo Jiang, Ye Liu, and WK Chan. Contractfuzzer: Fuzzing smart contracts for vul-
nerability detection. In Proceedings of the 33rd ACM/IEEE International Conference
on Automated Software Engineering, pages 259–269, 2018.
[17]
Caroline Lemieux, Rohan Padhye, Koushik Sen, and Dawn Song. Peruzz: Auto-
matically generating pathological inputs. In Proceedings of the 27th ACM SIGSOFT
International Symposium on Software Testing and Analysis, pages 254–265, 2018.
[18]
Fuchen Ma, Ying Fu, Meng Ren, Wanting Sun, Zhe Liu, Yu Jiang, Jun Sun, and
Jiaguang Sun. Gasfuzz: Generating high gas consumption inputs to avoid out-of-
gas vulnerability, 2019.
[19] Bernhard Mueller. https://github.com/b- mueller/solfuzz.
[20]
Carlos Pacheco, Shuvendu K. Lahiri, Michael D. Ernst, and Thomas Ball. Feedback-
directed random test generation. In International Conference on Software Engi-
neering, pages 75–84, 2007.
[21]
Mike Papadakis, Marinos Kintis, Jie Zhang, Yue Jia, Yves Le Traon, and Mark
Harman. Mutation testing advances: an analysis and survey. In Advances in
Computers, volume 112, pages 275–378. Elsevier, 2019.
[22]
Sunbeom So, Myungho Lee, Jisu Park, Heejo Lee, and Hakjoo Oh. VeriSmart: A
highly precise safety verier for ethereum smart contracts. In IEEE Symposium
on Security & Privacy, 2020.
[23]
Trail of Bits. Echidna: Ethereum fuzz testing framework. https://github.com/
crytic/echidna, 2018.
[24]
Trail of Bits. Trail of bits security reviews. https://github.com/trailof bits/
publications#security-reviews, 2019.
[25]
Gavin Wood. Ethereum: a secure decentralised generalised transaction ledger.
http://gavwood.com/paper.pdf, 2014.
[26]
Valentin Wüstholz and Maria Christakis. Targeted greybox fuzzing with static
lookahead analysis. In International Conference on Software Engineering, 2020.
[27]
Michal Zalewski. american fuzzy lop (2.35b). http://lcamtuf.coredump.cx/a/.
Accessed December 20, 2016.
... ContractFuzzer [33] firstly utilizes fuzzing to detect vulnerabilities by observing runtime behavior. Subsequent research has focused on enhancing aspects of smart contract fuzzing, including feedback-driven fuzzers like ContraMaster [63], Harvey [70], and Echidna [25], which generate varied inputs to uncover bugs effectively. sFuzz [48] introduced branch distance feedback for exploring difficult branches, while Liu et al. [44] developed a strategy for generating sequences that activate deeper contract states to detect complex vulnerabilities. ...
... Step 2: Fuzzing Configuration. We input the source code into an existing smart contract fuzzing tool, Echidna [25], which is an industrystandard tool widely used in many popular projects (e.g., 0x [27] and Balancer [20]). We provide a default configuration of the fuzzing tool, and users can modify it to fit their requirements, such as setting user numbers, defining balance limits for addresses, and specifying the contract owner. ...
Preprint
Full-text available
Smart contracts are the fundamental components of blockchain technology. They are programs to determine cryptocurrency transactions, and are irreversible once deployed, making it crucial for cryptocurrency investors to understand the cryptocurrency transaction behaviors of smart contracts comprehensively. However, it is a challenging (if not impossible) task for investors, as they do not necessarily have a programming background to check the complex source code. Even for investors with certain programming skills, inferring all the potential behaviors from the code alone is still difficult, since the actual behaviors can be different when different investors are involved. To address this challenge, we propose PrettiSmart, a novel visualization approach via execution simulation to achieve intuitive and reliable visual interpretation of smart contracts. Specifically, we develop a simulator to comprehensively capture most of the possible real-world smart contract behaviors, involving multiple investors and various smart contract functions. Then, we present PrettiSmart to intuitively visualize the simulation results of a smart contract, which consists of two modules: The Simulation Overview Module is a barcode-based design, providing a visual summary for each simulation, and the Simulation Detail Module is an augmented sequential design to display the cryptocurrency transaction details in each simulation, such as function call sequences, cryptocurrency flows, and state variable changes. It can allow investors to intuitively inspect and understand how a smart contract will work. We evaluate PrettiSmart through two case studies and in-depth user interviews with 12 investors. The results demonstrate the effectiveness and usability of PrettiSmart in facilitating an easy interpretation of smart contracts.
... Symbolic execution tools like Mythril [4], Oyente [53], ETHBMC [40], EOSAFE [46] and Manticore [56] explore potentially vulnerable paths using constraint solvers. Dynamic analysis approaches, such as fuzzing tools [37,44,45,49,54,63,72,74], generate random inputs or reorder historical transaction sequences to test contracts. Furthermore, GPTScan [66] leverages large language models [47] to localize vulnerabilities by defining vulnerability scenarios. ...
Preprint
Full-text available
Price manipulation attack is one of the notorious threats in decentralized finance (DeFi) applications, which allows attackers to exchange tokens at an extensively deviated price from the market. Existing efforts usually rely on reactive methods to identify such kind of attacks after they have happened, e.g., detecting attack transactions in the post-attack stage, which cannot mitigate or prevent price manipulation attacks timely. From the perspective of attackers, they usually need to deploy attack contracts in the pre-attack stage. Thus, if we can identify these attack contracts in a proactive manner, we can raise alarms and mitigate the threats. With the core idea in mind, in this work, we shift our attention from the victims to the attackers. Specifically, we propose SMARTCAT, a novel approach for identifying price manipulation attacks in the pre-attack stage proactively. For generality, it conducts analysis on bytecode and does not require any source code and transaction data. For accuracy, it depicts the control- and data-flow dependency relationships among function calls into a token flow graph. For scalability, it filters out those suspicious paths, in which it conducts inter-contract analysis as necessary. To this end, SMARTCAT can pinpoint attacks in real time once they have been deployed on a chain. The evaluation results illustrate that SMARTCAT significantly outperforms existing baselines with 91.6% recall and ~100% precision. Moreover, SMARTCAT also uncovers 616 attack contracts in-the-wild, accounting for \9.25Mfinanciallosses,withonly19casespubliclyreported.ByapplyingSMARTCATasarealtimedetectorinEthereumandBinanceSmartChain,ithasraised14alarms99secondsafterthecorrespondingdeploymentonaverage.Theseattackshavealreadyledto9.25M financial losses, with only 19 cases publicly reported. By applying SMARTCAT as a real-time detector in Ethereum and Binance Smart Chain, it has raised 14 alarms 99 seconds after the corresponding deployment on average. These attacks have already led to 641K financial losses, and seven of them are still waiting for their ripe time.
... In this category, we found two research articles that handle both functional and security issues. Indeed, Echidna [66] is an open-source smart contract fuzzer that involves static and dynamic testing techniques to generate efficient tests. With the aim of detecting assertion violations and some custom properties, it makes use of Slither [96], a smart contract static analysis framework. ...
... Basically, smart contract fuzzing is a process that generates inputs to execute the APIs of a target contract and explores transaction sequences to detect abnormal behaviors that indicate bugs. Mainstream fuzzers, such as those mentioned in [21,33,34], operate at the API level, combining transaction sequences for comprehensive testing. Different fuzzers adopt varied techniques: sFuzz [27] focuses on higher code coverage, Smartian [18] uses dataflow analysis, and ConFuzzius [37] integrates symbolic execution and taint analysis. ...
Preprint
Full-text available
Billions of dollars are transacted through smart contracts, making vulnerabilities a major financial risk. One focus in the security arms race is on profitable vulnerabilities that attackers can exploit. Fuzzing is a key method for identifying these vulnerabilities. However, current solutions face two main limitations: a lack of profit-centric techniques for expediting detection, and insufficient automation in maximizing the profitability of discovered vulnerabilities, leaving the analysis to human experts. To address these gaps, we have developed VERITE, a profit-centric smart contract fuzzing framework that not only effectively detects those profitable vulnerabilities but also maximizes the exploited profits. VERITE has three key features: 1) DeFi action-based mutators for boosting the exploration of transactions with different fund flows; 2) potentially profitable candidates identification criteria, which checks whether the input has caused abnormal fund flow properties during testing; 3) a gradient descent-based profit maximization strategy for these identified candidates. VERITE is fully developed from scratch and evaluated on a dataset consisting of 61 exploited real-world DeFi projects with an average of over 1.1 million dollars loss. The results show that VERITE can automatically extract more than 18 million dollars in total and is significantly better than state-of-the-art fuzzer ITYFUZZ in both detection (29/9) and exploitation (58 times more profits gained on average). Remarkbly, in 12 targets, it gains more profits than real-world attacking exploits (1.01 to 11.45 times more). VERITE is also applied by auditors in contract auditing, where 6 (5 high severity) zero-day vulnerabilities are found with over $2,500 bounty rewards.
... Static analysis tools, such as Vandal [6] , Securify [7] , and Slither [8] , detect potential vulnerabilities by analyzing the structure and logic of smart contract code. Dynamic analysis tools, such as Echidna [9] , ContractFuzzer [10] , and Harvey [11] , use fuzz testing techniques to generate test inputs and identify anomalies during the actual execution of smart contracts, thereby gaining insights into runtime vulnerabilities. Symbolic execution tools, such as Oyente [12] , Halmos [13] , and Mythril [14] , detect vulnerabilities by simulating contract execution paths and checking possible states along each path. ...
Article
Full-text available
With the rapid development of blockchain technology and smart contracts, the security issues of smart contracts have become increasingly serious. To address the significant limitations of traditional detection methods in handling the complexity and scale of smart contracts, a new framework for smart contract vulnerability detection that combines a vector database and a generative pre-trained transformer (GPT) model — VulnScan GPT — has been proposed. This framework comprises three main components: function signature extraction, vector database storage and retrieval, and GPT-based vulnerability detection. The framework uses the solc tool to generate an abstract syntax tree from smart contracts, extract function signatures, and vectorize the code for storage. By integrating the GPT model, the framework can preliminarily analyze and filter key functions based on common vulnerability scenarios and then retrieve relevant implementations from the vector database for in-depth assessment. This method gradually optimizes function analysis through an iterative detection mechanism, leveraging the efficient storage and retrieval capabilities of the vector database, combined with the deep natural language processing abilities of the GPT model, enhancing the accuracy and comprehensiveness of vulnerability detection. In the experimental evaluation, tests were conducted on various datasets to detect automated market maker price manipulation and initial risk deposit vulnerabilities, demonstrating that VulnScan GPT not only improves the accuracy of vulnerability detection, but also significantly reduces operational costs by optimizing token usage, resulting in an efficient and cost-effective detection solution.
... To address smart contract vulnerabilities, various verification techniques (surveyed in [2]) and tools (surveyed in [3]) have been developed, including testingbased approaches (like [11]) and static analysis based approaches (e.g., symbolic execution [13]). These approaches heavily rely on known patterns and cannot guarantee correctness and security. ...
Chapter
Full-text available
An important problem in smart contract security is understanding the likelihood and criticality of discovered, or potential, weaknesses in contracts. In this paper we provide a summary of Ethereum smart contract audits performed for 23 professional stakeholders, avoiding the common problem of reporting issues mostly prevalent in low-quality contracts. These audits were performed at a leading company in blockchain security, using both open-source and proprietary tools, as well as human code analysis performed by professional security engineers. We categorize 246 individual defects, making it possible to compare the severity and frequency of different vulnerability types, compare smart contract and non-smart contract flaws, and to estimate the efficacy of automated vulnerability detection approaches.
Chapter
Full-text available
We present the main concepts, components, and usage of Gasol, a Gas AnalysiS and Optimization tooL for Ethereum smart contracts. Gasol offers a wide variety of cost models that allow inferring the gas consumption associated to selected types of EVM instructions and/or inferring the number of times that such types of bytecode instructions are executed. Among others, we have cost models to measure only storage opcodes, to measure a selected family of gas-consumption opcodes following the Ethereum’s classification, to estimate the cost of a selected program line, etc. After choosing the desired cost model and the function of interest, Gasol returns to the user an upper bound of the cost for this function. As the gas consumption is often dominated by the instructions that access the storage, Gasol uses the gas analysis to detect under-optimized storage patterns, and includes an (optional) automatic optimization of the selected function. Our tool can be used within an Eclipse plugin for Solidity which displays the gas and instructions bounds and, when applicable, the gas-optimized Solidity function.
Conference Paper
Full-text available
This paper describes Slither, a static analysis framework designed to provide rich information about Ethereum smart contracts. It works by converting Solidity smart contracts into an intermediate representation called SlithIR. SlithIR uses Static Single Assignment (SSA) form and a reduced instruction set to ease implementation of analyses while preserving semantic information that would be lost in transforming Solidity to bytecode. Slither allows for the application of commonly used program analysis techniques like dataflow and taint tracking. Our framework has four main use cases: (1) automated detection of vulnerabilities, (2) automated detection of code optimization opportunities, (3) improvement of the user's understanding of the contracts, and (4) assistance with code review. In this paper, we present an overview of Slither, detail the design of its intermediate representation, and evaluate its capabilities on real-world contracts. We show that Slither's bug detection is fast, accurate, and outperforms other static analysis tools at finding issues in Ethereum smart contracts in terms of speed, robustness, and balance of detection and false positives. We compared tools using a large dataset of smart contracts and manually reviewed results for 1000 of the most used contracts.
Conference Paper
Decentralized cryptocurrencies feature the use of blockchain to transfer values among peers on networks without central agency. Smart contracts are programs running on top of the blockchain consensus protocol to enable people make agreements while minimizing trusts. Millions of smart contracts have been deployed in various decentralized applications. The security vulnerabilities within those smart contracts pose significant threats to their applications. Indeed, many critical security vulnerabilities within smart contracts on Ethereum platform have caused huge financial losses to their users. In this work, we present ContractFuzzer, a novel fuzzer to test Ethereum smart contracts for security vulnerabilities. ContractFuzzer generates fuzzing inputs based on the ABI specifications of smart contracts, defines test oracles to detect security vulnerabilities, instruments the EVM to log smart contracts runtime behaviors, and analyzes these logs to report security vulnerabilities. Our fuzzing of 6991 smart contracts has flagged more than 459 vulnerabilities with high precision. In particular, our fuzzing tool successfully detects the vulnerability of the DAO contract that leads to USD 60 million loss and the vulnerabilities of Parity Wallet that have led to the loss of USD 30 million and the freezing of USD 150 million worth of Ether.
Conference Paper
Performance problems in software can arise unexpectedly when programs are provided with inputs that exhibit worst-case behavior. A large body of work has focused on diagnosing such problems via statistical profiling techniques. But how does one find these inputs in the first place? We present PerfFuzz, a method to automatically generate inputs that exercise pathological behavior across program locations, without any domain knowledge. PerfFuzz generates inputs via feedback-directed mutational fuzzing. Unlike previous approaches that attempt to maximize only a scalar characteristic such as the total execution path length, PerfFuzz uses multi-dimensional feedback and independently maximizes execution counts for all program locations. This enables PerfFuzz to (1) find a variety of inputs that exercise distinct hot spots in a program and (2) generate inputs with higher total execution path length than previous approaches by escaping local maxima. PerfFuzz is also effective at generating inputs that demonstrate algorithmic complexity vulnerabilities. We implement PerfFuzz on top of AFL, a popular coverage-guided fuzzing tool, and evaluate PerfFuzz on four real-world C programs typically used in the fuzzing literature. We find that PerfFuzz outperforms prior work by generating inputs that exercise the most-hit program branch 5x to 69x times more, and result in 1.9x to 24.7x longer total execution paths.
Conference Paper
Mutation testing is widely used in research (even if not in practice). Mutation testing tools usually target only one programming language and rely on parsing a program to generate mutants, or operate not at the source level but on compiled bytecode. Unfortunately, developing a robust mutation testing tool for a new language in this paradigm is a difficult and time-consuming undertaking. Moreover, bytecode/intermediate language mutants are difficult for programmers to read and understand. This paper presents a simple tool, called universalmutator, based on regular-expression-defined transformations of source code. The primary drawback of such an approach is that our tool can generate invalid mutants that do not compile, and sometimes fails to generate mutants that a parser-based tool would have produced. Additionally, it is incompatible with some approaches to improving the efficiency of mutation testing. However, the regexp-based approach provides multiple compensating advantages. First, our tool is easy to adapt to new languages; e.g., we present here the first mutation tool for Apple's Swift programming language. Second, the method makes handling multi-language programs and systems simple, because the same tool can support every language. Finally, our approach makes it easy for users to add custom, project-specific mutations.