ArticlePDF Available

Abstract and Figures

With the advent of Bitcoin, the interest of the database community in blockchain systems has steadily grown. Many existing blockchain applications use blockchains as a platform for monetary transactions, however. We deviate from this philosophy and present ResilientDB, which can serve in a suite of non-monetary data-processing blockchain applications. Our ResilientDB uses state-of-the-art technologies and includes a novel visualization that helps in monitoring the state of the blockchain application.
Content may be subject to copyright.
Scalable, Resilient, and Configurable Permissioned
Blockchain Fabric
Sajjad Rahnama, Suyash Gupta, Thamir M. Qadah, Jelle Hellings, Mohammad Sadoghi
Moka Blox LLC
Exploratory Systems Lab
Purdue University
University of California, Davis
With the advent of Bitcoin, the interest of the database com-
munity in blockchain systems has steadily grown. Many
existing blockchain applications use blockchains as a plat-
form for monetary transactions, however. We deviate from
this philosophy and present ResilientDB, which can serve in
a suite of non-monetary data-processing blockchain appli-
cations. Our ResilientDB uses state-of-the-art technologies
and includes a novel visualization that helps in monitoring
the state of the blockchain application.
PVLDB Reference Format:
Sajjad Rahnama, Suyash Gupta, Thamir M. Qadah, Jelle Hellings,
Mohammad Sadoghi. Scalable, Resilient, and Configurable Per-
missioned Blockchain Fabric. PVLDB, 13(12): 2893-2896, 2020.
With the advent of Bitcoin [20], the interest of the dis-
tributed and database community has steadily grown to-
wards blockchain applications. A blockchain in its simplest
form is an immutable ledger. Initial blockchain applications
envisioned blockchain as a platform for monetary transac-
tions [20, 21]. The key aim of these applications is to provide
a monetary unit of exchange, a cryptocurrency, which can be
covertly exchanged between two or more parties. This covert
exchange requires these applications to allow any user to
participate (permissionless) and hide identities of the par-
ticipants. As a result, these applications suffer from low
throughputs (around 10 txn/s), high latencies, and face sev-
eral vulnerabilities [7, 8].
Evidently, permissionless applications do not satisfy the
needs of secure industry-grade applications. This led to the
rise of permissioned blockchain applications, which require
the identity of each participant to be known a priori. At the
core of any permissioned blockchain application is a Byzan-
tine Fault-Tolerant (Bft) consensus protocol that helps to
This work is licensed under the Creative Commons Attribution-
NonCommercial-NoDerivatives 4.0 International License. To view a copy
of this license, visit For
any use beyond those covered by this license, obtain permission by emailing Copyright is held by the owner/author(s). Publication rights
licensed to the VLDB Endowment.
Proceedings of the VLDB Endowment, Vol. 13, No. 12
ISSN 2150-8097.
achieve a single order of transactions among the partici-
pants [5, 10, 11, 12, 15, 16, 22, 18].
Contrary to their potential, existing permissioned block-
chain systems are limited by their design choices as they (i)
require exchange of a monetary transaction [2, 19]; (ii) ad-
here to a specific order-execute [4, 9] or execute-order design
paradigm [2]; (iii) achieve very low throughput and do not
scale beyond a small set of replicas replicas [1, 2, 7, 9]; (iv)
are in-flexible in their choice of Bft protocol [1, 4, 19]; and
(v) lack support for introducing thread parallelism and task
pipelining [1, 2, 3, 4, 9].
In this demo, we introduce ResilientDB, a high-throughput
permissioned blockchain fabric [13, 14]. ResilientDB1is a
modular open-source system for deploying and operating
permissioned blockchain applications. With ResilientDB,
we envision data-management beyond mere cryptocurren-
cies via Blockchain-as-a-Service. ResilientDB enables devel-
opers with the power to implement and test any Bft proto-
col. Moreover, our fabric, which is built from scratch, em-
ploys state-of-the-art software engineering principles, sup-
ports a lean design, and eases application deployment.
ResilientDB includes multi-threaded deep pipelines that
allow it to achieve high-throughput consensus among its
replicas. Any developer can easily configure the number of
threads and stages of the pipeline. ResilientDB also allows
application developers to test their applications using both
YCSB [6] transactions and Smart Contracts. Further, Re-
silientDB provides its users with a Graphical User Interface
(GUI) to compile, deploy, and run the platform and analyze
the results, all with just one click. This GUI is designed to
tightly monitor the performance of each individual replica.
Despite these features of ResilientDB, our focus remains
at designing a high-performance permissioned blockchain
fabric. We claim that such a design is possible by adopting
asystem-centric view rather than the protocol-centric view
employed by existing fabrics. Our system-centric view is mo-
tivated twofold: (i) no one Bft protocol fits all settings, and
(ii) there is much more to a blockchain system than just its
Bft protocol. To illustrate the impact of our system-centric
view, we compare in Figure 1 the throughput of ResilientDB
employing the PBFT [5] consensus protocol against a per-
missioned blockchain that uses the Zyzzyva [18] consensus
protocol and adopts practices suggested in BFTSmart [3].
Note that PBFT requires three phases, of which two phases
necessitate quadratic communication among the replicas,
1ResilientDB is open-sourced at
and code is available at
Figure 1: Two permissioned applications employing
distinct Bft protocols (80 k clients per experiment).
while Zyzzyva only requires a single linear phase. Despite
using the costlier PBFT consensus protocol, ResilientDB
outperforms the other system. We discuss reasons for the
high throughput of ResilientDB in Section 2.
In this section, we present the architecture and capabil-
ities of our ResilientDB fabric. ResilientDB is written en-
tirely in C++ and provides a GUI to ease user interaction
with the system. Further, we also provide a Dockerized de-
ployment that allows any user to experience and test the Re-
silientDB fabric (comprising of multiple replicas and clients)
on its local machine. In Figure 2, we illustrate the overall
architecture, which we describe in detail next.
Transport Layer. Permissioned blockchains use com-
munication-intensive Bft consensus protocols. Hence, they
expect an efficient transport layer to facilitate exchange of
messages between replicas.2ResilientDB employs Nanomsg
sockets to facilitate communication among clients and repli-
cas via TCP or UDP (depending on the choice of the devel-
oper). We also provide support for fast RDMA communica-
tion for replicas with RDMA capabilities.
To facilitate efficient communication, ResilientDB employs
multiple input/output threads with dedicated sockets. Note
that the number of input/output threads can be readily ad-
justed based on the network requirements and buffering bot-
tlenecks. ResilientDB also provides access to distinct mes-
sage queues. Depending on the type of a message, these
queues can be used by different threads to communicate with
each other and to place the message on the network.
Crypto Layer. Blockchains typically are designed to
deal with malicious adversaries. To secure communication
and prevent message tampering, ResilientDB employs NIST-
recommended cryptographic constructs from the Crypto++
library. Depending on specific needs, replicas and clients
can digitally sign their messages using either (i) asymmetric-
key cryptographic schemes such as ED25519 or RSA; or (ii)
symmetric-key cryptographic schemes such as CMAC and
AES [17]. ResilientDB also provides message digests via
either SHA256 or SHA3 hashes.
Parallel Pipelined Consensus Layer. At the core of
any permissioned blockchain application lies a Bft consen-
sus protocol that safely replicates client transactions among
all replicas. Decades of research has brought forth several
such protocols. No one protocol is the best-fit in all en-
vironments, however. For example, Zyzzyva achieves high
throughput if none of the replicas are faulty, HotStuff [22]
works well if latency is not critical, GeoBFT [13] scales well
2Permissionless systems are compute-intensive as they run
consensus by solving a complex cryptographic puzzle.
Transport Layer Crypto Layer
Parallel Pipelined Consensus Layer
Storage Layer
Chain Management
Memory Pool
Smart Contract
Execution Thread
Smart Contract
Execution Thread
Figure 2: Architecture of the ResilientDB fabric.
when replicas are geographically distant, and PBFT, al-
though typically-considered too slow, is most robust against
failures. These characteristics of existing Bft protocols per-
mit us to conclude that any resilient permissioned blockchain
fabric should facilitate testing and implementation of differ-
ent Bft protocols. ResilientDB’s consensus layer allows this
and to support this claim we provide implementation of all
of the aforementioned protocols (among many others).
Furthermore, as we argued in Section 1, there is more to
a blockchain system than just its Bft protocol. In specific,
we showed that a permissioned blockchain fabric adopting
a system-centric design and employing a slow Bft protocol
outperforms a protocol-centric fabric that uses a fast pro-
tocol. To yield such a system-centric design, ResilientDB
employs transaction batching, multi-threading, pipelining,
out-of-order processing, and memory pooling.
Batching. Prior works have batched client transactions
to reduce the cost of consensus [5, 18]. We permit batching
of transactions at both clients and replicas and developers
can specify any size for such batches. Batching reduces both
communication costs and computation costs by reducing the
number of messages that are exchanged (which also reduced
the number of message signatures necessary).
Transactions and Smart Contracts. ResilientDB sup-
ports YCSB transactions and customized Smart Contracts.
YCSB transactions can be used for benchmarking perfor-
mance and developers can easily vary the skew (read/write
percentage) of these transaction. ResilientDB also provides
APIs for designing and testing Smart Contracts, which are
similar to stored procedures in databases [7]. To demon-
strate this, we implemented Ethereum’s account-based smart
contracts for banking applications [21] (see Section 3.2).
We associate each transaction with a transaction manager
that manages the resources required for handling transac-
tions. We provide fast lookup of transaction managers via
indices on transaction identifier and batch identifier. Fur-
thermore, transaction managers are pooled and reused to
save on allocation and deallocation costs.
Order-Execute vs. Execute-Order Paradigm. Tradi-
tional permissioned blockchain systems employ the order-
execute paradigm, which states that a transaction needs to
be ordered across all replicas prior to its execution [4, 9].
This is in contrast with the execute-order paradigm pro-
posed by Hyperledger [2], which advocates to first execute
and then order the transaction. Both of these paradigms
have their pros and cons. Our ResilientDB fabric provides
support for both paradigms and allows developers to select
the paradigm that best fits their applications.
Nginx Backend
Web server
React Web
Google Cloud Platform
Data Storage and Analytics
Figure 3: Demo Platform Architecture.
Multi-Threaded Deep Pipeline. As stated before, per-
missioned blockchain systems are communication-intensive.
Hence, we ensure that our ResilientDB fabric is not under-
utilizing hardware and will only be limited by network ca-
pacity. To do so, we designed the consensus protocols in
ResilientDB such that the critical path is as simple as possi-
ble and all other tasks are split of in their separate threads.
E.g., threads to deal with message sending, with message
receiving, signing messages, verifying signatures, creating
transaction batches, performing checkpoints, and executing
transactions. Users can easily adjust the number of required
threads depending on the specific needs of their applications.
Memory Pool. Blockchain systems that process thou-
sands of transactions, smart contracts, and messages per
second require high-performance management of memory
resources. For ResilientDB’s memory management, we em-
ploy Jemalloc. Further, we minimize memory allocation and
deallocation by using distinct memory pools for messages,
transaction managers, and smart contracts. Depending on
the size of an allocation, each thread accesses the required
pool and fetches an unallocated memory object. At the time
of garbage collection, obsolete objects are marked as released
and placed back in the respective memory pool for reuse.
Chain Storage. ResilientDB provides support for se-
cure ledger (blockchain) management. To enable efficient ex-
ecution of client transactions, we also support efficient read
and write accesses to client records. In specific, each replica
can use popular databases such as SQLite and LevelDB to
achieve data persistence for the ledger and client records un-
der failure. ResilientDB provides several simple APIs that
allow developers to read and write to these databases and
modify their schema if necessary.
During our demonstration, each user will get access to a
graphical web-based interface of ResilientDB. Figure 3 illus-
trates the architecture of our demonstration environment.
We provide a web-based UI for specifying experiment pa-
rameters, for monitoring the real-time throughput and la-
tency of the system, and for the analysis of collected data.
In specific, users can specify their choice of parameters
on our React Web Dashboard, which uses REST APIs to
forward these parameters to our Nginx back end. The back
end compiles the code and deploys the executables on the
Google Cloud Platform (GCP). Once the executables start
running, any emitted result is continuously stored in In-
fluxDB. Throughout this process, our dashboard shows the
user the current state of the system. If the user wants to
visualize the ongoing results, our dashboard asks the back
end to fetch the data from InfluxDB and plot the required
graphs. This allows us to show the user real-time system
Parameter Options
Bft Protocol PBFT, Zyzzyva, GeoBFT, HotStuff
Transactions YCSB, Banking Smart Contracts
Requests/Txn 1, 5, 15, 50
Batch Size 1, 10, 100, 500, 1000, 4000
Message Size 0 kB, 100 kB, 200 kB, 400 kB
Pipeline Enable or Disable
Threads I/O, Worker, Execute Sign, Checkpoint
Storage In-memory, SQLite, LevelDB
Signatures Disable
Only Asymmetric ED25519, RSA
Only Symmetric CMAC, VMAC
Mix Use both
Hash Schemes SHA256, SHA3
Figure 4: ResilientDB parameters.
Figure 5: The interactive WebUI dashboard.
throughput and latency metrics. We employ React (open-
source), Nginx (performance), and InfluxDB (eases manage-
ment of time-series data) for their associated advantages.
We provide our users access to two demonstration sce-
narios. The first demonstration scenario focuses on making
users understand the different parameters that affect the
performance of a blockchain fabric. The second demonstra-
tion scenario allows interested users to create and deploy
their own smart contracts on-the-fly. We explain these next.
3.1 Mix-and-Match
The key takeaway of the mix-and-match demonstration
is to make users experiment and observe the different pa-
rameters that affect throughput (transactions per second)
and latency (time from the client request to the response)
of a permissioned blockchain application. We give users a
GUI (see Figure 5) and ask them to mix-and-match the pa-
rameters listed in Figure 4.
We first require the user to Sign-up/Sign-in to our Re-
silientDB portal. Next, the user can Configure experiments
of its choice. To do so, the user first selects a BFT proto-
col. At present, we have already implemented four state-of-
the-art protocols. Next, the user decides whether it wants
clients to send YCSB transactions or to run Banking Smart
Contracts. The user can set the number of requests each
client includes in its transactions and the size of the batch
(if batching is employed by replicas). To test the limit of
the network, we also provide capability to add a predefined
load to messages.
We allow our users to select whether they want to enable
or disable pipelining. Enabling pipelining in ResilientDB al-
lows the various phases of a Bft protocol to be executed in
parallel. For example, PBFT is a three-phase preprepare-
prepare-commit protocol. If we pipeline PBFT, then one
transaction is prepared while previous transactions are com-
1/r e t u r n : 1 f o r c omm it , 0 f o r ab o r t /
3int Tr a ns f e rM o n ey : : ex e c u t e ()
5int source bal = db>Get ( t h i s >s o u r c e ) ;
6int d e s t b a l = db>Ge t ( t h i s >de s t ) ;
7i f (th i s >amount <= so u r c e ) {
8db>S e t ( t h i s >sou r c e , so u r c e ba l amou nt ) ;
9db>S e t ( t h i s >des t , d e s t b a l + amo un t ) ;
10 return 1 ;
11 }
12 return 0 ;
13 }
Figure 6: Transfer Smart Contract in ResilientDB.
mitted and executed. To ensure safety, the ordering is de-
layed until execution [5, 13]. In similar ways, the phases
of other protocols can be pipelined. As stated earlier, Re-
silientDB also divides tasks across threads. We allow users
to choose the number of threads needed to create batches, to
sign messages, to fetch data from the network and to place
output on the network. Further, we allow users to select the
type of storage for their blockchain ledger and client records.
At present, we support the in-memory databases SQLite and
LevelDB for storing the ledger and client records. Finally,
users can decide the type of cryptographic constructs, signa-
tures and digests, they want to employ. Note that a user
need not specify all parameters. In such a case, the system
will proceed with the default parameters.
Finally, the user can deploy the experiment via the Run
button, which initiates the script that will compile, deploy,
and run the experiment. The user is presented with a web-
page that tracks the experiment progress. The user also
has an option of monitoring the results in real-time. Once
the experiment completes, the user can query the InfluxDB
database holding all results.
3.2 Deploying Smart-Contract
The key takeaway of the Deploying Smart-Contract demon-
stration is to show how users can design their own applica-
tions in ResilientDB. We believe that demonstrating users
the ease with which they can use ResilientDB to create new
applications illustrates ResilientDB’s general applicability.
Say we want to design a banking application. The trans-
fer transaction is a key utility, as it allows movement of
money from one account to another. To support transfers,
we create a smart contract that allows a user Bob to trans-
fer an amount Xfrom his account to the account of Alice
(see Figure 6). Prior to transferring X, the smart contract
also needs to check if Bob (source) has at least amount X
(source bal) in his account. The smart contract needs ac-
cess to the database with client records, for which we use
We provide a base class (SmartContract) that developers
can inherit to define their functionalities. Further, the client
needs to provide the required parameters for the new smart
contract. For example, the client specifies the source, des-
tination and the amount. Note that these changes do not
affect the process of compiling, deploying, and running the
code, which can still be done through our GUI. Hence, with
simple changes, users can build their own applications using
our ResilientDB fabric.
In this demonstration, we present ResilientDB, a high-
throughput permissioned blockchain fabric. The key aim of
this demonstration is to make users understand the different
parameters that affect the throughput of a blockchain fabric.
We allow users to mix-and-match different parameters and
illustrate how easy it is for users to make their applications
around ResilientDB.
Acknowledgments. This work is partially supported by
the U.S. Department of Energy, Office of Science, Office of
Small Business Innovation Research, under Award Number
[1] M. J. Amiri, D. Agrawal, and A. E. Abbadi. CAPER: A
cross-application permissioned blockchain. PVLDB,
12(11):1385–1398, 2019.
[2] E. Androulaki et al. Hyperledger Fabric: A distributed
operating system for permissioned blockchains. In Proceedings
of the Thirteenth EuroSys Conference, pages 30:1–30:15.
ACM, 2018.
[3] A. Bessani, J. Sousa, and E. E. Alchieri. State machine
replication for the masses with BFT-SMART. In 44th Annual
IEEE/IFIP International Conference on Dependable Systems
and Networks, pages 355–362. IEEE, 2014.
[4] E. Buchman, J. Kwon, and Z. Milosevic. Revisiting fast
practical byzantine fault tolerance, 2018.
[5] M. Castro and B. Liskov. Practical byzantine fault tolerance
and proactive recovery. ACM Transactions on Computer
Systems, 20(4):398–461, 2002.
[6] B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and
R. Sears. Benchmarking cloud serving systems with YCSB. In
Proceedings of the 1st ACM Symposium on Cloud Computing,
pages 143–154. ACM, 2010.
[7] T. T. A. Dinh, R. Liu, M. Zhang, G. Chen, B. C. Ooi, and
J. Wang. Untangling blockchain: A data processing view of
blockchain systems. IEEE Trans. Knowl. Data Eng.,
30(7):1366–1385, 2018.
[8] Eyal, A. E. Gencer, E. G. Sirer, and R. V. Renesse. Bitcoin-NG:
A scalable blockchain protocol. In 13th USENIX Symposium
on Networked Systems Design and Implementation, pages
45–59, Santa Clara, CA, 2016. USENIX Association.
[9] G. Greenspan. Multichain private blockchain, 2015.
[10] S. Gupta, J. Hellings, S. Rahnama, and M. Sadoghi.
Proof-of-Execution: Reaching Consensus through
Fault-Tolerant Speculation. abs/1911.00838, 2019.
[11] S. Gupta, J. Hellings, and M. Sadoghi. Brief announcement:
Revisiting consensus protocols through wait-free parallelization.
In 33rd International Symposium on Distributed Computing
(DISC 2019), volume 146, pages 44:1–44:3, 2019.
[12] S. Gupta, J. Hellings, and M. Sadoghi. Scaling blockchain
databases through parallel resilient consensus paradigm.
abs/1911.00837, 2019.
[13] S. Gupta, S. Rahnama, J. Hellings, and M. Sadoghi.
ResilientDB: Global scale resilient blockchain fabric. PVLDB,
13(6):868–883, 2020.
[14] S. Gupta, S. Rahnama, and M. Sadoghi. Permissioned
blockchain through the looking glass: Architectural and
implementation lessons learned. In 40th International
Conference on Distributed Computing Systems. IEEE, 2020.
[15] J. Hellings and M. Sadoghi. Brief announcement: The
fault-tolerant cluster-sending problem. In 33rd International
Symposium on Distributed Computing (DISC 2019), volume
146, pages 45:1–45:3, 2019.
[16] J. Hellings and M. Sadoghi. Coordination-free byzantine
replication with minimal communication costs. In Proceedings
of the 23rd International Conference on Database Theory,
volume 155, 2020.
[17] J. Katz and Y. Lindell. Introduction to Modern Cryptography.
Chapman and Hall/CRC, 2nd edition, 2014.
[18] R. Kotla, L. Alvisi, M. Dahlin, A. Clement, and E. Wong.
Zyzzyva: Speculative byzantine fault tolerance. ACM
Transactions on Computer Systems, 27(4):7:1–7:39, 2009.
[19] Libra Association Members. An introduction to libra, 2019.
[20] S. Nakamoto. Bitcoin: A peer-to-peer electronic cash system,
[21] G. Wood. Ethereum: a secure decentralised generalised
transaction ledger, 2016. EIP-150 revision.
[22] M. Yin, D. Malkhi, M. K. Reiter, G. G. Gueta, and I. Abraham.
HotStuff: BFT consensus with linearity and responsiveness. In
Proceedings of the 2019 ACM Symposium on Principles of
Distributed Computing, pages 347–356. ACM, 2019.
... We evaluate our FlexiTrust variants against five protocols: three trust-bft and two bft systems. For fair evaluation, we implement these protocols on the open-source ResilientDB fabric [37,38,65,71]. Our evaluation on a realworld setup of 97 replicas and up to 80 k clients shows that our FlexiTrust protocols achieve up to 185% and 100% more throughput than their trust-bft and bft counterparts, respectively. ...
Conference Paper
Full-text available
The growing interest in reliable multi-party applications has fostered widespread adoption of Byzantine Fault-Tolerant (bft) consensus protocols. Existing bft protocols need f more replicas than Paxos-style protocols to prevent equivocation attacks. trust-bft protocols seek to minimize this cost by making use of trusted components at replicas. This paper makes two contributions. First, we analyze the design of existing trust-bft protocols and uncover three fundamental limitations that preclude most practical deployments. Some of these limitations are fundamental, while others are linked to the state of trusted components today. Second, we introduce a novel suite of consensus protocols, FlexiTrust, that attempts to sidestep these issues. We show that our FlexiTrust protocols achieve up to 185% more throughput than their trust-bft counterparts.
... The data processing Membership services provider. It is responsible for providing all member management services on the chain, including member registration and certificate management [37]. Users need to authenticate and authorize to become members of the alliance chain first, and then operate and maintain the data of the alliance chain as members to share the data. ...
Full-text available
Human resource data sharing is very important for the cooperation of human resource institutions. Since the human resource data-sharing service needs to take into account the needs of individuals and talent service institutions, it faces issues such as the security of information sharing and the traceability of information. Therefore, this paper constructs a human resource data-sharing service system based on blockchain technology. Its trust mechanism is based on the Fabric alliance chain, and the system makes full use of its advantages of decentralization and consensus. Our research mainly includes data sharing architecture design, consensus mechanism analysis, smart contract design, data sharing process, and blockchain construction. The contribution of this paper is mainly in two aspects. On the one hand, it explores the trust mechanism of human resource data sharing and gives the scheme of the Fabric alliance chain. On the other hand, the overall architecture and smart contract design are given on the construction of the blockchain, which provides a reference for future research on human resource data sharing.
... Bitcoin is one of the most popular digital cryptocurrencies [1][2][3], exhibiting characteristics of decentralization and anonymity. First introduced in Bitcoin, blockchain [4][5][6][7] combines techniques from cryptography, consensus mechanism, peer-to-peer (P2P) network and other related fields to realize a distributed database to store transactions [8,9]. Bitcoin network is an unstructured network, and it uses gossip protocol [10] to synchronize messages. ...
Bitcoin is a popular decentralized cryptocurrency, and the Bitcoin network is essentially an unstructured peer-to-peer (P2P) network that can synchronize distributed database of replicated ledgers through message broadcasting. In the Bitcoin network, the average clustering coefficient of nodes is very high, resulting in low message propagation efficiency. In addition, average node degree in the Bitcoin network is also considerably large, causing high message redundancy when nodes use the gossip protocol to broadcast messages. These may affect message propagation speed, hindering Bitcoin from being applied to scenarios of high transactional throughputs. To illustrate, we have collected single-hop propagation data of transactions of 366 blocks from Bitcoin Core. The analysis results show that transaction verification and network delay are two major causes of low transaction propagation efficiency. In this paper, we propose a novel P2P network structure, called local clique network (LCN), for message broadcasting in the Bitcoin network. Specifically, to reduce transaction validation latency and message redundancy, in LCN local nodes (logically) form cliques, and only a few nodes in a clique broadcast messages to the other cliques, instead of each node sending messages to its neighboring nodes. We have conducted extensive experiments, and the results show that message redundancy is low in LCN, and message propagation speed increases significantly. Meanwhile, LCN exhibits excellent robustness when average node degree remains high in the Bitcoin network.
... Besides of addressing security vulnerabilities [5] of SCADA systems such as single point of failure vulnerabilities and RTU's overhead. The adoption of the HLF network introduces security features of blockchain platforms [31] such as fault tolerance and data replication. Combining a distributed ledger and the SPAI protocol prevents critical information disclosure through lightweight crypto solutions even if malicious users intercept part or all transmitted data. ...
Full-text available
Industrial Control Systems and Supervisory Control and Data Acquisition (ICS/SCADA) systems are profound backbones of the national critical infrastructures and are essential to the sustainability of society since they help monitoring and controlling the cyber-enable services, such as energy, transportation, healthcare, etc. Modern SCADA systems continue to use the legacy communication protocols that lack adequate security mechanisms to provide trusted device authentication and ensure data flow integrity. Furthermore, advent of state-of-the-art network-capable sensor technology exposes many unique vulnerabilities to the adversaries. Thus, integrity of the data originated from field sensors along with their identity must be managed carefully in order to enhance reliability of ICS/SCADA ecosystems. In this paper, we present a blockchain-based SRAM PUF Authentication and Integrity (BloSPAI) protocol that aims to ensure a continuous authentication of field sensors and provide a robust data flow integrity process by leveraging distributed ledger and hardware security primitives. The prototype of the protocol has been implemented in a sensor-integrated Raspberry PI testbed that is interfaced with a permissioned blockchain network. We discuss the performance and overhead aspects of the proposed BloSPAI protocol and compare with state-of-art cybersecurity solutions. Through experimental evaluation demonstrates the relationship between the size of the blockchain network impacts the throughput in terms of time to commit transactions and overall systems setup time.
... Gupta et al 2020c;Rahnama et al 2020) is a state-of-the-art permissioned blockchain fabric, which is designed with the aim of fostering academic and industry research. RESILIENTDB also acts as a reliable test-bed to implement and evaluate ...
Full-text available
A blockchain is a linked list of immutable tamper-proof blocks, which is stored at each participating node. Each block records a set of transactions and the associated metadata. Blockchain transactions act on the identical ledger data stored at each node. Blockchain was first perceived by Satoshi Nakamoto, as a peer-to-peer money exchange system. Nakamoto referred to the transactional tokens exchanged among clients in his system as Bitcoins.
Full-text available
The recent growth of blockchain technology has accelerated research on decentralized platforms. Initial such platforms decide on what should be added to the ledger based on the Proof-of-Work (PoW) consensus protocol. PoW protocol requires its participants to perform massive computations and leads to massive energy wastage. Existing solutions to replace the PoW protocol make use of the Proof-of-Stake (PoS) protocol or classical fault-tolerant consensus protocols. However, the safety of the ledger created by these protocols is at the mercy of the long-term safe-keeping of the private keys of participants subject to long-range attacks. To ameliorate this situation, we present the design of our novel HybridChain architecture, which requires each client transaction to undergo two consensus protocols: a fault-tolerant consensus followed by our novel Power-of-Collaboration (PoC) protocol. Despite this, we observe that our HybridChain system outperforms state-of-the-art blockchain systems yielding up to 2000x higher throughput and 10^5 times less energy costs.
Conference Paper
Full-text available
Agreement protocols have been extensively used by distributed data management systems to provide robustness and high availability. The broad spectrum of design dimensions, applications, and fault models have resulted in different flavors of agreement protocols. This proliferation of agreement protocols has made it hard to argue their correctness and has unintentionally created a disparity in understanding their design. To address this disparity, we study the chemistry behind agreement and present a unified framework that simplifies expressing different agreement protocols. Specifically, we extract essential elements of the agreement and define atoms that connect these elements. We illustrate how these elements can help to explain and design various agreement protocols.
Full-text available
Recent developments in blockchain technology have inspired innovative new designs in resilient distributed and database systems. At their core, these blockchain applications typically use Byzantine fault-tolerant consensus protocols to maintain a common state across all replicas, even if some replicas are faulty or malicious. Unfortunately, existing consensus protocols are not designed to deal with geo-scale deployments in which many replicas spread across a geographically large area participate in consensus. To address this, we present the Geo-Scale Byzantine Fault-Tolerant consensus protocol (GeoBFT). GeoBFT is designed for excellent scalability by using a topological-aware grouping of replicas in local clusters, by introducing parallelization of consensus at the local level, and by minimizing communication between clusters. To validate our vision of high-performance geo-scale resilient distributed systems, we implement GeoBFT in our efficient ResilientDB permissioned blockchain fabric. We show that GeoBFT is not only sound and provides great scalability, but also outperforms state-of-the-art consensus protocols by a factor of six in geo-scale deployments.
Full-text available
Despite recent intensive research, existing blockchain systems do not adequately address all the characteristics of distributed applications. In particular, distributed applications collaborate with each other following service level agreements (SLAs) to provide different services. While collaboration between applications, e.g., cross-application transactions, should be visible to all applications, the internal data of each application, e.g, internal transactions, might be confidential. In this paper, we introduce CAPER, a permissioned blockchain system to support both internal and cross-application transactions of collaborating distributed applications. In CAPER, the blockchain ledger is formed as a directed acyclic graph where each application accesses and maintains only its own view of the ledger including its internal and all cross-application transactions. CAPER also introduces three consensus protocols to globally order cross-application transactions between applications with different internal consensus protocols. The experimental results reveal the efficiency of CAPER in terms of performance and scalability.
Conference Paper
Full-text available
We present HotStuff, a leader-based Byzantine fault-tolerant replication protocol for the partially synchronous model. Once network communication becomes synchronous, HotStuff enables a correct leader to drive the protocol to consensus at the pace of actual (vs. maximum) network delay--a property called responsiveness---and with communication complexity that is linear in the number of replicas. To our knowledge, HotStuff is the first partially synchronous BFT replication protocol exhibiting these combined properties. Its simplicity enables it to be further pipelined and simplified into a practical, concise protocol for building large-scale replication services.
Conference Paper
Full-text available
The last fifteen years have seen an impressive amount of work on protocols for Byzantine fault-tolerant (BFT) state machine replication (SMR). However, there is still a need for practical and reliable software libraries implementing this technique. BFT-SMART is an open-source Java-based library implementing robust BFT state machine replication. Some of the key features of this library that distinguishes it from similar works (e.g., PBFT and UpRight) are improved reliability, modularity as a first-class property, multicore-awareness, reconfiguration support and a flexible programming interface. When compared to other SMR libraries, BFT-SMART achieves better performance and is able to withstand a number of real-world faults that previous implementations cannot.
Conference Paper
Fabric is a modular and extensible open-source system for deploying and operating permissioned blockchains and one of the Hyperledger projects hosted by the Linux Foundation ( Fabric is the first truly extensible blockchain system for running distributed applications. It supports modular consensus protocols, which allows the system to be tailored to particular use cases and trust models. Fabric is also the first blockchain system that runs distributed applications written in standard, general-purpose programming languages, without systemic dependency on a native cryptocurrency. This stands in sharp contrast to existing block-chain platforms that require "smart-contracts" to be written in domain-specific languages or rely on a cryptocurrency. Fabric realizes the permissioned model using a portable notion of membership, which may be integrated with industry-standard identity management. To support such flexibility, Fabric introduces an entirely novel blockchain design and revamps the way blockchains cope with non-determinism, resource exhaustion, and performance attacks. This paper describes Fabric, its architecture, the rationale behind various design decisions, its most prominent implementation aspects, as well as its distributed application programming model. We further evaluate Fabric by implementing and benchmarking a Bitcoin-inspired digital currency. We show that Fabric achieves end-to-end throughput of more than 3500 transactions per second in certain popular deployment configurations, with sub-second latency, scaling well to over 100 peers.
Blockchain technologies are gaining massive momentum in the last few years. Blockchains are distributed ledgers that enable parties who do not fully trust each other to maintain a set of global states. The parties agree on the existence, values and histories of the states. As the technology landscape is expanding rapidly, it is both important and challenging to have a firm grasp of what the core technologies have to offer, especially with respect to their data processing capabilities. In this paper, we first survey the state of the art, focusing on private blockchains (in which parties are authenticated). We analyze both in-production and research systems in four dimensions: distributed ledger, cryptography, consensus protocol and smart contract. We then present BLOCKBENCH, a benchmarking framework for understanding performance of private blockchains against data processing workloads. We conduct a comprehensive evaluation of three major blockchain systems based on BLOCKBENCH, namely Ethereum, Parity and Hyperledger Fabric. The results demonstrate several trade-offs in the design space, as well as big performance gaps between blockchain and database systems. Drawing from design principles of database systems, we discuss several research directions for bringing blockchain performance closer to the realm of databases.
A longstanding vision in distributed systems is to build reliable systems from unreliable components. An enticing formulation of this vision is Byzantine Fault-Tolerant (BFT) state machine replication, in which a group of servers collectively act as a correct server even if some of the servers misbehave or malfunction in arbitrary (“Byzantine”) ways. Despite this promise, practitioners hesitate to deploy BFT systems, at least partly because of the perception that BFT must impose high overheads. In this article, we present Zyzzyva, a protocol that uses speculation to reduce the cost of BFT replication. In Zyzzyva, replicas reply to a client's request without first running an expensive three-phase commit protocol to agree on the order to process requests. Instead, they optimistically adopt the order proposed by a primary server, process the request, and reply immediately to the client. If the primary is faulty, replicas can become temporarily inconsistent with one another, but clients detect inconsistencies, help correct replicas converge on a single total ordering of requests, and only rely on responses that are consistent with this total order. This approach allows Zyzzyva to reduce replication overheads to near their theoretical minima and to achieve throughputs of tens of thousands of requests per second, making BFT replication practical for a broad range of demanding services.
A purely peer-to-peer version of electronic cash would allow online payments to be sent directly from one party to another without going through a financial institution. Digital signatures provide part of the solution, but the main benefits are lost if a trusted third party is still required to prevent double-spending. We propose a solution to the double-spending problem using a peer-to-peer network. The network timestamps transactions by hashing them into an ongoing chain of hash-based proof-of-work, forming a record that cannot be changed without redoing the proof-of-work. The longest chain not only serves as proof of the sequence of events witnessed, but proof that it came from the largest pool of CPU power. As long as a majority of CPU power is controlled by nodes that are not cooperating to attack the network, they'll generate the longest chain and outpace attackers. The network itself requires minimal structure. Messages are broadcast on a best effort basis, and nodes can leave and rejoin the network at will, accepting the longest proof-of-work chain as proof of what happened while they were gone.
Conference Paper
While the use of MapReduce systems (such as Hadoop) for large scale data analysis has been widely recognized and studied, we have recently seen an explosion in the number of systems developed for cloud data serving. These newer systems address "cloud OLTP" applications, though they typically do not support ACID transactions. Examples of systems proposed for cloud serving use include BigTable, PNUTS, Cassandra, HBase, Azure, CouchDB, SimpleDB, Voldemort, and many others. Further, they are being ap- plied to a diverse range of applications that di!er consider- ably from traditional (e.g., TPC-C like) serving workloads. The number of emerging cloud serving systems and the wide range of proposed applications, coupled with a lack of apples- to-apples performance comparisons, makes it di"cult to un- derstand the tradeo!s between systems and the workloads for which they are suited. We present the Yahoo! Cloud Serving Benchmark (YCSB) framework, with the goal of fa- cilitating performance comparisons of the new generation of cloud data serving systems. We define a core set of benchmarks and report results for four widely used systems: Cassandra, HBase, Yahoo!'s PNUTS, and a simple sharded MySQL implementation. We also hope to foster the devel- opment of additional cloud benchmark suites that represent other classes of applications by making our benchmark tool available via open source. In this regard, a key feature of the YCSB framework/tool is that it is extensible—it supports easy definition of new workloads, in addition to making it easy to benchmark new systems.