No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Elastic Resource Allocation in Sharding-Based Blockchains
Abstract
This chapter studies the PBFT-based sharded permissioned blockchain, which executes in either a local datacenter or a rented cloud platform. In such permissioned blockchain, the transaction (TX) assignment strategy could be malicious such that the network shards may possibly receive imbalanced transactions or even bursty-TX injection attacks. An imbalanced transaction assignment brings serious threats to the stability of the sharded blockchain. A stable sharded blockchain can ensure that each shard processes the arrived transactions timely. Since the system stability is closely related to the blockchain throughput, how to maintain a stable sharded blockchain becomes a challenge. To depict the transaction processing in each network shard, we adopt the Lyapunov Optimization framework. Exploiting drift-plus-penalty (DPP) technique, we then propose an adaptive resource-allocation algorithm, which can yield the near-optimal solution for each network shard while the shard queues can also be stably maintained. We also rigorously analyze the theoretical boundaries of both the system objective and the queue length of shards. The numerical results show that the proposed algorithm can achieve a better balance between resource consumption and queue stability than other baselines. We particularly evaluate two representative cases of bursty-TX injection attacks, i.e., the continued attacks against all network shards and the drastic attacks against a single network shard. The evaluation results show that the DPP-based algorithm can well alleviate the imbalanced TX assignment, and simultaneously maintain high throughput while consuming fewer resources than other baselines.KeywordsSystem scalabilitySharded blockchainQueueing theoryImbalanced transaction assignment
ResearchGate has not been able to resolve any citations for this publication.
The Blockchain technology, featured with its decentralized tamper-resistance based on a Peer-to-Peer network, has been widely applied in financial applications, and even further been extended to industrial applications. However, the weak scalability of traditional Blockchain technology severely affects the wide adoption due to the well-known trillema of decentralization-security-scalability in Blockchains. In regards to this issue, a number of solutions have been proposed, targeting to boost the scalability while preserving the decentralization and security. They range from modifying the on-chain data structure and consensus algorithms to adding the off-chain technologies. Therein, one of the most practical methods to achieve horizontal scalability along with the increasing network size is sharding, by partitioning network into multiple shards so that the overhead of duplicating communication, storage, and computation in each full node can be avoided. This paper presents a survey focusing on sharding in Blockchains in a systematic and comprehensive way. We provide detailed comparison and quantitative evaluation of major sharding mechanisms, along with our insights analyzing the features and restrictions of the existing solutions.We also provide theoretical upper-bound of the throughput for each considered sharding mechanism. The remaining challenges and future research directions are also reviewed.
Satellite-based communication technology regains much attention in the past few years, where satellites play mainly the supplementary roles as relay devices to terrestrial communication networks. Unlike previous work, we treat the low-earth-orbit (LEO) satellites as secure data storage mediums [1]. We focus on data acquisition from an LEO satellite-based data storage system (also referred to as the LEO based datacenters), which has been considered as a promising and secure paradigm on data storage. Under the LEO based datacenter architecture, one fundamental challenge is to deal with energy-efficient downloading from space to ground while maintaining the system stability. In this paper, we aim to maximize the amount of data admitted while minimizing the energy consumption, when downloading files from LEO based datacenters to meet user demands. To this end, we first formulate a novel optimization problem and develop an online scheduling framework. We then devise a novel coflow-like "Join the first K-shortest Queues (JKQ)" based job-dispatch strategy, which can significantly lower backlogs of queues residing in LEO satellites, thereby improving the system stability. We also analyze the optimality of the proposed approach and system stability. We finally evaluate the performance of the proposed algorithm through conducting emulator based simulations, based on real-world LEO constellation and user demand traces. The simulation results show that the proposed algorithm can dramatically lower the queue backlogs and achieve high energy efficiency.
Existing blockchain systems scale poorly because of their distributed consensus protocols. Current attempts at improving blockchain scalability are limited to cryptocurrency. Scaling blockchain systems under general workloads (i.e., non-cryptocurrency applications) remains an open question. This work takes a principled approach to apply sharding to blockchain systems in order to improve their transaction throughput at scale. This is challenging, however, due to the fundamental difference in failure models between databases and blockchain. To achieve our goal, we first enhance the performance of Byzantine consensus protocols, improving individual shards' throughput. Next, we design an efficient shard formation protocol that securely assigns nodes into shards. We rely on trusted hardware, namely Intel SGX, to achieve high performance for both consensus and shard formation protocol. Third, we design a general distributed transaction protocol that ensures safety and liveness even when transaction coordinators are malicious. Finally, we conduct an extensive evaluation of our design both on a local cluster and on Google Cloud Platform. The results show that our consensus and shard formation protocols outperform state-of-the-art solutions at scale. More importantly, our sharded blockchain reaches a high throughput that can handle Visa-level workloads, and is the largest ever reported in a realistic environment.
With the support of the blockchain systems, the cryptocurrency has changed the world of virtual assets. Digital games, especially those with massive multi-player scenarios, will be significantly impacted by this novel technology. However, there are insufficient academic studies on this topic. In this work, we filled the blank by surveying the state-of-the-art blockchain games. We discuss the blockchain integration for games and then categorize existing blockchain games from the aspects of their genres and technical platforms. Moreover, by analyzing the industrial trend with a statistical approach, we envision the future of blockchain games from technological and commercial perspectives.
Internet of Things (IoT) is reshaping the incumbent industry to smart industry featured with data-driven decision-making. However, intrinsic features of IoT result in a number of challenges such as decentralization, poor interoperability, privacy and security vulnerabilities. Blockchain technology brings the opportunities in addressing the challenges of IoT. In this paper, we investigate the integration of blockchain technology with IoT. We name such synthesis of blockchain and IoT as Blockchain of Things (BCoT). This paper presents an in-depth survey of BCoT and discusses the insights of this new paradigm. In particular, we first briefly introduce IoT and discuss the challenges of IoT. Then we give an overview of blockchain technology. We next concentrate on introducing the convergence of blockchain and IoT and presenting the proposal of BCoT architecture. We further discuss the issues about using blockchain for 5G beyond in IoT as well as industrial applications of BCoT. Finally, we outline the open research directions in this promising area.
https://ieeexplore.ieee.org/document/8731639
Despite the growing interest in cryptocurrencies, the delays incurred to confirm transactions are one of the factors that hamper the wide adoption of systems such as Bitcoin. Bitcoin transactions usually are confirmed in short periods (minutes), but still much larger than conventional credit card systems (seconds). In this work, we propose a framework encompassing machine learning and a queueing theory model to (i) identify which transactions will be confirmed; and (ii) characterize the confirmation time of confirmed transactions. The proposed queueing theory model accounts for factors such as the activity time of blocks and the mean time between transactions. We parameterize the model for transactions that are confirmed within minutes, suggesting that its integration into a more general framework is a step towards building scalability to Bitcoin.
The report starts with an overview of the blockchain security system and then highlights the specific security threats and summarizes them. We review with some comments and possible research direction. This survey, we examines the security issues of blockchain model related technologies and their applications. The blockchain is considered a still growing like the internet in 1990. It has the potential to disrupt so many technology areas in the future. But as a new underdeveloped field, it is suffering many setbacks mostly resulting from the security area. Its security concerns coming not only from distributed/decentralized computing issue or Cryptography algorithm issue, from some unexpected field too. Here, in this paper, we tried to classify the security concerns for the blockchain based on our survey from recent research papers. We also tried to show which way blockchain development trends are going.
Designing a secure permissionless distributed ledger (blockchain) that performs on par with centralized payment processors, such as Visa, is a challenging task. Most existing distributed ledgers are unable to scale-out, i.e., to grow their total processing capacity with the number of validators; and those that do, compromise security or decentralization. We present OmniLedger, a novel scale-out distributed ledger that preserves longterm security under permissionless operation. It ensures security and correctness by using a bias-resistant public-randomness protocol for choosing large, statistically representative shards that process transactions, and by introducing an efficient cross-shard commit protocol that atomically handles transactions affecting multiple shards. OmniLedger also optimizes performance via parallel intra-shard transaction processing, ledger pruning via collectively-signed state blocks, and low-latency "trust-but-verify" validation for low-value transactions. An evaluation of our experimental prototype shows that OmniLedger's throughput scales linearly in the number of active validators, supporting Visa-level workloads and beyond, while confirming typical transactions in under two seconds.
Blockchain has recently been applied in many applications such as bitcoin, smart grid, and Internet of Things (IoT) as a public ledger of transactions. However, the use of blockchain in mobile environments is still limited because the mining process consumes too much computing and energy resources on mobile devices. Edge computing offered by the Edge Computing Service Provider can be adopted as a viable solution for offloading the mining tasks from the mobile devices, i.e., miners, in the mobile blockchain environment. However, a mechanism needs to be designed for edge resource allocation to maximize the revenue for the Edge Computing Service Provider and to ensure incentive compatibility and individual rationality is still open. In this paper, we develop an optimal auction based on deep learning for the edge resource allocation. Specifically, we construct a multi-layer neural network architecture based on an analytical solution of the optimal auction. The neural networks first perform monotone transformations of the miners' bids. Then, they calculate allocation and conditional payment rules for the miners. We use valuations of the miners as the data training to adjust parameters of the neural networks so as to optimize the loss function which is the expected, negated revenue of the Edge Computing Service Provider. We show the experimental results to confirm the benefits of using the deep learning for deriving the optimal auction for mobile blockchain with high revenue
Blockchain, an emerging decentralized security system, has been applied in many applications, such as bitcoin, smart grid, and Internet-of-Things. However, running the mining process may cost too much energy consumption and computing resource usage on handheld devices, which restricts the use of blockchain in mobile environments. In this paper, we consider deploying edge computing service to support the mobile blockchain. We propose an auction-based edge computing resource market of the edge computing service provider. Since there is competition among miners, the allocative externalities (positive and negative) are taken into account in the model. In our auction mechanism, we maximize the social welfare while guaranteeing the truthfulness, individual rationality and computational efficiency. Based on blockchain mining experiment results, we define a hash power function that characterizes the probability of successfully mining a block. Through extensive simulations, we evaluate the performance of our auction mechanism which shows that our edge computing resources market model can efficiently solve the social welfare maximization problem for the edge computing service provider.
This paper describes a new replication algorithm that is able to tolerate Byzantine faults. We believe that Byzantinefault -tolerant algorithms will be increasingly important in the future because malicious attacks and software errors are increasingly common and can cause faulty nodes to exhibit arbitrary behavior. Whereas previous algorithms assumed a synchronous system or were too slow to be used in practice, the algorithm described in this paper is practical: it works in asynchronous environments like the Internet and incorporates several important optimizations that improve the response time of previous algorithms by more than an order of magnitude. We implemented a Byzantine-fault-tolerant NFS service using our algorithm and measured its performance. The results show that our service is only 3% slower than a standard unreplicated NFS. 1 Introduction Malicious attacks and software errors are increasingly common. The growing reliance of industry and government on online information...
Since 2016, sharding has become an auspicious solution to tackle the scalability issue in legacy blockchain systems. Despite its potential to strongly boost the blockchain throughput, sharding comes with its own security issues. To ease the process of deciding which shard to place transactions, existing sharding protocols use a hash-based transaction sharding in which the hash value of a transaction determines its output shard. Unfortunately, we show that this mechanism opens up a loophole that could be exploited to conduct a single-shard flooding attack, a type of Denial-of-Service (DoS) attack, to overwhelm a single shard that ends up reducing the performance of the system as a whole. To counter the single-shard flooding attack, we propose a countermeasure that essentially eliminates the loophole by rejecting the use of hash-based transaction sharding. The countermeasure leverages the Trusted Execution Environment (TEE) to let blockchain's validators securely execute a transaction sharding algorithm with a negligible overhead. We provide a formal specification for the countermeasure and analyze its security properties in the Universal Composability (UC) framework. Finally, a proof-of-concept is developed to demonstrate the feasibility and practicality of our solution.
With the advancement and emergence of diverse network services in Internet of Vehicles (IoV), large volume of data are collected and stored, making data important properties. Data will be one of the most important commodities in the future blockchain-based IoV systems. However, efficiency challenges have been commonly found in blockchain-based data markets, which is mainly caused by transaction confirmation delays and the cold-start problems for new users. To address the efficiency challenges, we propose a secure, decentralized IoV data-trading system by exploiting the blockchain technology, and design an efficient debt-credit mechamism to support efficient data-trading in IoV. In the debt-credit mechanism, a vehicle with loan demand could loan from multi-vehicles by promising to pay interest and reward. In particular, we encourage loaning among vehicles by a motivation-based investing and pricing mechanism. We formulate a two-stage Stackelberg game to maximize the profits of borrower vehicle and lender vehicles jointly. In the first stage, the borrower vehicle set the interest rate and reward for the loan as its pricing strategies. In the second stage, the lender vehicles decide on their investing strategies. We apply backward induction to analyze the sub-game perfect equilibrium at each stage for both independent and uniform pricing schemes. We also validate the existence and uniqueness of Stackelberg equilibrium. The numerical results illustrate the efficiency of the proposed pricing schemes.
A major approach to overcoming the performance and scalability limitations of current blockchain protocols is to use sharding which is to split the overheads of processing transactions among multiple, smaller groups of nodes. These groups work in parallel to maximize performance while requiring significantly smaller communication, computation, and storage per node, allowing the system to scale to large networks. However, existing sharding-based blockchain protocols still require a linear amount of communication (in the number of participants) per transaction, and hence, attain only partially the potential benefits of sharding. We show that this introduces a major bottleneck to the throughput and latency of these protocols. Aside from the limited scalability, these protocols achieve weak security guarantees due to either a small fault resiliency (e.g., 1/8 and 1/4) or high failure probability, or they rely on strong assumptions (e.g., trusted setup) that limit their applicability to mainstream payment systems. We propose RapidChain, the first sharding-based public blockchain protocol that is resilient to Byzantine faults from up to a 1/3 fraction of its participants, and achieves complete sharding of the communication, computation, and storage overhead of processing transactions without assuming any trusted setup. RapidChain employs an optimal intra-committee consensus algorithm that can achieve very high throughputs via block pipelining, a novel gossiping protocol for large blocks, and a provably-secure reconfiguration mechanism to ensure robustness. Using an efficient cross-shard transaction verification technique, our protocol avoids gossiping transactions to the entire network. Our empirical evaluations suggest that RapidChain can process (and confirm) more than 7,300 tx/sec with an expected confirmation latency of roughly 8.7 seconds in a network of 4,000 nodes with an overwhelming time-to-failure of more than 4,500 years.
Exploiting both LTE and Wi-Fi links simultaneously enhances the performance of video streaming services in a smartphone. However, it is challenging to achieve seamless and high quality video while saving battery energy and LTE data usage to prolong the usage time of a smartphone. In this paper, we propose REQUEST, a video chunk request policy for Dynamic Adaptive Streaming over HTTP (DASH) in a smartphone, which can utilize both LTE and Wi-Fi. REQUEST enables seamless DASH video streaming with near optimal video quality under given budgets of battery energy and LTE data usage. Through extensive simulation and measurement in a real environment, we demonstrate that REQUEST significantly outperforms other existing schemes in terms of average video bitrate, rebuffering, and resource waste.
Cryptocurrencies, such as Bitcoin and 250 similar alt-coins, embody at their core a blockchain protocol --- a mechanism for a distributed network of computational nodes to periodically agree on a set of new transactions. Designing a secure blockchain protocol relies on an open challenge in security, that of designing a highly-scalable agreement protocol open to manipulation by byzantine or arbitrarily malicious nodes. Bitcoin's blockchain agreement protocol exhibits security, but does not scale: it processes 3--7 transactions per second at present, irrespective of the available computation capacity at hand.
In this paper, we propose a new distributed agreement protocol for permission-less blockchains called ELASTICO. ELASTICO scales transaction rates almost linearly with available computation for mining: the more the computation power in the network, the higher the number of transaction blocks selected per unit time. ELASTICO is efficient in its network messages and tolerates byzantine adversaries of up to one-fourth of the total computational power. Technically, ELASTICO uniformly partitions or parallelizes the mining network (securely) into smaller committees, each of which processes a disjoint set of transactions (or "shards"). While sharding is common in non-byzantine settings, ELASTICO is the first candidate for a secure sharding protocol with presence of byzantine adversaries. Our scalability experiments on Amazon EC2 with up to nodes confirm ELASTICO's theoretical scaling properties.
Spanner is Google’s scalable, multiversion, globally distributed, and synchronously replicated database. It is the first system to distribute data at global scale and support externally-consistent distributed transactions. This article describes how Spanner is structured, its feature set, the rationale underlying various design decisions, and a novel time API that exposes clock uncertainty. This API and its implementation are critical to supporting external consistency and a variety of powerful features: nonblocking reads in the past, lock-free snapshot transactions, and atomic schema changes, across all of Spanner.
We consider a stochastic model of jobs arriving at a cloud data center. Each job requests a certain amount of CPU, memory, disk space, etc. Job sizes (durations) are also modeled as random variables, with possibly unbounded support. These jobs need to be scheduled non preemptively on servers. The jobs are first routed to one of the servers when they arrive and are queued at the servers. Each server then chooses a set of jobs from its queues so that it has enough resources to serve all of them simultaneously. This problem has been studied previously under the assumption that job sizes are known and upper bounded, and an algorithm was proposed which stabilizes traffic load in a diminished capacity region. Here, we present a load balancing and scheduling algorithm that is throughput optimal, without assuming that job sizes are known or are upper bounded.
A purely peer-to-peer version of electronic cash would allow online payments to be sent directly from one party to another without going through a financial institution. Digital signatures provide part of the solution, but the main benefits are lost if a trusted third party is still required to prevent double-spending. We propose a solution to the double-spending problem using a peer-to-peer network. The network timestamps transactions by hashing them into an ongoing chain of hash-based proof-of-work, forming a record that cannot be changed without redoing the proof-of-work. The longest chain not only serves as proof of the sequence of events witnessed, but proof that it came from the largest pool of CPU power. As long as a majority of CPU power is controlled by nodes that are not cooperating to attack the network, they'll generate the longest chain and outpace attackers. The network itself requires minimal structure. Messages are broadcast on a best effort basis, and nodes can leave and rejoin the network at will, accepting the longest proof-of-work chain as proof of what happened while they were gone.
Blockchain queue theory
- Q L Li
- J Y Ma
- Y X Chang
Q.-L. Li, J.-Y. Ma, and Y.-X. Chang, "Blockchain queue theory," in
Proc. Int. Conf. Comput. Soc. Netw., 2018, pp. 25-40.
Monoxide: Scale out blockchains with asynchronous consensus zones
- J Wang
- H Wang
J. Wang and H. Wang, "Monoxide: Scale out blockchains with
asynchronous consensus zones," in Proc. 16th USENIX Symp.
Netw. Syst. Des. Implementation, 2019, pp. 95-112. [Online]. Available: https://www.usenix.org/conference/nsdi19/presentation/
wang-jiaping
Sharing medical questionnaries based on blockchain
- M G Kim
- A R Lee
- H J Kwon
- J W Kim
- I K Kim
M. G. Kim, A. R. Lee, H. J. Kwon, J. W. Kim, and I. K. Kim,
"Sharing medical questionnaries based on blockchain," in Proc.
IEEE Int. Conf. Bioinf. Biomed., 2018, pp. 2767-2769.
Solutions to scalability of blockchain: A survey
- Q Zhou
- H Huang
- Z Zheng
- J Bian
Q. Zhou, H. Huang, Z. Zheng, and J. Bian, "Solutions to scalability
of blockchain: A survey," IEEE Access, vol. 8, pp. 16 440-16 455,
2020.
Chainspace: A sharded smart contracts platform
- M Al-Bassam
- A Sonnino
- S Bano
- D Hrycyszyn
- G Danezis
M. Al-Bassam, A. Sonnino, S. Bano, D. Hrycyszyn, and G. Danezis, "Chainspace: A sharded smart contracts platform," in Proc.
Netw. Distrib. Syst. Secur. Symp., 2018, pp. 1-15.
Ethereum sharding: Overview and finality
- H.-W Wang
H.-W. Wang, "Ethereum sharding: Overview and finality," Dec.
2017. [Online]. Available: https://medium.com/@icebearhww/
ethereum-sharding-and-finality-65248951f649
He is currently an associate professor with Sun Yat-Sen University. He has served as a research fellow of JSPS, and an assistant professor with Kyoto University
- Huawei Huang
Huawei Huang (Member, IEEE) received the
PhD degree in computer science and engineering
from the University of Aizu, Aizuwakamatsu,
Japan, in 2016. He is currently an associate professor with Sun Yat-Sen University. He has
served as a research fellow of JSPS, and an
assistant professor with Kyoto University, Japan.
His research interests include blockchain and distributed computing. He is now serving as a guest
editor of the IEEE Journal on Selected Areas in
Communications and IEEE Open Journal of the Computer Society, the
operation-committee chair for the IEEE Symposium on Blockchain at
IEEE SERVICES 2021, and the TPC co-chair of GLOBECOM'2021/
ICC'2022 Workshop on scalable, secure, and intelligent blockchain.
Zhengyu Yue is currently working toward the
master's degree in the School of Computer Science and Engineering, Sun Yat-Sen University,
Guangzhou, China. His research interest include
blockchain.
He is currently an associate professor with Sun Yat-sen University
- Wuhui Chen
Wuhui Chen (Member, IEEE) received the bachelor's degree from Northeast University, Shengyang,
China, in 2008, and the master's and PhD degrees
from the University of Aizu, Aizuwakamatsu, Japan,
in 2011 and 2014, respectively. From 2014 to 2016,
he was a research fellow with the Japan Society for
the Promotion of Science, Tokyo, Japan. From 2016
to 2017, he was a researcher with the University of
Aizu. He is currently an associate professor with Sun
Yat-sen University, Guangzhou, China. His current
research interests include edge/cloud computing,
cloud robotics, and blockchain.