tMPT: Reconfiguration across Blockchain Shards via Trimmed Merkle Patricia Trie
Abstract and Figures
Sharding is one of the most promising techniques that can improve the scalability and storage issues of blockchain systems. In the past few years, many sharding protocols have been proposed to contribute to the technique matrix of blockchain sharding such as the coordination mechanisms of blockchain shards, the handling of cross-shard transactions, the security guarantee of blockchain shards, etc. Although those previous solutions are crucial for sharded blockchains, we still have not found any systematic implementation of shard reconfiguration, which determines the security of a sharded blockchain because the shuffling of blockchain nodes could prevent malicious nodes from corrupting a shard. However, the implementation of shard reconfiguration is not easy. When reallocating blockchain nodes to designated shards, typical challenges include the following: i) how to synchronize a large size of state data for a newly arrived node, and ii) how to mitigate the large reconfiguration latency of blockchain shards while keeping the liveness and consistency properties of a blockchain system. To overcome those challenges, we propose a dedicated protocol for shard reconfiguration in blockchain sharding using trimmed Merkle Patricia Trie (tMPT). The proposed tMPT-based protocol is designed to guarantee the high efficiency of the reconfiguration of blockchain shards while ensuring the uninterrupted services of the sharded blockchain. We implement the proposed tMPT-based reconfiguration protocol in a prototype, which enables the functionality of blockchain sharding. We then deploy our prototype in Alibaba Cloud. The experimental results show that the proposed tMPT-based protocol outperforms the existing methods in terms of reconfiguration efficiency. For example, the throughput of the proposed protocol shows 198% higher than Ethereum's full sync method.
Figures - uploaded by Huawei Huang
Author content
All figure content in this area was uploaded by Huawei Huang
Content may be subject to copyright.
... Moreover, the implementation of shard reconfiguration faces several challenges, including how to synchronize a large amount of state data and how to reduce the latency of largescale blockchain shard reconfigurations. Huang et al. [12] devise a new MPT structure, named tMPT, to make state data synchronization more efficient. Meanwhile, a new shard reconfiguration protocol is proposed to minimize the impact of shard reconfiguration on transaction processing. ...
Blockchain sharding is a significant technical area, improving the scalability of blockchain systems. It is regarded as one of the potential solutions that can achieve on-chain scaling, and significantly improve the scalability of blockchains without alleviating the decentralization feature of blockchain. To provide a reference and inspire participation from both the academic and industrial sectors in the area of blockchain sharding, we have researched the state-of-the-art studies published in the past three years. We have also conducted experiments to show the performance of representative sharding protocols such as Monoxide, LBF, Metis, and BrokerChain. We envision the potential challenges and promising future of sharding techniques in terms of the urgent demands of high throughput required by emerging applications such as Web3, Metaverse, and Decentralized Finance (DeFi). We hope that this article is helpful to researchers, engineers, and educators, and will inspire subsequent studies in the field of blockchain sharding.
Sharding technology is widely regarded as an effective way to solve blockchain scalability limitations, such as lower throughput and longer delay time. However, sharding encounters challenges related to high cross-shard transactions proportion and complex cross-shard transaction verification. Therefore, a transaction sharding algorithm based on account-weighted graph is proposed. Firstly, an account-based transaction sharding model is established from the viewpoint of data sharding. Secondly, based on this model, an account-weighted graph is constructed with accounts as nodes and transaction frequency between accounts as weights for the long-term accumulated transaction data of blockchain. The community discovery algorithm is adopted, and the accounts with the largest modularity gain are selected and merged according to the association relationship between the accounts, so multi-shards blockchain is formed initially. Finally, multi-shards blockchain are adjusted and rebuilt by splitting and merging. The proposed algorithm is compared with modular sharding algorithm and random sharding algorithm under the sharding size of 10. The cross-shard transaction proportion is reduced by 78%, the average cross-shard transaction delay of accounts is reduced by 36.1% and 44.5%, and the transaction throughput is increased by 93% and 122%. Under the other granularities, the proposed algorithm also outperforms the two algorithms in terms of cross-shard transaction proportion and account transaction delay. In conclusion, the method effectively reduces the cross-shard transaction proportion and minimizes cross-shard transaction delay.
Sharding technique is viewed as the most promising solution to improving blockchain scalability. However, to implement a sharded blockchain, developers have to address two major challenges. The first challenge is that the ratio of cross-shard transactions (TXs) across blockchain shards is very high. This issue significantly degrades the throughput of a blockchain. The second challenge is that the workloads across blockchain shards are largely imbalanced. If workloads are imbalanced, some shards have to handle an overwhelming number of TXs and become congested very possibly. Facing these two challenges, a dilemma is that it is difficult to guarantee a low cross-shard TX ratio and maintain the workload balance across all shards, simultaneously. We believe that a fine-grained account-allocation strategy can address this dilemma. To this end, we first formulate the tradeoff between such two metrics as a network-partition problem. We then solve this problem using a community-aware account partition algorithm. Furthermore, we also propose a sharding protocol, named Transformers, to apply the proposed algorithm into the sharded blockchain system. Finally, trace-driven evaluation results demonstrate that the proposed protocol outperforms other baselines in terms of throughput, latency, cross-shard TX ratio, and the queue size of transaction pool.
Sharding is the widely used approach to the trilemma of simultaneously achieving decentralization, security, and scalability in traditional blockchain systems. However, existing schemes generally involve problems such as uneven shard arithmetic power and insecure cross-shard transaction processing. In this study, we used the Practical Byzantine Fault Tolerance (PBFT) as the intra-shard consensus and, here, we propose a new sharding consensus mechanism. Firstly, we combined a jump consistent hash algorithm with signature Anchorhash to minimize the mapping of the node assignment. Then, we improved the process of the cross-shard transaction and used the activity of nodes participating in intra-shard transactions as the criterion for the shard reconfiguration, which ensured the security of the blockchain system. Meanwhile, we analyzed the motivation mechanism from two perspectives. Finally, through theoretical analysis and related experiments, we not only verified that the algorithm can ensure the security of the entire system, but also further clarified the necessary conditions to ensure the effectiveness of the shards and the system on the original basis.
Blockchain is simply defined as a decentralized, immutable, and distributed ledger for recording transactions, tracking digital assets and building trust. Blockchain, such as Bitcoin and Ethereum, has excellent security against adversarial attacks, but suffers from scalability issues. Indeed, the number of transactions per second (tx/s) that can be processed is too small and insufficient (e.g., up to 7 tx/s for Bitcoin and 15 tx/s for Ethereum). This is unacceptable for most traditional centralized payment systems that require thousands of transactions per second (e.g., Visa handles an average of 1700 tx/s). Sharding is one of the most promising and leading solutions to scale blockchain. The basic idea behind sharding is to divide the blockchain network into multiple committees/shards, where each processes a separate set of transactions, rather than the entire network processes all transactions. However, sharding raises security issues due to its 1 \% attacks. Therefore, the design of strong models to analyze the security of sharding-based blockchain protocols is needed.
This thesis consists of four main contributions. In the first contribution, we review and compare existing solutions to blockchain scalability. In particular, we focus on sharding as a promising solution to the scalability issue. Second, we propose a mathematical model (three probability bounds are used: Chebyshev, Hoeffding, and Chv\'atal) to analyze the security of sharding-based blockchain protocols. More specifically, we bound the failure for one committee and so for each epoch using probability bounds for sums of upper-bounded hypergeometric and binomial distributions. Moreover, we analyze the following fundamental question: ``how to keep the failure probability, for a given sharding protocol, smaller than a predefined threshold?''. In the third contribution, we propose a method to compute exactly the failure probability (instead of bounding it). The proposed method computes the failure probability for one sharding round taking into consideration the failure probabilities of all shards. However, using this method is intractable since we need to execute a huge number of trials, requiring high computational power and time, to accurately estimate the failure probability. In the fourth contribution, we propose a novel probabilistic method that is as accurate as the third contribution but tractable. In addition, we investigate the threat of Sybil attacks and compute the probability of a successful attack in these protocols.
State-of-the-art blockchain sharding solutions, say Monoxide, can induce imbalanced transaction (TX) distributions among all blockchain shards due to their account deployment mechanisms. Imbalanced TX distributions can then cause hot shards, in which the cross-shard TXs may experience an unlimited length of confirmation latency. Thus, how to address the hot shard issue and how to reduce cross-shard TXs become significant challenges in the context of blockchain state sharding. Through reviewing the related studies, we find that a cross-shard TX protocol that can achieve workload balance among all blockchain shards and simultaneously reduce the number of cross-shard TXs is still absent from the literature. To this end, we propose BrokerChain, which is a cross-shard blockchain protocol devised for the account/balance-based state sharding. Essentially, BrokerChain exploits fine-grained state partition and account segmentation. We also elaborate on how BrokerChain handles the cross-shard TXs through broker accounts. The security issues and other properties of BrokerChain are analyzed substantially. Finally, we conduct comprehensive evaluations using both a real cloud-based prototype and a transaction-driven simulator. The evaluation results show that BrokerChain outperforms the state-of-the-art solutions in terms of system throughput, transaction confirmation latency, the queue size of the transaction pool, and workload balance performance.
In a large-scale sharded blockchain, transactions are processed by a number of parallel committees collaboratively. Thus, the blockchain throughput can be strongly boosted. A problem is that some groups of blockchain nodes consume large latency to form committees at the beginning of each epoch. Furthermore, the heterogeneous processing capabilities of different committees also result in unbalanced consensus latency. Such unbalanced two-phase latency brings a large cumulative age to the transactions waited in the final committee. Consequently, the blockchain throughput can be significantly degraded because of the large transaction's cumulative age. We believe that a good committee-scheduling strategy can reduce the cumulative age, and thus benefit the blockchain throughput. However, we have not yet found a committee-scheduling scheme that works for accelerating block formation in the context of blockchain sharding. To this end, this paper studies a fine-balanced tradeoff between the transaction's throughput and their cumulative age in a large-scale sharded blockchain. We formulate this tradeoff as a utility-maximization problem, which is proved NP-hard. To solve this problem, we propose an online distributed Stochastic-Exploration (SE) algorithm, which guarantees a near-optimal system utility. The theoretical convergence time of the proposed algorithm as well as the performance perturbation brought by the committee's failure are also analyzed rigorously. We then evaluate the proposed algorithm using the dataset of blockchain-sharding transactions. The simulation results demonstrate that the proposed SE algorithm shows an overwhelming better performance comparing with other baselines in terms of both system utility and the contributing degree while processing shard transactions.
Blockchain like Bitcoin and Ethereum suffer from scalability issues. Sharding is one of the most promising and leading solutions to scale blockchain. The basic idea behind sharding is to divide the blockchain network into multiple committees, where each processing a separate set of transactions, rather than the entire network processes all transactions. In this paper, we propose a probabilistic approach to analyze the security of sharding-based blockchain protocols. Based on this approach, we investigate the threat of Sybil attacks in these protocols. The key contribution of our paper is a tractable probabilistic approach to accurately compute the failure probability that at least one committee fails and ultimately compute the probability of a successful attack. To show the effectiveness of our approach, we conduct a numerical and comparative analysis of the proposed approach with existing approaches.
NSF (Awards CNS-1413920 and CNS-1414119)
