BlockBench
Blockbench is an open-source benchmarking framework designed to analyze and evaluate the performance of blockchain systems. Established as the first framework specifically for benchmarking private blockchain systems in 2017, it enables the comparison of different platforms and provides insights into system design trade-offs.
History
Blockbench was created as a research project under the DB System Group at the National University of Singapore and Zhejiang University in 2017. Since then, it has become a widely used framework to evaluate blockchain performance, alongside other frameworks such as Hyperledger Caliper (2018) for Hyperledger Fabric. The framework has since evolved into Blockbench-v3, expanding its scope to address the architectural complexities of modern public blockchains and the Web3 ecosystem.
Blockchain Abstraction Model
To characterize blockchain bottlenecks and to design benchmarking workloads, Blockbench identifies the blockchain abstraction model by finding common layers across blockchain implementations. The model comprises four layers:
- Consensus: Get all nodes in the system to agree on the blockchain content (appending blocks to the blockchain).
- Data Model: Structure, content, and operations related to blockchain data (e.g., account, transaction, block data)
- Execution Engine: The runtime environment for executing transactions.
- Application: Blockchain applications (smart contracts)
Workloads
The workloads are designed to test each layer in the abstraction model, identify bottlenecks, and give insights into performance behavior. It employs a hierarchical approach to benchmarking, utilizing two primary categories of workloads:
- Macro-benchmark workloads: Used for evaluating the overall system performance from the application layer. This category consists of YCSB, Smallbank, EtherId, Doubler, and WavesPresale.
- Micro-benchmark workloads: Used for assessing the lower abstraction layers. This category includes DoNothing, Analytics, IOHeavy, and CPUHeavy.
The workloads are implemented in smart contracts written in Solidity. Users can integrate custom workloads to the framework.
Metrics
Blockbench includes metrics to measure different aspects of blockchain systems: throughput, latency, scalability, fault tolerance. In its recent update, an additional security metric to measure was introduced.
Implementation
Blockbench architecture comprises the Driver, the core component, which orchestrates the evaluation process. The driver accepts a workload and a user-defined configuration (specifying parameters such as the number of operations and client threads), executes the workload on the target blockchain, and outputs runtime statistics. It connects with workload clients and blockchains through two interfaces IBlockchainConnector and IWorkloadConnector.
To integrate a specific blockchain system, developers implement the IBlockchainConnector interface, which defines standard operations for:
- Deploying applications (smart contracts).
- Invoking applications via transactions.
- Querying the blockchain's state.
Users can utilize existing workloads or implement custom ones via the IWorkloadConnector interface.
Blockbench-v3
Blockbench-v3 is the successor to the original Blockbench framework, developed to solve the challenges of evaluating modern public blockchain architectures. It establishes standardized principles for blockchain evaluation, defining unified workload specifications, metric definitions, and methodologies to ensure fair comparison across heterogeneous systems.
Blockbench-v3 provides a modular, extensible implementation consisting of three core components:
- Workload Generator: Produces realistic Web3 transaction patterns.
- Multi-layer Driver Architecture: Interfaces with diverse blockchain APIs and protocols.
- Unified Metric Collection System: Captures both traditional performance indicators and layer-specific measurements.
This design enables researchers and practitioners to conduct reproducible, standardized evaluations across diverse blockchain environments. By incorporating new drivers, workloads, and a standardized design, it can adapt to test a wide range of cutting-edge blockchain systems, including parallel blockchains, Layer 2 solutions, and cross-chain bridges.
Supported Systems
Blockbench supports a range of blockchain implementations:
- Ethereum and its variants: Geth (Ethereum), Parity, and Quorum.
- Hyperledger Fabric: Versions 0.6, 1.4, and 2.2.
- Modern architectures: Supported via the extensible Blockbench-v3 driver system, including Layer 1 (Solana, Aptos, Sui), Layer 2 (Optimism, ZK-rollups), and Cross-chain solutions.
Later versions expanded support to include Layer 1 and Layer 2
See also
- Blockchain
- Benchmark
- Smart contract
- Hyperledger