Our paper "Hailstorm: Disaggregated Compute and Storage for Distributed LSM-based Databases" has been accepted at ASPLOS 2020, taking place in Lausanne, Switzerland!
What is Hailstorm?
Hailstorm addresses performance challenges in distributed LSM-based databases through a disaggregated storage and compute architecture. The system pools storage across a rack and offloads compaction tasks to remote nodes.
Key Results
Our evaluation demonstrates significant improvements:
- 60% average throughput improvement in MongoDB deployments with skewed workloads
- 5X tail latency reduction
- 22X throughput improvements in range query workloads
- 47-56% cost savings in OLTP scenarios
Technical Approach
The research focuses on addressing load imbalance and background task interference in distributed databases by deploying storage engines over a distributed filesystem that separates storage from processing capabilities.