Hailstorm improves load distribution and resource efficiency in LSM-based distributed databases.
Architecture
The system operates as a filesystem layer underneath LSM storage engines such as RocksDB, uniformly distributing data across all storage devices within a rack in fine-grained blocks. This storage architecture enables database instances handling significant loads to delegate compaction operations to less utilized machines.
Performance Results
Hailstorm delivers impressive performance improvements:
- 2X throughput improvement for write-heavy workloads in MongoDB
- 22X throughput improvement for range scans
- 50% performance improvements in TPC-C and TPC-E on TiDB
These results demonstrate the effectiveness of disaggregating compute and storage in distributed database deployments.