X-Stream is a system designed for analyzing large-scale graphs on a single machine.
Core Philosophy
The system is based on the philosophy that sequential access to data works best for all storage mediums: main memory, SSD, and magnetic disk.
Storage-Centric Design
The core approach centers on rearranging graph algorithms to stream data directly from storage rather than loading everything into memory first. This storage-centric design allows efficient processing across different storage types.
My Involvement
While I was involved as a developer on the project, I was not among the original paper authors. X-Stream laid the groundwork for our subsequent work on Chaos, which extended these ideas to distributed clusters.
We encourage developers to fork the project and participate in its development!
For detailed technical information, see the research paper.