Cyclades: Conflict-free Asynchronous Machine Learning

We present Cyclades, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. Cyclades is asynchronous during model updates, and requires no memory locking mechanisms, similar to Hogwild! -type algorithms. Unlike Hogwild! , Cyclades introduces no conflicts during parallel execution, and offers a black-box analysis for provable speedups across a large family of algorithms. Due to its inherent cache locality and conflict-free nature, our multi-core implementation of Cyclades consistently outperforms Hogwild! -type algorithms on sufficiently sparse datasets, leading to up to 40% speedup gains compared to Hogwild! , and up to 5\times gains over asynchronous implementations of variance reduction algorithms.

Authors

• Xinghao Pan UC Berkeley
• Maximilian Lam UC Berkeley
• Stephen Tu
• Dimitris Papailiopoulos
• Ce Zhang Institute of Automation, CASIA & School of Artificial Intelligence, UCAS
• Michael Jordan
• Kannan Ramchandran
• Christopher Ré Stanford University

Keywords

No keywords are indexed for this paper.