A sparse Johnson: Lindenstrauss transform

Dimension reduction is a key algorithmic tool with many applications including nearest-neighbor search, compressed sensing and linear algebra in the streaming model. In this work we obtain a sparse version of the fundamental tool in dimension reduction -- the Johnson-Lindenstrauss transform. Using hashing and local densification, we construct a sparse projection matrix with just ~O(1/ε) non-zero entries per column. We also show a matching lower bound on the sparsity for a large class of projection matrices. Our bounds are somewhat surprising, given the known lower bounds of Ω(1/ε 2 ) both on the number of rows of any projection matrix and on the sparsity of projection matrices generated by natural constructions. Using this, we achieve an ~O(1/ε) update time per non-zero element for a (1 ε)-approximate projection, thereby substantially outperforming the ~O(1/ε 2 ) update time required by prior approaches. A variant of our method offers the same guarantees for sparse vectors, yet its ~O(d) worst case running time matches the best approach of Ailon and Liberty.

Authors

• Anirban Dasgupta 0001
• Ravi Kumar 0001
• Tamás Sarlós

Keywords

• Johnson-Lindenstrauss transform
• dimensionality reduction