Structured BFGS Method for Optimal Doubly Stochastic Matrix Approximation

Doubly stochastic matrix plays an essential role in several areas such as statistics and machine learning. In this paper we consider the optimal approximation of a square matrix in the set of doubly stochastic matrices. A structured BFGS method is proposed to solve the dual of the primal problem. The resulting algorithm builds curvature information into the diagonal components of the true Hessian, so that it takes only additional linear cost to obtain the descent direction based on the gradient information without having to explicitly store the inverse Hessian approximation. The cost is substantially fewer than quadratic complexity of the classical BFGS algorithm. Meanwhile, a Newton-based line search method is presented for finding a suitable step size, which in practice uses the existing knowledge and takes only one iteration. The global convergence of our algorithm is established. We verify the advantages of our approach on both synthetic data and real data sets. The experimental results demonstrate that our algorithm outperforms the state-of-the-art solvers and enjoys outstanding scalability.

Authors

• Dejun Chu Hefei University of Technology
• Changshui Zhang Tsinghua University
• Shiliang Sun East China Normal University
• Qing Tao Army Academy of Artillery and Air Defense

Keywords

• CSO: Constraint Optimization
• CSO: Constraint Programming
• ML: Optimization