The Bit Complexity of Efficient Continuous Optimization

We analyze the bit complexity of efficient algorithms for fundamental optimization problems, such as linear regression, p-norm regression, and linear programming (LP). State-of-the-art algorithms are iterative, and in terms of the number of arithmetic operations, they match the current time complexity of multiplying two n-by-n matrices (up to polylogarithmic factors). However, previous work has typically assumed infinite precision arithmetic, and due to complicated inverse maintenance techniques, the actual running times of these algorithms are unknown. To settle the running time and bit complexity of these algorithms, we demonstrate that a core common subroutine, known as inverse maintenance, is backward-stable. Additionally, we show that iterative approaches for solving constrained weighted regression problems can be accomplished with bounded-error preconditioners. Specifically, we prove that linear programs can be solved approximately in matrix multiplication time multiplied by polylog factors that depend on the condition number $\kappa$ of the matrix and the inner and outer radius of the LP problem. p-norm regression can be solved approximately in matrix multiplication time multiplied by polylog factors in $\kappa$. Lastly, linear regression can be solved approximately in input-sparsity time multiplied by polylog factors in $\kappa$. Furthermore, we present results for achieving lower than matrix multiplication time for p-norm regression by utilizing faster solvers for sparse linear systems.

Authors

• Mehrdad Ghadiri
• Richard Peng
• Santosh S. Vempala

Keywords

• Linear systems
• Linear regression
• Maintenance engineering
• Linear programming
• Iterative algorithms
• Complexity theory
• Sparse matrices
• Optimization Problem
• Running Time
• Linear System
• Time Complexity
• Matrix Multiplication
• Linear Problem
• Regression Problem
• Arithmetic Operations
• Number Of Matrices
• Arbitrary Precision
• High Probability
• Data Structure
• Complex Problems
• Complexity Analysis
• Invertible
• Iterative Algorithm
• Worst Case
• Singular Value
• Sparse Matrix
• Interior Point Method
• Linear Regression Problem
• Tensor Decomposition
• Submatrix
• Random Matrix
• Linear Programming Algorithm
• Norm Minimization
• Iterative Refinement
• Interior Point
• component
• formatting
• style
• styling
• insert