Chuong B. Do

ICML Conference 2009 Conference Paper

A majorization-minimization algorithm for (multiple) hyperparameter learning

• Chuan-Sheng Foo
• Chuong B. Do
• Andrew Y. Ng

We present a general Bayesian framework for hyperparameter tuning in L 2 -regularized supervised learning models. Paradoxically, our algorithm works by first analytically integrating out the hyperparameters from the model. We find a local optimum of the resulting non-convex optimization problem efficiently using a majorization-minimization (MM) algorithm, in which the non-convex problem is reduced to a series of convex L 2 -regularized parameter estimation tasks. The principal appeal of our method is its simplicity: the updates for choosing the L 2 -regularized subproblems in each step are trivial to implement (or even perform by hand), and each subproblem can be efficiently solved by adapting existing solvers. Empirical results on a variety of supervised learning models show that our algorithm is competitive with both grid-search and gradient-based algorithms, but is more efficient and far easier to implement.

ICML Conference 2009 Conference Paper

Proximal regularization for online and batch learning

• Chuong B. Do
• Quoc V. Le
• Chuan-Sheng Foo

AAAI Conference 2004 Conference Paper

PROBCONS: Probabilistic Consistency-Based Multiple Alignment of Amino Acid Sequences

• Chuong B. Do

Obtaining an accurate multiple alignment of protein sequences is a difficult computational problem for which many heuristic techniques sacrifice optimality to achieve reasonable running times. The most commonly used heuristic is progressive alignment, which merges sequences into a multiple alignment by pairwise comparisons along the nodes of a guide tree. To improve accuracy, consistency-based methods take advantage of conservation across many sequences to provide a stronger signal for pairwise comparisons. In this paper, we introduce the concept of probabilistic consistency for multiple sequence alignments. We also present PROBCONS, an HMM-based protein multiple sequence aligner, based on an approximation of the probabilistic consistency objective function. On the BAliBASE benchmark alignment database, PROBCONS demonstrates a statistically significant improvement in accuracy compared to several leading alignment programs while maintaining practical running times. Source code and program updates are freely available under the GNU Public License at http: //probcons. stanford. edu/.