Michael E. Houle

ICLR Conference 2024 Conference Paper

LDReg: Local Dimensionality Regularized Self-Supervised Learning

• Hanxun Huang
• Ricardo J. G. B. Campello
• Sarah Monazam Erfani
• Xingjun Ma
• Michael E. Houle
• James Bailey 0001

Representations learned via self-supervised learning (SSL) can be susceptible to dimensional collapse, where the learned representation subspace is of extremely low dimensionality and thus fails to represent the full data distribution and modalities. Dimensional collapse ––– also known as the "underfilling" phenomenon ––– is one of the major causes of degraded performance on downstream tasks. Previous work has investigated the dimensional collapse problem of SSL at a global level. In this paper, we demonstrate that representations can span over high dimensional space globally, but collapse locally. To address this, we propose a method called *local dimensionality regularization (LDReg)*. Our formulation is based on the derivation of the Fisher-Rao metric to compare and optimize local distance distributions at an asymptotically small radius for each data point. By increasing the local intrinsic dimensionality, we demonstrate through a range of experiments that LDReg improves the representation quality of SSL. The results also show that LDReg can regularize dimensionality at both local and global levels.

ICLR Conference 2018 Conference Paper

Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality

• Xingjun Ma
• Bo Li 0026
• Yisen Wang 0001
• Sarah Monazam Erfani
• Sudanthi N. R. Wijewickrema
• Grant Schoenebeck
• Dawn Song
• Michael E. Houle

Deep Neural Networks (DNNs) have recently been shown to be vulnerable against adversarial examples, which are carefully crafted instances that can mislead DNNs to make errors during prediction. To better understand such attacks, a characterization is needed of the properties of regions (the so-called `adversarial subspaces') in which adversarial examples lie. We tackle this challenge by characterizing the dimensional properties of adversarial regions, via the use of Local Intrinsic Dimensionality (LID). LID assesses the space-filling capability of the region surrounding a reference example, based on the distance distribution of the example to its neighbors. We first provide explanations about how adversarial perturbation can affect the LID characteristic of adversarial regions, and then show empirically that LID characteristics can facilitate the distinction of adversarial examples generated using state-of-the-art attacks. As a proof-of-concept, we show that a potential application of LID is to distinguish adversarial examples, and the preliminary results show that it can outperform several state-of-the-art detection measures by large margins for five attack strategies considered in this paper across three benchmark datasets. Our analysis of the LID characteristic for adversarial regions not only motivates new directions of effective adversarial defense, but also opens up more challenges for developing new attacks to better understand the vulnerabilities of DNNs.

ICML Conference 2018 Conference Paper

Dimensionality-Driven Learning with Noisy Labels

• Xingjun Ma
• Yisen Wang 0001
• Michael E. Houle
• Shuo Zhou 0001
• Sarah Monazam Erfani
• Shu-Tao Xia
• Sudanthi N. R. Wijewickrema
• James Bailey 0001

Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace of training samples. We show that from a dimensionality perspective, DNNs exhibit quite distinctive learning styles when trained with clean labels versus when trained with a proportion of noisy labels. Based on this finding, we develop a new dimensionality-driven learning strategy, which monitors the dimensionality of subspaces during training and adapts the loss function accordingly. We empirically demonstrate that our approach is highly tolerant to significant proportions of noisy labels, and can effectively learn low-dimensional local subspaces that capture the data distribution.

ICML Conference 2007 Conference Paper

Best of both: a hybridized centroid-medoid clustering heuristic

• Nizar Grira
• Michael E. Houle

TCS Journal 2002 Journal Article

On local transformation of polygons with visibility properties

• Carmen Hernando
• Michael E. Houle
• Ferran Hurtado

One strategy for the enumeration of a class of objects is local transformation, in which new objects of the class are produced by means of a small modification of a previously-visited object in the same class. When local transformation is possible, the operation can be used to generate objects of the class via random walks, and as the basis for such optimization heuristics as simulated annealing. For general simple polygons on fixed point sets, it is still not known whether the class of polygons on the set is connected via a constant-size local transformation. In this paper, we exhibit a simple local transformation for which the following polygon classes are connected: monotone, x-monotone, star-shaped, (weakly) edge-visible and (weakly) externally visible. The latter class is particularly interesting as it is the most general polygon class known to be connected under local transformation. For each of the polygon classes, we also provide asymptotically-tight worst-case upper bounds on the minimum number of operations required to transform one member of the class to any other.