Author name cluster

Michael Muehlebach

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

23 papers

2 author rows

ICLR Conference 2025 Conference Paper

Adversarial Training for Defense Against Label Poisoning Attacks

Melis Ilayda Bal
Volkan Cevher
Michael Muehlebach

As machine learning models grow in complexity and increasingly rely on publicly sourced data, such as the human-annotated labels used in training large language models, they become more vulnerable to label poisoning attacks. These attacks, in which adversaries subtly alter the labels within a training dataset, can severely degrade model performance, posing significant risks in critical applications. In this paper, we propose $\textbf{Floral}$, a novel adversarial training defense strategy based on support vector machines (SVMs) to counter these threats. Utilizing a bilevel optimization framework, we cast the training process as a non-zero-sum Stackelberg game between an $\textit{attacker}$, who strategically poisons critical training labels, and the $\textit{model}$, which seeks to recover from such attacks. Our approach accommodates various model architectures and employs a projected gradient descent algorithm with kernel SVMs for adversarial training. We provide a theoretical analysis of our algorithm’s convergence properties and empirically evaluate $\textbf{Floral}$'s effectiveness across diverse classification tasks. Compared to robust baselines and foundation models such as RoBERTa, $\textbf{Floral}$ consistently achieves higher robust accuracy under increasing attacker budgets. These results underscore the potential of $\textbf{Floral}$ to enhance the resilience of machine learning models against label poisoning threats, thereby ensuring robust classification in adversarial settings.

Details

ICLR Conference 2025 Conference Paper

Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering

Klaus-Rudolf Kladny
Bernhard Schölkopf
Michael Muehlebach

Generative models lack rigorous statistical guarantees with respect to their predictions. In this work, we propose Sequential Conformal Prediction for Generative Models (SCOPE-Gen), a sequential conformal prediction method producing prediction sets that satisfy a rigorous statistical guarantee called conformal admissibility control. This guarantee means that the prediction sets contain at least one admissible (or valid) example, with high probability. To this end, our method first samples an initial set of i.i.d. examples from a black box generative model. Then, this set is iteratively pruned via so-called greedy filters. As a consequence of the iterative generation procedure, admissibility of the final prediction set factorizes as a Markov chain, where each factor can be controlled separately, using conformal prediction. In comparison to prior work, our method demonstrates a large reduction in the number of admissibility evaluations during calibration. This is crucial e.g. in safety-critical applications, where these evaluations must be conducted manually by domain experts and are therefore costly and time consuming. We highlight the advantages of our method in terms of admissibility evaluations and cardinality of the prediction set through experiments in natural language generation and molecular graph extension tasks.

Details

EWRL Workshop 2025 Workshop Paper

Constraint-Aware Diffusion Guidance for Imitation Learning

Hao Ma
Sabrina Bodmer
Andrea Carron
Melanie Zeilinger
Michael Muehlebach

We propose Constraint-Aware Diffusion Guidance (CoDiG), a constraint-aware imitation learning framework based on conditional diffusion models. Unlike conventional imitation learning methods, which often fail to generalize to unseen or constrained environments, CoDiG enforces safety and physical feasibility during inference via barrier function guidance. Our method learns from a limited number of expert demonstrations without reward supervision or environment interaction, and is capable of generating safe and feasible trajectories in real time. A warm-start strategy further accelerates sampling by reusing previous outputs. We evaluate CoDiG on a miniature autonomous racing platform in a challenging obstacle avoidance task, demonstrating robust generalization, near time-optimal performance, and 100% success rate in dynamic scenarios. Our results highlight the potential of constraint-aware diffusion models as a data-efficient and deployable solution for safe imitation learning in robotics.

PDF

ICML Conference 2025 Conference Paper

Distributed Event-Based Learning via ADMM

Güner Dilsad Er
Sebastian Trimpe
Michael Muehlebach

We consider a distributed learning problem, where agents minimize a global objective function by exchanging information over a network. Our approach has two distinct features: (i) It substantially reduces communication by triggering communication only when necessary, and (ii) it is agnostic to the data-distribution among the different agents. We can therefore guarantee convergence even if the local data-distributions of the agents are arbitrarily distinct. We analyze the convergence rate of the algorithm both in convex and nonconvex settings and derive accelerated convergence rates in a convex setting. We also characterize the effect of communication failures and demonstrate that our algorithm is robust to communication failures. The article concludes by presenting numerical results from distributed learning tasks on the MNIST and CIFAR-10 datasets. The experiments underline communication savings of 35% or more due to the event-based communication strategy, show resilience towards heterogeneous data-distributions, and highlight that our approach outperforms common baselines such as FedAvg, FedProx, SCAFFOLD and FedADMM.

Details

NeurIPS Conference 2025 Conference Paper

Fast Non-Log-Concave Sampling under Nonconvex Equality and Inequality Constraints with Landing

Kijung Jeon
Michael Muehlebach
Molei Tao

Sampling from constrained statistical distributions is a fundamental task in various fields including Bayesian statistics, computational chemistry, and statistical physics. This article considers the cases where the constrained distribution is described by an unconstrained density, as well as additional equality and/or inequality constraints, which often make the constraint set nonconvex. Existing methods for nonconvex constraint set $\Sigma \subset \mathbb{R}^d$ defined by equality or inequality constraints commonly rely on costly projection steps. Moreover, they cannot handle equality and inequality constraints simultaneously as each method only specialized in one case. In addition, rigorous and quantitative convergence guarantee is often lacking. In this paper, we introduce Overdamped Langevin with LAnding (OLLA), a new framework that can design overdamped Langevin dynamics accommodating both equality and inequality constraints. The proposed dynamics also deterministically corrects trajectories along the normal direction of the constraint surface, thus obviating the need for explicit projections. We show that, under suitable regularity conditions on the target density and $\Sigma$, OLLA converges exponentially fast in $W_2$ distance to the constrained target density $\rho_\Sigma(x) \propto \exp(-f(x))d\sigma_\Sigma$. Lastly, through experiments, we demonstrate the efficiency of OLLA compared to projection-based constrained Langevin algorithms and their slack variable variants, highlighting its favorable computational cost and reasonable empirical mixing.

PDF Details

EWRL Workshop 2025 Workshop Paper

Online Optimization of Closed-Loop Control Systems

Hao Ma
Melanie Zeilinger
Michael Muehlebach

We propose a novel gradient-based online optimization framework for solving stochastic programming problems that frequently arise in the context of cyber-physical and robotic systems. We establish the connection between our algorithms and the cyber-physical systems through the classic two-degree-of-freedom control loop. We also incorporate an approximate model of the dynamics as prior knowledge into the learning process, and characterize the impact of modeling errors in the system dynamics on the convergence rate of the algorithms. We show that even rough estimates of the dynamics can significantly improve the convergence of our algorithms. Finally, we evaluate our algorithms in simulations of a flexible beam and a four-legged walking robot.

PDF

ICML Conference 2025 Conference Paper

Partially Observable Reinforcement Learning with Memory Traces

Onno Eberhard
Michael Muehlebach
Claire Vernade

Partially observable environments present a considerable computational challenge in reinforcement learning due to the need to consider long histories. Learning with a finite window of observations quickly becomes intractable as the window length grows. In this work, we introduce memory traces. Inspired by eligibility traces, these are compact representations of the history of observations in the form of exponential moving averages. We prove sample complexity bounds for the problem of offline on-policy evaluation that quantify the return errors achieved with memory traces for the class of Lipschitz continuous value estimates. We establish a close connection to the window approach, and demonstrate that, in certain environments, learning with memory traces is significantly more sample efficient. Finally, we underline the effectiveness of memory traces empirically in online reinforcement learning experiments for both value prediction and control.

Details

EWRL Workshop 2025 Workshop Paper

Partially Observable Reinforcement Learning with Memory Traces

Onno Eberhard
Michael Muehlebach
Claire Vernade

Partially observable environments present a considerable computational challenge in reinforcement learning due to the need to consider long histories. Learning with a finite window of observations quickly becomes intractable as the window length grows. In this work, we introduce *memory traces*. Inspired by eligibility traces, these are compact representations of the history of observations in the form of exponential moving averages. We prove sample complexity bounds for the problem of offline on-policy evaluation that quantify the return errors achieved with memory traces for the class of Lipschitz continuous value estimates. We establish a close connection to the window approach, and demonstrate that, in certain environments, learning with memory traces is significantly more sample efficient. Finally, we underline the effectiveness of memory traces empirically in online reinforcement learning experiments for both value prediction and control.

PDF

NeurIPS Conference 2025 Conference Paper

Quantization-Free Autoregressive Action Transformer

Ziyad Sheebaelhamd
Michael Tschannen
Michael Muehlebach
Claire Vernade

Current transformer-based imitation learning approaches introduce discrete action representations and train an autoregressive transformer decoder on the resulting latent code. However, the initial quantization breaks the continuous structure of the action space thereby limiting the capabilities of the generative model. We propose a quantization-free method instead that leverages Generative Infinite-Vocabulary Transformers (GIVT) as a direct, continuous policy parametrization for autoregressive transformers. This simplifies the imitation learning pipeline while achieving state-of-the-art performance on a variety of popular simulated robotics tasks. We enhance our policy roll-outs by carefully studying sampling algorithms, further improving the results.

PDF Details

EWRL Workshop 2025 Workshop Paper

The Sample Complexity of Online Reinforcement Learning: A Multi-model Perspective

Michael Muehlebach
Zhiyu He
Michael I. Jordan

We study the sample complexity of online reinforcement learning in the general setting of nonlinear dynamical systems with continuous state and action spaces. Our analysis accommodates a large class of dynamical systems ranging from a finite set of nonlinear candidate models to models with bounded and Lipschitz continuous dynamics, to systems that are parametrized by a compact and real-valued set of parameters. In the most general setting, our algorithm achieves a policy regret of $\mathcal{O}(N \epsilon^2 + \mathrm{ln}(m(\epsilon))/\epsilon^2)$, where $N$ is the time horizon, $\epsilon$ is a user-specified discretization width, and $m(\epsilon)$ measures the complexity of the function class under consideration via its packing number. In the special case where the dynamics are parametrized by a compact and real-valued set of parameters (such as neural networks, transformers, etc.), we prove a policy regret of $\mathcal{O}(\sqrt{N p})$, where $p$ denotes the number of parameters, recovering earlier sample-complexity results that were derived for \emph{linear} \emph{time-invariant} dynamical systems. While this article focuses on characterizing sample complexity, the proposed algorithms are likely to be useful in practice, due to their simplicity, their ability to incorporate prior knowledge, and their benign transient behaviors.

PDF

NeurIPS Conference 2025 Conference Paper

Zeroth-Order Optimization Finds Flat Minima

Liang Zhang
Bingcong Li
Kiran Thekumparampil
Sewoong Oh
Michael Muehlebach
Niao He

Zeroth-order methods are extensively used in machine learning applications where gradients are infeasible or expensive to compute, such as black-box attacks, reinforcement learning, and language model fine-tuning. Existing optimization theory focuses on convergence to an arbitrary stationary point, but less is known on the implicit regularization that provides a fine-grained characterization on which particular solutions are finally reached. We show that zeroth-order optimization with the standard two-point estimator favors solutions with small trace of Hessian, which is widely used in previous work to distinguish between sharp and flat minima. We further provide convergence rates of zeroth-order optimization to approximate flat minima for convex and sufficiently smooth functions, where flat minima are defined as the minimizers that achieve the smallest trace of Hessian among all optimal solutions. Experiments on binary classification tasks with convex losses and language model fine-tuning support our theoretical findings.

PDF Details

TMLR Journal 2024 Journal Article

Deep Backtracking Counterfactuals for Causally Compliant Explanations

Klaus-Rudolf Kladny
Julius von Kügelgen
Bernhard Schölkopf
Michael Muehlebach

Counterfactuals answer questions of what would have been observed under altered circumstances and can therefore offer valuable insights. Whereas the classical interventional interpretation of counterfactuals has been studied extensively, backtracking constitutes a less studied alternative where all causal laws are kept intact. In the present work, we introduce a practical method called deep backtracking counterfactuals (DeepBC) for computing backtracking counterfactuals in structural causal models that consist of deep generative components. We propose two distinct versions of our method—one utilizing Langevin Monte Carlo sampling and the other employing constrained optimization—to generate counterfactuals for high-dimensional data. As a special case, our formulation reduces to methods in the field of counterfactual explanations. Compared to these, our approach represents a causally compliant, versatile and modular alternative. We demonstrate these properties experimentally on a modified version of MNIST and CelebA.

PDF Details

UAI Conference 2023 Conference Paper

Causal effect estimation from observational and interventional data through matrix weighted linear estimators

Klaus-Rudolf Kladny
Julius von Kügelgen
Bernhard Schölkopf
Michael Muehlebach

We study causal effect estimation from a mixture of observational and interventional data in a confounded linear regression model with multivariate treatments. We show that the statistical efficiency in terms of expected squared error can be improved by combining estimators arising from both the observational and interventional setting. To this end, we derive methods based on matrix weighted linear estimators and prove that our methods are asymptotically unbiased in the infinite sample limit. This is an important improvement compared to the pooled estimator using the union of interventional and observational data, for which the bias only vanishes if the ratio of observational to interventional data tends to zero. Studies on synthetic data confirm our theoretical findings. In settings where confounding is substantial and the ratio of observational to interventional data is large, our estimators outperform a Stein-type estimator and various other baselines.

Details

IROS Conference 2023 Conference Paper

Data-Efficient Online Learning of Ball Placement in Robot Table Tennis

Philip Tobuschat
Hao Ma
Dieter Büchler
Bernhard Schölkopf
Michael Muehlebach

We present an implementation of an online op-timization algorithm for hitting a predefined target when returning ping-pong balls with a table tennis robot. The online algorithm optimizes over so-called interception policies, which define the manner in which the robot arm intercepts the ball. In our case, these are composed of the state of the robot arm (position and velocity) at interception time. Gradient information is provided to the optimization algorithm via the mapping from the interception policy to the landing point of the ball on the table, which is approximated with a black-box and a grey-box approach. Our algorithm is applied to a robotic arm with four degrees of freedom that is driven by pneumatic artificial muscles. As a result, the robot arm is able to return the ball onto any predefined target on the table after about 2–5 iterations. We highlight the robustness of our approach by showing rapid convergence with both the black-box and the grey-box gradients. In addition, the small number of iterations required to reach close proximity to the target also underlines the sample efficiency. A demonstration video can be found here: https://youtu.be/VC3KJoCss0k.

Details

NeurIPS Conference 2023 Conference Paper

Online Learning under Adversarial Nonlinear Constraints

Pavel Kolev
Georg Martius
Michael Muehlebach

In many applications, learning systems are required to process continuous non-stationary data streams. We study this problem in an online learning framework and propose an algorithm that can deal with adversarial time-varying and nonlinear constraints. As we show in our work, the algorithm called Constraint Violation Velocity Projection (CVV-Pro) achieves $\sqrt{T}$ regret and converges to the feasible set at a rate of $1/\sqrt{T}$, despite the fact that the feasible set is slowly time-varying and a priori unknown to the learner. CVV-Pro only relies on local sparse linear approximations of the feasible set and therefore avoids optimizing over the entire set at each iteration, which is in sharp contrast to projected gradients or Frank-Wolfe methods. We also empirically evaluate our algorithm on two-player games, where the players are subjected to a shared constraint.

PDF Details

EWRL Workshop 2023 Workshop Paper

Online Learning under Adversarial Nonlinear Constraints

Pavel Kolev
Georg Martius
Michael Muehlebach

PDF

JMLR Journal 2022 Journal Article

On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systems

Michael Muehlebach
Michael I. Jordan

We introduce a class of first-order methods for smooth constrained optimization that are based on an analogy to non-smooth dynamical systems. Two distinctive features of our approach are that (i) projections or optimizations over the entire feasible set are avoided, in stark contrast to projected gradient methods or the Frank-Wolfe method, and (ii) iterates are allowed to become infeasible, which differs from active set or feasible direction methods, where the descent motion stops as soon as a new constraint is encountered. The resulting algorithmic procedure is simple to implement even when constraints are nonlinear, and is suitable for large-scale constrained optimization problems in which the feasible set fails to have a simple structure. The key underlying idea is that constraints are expressed in terms of velocities instead of positions, which has the algorithmic consequence that optimizations over feasible sets at each iteration are replaced with optimizations over local, sparse convex approximations. In particular, this means that at each iteration only constraints that are violated are taken into account. The result is a simplified suite of algorithms and an expanded range of possible applications in machine learning. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2022. ( edit, beta )

PDF Details

NeurIPS Conference 2022 Conference Paper

Sampling without Replacement Leads to Faster Rates in Finite-Sum Minimax Optimization

Aniket Das
Bernhard Schölkopf
Michael Muehlebach

We analyze the convergence rates of stochastic gradient algorithms for smooth finite-sum minimax optimization and show that, for many such algorithms, sampling the data points \emph{without replacement} leads to faster convergence compared to sampling with replacement. For the smooth and strongly convex-strongly concave setting, we consider gradient descent ascent and the proximal point method, and present a unified analysis of two popular without-replacement sampling strategies, namely \emph{Random Reshuffling} (RR), which shuffles the data every epoch, and \emph{Single Shuffling} or \emph{Shuffle Once} (SO), which shuffles only at the beginning. We obtain tight convergence rates for RR and SO and demonstrate that these strategies lead to faster convergence than uniform sampling. Moving beyond convexity, we obtain similar results for smooth nonconvex-nonconcave objectives satisfying a two-sided Polyak-\L{}ojasiewicz inequality. Finally, we demonstrate that our techniques are general enough to analyze the effect of \emph{data-ordering attacks}, where an adversary manipulates the order in which data points are supplied to the optimizer. Our analysis also recovers tight rates for the \emph{incremental gradient} method, where the data points are not shuffled at all.

PDF Details

JMLR Journal 2021 Journal Article

Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives

Michael Muehlebach
Michael I. Jordan

We analyze the convergence rate of various momentum-based optimization algorithms from a dynamical systems point of view. Our analysis exploits fundamental topological properties, such as the continuous dependence of iterates on their initial conditions, to provide a simple characterization of convergence rates. In many cases, closed-form expressions are obtained that relate algorithm parameters to the convergence rate. The analysis encompasses discrete time and continuous time, as well as time-invariant and time-variant formulations, and is not limited to a convex or Euclidean setting. In addition, the article rigorously establishes why symplectic discretization schemes are important for momentum-based optimization algorithms, and provides a characterization of algorithms that exhibit accelerated convergence. [abs] [ pdf ][ bib ] &copy JMLR 2021. ( edit, beta )

PDF Details

ICML Conference 2020 Conference Paper

Continuous-time Lower Bounds for Gradient-based Algorithms

Michael Muehlebach
Michael I. Jordan

This article derives lower bounds on the convergence rate of continuous-time gradient-based optimization algorithms. The algorithms are subjected to a time-normalization constraint that avoids a reparametrization of time in order to make the discussion of continuous-time convergence rates meaningful. We reduce the multi-dimensional problem to a single dimension, recover well-known lower bounds from the discrete-time setting, and provide insight into why these lower bounds occur. We present algorithms that achieve the proposed lower bounds, even when the function class under consideration includes certain nonconvex functions.

Details

ICML Conference 2019 Conference Paper

A Dynamical Systems Perspective on Nesterov Acceleration

Michael Muehlebach
Michael I. Jordan

We present a dynamical system framework for understanding Nesterov’s accelerated gradient method. In contrast to earlier work, our derivation does not rely on a vanishing step size argument. We show that Nesterov acceleration arises from discretizing an ordinary differential equation with a semi-implicit Euler integration scheme. We analyze both the underlying differential equation as well as the discretization to obtain insights into the phenomenon of acceleration. The analysis suggests that a curvature-dependent damping term lies at the heart of the phenomenon. We further establish connections between the discretized and the continuous-time dynamics.

Details

ICRA Conference 2017 Conference Paper

Implementation of a parametrized infinite-horizon model predictive control scheme with stability guarantees

Michael Muehlebach
Carmelo Sferrazza
Raffaello D'Andrea

This article discusses the implementation of an infinite-horizon model predictive control approach that is based on representing input and state trajectories by a linear combination of basis functions. An iterative constraint sampling strategy is presented for guaranteeing constraint satisfaction over all times. It will be shown that the proposed method converges. In addition, we will discuss the implementation of the resulting (online) model predictive control algorithm on an unmanned aerial vehicle and provide experimental results. The computational efficiency of the algorithm is highlighted by the fact that a sampling rate of 100 Hz was achieved on an embedded platform.

Details

ICRA Conference 2016 Conference Paper

Application of an approximate model predictive control scheme on an unmanned aerial vehicle

Matthias Hofer 0003
Michael Muehlebach
Raffaello D'Andrea

An approximate model predictive control approach is applied on an unmanned aerial vehicle with limited computational resources. A novel method using a continuous time parametrization of the state and input trajectory is used to derive a compact description of the optimal control problem. Different first order methods for the online optimization are discussed in terms of memory requirements and execution time. The generalized fast dual gradient method is implemented on the aerial vehicle. The approximate model predictive control algorithm runs on an embedded platform with a STM32 Cortex M4 processor. Simulation studies show that the model predictive controller outperforms a linear quadratic regulator in aggressive maneuvers. The model predictive control approach is evaluated in practice and shown to yield satisfactory flight behavior.

Details