Author name cluster

Tobias Weber

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

ICLR Conference 2025 Conference Paper

Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries

Chris Kolb
Tobias Weber
Bernd Bischl
David Rügamer

Sparse regularization techniques are well-established in machine learning, yet their application in neural networks remains challenging due to the non-differentiability of penalties like the $L_1$ norm, which is incompatible with stochastic gradient descent. A promising alternative is shallow weight factorization, where weights are decomposed into two factors, allowing for smooth optimization of $L_1$-penalized neural networks by adding differentiable $L_2$ regularization to the factors. In this work, we introduce deep weight factorization, extending previous shallow approaches to more than two factors. We theoretically establish equivalence of our deep factorization with non-convex sparse regularization and analyze its impact on training dynamics and optimization. Due to the limitations posed by standard training practices, we propose a tailored initialization scheme and identify important learning rate requirements necessary for training factorized networks. We demonstrate the effectiveness of our deep weight factorization through experiments on various architectures and datasets, consistently outperforming its shallow counterpart and widely used pruning methods.

Details

ICML Conference 2025 Conference Paper

Linearization Turns Neural Operators into Function-Valued Gaussian Processes

Emilia Magnani
Marvin Pförtner
Tobias Weber
Philipp Hennig

Neural operators generalize neural networks to learn mappings between function spaces from data. They are commonly used to learn solution operators of parametric partial differential equations (PDEs) or propagators of time-dependent PDEs. However, to make them useful in high-stakes simulation scenarios, their inherent predictive error must be quantified reliably. We introduce LUNO, a novel framework for approximate Bayesian uncertainty quantification in trained neural operators. Our approach leverages model linearization to push (Gaussian) weight-space uncertainty forward to the neural operator’s predictions. We show that this can be interpreted as a probabilistic version of the concept of currying from functional programming, yielding a function-valued (Gaussian) random process belief. Our framework provides a practical yet theoretically sound way to apply existing Bayesian deep learning methods such as the linearized Laplace approximation to neural operators. Just as the underlying neural operator, our approach is resolution-agnostic by design. The method adds minimal prediction overhead, can be applied post-hoc without retraining the network, and scales to large models and datasets. We evaluate these aspects in a case study on Fourier neural operators.

Details

ICML Conference 2024 Conference Paper

Generalizing Orthogonalization for Models with Non-Linearities

David Rügamer
Chris Kolb
Tobias Weber
Lucas Kook
Thomas Nagler

The complexity of black-box algorithms can lead to various challenges, including the introduction of biases. These biases present immediate risks in the algorithms’ application. It was, for instance, shown that neural networks can deduce racial information solely from a patient’s X-ray scan, a task beyond the capability of medical experts. If this fact is not known to the medical expert, automatic decision-making based on this algorithm could lead to prescribing a treatment (purely) based on racial information. While current methodologies allow for the "orthogonalization" or "normalization" of neural networks with respect to such information, existing approaches are grounded in linear models. Our paper advances the discourse by introducing corrections for non-linearities such as ReLU activations. Our approach also encompasses scalar and tensor-valued predictions, facilitating its integration into neural network architectures. Through extensive experiments, we validate our method’s effectiveness in safeguarding sensitive data in generalized linear models, normalizing convolutional neural networks for metadata, and rectifying pre-existing embeddings for undesired attributes.

Details

IROS Conference 2020 Conference Paper

An External Stabilization Unit for High-Precision Applications of Robot Manipulators

Tobias F. C. Berninger
Tomas Slimak
Tobias Weber
Daniel J. Rixen

Because of their large workspace, robot manipulators have the potential to be used for high precision non-contact manufacturing processes, such as laser cutting or welding, on large complex work pieces. However, most industrial manipulators are not able to provide the necessary accuracy requirements. Mainly because of their flexible structures, they are subject to point to point positioning errors and also vibration errors on a smaller scale. The vibration issues are especially hard to deal with. Many published solutions propose to modify the robot's own control system to deal with these problems. However, most modern control techniques require high fidelity models of the underlying system dynamics, which are quite difficult to obtain for robot manipulators. In this work, we propose an external stabilization unit with an additional set of actuators/sensors to stabilize the process tool, similar to Optical Image Stabilization systems. We show that, because of collocated control, a model of the robot's own dynamic behavior is not needed to achieve high tracking accuracy. We also provide testing results of a prototype stabilizing a dummy tool in two degrees of freedom on a UR10 robot, which reduced its tracking error by two orders of magnitude below 20 micrometers.

Details