Author name cluster

Maciej Wołczyk

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

1 author row

AAAI Conference 2025 Short Paper

On-Policy Algorithms for Continual Reinforcement Learning (Student Abstract)

Tadeusz Dziarmaga
Tomasz Arczewski
Marcin Mazur
Maciej Wołczyk

Continual reinforcement learning (CRL) is the study of optimal strategies for maximizing rewards in sequential environments that change over time. This is particularly crucial in domains such as robotics, where the operational environment is inherently dynamic and subject to continual change. Nevertheless, research in this area has thus far concentrated on off-policy algorithms with replay buffers that are capable of amortizing the impact of distribution shifts. Such an approach is not feasible with on-policy reinforcement learning algorithms that learn solely from the data obtained from the current policy. In this paper, we examine the performance of proximal policy optimization (PPO), a prevalent on-policy reinforcement learning (RL) algorithm, in a classical CRL benchmark. Our findings suggest that the current methods are suboptimal in terms of average performance. Nevertheless, they demonstrate encouraging competitive outcomes with respect to forward transfer and forgetting metrics. This highlights the need for further research into continual on-policy reinforcement learning. The source code is available at https://github.com/Teddy298/continualworld-ppo.

PDF Details DOI

AAAI Conference 2022 Conference Paper

PluGeN: Multi-Label Conditional Generation from Pre-trained Models

Maciej Wołczyk
Magdalena Proszewska
Łukasz Maziarka
Maciej Zieba
Patryk Wielopolski
Rafał Kurczab
Marek Smieja

Modern generative models achieve excellent quality in a variety of tasks including image or text generation and chemical molecule modeling. However, existing methods often lack the essential ability to generate examples with requested properties, such as the age of the person in the photo or the weight of the generated molecule. Incorporating such additional conditioning factors would require rebuilding the entire architecture and optimizing the parameters from scratch. Moreover, it is difficult to disentangle selected attributes so that to perform edits of only one attribute while leaving the others unchanged. To overcome these limitations we propose PluGeN (Plugin Generative Network), a simple yet effective generative technique that can be used as a plugin to pre-trained generative models. The idea behind our approach is to transform the entangled latent representation using a flow-based module into a multi-dimensional space where the values of each attribute are modeled as an independent one-dimensional distribution. In consequence, PluGeN can generate new samples with desired attributes as well as manipulate labeled attributes of existing examples. Due to the disentangling of the latent representation, we are even able to generate samples with rare or unseen combinations of attributes in the dataset, such as a young person with gray hair, men with make-up, or women with beards. We combined PluGeN with GAN and VAE models and applied it to conditional generation and manipulation of images and chemical molecule modeling. Experiments demonstrate that PluGeN preserves the quality of backbone models while adding the ability to control the values of labeled attributes. Implementation is available at https: //github. com/gmum/plugen.

PDF Details

NeurIPS Conference 2021 Conference Paper

Continual World: A Robotic Benchmark For Continual Reinforcement Learning

Maciej Wołczyk
Michał Zając
Razvan Pascanu
Łukasz Kuciński
Piotr Miłoś

Continual learning (CL) --- the ability to continuously learn, building on previously acquired knowledge --- is a natural requirement for long-lived autonomous reinforcement learning (RL) agents. While building such agents, one needs to balance opposing desiderata, such as constraints on capacity and compute, the ability to not catastrophically forget, and to exhibit positive transfer on new tasks. Understanding the right trade-off is conceptually and computationally challenging, which we argue has led the community to overly focus on catastrophic forgetting. In response to these issues, we advocate for the need to prioritize forward transfer and propose Continual World, a benchmark consisting of realistic and meaningfully diverse robotic tasks built on top of Meta-World as a testbed. Following an in-depth empirical evaluation of existing CL methods, we pinpoint their limitations and highlight unique algorithmic challenges in the RL setting. Our benchmark aims to provide a meaningful and computationally inexpensive challenge for the community and thus help better understand the performance of existing and future solutions. Information about the benchmark, including the open-source code, is available at https: //sites. google. com/view/continualworld.

PDF Details

AAAI Conference 2021 Short Paper

Remember More by Recalling Less: Investigating the Role of Batch Size in Continual Learning with Experience Replay (Student Abstract)

Maciej Wołczyk
Andrii Krutsylo

Experience replay is a simple and well-performing strategy for continual learning problems, often used as a basis for more advanced methods. However, the dynamics of experience replay are not yet well understood. To showcase this, we focus on a single component of this problem, namely choosing the batch size of the buffer samples. We find that small batches perform much better at stopping forgetting than larger batches, contrary to the intuitive assumption that it is better to recall more samples from the past to avoid forgetting. We show that this phenomenon does not disappear under learning rate tuning and we propose possible directions for further analysis.

PDF Details

NeurIPS Conference 2021 Conference Paper

Zero Time Waste: Recycling Predictions in Early Exit Neural Networks

Maciej Wołczyk
Bartosz Wójcik
Klaudia Bałazy
Igor T Podolak
Jacek Tabor
Marek Śmieja
Tomasz Trzcinski

The problem of reducing processing time of large deep learning models is a fundamental challenge in many real-world applications. Early exit methods strive towards this goal by attaching additional Internal Classifiers (ICs) to intermediate layers of a neural network. ICs can quickly return predictions for easy examples and, as a result, reduce the average inference time of the whole model. However, if a particular IC does not decide to return an answer early, its predictions are discarded, with its computations effectively being wasted. To solve this issue, we introduce Zero Time Waste (ZTW), a novel approach in which each IC reuses predictions returned by its predecessors by (1) adding direct connections between ICs and (2) combining previous outputs in an ensemble-like manner. We conduct extensive experiments across various datasets and architectures to demonstrate that ZTW achieves a significantly better accuracy vs. inference time trade-off than other recently proposed early exit methods.

PDF Details