Author name cluster

Thomas Runkler

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

1 author row

EAAI Journal 2021 Journal Article

Overcoming model bias for robust offline deep reinforcement learning

Phillip Swazinna
Steffen Udluft
Thomas Runkler

State-of-the-art reinforcement learning algorithms mostly rely on being allowed to directly interact with their environment to collect millions of observations. This makes it hard to transfer their success to industrial control problems, where simulations are often very costly or do not exist, and exploring in the real environment can potentially lead to catastrophic events. Recently developed, model-free, offline RL algorithms, can learn from a single dataset (containing limited exploration) by mitigating extrapolation error in value functions. However, the robustness of the training process is still comparatively low, a problem known from methods using value functions. To improve robustness and stability of the learning process, we use dynamics models to assess policy performance instead of value functions, resulting in MOOSE (MOdel-based Offline policy Search with Ensembles), an algorithm which ensures low model bias by keeping the policy within the support of the data. We compare MOOSE with state-of-the-art model-free, offline RL algorithms BRAC, BEAR and BCQ on the Industrial Benchmark and MuJoCo continuous control tasks in terms of robust performance, and find that MOOSE outperforms its model-free counterparts in almost all considered cases, often even by far.

Details DOI

AAAI Conference 2019 Conference Paper

Neural Relation Extraction within and across Sentence Boundaries

Pankaj Gupta
Subburam Rajaram
Hinrich Schütze
Thomas Runkler

Past work in relation extraction mostly focuses on binary relation between entity pairs within single sentence. Recently, the NLP community has gained interest in relation extraction in entity pairs spanning multiple sentences. In this paper, we propose a novel architecture for this task: inter-sentential dependency-based neural networks (iDepNN). iDepNN models the shortest and augmented dependency paths via recurrent and recursive neural networks to extract relationships within (intra-) and across (inter-) sentence boundaries. Compared to SVM and neural network baselines, iDepNN is more robust to false positives in relationships spanning sentences. We evaluate our models on four datasets from newswire (MUC6) and medical (BioNLP shared task) domains that achieve state-of-the-art performance and show a better balance in precision and recall for inter-sentential relationships. We perform better than 11 teams participating in the BioNLP shared task 2016 and achieve a gain of 5. 2% (0. 587 vs 0. 558) in F1 over the winning team. We also release the crosssentence annotations for MUC6.

PDF Details

NeurIPS Conference 2018 Conference Paper

Bayesian Alignments of Warped Multi-Output Gaussian Processes

Markus Kaiser
Clemens Otte
Thomas Runkler
Carl Henrik Ek

We propose a novel Bayesian approach to modelling nonlinear alignments of time series based on latent shared information. We apply the method to the real-world problem of finding common structure in the sensor data of wind turbines introduced by the underlying latent and turbulent wind field. The proposed model allows for both arbitrary alignments of the inputs and non-parametric output warpings to transform the observations. This gives rise to multiple deep Gaussian process models connected via latent generating processes. We present an efficient variational approximation based on nested variational compression and show how the model can be used to extract shared information between dependent time series, recovering an interpretable functional decomposition of the learning problem. We show results for an artificial data set and real-world data of two wind turbines.

PDF Details

EAAI Journal 2017 Journal Article

Particle swarm optimization for generating interpretable fuzzy reinforcement learning policies

Daniel Hein
Alexander Hentschel
Thomas Runkler
Steffen Udluft

Fuzzy controllers are efficient and interpretable system controllers for continuous state and action spaces. To date, such controllers have been constructed manually or trained automatically either using expert-generated problem-specific cost functions or incorporating detailed knowledge about the optimal control strategy. Both requirements for automatic training processes are not found in most real-world reinforcement learning (RL) problems. In such applications, online learning is often prohibited for safety reasons because it requires exploration of the problem’s dynamics during policy training. We introduce a fuzzy particle swarm reinforcement learning (FPSRL) approach that can construct fuzzy RL policies solely by training parameters on world models that simulate real system dynamics. These world models are created by employing an autonomous machine learning technique that uses previously generated transition samples of a real system. To the best of our knowledge, this approach is the first to relate self-organizing fuzzy controllers to model-based batch RL. FPSRL is intended to solve problems in domains where online learning is prohibited, system dynamics are relatively easy to model from previously generated default policy transition samples, and it is expected that a relatively easily interpretable control policy exists. The efficiency of the proposed approach with problems from such domains is demonstrated using three standard RL benchmarks, i. e. , mountain car, cart-pole balancing, and cart-pole swing-up. Our experimental results demonstrate high-performing, interpretable fuzzy policies.

Details DOI

IJCAI Conference 2016 Conference Paper

Semantic Framework for Industrial Analytics and Diagnostics

Gulnar Mehdi
Sebastian Brandt
Mikhail Roshchin
Thomas Runkler

Massive data streams from sensors and devices are prominent form of industrial data generated during condition-monitoring and diagnosis of complex systems. Data analytics and reasoning has emerged as a vital tool to harness massive data sets, providing insights into historical and real-time system conditions; enhanced decision support, reliability and cost reduction. However, application of data analytics is mainly challenged by the complexity of data-access, integration, domain-specific query support and contextual reasoning capabilities. The current state-of-the-art only uses dedicated scenarios and sensors, but this limits reuse, scalability and are not sufficient for an integrated solution. Our thesis investigates if semantic technology can be a potential solution to interact and leverage data analytics for operational use. First, we have studied related work and utilized ontology-based data access (OBDA) techniques for semantic interpretation of diagnosis for Siemens Turbine use-case. Secondly, we have extended our solution to support any analytical workflow by means of an ontology.

PDF Details