Author name cluster

Andrei Haidu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

12 papers

2 author rows

KR Conference 2023 Conference Paper

Knowledge-Driven Robot Program Synthesis from Human VR Demonstrations

Benjamin Alt
Franklin Kenghagho Kenfack
Andrei Haidu
Darko Katic
Rainer Jäkel
Michael Beetz

Aging societies, labor shortages and increasing wage costs call for assistance robots capable of autonomously performing a wide array of real-world tasks. Such open-ended robotic manipulation requires not only powerful knowledge representations and reasoning (KR&R) algorithms, but also methods for humans to instruct robots what tasks to perform and how to perform them. In this paper, we present a system for automatically generating executable robot control programs from human task demonstrations in virtual reality (VR). We leverage common-sense knowledge and game engine-based physics to semantically interpret human VR demonstrations, as well as an expressive and general task representation and automatic path planning and code generation, embedded into a state-of-the-art cognitive architecture. We demonstrate our approach in the context of force-sensitive fetch-and-place for a robotic shopping assistant. The source code is available at https: //github. com/ease-crc/vr-program-synthesis.

PDF Details DOI

ICRA Conference 2021 Conference Paper

Automated acquisition of structured, semantic models of manipulation activities from human VR demonstration

Andrei Haidu
Michael Beetz

In this paper we present a system capable of collecting and annotating, human performed, robot understandable, everyday activities from virtual environments. The human movements are mapped in the simulated world using off-the-shelf virtual reality devices with full body, and eye tracking capabilities. All the interactions in the virtual world are physically simulated, thus movements and their effects are closely relatable to the real world. During the activity execution, a subsymbolic data logger is recording the environment and the human gaze on a per-frame basis, enabling offline scene reproduction and replays. Coupled with the physics engine, online monitors (symbolic data loggers) are parsing (using various grammars) and recording events, actions, and their effects in the simulated world.

Details

IROS Conference 2020 Conference Paper

Learning Motion Parameterizations of Mobile Pick and Place Actions from Observing Humans in Virtual Environments

Gayane Kazhoyan
Alina Hawkin
Sebastian Koralewski
Andrei Haidu
Michael Beetz

In this paper, we present an approach and an implemented pipeline for transferring data acquired from observing humans in virtual environments onto robots acting in the real world, and adapting the data accordingly to achieve successful task execution. We demonstrate our pipeline by inferring seven different symbolic and subsymbolic motion parameters of mobile pick and place actions, which allows the robot to set a simple breakfast table. We propose an approach to learn general motion parameter models and discuss, which parameters can be learned at which abstraction level.

Details

ICRA Conference 2019 Conference Paper

Automated Models of Human Everyday Activity based on Game and Virtual Reality Technology

Andrei Haidu
Michael Beetz

In this paper, we will describe AMEvA (Automated Models of Everyday Activities), a special-purpose knowledge acquisition, interpretation, and processing system for human everyday manipulation activity that can automatically: (1) create and simulate virtual human living and working environments (such as kitchens and apartments) with a scope, extent, level of detail, physics, and close to photorealism that facilitates and promotes the natural and realistic execution of human everyday manipulation activities; (2) record human manipulation activities performed in the respective virtual reality environment as well as their effects on the environment and detect force-dynamic states and events; (3) decompose and segment the recorded activity data into meaningful motions and categorize the motions according to action models used in cognitive science; and (4) represent the interpreted activities symbolically in KNOWROB [1] using a first-order time interval logic representation.

Details

IROS Conference 2018 Conference Paper

Cognition-enabled Framework for Mixed Human-Robot Rescue Teams

Fereshta Yazdani
Gayane Kazhoyan
Asil Kaan Bozcuoglu
Andrei Haidu
Ferenc Balint-Benczedi
Daniel Beßler
Mihai Pomarlan
Michael Beetz

With the advancements in robotic technology and the progress in human-robot interaction research, the interest in deploying mixed human-robot teams in rescue missions is increasing. Due to their complementary capabilities in terms of locomotion, visibility and reachability of areas, human-robot teams are considerably deployed in real-world settings, albeit the robotic agents in such scenarios are normally fully teleoperated. A major barrier to successful and efficient mission execution in those teams is the lack of cognitive skills in robotic systems. In this paper, we present a cognition-enabled framework and an implemented system where robotic agents are equipped with cognitive capabilities to naturally communicate with humans and autonomously perform tasks. The framework allows for natural tasking of robots, reasoning about robot behavior, capabilities and actions, and a common belief state representation for shared mission awareness of robots and human operators.

Details

ICRA Conference 2018 Conference Paper

Know Rob 2. 0 - A 2nd Generation Knowledge Processing Framework for Cognition-Enabled Robotic Agents

Michael Beetz
Daniel Beßler
Andrei Haidu
Mihai Pomarlan
Asil Kaan Bozcuoglu
Georg Bartels

In this paper we present KnowRob2, a second generation knowledge representation and reasoning framework for robotic agents. KnowRob2 is an extension and partial redesign of KnowRob, currently one of the most advanced knowledge processing systems for robots that has enabled them to successfully perform complex manipulation tasks such as making pizza, conducting chemical experiments, and setting tables. The knowledge base appears to be a conventional first-order time interval logic knowledge base, but it exists to a large part only virtually: many logical expressions are constructed on demand from data structures of the control program, computed through robotics algorithms including ones for motion planning and solving inverse kinematics problems, and log data stored in noSQL databases. Novel features and extensions of KnowRob2 substantially increase the capabilities of robotic agents of acquiring open-ended manipulation skills and competence, reasoning about how to perform manipulation actions more realistically, and acquiring commonsense knowledge.

Details

IROS Conference 2018 Conference Paper

KnowRobSIM - Game Engine-Enabled Knowledge Processing Towards Cognition-Enabled Robot Control

Andrei Haidu
Daniel Beßler
Asil Kaan Bozcuoglu
Michael Beetz

AI knowledge representation and reasoning methods consider actions to be blackboxes that abstract away from how they are executed. This abstract view does not suffice for the decision making capabilities required by robotic agents that are to accomplish manipulation tasks. Such robots have to reason about how to pour without spilling, where to grasp a pot, how to open different containers, and so on. To enable such reasoning it is necessary to consider how objects are perceived, how motions can be executed and parameterized, and how motion parameterization affects the physical effects of actions. To this end, we propose to complement and extend symbolic reasoning methods with KnowRob SIM, an additional reasoning infrastructure based on modern game engine technology, including the subsymbolic world modeling through data structures, action simulation based on physics engine, and world scene rendering. We demonstrate how KnowRob SIM can perform powerful reasoning, prediction, and learning tasks that are required for informed decision making in object manipulation.

Details

IROS Conference 2016 Conference Paper

Action recognition and interpretation from virtual demonstrations

Andrei Haidu
Michael Beetz

To properly perform tasks based on abstract instructions, autonomous robots need refined reasoning skills in order to bridge the gap between the ambiguous descriptions and the comprehensive information needed to execute the implied actions. In this article, we present an automated knowledge acquisition system from human executed tasks in virtual environments, and extend the knowledge processing system KNOWROB[1] to be capable to reason on the acquired data. We have set up two scenarios in a physics based simulator: creating a pancake, and garnishing a pizza dough. Users where asked to execute these tasks using the provided tools and ingredients. Using a data processing module we then collect the low-level data and the relevant abstract events from the performed episodes. The recorded data is then made available in a format that robots can understand, by using a symbolic layer to interconnect the two data types in a seamless way.

Details

ICRA Conference 2016 Conference Paper

Open robotics research using web-based knowledge services

Michael Beetz
Daniel Beßler
Jan Oliver Winkler
Jan-Hendrik Worch
Ferenc Balint-Benczedi
Georg Bartels
Aude Billard
Asil Kaan Bozcuoglu

In this paper we discuss how the combination of modern technologies in “big data” storage and management, knowledge representation and processing, cloud-based computation, and web technology can help the robotics community to establish and strengthen an open research discipline. We describe how we made the demonstrator of a EU project review openly available to the research community. Specifically, we recorded episodic memories with rich semantic annotations during a pizza preparation experiment in autonomous robot manipulation. Afterwards, we released them as an open knowledge base using the cloud- and web-based robot knowledge service OPENEASE. We discuss several ways on how this open data can be used to validate our experimental reports and to tackle novel challenging research problems.

Details

IROS Conference 2015 Conference Paper

Learning action failure models from interactive physics-based simulations

Andrei Haidu
Daniel Kohlsdorf
Michael Beetz

Predicting the outcome of an action can help a robot detect failures in advance, and schedule action replanning before an error occurs. We propose using an interactive physics based simulator with the aim of collecting realistic data to be used for learning. We then show how we save and query for specific information from the data more effectively. The data from the simulation is used to learn a failure detection model which is utilized by a real robot performing the same actions. We show that learning from simulation data is realistic enough to be applied on a real robot. The learning algorithm is more simple in design and outperforms the more complex one from our previous work.

Details

IROS Conference 2014 Conference Paper

Learning task outcome prediction for robot control from interactive environments

Andrei Haidu
Daniel Kohlsdorf
Michael Beetz

In order to manage complex tasks such as cooking, future robots need to be action-aware and posses common sense knowledge. For example flipping a pancake requires a robot to know that a spatula has to be under a pancake in order to succeed. We present a novel approach for the extraction and learning of action and common sense knowledge, and developed a game using a robot-simulator with realistic physics for data acquisition. The game environment is a virtual kitchen, in which a user has to create a pancake by pouring pancake-mix on an oven and flipping it using a spatula. The interaction is done by controlling a virtual robot hand with a 3D input sensor. We incorporate a realistic fluid simulation in order to gather appropriate data of the pouring action. Furthermore, we present a task outcome prediction algorithm for this specific system and show how to learn a failure model for the pouring and flipping action.

Details

IROS Conference 2013 Conference Paper

Acquiring task models for imitation learning through games with a purpose

Lars Kunze
Andrei Haidu
Michael Beetz

Teaching robots everyday tasks like making pancakes by instructions requires interfaces that can be intuitively operated by non-experts. By performing novel manipulation tasks in a virtual environment using a data glove task-related information of the demonstrated actions can directly be accessed and extracted from the simulator. We translate low-level data structures of these simulations into meaningful first-order representations whereby we are able to select data segments and analyze them at an abstract level. Hence, the proposed system is a powerful tool for acquiring examples of manipulation actions and for analyzing them whereby robots can be informed how to perform a task.

Details