Author name cluster

Daniel Barath

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

Object-X: Learning to Reconstruct Multi-Modal 3D Object Representations

Gaia Di Lorenzo
Federico Tombari
Marc Pollefeys
Daniel Barath

Learning effective multi-modal 3D representations of objects is essential for numerous applications, such as augmented reality and robotics. Existing methods often rely on task-specific embeddings that are tailored either for semantic understanding or geometric reconstruction. As a result, these embeddings typically cannot be decoded into explicit geometry and simultaneously reused across tasks. In this paper, we propose Object-X, a versatile multi-modal object representation framework capable of encoding rich object embeddings (e. g. , images, point cloud, text) and decoding them back into detailed geometric and visual reconstructions. Object-X operates by geometrically grounding the captured modalities in a 3D voxel grid and learning an unstructured embedding fusing the information from the voxels with the object attributes. The learned embedding enables 3D Gaussian Splatting-based object reconstruction, while also supporting a range of downstream tasks, including scene alignment, single-image 3D object reconstruction, and localization. Evaluations on two challenging real-world datasets demonstrate that Object-X produces high-fidelity novel-view synthesis comparable to standard 3D Gaussian Splatting, while significantly improving geometric accuracy. Moreover, Object-X achieves competitive performance with specialized methods in scene alignment and localization. Critically, our object-centric descriptors require 3-4 orders of magnitude less storage compared to traditional image- or point cloud-based approaches, establishing Object-X as a scalable and highly practical solution for multi-modal 3D scene representation.

PDF Details

ICRA Conference 2024 Conference Paper

Semantically Guided Feature Matching for Visual SLAM

Oguzhan Ilter
Iro Armeni
Marc Pollefeys
Daniel Barath

We introduce a new algorithm that utilizes semantic information to enhance feature matching in visual SLAM pipelines. The proposed method constructs a high-dimensional semantic descriptor for each detected ORB feature. When integrated with traditional visual ones, these descriptors aid in establishing accurate tentative point correspondences between consecutive frames. Additionally, our semantic descriptors enrich 3D map points, enhancing loop closure detection by providing deeper insights into the underlying map regions. Experiments on public large-scale datasets demonstrate that our technique surpasses the accuracy of established methods. Importantly, given its detector-agnostic nature, our algorithm also amplifies the efficacy of modern keypoint detectors, such as SuperPoint. The implementation of our algorithm can be found on Github 3.

Details

IROS Conference 2024 Conference Paper

Volumetric Semantically Consistent 3D Panoptic Mapping

Yang Miao
Iro Armeni
Marc Pollefeys
Daniel Barath

We introduce an online 2D-to-3D semantic instance mapping algorithm aimed at generating comprehensive, accurate, and efficient semantic 3D maps suitable for autonomous agents in unstructured environments. The proposed approach is based on a Voxel-TSDF representation used in recent algorithms. It introduces novel ways of integrating semantic prediction confidence during mapping, producing semantic and instance-consistent 3D regions. Further improvements are achieved by graph optimization-based semantic labeling and instance refinement. The proposed method achieves accuracy superior to the state of the art on public large-scale datasets, improving on a number of widely used metrics. We also highlight a downfall in the evaluation of recent studies: using the ground truth trajectory as input instead of a SLAM-estimated one substantially affects the accuracy, creating a large gap between the reported results and the actual performance on real-world data. The code is available: https://github.com/y9miao/ConsistentPanopticSLAM.

Details

ICRA Conference 2021 Conference Paper

Efficient Recovery of Multi-Camera Motion from Two Affine Correspondences

Banglei Guan
Ji Zhao 0001
Daniel Barath
Friedrich Fraundorfer

We propose an efficient method to estimate the relative pose of a multi-camera system from a minimum of two affine correspondences (ACs). Our solution is novel as it computes the 6DOF relative pose by utilizing a first-order rotation approximation. We directly derive a single polynomial based on the constraint between ACs and the generalized camera model. Then a closed-form solution is found analytically and it produces an accurate relative pose estimation efficiently. Benefiting from the low number of exploited correspondences and the speed of the solver, it speeds up robust estimators, e. g. RANSAC, significantly. The proposed method is evaluated both on synthetic data and real-world image sequences from the KITTI benchmark. It is shown that the proposed solver is superior to the state-of-the-art algorithms in terms of accuracy.

Details

ICRA Conference 2021 Conference Paper

Pose Estimation for Vehicle-mounted Cameras via Horizontal and Vertical Planes

Istan Gergo Gal
Daniel Barath
Levente Hajder

We propose novel solvers for estimating the egomotion of a calibrated camera mounted to a moving vehicle from a single affine correspondence via recovering special homographies. For the first, second and third classes of solvers, the sought plane is expected to be perpendicular to one of the camera axes. For the fourth class, the plane is orthogonal to the ground with unknown normal, e. g. , it is a building facade. All methods are solved via a linear system with a small coefficient matrix, thus, being extremely efficient. Both the minimal and over-determined cases can be solved by the proposed solvers. They are tested on synthetic data and on publicly available real-world datasets. The novel methods are more accurate or comparable to the traditional algorithms and are faster when included in state-of-the-art robust estimators. The source code is publicly available[1].

Details

ICRA Conference 2020 Conference Paper

Least-squares Optimal Relative Planar Motion for Vehicle-mounted Cameras

Levente Hajder
Daniel Barath

A new closed-form solver is proposed minimizing the algebraic error optimally, in the least squares sense, to estimate the relative planar motion of two calibrated cameras. The main objective is to solve the over-determined case, i. e. , when a larger-than-minimal sample of point correspondences is given - thus, estimating the motion from at least three correspondences. The algorithm requires the camera movement to be constrained to a plane, e. g. mounted to a vehicle, and the image plane to be orthogonal to the ground. 1 The solver obtains the motion parameters as the roots of a 6th degree polynomial. It is validated both in synthetic experiments and on publicly available real-world datasets that using the proposed solver leads to results superior to the state-of-the-art in terms of geometric accuracy with no noticeable deterioration in the processing time.

Details

ICRA Conference 2020 Conference Paper

Relative planar motion for vehicle-mounted cameras from a single affine correspondence

Levente Hajder
Daniel Barath

Two solvers are proposed for estimating the extrinsic camera parameters from a single affine correspondence assuming general planar motion. In this case, the camera movement is constrained to a plane and the image plane is orthogonal to the ground. The algorithms do not assume other constraints, e. g. the non-holonomic one, to hold. A new minimal solver is proposed for the semi-calibrated case, i. e. the camera parameters are known except a common focal length. Another method is proposed for the fully calibrated case. Due to requiring a single correspondence, robust estimation, e. g. histogram voting, leads to a fast and accurate procedure. The proposed methods are tested in our synthetic environment and on publicly available real datasets consisting of videos through tens of kilometers. They are superior to the state-of-the-art both in terms of accuracy and processing time.

Details