Manfred Jaeger

TMLR Journal 2025 Journal Article

A Self-Explainable Heterogeneous GNN for Relational Deep Learning

• Francesco Ferrini
• Antonio Longa
• Andrea Passerini
• Manfred Jaeger

Recently, significant attention has been given to the idea of viewing relational databases as heterogeneous graphs, enabling the application of graph neural network (GNN) technology for predictive tasks. However, existing GNN methods struggle with the complexity of the heterogeneous graphs induced by databases with numerous tables and relations. Traditional approaches either consider all possible relational meta-paths, thus failing to scale with the number of relations, or rely on domain experts to identify relevant meta-paths. A recent solution does manage to learn informative meta-paths without expert supervision, but assumes that a node’s class depends solely on the existence of a meta-path occurrence. In this work, we present a self-explainable heterogeneous GNN for relational data, that supports models in which class membership depends on aggregate information obtained from multiple occurrences of a meta-path. Experimental results show that in the context of relational databases, our approach effectively identifies informative meta-paths that faithfully capture the model’s reasoning mechanisms. It significantly outperforms existing methods in both synthetic and real-world scenarios.

NeurIPS Conference 2025 Conference Paper

Bridging Theory and Practice in Link Representation with Graph Neural Networks

• Veronica Lachi
• Francesco Ferrini
• Antonio Longa
• Bruno Lepri
• Andrea Passerini
• Manfred Jaeger

Graph Neural Networks (GNNs) are widely used to compute representations of node pairs for downstream tasks such as link prediction. Yet, theoretical understanding of their expressive power has focused almost entirely on graph-level representations. In this work, we shift the focus to links and provide the first comprehensive study of GNN expressiveness in link representation. We introduce a unifying framework, the $k_\phi$-$k_\rho$-$m$ framework, that subsumes existing message-passing link models and enables formal expressiveness comparisons. Using this framework, we derive a hierarchy of state-of-the-art methods and offer theoretical tools to analyze future architectures. To complement our analysis, we propose a synthetic evaluation protocol comprising the first benchmark specifically designed to assess link-level expressiveness. Finally, we ask: does expressiveness matter in practice? We use a graph symmetry metric that quantifies the difficulty of distinguishing links and show that while expressive models may underperform on standard benchmarks, they significantly outperform simpler ones as symmetry increases, highlighting the need for dataset-aware model selection.

JMLR Journal 2022 Journal Article

The AIM and EM Algorithms for Learning from Coarse Data

• Manfred Jaeger

Statistical learning from incomplete data is typically performed under an assumption of ignorability for the mechanism that causes missing values. Notably, the expectation maximization (EM) algorithm is based on the assumption that values are missing at random. Most approaches that tackle non-ignorable mechanisms are based on specific modeling assumptions for these mechanisms. The adaptive imputation and maximization (AIM) algorithm has been introduced in earlier work as a general paradigm for learning from incomplete data without any assumptions on the process that causes observations to be incomplete. In this paper we give a thorough analysis of the theoretical properties of the AIM algorithm, and its relationship with EM. We identify conditions under which EM and AIM are in fact equivalent, and show that when these conditions are not met, then AIM can produce consistent estimates in non-ignorable incomplete data scenarios where EM becomes inconsistent. Convergence results for AIM are obtained that closely mirror the available convergence guarantees for EM. We develop the general theory of the AIM algorithm for discrete data settings, and then develop a general discretization approach that allows to apply the method also to incomplete continuous data. We demonstrate the practical usability of the AIM algorithm by prototype implementations for parameter learning from continuous Gaussian data, and from discrete Bayesian network data. Extensive experiments show that the theoretical differences between AIM and EM can be observed in practice, and that a combination of the two methods leads to robust performance for both ignorable and non-ignorable mechanisms. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2022. ( edit, beta )

IJCAI Conference 2021 Conference Paper

Learning Aggregation Functions

• Giovanni Pellegrini
• Alessandro Tibo
• Paolo Frasconi
• Andrea Passerini
• Manfred Jaeger

Learning on sets is increasingly gaining attention in the machine learning community, due to its widespread applicability. Typically, representations over sets are computed by using fixed aggregation functions such as sum or maximum. However, recent results showed that universal function representation by sum- (or max-) decomposition requires either highly discontinuous (and thus poorly learnable) mappings, or a latent dimension equal to the maximum number of elements in the set. To mitigate this problem, we introduce LAF (Learning Aggregation Function), a learnable aggregator for sets of arbitrary cardinality. LAF can approximate several extensively used aggregators (such as average, sum, maximum) as well as more complex functions (e. g. variance and skewness). We report experiments on semi-synthetic and real data showing that LAF outperforms state-of-the-art sum- (max-) decomposition architectures such as DeepSets and library-based architectures like Principal Neighborhood Aggregation, and can be effectively combined with attention-based architectures.

IJCAI Conference 2020 Conference Paper

A Complete Characterization of Projectivity for Statistical Relational Models

• Manfred Jaeger
• Oliver Schulte

A generative probabilistic model for relational data consists of a family of probability distributions for relational structures over domains of different sizes. In most existing statistical relational learning (SRL) frameworks, these models are not projective in the sense that the marginal of the distribution for size-n structures on induced substructures of size k<n is equal to the given distribution for size-k structures. Projectivity is very beneficial in that it directly enables lifted inference and statistically consistent learning from sub-sampled relational structures. In earlier work some simple fragments of SRL languages have been identified that represent projective models. However, no complete characterization of, and representation framework for projective models has been given. In this paper we fill this gap: exploiting representation theorems for infinite exchangeable arrays we introduce a class of directed graphical latent variable models that precisely correspond to the class of projective relational models. As a by-product we also obtain a characterization for when a given distribution over size-k structures is the statistical frequency distribution of size-k substructures in much larger size-n structures. These results shed new light onto the old open problem of how to apply Halpern et al. 's ``random worlds approach'' for probabilistic inference to general relational signatures.

JMLR Journal 2020 Journal Article

Learning and Interpreting Multi-Multi-Instance Learning Networks

• Alessandro Tibo
• Manfred Jaeger
• Paolo Frasconi

We introduce an extension of the multi-instance learning problem where examples are organized as nested bags of instances (e.g., a document could be represented as a bag of sentences, which in turn are bags of words). This framework can be useful in various scenarios, such as text and image classification, but also supervised learning over graphs. As a further advantage, multi-multi instance learning enables a particular way of interpreting predictions and the decision function. Our approach is based on a special neural network layer, called bag-layer, whose units aggregate bags of inputs of arbitrary size. We prove theoretically that the associated class of functions contains all Boolean functions over sets of sets of instances and we provide empirical evidence that functions of this kind can be actually learned on semi-synthetic datasets. We finally present experiments on text classification, on citation graphs, and social graph data, which show that our model obtains competitive results with respect to accuracy when compared to other approaches such as convolutional networks on graphs, while at the same time it supports a general approach to interpret the learnt model, as well as explain individual predictions. [abs] [ pdf ][ bib ] &copy JMLR 2020. ( edit, beta )

AIJ Journal 2013 Journal Article

Type Extension Trees for feature construction and learning in relational domains

• Manfred Jaeger
• Marco Lippi
• Andrea Passerini
• Paolo Frasconi

Type Extension Trees are a powerful representation language for “count-of-count” features characterizing the combinatorial structure of neighborhoods of entities in relational domains. In this paper we present a learning algorithm for Type Extension Trees (TET) that discovers informative count-of-count features in the supervised learning setting. Experiments on bibliographic data show that TET-learning is able to discover the count-of-count feature underlying the definition of the h-index, and the inverse document frequency feature commonly used in information retrieval. We also introduce a metric on TET feature values. This metric is defined as a recursive application of the Wasserstein–Kantorovich metric. Experiments with a k-NN classifier show that exploiting the recursive count-of-count statistics encoded in TET values improves classification accuracy over alternative methods based on simple count statistics.

ICML Conference 2007 Conference Paper

Parameter learning for relational Bayesian networks

• Manfred Jaeger

UAI Conference 2006 Conference Paper

The AI&M Procedure for Learning from Incomplete Data

• Manfred Jaeger

ICML Conference 2003 Conference Paper

Probabilistic Classifiers and the Concepts They Recognize

• Manfred Jaeger

AIJ Journal 2000 Journal Article

On the complexity of inference about probabilistic relational models

• Manfred Jaeger

We investigate the complexity of probabilistic inference from knowledge bases that encode probability distributions on finite domain relational structures. Our interest here lies in the complexity in terms of the domain under consideration in a specific application instance. We obtain the result that assuming NETIME≠ETIME this problem is not polynomial for reasonably expressive representation systems. The main consequence of this result is that it is unlikely to find inference techniques with a better worst-case behavior than the commonly employed strategy of constructing standard Bayesian networks over ground atoms (knowledge based model construction).

UAI Conference 1998 Conference Paper

Measure Selection: Notions of Rationality and Representation Independence

• Manfred Jaeger

UAI Conference 1997 Conference Paper

Relational Bayesian Networks

• Manfred Jaeger

IJCAI Conference 1995 Conference Paper

Minimum Cross-En cropy Reasoning A Statistical Justification

• Manfred Jaeger

Degrees of belief are formed using observed ev­ idence and statistical background information. In this paper we examine the process of how prior degrees of belief derived from the evidence are combined with statistical data to form more specific degrees of belief. A statistical model for this process then is shown to vindicate the cross-entropy minimization principle as a rule for probabilistic default-inference.

UAI Conference 1994 Conference Paper

A Logic for Default Reasoning About Probabilities

• Manfred Jaeger

AIJ Journal 1993 Journal Article

Circumscription

• Manfred Jaeger

In this paper we demonstrate that some results on the completeness of P-defining theories published earlier are incorrect. We point out that by restricting the original propositions to well-founded theories results somewhat weaker than the original ones can be retained. We also present a theorem that provides some insight into the relation between completeness and reducibility and helps to identify the theories whose minimal models can be adequately handled with circumscription.