Arrow Research search

Author name cluster

Saso Dzeroski

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

13 papers
2 author rows

Possible papers

13

ECAI Conference 2020 Conference Paper

Feature Importance Estimation with Self-Attention Networks

  • Blaz Skrlj
  • Saso Dzeroski
  • Nada Lavrac
  • Matej Petkovic

Black-box neural network models are widely used in industry and science, yet are hard to understand and interpret. Recently, the attention mechanism was introduced, offering insights into the inner workings of neural language models. This paper explores the use of attention-based neural networks mechanism for estimating feature importance, as means for explaining the models learned from propositional (tabular) data. Feature importance estimates, assessed by the proposed Self-Attention Network (SAN) architecture, are compared with the established ReliefF, Mutual Information and Random Forest-based estimates, which are widely used in practice for model interpretation. For the first time we conduct scale-free comparisons of feature importance estimates across algorithms on ten real and synthetic data sets to study the similarities and differences of the resulting feature importance estimates, showing that SANs identify similar high-ranked features as the other methods. We demonstrate that SANs identify feature interactions which in some cases yield better predictive performance than the baselines, suggesting that attention extends beyond interactions of just a few key features and detects larger feature subsets relevant for the considered learning task.

Details

ECAI Conference 2014 Conference Paper

Multi-objective learning of hybrid classifiers

  • Rok Piltaver
  • Mitja Lustrek
  • Jernej Zupancic
  • Saso Dzeroski
  • Matjaz Gams

We propose a multi-objective machine learning approach guaranteed to find the Pareto optimal set of hybrid classification models consisting of comprehensible and incomprehensible submodels. The algorithm run-times are below 1 s for typical applications despite the exponential worst-case time complexity. The user chooses the model with the best comprehensibility-accuracy trade-off from the Pareto front which enables a well informed decision or repeats finding new Pareto fronts with modified seeds. For a classification trees as the comprehensible seed, the hybrids include single black-box model, invoked in hybrid leaves. The comprehensibility of such hybrid classifiers is measured with the proportion of examples classified by the regular leaves. We propose one simple and one computationally efficient algorithm for finding the Pareto optimal hybrid trees, starting from an initial classification tree and a black-box classifier. We evaluate the proposed algorithms empirically, comparing them to the baseline solution set, showing that they often provide valuable improvements. Furthermore, we show that the efficient algorithm outperforms the NSGA-II algorithm in terms of quality of the result set and efficiency (for this optimisation problem). Finally we show that the algorithm returns hybrid classifiers that reflect the expert's knowledge on activity recognition problem well.

Details

JELIA Conference 2006 Invited Paper

From Inductive Logic Programming to Relational Data Mining

  • Saso Dzeroski

Abstract Situated at the intersection of machine learning and logic programming, inductive logic programming (ILP) has been concerned with finding patterns expressed as logic programs. While ILP initially focussed on automated program synthesis from examples, it has recently expanded its scope to cover a whole range of data analysis tasks (classification, regression, clustering, association analysis). ILP algorithms can this be used to find patterns in relational data, i. e. , for relational data mining (RDM). This paper briefly introduces the basic concepts of ILP and RDM and discusses some recent research trends in these areas.

Details

IJCAI Conference 1997 Conference Paper

Integrating Explanatory and Descriptive Learning in ILP

  • Yannis Dimopoulos
  • Saso Dzeroski
  • Antonis Kakas

A learning framework that combines the two frameworks of explanatory and descriptive In­ ductive Logic Programming (ILP) is presented. The induced hypotheses in this framework are pairs of the form (T, IC) where T is a defi­ nite clausal theory and IC is a set of integrity constraints. The two components allow us to combine complementary information from the same data by applying both explanatory and descriptive learning methods. This non-trivial integration is achieved using a nonmonotonic entailment relation for the basic notion of cov­ erage in the combined language of rules and constraints where the constraints can restrict the conclusions derivable by the rules. We present a semantics for the new framework and then discuss different cases where combin­ ing information from explanatory and descrip­ tive ILP could be useful. We present some basic algorithmic frameworks for learning in the new framework, and report on some preliminary ex­ periments with encouraging results.

PDF
v2026.09.13