Probabilistic AND-OR Attribute Grouping for Zero-Shot Learning

In zero-shot learning (ZSL), a classifier is trained to recognize visual classes without any image samples. Instead, it is given semantic information about the class, like a textual description or a set of attributes. Learning from attributes could benefit from explicitly modeling structure of the attribute space. Unfortunately, learning of general structure from empirical samples is hard with typical dataset sizes. Here we describe LAGO1, a probabilistic model designed to capture natural soft andor relations across groups of attributes. We show how this model can be learned end-toend with a deep attribute-detection model. The soft group structure can be learned from data jointly as part of the model, and can also readily incorporate prior knowledge about groups if available. The soft and-or structure succeeds to capture meaningful and predictive structures, improving the accuracy of zero-shot learning on two of three benchmarks. Finally, LAGO reveals a unified formulation over two ZSL approaches: DAP (Lampert et al. , 2009) and ESZSL (Romera-Paredes & Torr, 2015). Interestingly, taking only one singleton group for each attribute, introduces a new soft-relaxation of DAP, that outperforms DAP by ∼40%. 1 A video of the highlights, and code is available at: http: //chechiklab. biu. ac. il/˜yuvval/LAGO/ Figure 1: Classifying a bird species based on attributes from (Wah et al. , 2011). The Mourning Warbler can be distinguished from other species by a combination of a grey head and olive-green underparts. Both human raters and machine learning models may confuse semantically-similar attributes like olive or green wings. These attribute naturally cluster into ”OR” groups, where we aim to recognize this species if the wing is labeled as either green or olive. The LAGO model (Eq. 4) weighs attributes detection, inferring classes based on within-group soft-OR and across-groups soft-AND. In general, OR-groups include alternative choices of a property (wing color: {red, olive, green}) and soft-OR allows to weigh down class-irrelevant attributes (here, wing: red).

Authors

• Yuval Atzmon
• Gal Chechik

Keywords

No keywords are indexed for this paper.