Toggle Main Menu Toggle Search

Open Access padlockePrints

Mutual Balancing in State-Object Components for Compositional Zero-Shot Learning

Lookup NU author(s): Dr Shidong WangORCiD

Downloads

Full text for this publication is not currently held within this repository. Alternative links are provided below where available.


Abstract

© 2024 Elsevier LtdCompositional Zero-Shot Learning (CZSL) aims to recognize unseen compositions from seen states and objects. The disparity between the manually labeled semantic information and its actual visual features causes a significant imbalance of visual deviation in the distribution of various object classes and state classes, which is ignored by existing methods. To ameliorate these issues, we consider the CZSL task as an unbalanced multi-label classification task and propose a novel method called MUtual balancing in STate-object components (MUST) for CZSL, which provides a balancing inductive bias for the model. In particular, we split the classification of the composition classes into two consecutive processes to analyze the entanglement of the two components to get additional knowledge in advance, which reflects the degree of visual deviation between the two components. We use the knowledge gained to modify the model's training process in order to generate more distinct class borders for classes with significant visual deviations. Extensive experiments demonstrate that our approach significantly outperforms the state-of-the-art on MIT-States, UT-Zappos, and C-GQA when combined with the basic CZSL frameworks, and it can improve various CZSL frameworks. Our code is available at https://github.com/LanchJL/MUST.


Publication metadata

Author(s): Jiang C, Ye Q, Wang S, Shen Y, Zhang Z, Zhang H

Publication type: Article

Publication status: Published

Journal: Pattern Recognition

Year: 2024

Volume: 152

Online publication date: 26/03/2024

Acceptance date: 22/03/2024

ISSN (print): 0031-3203

ISSN (electronic): 1873-5142

Publisher: Elsevier Ltd

URL: https://doi.org/10.1016/j.patcog.2024.110451

DOI: 10.1016/j.patcog.2024.110451


Altmetrics

Altmetrics provided by Altmetric


Share