Universität zu Köln

Melsbach, Johannes ORCID: 0000-0001-6904-5037, Haase, Frederic ORCID: 0009-0004-2138-0441, Stahlmann, Sven ORCID: 0000-0001-5989-6073, Hirschmeier, Stefan ORCID: 0000-0002-3754-5261 and Schoder, Detlef (2025). Contrastive Transformer Network for Long Tail Classification. Knowledge-Based Systems, 320. pp. 1-10. Elsevier. ISSN 0950-7051

PDF 1-s2.0-S0950705125006537-main.pdf Bereitstellung unter der CC-Lizenz: Creative Commons Attribution Non-commercial No Derivatives. Download (1MB)

Abstract

[Artikel-Nr.: 113607] In the context of big data, multi-label text classification presents considerable challenges, most notably the long-tail problem, wherein a small number of labels account for the majority of instances, while the vast majority of labels occur only rarely. This imbalance creates a critical bias in classification models, leading to suboptimal performance on tail labels that significantly impacts applications such as recommender systems and search engines. We present CTN-LT (Contrastive Transformer Network for Long Tail Classification), a novel dual-encoder architecture that combines adapted loss functions, contrastive learning and reframes the multi- label text classification as a semantic similarity task to specifically enhance tail label performance. Our method achieves state-of-the-art performance on tail labels while maintaining competitive performance on head labels across multiple benchmark datasets. The model demonstrates superior few-shot and zero-shot capabilities, making it particularly valuable for dynamic environments where new categories frequently emerge. We release our code at https://github.com/jmelsbach/CTN-LT.

Item Type:	Article
Creators:	Creators Email ORCID ORCID Put Code Melsbach, Johannes UNSPECIFIED https://orcid.org/0000-0001-6904-5037 UNSPECIFIED Haase, Frederic UNSPECIFIED https://orcid.org/0009-0004-2138-0441 UNSPECIFIED Stahlmann, Sven UNSPECIFIED https://orcid.org/0000-0001-5989-6073 UNSPECIFIED Hirschmeier, Stefan UNSPECIFIED https://orcid.org/0000-0002-3754-5261 UNSPECIFIED Schoder, Detlef UNSPECIFIED UNSPECIFIED UNSPECIFIED
URN:	urn:nbn:de:hbz:38-804532
Identification Number:	10.1016/j.knosys.2025.113607
Journal or Publication Title:	Knowledge-Based Systems
Volume:	320
Page Range:	pp. 1-10
Number of Pages:	10
Date:	June 2025
Publisher:	Elsevier
ISSN:	0950-7051
Language:	English
Faculty:	Faculty of Management, Economy and Social Sciences
Divisions:	Faculty of Management, Economics and Social Sciences > Business Administration > Information Systems > Professorship for Integrated Information Systems
Subjects:	Economics
['eprint_fieldname_oa_funders' not defined]:	Publikationsfonds UzK
Refereed:	Yes
URI:	http://kups.ub.uni-koeln.de/id/eprint/80453