Type of Publication

Others

Date:

3 /

2023

Status

Published

DOI:

10.32604/cmc.2023.037463

TECMH: Transformer-Based Cross-Modal Hashing For Fine-Grained Image-Text Retrieval

Featured in:

Computers, Materials & Continua 2023

Authors:

Qiqi Li, Longfei Ma, Zheng Jiang, Mingyong Li and Bo Jin

Abstract

In recent years, cross-modal hash retrieval has become a popular research field because of its advantages of high efficiency and low storage. Cross-modal retrieval technology can be applied to search engines, cross-modal medical processing, etc. The existing main method is to use a multi-label matching paradigm to finish the retrieval tasks. However, such methods do not use fine-grained information in the multi-modal data, which may lead to sub-optimal results. To avoid cross-modal matching turning into label matching, this paper proposes an end-to-end fine-grained cross-modal hash retrieval method, which can focus more on the fine-grained semantic information of multi-modal data. First, the method refines the image features and no longer uses multiple labels to represent text features but uses BERT for processing. Second, this method uses the inference capabilities of the transformer encoder to generate global fine-grained features. Finally, in order to better judge the effect of the fine-grained model, this paper uses the datasets in the image text matching field instead of the traditional label-matching datasets. This article experiment on Microsoft COCO (MS-COCO) and Flickr30K datasets and compare it with the previous classical methods. The experimental results show that this method can obtain more advanced results in the cross-modal hash retrieval field.

Citation
Qiqi Li, Longfei Ma, Zheng Jiang, Mingyong Li and Bo Jin (2023). TECMH: Transformer-Based Cross-Modal Hashing For Fine-Grained Image-Text Retrieval. Computers, Materials & Continua, 75(2). DOI: 10.32604/cmc.2023.037463

Related Content

Post-Doc Researcher
No tagged content to show
No tagged content to show
No tagged content to show

RECENT PUBLICATIONS

Geometric implicit neural representations for signed distance functions

Authors: Luiz Schirmer, Tiago Novello, Vinícius da Silva, Guilherme Schardong, Daniel Perazzo, Hélio Lopes, Nuno Gonçalves, Luiz Velho
Featured in: Special Section on SIBGRAPI 2023 Tutorials

Towards Secure Biometric Solutions: Enhancing Facial Recognition while Protecting User Data

Authors: Jose Silva, Aniana Cruz, Bruno Sousa and Nuno Gonçalves
Featured in: 14th International Conference on Pattern Recognition Applications and Methods (ICPRAM) 2025

StylePuncher: encoding a hidden QR code into images

Authors: Farhad Shadmand, Luiz Schirmer and Nuno Gonçalves
Featured in: 14th International Conference on Pattern Recognition Applications and Methods (ICPRAM) 2025

suggested news

Best Paper Award @ICPRAM 2025
Nuno Gonçalves serves as jury member for PhD...
ACHILLES project launches official Website and Newsletter

RECENT PROJECTS

FACING2 – Face Image Understanding
VISUAL-ID – Unique Visual Identities in Graphics, Images and Faces
UniqueMark

Institute of Systems and Robotics Department of Electrical and Computers Engineering University of Coimbra