Simulated multimodal deep facial diagnosis; Deep facial diagnosisSimulated multimodalFace depth estimationFacial phenotypesCondition-specific facesBilinear

Type of Publication

Journal Articles

Date:

10 /

2024

Status

Published

DOI:

10.1016/j.eswa.2024.123881

Simulated multimodal deep facial diagnosis

Featured in:

Expert Systems with Applications

Authors:

Bo Jin, Nuno Gonçalves, Leandro Cruz, Iurii Medvedev, Yuanyu Yu and Jiujiang Wang

Abstract

Facial phenotypes are extensively studied in medical and biological research, serving as critical markers that potentially indicate underlying genetic traits or medical conditions. With the recent advancements in big data, algorithms, and hardware, deep facial diagnosis, which employs deep learning techniques to systematically examine facial phenotypes and identify signs of certain diseases or medical conditions, has attracted significant attention and research, gradually emerging as a promising tool in precision medicine. Primarily limited by the scarcity of data for training facial diagnosis models, the accuracy of facial diagnosis for various conditions remains low up to now. In the past decade, RGB-D cameras, measuring depth information along with standard RGB capabilities, have proven superior in processing spatial details with more stability and accuracy. Motivated by the facts mentioned above, in this paper, we propose a Simulated Multimodal Framework, which effectively improves the computer-aided facial diagnosis performance of state-of-the-art models in experiments under different conditions. The underlying principle is to leverage the simulated depth by generative models to improve the performance of RGB image recognition. Furthermore, as a rapid and non-invasive tool for disease screening and detection, our proposal demonstrated an average accuracy improvement of over 20% compared to practicing physicians in the study.

Citation
Bo Jin, Nuno Gonçalves, Leandro Cruz, Iurii Medvedev, Yuanyu Yu and Jiujiang Wang. Simulated multimodal deep facial diagnosis. Expert Systems with Applications, Volume 252, Part A, 2024. DOI: 10.1016/j.eswa.2024.123881.

Related Content

Researcher Coordinator, VIS TEAM Leader
Post-Doc Researcher
Researcher
Post-Doc Researcher
No tagged content to show
No tagged content to show
No tagged content to show

RECENT PUBLICATIONS

MorFacing: A Benchmark for Estimation Face Recognition Robustness to Face Morphing Attacks

Authors: Iurii Medvedev and Nuno Gonçalves
Featured in: IEEE International Joint Conference on Biometrics (IJCB 2024)

Face Liveness Detection Competition (LivDet-Face)

Authors: Lambert Igene, Afzal Hossain, Stephanie Schuckers, Mohammad Zahir Uddin Chowdhury, Humaira Rezaie, Ayden Rollins, Jesse Dykes, Rahul Vijaykumar, Sebastien Marcel, Juan Tapia, Carlos Aravena, Daniel Schulz, Nima Karimian and Anafsheh Adami, Diogo Nunes, João Marcos, Nuno Gonçalves, Lovro Sikošek, Borut Batagelj, Nima Schei, David Pabon, Manuela Tiedemann, Vasiliy Pryadchenko, Aleksandr Alenin, Alhasan Alkhaddour, Anton Pimenov, Artem Tregubov, Igor Avdonin, Maxim Lazantsev and Mikhail Pozigun
Featured in: IEEE International Joint Conference on Biometrics Competitions, 2024

Social NSTransformers: Low-Quality Pedestrian Trajectory Prediction

Authors: Zihan Jiang, Yiqun Ma, Bingyu Shi, Xin Lu, Jian Xing, Nuno Gonçalves and Bo Jin
Featured in: IEEE Transactions on Artificial Intelligence

suggested news

Laser engraving of precious metal artifacts (UniqueMark® deterministic...
UniqueMark® and UniQode® Glitter patent published
Paper about protecting facial recognition systems against morphing...

RECENT PROJECTS

FACING2 – Face Image Understanding
VISUAL-ID – Unique Visual Identities in Graphics, Images and Faces
UniqueMark

Institute of Systems and Robotics Department of Electrical and Computers Engineering University of Coimbra