Application of Multi-Modal Neural Networks in Verifying the Authenticity of News Text and Images

• Published in: TRAITEMENT DU SIGNAL
• Main Authors: Li, Feng; Xu, Meiling; Rosli, Marshima Mohd
• Format: Article
• Language: English
• Published: INT INFORMATION & ENGINEERING TECHNOLOGY ASSOC 2023
• Subjects: Computer Science; Engineering
• Online Access: https://www-webofscience-com.uitm.idm.oclc.org/wos/woscc/full-record/WOS:001137494800004
• ISSN: 0765-0019; 1958-5608
• DOI: 10.18280/ts.400606

In the digital information era, the authenticity of news media is crucial for shaping public opinion and maintaining social stability. The verification of authenticity, especially in news content rich in both text and images, has become a significant task in safeguarding online information security. Traditional methods, often relying on single-modality analysis, are ineffectual against the complex interplay and manipulation techniques possible between news text and accompanying images. Addressing this, research in multi-modal neural networks has emerged, aiming to enhance verification effectiveness by integrating information from both text and images. However, limitations exist in existing research regarding key issue resolution and modality fusion strategies, including inadequate exploration in searching for anomalously similar image blocks, reducing false alarms, and post-processing of tampered images. This study systematically investigates these challenges in news text and image authenticity verification, proposing a novel multi-modal fusion method. This method encompasses three components: a joint gated co-attention mechanism, a filtering gate mechanism, and a joint cooperative representation. These effectively combine text and image information, enhancing the model's ability to discern complex manipulations. The findings of this study not only advance theoretical research in multi -modal fusion but also provide a powerful tool for news media to verify authenticity, significantly contributing to the fight against fake news and maintaining the integrity of information dissemination.