Automated Generation of Chinese Text-Image Summaries Using Deep Learning Techniques

• Published in: TRAITEMENT DU SIGNAL
• Main Authors: Xu, Meiling; Abd Rahman, Hayati; Li, Feng
• Format: Article
• Language: English
• Published: INT INFORMATION & ENGINEERING TECHNOLOGY ASSOC 2023
• Subjects: Computer Science; Engineering
• Online Access: https://www-webofscience-com.uitm.idm.oclc.org/wos/woscc/full-record/WOS:001137494800030
• ISSN: 0765-0019; 1958-5608
• DOI: 10.18280/ts.400644

In the era of the internet, an abundance of Chinese text-image content is continuously produced, necessitating effective automated technologies for processing and summarizing these materials. Automated generation of Chinese text-image summaries facilitates rapid comprehension of key information, thereby enhancing the efficiency of information consumption. Due to the unique characteristics of the Chinese language, traditional automatic summarization techniques are inadequately transferable, prompting the development of text-image summary generation technologies tailored to Chinese features. Research indicates that while existing natural language processing and deep learning techniques have made strides in text summarization, deficiencies remain in the deep semantic mining and integration of text-image content. This study primarily focuses on two aspects: Firstly, a generative approach based on an enhanced MaliGAN model, employing deep learning models to improve text generation quality. Secondly, a retrieval-based approach, utilizing cross-modal similarity retrieval to extract text information most relevant to the image content, guiding summary generation. Additionally, this study innovatively proposes a model architecture comprising segmentation, cross-modal retrieval, and adaptive fusion strategy modules, significantly augmenting the accuracy and reliability of text-image summary generation.