Feature Substitution Using Latent Dirichlet Allocation for Text Classification

• Published in: International Journal of Advanced Computer Science and Applications
• Main Author: Mathivanan N.M.N.; Janor R.M.; Razak S.A.; Md. Ghani N.A.
• Format: Article
• Language: English
• Published: Science and Information Organization 2025
• Online Access: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85216865449&doi=10.14569%2fIJACSA.2025.01601105&partnerID=40&md5=abf0789c4fb07f118c7d1b6f7437a0d3

Text classification plays a pivotal role in natural language processing, enabling applications such as product categorization, sentiment analysis, spam detection, and document organization. Traditional methods, including bag-of-words and TF-IDF, often lead to high-dimensional feature spaces, increasing computational complexity and susceptibility to overfitting. This study introduces a novel Feature Substitution technique using Latent Dirichlet Allocation (FS-LDA), which enhances text representation by replacing non-overlapping high-probability topic words. FS-LDA effectively reduces dimensionality while retaining essential semantic features, optimizing classification accuracy and efficiency. Experimental evaluations on five e- commerce datasets and an SMS spam dataset demonstrated that FS-LDA, combined with Hidden Markov Models (HMMs), achieved up to 95% classification accuracy in binary tasks and significant improvements in macro and weighted F1-scores for multiclass tasks. The innovative approach lies in FS-LDA's ability to seamlessly integrate dimensionality reduction with feature substitution, while its predictive advantage is demonstrated through consistent performance enhancement across diverse datasets. Future work will explore its application to other classification models and domains, such as social media analysis and medical document categorization, to further validate its scalability and robustness. © (2025), (Science and Information Organization). All rights reserved.