DeepPre-OM: An Enhanced Pre-processing Framework for Opinion Classification of Microblog Data

Abstract

In the present scenario, the world is moving rapidly with the development of new technologies and innovations. Social networks play a crucial role in society in terms of communication, interaction and sharing of views. Social media platforms, particularly Twitter, generate a continuous stream of short, informal, and often noisy texts. This creates challenges for sentiment analysis. Existing pre-processing approaches failed to retain sentiment cues by integrating emoji to text conversion and hashtag segmentation. To address these challenges, this research work introduces DeepPre_OM, a structured pre-processing framework that incorporates case normalization, hashtag segmentation, emoji translation, slang normalization, tokenization, and lemmatization. Glove embeddings are used to convert the pre-processed text into numerical vectors to maintain the semantic relationships. The experimental results shows that LSTM accuracy improved from 76.8 % to 82.5 % and BiLSTM from 79.2 % to 84.3 % demonstrating the effectiveness of the proposed pipeline. DeepPre_OM not only enhances the accuracy but also enables a more nuanced understanding of user emotions. By using this approach, the researchers and decision makers can gain deeper understanding of public opinions and sentiments to refine the data.