A scalable hybrid deep learning framework for mining actionable knowledge from large-scale and uncertain Twitter data

Abstract

Existing deep learning approaches often exhibit limitations in contextual comprehension, high computational overhead, and restricted generalization when processing large-scale, tweet-level, and semantically ambiguous text. Moreover, deploying such computationally intensive models in real-time internet of things (IoT)-enabled monitoring systems and embedded platforms introduces additional constraints related to latency, memory footprint, and energy efficiency. To address these challenges, this work proposes a scalable hybrid deep learning framework (SHDLF). The proposed framework effectively captures semantic, syntactic, and temporal dependencies in both short and long social media texts through a novel integration of transformer-based representations and attention-driven feature fusion mechanisms. The architecture is designed with a modular and parallelizable structure to facilitate hardware-aware optimization and potential deployment on embedded and reconfigurable computing platforms, enabling efficient edge-level processing of high-velocity Twitter streams. Extensive experimental evaluations conducted on a large benchmark Twitter dataset demonstrate that SHDLF consistently outperforms state-of-the-art models, including convolutional neural network (CNN), bidirectional long short-term memory (BiLSTM), and baseline bidirectional encoder representations from transformers (BERT)-based architectures, in terms of accuracy, F1-score, and robustness under noisy conditions. The results confirm that SHDLF offers a robust, scalable, and computationally efficient solution for extracting reliable sentiment insights from noisy and dynamically evolving social media data.