Deep learning-based spam detection for WhatsApp chatbot fallback reduction
10.11591/ijai.v15.i1.pp909-918
Satrio Sadewo
,
Amalia Zahra
Chatbots on WhatsApp are widely used for customer service, but their effectiveness is often undermined by fallback responses when user input cannot be understood. A major cause of these fallbacks is unsolicited spam, which disrupts interactions and reduces service quality. This study develops and evaluates a spam detection system aimed at reducing fallback rates and enhancing user experience. A comparative analysis was conducted between traditional machine learning models (support vector machine (SVM) and decision tree (DT)) and advanced deep learning architectures, including long short-term memory (LSTM) variants (vanilla, bidirectional, stacked, convolutional neural network (CNN)-LSTM, and encoder-decoder) and transformer-based models (bidirectional encoder representations from transformers (BERT)-base, DistilBERT, and cross-lingual language model robustly optimized BERT pretraining approach (XLM-ROBERTa)). Using 170,000 messages sampled from 18 million interactions collected between July 2022 and December 2023, the models were assessed with standard evaluation metrics. Results show that CNN-LSTM and DistilBERT achieved the most robust performance. CNN-LSTM attained a precision of 0.92, recall of 0.91, F1-score of 0.91, and accuracy of 0.94, while DistilBERT achieved precision of 0.92, recall of 0.89, F1-score of 0.90, and accuracy of 0.93. These findings highlight their superior ability to capture contextual patterns in spam messages. Implementing such models is expected to significantly lower fallback rates, thereby improving chatbot reliability and user satisfaction.
