Study of a real time voice phishing detection system with machine learning technology

Thumbnail Image




Σκαρπέτης, Ιωάννης

Journal Title

Journal ISSN

Volume Title



This study investigated the development and performance of a real-time vishing (voice phishing) detection system employing machine learning algorithms. The research methodology encompassed the creation of a custom dataset utilizing advanced generative pretrained transformers (ChatGPT), preprocessing techniques, and the evaluation and hyper parameter tuning of various machine learning models, including Logistic Regression, Random Forest, Gradient Boosted Trees (GBT) and deep neural network architectures. Furthermore, to support the real-time detection of such attacks a Speech-To Text system based on the Google API was implemented. Results from the real-time testing of these models indicate a promising capability for distinguishing between legitimate and fraudulent telecommunications in real-time. The hyperband tuned neural network model alongside the gradient boosted trees model displayed the most promising performance, since these models displayed the best accuracy in classifications during the final testing process, suggesting their potential to be suitable for deployment in real-world applications. This study contributes to the ongoing efforts to safeguard individuals against vishing scams and highlights the need for further research on the subject. Future enhancements of the work presented in this study include improvements on the dataset diversity and the research of alternative prediction algorithms and preprocessing methods like transformers. The findings underscore the potential of machine learning in effectively bolstering cybersecurity measures against the evolving threat of voice phishing.



Artificial intelligence and machine learning, Speech to text transcription in real time, Voice phishing detection