Comparative Analysis of Word Embeddings for Multiclass Cyberbullying Detection


  • Azhi Faraj Department of Computer Engineering, Faculty of Engineering, Dokuz Eylul University, Izmir, Turkey, Department of Information Technology, College of Commerce, Sulaimani University, Sulaymaniyah, Iraq
  • Semih Utku Department of Computer Engineering, Faculty of Engineering, Dokuz Eylul University, Izmir, Turkey



Cyberbullying detection, Word embeddings, Deep learning, Machine learning, Text classification


Cyberbullying has emerged as a pervasive concern in modern society, particularly within social media platforms. This phenomenon encompasses employing digital communication to instill fear, threaten, harass, or harm individuals. Given the prevalence of social media in our lives, there is an escalating need for effective methods to detect and combat cyberbullying. This paper aims to explore the utilization of word embeddings and to discern the comparative effectiveness of trainable word embeddings, pre-trained word embeddings, and fine-tuned language models in multiclass cyberbullying detection. Distinguishing from previous binary classification methods, our research delves into nuanced multiclass detection. The exploration of word embeddings holds significant promise due to its ability to transform words into dense numerical vectors within a high-dimensional space. This transformation captures intricate semantic and syntactic relationships inherent in language, enabling machine learning (ML) algorithms to discern patterns that might signify cyberbullying. In contrast to previous research, this work delves beyond primary binary classification and centers on the nuanced realm of multiclass cyberbullying detection. The research employs diverse techniques, including convolutional neural networks and bidirectional long short-term memory, alongside well-known pre-trained models such as word2vec and bidirectional encoder representations from transformers (BERT). Moreover, traditional ML algorithms such as K-nearest neighbors, Random Forest, and Naïve Bayes are integrated to evaluate their performance vis-à-vis deep learning models. The findings underscore the promise of a fine-tuned BERT model on our dataset, yielding the most promising results in multiclass cyberbullying detection, and achieving the best-recorded accuracy of 85% on the dataset.


R. S. Tokunaga. “Following you home from school: A critical review and synthesis of research on cyberbullying victimization”. Computers in Human Behavior, vol. 26, no. 3, pp. 277-287, 2010.

K. Hellfeldt, L. López-Romero and H. Andershed. “Cyberbullying and psychological well-being in young adolescence: The potential protective mediation effects of social support from family, friends, and teachers”. International Journal of Environmental Research and Public Health, vol. 17, no. 1, p. 45, 2020.

K. Rudnicki, H. Vandebosch, P. Voué and K. Poels. “Systematic review of determinants and consequences of bystander interventions in online hate and cyberbullying among adults”. Behaviour and Information Technology, vol. 42, no. 5, pp. 527-544, 2023.

H. Rosa, N. Pereira, R. Ribeiro, P. Ferreira, J. Carvalho, S. Oliveira, L. Coheur, P. Paulino, A. V. Simão and I. Trancoso. “Automatic cyberbullying detection: A systematic review”. Computers in Human Behavior, vol. 93, pp. 333-345, 2019.

F. Elsafoury, S. Katsigiannis, Z. Pervez and N. Ramzan. “When the timeline meets the pipeline: A survey on automated cyberbullying detection”. IEEE Access, vol. 9, pp. 103541-103563, 2021.

D. Yin, Z. Xue, L. Hong, B. D. Davison, A. Kontostathis and L. Edwards. “Detection of harassment on web 2.0”. Proceedings of the Content Analysis in the Web, vol. 2, pp. 1-7, 2009.

F. Almeida and G. Xexéo. “Word Embeddings: A Survey”. arXiv, 2023. Available from: [Last accessed on 2023 Nov 13].

D . Yin, Z. Xue and L. Hong. “Detection of Harassment on Web 2.0. In: Proceedings of the Content Analysis in the WEB, vol. 2, pp. 1-7, 2009.

C. Iwendi, G. Srivastava, S. Khan and P. K. R. Maddikunta. “Cyberbullying detection solutions based on deep learning architectures”. Multimedia Systems, vol. 29, no. 3, pp. 1839- 1852, 2023.

B. A. Talpur and D. O’Sullivan. “Multi-class imbalance in text classification: A feature engineering approach to detect cyberbullying in twitter”. Informatics, vol. 7, p. 52, 2020.

A. Bozyiğit, S. Utku and E. Nasibov. “Cyberbullying detection: Utilizing social media features”. Expert Systems with Applications, vol. 179, p. 115001, 2021.

A. Aizawa. “An information-theoretic perspective of tf-idf measures”. Information Processing and Management, vol. 39, no. 1, pp. 45-65, 2003.

K. Dinakar, R. Reichart and H. Lieberman. “Modeling the Detection of Textual Cyberbullying. In: Proceedings of the International AAAI Conference on Web and Social Media”, pp. 11-17, 2011. Available from: [Last accessed on 2023 Nov 13].

A. Dewani, M. A. Memon and S. Bhatti. “Cyberbullying detection: Advanced preprocessing techniques and deep learning architecture for Roman Urdu data”. Journal of Big Data, vol. 8, no. 1, p. 160, 2021.

S. Agrawal and A. Awekar. Deep learning for detecting cyberbullying across multiple social media platforms. In: G. Pasi, B. Piwowarski, L. Azzopardi and A. Hanbury, Eds. “Advances in Information Retrieval. Lecture Notes in Computer Science”. vol. 10772. Springer International Publishing, Cham, pp. 141-153, 2018.

P. Badjatiya, S. Gupta, M. Gupta and V. Varma. “Deep Learning for Hate Speech Detection in Tweets. In: Proceedings of the 26th International Conference on World Wide Web Companion- WWW ’17 Companion”. ACM Press, Perth, Australia, 2017, pp. 759-760.

E. Wulczyn, N. Thain and L. Dixon. “Ex Machina: Personal Attacks Seen at Scale. In: Proceedings of the 26th International Conference on World Wide Web”. International World Wide Web Conferences Steering Committee, Perth Australia, pp. 1391-1399, 2017.

M. H. Obaid, S. K. Guirguis and S. M. Elkaffas. “Cyberbullying detection and severity determination model”. IEEE Access, vol. 11, pp. 97391-97399, 2023.

T. Mikolov, W. Yih and G. Zweig. “Linguistic Regularities in Continuous Space Word Representations. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies”, pp. 746-751, 2013. Available from: 1090.pdf [Last accessed on 2024 Jan 10].

C. Wang, P. Nulty and D. Lillis. “A Comparative Study on Word Embeddings in Deep Learning for Text Classification. In: Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval”. ACM, Seoul Republic of Korea, pp. 37-46, 2020.

M. Das, S. Banerjee and P. Saha. “Abusive and Threatening Language Detection in Urdu Using Boosting based and BERT based Models: A Comparative Approach”. arXiv, 2021. Available from: [Last accessed on 2024 Jan 10].

S. Gaikwad, T. Ranasinghe, M. Zampieri and C. M. Homan. “Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of Marathi”. arXiv, 2021. Available from: [Last accessed on 2024 Jan 10].

D. Saha, N. Paharia, D. Chakraborty, P. Saha and A. Mukherjee, “Hate-Alert@DravidianLangTech-EACL2021: Ensembling Strategies for Transformer-based Offensive Language Detection”. arXiv, 2021.

A. M. Ishmam and S. Sharmin. “Hateful Speech Detection in Public Facebook Pages for the Bengali Language. In: 2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA)”, IEEE, pp. 555-560, 2019.

R. Cao, R. K. W. Lee and T. A. Hoang. “DeepHate: Hate Speech Detection Via Multi-Faceted Text Representations. In: 12th ACM Conference on Web Science”. ACM, Southampton United Kingdom, pp. 11-20, 2020.

J. Wang, K. Fu and C. T. Lu. “Sosnet: A Graph Convolutional Network Approach to Fine-grained Cyberbullying Detection. In: 2020 IEEE International Conference on Big Data (Big Data)”, IEEE, pp. 1699-1708, 2020.

T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado and J. Dean, “Distributed Representations of Words and Phrases and Their Compositionality. In: Advances in Neural Information Processing Systems”. vol. 26, 2013. Available from: [Last accessed on 2024 Jan 10].

J. Pennington, R. Socher and C. D. Manning. “Glove: Global Vectors for Word Representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)”, pp. 1532-1543, 2014.

J. Devlin, M. W. Chang, K. Lee and K. Toutanova. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”. arXiv, 2019. Available from: abs/1810.04805 [Last accessed on 2024 Jan 10].

T. H. Aldhyani, M. H. Al-Adhaileh and S. N. Alsubari. “Cyberbullying identification system based deep learning algorithms”. Electronics, vol. 11, no. 20, p. 3273, 2022.

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas,… and E. Duchesnay. “Scikit-learn: Machine learning in python”. Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011.



How to Cite

Faraj, A., & Utku, S. (2024). Comparative Analysis of Word Embeddings for Multiclass Cyberbullying Detection. UHD Journal of Science and Technology, 8(1), 55–63.