Analyzing Public opinion on Climate Change via Twitter: A Machine Learning Approach Using Historical Data

Salah, Ahmad; Mahdi, Mahmoud; Omar, Mohamed

doi:10.21608/jocc.2025.446646

Analyzing Public opinion on Climate Change via Twitter: A Machine Learning Approach Using Historical Data

Document Type : Original Article

Authors

Faculty of Computers and Informatics, Zagazig University

10.21608/jocc.2025.446646

Abstract

Examining public perspective regarding critical issues such as climate change in the context of a vast social media stream necessitates a computational approach. This study provides an initial benchmark of five classical machine learning classifiers (Multinomial Naive Bayes, Logistic Regression, Linear Support Vector classifier (SVM), Random Forest, Gradient Boosting) on multi-class categorization (Anti, Neutral, Pro, News) using the openly available Twitter Climate Change Sentiment dataset (2015-2018). Models were built using Term Frequency-Inverse Document Frequency (TF-IDF) for feature representation and performance was assessed using standard metrics (Accuracy, F1-score), as well as modeling generalizability by comparing training versus testing performance to identify overfitting. Experimental results showed that Linear SVC achieved the highest test F1-score (~70.7) but exhibited significant overfitting (≈28%), while Logistic Regression provided the best compromise producing a competitive F1-score (~67.8) and a notably higher degree of generalizability (≈11.5% drop in F1). Gradient Boosting showed remarkable robustness with minimal overfitting (≈1.4% drop in F1) but had less absolute performance (~58.3% F1). This study provides an important baseline for this classification task and highlights the importance of generalization in evaluating model performance in addition to predictions for reliable stance analysis, particularly in computational social science.

Keywords

References

[1] Chatterjee, Moumita, Piyush Kumar, and Dhrubasish Sarkar. "A novel framework for analyzing climate change tweets from online social media using supervised and unsupervised algorithms." In 2024 IEEE Calcutta Conference (CALCON), pp. 1-5. IEEE, 2024.

[2] Maynard D, Bontcheva K. Understanding climate change tweets: an open source toolkit for social media analysis. InEnviroInfo and ICT for Sustainability 2015 2015 Sep (pp. 242-250). Atlantis Press.

[3] Upadhyaya, Apoorva, Marco Fisichella, and Wolfgang Nejdl. "A multi-task model for sentiment aided stance detection of climate change tweets." In Proceedings of the international AAAI conference on web and social media, vol. 17, pp. 854-865. 2023.

[4] Toupin, Rémi, Florence Millerand, and Vincent Larivière. "Who tweets climate change papers? Investigating publics of research through users’ descriptions." Plos one 17, no. 6 (2022): e0268999.

[5] Mumenthaler, Christian, O. Renaud, R. Gava, and Tobias Brosch. "The impact of local temperature volatility on attention to climate change: Evidence from Spanish tweets." Global environmental change 69 (2021): 102286.

[6] Fownes, Jennifer R., Chao Yu, and Drew B. Margolin. "Twitter and climate change." Sociology Compass 12, no. 6 (2018): e12587.

[7] Marcondes, Francisco S., Adelino Gala, Renata Magalhães, Fernando Perez de Britto, Dalila Durães, and Paulo Novais. "Case Study: LLM-Based Anxiety Climate Index." In Natural Language Analytics with Generative Large-Language Models: A Practical Approach with Ollama and Open-Source LLMs, pp. 53-73. Cham: Springer Nature Switzerland, 2025.

[8] Dahal, Biraj, Sathish AP Kumar, and Zhenlong Li. "Topic modeling and sentiment analysis of global climate change tweets." Social network analysis and mining 9 (2019): 1-20.

[9] Maynard, Diana G., and Mark A. Greenwood. "Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis." In Lrec 2014 proceedings. ELRA, 2014.

[10] Amendola, Miriam, Danilo Cavaliere, Carmen De Maio, Giuseppe Fenza, and Vincenzo Loia. "Towards echo chamber assessment by employing aspect-based sentiment analysis and gdm consensus metrics." Online Social Networks and Media 39 (2024): 100276.

[11] Qian, Edward. 2015. “Twitter Climate Change Sentiment Dataset.” Kaggle.com. 2015. https://www.kaggle.com/datasets/edqian/twitter-climate-change-sentiment-dataset?resource=download.

[12] Mi, Zhewei, and Hongwei Zhan. "Text mining attitudes toward climate change: Emotion and sentiment analysis of the twitter corpus." Weather, Climate, and Society 15, no. 2 (2023): 277-287.

[13] Shyrokykh, Karina, Max Girnyk, and Lisa Dellmuth. "Short text classification with machine learning in the social sciences: The case of climate change on Twitter." Plos one 18, no. 9 (2023): e0290762.

[14] Uthirapathy, Samson Ebenezar, and Domnic Sandanam. "Topic Modelling and Opinion Analysis On Climate Change Twitter Data Using LDA And BERT Model." Procedia Computer Science 218 (2023): 908-917.

[15] Kvasničková Stanislavská, Lucie, Ladislav Pilař, Xhesilda Vogli, Tomas Hlavsa, Kateřina Kuralová, Abby Feenstra, Lucie Pilařová, Richard Hartman, and Joanna Rosak-Szyrocka. "Global analysis of Twitter communication in corporate social responsibility area: sustainability, climate change, and waste management." PeerJ Computer Science 9 (2023): e1390.

[16] Webb, Geoffrey I., Eamonn Keogh, and Risto Miikkulainen. "Naïve Bayes." Encyclopedia of machine learning 15, no. 1 (2010): 713-714.

[17] LaValley, Michael P. "Logistic regression." Circulation 117, no. 18 (2008): 2395-2399.

[18] Hearst, Marti A., Susan T. Dumais, Edgar Osuna, John Platt, and Bernhard Scholkopf. "Support vector machines." IEEE Intelligent Systems and their applications 13, no. 4 (1998): 18-28.

[19] Rigatti, S.J., 2017. Random forest. Journal of Insurance Medicine, 47(1), pp.31-39.

[20] Natekin, Alexey, and Alois Knoll. "Gradient boosting machines, a tutorial." Frontiers in neurorobotics 7 (2013): 21.

[21] Jost, François, Ann Dale, and Shoshana Schwebel. "How positive is “change” in climate change? A sentiment analysis." Environmental Science & Policy 96 (2019): 27-36.

[22] Taufek, Tasha Erina, Nor Fariza Mohd Nor, Azhar Jaludin, and Sabrina Tiun. "Public Perceptions on Climate Change: A Sentiment Analysis Approach." GEMA Online Journal of Language Studies 21, no. 4 (2021).

[23] Mohamad Sham, Nabila, and Azlinah Mohamed. "Climate change sentiment analysis using lexicon, machine learning and hybrid approaches." Sustainability 14, no. 8 (2022): 4723.

[24] Rosenberg, Emelie, Carlota Tarazona, Fermín Mallor, Hamidreza Eivazi, David Pastor-Escuredo, Francesco Fuso-Nerini, and Ricardo Vinuesa. "Sentiment analysis on Twitter data towards climate action." Results in Engineering 19 (2023): 101287.

[25] Loureiro, Maria L., and Maria Alló. "Sensing climate change and energy issues: Sentiment and emotion analysis with social media in the UK and Spain." Energy Policy 143 (2020): 111490.

[26] Klein, G., Kim, Y., Deng, Y., Senellart, J., & Rush, A. M. (2017). Opennmt: Open-source toolkit for neural machine translation. arXiv preprint arXiv:1701.02810.