Authors :
Suhag Pandya
Volume/Issue :
Volume 9 - 2024, Issue 12 - December
Google Scholar :
https://tinyurl.com/3vyvj3vp
Scribd :
https://tinyurl.com/yehve698
DOI :
https://doi.org/10.5281/zenodo.14575886
Abstract :
Sentiment analysis is widely recognised as the
most actively researched area in data mining. These days,
a number of social media platforms have been created, and
Twitter is a crucial instrument for exchanging and
gathering people's thoughts, feelings, opinions, and
attitudes about specific things. This paper presents a
comparative analysis of Large Language Models (LLMs)
and ML models for sentiment analysis on a Twitter
dataset. The study evaluates a performance of XLNet,
advance algorithms including KNN, RF, and XGBoost.
The sentiment analysis methodology involves pre-
processing the Twitter dataset through noise removal and
tokenisation, followed by feature extraction using methods
like Bag-of-Words and Word2Vec. Results show that
XLNet is superior to the conventional models; It has 99%
precision, recall, and F1-score values and an accuracy rate
of 99.54%. In comparison, KNN achieves 78% accuracy,
85% precision, 88% recall, and 86% F1 score, while RF
and XGBoost exhibit lower performance with accuracy
rates of 69% and 60%, respectively. The performance
comparison highlights the superior capabilities of XLNet
for sentiment classification tasks, indicating its potential
for enhancing text classification applications. Research in
the future can look at ways to improve XLNet's
performance on bigger and more complicated datasets by
combining it with more sophisticated deep learning
methods like attention mechanisms and transfer learning.
Keywords :
Sentiment Analysis, LLM, Twitter Dataset, Natural Language Processing, Text Classification, Social Media Analytics.
References :
- B. Patel, V. K. Yarlagadda, N. Dhameliya, K. Mullangi, and S. C. R. Vennapusa, “Advancements in 5G Technology: Enhancing Connectivity and Performance in Communication Engineering,” Eng. Int., vol. 10, no. 2, pp. 117–130, 2022, doi: 10.18034/ei.v10i2.715.
- S. Bauskar and S. Clarita, “AN ANALYSIS: EARLY DIAGNOSIS AND CLASSIFICATION OF PARKINSON’S DISEASE USING MACHINE LEARNING TECHNIQUES,” Int. J. Comput. Eng. Technol., vol. 12, no. 01, pp. 54-66., 2021, doi: 10.5281/zenodo.13836264.
- V. V. Kumar, A. Sahoo, S. K. Balasubramanian, and S. Gholston, “Mitigating healthcare supply chain challenges under disaster conditions: a holistic AI-based analysis of social media data,” Int. J. Prod. Res., 2024, doi: 10.1080/00207543.2024.2316884.
- J. Thomas, “A Survey of E-Commerce Integration in Supply Chain Management for Retail and Consumer Goods in Emerging Markets,” J. Emerg. Technol. Innov. Res., vol. 10, no. 12, 2023.
- V. A. and S. S. Sonawane, “Sentiment Analysis of Twitter Data: A Survey of Techniques,” Int. J. Comput. Appl., 2016, doi: 10.5120/ijca2016908625.
- Sahil Arora and Pranav Khare, “AI/ML-Enabled Optimization of Edge Infrastructure: Enhancing Performance and Security,” Int. J. Adv. Res. Sci. Commun. Technol., vol. 6, no. 1, pp. 1046–1053, 2024, doi: 10.48175/568.
- Pranav Khare and Shristi Srivastava, “Data-driven product marketing strategies: An in-depth analysis of machine learning applications,” Int. J. Sci. Res. Arch., vol. 10, no. 2, pp. 1185–1197, Dec. 2023, doi: 10.30574/ijsra.2023.10.2.0933.
- K. Patel, “AN ANALYSIS OF QUALITY ASSURANCE FOR BUSINESS INTELLIGENCE PROCESS IN EDUCATION SECTOR,” IJNRD - Int. J. Nov. Res. Dev., vol. 9, no. 9, pp. a884–a896, 2024.
- V. S. Thokala, “Improving Data Security and Privacy in Web Applications : A Study of Serverless Architecture,” Int. Res. J., vol. 11, no. 12, pp. 74–82, 2024.
- S. Elmitwalli and J. Mehegan, “Sentiment analysis of COP9-related tweets: a comparative study of pre-trained models and traditional techniques,” Front. Big Data, vol. 7, 2024, doi: 10.3389/fdata.2024.1357926.
- S. G. Thomas Jubin, Kirti Vinod Vedi, “Enhancing Supply Chain Resilience Through Cloud-Based SCM and Advanced Machine Learning: A Case Study of Logistics,” J. Emerg. Technol. Innov. Res., vol. 8, no. 9, 2021.
- N. Abid, “A Climbing Artificial Intelligence for Threat Identification in Critical Infrastructure Cyber Security,” Int. J. Res. Anal. Rev., vol. 9, no. 4, 2022.
- S. S. Pranav Khare, “The Impact of AI on Product Management : A Systematic Review and Future Trends,” Int. J. Res. Anal. Rev. (IJRAR, vol. 9, no. 4, 2022.
- A. P. A. Singh and N. Gameti, “Leveraging Digital Twins for Predictive Maintenance : Techniques , Challenges , and Application,” IJSART, vol. 10, no. 9, 2024.
- V. V. Kumar, F. W. Liou, S. N. Balakrishnan, and V. Kumar, “Economical impact of RFID implementation in remanufacturing: a Chaos-based Interactive Artificial Bee Colony approach,” J. Intell. Manuf., vol. 26, no. 4, pp. 815–830, Aug. 2015, doi: 10.1007/s10845-013-0836-9.
- S. G. Jubin Thomas, Kirti Vinod Vedi, “Effects of supply chain management strategies on the overall performance of the organization,” Int. J. Sci. Res. Arch., vol. 13, no. 01, pp. 709–719, 2024.
- S. Bauskar, “A Review on Database Security Challenges in Cloud Computing Environment,” Int. J. Comput. Eng. Technol., vol. 15, pp. 842–852, 2024, doi: 10.5281/zenodo.13922361.
- K. Patel, “The Impact of Data Quality Assurance Practices in Internet of Things (IoT) Technology,” Int. J. Tech. Innov. Mod. Eng. Sci., vol. 10, no. 10, pp. 1–8, 2024.
- V. S. Thokala, “Integrating Machine Learning into Web Applications for Personalized Content Delivery using Python,” Int. J. Curr. Eng. Technol., vol. 11, no. 6, pp. 652–660, 2021, doi: https://doi.org/10.14741/ijcet/v.11.6.9.
- N. Parveen, P. Chakrabarti, B. T. Hung, and A. Shaik, “Twitter sentiment analysis using hybrid gated attention recurrent network,” J. Big Data, 2023, doi: 10.1186/s40537-023-00726-3.
- S. K. R. Anumandla, V. K. Yarlagadda, S. C. R. Vennapusa, and K. R. V Kothapalli, “Unveiling the Influence of Artificial Intelligence on Resource Management and Sustainable Development: A Comprehensive Investigation,” Technol. \& Manag. Rev., vol. 5, no. 1, pp. 45–65, 2020.
- Abhishek Goyal, “Driving Continuous Improvement in Engineering Projects with AI-Enhanced Agile Testing and Machine Learning,” Int. J. Adv. Res. Sci. Commun. Technol., vol. 3, no. 3, pp. 1320–1331, Jul. 2023, doi: 10.48175/IJARSCT-14000T.
- H. Sinha, “ANALYZING MOVIE REVIEW SENTIMENTS ADVANCED MACHINE LEARNING AND NATURAL LANGUAGE PROCESSING METHODS,” Int. Res. J. Mod. Eng. Technol. Sci. (, vol. 06, no. 08, pp. 1326–1337, 2024, doi: https://www.doi.org/10.56726/IRJMETS60895.
- R. Tandon, “An Analysis Of COVID-19 Tweets Sentiments Based On Large Language Models (Llms),” Int. J. Res. Anal. Rev., vol. 11, no. 3, 2024.
- N. Yadav, O. Kudale, A. Rao, S. Gupta, and A. Shitole, “Twitter Sentiment Analysis Using Supervised Machine Learning,” in Lecture Notes on Data Engineering and Communications Technologies, 2021. doi: 10.1007/978-981-15-9509-7_51.
- A. and P. Khare, “Cloud Security Challenges : Implementing Best Practices for Secure SaaS Application Development,” Int. J. Curr. Eng. Technol., vol. 11, no. 6, pp. 669–676, 2021, doi: https://doi.org/10.14741/ijcet/v.11.6.11.
- R. Tandon, “Social Media Texts Classification and Analysis Based on Large Language Models (LLMs),” J. Emerg. Technol. Innov. Res., vol. 11, no. 9, pp. a751–a757, 2024.
- B. Ihnaini, W. Sun, Y. Cai, Z. Xu, and R. Sangi, “Sentiment Analysis of Song Dynasty Classical Poetry Using Fine-Tuned Large Language Models: A Study with LLMs,” in 2024 7th International Conference on Artificial Intelligence and Big Data (ICAIBD), 2024, pp. 590–597. doi: 10.1109/ICAIBD62003.2024.10604440.
- H. A. Patrick, J. Praveen Gujjar, M. H. Sharief, and U. Mukherjee, “Sentiment Analysis Perspective using Supervised Machine Learning Method,” in 2023 5th International Conference on Electrical, Computer and Communication Technologies, ICECCT 2023, 2023. doi: 10.1109/ICECCT56650.2023.10179807.
- R. D. Gore, V. Y. Borole, R. S. Gupta, V. Sonaje, P. Chaudhari, and B. W. Gawali, “Sentiments Analysis on Amazon Product Reviews Using Supervised Machine Learning Algorithms,” in 2023 International Conference on Integration of Computational Intelligent System, ICICIS 2023, 2023. doi: 10.1109/ICICIS56802.2023.10430298.
- B. Waspodo, Q. Aini, F. R. Singgih, R. H. Kusumaningtyas, and E. Fetrina, “Support Vector Machine and Lexicon based Sentiment Analysis on Kartu Prakerja (Indonesia Pre-Employment Cards Government Initiatives),” in 2022 10th International Conference on Cyber and IT Service Management, CITSM 2022, 2022. doi: 10.1109/CITSM56380.2022.9935990.
- E. Asmawati, A. Saikhu, and D. Siahaan, “Sentiment Analysis of Text Memes: A Comparison among Supervised Machine Learning Methods,” in International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), 2022. doi: 10.23919/EECSI56542.2022.9946506.
- J. C. Gope, T. Tabassum, M. M. Mabrur, K. Yu, and M. Arifuzzaman, “Sentiment Analysis of Amazon Product Reviews Using Machine Learning and Deep Learning Models,” in 2022 International Conference on Advancement in Electrical and Electronic Engineering, ICAEEE 2022, 2022. doi: 10.1109/ICAEEE54957.2022.9836420.
- S. Al-Hagree and G. Al-Gaphari, “Arabic Sentiment Analysis Based Machine Learning for Measuring User Satisfaction with Banking Services’ Mobile Applications : Comparative Study,” in 2022 2nd International Conference on Emerging Smart Technologies and Applications, eSmarTA 2022, 2022. doi: 10.1109/eSmarTA56775.2022.9935486.
- P. K. Vishwakarma and M. R. Suyambu, “An Analysis of Engineering , Procurement And Construction ( EPC ) -Contracts Based on Renewable Energy,” IJSART, vol. 10, no. 10, 2024.
- R. Goyal, “EXPLORING THE PERFORMANCE OF MACHINE LEARNING MODELS FOR CLASSIFICATION AND IDENTIFICATION OF FRAUDULENT INSURANCE CLAIMS,” Int. J. Core Eng. Manag., vol. 7, no. 10, 2024.
- P. Khare and S. Srivastava, “AI-Powered Fraud Prevention: A Comprehensive Analysis of Machine Learning Applications in Online Transactions,” J. Emerg. Technol. Innov. Res., vol. 10, pp. f518–f525, 2023.
- R. K. Arora, Mohd.Muqeem, and M. Saxena, “Developing a Comprehensive Security Framework for Detecting and Mitigating IoT device Attack,” Sep. 30, 2024. doi: 10.21203/rs.3.rs-5165811/v1.
- N. G. Abhinav Parashar, “Asset Master Data Management: Ensuring Accuracy and Consistency in Industrial Operations,” IJNRD - Int. J. Nov. Res. Dev., vol. 9, no. 9, pp. 861-a867, 2024.
- S. Bauskar, “Enhancing System Observability with Machine Learning Techniques for Anomaly Detection,” Int. J. Manag. IT Eng., vol. 14, no. 10, pp. 64–70, 2024.
- R. Arora, S. Gera, and M. Saxena, “Mitigating security risks on privacy of sensitive data used in cloud-based ERP applications,” in Proceedings of the 2021 8th International Conference on Computing for Sustainable Global Development, INDIACom 2021, 2021. doi: 10.1109/INDIACom51348.2021.00081.
- S. Bauskar, “BUSINESS ANALYTICS IN ENTERPRISE SYSTEM BASED ON APPLICATION OF ARTIFICIAL INTELLIGENCE,” Int. Res. J. Mod. Eng. Technol. Sci., vol. 04, no. 01, pp. 1861–1870, 2022, doi: DOI : https://www.doi.org/10.56726/IRJMETS18127.
- V. V. Kumar, A. Sahoo, and F. W. Liou, “Cyber-enabled Product Lifecycle Management: A Multi-agent Framework,” Procedia Manuf., vol. 39, pp. 123–131, 2019, doi: 10.1016/j.promfg.2020.01.247.
- Rajesh Goyal, “THE ROLE OF BUSINESS ANALYSTS IN INFORMATION MANAGEMENT PROJECTS,” Int. J. Core Eng. Manag., vol. 6, no. 9, 2020.
- N. G. Singh, Abhinav Parashar A, “Streamlining Purchase Requisitions and Orders : A Guide to Effective Goods Receipt Management,” J. Emerg. Technol. Innov. Res., vol. 8, no. 5, 2021.
- V. K. Yarlagadda, “Cutting-edge developments in Robotics for Smart Warehousing and Logistics Optimization,” Robot. Xplore USA Tech Dig., vol. 1, no. 1, pp. 61–79, 2024.
- A. P. A. Singh, “Best Practices for Creating and Maintaining Material Master Data in Industrial Systems,” Int. J. Res. Anal. Rev., vol. 10, no. 1, pp. 112–119, 2023.
- S. Pandya, “Predictive Analytics in Smart Grids : Leveraging Machine Learning for Renewable Energy Sources,” Int. J. Curr. Eng. Technol., vol. 11, no. 6, pp. 677–683, 2021, doi: https://doi.org/10.14741/ijcet/v.11.6.12.
- P. K. and S. Arora, “the Impact of Machine Learning and Ai on Enhancing Risk-Based Identity Verification Processes,” Int. Res. J. Mod. Eng. Technol. Sci., vol. 06, no. 05, pp. 8246–8255, 2024, doi: 10.56726/IRJMETS57063.
- B. Boddu, “Serverless Databases Are the Future of Database Management,” https://jsaer.com/download/vol-6-iss-1-2019/JSAER2019-6-1-277-282.pdf, vol. 6, no. 1, p. 5, 2020.
- A. P. A. Singh, “STRATEGIC APPROACHES TO MATERIALS DATA COLLECTION AND INVENTORY MANAGEMENT,” Int. J. Bus. Quant. Econ. Appl. Manag. Res., vol. 7, no. 5, 2022.
- P. Khare, S. Arora, and S. Gupta, “Integration of Artificial Intelligence (AI) and Machine Learning (ML) into Product Roadmap Planning,” in 2024 First International Conference on Electronics, Communication and Signal Processing (ICECSP), IEEE, Aug. 2024, pp. 1–6. doi: 10.1109/ICECSP61809.2024.10698502.
- Muthuvel Raj Suyambu and Pawan Kumar Vishwakarma, “Improving grid reliability with grid-scale Battery Energy Storage Systems (BESS),” Int. J. Sci. Res. Arch., vol. 13, no. 1, pp. 776–789, Sep. 2024, doi: 10.30574/ijsra.2024.13.1.1694.
- P. Rajapaksha, R. Farahbakhsh, and N. Crespi, “BERT, XLNet or RoBERTa: The Best Transfer Learning Model to Detect Clickbaits,” IEEE Access, 2021, doi: 10.1109/ACCESS.2021.3128742.
- R. Tandon, “Face mask detection model based on deep CNN techniques using AWS,” Int. J. Eng. Res. Appl., vol. 13, no. 5, pp. 12–19, 2023, doi: 10.9790/9622-13051219.
- N. Abid, “Enhanced IoT Network Security with Machine Learning Techniques for Anomaly Detection and Classification,” Int. J. Curr. Eng. Technol., vol. 13, no. 6, pp. 536–544, 2023, doi: https://doi.org/10.14741/ijcet/v.13.6.5.
- S. Arora and A. Tewari, “AI-Driven Resilience : Enhancing Critical Infrastructure with Edge Computing,” Int. J. Curr. Eng. Technol., vol. 12, no. 2, pp. 151–157, 2022, doi: https://doi.org/10.14741/ijcet/v.12.2.9.
- M. Z. Hasan, R. Fink, M. R. Suyambu, M. K. Baskaran, D. James, and J. Gamboa, “Performance evaluation of energy efficient intelligent elevator controllers,” in IEEE International Conference on Electro Information Technology, 2015. doi: 10.1109/EIT.2015.7293320.
- R. Goyal, “THE ROLE OF REQUIREMENT GATHERING IN AGILE SOFTWARE DEVELOPMENT: STRATEGIES FOR SUCCESS AND CHALLENGES,” Int. J. Core Eng. Manag., vol. 6, no. 12, pp. 142–152, 2021.
- Prity Choudhary, Rahul Choudhary and S. Garaga, “Enhancing Training by Incorporating ChatGPT in Learning Modules: An Exploration of Benefits, Challenges, and Best Practices,” Int. J. Innov. Sci. Res. Technol., vol. 9, no. 11, 2024.
- K. Ullah et al., “Short-Term Load Forecasting: A Comprehensive Review and Simulation Study With CNN-LSTM Hybrids Approach,” IEEE Access, vol. 12, pp. 111858–111881, 2024, doi: 10.1109/ACCESS.2024.3440631.
- R. Tandon, “Prediction of Stock Market Trends Based on Large Language Models,” J. Emerg. Technol. Innov. Res., vol. 11, no. 9, pp. a615–a622, 2024.
- M. H. Hashmi, M. Affan, and R. Tandon, “A Customize Battery Management Approach for Satellite,” in Proceedings of the 2023 24th International Carpathian Control Conference, ICCC 2023, 2023. doi: 10.1109/ICCC57093.2023.10178893.
- Ramesh Bishukarma, “Privacy-preserving based encryption techniques for securing data in cloud computing environments,” Int. J. Sci. Res. Arch., vol. 9, no. 2, pp. 1014–1025, Aug. 2023, doi: 10.30574/ijsra.2023.9.2.0441.
- Sahil Arora and Apoorva Tewari, “Fortifying Critical Infrastructures: Secure Data Management with Edge Computing,” Int. J. Adv. Res. Sci. Commun. Technol., vol. 3, no. 2, pp. 946–955, Aug. 2023, doi: 10.48175/IJARSCT-12743E.
- S. Bauskar, “ADVANCED ENCRYPTION TECHNIQUES FOR ENHANCING DATA SECURITY IN CLOUD COMPUTING ENVIRONMENT,” Int. Res. J. Mod. Eng. Technol. Sci., vol. 05, no. 10, 2023, doi: 10.56726/IRJMETS45283.
- H. Sinha, “Benchmarking Predictive Performance Of Machine Learning Approaches For Accurate Prediction Of Boston House Prices : An In-Depth Analysis,” iternational J. Res. Anal. Rev., vol. 11, no. 3, 2024.
- M. R. S. and P. K. Vishwakarma, “THE ASSESSMENTS OF FINANCIAL RISK BASED ON RENEWABLE ENERGY INDUSTRY,” Int. Res. J. Mod. Eng. Technol. Sci., vol. 06, no. 09, pp. 758–770, 2024.
- H. Sinha, “Predicting Employee Performance in Business Environments Using Effective Machine Learning Models,” IJNRD - Int. J. Nov. Res. Dev., vol. 9, no. 9, pp. a875–a881, 2024.
- R. Bishukarma, “Optimising Cloud Security in Multi-Cloud Environments : A Study of Best Practices,” Int. Res. J., vol. 11, no. 11, pp. 590–598, 2024.
- H. Cam, A. V. Cam, U. Demirel, and S. Ahmed, “Sentiment analysis of financial Twitter posts on Twitter with the machine learning classifiers,” Heliyon, 2024, doi: 10.1016/j.heliyon.2023.e23784.
- T. Zahra and I. Hussain, “Sentiment Analysis of Twitter Dataset Using Lle and Classification Methods,” no. 01, pp. 1151–1164, 2021.
- K. Deepa, H. Sangita, and H. Shruthi, “Sentiment Analysis of Twitter Data Using Machine Learning,” Lect. Notes Electr. Eng., vol. 905, no. June, pp. 259–267, 2022, doi: 10.1007/978-981-19-2177-3_26.
Sentiment analysis is widely recognised as the
most actively researched area in data mining. These days,
a number of social media platforms have been created, and
Twitter is a crucial instrument for exchanging and
gathering people's thoughts, feelings, opinions, and
attitudes about specific things. This paper presents a
comparative analysis of Large Language Models (LLMs)
and ML models for sentiment analysis on a Twitter
dataset. The study evaluates a performance of XLNet,
advance algorithms including KNN, RF, and XGBoost.
The sentiment analysis methodology involves pre-
processing the Twitter dataset through noise removal and
tokenisation, followed by feature extraction using methods
like Bag-of-Words and Word2Vec. Results show that
XLNet is superior to the conventional models; It has 99%
precision, recall, and F1-score values and an accuracy rate
of 99.54%. In comparison, KNN achieves 78% accuracy,
85% precision, 88% recall, and 86% F1 score, while RF
and XGBoost exhibit lower performance with accuracy
rates of 69% and 60%, respectively. The performance
comparison highlights the superior capabilities of XLNet
for sentiment classification tasks, indicating its potential
for enhancing text classification applications. Research in
the future can look at ways to improve XLNet's
performance on bigger and more complicated datasets by
combining it with more sophisticated deep learning
methods like attention mechanisms and transfer learning.
Keywords :
Sentiment Analysis, LLM, Twitter Dataset, Natural Language Processing, Text Classification, Social Media Analytics.