Authors : Shatrunjay Kumar Singh

Volume/Issue : Volume 10 - 2025, Issue 11 - November

Google Scholar : https://tinyurl.com/mr3w3ya2

Scribd : https://tinyurl.com/5t33prmz

DOI : https://doi.org/10.38124/ijisrt/25nov1315

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Note : Google Scholar may take 30 to 40 days to display the article.

Abstract : Large Language Models (LLMs) bring revolutionary changes to legal practice yet their ability to produce fake legal information through hallucination remains a major obstacle. The current evaluation methods for legal hallucinations fail to meet the needs of the legal field because they measure factual errors without considering the severe consequences of legal mistakes (Liang, 2024). The paper establishes a vital knowledge gap through its introduction of Risk-Weighted Hallucination Score (RWHS) as a new evaluation method. The trustworthiness of AI-generated legal answers requires more than error volume because legal risk assessment needs to evaluate the severity of hallucinations based on their ability to lead to malpractice or procedural failures or legal injustices. The research establishes a systematic classification system which ranks legal hallucinations based on their consequences from severe to insignificant and creates a method to evaluate them (Chen, 2023) (Clapp, 2022). The framework enables AI developers to focus on fixing critical system failures while legal professionals can use it to validate AI outputs and maintain their technological competence and policymakers can create effective standards and oversight systems (Cohen, 2022). The paper creates a fundamental framework which enables developers to create artificial intelligence systems for legal work that are dependable and ethical and responsible. The paper introduces a new method to assess artificial intelligence systems in law by evaluating their actual impact instead of their accuracy rate.

Keywords : Large Language Models (LLMs); AI Hallucination; Legal Technology; Legal Risk Management; AI Evaluation; Computational Law; Legal Ethics; Responsible AI.

References :

1. Ahmadi, A. (2024). Unravelling the Mysteries of Hallucination in Large Language Models: Strategies for Precision in Artificial Intelligence Language Generation. Asian Journal of Computer Science and Technology, 13(1), 1–10. https://doi.org/10.70112/ajcst-2024.13.1.4144.
2. Liu, X. (2024). A Survey of Hallucination Problems Based on Large Language Models. Applied and Computational Engineering, 97(1), 24–30. https://doi.org/10.54254/2755-2721/2024.17851.
3. Aditya, G. (2024). Understanding and Addressing AI Hallucinations in Healthcare and Life Sciences. International Journal of Health Sciences, 7(3), 1–11. https://doi.org/10.47941/ijhs.1862.
4. Dahl, M., Magesh, V., Ho, D. E., & Suzgun, M. (2024). Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models. Journal of Legal Analysis, 16(1), 64–93. https://doi.org/10.1093/jla/laae003.
5. Mather, L., Maiman, R. J., & Mcewen, C. A. (2001). Divorce Lawyers at Work. Oxford University New York Ny. https://doi.org/10.1093/oso/9780195145151.001.0001.
6. Kerikmäe, T., Hoffmann, T., & Chochia, A. (2018). Legal Technology for Law Firms: Determining Roadmaps for Innovation. Croatian International Relations Review, 24(81), 91–112. https://doi.org/10.2478/cirr-2018-0005.
7. Shapovalov, V. (2023). Medical Errors in Health Care Institutions: an Interdisciplinary Study of the Competences of Specialists Based on Medical and Pharmaceutical Law. SSP Modern Law and Practice, 3(4), 1–14. https://doi.org/10.53933/sspmlp.v3i4.121.
8. Susskind, R. (2008). The End of Lawyers? Oxford University Pressoxford. https://doi.org/10.1093/oso/9780199541720.001.0001.
9. Berberette, E., Hutchins, J., & Sadovnik, A. (2024). Redefining “Hallucination” in LLMs: Towards a psychology-informed framework for mitigating misinformation. https://doi.org/10.48550/arxiv.2402.01769.
10. Huang, L., Yu, W., Liu, T., Ma, W., Qin, B., Feng, X., Feng, Z., Wang, H., Peng, W., Chen, Q., & Zhong, W. (2025). A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Transactions on Information Systems, 43(2), 1–55. https://doi.org/10.1145/3703155.
11. Tonmoy, S., Zaman, S., Jain, V., Rani, A., Rawte, V., Chadha, A., & Das, A. (2024). A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models. https://doi.org/10.48550/arxiv.2401.01313.
12. Kojola, E. (2018). Indigeneity, gender and class in decision-making about risks from resource extraction. Environmental Sociology, 5(2), 130–148. https://doi.org/10.1080/23251042.2018.1426090.
13. Tyler, T. R., & Bies, R. J. (2015). Beyond Formal Procedures: The Interpersonal Context of Procedural Justice (pp. 77–98). Psychology. https://doi.org/10.4324/9781315728377-4.
14. Djankov, S., Porta, R. L., Lopez-De-Silane, F., & Shleifer, A. (2002). Courts: the Lex Mundi Project. National Bureau of Economic Research. https://doi.org/10.3386/w8890.
15. Leiter, B. (2010). LEGAL FORMALISM AND LEGAL REALISM: WHAT IS THE ISSUE? Legal Theory, 16(2), 111–133. https://doi.org/10.1017/s1352325210000121.
16. Larøi, F. (2006). The Phenomenological Diversity of Hallucinations: Some theoretical and clinical implications. Psychologica Belgica, 46(1–2), 163. https://doi.org/10.5334/pb-46-1-2-163.
17. Ffytche, D. H. (2009). Visual hallucinations in eye disease. Current Opinion in Neurology, 22(1), 28–35. https://doi.org/10.1097/wco.0b013e32831f1b3f.
18. Biddle, L., Wahedi, K., & Bozorgmehr, K. (2020). Health system resilience: a literature review of empirical research. Health Policy and Planning, 35(8), 1084–1109. https://doi.org/10.1093/heapol/czaa032.
19. Copeland, S., Hinrichs-Krapels, S., Fecondo, F., Santizo, E. R., Bal, R., & Comes, T. (2023). A resilience view on health system resilience: a scoping review of empirical studies and reviews. BMC Health Services Research, 23(1). https://doi.org/10.1186/s12913-023-10022-8.
20. Paulsen, J. S., Weisstein–Jenkins, C., Romero, R., Salmon, D. P., Jeste, D. V., Galasko, D., Hofstetter, C. R., Thomas, R., Grant, I., & Thal, L. J. (2000). Incidence of and risk factors for hallucinations and delusions in patients with probable AD. Neurology, 54(10), 1965–1971. https://doi.org/10.1212/wnl.54.10.1965.
21. Berrios, G. E. (1982). Tactile hallucinations: conceptual and historical aspects. Journal of Neurology, Neurosurgery & Psychiatry, 45(4), 285–293. https://doi.org/10.1136/jnnp.45.4.285.
22. Vercammen, A., & Aleman, A. (2008). Semantic Expectations Can Induce False Perceptions in Hallucination-Prone Individuals. Schizophrenia Bulletin, 36(1), 151–156. https://doi.org/10.1093/schbul/sbn063.
23. Larøi, F., Fernyhough, C., Jenkins, J., Deshpande, S., Bell, V., Christian, W. A., Luhrmann, T. M., & Woods, A. (2014). Culture and hallucinations: overview and future directions. Schizophrenia Bulletin, Suppl 40 4(Suppl 4), S213–S220. https://doi.org/10.1093/schbul/sbu012.
24. Bai, Z., Wang, P., Xiao, T., He, T., Han, Z., Zhang, Z., & Shou, M. (2024). Hallucination of Multimodal Large Language Models: A Survey. https://doi.org/10.48550/arxiv.2404.18930.
25. Constantinides, M., Muller, M., Wilcox, L., Madaio, M., Baeza-Yates, R., Cramer, H., Vitak, J., Stumpf, S., Blumenfeld, I. G., Kennedy, S., Bogucka, E. P., Holbrook, J., Luger, E., Pistilli, G., Quercia, D., & Tahaei, M. (2024). Implications of Regulations on the Use of AI and Generative AI for Human-Centered Responsible Artificial Intelligence. 1–4. https://doi.org/10.1145/3613905.3643979.
26. Butt, J. S. (2024). Analytical Study of the World’s First EU Artificial Intelligence (AI) Act, 2024. International Journal of Research Publication and Reviews, 5(3), 7343–7364. https://doi.org/10.55248/gengpi.5.0324.0914.
27. Dokumacı, M. (2024). Legal Frameworks for AI Regulations. Human Computer Interaction, 8(1), 133. https://doi.org/10.62802/ytst2927.
28. Lakkshmanan, A., Seranmadevi, R., Tyagi, A. K., & Sree, P. H. (2024). Engineering Applications of Artificial Intelligence (pp. 166–179). Igi Global. https://doi.org/10.4018/979-8-3693-5261-8.ch010.
29. Mariani, M. M., Machado, I., Magrelli, V., & Dwivedi, Y. K. (2022). Artificial intelligence in innovation research: A systematic review, conceptual framework, and future research directions. Technovation, 122, 102623. https://doi.org/10.1016/j.technovation.2022.102623.
30. Benneh Mensah, G., & Dutta, P. K. (2024). Evaluating if Ghana’s Health Institutions and Facilities Act 2011 (Act 829) Sufficiently Addresses Medical Negligence Risks from Integration of Artificial Intelligence Systems. Mesopotamian Journal of Artificial Intelligence in Healthcare, 2024, 35–41. https://doi.org/10.58496/mjaih/2024/006.
31. Franceschelli, G., & Musolesi, M. (2024). Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges. Journal of Artificial Intelligence Research, 79, 417–446. https://doi.org/10.1613/jair.1.15278.
32. Vargas-Murillo, A. R., Delgado-Chávez, C. A., Sanchez-Paucar, F., Turriate-Guzman, A. M., & Pari-Bedoya, I. (2024). Transforming Justice: Implications of Artificial Intelligence in Legal Systems. Academic Journal of Interdisciplinary Studies, 13(2), 433. https://doi.org/10.36941/ajis-2024-0059.
33. Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C., & Ho, D. (2024). Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. https://doi.org/10.48550/arxiv.2405.20362.
34. Rodrigues, R. (2020). Legal and human rights issues of AI: Gaps, challenges and vulnerabilities. Journal of Responsible Technology, 4, 100005. https://doi.org/10.1016/j.jrt.2020.100005.
35. Athaluri, S. A., Dave, T., Kesapragada, V. S. R. K. M., Yarlagadda, V., Manthena, S. V., & Duddumpudi, R. T. S. (2023). Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References. Cureus, 15(4). https://doi.org/10.7759/cureus.37432.
36. Zakir, M. H., Ali, R. N., Bashir, S., & Khan, S. H. (2024). Artificial Intelligence and Machine Learning in Legal Research: A Comprehensive Analysis. Qlantic Journal of Social Sciences, 5(1), 307–317. https://doi.org/10.55737/qjss.203679344.
37. Quteishat, E. (2024). Exploring the Role of AI in Modern Legal Practice: Opportunities, Challenges, and Ethical Implications. Journal of Electrical Systems, 20(6s), 3040–3050. https://doi.org/10.52783/jes.3320.
38. Liang, X., Niu, S., Tang, B., Deng, H., Li, Z., He, D., Wang, Z., Peng, C., Song, S., Xiong, F., & Wang, Y. (2024). UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation. 5266–5293. https://doi.org/10.18653/v1/2024.acl-long.288.
39. Chen, Y., Zhang, D., Fan, G., Yuan, Y., Liu, D., Xiao, Y., Wen, Z., Li, Z., & Fu, Q. (2023). Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models. 35, 245–255. https://doi.org/10.1145/3583780.3614905.
40. Clapp, M. A., Kim, E., James, K. E., Perlis, R. H., Kaimal, A. J., Mccoy, T. H., & Easter, S. R. (2022). Comparison of Natural Language Processing of Clinical Notes with a Validated Risk-Stratification Tool to Predict Severe Maternal Morbidity. JAMA Network Open, 5(10), e2234924. https://doi.org/10.1001/jamanetworkopen.2022.34924.
41. Cohen, J., Klatt, T. W., Wright-Berryman, J., Daniel, L., Rohlfs, L., & Trocinski, D. (2022). Integration and Validation of a Natural Language Processing Machine Learning Suicide Risk Prediction Model Based on Open-Ended Interview Language in the Emergency Department. Frontiers in Digital Health, 4. https://doi.org/10.3389/fdgth.2022.818705.