International Research Journal of Engineering and Technology (IRJET) Volume: 13 Issue: 01 | Jan 2026
www.irjet.net
e-ISSN: 2395-0056 p-ISSN: 2395-0072
From Human Judgment to AI Algorithms: A Literature-Based Analysis of Automated Essay Scoring Accuracy and Limitations Aaryan Karlapalem1 1Student pursuing Computer Science at Tomball Memorial High School, Tomball, Texas, USA ----------------------------------------------------------------------------------***-------------------------------------------------------------------------------
Abstract - Since its release in 2022, arti/icial intelligence (AI) has become a prominent part of most education systems, with nearly 20% of American schools using some form of AI in the classroom [3]. While most of these schools just use AI for lesson planning and personalized learning, however, one of the most labor-intensive tasks in education, grading essays, has also begun to shift toward automation. To reduce teacher workload and improve grading ef/iciency, teachers and state agencies such as the Texas Education Agency have been exploring AI-based tools for evaluating student writing. These tools, called automated essay scoring (AES) systems, use natural language processing (NLP) and machine learning to mimic human judgment in reviewing human writing. This paper examines the capabilities and limitations of modern AES technologies, comparing their grading accuracy, feedback quality, and overall fairness to traditional human grading. Using recent research, it explores whether AI is capable of permanently replacing human graders. Ultimately, it concludes that while current AES systems offer some bene/its in scalability and ef/iciency, they are not yet advanced enough to fully replace human judgment in high-stakes or nuanced essay evaluation contexts. Keywords: Artificial intelligence (AI), automated essay scoring (AES), Natural language processing (NLP)
1. Introduction During the COVID-19 pandemic, virtually every major industry experienced huge impacts because of the global outbreak, and the education industry was no exception to that, with thousands of schools worldwide being forced to switch to virtual school and video calls to replace physical schools and classes. During this time, the burden put on teachers was immense, with teachers having to juggle their personal health as well as their duty to provide a quality learning experience to students despite the pandemic. This quickly resulted in teachers mass-quitting their jobs because of increased stress and fatigue. This continues to be a major problem even today, with a teacher shortage being marked as a serious community problem. Schools and universities globally have been struggling to hire qualiGied teachers, a problem that has only become worse because of the global pandemic. Furthermore, young people today have also been showing less interest in pursuing a career in education, with Millennials and Generation Z being the least interested in becoming future teachers. All these factors together intensi Gied the overall problem and the need to accelerate ways to bring in more teachers. Additionally, according to a study done by Merrimack College in 2022, it was shown that from 2012 to 2022, the percentage of K-12 teachers who were “very satisGied” with their jobs fell from 39% to 12%, the lowest this statistic has ever been. Additionally, more teachers have reported that they want to leave their jobs to pursue a career outside of education, further showing the growing disinterest in teaching.
2. The Rise of Arti Bicial Intelligence in Education Ever since the early 2020s, artiGicial intelligence has been experiencing rapid adoption among various education systems around the world, and for good reason. Ever since the global COVID-19 pandemic, teachers have been under more stress than ever, and they desperately need to streamline their work Glows to keep up with the increasing amount of work they have. Thus, AI technologies presented themselves as the solution to this, providing tools to create lesson plans, slideshows, and even grading to an extent. Initially, these AI services were simply being used to automate parts of a teacher’s job, such as the generation of lesson materials and assessments. However, with the recent rise of various LLMs (large language models) that specialize in certain tasks, such as GPT-3.5, GPT-4o mini, Gemini 2.5 Glash, and more, the extent to which AI is being utilized in the classroom has only been increasing. Today, these models are being tested and deployed for more complex tasks, like essay evaluation and feedback, grading, and even tutoring to some extent. These advancements bring about a shift from simply helping teachers with their jobs to potentially trying to replace them, especially in areas like writing assessment that are time-intensive for teachers. One of the emerging concepts tied to this shift is precision education, which aims to use artiGicial intelligence to customize instruction and grading to each student’s needs [5]. This kind of system integrates deep learning, transfer learning, and learning analytics to help students personally. Within this new framework, automated essay scoring (AES) systems are often used to not
© 2026, IRJET
|
Impact Factor value: 8.315
|
ISO 9001:2008 Certified Journal
|
Page 239