TY - JOUR
T1 - Generative AI in graduate bioprocess engineering exams
T2 - Is attention all students need?
AU - Xiao, Zhengyang
AU - Lee, Eunseo
AU - Yuan, Sophia
AU - Ding, Roland
AU - Tang, Yinjie J.
N1 - Publisher Copyright:
© 2025 The Authors
PY - 2025/7
Y1 - 2025/7
N2 - State-of-the-art large language models (LLMs) can now answer conceptual textbook questions with near-perfect accuracy and perform complex equation derivations, raising significant concerns for higher education. This study evaluates the performance of LLMs on graduate-level bioprocess engineering exams, which include multiple-choice, short-answer, and long-form questions requiring calculations. First, allowing students to use LLMs led to a 36 % average score increase compared to exams taken with only textbooks and notes. Second, as students gained more experience using LLMs, their performance improved further, particularly among students with disabilities. Third, under optimized conditions on two exams, OpenAI's GPT-4o scored approximately 70 out of 100, while more advanced models, such as OpenAI o1, o3, GPT-4.5, Qwen3–235B-A22B, and DeepSeek R1, scored above 84, outperforming 96 % of human test-takers. This indicates that students with access to more capable AI tools may gain an unfair advantage. Fourth, we propose guidelines for developing exam questions that are less susceptible to LLM-generated solutions. These include tasks such as interpreting graphical biological pathways, answering negatively worded conceptual questions, performing complex numerical calculations and optimizations, and solving open-ended research problems that demand critical thinking. This article calls for urgent reforms to bioprocess engineering education, advocating for the integration of LLM literacy through hands-on activities that address both practical applications and ethical considerations.
AB - State-of-the-art large language models (LLMs) can now answer conceptual textbook questions with near-perfect accuracy and perform complex equation derivations, raising significant concerns for higher education. This study evaluates the performance of LLMs on graduate-level bioprocess engineering exams, which include multiple-choice, short-answer, and long-form questions requiring calculations. First, allowing students to use LLMs led to a 36 % average score increase compared to exams taken with only textbooks and notes. Second, as students gained more experience using LLMs, their performance improved further, particularly among students with disabilities. Third, under optimized conditions on two exams, OpenAI's GPT-4o scored approximately 70 out of 100, while more advanced models, such as OpenAI o1, o3, GPT-4.5, Qwen3–235B-A22B, and DeepSeek R1, scored above 84, outperforming 96 % of human test-takers. This indicates that students with access to more capable AI tools may gain an unfair advantage. Fourth, we propose guidelines for developing exam questions that are less susceptible to LLM-generated solutions. These include tasks such as interpreting graphical biological pathways, answering negatively worded conceptual questions, performing complex numerical calculations and optimizations, and solving open-ended research problems that demand critical thinking. This article calls for urgent reforms to bioprocess engineering education, advocating for the integration of LLM literacy through hands-on activities that address both practical applications and ethical considerations.
KW - AI
KW - Bioprocess
KW - DeepSeek
KW - LLM
KW - OpenAI
KW - Pathway
UR - https://www.scopus.com/pages/publications/105007441460
U2 - 10.1016/j.ece.2025.05.006
DO - 10.1016/j.ece.2025.05.006
M3 - Article
AN - SCOPUS:105007441460
SN - 1749-7728
VL - 52
SP - 133
EP - 140
JO - Education for Chemical Engineers
JF - Education for Chemical Engineers
ER -