ATLAS fellows pursue smarter assessment design

LAWRENCE — Two graduate researchers will explore new ways to develop assessments and deliver actionable test results through a research fellowship program at the University of Kansas. 

Each year, Accessible Teaching, Learning, and Assessment Systems (ATLAS), a center within the Achievement & Assessment Institute, sets research priorities for its fellowship program. The 2026–2027 priority topics are AI applications used within an operational assessment and literature syntheses on dashboard designs. 

Xiyu Wang and Wenjie Zhou were selected for the program. Wang is a fifth-year doctoral student studying educational psychology and research methodology at Purdue University. Zhou is a first-year doctoral student at the University of California, Berkeley studying social research methodology. 

Improving data dashboards for teachers 

Wang will conduct a systematic review of teacher-facing dashboard design in K–12 assessment contexts. This research will help psychometricians better understand how teachers interpret and use student performance data. 

“We know the student’s performance, but how can we make sure the teachers fully understand?” Wang said. “I think it's our responsibility to help the teachers know what the results mean.” 

Although existing research has examined dashboards across a range of users and contexts, there is little research on which design features support teacher interpretation and instructional decision-making. Wang will also investigate whether existing dashboard designs are aligned with established instructional design principles. 

Wang said she is excited to work with ATLAS researchers and help teachers turn assessment analytics into instructional decisions that support student learning.  

“ATLAS gave me this opportunity,” Wang said. “If I can contribute even a small but meaningful piece, that will mean a lot to me.” 

Streamlining test development 

Zhou will research the use of large language models (LLMs) to simulate student responses, allowing researchers to test assessments more rigorously before piloting them with actual students.  

Using real students in test development and research takes significant time and money. This can be especially challenging for researchers with limited funds and when developers need data from specific student populations. Zhou said his research could help reduce those barriers.  

“This framework can change the test development process,” Zhou said. “Right now, there is a big barrier in test development because you can write great questions, but you cannot use them until real students have tried them out. If this project works, it could help alleviate this bottleneck.”  

Zhou said existing off-the-shelf LLMs tend to produce responses reflecting high ability with narrow variability, underrepresenting low- and mid-performing students.  

He will be pushing the limits of LLMs in test development by creating AI bots with varied profiles of skills that better align with real student populations.  

“Instead of just telling the AI to pretend it represents students across skill and ability levels, I give it a detailed cognitive profile,” Zhou said. “For example, I might tell the AI that the student has mastered fractions but struggles with decimals.”  

Zhou said the AI bots are not meant to replace real students in the test development process but can hone the process and get higher quality tests in front of students. 

2025 fellows reach the finish line  

Wang and Zhou are preceded by the 2025 ATLAS fellows, Victoria Quirk and Pragati Maheshwary. 

Over the past year, Quirk investigated the differences between hierarchical models within computerized adaptive assessments. 

Quirk said the freedom and flexibility offered by the fellowship helped her ask the right questions and pivot as needed, setting her on the right path forward.