W. Jake Thompson


Jake Thompson
  • Assistant Director of Psychometrics

Biography —

Jake Thompson, PhD, has worked in large-scale operational assessments since 2014, bridging the gap between psychometrics and assessment use. His research focuses on the implementation of diagnostic classification models in applied settings to improve instructional decision-making and student outcomes. He has served as principal investigator on IES-funded grants to develop software to support of the use of diagnostic models in applied settings (R305D210045, R305D240032) and has collaborated on other OSEP-funded grants to implement diagnostic models to understand student learning and provide actionable feedback to educators (S368A170009, S368A220019).

Thompson is the lead psychometrician for the Dynamic Learning Maps® Alternate Assessment System, supporting the operational delivery of assessments in 25 states and implementing a research agenda to support and evaluate the intended uses of the results. He supports the year-end and instructionally-embedded assessments, both of which have met US Department of Education assessment peer review requirements.

Thompson’s research interests also include Bayesian statistics and data visualization methods for effective communication. He has co-authored more than 100 journal articles, book chapters, technical reports, conference presentations, and software packages.

Education —

Ph.D. in Research, Evaluation, Measurement, and Statistics, University of Kansas

Selected Publications —

Publications

Thompson, W. J. (2025). Evaluating methods for assessing model fit in diagnostic classification models. In J.-S. Kim, H. Wu, T. Sweet, D. Molenaar, B. Junker, I. Moustaki, J. Harring, O. Bulut, X. Tong, G. Wallin, & S. Di Plinio (Ed.) Proceedings of the International Meeting of the Psychometric Society: The 89th Annual Meeting, Prague, Czech Republic, 2024. https://doi.org/10.64028/rgtk456752

Swinburne, R. R., Schuster, J., Karvonen, M., Thompson, W. J., Erickson, K., Simmering, V., & Bechard, S. (2025). Learning maps as cognitive models for instruction and assessment. Education Sciences, 15(3), Article 365. https://doi.org/10.3390/educsci15030365

Clark, A. K., Hirt, A., Whitcomb, D., Thompson, W. J., Wine, M., & Karvonen, M. (2025). Artificial intelligence in science and mathematics assessment for students with disabilities: Opportunities and challenges. Education Sciences, 15(2), Article 233. https://doi.org/10.3390/educsci15020233

Thompson, W. J. & Clark, A. K. (2024). Improving instructional decision-making using diagnostic classification models. Educational Measurement: Issues and Practice, 43(4), 146–156. https://doi.org/10.1111/emip.12619 [Preprint]

Thompson, W. J. (2023). measr: Bayesian psychometric measurement using Stan. Journal of Open Source Software, 8(91), Article 5742. https://doi.org/10.21105/joss.05742

Thompson, W. J. (2023). Visualizing distributions across grades with ridgeline plots [Cover graphic]. Educational Measurement: Issues and Practice, 42(2). https://doi.org/10.1111/emip.12514Thompson, W. J., Nash, B., Clark, A. K., & Hoover, J. C. (2023). Using simulated retests to estimate the reliability of diagnostic assessment systems. Journal of Educational Measurement. [Preprint]

Kobrin, J. L., Karvonen, M., Clark, A. K., & Thompson, W. J. (2022). Developing and refining a model for measuring implementation fidelity for an instructionally embedded assessment system. Practical Assessment, Research, and Evaluation, 27(1), Article 24.

Thompson, W. J. (2022). Gibbs sampler. In B. B. Frey (Ed.) The SAGE encyclopedia of research design (2nd ed., pp. 621– 622). SAGE.

Thompson, W. J. & Nash, B. (2022). A diagnostic framework for the empirical evaluation of learning maps. Frontiers in Education, 6, 714736.

Karvonen, M., Kingston, N. M., Wehmeyer, M. L., & Thompson, W. J. (2020). New approaches to designing and administering inclusive assessments. Oxford research encyclopedia of education (pp. 1–23). Oxford University Press.

Thompson, W. J., Clark, A. K., & Nash, B. (2019). Measuring the reliability of diagnostic mastery classifications at multiple levels of reporting. Applied Measurement in Education, 32(4), 298–309. [Preprint]

Thompson, W. J. (2018). Construct irrelevance. In B. B. Frey (Ed.) The SAGE encyclopedia of educational research, measurement, and evaluation (pp. 375–376). SAGE.

Thompson, W. J. (2018). Evaluating model estimation processes for diagnostic classification models (Publication No. 10785604) [Doctoral dissertation, University of Kansas]. ProQuest Dissertations and Theses Global.

Chrysikou, E. G. & Thompson, W. J. (2016). Assessing cognitive and affective empathy through the Interpersonal Reactivity Index: An argument against a two-factor model. Assessment, 23(6), 769–777.

Technical Reports

Thompson, W. J. (2025). Using multilevel regression and poststratification to estimate reliability for student groups (Technical Report No. 25-01). University of Kansas; Accessible Teaching, Learning, and Assessment Systems. [PDF]

Accessible Teaching, Learning, and Assessment Systems (2021). 2020–2021 DLM administration during COVID-19: Participation, performance, and educational experience (Technical Report No. 21-02). University of Kansas. [PDF]

Thompson, W. J. & Hoover, J. C. (2021). Using propensity scores to evaluate changes in cross-year performance distributions (pdf) (Technical Report No. 21-01). University of Kansas; Accessible Teaching, Learning, and Assessment Systems.

Thompson, W. J. (2020). Reliability for the Dynamic Learning Maps assessments: A comparison of methods (pdf) (Technical Report No. 20-03). University of Kansas; Accessible Teaching, Learning, and Assessment Systems.

Thompson, W. J. (2019). Bayesian psychometrics for diagnostic assessments: A proof of concept (Research Report No. 19- 01). University of Kansas; Accessible Teaching, Learning, and Assessment Systems.

Clark, A. K., Thompson, W. J., & Karvonen, M. (2019). Instructionally embedded assessment: Patterns of use and outcomes (pdf) (Technical Report No. 19-01). University of Kansas; Accessible Teaching, Learning, and Assessment Systems.

Thompson, W. J. (2018). Assessing model fit for the Dynamic Learning Maps alternate assessment using a Bayesian estimation (pdf) (Technical Report No. 18-01). University of Kansas; Accessible Teaching, Learning, and Assessment Systems.

R Packages

Thompson, W. J., Pablo, N., & Hoover, J. (2023). ratlas: ATLAS formatting functions and templates. R package version 0.0.0.9000.

Thompson, W. J. (2023). taylor: Lyrics and song data for Taylor Swift’s discography. R package version 2.0.1.9000.

Thompson, W. J. (2023). measr: Bayesian psychometric measurement using ‘Stan’. R package version 0.2.1.9000.

Hoover, J. & Thompson, W. J. (2023).dcm2: Calculating the M2 model fit statistic for diagnostic classification models. R package version 1.0.2.

Hoover, J. & Thompson, W. J. (2023). tdcmStan: Automating the creation of Stan code for TDCMs. R package version 2.0.0.

Selected Presentations —

Nash, B. & Thompson, W. J. (2026, September 27–October 2). Through-year assessment as a summative tool: Proof-of-concept evidence from the Pathways for Instructionally Embedded Assessment (PIE) project [Conference session]. International Association for Educational Assessment Annual Conference, Toronto, Canada.

Thompson, W. J. (2026, September 27–October 2). Using diagnostic classification models to improve instructional decision-making [Conference session]. International Association for Educational Assessment Annual Conference, Toronto, Canada.

Thompson, W. J. (2026, September 27–October 2). Open and transparent scoring with measr: An R package for diagnostic classification models [Conference session]. International Association for Educational Assessment Annual Conference, Toronto, Canada.

Jimenez, A., Thompson, W. J., & Nash, B. (2026, April 8–11). Aggregating instructionally embedded assessment data using beta IRT models [Paper presentation]. National Council on Measurement in Education Annual Meeting, Los Angeles, CA.

Nash, B., Thompson, W. J., & Majerus, M. (2026, April 8–11). Instructionally embedded assessments to meet instructional and summative uses: Evidence from a pilot study [Paper presentation]. National Council on Measurement in Education Annual Meeting, Los Angeles, CA.

Thompson, W. J. (2026, April 8–11). Estimating and evaluating diagnostic models with the R package measr [Innovation demonstration]. National Council on Measurement in Education Annual Meeting, Los Angeles, CA. [Slides]

Thompson, W. J. & Nash, B. (2026, April 8–11). Creating summative scores from instructionally embedded results. In K. McClure (Chair), Combining through-year scores: Operational approaches, challenges, and practical implications [Symposium]. National Council on Measurement in Education Annual Meeting, Los Angeles, CA. [Slides]

Thompson, W. J. (2026, January 3–7). Using diagnostic classification models to improve instructional decision-making [Conference session]. IAFOR International Conference on Education in Hawaii, Honolulu, HI. [PDF / Slides]

Thompson, W. J., Nash, B., & Bates, S. (2025, June 23–25). Pathways for instructionally embedded assessment (PIE) proof of concept: Potential future summative uses of an instructionally embedded assessment model [Symposium]. National Conference on Student Assessment, Denver, CO. [Slides]

Jimenez, A., Thompson, W. J., & Nash, B. (2025, April 23–26). Evaluating learning progression hierarchies with diagnostic models [Paper presentation]. National Council on Measurement in Education Annual Meeting, Denver, CO. [Slides]

Nash, B., Thompson, W. J., Jimenez, A., Majerus, M., & Bates, S. (2025, April 23–26). Patterns of use from an instructionally embedded assessment pilot study [Paper presentation]. National Council on Measurement in Education Annual Meeting, Denver, CO. [Slides]

Thompson, W. J. (2025, April 23–26). Modeling attribute relationships in diagnostic models with the R package measr [Innovation demonstration]. National Council on Measurement in Education Annual Meeting, Denver, CO. [Slides]

Thompson, W. J. (2025, April 23–26). Applications of diagnostic models to learning progressions to improve student learning. In S. Wang (Chair), Bridging theory and practice: Fostering innovation in diagnostic measurement to enhance education [Symposium]. National Council on Measurement in Education Annual Meeting, Denver, CO. [Slides]

Thompson, W. J. (2024, July 15–19). Evaluating methods for assessing model fit in diagnostic classification models [Paper presentation]. International Meeting of the Psychometric Society, Prague, Czech Republic. [PDF / Slides]

Thompson, W. J. (2024, July 8–11). Diagnostic modeling for educational and psychological assessment [Conference session]. useR!, Salzburg, Austria. [Slides]

Hoover, J. C. & Thompson, W. J. (2024, June 24–26). Evaluating Bayesian transition diagnostic classification models for reporting within-year progress [Conference session]. Modern Modeling Methods, University of Connecticut, Storrs, CT.

Invited discussion
Thompson, W. J. (2024, April 11–14). Invited discussion. In S. Wang (Chair), Reconceptualizing diagnostic classification models: Applications and new developments [Symposium]. National Council on Measurement in Education Annual Meeting, Philadelphia, PA. [Slides]

Thompson, W. J., Nash, B., & Hoover, J. C. (2023, September 6–7). Using diagnostic models to evaluate student learning hierarchies in a large-scale assessment [Conference session]. Frontier Research in Educational Measurement, Oslo, Norway. [PDF / Slides]

Thompson, W. J. & Clark, A. K. (2023, April 12–15). A simulated retest method for estimating classification reliability. In Y. Bao, M. Madison, & Q. Pan (Chair), Diagnostic measurement: Operational and implementational issues [Symposium]. National Council on Measurement in Education Annual Meeting, Chicago, IL. [Slides]

Thompson, W. J. (2023, March 28–30). Applied diagnostic classification modeling with the R package measr [Paper presentation]. National Council on Measurement in Education Annual Meeting, Virtual Sessions. [Slides]

Clark, A. K., Thompson, W. J., & Kobrin, J. (2022, April 22–25). Visualizing validity evidence: Considering strength of evidence following disrupted administration [Paper presentation]. National Council on Measurement in Education Annual Meeting, San Diego, CA.

Hoover, J. C. & Thompson, W. J. (2022, April 22–25). Modifying the M2 statistic to handle missing data [Paper presentation]. National Council on Measurement in Education Annual Meeting, San Diego, CA.

Kobrin, J., Thompson, W. J., Wang, W., & Hoover, J. C. (2022, April 22–25). Development and evaluation of a composite item-fit statistic for diagnostic classification models [Paper presentation]. National Council on Measurement in Education Annual Meeting, San Diego, CA.

Hoover, J. C., Thompson, W. J., Nash, B., & Kobrin, J. (2021, June 8–11). The I-SMART project: Empirical map validation (pdf)[Paper presentation]. National Council on Measurement in Education Annual Meeting, Virtual Conference.

Thompson, W. J., Clark, A. K., & Nash, B. (2021, June 8–11). Technical evidence for diagnostic assessments. In W. J. Thompson (Chair), Diagnostic assessments: Moving from theory to practice (pdf)[Symposium]. National Council on Measurement in Education Annual Meeting, Virtual Conference. [Slides]

Thompson, W. J. & Pablo, N. (2020, January 29–30). Branding and packaging reports with R Markdown [Conference session]. rstudio::conf(2020), San Francisco, CA.

Thompson, W. J. & Nash, B. (2019, April 4–8). Empirical methods for evaluating maps: Illustrations and results. In M. Karvonen (Chair), Beyond learning progressions: Maps as assessment architecture (pdf) [Symposium]. National Council on Measurement in Education Annual Meeting, Toronto, Canada. [Slides]

Brussow, J. A., Skorupski, W. P., & Thompson, W. J. (2018, April 12–16). A hierarchical IRT model for identifying group-level aberrant growth [Paper presentation]. National Council on Measurement in Education Annual Meeting, New York, NY.

Nash, B., Clark, A. K., & Thompson, W. J. (2018, April 12–16). Using simulation to evaluate retest reliability of assessment results (pdf) [Paper presentation]. National Council on Measurement in Education Annual Meeting, New York, NY.

Thompson, W. J., Clark, A. K., & Nash, B. (2018, April 12–16). Measuring the reliability of student mastery classifications at multiple levels of reporting (pdf) [Paper presentation]. National Council on Measurement in Education Annual Meeting, New York, NY. [Slides]

Nash, B. & Thompson, W. J. (2017, April 26–30). Evaluating an initialization tool for student placement into a map-based assessment (pdf) [Paper presentation]. National Council on Measurement in Education Annual Meeting, San Antonio, TX.

Awards & Honors —

  • AERA Division H Outstanding Publication Award for Advances in Methodology (2023)
  • Educational Measurement: Issues and Practice Cover Showcase Winner (2023)
  • AERA Inclusion and Accessibility in Educational Assessment SIG Annual Award (2022)
  • Educational Measurement: Issues and Practice Cover Showcase Winner (2020)
  • Educational Measurement: Issues and Practice Cover Showcase Winner (2017)
  • Educational Measurement: Issues and Practice Cover Showcase Top 10 (2016)
  • Chancellor’s Doctoral Fellowship (2014–2018)

Grants & Other Funded Activity —

Currently Funded Projects

Principal Investigator: Improving Software and Methods for Estimating and Evaluating Diagnostic Classification Models (2021–2023). USED, Institute of Education Sciences; $225,000.

Co-Principal Investigator: Dynamic Learning Maps (DLM) Alternate Assessment System. Ongoing state contracts. PI: Meagan Karvonen.

Other Personnel (Psychometrician): Pathways for Instructionally Embedded Assessment (PIE) (2022–2026). USED, Office of Elementary and Secondary Education, Office of School Support and Accountability; $2,500,000. PI: Brooke Nash.

Previously Funded Projects

Other Personnel (Psychometrician): Innovations in Science Map, Assessment, and Reporting Technology (I-SMART) (2016– 2020). USED, Office of Elementary and Secondary Education. PI: Meagan Karvonen.

Unfunded Projects

Principal Investigator: Improving Software and Methods for Estimating and Evaluating Diagnostic Classification Models (2020–2022). USED, Institute of Education Sciences; $225,000.

Memberships —

Professional Affiliations

  • American Educational Research Association
  • American Statistical Association
  • National Council on Measurement in Education

Reviewer

  • Behaviormetrika
  • International Journal of Research in Education and Science
  • Journal of Educational Measurement
  • Journal of Open Source Software
  • National Council on Measurement in Education annual meeting