The black box of AI assessment

Summary

A vendor’s validity review recognizes that stable scores can still reward the wrong learning.

Christodoulou and Wheadon distinguish reliability from validity, discussing human agreement, repeated-assessment correlations and developmental patterns. They retain human reading and judgment.

They identify unfinished work on unusual responses, external tests, adversarial gaming and instructional washback. Their inability to find a gaming strategy is not equivalent to systematic adversarial validation.

Vendor validity argument; raw data, bias and adversarial robustness were not independently tested.