Playing Pretend: Expert Personas Don’t Improve Factual Accuracy

Summary

This report tests whether assigning expert identities reliably improves answers to difficult factual questions.

Across models and benchmarks, expert personas produce no dependable general gain, and some mismatched or low-knowledge personas worsen performance.

For AI literacy, distinguish a role that shapes perspective or a simulation from evidence of factual competence. The report explicitly leaves room for useful non-accuracy purposes of role prompting.

Tests factual-question accuracy, not the full value of roleplay, writing style or pedagogical dialogue.