Debiased Inference for AI-Generated Data without Gold-Standard Labels: Identification via Multiple Imperfect Measurements

Summary

Researchers increasingly use AI to measure variables for downstream analyses, but ignoring prediction errors—even with accuracy above 90%—can cause substantial bias and invalid confidence intervals. Existing remedies like design-based supervised learning and prediction-powered inference rely on costly gold-standard labels. This paper introduces debiased inference with multiple imperfect measurements (DMM), a framework that leverages several error-prone AI measurements to achieve valid inference without gold-standard labels. Building on CP decomposition, DMM assumes measurements are independent conditional on the latent true label and unit-level features, allowing misclassification rates to vary across annotators and units. Semiparametric theory proves the DMM estimator is consistent and asymptotically normal. Simulations show valid inference and efficiency gains from adding imperfect measurements. Diagnostics for the conditional independence assumption are also developed.