PRISM: Prototype-Rectified Iterative Self-Supervised Manifold Denoising

Summary

Audio-Text Foundation Models (ATMs) are prone to catastrophic failure under severe acoustic noise, but existing adaptation methods either reinforce noise instead of signal or require privileged noise annotations. PRISM (Prototype-Rectified Iterative Self-Supervised Manifold Denoising) is a training-free, source-free TTA framework that leverages the Affine Noise Hypothesis, positing that severe acoustic noise induces a low-rank affine shift in the latent space, with over 90% of distortion energy in the leading 60 principal components. PRISM uses frozen text prototypes as geometric anchors to estimate and reverse this distortion via three closed-form corrections combined into one static projection matrix by Affine Bias Regression. At inference, adaptation is a single matrix-vector multiplication taking 0.0009 ms. On UrbanSound8K, PRISM improves over the zero-shot baseline by 12.94 percentage points and surpasses an oracle-assisted TTA baseline by 9.41 percentage points without using its privileged noise prompts. It also identifies the Polyphonic Trap, a failure mode of subspace deflation, and resolves it via Confidence-Aware Regression, recovering up to 8.16 percentage points for the worst-affected class.