Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification

Summary

Open-weight language models are often fine-tuned, quantized, pruned, or merged without documented provenance. We propose a data-free, white-box method for lineage verification: by removing the identity-aligned component that residual training imparts to branch products, we isolate checkpoint-specific structure across residual blocks and compute a symmetric lineage score. On residual-MLP and GPT-2 benchmarks, the score perfectly separates fine-tuned, LoRA-merged, pruned, and quantized descendants from independent and distilled models (AUROC=1.0), distinguishing weight ancestry from behavioral similarity. Under function-preserving checkpoint laundering, weight-space baselines lose margin or fail, while our score remains unchanged and runs 76x faster than the nearest robust baseline on GPT-2. The projection-pairing signal generalizes across six language-model families and beyond; a case study correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints. These results establish a passive, data-free provenance signal for compatible open-weight language-model checkpoints.