anthology-of-the-sota

LIT-524: Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signatures

2 min read

Properties5
StatusActive
Tagsdeployment-and-society, analysis-and-evaluation
Cited as “introduced by” bySOTA-321 — Verify a model's lineage from the singular spectra of its attention products, which survive permutation and rescaling
Cited as “paper” byNOTE-268 — A data-free lineage check built on the products that survive permutation and scaling
Cited as “source” bySOTA-321 — Verify a model's lineage from the singular spectra of its attention products, which survive permutation and rescaling

Graph View

Wang, Ma, Xinyi and Li (2025) — ARXIV-2511.06390, AAAI 2026. Read as NOTE-268.

Key takeaways

  • The invariant is the whole idea. Comparing W_q, W_k, W_v, W_o directly is defeated by permutation and scaling, which change the weights and not the function. The products M_qk = W_q W_k^T and M_vo = W_v W_o are not, so their singular spectra survive the obfuscation.
  • Data-free and non-invasive. No training data, no queries, no modification of the model. It reads weights.
  • Two metrics. GhostSpec-mse compares truncated spectra directly; GhostSpec-corr compares the trend of the top-K normalized singular values across layers, aligned by dynamic sequence alignment and scored by distance correlation.
  • 55 model pairs, Llama-2-7b and Mistral-7B as base models, covering fine-tuning, pruning, merging, expansion and adversarial transforms. F1: GhostSpec-mse 0.9867, GhostSpec-corr 0.9730, REEF 0.9610, Logits 0.9268, QueRE 0.9157, PCS 0.9014.

Standing in the anthology

It opens a subject the record does not hold at all. Grepping the practices for watermarking, fingerprinting or provenance returns nothing. The corpus has three hundred recommendations about building and serving models and none about establishing where a model came from — which is a question that has already had public instances, and one the deployment-and-society topic exists for.

It is the same invariant the stability cluster is about, used for a different purpose. W_q W_k^T is the matrix whose spectral energy THEORY-062 says predicts a crash and whose spectral norm THEORY-061 bounds attention entropy by. Here its spectrum is a serial number. Nobody in either literature cites the other, and the coincidence is worth noticing: the quantity that governs whether training survives is also the quantity that identifies the model afterwards.

Its limits are about adversaries, not accuracy. The evaluation covers transformations someone might apply innocently or to hide provenance, and the method is offered as evidence rather than proof. A determined adversary who knows the fingerprint is a different threat model and the paper does not claim to cover it.

Backlinks