The claim
What a system learns to preserve is constrained by what its training signal can distinguish. What can be verified that it preserved is constrained by the observations used to evaluate it. When the two are the same probe family, evaluation measures fit to the training signal, not preservation. Optimization moves the system toward whatever the probes reward, and that includes differences the probes cannot see. So preservation has to be shown under probes the system was not trained against.
The documented case (CASE-041) is worse than blindness. A generator trained against ImageNet features matches a stronger generator’s FID, computed in the same features, while doing worse in an independently trained feature space and with human judges.
Where the record applies it
CLAIM-149 applies it to translation and reconstruction models tuned against pragmatic classifiers or judges. CLAIM-190 applies it to a record’s anchoring measures. Both now rest on this claim for the general point.
What it does not say
- It does not say metrics in the training probes are worthless, only that they are not a certificate.
- It does not say a system trained against a family will introduce the differences the family cannot see. It says the family cannot show that it did not.
- It does not say held-out probes are neutral. They have their own training data.
- That its generality rests on its cases. Its ground is the argument above: a family the system was optimized against scores it well whether or not it preserved what the family cannot see, so the score cannot tell the two apart. CASE-041 shows the bias in image generation, by training; CASE-071 shows it in translation, by selection against the metric, which the title’s “training signal” does not literally cover.