Why it is a question
THEORY-229 now holds what Simon et al. (LIT-242) prove: from a small initialization aligned with the top eigenvectors of the pairs’ symmetrized cross-correlation, at a common scale, each mode evolves on its own and rises at a time set by its eigenvalue and the initial scale, larger eigenvalues first; and the top spectral solutions are exactly the minimum-norm zero-loss ones. Their Result 4.6 says a generic small initialization approaches the aligned trajectory as its scale goes to zero, but they call it a Result “because we give an informal derivation rather than a formal proof”, checked numerically at d = 3 and on 500 CIFAR-10 crop pairs (NOTE-741).
What would answer it
A formal proof of Result 4.6, or of the same stepwise order for a second spectral self-supervised loss on a linear or fixed-kernel model, would answer it yes in the regime it claims. A linear or fixed-kernel case of this loss, from vanishing generic initialization with distinct positive eigenvalues, that does not reach the minimum-norm solution or learns a lower mode first, would answer it no. More ResNet runs showing stepwise rank growth do not settle it, since the theory does not cover a moving tangent kernel.