nucleation

QUESTION-058: When a trained network's representation carries a group's structure, how much of it comes from the task, the architecture, the training set's relations and the symmetry of the update, and how much can be read off the network without supplying the group?

2 min read

Properties5
StatusOpen
Tagsrepresentation-learning, learning-theory, mathematics
Linelearned-structure
Refined byQUESTION-145 — In trained models, do the features' dimensionality-weighted projectors sum to the identity on their span, as they do in the exact polytope and tegum configurations of uniform superposition?, QUESTION-199 — Does the training fraction at which networks summing two learned embeddings generalise follow the fraction at which the training set's same-answer equations leave only translation and scale free, across group sizes?, QUESTION-206 — Does stochastic gradient noise drive training into a mirror-symmetric subspace where it is a saddle, beyond the second-order, diagonal-Hessian argument, and does a trained network's distance to that subspace fall to zero past a threshold set by noise and curvature?
BearingTHEORY-288 — In small CIFAR and MNIST autoencoders whose only basis-privileging primitive is the activation function, elementwise tanh and Leaky-ReLU concentrate latent representations into narrow beams arranged by the activation's discrete symmetry group, even in layers no activation precedes, while their orthogonally equivariant radial counterparts leave them spread, so axis-aligned discreteness there comes from the primitive, not the task, THEORY-210 — Continuous parameter symmetries stretch a network's minima into connected manifolds along which the loss is flat, and give gradient flow conserved quantities that fix where on such a manifold training ends; finite steps, gradient noise and weight decay break them, so the position along a symmetry orbit is set by initialization and optimizer, not by the loss, THEORY-022 — The reverse-engineered grokking network computes modular addition by multiplying characters of ℤ/p at a few frequencies, the same real two-dimensional irreducibles later found as circles in language models; in both the group comes from the task, not the network, THEORY-445 — A network built from group correlations over a discrete split group of plane symmetries, pointwise nonlinearities and coset pooling is equivariant to that group by construction, through the regular representation alone: the group is fixed before training and built into the weight sharing, not found in the data, THEORY-284 — If a feature's values form a compact metric space carried continuously and injectively onto the unit sphere, and cosine similarity is locally a C² decreasing function of squared feature distance, then every path on the representation manifold is √(−2g′(0)) times as long as the feature path it images, so on-manifold shortest paths recover the feature's intrinsic length metric up to one scale; a homeomorphism alone does not fix it, THEORY-213 — Every mirror symmetry of a network's per-sample loss makes its symmetric subspace invariant under gradient descent and SGD, and when the loss's Hessian is bounded below by −c, weight decay γ‖θ‖² with γ > c/2 puts every minimum in that subspace; rescaling symmetries that hold at λ = −1, rotation symmetries and permutation symmetries contain such mirrors, whose subspaces are zeroed weights, reduced rank and identical neurons, THEORY-184 — Under a permutation-equivariant update whose map is K-Lipschitz, coincident neurons stay coincident at every step size, and at step size below 1/K distinct neurons cannot merge in finitely many steps, THEORY-021 — In the exact configurations of uniform superposition (regular polygons, the tetrahedron, antipodal pairs, and tegum products of these that span the hidden space, with equal norms within each factor) the features form a rank-one POVM compressed from the n-feature basis: their dimensionality-weighted projectors sum to the identity, THEORY-181 — In a model that decodes the sum of two learned embeddings, at zero training loss with an injective decoder, a training set fixes the generalising representation of addition when its same-answer pairs leave only translation and scale free; for non-modular addition with p = 10 and one-dimensional embeddings, generalisation sets in near the training fraction, about 0.4, at which that becomes likely

Graph View

The question

Trained networks are found with characters of ℤ/p, with circles for months, and with polytopes in superposition. The question is where that structure comes from, and what an analyst can recover from the weights alone. One possible source is the data’s symmetry. Others are an operation built into the architecture, the number of relations in the training set, and the equivariance of the update rule.

Why the record needs it

QUESTION-036 asks whether a kind’s symmetries can be reconstructed from its realizations. CLAIM-137 identifies kinds by character-like invariants, and CLAIM-132 limits what characters fix. Trained networks are the record’s only measured cases of group structure in a representation. They show what the group’s source is, and whether reconstruction worked or the group was supplied. CLAIM-135 and CLAIM-148 frame recovery as identification up to stated ambiguities.

What the record holds

  • THEORY-022: the network multiplies characters, but “the group comes from the task, not the network”. A spectral test sees the irreducibles “only as pairs”. Limit: one weight-decay model; dropout models are not Fourier-sparse.
  • THEORY-181: the training set’s same-answer pairs fix the representation once they “leave only translation and scale free”, with a critical training fraction. Limit: the sum is built into the architecture.
  • THEORY-021: the dimensionality-weighted projectors of the packed geometries sum to the identity. But the POVM “needs an inner product”, and the feature basis is stipulated. Limit: toy geometry only.
  • THEORY-184: under a permutation-equivariant update “coincident neurons stay coincident”, and below step size 1/K distinct ones cannot merge. The update preserves the symmetry it starts with. Limit: it says nothing about the function computed.
  • THEORY-284: if cosine similarity is locally a decreasing function of squared feature distance, on-manifold shortest paths recover the feature’s intrinsic length metric up to one scale. Limit: it reads off topology and the length metric only, under hypotheses assumed and not derived, with the candidate metric supplied by hand; it recovers no group.
  • THEORY-288: elementwise tanh and Leaky-ReLU arrange autoencoder latents into beams by the activation’s discrete group (B_n or S_n), and their radial counterparts do not. A group read off a network can be the network’s own. Limit: small autoencoders, density plots with no statistic.

What would answer it

A non-abelian task learned without a matrix representation imposed, analysed without the group being supplied in advance. Its irreducibles would be either recovered from the weights or shown to be unrecoverable.

Backlinks