nucleation

QUESTION-294: When a network's architecture can represent a structured system's renormalization step exactly, must training by distribution fitting make the variables it represents the system's relevant ones?

1 min read

Properties5
StatusAnswered
Tagsnatural-sciences, representation-learning, mathematics
Linelearned-structure
RefinesQUESTION-060 — When a network is trained on data from a structured system, are the variables it comes to represent fixed by its training objective or by what its architecture can represent cheaply, and since an architecture that can represent the system's coarse-graining need not learn it, when do they coincide with the system's own relevant variables?
Cited as “answers” byTHEORY-197 — Mehta and Schwab's correspondence between Kadanoff's variational renormalization group and restricted Boltzmann machines is an identity of parametrizations, under which an exact RG step is a perfect fit of the data distribution; away from that point it does not make training by distribution fitting a renormalization

Graph View

The question

QUESTION-060 asks whether the variables a trained network represents are fixed by its objective or by what its architecture can represent, and when they coincide with the system’s relevant variables. One candidate condition is that the architecture can express the system’s own coarse-graining exactly. This question asks whether that is enough.

The answer

THEORY-197, Active, answers no:

  • The identity. With the RG kernel T = −E + H, Kadanoff’s coarse Hamiltonian equals the RBM’s hidden-marginal Hamiltonian for every parameter value, and an exact RG step is exactly a perfect fit of the data distribution (Mehta and Schwab, Eqs. 18–22).
  • The counterexample. Koch-Janusz and Ringel train an RBM by contrastive divergence on fully packed dimers with added decoupled spin pairs. It spends its hidden units on the noise, then on columnar textures, and not on the staggered patterns that carry the RG-relevant fields.

So a network that can represent the renormalization need not learn it. A negative answer by one counterexample, for Boltzmann machines. When distribution-fitting networks do keep the relevant variables remains QUESTION-060’s.

Backlinks