ADR-027: loss-landscapes joins the topics

Context

The owner asked for 33 readings to be filed here. Most are about the geometry of trained solutions in weight space:

• mode connectivity, both linear and along curves;
• the permutation symmetries behind barriers between minima, and Git Re-Basin;
• star domains and connecting simplexes;
• mode connectivity in generative models, GNNs, transformers and mixtures of experts;
• model merging (TIES, ZipIt!, and weight-space regions of fine-tuned models).

The rest are on statistical-mechanics and random-matrix accounts of generalization and double descent, and on where in a network generalization is carried.

The anthology holds two of the set: Git Re-Basin and Entezari et al. on permutation invariance. This record held none of it.

No topic word fits. information-geometry is the Fisher metric and the curvature of statistical models, not the arrangement of minima. learning-theory is why learning generalizes, which only some of the batch addresses.

Decision

Add loss-landscapes: the geometry of trained solutions in weight space. It covers:

• mode connectivity, linear and along curves;
• barriers and the permutation symmetries that hide or remove them;
• re-basin alignment;
• star domains and connecting simplexes;
• local versus global landscape structure;
• model merging, as the practical use of the above.

The works on double descent and statistical-mechanics generalization take learning-theory, and also loss-landscapes where they are about the landscape.

Every work in the batch is a machine-learning work that an anthology topic could hold, so each carries anthology-candidate (ADR-005). The two the anthology holds are filed here as ADR-013 second readings, and each pair names the other.

Alternatives considered

• information-geometry or learning-theory. Either would then hold two subjects, and a 33-paper line would be invisible as one.
• A separate model-merging word. Merging in this batch is argued from connectivity: weights can be averaged because the minima lie in one basin. It is a use of the landscape, not a separate subject, so it stays under the one word until a merging work arrives that does not rest on connectivity.