Lines of inquiry — A named line of the record’s own argument, so that a line can be set aside as a whole and what depends on it can be found. Retracting a line is a status change on its members, never a deletion (ADR-032).
How training finds structure — why trained networks come to have the structure they have, which features, symmetries and connected solution sets training reaches and why — understanding, not instruction for practice, which belongs to the anthology.
58 of 371 QUESTION documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| QUESTION-039 v2 | What sets the order and the moment, in training time or in examples, at which a network acquires a structure, and is acquisition one kind of event or several? | Filed in the fourth synthesis pass to give readings no question asked for a place in the argument. It bears on CLAIM-164. THEORY-086, THEORY-187, THEORY-200, THEORY-204 and THEORY-039 answer it in part; the body gives each one’s limit. | Open |
| QUESTION-051 v2 | What structure does training for one classification task leave in a network’s last layers, and what does that structure cost for decisions and distributions the network was not trained on? | Filed in the fourth synthesis pass to give readings no question asked for a place in the argument. It bears on QUESTION-010 and CLAIM-164. THEORY-083, THEORY-104 and THEORY-085 answer it in part; the body gives each one’s limit. | Open |
| QUESTION-058 | When a trained network’s representation carries a group’s structure, how much of it comes from the task, the architecture, the training set’s relations and the symmetry of the update, and how much can be read off the network without supplying the group? | Filed in the fourth synthesis pass to give readings no question asked for a place in the argument. It bears on QUESTION-036, CLAIM-137, CLAIM-132, CLAIM-148, CLAIM-135 and CLAIM-193. THEORY-022, THEORY-181, THEORY-021 and THEORY-184 answer it in part; the body gives each one’s limit. | Open |
| QUESTION-060 v2 | When a network is trained on data from a structured system, are the variables it comes to represent fixed by its training objective or by what its architecture can represent cheaply, and since an architecture that can represent the system’s coarse-graining need not learn it, when do they coincide with the system’s own relevant variables? | Filed in the fourth synthesis pass to give readings no question asked for a place in the argument. It bears on CLAIM-152 and QUESTION-025. THEORY-197 and THEORY-206 answer it in part; the body gives each one’s limit. | Open |
| QUESTION-069 | Is the connectedness of trained solutions a property of the solutions, or of the path family searched, the shared history that produced them and the regime they were trained in? | Filed in the fourth synthesis pass to give readings no question asked for a place in the argument. It bears on CLAIM-187 and CLAIM-142. THEORY-105, THEORY-107, THEORY-108, THEORY-109, THEORY-110 and THEORY-114 answer it in part; the body gives each one’s limit. | Open |
| QUESTION-080 | When a trained network’s parameters keep moving after its loss has converged, what about the network stays fixed, and is the last-layer geometry of neural collapse among the things that do? | The record holds two pictures of the end of training. In one, the last layer converges to a fixed geometry through the terminal phase (THEORY-083, neural collapse). In the other, the parameters never stop: they sample a stationary distribution at constant speed (THEORY-209). The two may describe different regimes. The terminal phase of neural collapse is cross-entropy training past zero error, with no finite minimum, while the limiting dynamics assume a quadratic minimum. Nothing in the record relates them. Drawn by the assistant from the reading of Kunin et al., and filed at the owner’s request. | Open |
| QUESTION-081 | When the relative scale of a network’s layers sets its learning regime under gradient flow, does that regime survive SGD, whose minibatch noise drives the relative scale toward balance, and on what timescale compared with learning? | Two readings meet here. Under gradient flow, the relative scale of adjacent layers is conserved and sets whether learning is rich or lazy (THEORY-212, Dominé et al.). Under SGD, that scale is not conserved: minibatch noise drives it toward the point where the layers’ gradient noise balances, at a rate set by η/S (THEORY-211, Ziyin et al.). If the drift is fast compared with learning, an unbalanced start chosen to be lazy would drift toward balance and richness. If it is slow, the regime set at initialization stands. Neither paper addresses the other. Drawn by the assistant from reading both, and filed at the owner’s request. | Open |
| QUESTION-122 | Can a deep network during training have meaningful affect-like states, such as frustration of its demands, and to which unit, the network, the network–optimizer pair or the training process, would they belong? | The owner’s, U30 of the exchange of 2026-10-17: “the possibility of meaningful affective states attributable to a deep neural network specifically during training”. A30: the framework supports “a meaningful, qualified notion of training-time affective organization” and not “the conclusion that a neural network feels distress”. U31 added the bearer half, and CLAIM-367 answers it for conventional training, in part. U33 moved the primitive from distress to frustration (CLAIM-368). An answer says what training-time condition, if any, is affect-like, and whose. Organizational-affect is the natural line, set aside; whether it reopens is the owner’s decision, not made here. | Open |
| QUESTION-127 v2 | Which diagnoses of frustration among a learner’s demands survive quotienting by function-preserving symmetries, which belong only to an update geometry, and which only to a particular training organization with its optimizer? | A34 §6G of the exchange of 2026-10-17: “When do two functionally equivalent parameterizations yield equivalent frustration diagnoses?” The owner, at U35: “interesting question, double click on this.” A35 answered the first-order coordinate case, filed as CLAIM-366: the common-descent test is chart-invariant, gradient cosines are not, and a budgeted test needs the metric pulled back. The rest is open: function- preserving symmetries, which are not charts of one space, and training trajectories, where “same current function ≠ same learning dynamics”. An answer sorts each diagnosis by what it is a property of. | Open |
| QUESTION-145 | In trained models, do the features’ dimensionality-weighted projectors sum to the identity on their span, as they do in the exact polytope and tegum configurations of uniform superposition? | The promotion condition formerly on THEORY-021, moved here unchanged when THEORY-021 was narrowed to the exact configurations and promoted on 2026-10-19. The algebra is settled for those configurations. Whether trained weights reach them is a measurement no one has made. | Open |
| QUESTION-149 | In released language models, do the relational linear properties reported so far lie in the subspaces that enter the logits, with Γ ⊆ N and Γ_q ⊆ M, so that they are properties of the distribution and not of one fitted model? | The trained-model half of the promotion condition formerly on THEORY-234, moved here unchanged when THEORY-234 was narrowed and promoted on 2026-10-19. Theorem 14 of Marconato et al. says a relational linear property is shared by every model inducing the same distribution when it lies in N with Γ_q ⊆ M. Whether the linearities reported in trained models meet that condition has not been measured. | Open |
| QUESTION-150 | In the Poincaré ball H^r for r > 2, does embedding a tree with worst-case distortion 1 + ε need Ω((ℓ/(εr)) log deg_max) bits per coordinate for r up to about log deg_max, and Ω(ℓ/ε) beyond, matching the code-based upper bound? | The higher-dimensional half of the promotion condition formerly on THEORY-196, moved here when THEORY-196’s planar lower bound was split off as THEORY-734 on 2026-10-19. LIT-874’s Proposition 3.1 gives the upper bound in H^r. No lower bound in H^r is held, so whether dimension helps exactly as much as the construction uses is open. | Open |
| QUESTION-153 | Do the hyperbolic Betti-curve matches in odour space and in CA1 separate a hyperbolic metric from a flat sphere or a flat multiscale code, and do they replicate on other data? | The former promote_when clauses of THEORY-191 and THEORY-192, moved here unchanged when both were narrowed to what was measured and promoted under ADR-047 v3. Each Betti-curve test rejected only uniform Euclidean cubes. On a thin shell, hyperbolic distance is nearly a monotone function of angle, and uniform hyperbolic sampling is nearly an exponential spread of scales, so flat rivals remain. Neither study has been repeated on other data or by another group. | Open |
| QUESTION-155 | Does a lossy-compression bound on generalisation error give a non-trivial value for an algorithm whose output carries infinite information about its sample, such as deterministic training of a network with continuous weights? | The promotion condition formerly on THEORY-248, moved here unchanged when THEORY-248 was narrowed and promoted on 2026-10-19. The bound is proved and finite wherever the output law’s rate-distortion function is. No one has evaluated it for a learned model whose I(S;W) is infinite. | Open |
| QUESTION-157 | Do networks trained on factored sequence processes other than Mess3 and Bloch Walk factors, or under an objective other than next-token prediction, also settle near the factored dimension in per-factor subspaces, and what loss gap does the factored representation pay when noise makes it lossy? | The promotion condition formerly on THEORY-256, moved here unchanged when THEORY-256 was narrowed to the transformers and processes tested and promoted on 2026-10-19 (ADR-047 v3). The embedding theorem is settled. Whether the training behaviour holds beyond one synthetic family and one objective, and what the lossy case costs, has not been measured. | Open |
| QUESTION-158 | Does gradient descent on a two-layer network, without a spectral initialization or a data split, recover direction k of a scale-free hierarchical multi-index target at n = Θ(a_k^(−2) d) samples and attain the optimal weighted error, and do the same thresholds hold for an input law that is not isotropic Gaussian? | The condition formerly on THEORY-283, moved here unchanged when THEORY-283 was narrowed and promoted on 2026-10-19. The thresholds and rates are proved for the Bayes-optimal and spectral estimators, and for a network whose first layer is the spectral estimator. Whether gradient descent reaches them, or whether they survive a second input law, is open. | Open |
| QUESTION-159 | Does an exact train-then-predict identity, of the kind proved for unnormalized linear attention and for softmax attention, hold for further sequence-layer classes such as delta-rule or mini-batch inner updates, and is there a standard attention layer whose dependence on its context admits no description as any learner’s train-then-predict step? | The scope test formerly in THEORY-169’s promote_when, moved here unchanged when THEORY-169 was narrowed and promoted on 2026-10-19. The identities are proved for unnormalized causal linear attention (one batch gradient step of a linear model from zero) and for softmax attention (kernel regression that stores the context). Whether the pattern reaches other layer classes, or is bounded by one, is open. | Open |
| QUESTION-160 | Does the heat kernel of a metric tree satisfy the Hellinger Hölder and separation bounds of the multiscale hyperbolic diffusion theorem, with which Hölder exponent, and when does the diffusion operator of sampled data approximate a latent tree’s heat kernel? | The promotion condition formerly on THEORY-478, moved here unchanged when THEORY-478 was narrowed on 2026-10-19 to the conditional equivalence under two Hellinger bounds, with the admissible range α < θ/β. Whether a metric tree’s heat kernel meets those bounds, with which θ, and whether sampled data yield such a kernel, is not settled by anything the record holds. | Open |
| QUESTION-161 | Do trained language models learn in context by implicit Bayesian inference over a latent concept: do their in-context predictions track the posterior predictive of a latent-concept mixture fitted to their pretraining data, or carry a readable posterior that concentrates as the theorems say? | The promotion condition formerly on THEORY-614, moved here unchanged on 2026-10-19. THEORY-614’s title is a conditional theorem about the pretraining distribution under exact fit, and it says that whether trained networks learn in context this way is not shown. The old condition asked for evidence on exactly that disclaimed point, so it is a question in its own right. | Open |
| QUESTION-162 | In hyperbolic neural networks on tree-like data, does the advantage over Euclidean networks go with the Möbius bias translations and the distance-based output layer, the only operations through which curvature acts, or do networks without them keep it? | The promotion condition formerly on THEORY-566, moved here unchanged when THEORY-566 was promoted on 2026-10-19 on its structural claim. That the curvature of Ganea et al.’s networks acts only through two operations is settled; whether their gains on tree-like data come from those two operations is untested, since no ablation removes either. | Open |
| QUESTION-163 | Do Blackwell-equivalent representations of trained networks carry different stitching penalties under the paper’s own protocol, by more than the spread across stitching seeds and top networks, or does an independent treatment relate stitching to the comparison of experiments? | The promotion condition formerly on THEORY-176, moved here unchanged when THEORY-176 was promoted on 2026-10-19. That Blackwell-equivalent representations can stitch differently is shown by a constructed example. Whether they do, by a measurable margin, in trained networks under the paper’s protocol is untested. | Open |
| QUESTION-164 | Does the law of balance hold in a deep network trained on real data during its limiting dynamics: for parameter groups related by a rescaling symmetry, does the difference of squared norms move toward the value where the groups’ gradient-noise traces are equal, at a rate that scales with learning rate over batch size? | The promotion condition formerly on THEORY-211, moved here unchanged when THEORY-211 was narrowed and promoted on 2026-10-19 to the SDE model in which the law of balance is proved. Whether the law holds for a deep network trained on real data is not shown; the tests are small synthetic networks. | Open |
| QUESTION-165 | On the Random Hierarchy Model, is the number of examples a deep network needs set by when patch-label correlations become detectable, or by the size of the grammar, once the two are separated by moving the correlations with n_c, m and L fixed? | The promotion condition formerly on THEORY-195, moved here unchanged when THEORY-195 was narrowed and promoted on 2026-10-19 to what the experiments measured. In the model as given, the correlation threshold and the grammar’s size are the same expression, n_c m^L, so the measurements cannot say which one sets the number of examples. | Open |
| QUESTION-166 | Does the depth law of deep linear networks hold from starts off the decoupled set: learning time, counted in iterations at the stable step size, bounded in depth when every mode’s initial end-to-end gain is of order one, and growing as a power of 1/ε when it is ε, for random orthogonal or small random weights not aligned with the task’s singular vectors? | The promotion condition formerly on THEORY-187, moved here unchanged when THEORY-187 was narrowed and promoted on 2026-10-19. The depth law is exact for decoupled, balanced starts on whitened inputs. Whether it holds from random orthogonal or small random weights is supported by one MNIST experiment and an argument from dynamical isometry, not derived. | Open |
| QUESTION-171 | Do volume-maximizing joint-embedding objectives prefer equal-sized clusters as they are actually trained, and does the preference cost semantic transfer on naturally long-tailed data? | The promotion condition formerly on THEORY-306, moved here when THEORY-306 was narrowed to MSN’s and SwAV’s explicit prior and one study’s imbalanced-batch measurement, and promoted on 2026-10-19 (ADR-047 v3). LIT-231 reaches the same prior for VICReg only through an informal reduction of a simplified loss to K-means, and for SimCLR only through a cited equivalence. Its imbalance experiment uses two to eight ImageNet classes per batch, not a natural long tail. | Open |
| QUESTION-174 | Outside finite alphabets, is a variable-rate covering argument, rather than a change of measure alone, what yields data-dependent PAC-Bayes bounds? | Filed when the promotion audit (batch 036, ADR-047 v2) promoted THEORY-263. Its second clause was re-derived in general spaces by the disintegrated change of measure, with no coding argument. The former promote_when also asked for a second source deriving a PAC-Bayes bound from a variable-rate covering outside finite alphabets; this question holds that request. | Open |
| QUESTION-175 | From generic small initialization, does gradient flow on a symmetrized Barlow Twins loss with a linear or fixed-kernel model follow the aligned trajectory, learning the top eigenvectors one at a time in order of eigenvalue? | Filed when the promotion audit (batch 036, ADR-047 v2) narrowed and promoted THEORY-229 to initialization aligned with the top eigenvectors. Simon et al.’s Result 4.6, for generic small initialization, is an informal derivation; the former promote_when asked for its proof, and this question holds it. | Open |
| QUESTION-176 | Do networks trained by gradient descent on a mixture of random facts and a second domain reach the capacity-optimal allocation, so that their threshold size and mixing ratio are the optimal learner’s? | Filed when the promotion audit (batch 036, ADR-047 v2 and v3) narrowed and promoted THEORY-293. The theorem about the optimal capacity-bounded learner was re-derived, and Gu et al.’s Pythia runs were accepted as showing a threshold. That the trained networks’ thresholds are the optimal learner’s was the former promote_when, and this question holds it. | Open |
| QUESTION-179 | Is the latent-variable MDL bound of Theorem 4 in Sefidgaran et al. non-vacuous for a trained encoder, with a type-III symmetric prior that does not use the ghost data? | Filed when the promotion audit (batch 038, ADR-047 v2) narrowed and promoted THEORY-238. The constant in Theorem 4 was re-derived and the bound holds as stated. The former promote_when also asked whether the bound is small for a real network, which the account does not claim. This question holds that request. | Open |
| QUESTION-180 | Does the rate-distortion function of an optimisation trajectory given its sample, with the coupling coefficient M bounded or estimated and the 4Lε term kept, account for the generalisation of trained networks? | Filed when the promotion audit (batch 038) narrowed and promoted THEORY-266, Theorem 7 of LIT-233, on its proof. The former promote_when asked for an evaluation that would settle whether the trajectory’s compressibility given the sample accounts for generalisation. That is more than the theorem says. This question holds it. | Open |
| QUESTION-181 | Do trained networks follow the continually renormalized gradient flow toward neural collapse under MSE loss, and does the classifier stay near the least-squares classifier of its features outside Han, Papyan and Donoho’s settings? | Filed when the promotion audit (batch 038, ADR-047 v2 and v3) narrowed and promoted THEORY-278 to LIT-974’s measured cells and its renormalized model. The former promote_when asked whether ordinary training follows that model’s dynamics, and whether the central path holds in settings the paper did not test. This question holds both. | Open |
| QUESTION-192 | Does lazy-regime training leave the within-class variability of a network’s features at its initial value beyond one-dimensional, zero-bias, two-class data, where the tangent and initialization kernels are not proportional? | The promotion condition formerly on THEORY-294. THEORY-294 was narrowed to Kothapalli and Tirer’s Theorem 5.1 and promoted on 2026-10-19 (ADR-047 v2): for a wide two-layer ReLU network with zero bias on one-dimensional data whose classes lie on opposite sides of the origin, the tangent kernel is a constant multiple of the initialization kernel, so NC1 is unchanged. Whether lazy training leaves NC1 in place where the kernels are not proportional is open. | Open |
| QUESTION-193 v2 | Does sign descent provably delay each weaker skill until the stronger one is learned when a loss’s eigendirections are misaligned with the coordinates, and does changing only the optimizer turn a trained network’s simultaneous skill onsets into a sequence? | The promotion condition formerly on THEORY-253. THEORY-253 was narrowed to what Liu, Liu, Michaud, Gore and Tegmark measured in their Geometry model and promoted on 2026-10-19 (ADR-047 v3): skills progress together under gradient descent and in sequence under sign descent, with the ratio of onset times saturating at 2, and on a rotated quadratic only when the eigendirections are misaligned with the coordinates. That is simulation. A derivation, and a test on a trained network, are open. | Open |
| QUESTION-199 | Does the training fraction at which networks summing two learned embeddings generalise follow the fraction at which the training set’s same-answer equations leave only translation and scale free, across group sizes? | Filed when the promotion audit (batch 043, ADR-047 v2 and v3) narrowed and promoted THEORY-181 to the zero-loss injective model and to the one size Liu et al. measured. The former promote_when asked for the multi-size test; this question holds it. | Open |
| QUESTION-202 | Does neural collapse hold outside its first protocol, under augmentation, label noise, class imbalance and several seeds per setting, and does gradient descent on cross-entropy provably reach it? | The promotion condition formerly on THEORY-083, moved here when THEORY-083 was narrowed to the runs Papyan, Han and Donoho plotted and promoted on 2026-10-20 (ADR-047 v3). Those runs had balanced classes, no augmentation, dropout removed and one plotted run per learning rate. Whether collapse holds outside that protocol, and why training reaches it, is open. | Open |
| QUESTION-206 | Does stochastic gradient noise drive training into a mirror-symmetric subspace where it is a saddle, beyond the second-order, diagonal-Hessian argument, and does a trained network’s distance to that subspace fall to zero past a threshold set by noise and curvature? | The promotion condition formerly on THEORY-213. THEORY-213 was narrowed and promoted on 2026-10-20 (ADR-047 v2) to the exact half of Ziyin’s Theorem 1: a mirror symmetry’s subspace is invariant under gradient training, and weight decay above half the lower bound on the Hessian puts every minimum in it. The SGD half, attraction to the subspace even at a saddle, rests on a diagonal approximation and is open. | Open |
| QUESTION-207 | Without choosing checkpoints on the unseen pairs, does coverage of combinations still beat volume for compositional generalization, in a second architecture or with more than two concepts? | The promotion condition formerly on THEORY-117. THEORY-117 was narrowed to what Uselis, Dittadi and Oh measured and promoted on 2026-10-20 (ADR-047 v3), with a determination result the record re-derived: from-scratch ResNet-50s on two-concept grids, with checkpoints chosen on the unseen pairs, reach unseen combinations as coverage grows and not under up to four times more data of the same combinations. Whether that survives honest model selection, another architecture or more concepts is open. | Open |
| QUESTION-217 v2 | When a learning system is said to go from coarse to fine, do the order in which its representations refine one another, the order of their relevance to a task and the order in which training acquires them admit one common chain, and under what conditions on the learning operator? | A1 and A2 of the exchange of 2026-10-19 on coarse-to-fine processes. The record already holds that “coarse to fine” names different things: an order in training time by singular value (THEORY-186), an order in noise (THEORY-199), and a sample-complexity result in which every level arrives at once (THEORY-195). No entry separates refinement, relevance and acquisition as three relations and asks when they agree. An answer is a theorem giving sufficient conditions for a common chain, or a minimal family of counterexamples. | Open |
| QUESTION-220 v3 | Given that information-bottleneck optima at two budgets need not be nested, what condition on P(X, Y), beyond Y a function of X or X and Y both binary, guarantees that the coarser optimum is a (possibly noisy) function of the finer, X → Z₂ → Z₁, and how much is lost by forcing nesting when it fails? | A2 of the exchange of 2026-10-19 on coarse-to-fine processes, which called it “the most consequential missing bridge”. A hierarchy of optimal abstractions needs more than an optimum at each budget: it needs the optima to be compatible. NOTE-676 gets nesting by restricting the encoder to prunings of a fixed tree; nothing in the record says when it holds without that restriction. Rate–distortion theory’s answer is successive refinability (Equitz and Cover, registered unread). | Open |
| QUESTION-233 | Do language models show the data-limited exponent γ/(2β) because data limits the usable horizon: does the exponent track γ/(2β) when one corpus statistic is moved and the other held, and does the match survive at horizons of thousands of tokens with γ estimated outside the model class? | Filed when the promotion audit (batch 062, ADR-047 v2 and v3) narrowed and promoted THEORY-204. That the fitted data-limited exponent matches γ/(2β) on TinyStories and WikiText-103, and that assumed per-lag scaling forms give min(δ, γ/(2β)), is settled. Whether real learners show the exponent for the horizon reason was the old promote_when, and it moves here. | Open |
| QUESTION-234 v3 | Do interchange interventions leave the mediating direction underdetermined in trained networks: do two basis-related alignments in a trained model tie on every interchange drawn from the data, and for which networks are alignments that agree on all such interventions unique up to a stated symmetry? | Filed when the promotion audit (batch 062, ADR-047 v2) narrowed and promoted THEORY-243 to what the constructed three-unit network proves: two basis-related alignments tie on every interchange intervention on the input. Whether such ties occur in trained networks, or are ruled out for some class up to a symmetry, was the old promote_when, and it moves here. | Open |
| QUESTION-236 | Does fixed-feature transfer to new classes fall as penultimate class separation rises outside ImageNet-trained ResNet-50s, and does it move with within-class spread when the loss is held fixed? | The promotion condition formerly on THEORY-246, moved unchanged when that account was narrowed to what Kornblith et al. (2021) measured and promoted on 2026-10-20 (ADR-047 v3). Open: whether the inverse relation between penultimate class separation and new-class transfer holds in other pretraining settings, and whether separation, rather than the loss, is what moves transfer. | Open |
| QUESTION-244 | Must the optimal representations of an information-bottleneck objective at two budgets be nested, so that the coarser optimum is a (possibly noisy) function of the finer? | The yes-or-no part of QUESTION-220, split off under ADR-049. No, for deterministic encoders (CLAIM-387) and for stochastic ones (CLAIM-392), on one four-state source with binary Y. Raising the budget can reorganize the optimum rather than split it. What condition on P(X, Y) guarantees nesting stays with QUESTION-220. | Answered |
| QUESTION-274 v2 | Do two dictionaries of equal fidelity, trained on the same model with different seeds or widths, give the same validated circuit for the same prompt and behaviour, or is a feature circuit fixed only relative to the dictionary that supplies its units? | Feature circuits (sparse feature circuits, transcoder circuits, attribution graphs) are stated over the units of a learned dictionary. The dictionaries are not unique: features split and merge as the dictionary grows, and which concepts get a feature depends on their frequency relative to its size. Whether the circuit found for a behaviour survives a change of dictionary at equal fidelity is the question THEORY-802 asks of alignments, asked of dictionaries. No work the record holds has run the comparison. | Open |
| QUESTION-275 | Can the development of a transformer’s circuits in training be described as organizational regimes forming over shared, overlapping latent features, so that a model coupling feature allocation to circuit interaction, fitted to early checkpoints, predicts the effects of held-out training interventions that feature-only and circuit-only descriptions fitted to the same checkpoints miss? | The owner, at T24: “could we potentially apply this setup as an interpretive framework for understanding the learning dynamics of transformers?” “This setup” is the dynamic latent-feature model of T23 (CASE-165). The reply answered with a test: “Can the emergence, specialization, overlap, and persistence of transformer circuits be explained by modeling them as developing organizational regimes over shared latent features—and does that description predict training responses that existing feature- or circuit-based analyses miss?” It is the second time the owner has applied the record’s frameworks to a network in training. The first was CASE-113 and CLAIM-367, and CASE-117 is the population form. The sharp edge is CLAIM-427’s third grade: positive results would have to be prospective, against strong conventional baselines at matched complexity. Nothing here says that a circuit is a perspective. | Open |
| QUESTION-286 | Under smooth invertible changes of the parameter coordinates, which first-order frustration diagnoses among a learner’s demands are invariant: pairwise Euclidean gradient angles, common strict first-order descent, or budgeted feasibility with the update geometry transported? | The coordinate part of QUESTION-127, split off by the status review of 2026-10-27 under ADR-049, because CLAIM-366 (Active) settles it in full. Common strict descent, 0 ∉ conv{g_i}, is invariant under smooth invertible reparameterization. The sign of a Euclidean gradient inner product is not, and a negative inner product does not exclude common descent. A budgeted test is invariant when the metric is pulled back. These are the first three rows of A35 §2’s table. | Answered |
| QUESTION-291 | For finite X and Y, does every finite-budget information-bottleneck optimum nest under the β → ∞ optimum, what test on a pair of optimal encoders decides whether two finite budgets nest, and do the optima nest at every pair of budgets when Y is a function of X or when X and Y are both binary? | The settled part of QUESTION-220, split off by the status review of 2026-10-27 under ADR-049, because THEORY-743, THEORY-753 and THEORY-757 (Active) answer it. Every finite-β optimum is a stochastic coarsening of the causal partition. Two budgets nest iff some pair of optimal encoders forms X → T₂ → T₁, a convex-hull test on decoder points. The optima nest at every pair of budgets when Y is a function of X and when X and Y are both binary. | Answered |
| QUESTION-294 | When a network’s architecture can represent a structured system’s renormalization step exactly, must training by distribution fitting make the variables it represents the system’s relevant ones? | The representability part of QUESTION-060, split off by the status review of 2026-10-27 under ADR-049, because THEORY-197 (Active) settles it, in the negative. Mehta and Schwab’s RBM–renormalization correspondence is an identity of parametrizations, and an exact RG step is a perfect fit of the data distribution. Away from that point a contrastive-divergence RBM spends its hidden units on noise, not on the relevant fields. The architecture’s match does not fix what training keeps. | Answered |
| QUESTION-305 | From small random weights not aligned with the data’s eigenbasis, does a linear-attention layer trained on in-context regression come into alignment with that basis before or while its modes are learned, so that its learning is staged by the input spectrum without the alignment being assumed? | The promotion condition formerly on THEORY-481, moved here when the ninth status review narrowed it to the aligned case it derives and promoted it, as THEORY-285’s moved to QUESTION-303. THEORY-186 holds the same open claim for two-layer linear networks. | Open |
| QUESTION-312 | Does linear mode connectivity emerge during training rather than at initialization outside image classification and beyond MLPs: in a language model or transformer, for copies spawned from a shared state and for independent runs aligned under the architecture’s symmetry group, and for standard-width convnets or ResNets after alignment? | The promotion condition formerly on THEORY-114, moved here when the tenth status review narrowed it to the image-classification networks and MLPs its two sources measured and promoted it, as THEORY-285’s moved to QUESTION-303. | Open |
| QUESTION-316 | Is it the difference in task, and not depth and initialisation alone, that leaves features with no counterpart in the other model and makes merging fail: at matched width, do models trained from different initialisations on disjoint tasks lose cross-model feature correlation faster with depth than models trained on the same task, with merging failing at that depth? | The promotion condition formerly on THEORY-113, moved here when the eleventh status review narrowed it to what Stoica et al. measured, with their explanation given as theirs, and promoted it, as THEORY-114’s moved to QUESTION-312. | Open |
| QUESTION-317 | Does aligning independently trained networks over the architecture’s full symmetry group remove most of the linear barrier at standard width, on ImageNet or a language corpus, with an alignment computed without data; is the barrier that remains a difference in what the endpoints compute; and does it rise with depth? | The promotion condition formerly on THEORY-112, with its dropped depth clause, moved here when the eleventh status review narrowed it to what alignment removed in the networks its sources measured and promoted it, as THEORY-114’s moved to QUESTION-312. | Open |
| QUESTION-318 | Does layerwise linear feature connectivity hold outside image classification, in transformers and language models connected by spawning, shared pretraining or alignment, and does it carry from pairs to the many-model averages practice uses, model soups and SWA? | The promotion condition formerly on THEORY-111, moved here when the eleventh status review narrowed it to the pairs of image classifiers its source measured and promoted it, as THEORY-114’s moved to QUESTION-312. | Open |
| QUESTION-344 | Does raising or holding below threshold one relation’s subject–object co-occurrences in pretraining, with the rest of the corpus unchanged, make its LRE causality cross 0.9 or fail to, and is the threshold, with its spread, recovered across more model sizes and families? | The promotion condition formerly on THEORY-269, moved here when the status review of 2026-10-30 promoted it under ADR-065. Merullo et al. measured a correlation in three models and say they can draw neither causal claims nor conclusions about scale from it. | Open |
| QUESTION-357 | Outside MNIST and CIFAR-10, or under a second probe family such as a regularized nonlinear readout, do encoders constrained to preserve their input’s information exactly still span a range of downstream quality that includes worse than the raw input? | The promotion condition formerly on THEORY-227, moved here when the status review of 2026-10-30 promoted it under ADR-065. Tschannen et al. (LIT-247, read in NOTE-749) showed the spread on MNIST halves, with CIFAR-10 for two experiments, under a linear probe only. | Open |
| QUESTION-360 | When does a learned distinction become a concept rather than a discriminator or a habit (by being reused systematically, transferring to counterfactual cases, integrating with other capacities, or being correctable), is the transition graded or categorical, and how is a concept told apart from the organization that realizes it and from the thing it is about? | From the owner’s questions at U4 to U6 of the exchange “Analyze nucleation concepts” (2026-10-31): “how does our current position characterize the notion of a ‘concept’?”, “can we distinguish between a ‘concept’ and a ‘thing’?” and “how about ‘entity’?”. The assistant defined “concept” three times in three turns, each a little differently, which is the record’s signal to give it a code. A candidate term is filed beside it, Proposed (TERM-098); this question asks what would settle it. | Open |
| QUESTION-363 | When does the order in which stochastic gradient descent comes to occupy a network’s organizations over training time follow the order in which they come to dominate the Bayesian posterior as the sample grows, and what fixes the correspondence where it holds: an effective temperature set by learning rate and batch size, the accessibility of regions and the barriers between them, or the time allowed to equilibrate? | The owner’s challenge at U4 of the exchange “Analyze nucleation concepts” (2026-10-31). A3 had said that “Bayesian posterior dominance as sample size n changes is not the same process as gradient-based optimization as training time advances”. The owner: “Isn’t it? I think there’s a David M Blei paper that discusses this.” The paper is Mandt, Hoffman and Blei 2017 (LIT-1898). A4 conceded the separation was “too strong” and kept a narrower one, which is this question. It refines QUESTION-039, which asks what sets the order and moment of acquisition. | Open |
| QUESTION-370 | Do the concepts a language model’s workspace makes reportable across an Assistant episode share an episode-level mixture, so that an LDA-style admixture over J-lens readouts predicts their co-occurrence better than one persona selected per conversation and better than independent draws per position; does intervening on the inferred mixture change which concepts reach report while leaving the consumers that bypass the workspace unchanged; and if two models matched on persona distribution and mixture differ under such interventions, whose will-organization is that difference: the simulated Assistant’s, the network’s or the deployed system’s? | From the owner’s U10 and U12 of the exchange “Assess SDT Coverage” (2026-11-02): “the J-Space is basically this kind of LDA”, and H2, not H1, is the LDA because of the Dirichlet over each episode’s mixture (CLAIM-536). With U9’s question, how to read an LDA as will-organization under the Persona Selection Model. The paper is held and read (LIT-1347, NOTE-1069). At the level of a concept the analogy holds, as the owner put it at review: “J-space is a concept space. concepts are distributions over words. concepts are directions in a word space.” What keeps it a question: the paper fits no episode-level mixture, concepts combine additively in logits rather than as a mixture, and the workspace’s privilege is selective, where an LDA topic mediates every word. | Open |