nucleation

LIT-918: Neural networks and physical systems with emergent collective computational abilities

3 min read

Properties6
StatusActive — read 2026-10-14 (NOTE-819) in full; worth reading as the paper that made memory a property of a flow: with symmetric couplings the asynchronous dynamics descend an energy function, Hebbian storage makes random patterns attractors, and a cue a few bits off settles into the nearest one. Read it knowing that every number in it (the 0.15N capacity, the basin width, the fusion of memories closer than about a tenth of N, the loss of all memories under overload) comes from simulations at N = 30 and 100, and that its "stable states of no obvious meaning" appear at a low load, not only past capacity.
Version2
Tagscomplex-systems, neuroscience, anthology-candidate
SupportsCLAIM-180 — Drift is retrieval failure outside a critical distance: a restatement close enough to an idea retrieves it, one that has drifted too far is taken for new, and a code bypasses retrieval by content altogether
Cited as “paper” byNOTE-819 — Hopfield — Neural networks and physical systems with emergent collective computational abilities
Cited as “source” byTHEORY-272 — In Hopfield's 1982 simulations at N = 30 and 100, a network of threshold units with symmetric Hebbian couplings acts as a content-addressable memory whose stored random patterns are attractors: recall degrades severely near 0.15N patterns, a pattern placed 10 bits from another at N = 100 often fuses with it, and at 25-fold overload no stored pattern stays stable, THEORY-800 — A network of N threshold units with symmetric Hebbian couplings stores random patterns as attractors up to a load proportional to N, about 0.15N for all large N, past which recall fails for every pattern at once, and a provision for forgetting old memories, such as couplings that saturate, keeps the most recent ones retrievable however many are added

Graph View

J. J. Hopfield (1982), Proceedings of the National Academy of Sciences 79(8), 2554–2558

Key takeaways

From the reading, NOTE-819.

  • Memory as attractors. “Any physical system whose dynamics in phase space is dominated by a substantial number of locally stable states to which it is attracted can therefore be regarded as a general content-addressable memory.”
  • The energy function is derived. For symmetric couplings, E = −½ Σᵢ≠ⱼ TᵢⱼVᵢVⱼ never rises under asynchronous threshold updates (Eqs. 7–8). This is the paper’s one derivation; the rest is simulation.
  • Storage is Hebbian. Tᵢⱼ = Σₛ (2Vᵢˢ − 1)(2Vⱼˢ − 1), with Tᵢᵢ = 0. Unlike a linear associative net, which answers a mixed cue with a mixed output, the network chooses one memory.
  • Basins. At N = 30 with five memories, a start within 5 bits of a memory reaches it more than 90% of the time, falling to 0.2 at 12 bits. An ambiguous cue gets “a statistical choice between the memory states it most resembles”.
  • Capacity. “About 0.15 N states can be simultaneously remembered before error in recall is severe”: the load at which about half the memories are well retained, at N = 30 and 100.
  • Two different failures. Memories closer than about a tenth of N fuse, at any load. Past capacity, new memories make “all memory states irretrievable unless there is a provision for forgetting”; saturating synapses keep the recent and lose the distant past.
  • Spurious states at low load. At five memories in 30 units, 10% of random starts end in “stable states of no obvious meaning”.
  • Robustness. The memory survives asymmetric, clipped and one-way couplings, failing “softly”. With a threshold, a cue that resembles no memory falls to a state that signals unfamiliarity.

Standing in the record

Filed on 2026-10-11 at the owner’s request, with Hebb (LIT-922) and Kanerva (LIT-928, LIT-921), and read on 2026-10-14 in NOTE-819. It is the dynamical source for CLAIM-180’s picture of retrieval by content, and the reading supports that picture while correcting it in two places: merging comes from two memories being close, not from load, and the model has no analogue of a cue “taken for new” or of a code. Its overload result is the sharper form of the flooding analogy in CLAIM-179: past capacity a memory loses everything unless it forgets. THEORY-272, filed from the reading, holds the attractor account with its limits. Kanerva’s report (LIT-921) puts the same capacity per storage element on sparse distributed memory. Flagged anthology-candidate.

Backlinks