nucleation

THEORY-800: A network of N threshold units with symmetric Hebbian couplings stores random patterns as attractors up to a load proportional to N, about 0.15N for all large N, past which recall fails for every pattern at once, and a provision for forgetting old memories, such as couplings that saturate, keeps the most recent ones retrievable however many are added

2 min read

Properties5
StatusProposed
Tagscomplex-systems, neuroscience, cognition
SourceLIT-918 — Neural networks and physical systems with emergent collective computational abilities
ExtendsTHEORY-272 — In Hopfield's 1982 simulations at N = 30 and 100, a network of threshold units with symmetric Hebbian couplings acts as a content-addressable memory whose stored random patterns are attractors: recall degrades severely near 0.15N patterns, a pattern placed 10 bits from another at N = 100 often fuses with it, and at 25-fold overload no stored pattern stays stable
AnswersQUESTION-201 — Do random patterns stored by the Hebbian outer-product rule stay attractors with basins of nonzero width up to a load near 0.14N at large N, does recall then collapse for every pattern at once, and at what overlap do two stored patterns fuse?

Graph View

Source

Hopfield (1982), LIT-918, read in full in NOTE-819. The crosstalk on a bit is approximately Gaussian for large nN (Eq. 10), and the retention seen at n = 0.15N “is expected to be true for all large N”. On overload (p. 2557): “the addition of new memories beyond the capacity overloads the system and makes all memories irretrievable unless there is a provision for forgetting old memories”; with couplings saturating at 0, ±1, ±2, ±3, only recent memories stay stable.

What is asserted, and by whom

Two things, both Hopfield’s: that capacity scales with N at about 0.15N for large networks, with collapse for all patterns past it; and that forgetting, by saturating couplings, prevents that collapse for recent memories. The first rests on an estimate, the second on a statement with no figure.

What the record holds toward it

  • THEORY-272 (Active): the simulations at N = 30 and 100, with recall degrading near 0.15N, fusion of a pattern placed 10 bits from another at N = 100, and no pattern stable at 25-fold overload; the energy argument and the crosstalk estimate checked.
  • The analytic literature the promote_when names (not read here) is widely cited for a capacity near 0.138N at large N. If the record reads it, the figure in the title may need that correction.

What would confirm or reject it

The large-N treatment and a bounded-coupling model, as the promote_when says. QUESTION-201 asks the capacity half as a question.

Backlinks