nucleation

LIT-921: Sparse Distributed Memory and Related Models

2 min read

Properties6
StatusActive — read 2026-10-14 (NOTE-779) in full; worth reading as the compact statement of sparse distributed memory in matrix form, with the ordinary RAM as its zero-radius special case and a simple signal-to-noise analysis: fidelity Φ(ρ) with ρ² = pM/[1 + pT(1 + p²M)], best at p = (2MT)^−1/3, and capacity near a tenth of the hard locations at fidelity 0.999. Read it knowing that every number assumes uniform random data and exact retrieval cues, that the noisy-cue case and the critical distance are not analysed here, and that it relates SDM to the Hopfield net only through a cited capacity comparison.
Version2
Tagscognition, neuroscience, information-retrieval
SupportsCLAIM-180 — Drift is retrieval failure outside a critical distance: a restatement close enough to an idea retrieves it, one that has drifted too far is taken for new, and a code bypasses retrieval by content altogether
Cited as “paper” byNOTE-779 — Kanerva — Sparse Distributed Memory and Related Models
Cited as “source” byTHEORY-261 — In sparse distributed memory, recall of uniform random data from exact cues is a signal-to-noise problem whose optimum activates about (2MT)^-1/3 of the hard locations, and capacity then grows in proportion to the number of hard locations, rising toward about a tenth of them at 0.999 bit fidelity, not with the word length

Graph View

Pentti Kanerva (1992), RIACS Technical Report 92.10, NASA Ames; published 1993 in M. H. Hassoun (ed.), Associative Neural Memories, Oxford University Press

Key takeaways

From the reading, NOTE-779.

  • Two matrices. A fixed random address matrix A selects the hard locations within a radius of the cue; a contents matrix C of counters is trained by the outer-product rule, C := C + y(Ax) ⊗ w, and read by z = sign(Cᵀy(Ax)).
  • A RAM is the special case. With all 2ᴺ addresses, radius zero and one-bit counters, the memory returns exactly the word last written at an exact address.
  • Expansion buys capacity. Addressing through A turns an N-bit cue into an M-dimensional, nearly orthogonal activation vector, which makes capacity “independent of the input dimension N”.
  • Signal and noise. For uniform random data and an exact cue, a bit’s sum has mean pM and variance pM[1 + pT(1 + p²M)]; fidelity is Φ of their ratio, best at p = (2MT)
  • Capacity. It tends to 1/[Φ⁻¹(φ)]² of the hard locations: 0.105 at φ = 0.999 (0.096 for a million locations). Keeler’s result, cited, gives the Hopfield net the same capacity per storage element. In practice, store 1–5% of M.
  • Not analysed here. Noisy cues, which lower capacity (Chou 1989), and the critical distance of iterated reads, which is only illustrated.
  • Names and patterns. A name is “an ideal encoding, except that it is arbitrary”; in pattern computing the units “serve both as pointers and as data, and they need not be precise”.

Standing in the record

Filed on 2026-10-11 at the owner’s request, under ADR-013 beside LIT-928; the anthology holds it as ANTH-LIT-669. Read on 2026-10-14 in NOTE-779. It supplies the half of CLAIM-180 that Hopfield (LIT-918) does not: retrieval by exact address and retrieval by content as two ends of one machine. It also does not hold the critical distance that CLAIM-180 quotes; that is derived in the book, LIT-928. THEORY-261, filed from the reading, holds its signal-to-noise and capacity account under the conditions it states.

Backlinks