nucleation

LIT-1075: Deictic codes for the embodiment of cognition

2 min read

Properties4
StatusActive — read in full 2026-10-15 (the target article, pp. 723–742; the open peer commentaries and the authors' response, pp. 742–767, not read) from the typeset BBS PDF posted on Dana Ballard's University of Texas page (www.cs.utexas.edu/~dana/bbs.pdf; the same file is on Rajesh Rao's University of Washington page). Unpaywall and OpenAlex report the DOI closed. Worth reading as the founding statement of fixation as a pointer that binds a program variable to a place in the world at about a third of a second, and for the block-copying evidence that people re-look rather than memorize: about 1.3 looks at the model per block, the slowest lowest-memory strategy (about 3 s a block, against 1.5 s from memory) chosen most often under instructions to be fast, falling to 1.0 when looking is made dearer (70° separation) and 0.7 when patterns can be chunked. For QUESTION-095 it shows the trade stated as a cost comparison ("carrying cost" of working memory against on-line acquisition) that moves with both costs, but the cost is inferred from choice, not measured or modelled, what is left in the world is raw input rather than a computed result, and the authors argue against precomputing where needs are unpredictable, so it is the deferral policy opposite to the cockpit's, not amortization; for TERM-055 it shows people choosing between internal and external sources by their relative retrieval cost, and moving retrieval later, not earlier; for CLAIM-298 it offers only an analogy (cue-driven matching, the same image read differently by task step) and no test of framing or judgement.
Tagscognition, embodied-cognition, neuroscience
Cited as “paper” byNOTE-883 — Ballard, Hayhoe, Pook & Rao — Deictic codes for the embodiment of cognition
Cited as “source” byTHEORY-334 — In copying a pattern of coloured blocks that stays in view, seven subjects told to work quickly looked back at the model about 1.3 times per block, and their commonest cycle, which looks at the model both before picking up a block and before dropping it, was the slowest per block in the authors' observed tallies (about 3 s, against 1.5 s for a cycle that copies from memory); looks fell to 1.0 per block with model and copy 70° apart and to 0.7 with one-colour patterns that could be chunked, and in three subjects a colour change made during a saccade raised fixation time by about 129 ms after pickup against about 50 ms before; no statistics are reported

Graph View

Ballard, D. H., Hayhoe, M. M., Pook, P. K. and Rao, R. P. N. (1997). Deictic codes for the embodiment of cognition. Behavioral and Brain Sciences 20(4), 723–742. DOI 10.1017/S0140525X97001611

Key takeaways

Read in full (NOTE-883). At about a third of a second, the time scale of a saccade, the article models fixation as a pointer that binds a variable in a behavioural program to a place in the world, so that what is bound is only the task-relevant feature, fetched just before use. In a block-copying task subjects looked back at the model about 1.3 times per block on average, the commonest cycle being two looks per block (one apparently for colour, one for position), and chose this slowest, lowest-memory strategy (about 3 s a block, against about 1.5 s for copying from memory) though told to be fast; separating model and copy by 70° cut looks to 1.0 per block, and one-colour patterns that could be chunked cut them to 0.7. The authors read this as working memory having a “carrying cost” that exceeds the cost of re-acquiring from the world, but that cost is inferred from the choices and never measured or modelled, and the evidence is one task with small samples (seven subjects for the main data, three for the colour-change experiment) and no statistics reported in the article. What is left in the world is unprocessed input, not a computed result, and the article argues against precomputation where the needed products cannot be foreseen.

Why this is here

The owner, after the cockpit reading (LIT-1066): “amortized retrieval is actually a kind of amortized computation. there’s probably a clearer relationship here that I imagine has likely been clarified/formalized at some point in the 30 years since the cockpit essay was published.” Registered with six others for QUESTION-095.

The case for the world as a memory consulted when needed, with its costs: the alternative to the cockpit’s precomputation.

Backlinks