Fiderer et al. (2025) — ARXIV-2504.06209
Key takeaways
- The setting. An agent and an environment are both finite-state hidden Markov channels (input–output HMMs, or transducers) taking turns on a shared register. Each round, the agent maps a percept to an action and the environment maps the action to the next percept. The agent’s work per round is bounded by the entropy change of its memory plus the register. The asymptotic work rate is W = ⟨H(Aₜ|Mₜ) − H(Sₜ|Mₜ)⟩ₜ, in units of k_BT ln 2 (eq. 17). This bound is tight under idealized isothermal protocols.
- Work capacity is the supremum of W over all agents compatible with the environment. Theorem 1 shows that it exists, that it lies between 0 and log₂|S|, and that it is subadditive when environments are cascaded. The identity channel has zero capacity.
- Without feedback the old design rules hold. For “product” environments, whose percepts do not depend on actions (the tape setting), Theorem 2 says two principles are compatible and reach the optimum: randomize actions without remembering them, and keep a predictive memory. This recovers the Boyd–Mandal–Crutchfield tape-setting results.
- With feedback they break (Theorem 3). There is an environment, a memoryless one with binary alphabets, in which for every memory size the set of work-efficient agents is disjoint from both the predictive set and the maximum-entropy-action set. Predictive agents in it keep complete information about their actions, H(Aₜ|Mₜ) = 0, and their work rate is at most zero.
- Worked example (§VI, Fig. 8). A two-state environment has a
feedback parameter r ∈ [0, 1].
- At r = 0 there is no feedback. Predictive agents are optimal, and the gap to a memoryless agent is (log₂27 − 4)/3 k_BT ln 2.
- For 0 < r < 1 the best predictive work rate drops sharply below the numerically optimized two-state agent. That agent is less predictive than the maximally predictive one but more predictive than a memoryless one. Optimized work rates for memory sizes 2, 3 and 4 are indistinguishable.
- At r = 1 the environment is deterministic and prediction is optimal again.
- The optimized agent’s predictive gap jumps discontinuously near r ≈ 0.71, which the authors attribute to a change in its memory mechanism.
- What it does not cover. Every agent is a fixed transducer optimized for its environment, whose description is stored in read-only memory. There is no learning, no reward, and no implementation cost beyond the second-law bound. The trade-off is shown by an existence theorem and one numerical family, not across a class of environments. The authors list goal-directed agents, as a reinforcement-learning analogue in which rewards are low-entropy percepts, and dissipation as future work.
Standing in the anthology
Filed under agents-and-environments, the topic added for documents about
systems that perceive and act. Its result is about physical limits on
idealized agents. It is not a method and it is not a claim about trained
models, so nothing in the record depends on it.
The nearest document is LIT-759 (Boyd et al., From monoliths to modules). It uses the same formalism, world models as transducers, and Boyd’s earlier tape-setting results are the ones this paper extends to feedback loops. Read together, the two cover the formalism from both sides: LIT-759 factorizes environments into modules, and this paper asks what an agent embedded in such an environment can extract from it.
The paper’s one sentence aimed at machine learning comes in §VII. Systems nearing thermodynamic efficiency limits may need memory and action policies that depart from maximal prediction. This is speculative, and no practice follows from it. It is a seed, not a gap.
The paper is also held in the companion record: nucleation’s note, which has a full reading of v1 and its proofs.
Unread — no NOTE.