Topics — What a work is about. Nucleation holds whatever reading the owner files here, whether or not an Anthology of the SOTA topic could also hold it; instruction for ML practice belongs to the anthology (ADR-048). The words are seeded from the clusters the reading-time triage of 2026-09-25 found, and grow by decision.
Mathematical statistics — the theory of statistical inference read for itself — comparison and sufficiency of experiments (Blackwell, Le Cam deficiency), statistical decision theory, the sufficiency theorems (Fisher–Neyman, Basu, Bahadur) and bounds on estimation. Bayesian modelling is probabilistic-modeling (ADR-031).
31 of 2004 LIT documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| LIT-349 v3 | Information and the Accuracy Attainable in the Estimation of Statistical Parameters | Rao (1945), DOI-10.1007/978-1-4612-0919-5_16. Proves the lower bound on the variance of unbiased estimators in terms of Fisher information (the Cramér–Rao bound), and in the same paper introduces the Fisher-information Riemannian metric and the Rao–Blackwell improvement idea. | Unavailable — seeded 2026-09-29 from metadata and published descriptions, not read: registered because the owner’s prior-art novelty map cites it. A close reading was attempted on 2026-09-29 and no legitimate full text could be reached; it stays Unavailable until a copy is supplied. |
| LIT-772 | A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics | Fritz (2020), Adv. Math. 370:107239. Markov categories as a framework for synthetic probability and statistics: conditioning, conditional independence, almost-sure equality and sufficient statistics stated in categorical terms, with the Fisher–Neyman, Basu and Bahadur theorems proved there. Basu needs no extra axiom; Fisher–Neyman and Bahadur need strict positivity, not conditionals. Fisher–Neyman is matched to the classical theorem only in FinStoch. | Active — read 2026-10-09 (NOTE-588); worth reading as the reference account of Markov categories and of how much probability and statistics follows from copying, discarding and composing kernels: determinism, four notions of conditional independence with the semigraphoid rules (weak union only with conditionals), almost-sure equality, and sufficiency, completeness and ancillarity. Basu’s theorem holds in every Markov category, and Fisher–Neyman and Bahadur under strict positivity, which Stoch has without conditionals. The abstract Fisher–Neyman is matched to the classical theorem only for finite sets. |
| LIT-778 v3 | Comparison of Statistical Experiments | Torgersen (1991), Cambridge University Press. The comprehensive treatment of comparing statistical experiments, from Wald, Blackwell and Le Cam: when one experiment is more informative than another, Le Cam’s deficiency as the measure of the difference, and its relations to sufficiency and randomisation. Not read: no lawful full text. | Unavailable — filed 2026-10-09 from Crossref and the publisher’s pages, not read: no lawful full text was reachable. Cambridge Core sells the book and its chapters (OpenAlex: closed access, no repository copy); its public pages carry only the summary and each chapter’s opening paragraph. The Internet Archive copy is lending-only (access-restricted, printdisabled), and HathiTrust has no record for the ISBN. No NOTE is filed. It stays Unavailable until a copy is supplied, not on merit. |
| LIT-781 | Equivalent Comparisons of Experiments | Blackwell (1953), Ann. Math. Statist. 24(2):265–272. Completes the Blackwell order: for finitely many states, one experiment is at least as informative as another for every decision problem exactly when the other is a garbling of it. The easy direction is Blackwell (1951) and the finite converse Sherman and Stein’s; this paper re-proves the converse by minimax, extends it to arbitrary outcome spaces by a martingale argument, introduces comparison by k-action problems, and reduces two-state comparison to the type I/type II error trade-off. | Active — read 2026-10-09 (NOTE-595); worth reading as the paper in which the comparison of experiments is settled in general: for experiments with finitely many states, being at least as informative in every decision problem (Bohnenblust, Shapley and Sherman’s order) and being able to reproduce the other by a stochastic transformation (Blackwell’s 1951 sufficiency, later called garbling) are the same order. The finite-outcome converse is Sherman and Stein’s, re-proved here by a minimax argument; the extension to arbitrary outcome spaces, by martingale convergence, is new. It adds comparison by k-action problems, and shows that for two states testing problems, and so the type I/type II error curves, decide the order. |
| LIT-840 v2 | Asymptotic Methods in Statistical Decision Theory | Le Cam (1986), Springer Series in Statistics, xxvi + 742 pp. Asymptotic statistics organised around Blackwell’s experiments, Wald’s decision theory and Le Cam’s deficiency distance between experiments, with approximation by Gaussian shift and exponential families as the central theme. Not read: no lawful full text; known here from its front matter. | Unavailable — filed 2026-10-09 from catalogue records and the publisher’s free front matter, not read: no lawful full text was reachable. Springer sells the book and its chapters (OpenAlex: closed access, no repository copy); the Internet Archive copy is lending-only (access-restricted, printdisabled); HathiTrust holds it search-only. The front matter Springer serves freely (26 pages: contents, acknowledgments and the author’s “To the Reader” with a chapter-by-chapter summary) was read; the main text was not. No NOTE is filed. It stays Unavailable until a copy is supplied, not on merit. |
| LIT-964 v2 | The reusable holdout: Preserving validity in adaptive data analysis | Dwork et al. (2015). Adaptive reuse of data inflates false discoveries. By the Supplementary Material: if the max-information between the data and the function an analyst picks is at most log₂e·τ²n, that function’s empirical mean exceeds its true mean by τ with probability at most e^{−τ²n}, and ε-differential privacy bounds max-information by log₂e·εn. Thresholdout answers m adaptively chosen [0, 1]-valued queries on a holdout to accuracy τ with probability 1 − β for n = O(ln(m/β)/τ²·min{B, √(B ln(m/β))/τ}), B the budget of detected overfits. The main text was not read. | Active — read 2026-10-14 (NOTE-848) in part; worth reading for QUESTION-040 as a high-probability bound on overfitting under adaptive reuse: bounded max-information between the data and the analyst’s choices bounds the chance a chosen statistic overfits, differential privacy bounds max-information, and Thresholdout validates m adaptively chosen statistical queries with a holdout growing as ln m. Read it knowing that only the Supplementary Material was reached, not the main text, that four of its six bounds are proved in companion papers, and that it covers [0, 1]-valued queries on a discrete universe. |
| LIT-1028 | A cluster process representation of a self-exciting process | Hawkes and Oakes (1974), Journal of Applied Probability 11(3). Represents a stationary self-exciting process as a cluster process of immigrants and their offspring, the branching view in which a branching ratio below one gives finite cascades. | Unavailable — filed 2026-10-13, not read. Registered from the 2026-10-13 exchange on meaning operators and selfhood (the workbench entry of 2026-10-13, 21:29). No NOTE is filed, and it stays Unavailable until it is read, not on merit. Looked for a lawful full text on 2026-10-13 and found none: OpenAlex reports it closed (no open-access location, no repository full text) and Semantic Scholar gives no openAccessPdf; the Cambridge Core page (10.1017/S0021900200096273) offers only purchase or institutional access, and its PDF link returns the access page; the JSTOR copy (10.2307/3212693) is not free; web searches found no author-posted, institutional or course-hosted copy. Brémaud and Massoulié (LIT-1035, read in NOTE-728) describe its results: existence of the stationary linear excitatory process for ∫h < 1 by a cluster (branching) construction, and uniqueness and stability in distribution from an empty past. |
| LIT-1035 | Stability of nonlinear Hawkes processes | Brémaud and Massoulié (1996), Ann. Probab. 24(3): 1563–1588. Sufficient conditions for a Hawkes process with a nonlinear rate function and kernels of either sign to have a unique stationary version that versions started from a suitable past converge to: Lipschitz constant times the kernel’s absolute mass below one (for networks, a spectral radius below one), or a bounded Lipschitz rate with no limit on kernel size. Kernels are fixed in time; nothing is proved past the threshold or about cascade sizes. | Active — read in full 2026-10-13 (NOTE-728) from the publisher’s typeset PDF, open access on Project Euclid (projecteuclid.org/journals/annals-of-probability/volume-24/issue-3/Stability-of-nonlinear-Hawkes-processes/10.1214/aop/1065725193.pdf), fetched as the Internet Archive’s capture of that URL because Project Euclid refused direct requests from this environment. Worth reading as the standard source of stability conditions for Hawkes processes that are nonlinear and may inhibit: a Lipschitz-weighted kernel mass, or a network spectral radius, below one, or a bounded rate with no size condition (THEORY-215). For QUESTION-091 it shows that outside the linear excitatory case that quantity is a sufficient bound, not a branching ratio, and that saturating rates have no critical point; for CLAIM-163 it supplies a coupling and a control parameter, not an order parameter or transitions. |
| LIT-1036 | Spectra of some self-exciting and mutually exciting point processes | Hawkes (1971), Biometrika 58(1). Introduces point processes whose event rate is raised by their own past events, and by other processes’ events in the multivariate case, and derives their spectra. | Unavailable — filed 2026-10-13, not read. Registered from the 2026-10-13 exchange on meaning operators and selfhood (the workbench entry of 2026-10-13, 21:29). No NOTE is filed, and it stays Unavailable until it is read, not on merit. Looked for a lawful full text on 2026-10-13 and found none: OpenAlex reports it closed (no open-access location, no repository full text) and Semantic Scholar gives no openAccessPdf; the Oxford Academic version of record is for subscribers only; the JSTOR copy (10.2307/2334319) falls outside its free early-journal content; web searches found no author-posted, institutional or course-hosted copy. A scan is posted on a third party’s personal site (dcscience.net), which is not the author’s, an institution’s or a course’s copy, so it was not used. Hawkes’s retrospective with Chen (Cardiff ORCA, eprint 141988) is a different work. Brémaud and Massoulié (LIT-1035, read) cite it as the source of the process. |
| LIT-1160 v2 | Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing | Donoho & Tanner (2009), Phil. Trans. R. Soc. A 367: 4273–4293, ARXIV-0906.2530. For Gaussian matrices, recovery of a k-sparse vector from n measurements in dimension N by ℓ1 minimization, or by its nonnegative LP form, succeeds with probability tending to 1 below a curve ρ(δ; Q) in δ = n/N, ρ = k/n and to 0 above it, a result proved earlier and restated here. In 2,948,000 linear programs over eight non-Gaussian ensembles at N up to 1600, the 50% success points fall on the same curves. Exact agreement at finite N is rejected; the differences decay as N^(-1/2), so the paper supports an asymptotic universality, observed and not proved. | Active — read 2026-10-14 (NOTE-933) in full, from the 48-page arXiv copy, not the journal text; worth reading as the measured case for how far the sharp ℓ1 recovery thresholds proved for Gaussian matrices hold for other random matrices. Read it knowing that the transitions sit on the Gaussian curves only in the large-N limit, that finite-N universality is rejected, that the sparse and Hadamard ensembles miss by about one transition width at small N, that the curves belong to (LP) and (P1) and not to other decoders, and that nothing about non-Gaussian matrices is proved. |
| LIT-1162 | Quantifying unique information | Bertschinger, Rauh, Olbrich, Jost & Ay (2014), Entropy 16(4): 2161–2183, ARXIV-1311.2852. For a target X and sources Y, Z, defines the unique information of Y as the minimum of MI_Q(X:Y|Z) over Δ_P, the joint distributions with the same (X,Y) and (X,Z) marginals as P; shared and synergistic information follow from the consistency equations. If unique information depends only on the pair marginals, these functions bound every other such measure (Lemma 3). The unique information is zero exactly when Z is Blackwell-sufficient for Y with respect to X (Lemma 6, Corollary 7). The three measures are non-negative, satisfy the bivariate PI axioms and the identity property, and are computed by a convex programme. They do not extend to three or more sources on the PI lattice. | Active — read in full 2026-10-17 (NOTE-937); worth reading as the bivariate partial information decomposition that the redundancy-measure-dependent numbers in the record can be checked against: unique information is the minimum of the conditional mutual information over all joints with the same source–target pair marginals, so it vanishes exactly when Blackwell’s order says the other source knows everything this one does, and synergy is what the pair marginals cannot reveal. Read it knowing that it is bivariate only (it cannot be extended to the PI lattice for three sources, by its own §6), that its optimisation is ill-conditioned in practice (Example 31), and that Table 1’s Unq and Copy rows conflict with its own Proposition 18. Filed as THEORY-705. |
| LIT-1163 | Estimating Mutual Information | Kraskov, Stögbauer & Grassberger (2004), Physical Review E 69, 066138, ARXIV-cond-mat/0305641. Two estimators of I(X,Y) from N i.i.d. samples, built from Kozachenko–Leonenko k-nearest-neighbour entropy estimates with the joint space in the maximum norm: I⁽¹⁾ = ψ(k) − ⟨ψ(n_x+1) + ψ(n_y+1)⟩ + ψ(N), and I⁽²⁾ = ψ(k) − 1/k − ⟨ψ(n_x) + ψ(n_y)⟩ + ψ(N), where n_x, n_y count marginal neighbours within the joint k-th-neighbour distance. Using one distance scale across joint and marginal spaces makes the biases largely cancel. Numerically they are exact for independent variables (conjectured, not proved) and have systematic errors that scale with k/N. | Active — read 2026-10-16 (NOTE-938), the formal sections in full and the numerical and application sections at lower resolution; worth reading as the source of the k-nearest-neighbour (KSG) mutual-information estimators that the record’s readings use off the shelf (NOTE-788, NOTE-320, NOTE-828). Read it knowing that the estimators rest on a locally-constant-density approximation with no bias bound, that their exactness for independent variables is a numerical conjecture the paper’s own Weinman example strains, and that it estimates mutual information and multi-information only: conditional mutual information, and so transfer entropy, is not treated. Filed as THEORY-709. |
| LIT-1164 | Causal Reasoning in Graphical Time Series Models | Eichler & Didelez (2007), UAI 2007: 109–116, ARXIV-1206.5246. Defines the causal effect of an intervention on one component of a stationary multivariate time series on another at a later time, and links it to strong Granger noncausality through mixed graphs (directed edges for Granger causality, dashed edges for contemporaneous dependence). Under its intervention assumptions, which require the system to include every relevant process and exclude instantaneous causality, a missing directed edge means no effect at the next step, non-ancestry means none at any horizon, and back-door and front-door criteria identify the effect from observational data. In a Gaussian VAR the identified effect is a coefficient of the multi-step linear predictor. | Active — read in full 2026-10-16 (NOTE-941); worth reading as the short statement of exactly when a Granger-causal analysis says something about interventions: under its intervention assumptions, which need the series to include every relevant process (observed or not) and exclude instantaneous causation, a missing Granger edge means no causal effect at the next step and non-ancestry means none at any horizon, and time-series back-door and front-door criteria identify effects from observational data. Read it knowing that the front-door theorem’s proof is deferred to a paper “in preparation”, that the results are sufficient conditions, and that the examples are Gaussian VARs. Filed as THEORY-704. |
| LIT-1169 | Measurement of Linear Dependence and Feedback between Multiple Time Series | Geweke (1982), Journal of the American Statistical Association 77(378): 304–313. Measures the linear dependence between two multivariate time series as the sum of three parts: feedback from the first to the second, feedback from the second to the first, and instantaneous feedback. The directed measures are log ratios of generalized variances (determinants of residual covariance matrices), have asymptotic χ² null distributions, and decompose additively by frequency. Unread; known through Barnett, Barrett and Seth and the abstract. | Unavailable — unread, registered 2026-10-16. A lawful copy was sought and not reached: OpenAlex marks it closed with no repository full text, Crossref lists only the Taylor & Francis and JSTOR copies, and web searches found no author-posted or course-hosted copy. Its generalized-variance measure and χ² asymptotics are restated by Barnett, Barrett and Seth (LIT-1171), which is how the record reads it until a copy is supplied. |
| LIT-1173 | Investigating Causal Relations by Econometric Models and Cross-spectral Methods | Granger (1969), Econometrica 37(3): 424–438. Proposes testable definitions of causality and feedback between stationary time series: Y causes X if the past of Y improves the prediction of X beyond what the other relevant past information gives. Shows the cross spectrum can be decomposed into two parts, one for each causal arm of a feedback loop. Unread; known through the works that restate it. | Unavailable — unread, registered 2026-10-16. A lawful copy was sought and not reached: Crossref and the Econometric Society give only the JSTOR copy (stable 1912791), behind a login; a course-hosted copy on a University of Freiburg seminar page (jeti.uni-freiburg.de, stud_sem_SS_09/grangercausality.pdf) refused this client’s connection three times and answered WebFetch with HTTP 503; a Bilkent course copy found by search returned 404. Searched again on 2026-10-17: the Freiburg host still reset the connection, and the only other copies found (ResearchGate, Scribd) are user uploads whose authorization could not be established. The record reads it through Barnett, Barrett and Seth (LIT-1171), Seth (LIT-1170) and Eichler and Didelez (LIT-1164), which restate its definition, until a copy is supplied. |
| LIT-1439 | Statistical ranking and combinatorial Hodge theory | Jiang, Lim, Yao & Ye (2011), Mathematical Programming 127:203–244. HodgeRank: an edge flow of aggregated pairwise comparisons splits orthogonally into a gradient flow, whose potential is the least-squares global ranking (a Borda count generalized to incomplete data), and a cyclic residual, which splits into curl (local) and harmonic (global) inconsistency (Theorems 4.8, 5.1). The residual’s share, the “cyclicity ratio”, certifies the ranking. | Active — read 2026-10-23 (NOTE-1099): in full, from arXiv 0811.1067v2 (42 pp.). HodgeRank. Pairwise comparisons, aggregated into a weighted edge flow on the graph of alternatives, decompose orthogonally into a gradient flow, the l2-best global ranking found by weighted least squares, and a divergence-free residual. The residual splits into curl (local, triangular inconsistency) and harmonic (global inconsistency) parts. Binary win/loss data enter as averaged signs, and on complete balanced binary data the ranking is Borda’s. The decomposition is cardinal: curl-freeness is stronger than transitivity. It is the tool for reading the record’s conflict record. |
| LIT-1523 v3 | Sequential Tests of Statistical Hypotheses | Wald (1945). Introduces the sequential probability ratio test: observations are taken one at a time, the log-likelihood ratio of two simple hypotheses is accumulated, and sampling stops when it leaves an interval whose bounds are set approximately from the desired error probabilities. The paper derives the test’s operating characteristic and its expected number of observations, which for given error rates is substantially smaller than the fixed sample a test of the same strength needs, and proves that no test of the same strength needs fewer on average once the overshoot of the bounds is neglected (checked against the reading). | Active — read 2026-10-25 (NOTE-1165) in full, from the copy the owner supplied on 2026-10-25, a JSTOR scan of the Annals article (this client had been stopped by Project Euclid’s bot check the same day). Worth reading as the source of the sequential probability ratio test: its boundaries follow from the error rates alone, A = (1 − β)/α and B = β/(1 − α) (THEORY-952), and its operating characteristic, expected sample size and optimality against every test of the same strength are derived, with savings of about half against the fixed-sample test (THEORY-951). Read it knowing that optimality is proved only when the overshoot of the boundaries is neglected, the exact case being a conjecture (footnote 12), that the operating characteristic and sample-size formulas are approximations that need i.i.d. observations, and that the paper never mentions diffusion: the drift-diffusion limit is later work’s, and is the case in which these approximations are exact. |
| LIT-1525 v2 | Bayesian Experimental Design: A Review | Chaloner & Verdinelli (1995). A review of Bayesian experimental design from a decision-theoretic standpoint, which casts the Bayesian design criteria as parts of one coherent approach covering linear and nonlinear problems, suggests new directions through new utility functions, shows that in some linear problems Bayesian solutions change sensibly as prior and utility are adapted to the experiment, and justifies the choice of optimality criterion mathematically (abstract). | Active — read 2026-10-25 (NOTE-1161) in full, from the authors’ Technical Report #607 (University of Minnesota, July 1995) in the University of Minnesota Digital Conservancy; worth reading for Lindley’s decision-theoretic formulation of design, for the derivation of Bayes D- and A-optimality from Shannon information and quadratic loss in the normal linear model (THEORY-956), and for the account of nonlinear design as prior-averaged local criteria, with local optimality as the point-mass case and its failure for the turning point (THEORY-954). Read it knowing that it is a 1995 survey, mostly of normal linear models, that its nonlinear criteria are approximations of unknown accuracy, that its treatment of model discrimination is brief, and that the copy read is the technical report, with equations lost to OCR, not the journal text. |
| LIT-1537 v2 | Hierarchical Beta Processes and the Indian Buffet Process | Thibaux & Jordan (2007), AISTATS 2007, PMLR 2: 564–571. The beta process is the de Finetti mixing distribution underlying the Indian buffet process, playing for it the role the Dirichlet process plays for the Chinese restaurant process. The paper uses the parallel to define hierarchies of beta processes and to derive posterior inference algorithms for the IBP, with an application to document classification related to smoothed naive Bayes (abstract). | Active — read 2026-10-26 (NOTE-1171) in full, from PMLR’s own PDF; worth reading for the beta process as the IBP’s de Finetti measure, with the predictive and the count of distinct features that follow (THEORY-985), and for the hierarchical beta process, the established way for groups to share one feature library at group-specific prevalences (THEORY-1015). Read it knowing that conjugacy is cited rather than proved, that its groups are disjoint and static, that its one experiment is a single text-classification split, and that its stated reason for a small group concentration reads against its own (9). |
| LIT-1548 v2 | Dependent Indian Buffet Processes | Williamson, Orbanz & Ghahramani (2010), AISTATS 2010, PMLR 9: 924–931. To let binary latent feature models follow a distribution that changes over time, space or another covariate, a dependent IBP generates for each covariate value a binary matrix with unboundedly many columns, its evolution over the covariate controlled by a hierarchical Gaussian process whose covariance sets the dependence and exchangeability. MCMC inference; experiments find explicit dependence improves prediction (abstract). | Active — read 2026-10-26 (NOTE-1175) in full, from PMLR’s own PDF; worth reading for the thresholded-Gaussian-process construction that keeps each feature’s prevalence fixed over a covariate while its covariance sets persistence, the established comparator for a dynamic latent feature model (THEORY-1006). Read it knowing that its claim of IBP marginals at every covariate value holds for the whole matrix only in its collapsed model, that features stay independent of one another, and that its evidence is two prediction comparisons with the plain IBP. |
| LIT-1556 v4 | The Indian Buffet Process: An Introduction and Review | Griffiths & Ghahramani (2011), JMLR 12: 1185–1224. The Indian buffet process is a distribution over equivalence classes of sparse binary matrices with finitely many rows and an unbounded number of columns, usable as a prior for objects represented by a potentially infinite array of features. The paper derives it as the limit of a finite beta–Bernoulli model, illustrates it in an infinite latent feature model, and reviews its applications, extensions and connections to other stochastic processes (abstract). | Active — read 2026-10-26 (NOTE-1172) in full, from JMLR’s own PDF; worth reading for the derivation of the IBP as the limit of a finite beta–Bernoulli model, for K+ ~ Poisson(αH_N) and Poisson(α) features per object, and as a map of the IBP’s variants to 2010, including the Markov IBP closest to a dynamic feature model. Read it knowing that αH_N is stated for the limit only and the finite model’s expected count of occupied features is strictly below it (THEORY-999), that its posterior Gibbs sampler is by its own account heuristic, and that its demonstrations are single runs on data built to suit the model. |
| LIT-1577 v3 | The Infinite Factorial Hidden Markov Model | Van Gael, Teh & Ghahramani (2008), NIPS 21: 1697–1704. The Markov Indian buffet process is a distribution over binary matrices with one row per time step and unboundedly many columns, each column a two-state Markov chain from an off state, with its own probability of switching on, a_m ~ Beta(α/M, 1), and of staying on, b_m ~ Beta(γ, δ). Its limit as M → ∞, a sequential buffet and a stick-breaking construction are derived. The number of features ever on is Poisson(αH_T). It is used as the prior of a nonparametric factorial HMM, which separates five speakers in mixed speech better than an IBP prior does. | Active — read 2026-10-27 (NOTE-1199) in full, from the NeurIPS proceedings’ own PDF; worth reading as the origin of the Markov Indian buffet process, the IBP’s extension to features that switch on and off over time, in which each feature has its own probability of turning on and, drawn independently of it, of staying on (THEORY-1034). Read it knowing that its sequential buffet misprints both of its predictive rules, that after the first time step the expected number of active features is no longer the IBP’s α (THEORY-1019), that it models one sequence and not several objects over time, and that its experiments are two single runs on mixed speech. |
| LIT-1622 v2 | Beam Sampling for the Infinite Hidden Markov Model | Van Gael, Saatci, Teh & Ghahramani (2008), ICML ‘08: 1088–1095. For the infinite hidden Markov model (the HDP-HMM), an auxiliary variable u_t ~ Uniform(0, π_{s_{t−1}s_t}) per time step leaves only finitely many trajectories with positive probability, so a whole state sequence can be resampled by dynamic programming without truncation. On synthetic data, a well-log changepoint series and character prediction in Alice in Wonderland, the beam sampler mixes in fewer iterations and is less sensitive to initialization and priors than the Gibbs sampler, at a cost per iteration close to O(TK) in practice. | Active — read 2026-10-27 (NOTE-1204) in full, from the authors’ own copy of the ICML paper; worth reading for the beam sampler, which adds a uniform slice variable under each transition so that only finitely many state trajectories of the infinite HMM have positive probability, and then samples a whole trajectory exactly by forward filtering, backward sampling (THEORY-1037). The infinite factorial HMM’s inference is built on it. Read it knowing that its case for faster mixing than Gibbs sampling rests on charts, synthetic data and one changepoint series judged by eye (THEORY-1044), and that one of its fixed hyperparameter settings was chosen from the samplers’ own results. |
| LIT-1623 v2 | Infinite Latent Feature Models and the Indian Buffet Process | Griffiths & Ghahramani (2005), NIPS 18. A finite model in which each of K features has prevalence π_k ~ Beta(α/K, 1) and each of N objects holds it with probability π_k has, as K → ∞, a limit on left-ordered equivalence classes of binary matrices; the Indian buffet process, in which customer i takes each earlier dish with probability m_k/i and then Poisson(α/i) new ones, produces the same distribution. It is used as the prior of an infinite binary linear-Gaussian latent feature model, with a Gibbs sampler, on 100 photographs of up to four objects. | Active — read 2026-10-27 (NOTE-1208) in full, from the NeurIPS proceedings’ own PDF; worth reading as the paper that introduces the Indian buffet process, the limit of a finite beta–Bernoulli feature model taken on left-ordered equivalence classes, and its sequential buffet, which produce the same distribution (THEORY-1043). Read it knowing that its eight pages state the limit and defer the technical details to a technical report, that it gives K+ ~ Poisson(αH_N) only through the buffet, with no count for the finite model, and that its one demonstration is a single run on 100 staged photographs, in which three of the four main features code an object’s absence. |
| LIT-1626 v2 | Infinite Sparse Factor Analysis and Infinite Independent Components Analysis | Knowles & Ghahramani (2007), ICA 2007, LNCS 4666: 381–388. Data are modelled as Y = G(Z ⊙ X) + E, with a potentially infinite number of hidden sources X, a binary mask Z drawn from the one- or two-parameter Indian buffet process, and a Gaussian (infinite sparse factor analysis) or Laplacian (infinite ICA) prior on the sources. A Gibbs sampler with Metropolis–Hastings steps for new features recovers a synthetic mask and beats FastICA in Amari error on sparse synthetic data, and finds tissue-specific gene signatures in ovarian cancer microarray data. | Active — read 2026-10-27 (NOTE-1209) in full, from the authors’ own copy of the ICA 2007 paper; worth reading for infinite sparse factor analysis and infinite ICA, in which an Indian buffet process masks which of unboundedly many sources is active at each data point, the model the infinite factorial HMM’s experiments compare against, and for its sampler, whose ratio for a source being on or off shows how far the data can tell an inactive source from an active one at zero (THEORY-1046). Read it knowing that its results are boxplots and a Hinton diagram, that its text and figure caption disagree on the synthetic data’s size, and that it misstates the one-parameter IBP’s total number of features. |
| LIT-1627 v2 | Stick-breaking Construction for the Indian Buffet Process | Teh, Görür & Ghahramani (2007), AISTATS, PMLR 2: 556–563. In the limit of the IBP’s finite model, the feature probabilities in decreasing order are μ_(k) = ∏_{l≤k} ν_l with ν_l ~ Beta(α, 1) independent; the pieces broken off are the Dirichlet process’s GEM(α) weights. A slice sampler built on it truncates the representation adaptively and needs no conjugacy; a semi-ordered variant, on synthetic linear-Gaussian data, mixes as fast as the conjugate Gibbs sampler, and it is demonstrated on 1000 MNIST threes. | Active — read 2026-10-27 (NOTE-1210) in full, from PMLR’s own PDF; worth reading for the stick-breaking construction of the Indian buffet process, in which the feature probabilities in decreasing order are products of independent Beta(α, 1) draws, for its duality with the Dirichlet process’s sticks, and for the slice samplers it licenses, which the Markov IBP’s inference uses (THEORY-1041). Read it knowing that its truncation is described but its error is not computed, that its sampler comparison is reported only as a chart of autocorrelation times, and that its MNIST demonstration is a single run judged by eye. |
| LIT-1675 v2 | On a Measure of the Information Provided by an Experiment | Lindley (1956). Proposes Shannon’s expected information, the expected reduction in entropy of the prior over parameters, as a measure of what an experiment provides, and uses it to compare and choose experiments. | Unfetched — searched 2026-10-27 for a lawful full text; one exists but this client could not fetch it, so it is not read and no NOTE is filed (ADR-053). Unpaywall lists the publisher’s free copy at Project Euclid (bronze, the published version); it answered curl with a bot challenge (“Just a moment…”), and a headless-browser fetch was not attempted, since it would be working around the challenge. The failure is on access, not availability. A browser fetch of that copy is the cheapest way to read it. It stays Unfetched until a copy is supplied. |
| LIT-1847 | Lecture 4: Statistical decision theory: model distance and equivalence | Han (2021), Stanford EE378C lecture notes, Lecture 4. Defines Le Cam’s deficiency through decision problems: M is ε-deficient relative to N if for every finite parameter subset, finite action set, loss with values in [0, 1] and rule for N, some rule for M has risk at most ε more at every parameter. States that this holds exactly when a stochastic kernel K has sup_θ ‖Q_θ − K P_θ‖_TV ≤ ε, proves the easy direction, and sketches the converse for finite Θ by a minimax swap. Defines Le Cam’s distance as the larger of the two deficiencies, and illustrates it with sufficiency (distance zero exactly for a sufficient statistic), Poissonization and the equivalence of nonparametric regression, white noise and density estimation. | Active — read 2026-10-30 (NOTE-1426) in full, all 26 slides with their handwritten annotations, from the course’s own copy at web.stanford.edu/class/ee378c/lecture4_annotated.pdf. Worth reading as a compact statement of Le Cam’s deficiency for decision problems with [0, 1]-valued losses and of the randomization criterion: a model is ε-deficient relative to another exactly when some stochastic kernel brings it within ε of the other in total variation, uniformly in θ. Read it knowing that only the easy direction (a kernel within ε bounds every risk gap by ε) is proved line by line; the converse is sketched for a finite parameter set by one inf–sup swap, with the minimax theorem’s conditions and the finiteness of the observation spaces left unstated, and the passage to an infinite parameter set is not given. |
| LIT-1859 | On Experiments | van Rooyen (arXiv 2025). Experiments are Markov transitions from a finite set of unknowns to a finite observation set. A general data processing theorem: any risk minimised over the strategies an experiment allows is monotone under garbling. Blackwell–Sherman–Stein follows. With a prior π fixed, the directed deficiency ξ^π(e, e′) is the least, over garblings f, of the π-average total variation between f∘e and e′; Theorem 26 (Le Cam’s randomization theorem) says ξ^π ≤ ε exactly when e’s minimum Bayes risk is within ε‖L‖∞ of e′‘s for every action set and loss. The symmetrised deficiency is a metric for a prior of full support, and ξ^π is a linear programme. | Active — read 2026-10-30 (NOTE-1430), Skimmed: §§1–3 and 5–8 in full, with Theorem 26 and its proof checked line by line, from arXiv’s v2 (13 October 2025, 23 pages); §4, on representing loss functions, was not read. Worth reading as a short, self-contained proof of Le Cam’s randomization theorem for finite experiments against a fixed prior, after Torgersen: the least prior-averaged total variation between a garbling of e and e′ is at most ε exactly when e’s minimum Bayes risk exceeds e′‘s by at most ε‖L‖∞ for every action set and loss. Read it knowing that the result is for one prior, not uniform in the unknown; that its reverse implication, and so the theorem as stated for every loss, loses a factor of 2 for losses that take both signs (it holds for losses in [0, 1]); that the saddle point it invokes is more than Sion’s theorem gives, though the proof needs only the minimising rule; and that Lemma 30’s linear programme computes twice the directed deficiency. |
| LIT-1882 v2 | Correspondence between thermodynamics and inference | LaMont & Wiggins (2019). Treats the Bayesian evidence as a partition function with sample size N as inverse temperature and derives the thermodynamic quantities. The internal energy is the leave-one-out predictive loss, whose excess K/2N is equipartition. The heat capacity becomes a learning capacity, K/2 in a regular model and the learning coefficient in a singular one, lower where the prior or discreteness fixes parameters (“freeze-out”), proposed as the definition of sloppiness. A prior chosen to make the Gibbs entropy zero is Jeffreys-like, depends on N, reproduces AIC and avoids the Lindley–Bartlett paradox in one simulation. | Active — read 2026-10-31 (NOTE-1481; appendices B, C1, C3, C4 skimmed); worth reading as the fullest working of the evidence-as-partition-function dictionary with sample size as inverse temperature: generalization error as equipartition, a “learning capacity” equal to K/2 in regular and to the RLCT in singular models (THEORY-1253), and a sample-size-dependent prior that reproduces AIC. The dictionary is formal; it has no physical currents or dissipation. |
| LIT-1884 v2 | Why Self-Training Helps and Hurts: Denoising vs. Signal Forgetting | Wu, Yang & Sun (2026). In overparameterized linear regression, iterated self-training with fresh covariates, noiseless pseudo-labels and minimum-norm students projects the estimate each round onto the row space of a new design. Noise from the first fit decays at (1 + τ)⁻¹ per round, and signal along a direction of variance s at s/(s + τ). Risk is U-shaped under anisotropy, monotone under isotropy, and tends to that of predicting zero. Deterministic equivalents concentrate uniformly over ridge parameters, and an iterated GCV picks the stopping round. One CIFAR-10 experiment shows the U-shape. | Active — read 2026-10-31 (NOTE-1471), body and appendices, with the proofs skimmed; worth reading for an exact account of iterated reconstruction through fresh low-rank views, in which forgetting the ancestor first improves and then destroys fidelity to the world, and every student fits its teacher exactly on its own sample (THEORY-1269). |