mathematical-statistics.
23 of 1538 NOTE documents. Back to the full index.
| # | Title | Summary | Status |
|---|---|---|---|
| NOTE-588 | Fritz — A synthetic approach to Markov kernels | Markov categories (symmetric monoidal, a commutative copy/discard comonoid on every object, discard natural) carry enough structure to define determinism, conditionals, four kinds of conditional independence with the semigraphoid rules, almost-sure equality, sufficiency, completeness and ancillarity. Basu’s theorem then holds in every Markov category; Fisher–Neyman (Theorem 14.5) and Bahadur (Theorem 16.3) need strict positivity, which existence of conditionals implies but which Stoch has without them. The abstract Fisher–Neyman is checked against the classical one only in FinStoch. | Read |
| NOTE-595 | Equivalent Comparisons of Experiments | Gives a new minimax proof of the Sherman–Stein theorem (for finite outcome spaces, an experiment at least as informative as another in every decision problem can reproduce it by a Markov matrix) and, by a martingale argument, removes the finiteness restriction, so for a finite parameter set “more informative” and “sufficient for” coincide in general. Adds k-decision problems, with three equivalent forms, and shows that for dichotomies two-action problems, and so the type I/type II error trade-off, already decide the order. | Read |
| NOTE-728 | Brémaud & Massoulié — Stability of nonlinear Hawkes processes | Sufficient conditions for a Hawkes process with a nonlinear rate function to have a unique stationary version and for any suitably started version to converge to it, in distribution or in variation. The main conditions are a Lipschitz rate function with Lipschitz constant times the kernel’s absolute mass below one (a spectral radius below one for a network), or a bounded Lipschitz rate with no limit on the kernel’s size. Kernels may be negative, so inhibition is allowed. Kernels are fixed in time, the conditions are sufficient and not necessary, and nothing is said about cascade sizes or about what happens past the threshold. | Read |
| NOTE-848 | Dwork et al. — The reusable holdout (supplementary material) | From the Supplementary Material: if the max-information between the dataset and the analyst’s chosen function is at most log₂e·τ²n, the chance that the function’s empirical mean is more than τ above its true mean is at most e^{−τ²n}; pure ε-differential privacy bounds max-information by log₂e·εn. Thresholdout answers m adaptively chosen statistical queries on a holdout of size n = O(ln(m/β)/τ² · min{B, √(B·ln(m/β))/τ}) to accuracy τ with probability 1 − β, where B budgets the times the training estimate is found to overfit. The main text was not read. | Skimmed |
| NOTE-933 | Donoho & Tanner — observed universality of phase transitions | For Gaussian matrices, ℓ1 minimization (and its nonnegative LP form) recovers a k-sparse vector from n random measurements in dimension N with probability tending to 1 below a curve ρ(δ; Q) in the plane δ = n/N, ρ = k/n, and to 0 above it; this was proved earlier and is restated here as Theorem 2.1. In 2,948,000 linear programs over eight non-Gaussian ensembles at N = 200, 400 and 1600, the 50% success points fall on the same curves, but exact finite-N agreement is rejected: the differences from the Gaussian decay as N^(-1/2), so the paper supports only an asymptotic (“weak”) universality, as an observation, not a theorem. | Read |
| NOTE-937 | Bertschinger, Rauh, Olbrich, Jost and Ay — Quantifying unique information | For a target X and sources Y, Z, the unique information of Y is UĨ(X : Y \ Z) = min over Q ∈ Δ_P of MI_Q(X : Y | Z), where Δ_P is the set of joints with the same (X,Y) and (X,Z) marginals. If unique information should depend only on those marginals, this is an upper bound on any such measure, and the shared and synergistic information it implies are lower bounds. It is zero exactly when the other source is Blackwell-sufficient. The measures are non-negative, satisfy the bivariate PI axioms and the identity property, and are a convex programme, but are bivariate only. | Read |
| NOTE-938 | Kraskov, Stögbauer and Grassberger — Estimating mutual information | Two k-nearest-neighbour estimators of mutual information, I⁽¹⁾ and I⁽²⁾ (Eqs. 8–9), from the Kozachenko–Leonenko entropy estimate (Eq. 20) taken at one length scale in the joint and marginal spaces, so that the biases largely cancel. The only approximation is a locally constant density. Numerically the systematic error scales with k/N and vanishes for independent variables, a conjecture without proof; the ordered Weinman exponential is the one case where the stronger form fails. Extensions to multi-information (Eqs. 23, 30). Conditional mutual information is not treated. | Read |
| NOTE-941 | Eichler and Didelez — Causal reasoning in graphical time series models | Defines the causal effect of intervening on one component of a stationary multivariate time series on another at a later time, and relates it to strong Granger noncausality through mixed graphs. Under Assumption 2.3 (an intervention independent of the system’s past and of the other contemporaneous variables, and acting on the future only through the present), a missing Granger edge a → b means no effect on b at the next step, a ∉ an(b) means no effect at any horizon, and time-series back-door and front-door criteria identify effects from observational data. A present edge is only “potentially causal”. | Read |
| NOTE-1099 | Statistical ranking and combinatorial Hodge theory | Rank aggregation as combinatorial Hodge theory. Aggregated pairwise comparisons form an edge flow, and its orthogonal decomposition gives a least-squares global ranking (the gradient), local inconsistency (curl) and global inconsistency (harmonic). The size of the cyclic residual certifies whether any global ranking is meaningful. The framework is cardinal, generalizes Borda, and is not Kemeny. | Read |
| NOTE-1161 | Chaloner and Verdinelli — Bayesian experimental design: a review | Bayesian experimental design is Lindley’s decision problem: choose the design that maximizes the expected utility of the best terminal decision, so the criterion follows from what the experiment is for. In the normal linear model Shannon information gives Bayes D-optimality, det(nM + R), and quadratic loss gives Bayes A-optimality, tr A(nM + R)^-1. E- and G-optimality have no utility behind them. In nonlinear problems the criteria are normal approximations that average a local criterion over the prior. Locally optimal design is the point-mass case, and it can leave the quantity of interest inestimable. | Read |
| NOTE-1165 | Wald — Sequential tests of statistical hypotheses | Defines the sequential probability ratio test: sample until the likelihood ratio of two simple hypotheses leaves an interval (B, A). Exact inequalities tie A and B to the error rates, so A = (1 − β)/α and B = β/(1 − α) are set with no distribution theory, and the realized errors can rise above α or β only slightly. It derives the operating characteristic and the expected sample size. It proves that no test with the same errors needs fewer observations on average under either hypothesis, once the overshoot of the boundaries is neglected, and for a normal mean it saves about half the fixed-sample test. It never mentions diffusion or Brownian motion. | Read |
| NOTE-1171 | Thibaux and Jordan — Hierarchical beta processes and the Indian buffet process | The beta process BP(c, B0) is the de Finetti mixing measure of the Indian buffet process: by beta–Bernoulli conjugacy, customer n + 1 takes dish j with probability m_{n,j}/(c + n) and Poisson(cγ/(c + n)) new dishes, which is the IBP at (c, γ) = (1, α) and its two-parameter generalisation otherwise. A hierarchy of beta processes lets groups share features, which requires a discrete shared base, at group-specific prevalences Beta(c_j b, c_j(1 − b)). On one unbalanced 20 Newsgroups split it beats tuned Laplace-smoothed naive Bayes, 58% against 50%. | Read |
| NOTE-1172 v3 | Griffiths and Ghahramani — The Indian buffet process: an introduction and review | Derives the Indian buffet process as the K → ∞ limit of a finite beta–Bernoulli model, πk ~ Beta(α/K, 1), on left-ordered equivalence classes of binary matrices: the number of occupied features K+ is Poisson(αH_N) and each object holds Poisson(α) features. For finite K the paper gives only the bound E[1ᵀZ1] = Nα/(1 + α/K) ≤ Nα; the αH_N count holds in the limit, and at finite K the expected count is strictly below it. It reviews applications, the two-parameter, stick-breaking, beta process, Markov and phylogenetic variants, and names column independence as an assumption later relaxed. | Read |
| NOTE-1175 v2 | Williamson, Orbanz and Ghahramani — Dependent Indian buffet processes | Couples IBPs over a covariate by thresholding Gaussian processes: z_nk(t) = 1 when h_nk(t) falls below the b_k quantile of its marginal, with stick-breaking sticks b_k that do not depend on t. Each item’s indicator is then Bernoulli(b_k) at every t, so prevalence is fixed and persistence is set apart from it by the GP covariance. The paper says Z(t) is IBP-distributed at each t; that holds exactly in its collapsed model, while with a shared feature-level GP the items at one t are correlated and fewer features are occupied. Truncated Gibbs with HMC; lower held-out RMSE than the IBP on two datasets. | Read |
| NOTE-1199 v2 | Van Gael, Teh and Ghahramani — The infinite factorial hidden Markov model | Introduces the Markov Indian buffet process: each feature is a two-state Markov chain over time steps, starting off, with its own probability of switching on, a_m ~ Beta(α/M, 1), and of staying on, b_m ~ Beta(γ, δ), drawn independently. In the limit the number of features ever on is Poisson(αH_T). Used as the prior of an infinite factorial HMM, it recovers who is speaking when in mixed speech better than an IBP prior. The sequential buffet as printed has both of its predictive rules wrong. | Read |
| NOTE-1204 | Van Gael, Saatci, Teh and Ghahramani — Beam sampling for the infinite hidden Markov model | Introduces the beam sampler for the infinite HMM: a slice variable u_t ~ Uniform(0, π_{s_{t−1}s_t}) at each step cuts the infinitely many transitions out of each state to the finitely many above u_t, so whole trajectories are sampled exactly by forward filtering, backward sampling. Against the Gibbs sampler it mixes in fewer iterations on strongly correlated synthetic sequences, is less sensitive to the hyperprior, handles a non-conjugate Cauchy likelihood on well-log data, and costs close to O(TK) per iteration in practice; the comparisons are charts and visual judgements. | Read |
| NOTE-1208 | Griffiths and Ghahramani — Infinite latent feature models and the Indian buffet process | Introduces the Indian buffet process: the finite beta–Bernoulli feature model with π_k ~ Beta(α/K, 1) has a limit as K → ∞ on left-ordered equivalence classes, eq. (4), and a sequential buffet produces the same distribution, so objects are exchangeable and K+ ~ Poisson(αH_N). Used as the prior of an infinite binary linear-Gaussian model, its Gibbs sampler settles on about seven features for 100 photographs of four objects. The paper is the original of the authors’ 2011 review. | Read |
| NOTE-1209 | Knowles and Ghahramani — Infinite sparse factor analysis and infinite independent components analysis | Defines infinite sparse factor analysis and infinite ICA: Y = G(Z ⊙ X) + E, an IBP mask Z over unboundedly many sources, Gaussian or Laplacian sources, and four variants with the one- or two-parameter IBP. Its Gibbs sampler integrates a source’s value out when deciding whether it is active. On 30 sparse synthetic data sets the IBP variants beat FastICA in Amari error; on ovarian cancer microarray data it finds a tissue-specific signature also found by ICA. Results are figures only, and the text misstates the IBP’s total number of features. | Read |
| NOTE-1210 | Teh, Görür and Ghahramani — Stick-breaking construction for the Indian buffet process | Derives the IBP’s stick-breaking construction: in decreasing order the feature probabilities are μ_(k) = ∏{l≤k} ν_l, ν_l ~ Beta(α, 1), so E[μ(k)] = (α/(α + 1))^k, and the discarded pieces are the Dirichlet process’s sticks. Slice samplers built on it, one ordered and one semi-ordered, need no conjugacy; on 120 synthetic datasets the semi-ordered one mixes as fast as Gibbs. The truncation CASE-165 quotes, 8.97 features at K = 16, N = 12, α = 3, is exact here at 8.974. | Read |
| NOTE-1426 | Han — Model distance and equivalence (EE378C Lecture 4) | The lecture defines Le Cam’s deficiency by decision problems with losses in [0, 1], states that a model is ε-deficient relative to another exactly when a stochastic kernel brings it within ε in total variation uniformly in θ, proves the easy direction in four lines, and sketches the converse for finite Θ by one inf–sup swap whose minimax conditions are not stated. Le Cam’s distance is the larger of the two deficiencies; sufficiency, Poissonization and the regression–white-noise–density equivalences are its examples. | Read |
| NOTE-1430 | van Rooyen — On Experiments | For finite sets and a fixed prior π, Theorem 26 proves Le Cam’s randomization theorem: the least π-averaged total variation between a garbling of e and e′ is at most ε exactly when e’s minimum Bayes risk exceeds e′‘s by at most ε‖L‖∞ for every action set and loss. The proof is a minimax argument after Torgersen. It is right for losses in [0, 1] and loses a factor of 2 for signed ones, and it is not the uniform statement; the uniform one follows from it at every prior with one more minimax exchange. | Skimmed |
| NOTE-1471 | Wu, Yang & Sun — Self-training: denoising vs. signal forgetting | In overparameterized linear regression, iterating self-training with fresh covariates, noiseless pseudo-labels and minimum-norm students multiplies the estimate by a fresh random orthogonal projection each round. Noise inherited from the first fit decays geometrically; signal along a direction of variance s decays at the slower rate s/(s + τ). Risk is therefore U-shaped when the covariance is anisotropic, monotone when it is isotropic, and tends to the risk of predicting zero. Each student fits its teacher exactly on its own sample while losing the teacher’s function elsewhere. | Read |
| NOTE-1481 | LaMont & Wiggins — thermodynamics and inference | Takes the evidence as a partition function with sample size N as inverse temperature, and reads off the thermodynamic derivatives. The internal energy is the leave-one-out predictive loss; the heat capacity becomes a “learning capacity” equal to K/2 in a regular model (equipartition) and to Watanabe’s learning coefficient in a singular one, and smaller when prior or discreteness fixes parameters (“freeze-out”). A prior making the Gibbs entropy zero reproduces AIC. The correspondence is formal; nothing in it is a physical current or dissipation. | Read |