nucleation

QUESTION-182: Does the landmark readout of a learned contrastive density ratio keep its 1/m rate under a finite second moment of the ratio alone, without a uniform bound on it?

1 min read

Properties4
StatusOpen
Tagsrepresentation-learning, learning-theory
Linepragmatic-transport
RefinesQUESTION-052 — Beyond linear decisions at the population level, if a kind's admissible transformations are given as a distribution over pairs of its realizations, is the top eigenspace of the pair operator, or the top left singular space when the transformations are directed, its stable, decision-sufficient signature, and what makes one span the signature when other spans tie?

Graph View

Why it is a question

THEORY-319 now states that, for the exact ratio g* = p(x,z)/(p(x)p(z)), a bounded label and E[g*²] finite, a linear readout of m landmark evaluations comes within O(1/m) of the infer-then-predict function in mean squared error. The source’s error-propagation results for a learned ratio (Theorems 10–11) add terms in (1 + g_max)²√ε_opt and (1 + g_max)⁴ε_opt, where g_max is a uniform bound on the ratio. Many natural joint laws have an unbounded ratio with a finite second moment, the paper’s Gaussian model among them.

What would answer it

A bound on the readout’s excess risk in terms of the excess logistic (or direct) loss and E[g²], with no g_max, would answer it yes. A family of joint laws with E[g²] bounded and learned ratios of vanishing excess loss whose landmark readouts stay far from the infer-then-predict function would answer it no. What cannot settle it: linear-probe accuracies of trained contrastive networks, where neither the ratio nor its excess loss is measured.

Backlinks