Source
Airoldi, Blei, Fienberg and Xing (2008), LIT-1553, §4.3.1–4.3.2, Table 2 and Figs. 10–12, read in full in NOTE-1179.
What was actually shown
871 proteins from MIPS’s hand-curated physical interactions, which exclude high-throughput data, were fitted with K = 15. MIPS also gives each protein its top-level functions, 15 in all, “with an average of ≈ 2.4 functional annotations for each protein”. The annotations were not used in fitting. Because “the K = 15 blocks in the blockmodel B are not directly identifiable in terms of functional categories”, a permutation from blocks to functions was estimated greedily, to maximize accuracy on a sample of 87 proteins, with memberships thresholded at τ, the 95th percentile of those proteins’ membership values. Applied to the remaining 90% of the proteins: “The accuracy on the 90% testing set is about 87%. An algorithm that randomly guesses annotations, knowing the right proportions of annotations in each category, leads to a baseline accuracy of about 70%.”
What could have come out otherwise: memberships fitted to interactions alone could have carried no functional information, and scored at the baseline.
Why it is Proposed
The paper does not define accuracy. If it is the share of correct protein–function entries, the base rate of an annotation is about 2.4/15 = 0.16. Then random guessing at the right proportions scores about 0.16² + 0.84² ≈ 0.73, close to the paper’s “about 70%”, which makes the per-entry reading likely. But predicting that no protein has any function scores about 0.84 on that reading, so “about 87%” is three points above the trivial predictor, not seventeen. The threshold at the 95th percentile predicts about 5% of entries present, against 16% true, which also bounds what any mapping could reach. The record’s arithmetic, from the paper’s own numbers. The direction of the result may well stand. Its size is not shown.
What this does not say
- Not that blocks are functions. The mapping is chosen to fit, and proteins have several functions each, so a block can match a function partly.
- Not about §4.3.3. The K = 50 analysis of predicted interactions’ functional precision is a separate result, and not filed.
- Not that memberships are identified. The permutation the paper estimates is the part of the gauge it admits (THEORY-993).
Connections
- CLAIM-427. Recovering structure in the model’s own terms counts for nothing there. This is the right kind of check, structure against labels the model never saw, done with too weak a baseline to count for much yet.
- QUESTION-276. An inferred mixed membership of an organization’s members would need the same kind of external test, against a stated trivial predictor.