anthology-of-the-sota

LIT-583: Highly accurate protein structure prediction with AlphaFold

3 min read

Properties7
StatusActive
Tagsbiomolecular-modeling, model-architecture, data-pipeline
Extended byLIT-584 — Accurate structure prediction of biomolecular interactions with AlphaFold 3, LIT-754 — Efficient AlphaFold2 Training using Parallel Evoformer and Branch Parallelism
Compared againstLIT-754 — Efficient AlphaFold2 Training using Parallel Evoformer and Branch Parallelism
Cited as “introduced by” bySOTA-353 — Train a head that predicts the model's own accuracy on each output, against the metric you care about, and use it to rank samples and filter predictions, SOTA-354 — When unlabelled inputs vastly outnumber labelled ones, retrain from scratch on the labelled data mixed with the model's own high-confidence predictions for unlabelled inputs, under augmentation that stops it copying them
Cited as “paper” byNOTE-321 — AlphaFold 2
Cited as “source” bySOTA-353 — Train a head that predicts the model's own accuracy on each output, against the metric you care about, and use it to rank samples and filter predictions, SOTA-354 — When unlabelled inputs vastly outnumber labelled ones, retrain from scratch on the labelled data mixed with the model's own high-confidence predictions for unlabelled inputs, under augmentation that stops it copying them, THEORY-084 — AlphaFold 2 needs the sequence alignment to find a protein's coarse fold, not to refine it, so accuracy collapses below a minimum alignment depth and barely improves above it

Graph View

Jumper, Evans, Pritzel, Green, Figurnov, Ronneberger et al., DeepMind (Nature 596, 2021) — DOI 10.1038/s41586-021-03819-2

Key takeaways

  • Accuracy: CASP14 median backbone 0.96 Å r.m.s.d.95 (95% CI 0.85–1.16), next best 2.8 Å. All-atom 1.5 Å against 3.5 Å. The accuracy holds on 10,795 PDB chains released after the training cutoff
  • Architecture: the Evoformer (48 blocks) updates an MSA representation and a pair representation, with triangle updates and attention on the pair. A structure module with invariant point attention builds per-residue frames, trained end to end with the frame-aligned point error (FAPE). “Recycling” runs the whole network three more times on its own outputs
  • Confidence (Figure 2c, d): the predicted lDDT (pLDDT) fits true lDDT-Cα as 0.997·pLDDT − 1.17 (r = 0.76), and predicted TM-score fits TM-score with r = 0.85
  • Self-distillation: predict structures for about 350,000 Uniclust30 sequences, keep a high-confidence subset, and retrain the same architecture from scratch on PDB plus these. Cropping and MSA subsampling make copying hard. The ablation shows it improving accuracy (Figure 4a). A BERT-style masked-MSA loss is trained jointly, not as pretraining
  • Ablations (Figure 4a, 87 CASP14 domains and 2,261 low-template PDB chains): removing invariant point attention together with recycling, or end-to-end structure gradients, costs the most. Removing templates changes little. The numbers are plotted, not tabulated
  • Limits (Figure 5): accuracy drops sharply when the median alignment depth is below about 30 sequences, with small gains above about 100. Removing both metagenomic databases costs 6.1 GDT on CASP14, mostly through a few targets losing 20 or more. Chains whose shape comes from other chains in a complex are predicted poorly

Standing in the anthology

Filed from #163 (“AlphaFold”). It was the item that prompted biomolecular-modeling (ADR-059). It is extended_by AlphaFold 3 (LIT-584). It sources SOTA-354 (self-distillation), SOTA-353 (a head that predicts the model’s own accuracy) and THEORY-084 (what the alignment is for).

Which evidence is which. CASP14 is a blind test and the strongest evidence here. The ablations are three-seed comparisons shown only as a plot, so the practices cite their direction, not magnitudes. The protein language model line in the record (LIT-505) is the MSA-free alternative, and ESMFold, which puts the two together, is not filed.

Read — NOTE-321.

Parallel Evoformer (LIT-754) is a measured comparison against this paper’s block: it moves only the outer product mean so the two tracks run independently, trains the original Evoformer from scratch as one arm of its accuracy comparison and times it as one arm of its speed table, and the original block matches the parallel one on accuracy. AlphaFold 2’s own end-to- end training time is quoted there, not rerun.

Backlinks