Rives, Meier, Sercu, Goyal, Lin, Liu, Guo, Ott, Zitnick, Ma and Fergus
(2021) — doi:10.1073/pnas.2016239118.
Standing
The trunk of the biological-sequence line, and the record’s first document in it. Until this filing the corpus held no protein, RNA or genomic sequence model at all, which is why #243’s lesson applies here: a descendant filed onto an empty trunk cannot declare the relations that say what it is. This is filed so LIT-506 can.
Not read. This record holds no NOTE on it and has not checked its
numbers against its tables. It is filed on its standing in the field and on
its role in LIT-506, which cites it as reference 66 and takes its
downstream architecture from it verbatim — “Similar to ESM-1b, our deep
residual network consists of 32 blocks, where each contains two convolution
layers with a filter size of 64.” Anything this record says about its
contents beyond that citation would be recall rather than reading, and the
summary above is deliberately at the level the citation supports.
What a reading would be for. The claim that structure and function emerge from sequence statistics alone is the load-bearing one for everything downstream of it, including the practice this record declined to file from LIT-506. Two things would need checking: whether the structural information is recovered by a linear probe or by a trained head, which decides how much the pretraining is doing; and how the evaluation splits relate to the pretraining corpus, which is the question NOTE-252 raises about the RNA descendant and cannot answer for either.