anthology-of-the-sota

LIT-579: Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark

2 min read

Properties6
StatusActive
Tagsgraphs-and-networks, analysis-and-evaluation, model-architecture
Compared againstLIT-580 — How Powerful are Graph Neural Networks?
Cited as “introduced by” bySOTA-349 — Before crediting a graph transformer with long-range gains, retune the message-passing baselines under the same budget, including an MLP prediction head, positional encodings and input feature normalization
Cited as “paper” byNOTE-317 — Where did the gap go?
Cited as “source” bySOTA-349 — Before crediting a graph transformer with long-range gains, retune the message-passing baselines under the same budget, including an MLP prediction head, positional encodings and input feature normalization

Graph View

Tönshoff, Ritzert, Rosenbluth, Grohe, RWTH Aachen and Göttingen (2023) — ARXIV-2309.00367

Key takeaways

  • Peptides (Table 1a): after tuning, GCN reaches AP 0.686 (reported 0.593) and MAE 0.246 (reported 0.350). GINE and GatedGCN improve similarly. GPS re-tuned is unchanged (0.653, 0.251). Every MPGNN now beats GPS, and GCN matches the best published result on Peptides-struct
  • The head (Figure 1b): swapping the linear prediction head for a 2-layer MLP, with nothing else changed, accounts for most of the MPGNN gain, and almost all of it on Peptides-struct. GPS is insensitive to the head
  • Superpixels (Table 2): channel-wise feature normalization helps every model. It accounts for at least half the gain for all but GatedGCN, where it mainly cuts seed variance. After tuning, GatedGCN reaches F1 0.388 on PascalVOC-SP, above GPS’s originally reported 0.375. GPS itself rises to 0.444 and stays ahead
  • PCQM-Contact: the official code computes raw MRR, not the filtered MRR the benchmark defines. Self-loop candidates also bias the symmetric scoring function

Standing in the anthology

Filed from #163 (“graph representation / gnn”). It is compared_against GIN (LIT-580), whose edge-feature variant GINE is one of the retuned baselines. It sources SOTA-349.

Scope of “the gap vanished”. It vanished on the two Peptides datasets. On PascalVOC-SP and COCO-SP, tuned GPS still leads tuned MPGNNs, by 5.6 and 9.6 F1 points. The paper is a short preprint with a 500k-parameter budget, and the LRGB paper it re-examines is not in the record.

Read — NOTE-317.

Backlinks