anthology-of-the-sota

LIT-041: Clusterability in Neural Networks

1 min read

Properties3
StatusRejected — the result does not look interesting enough to carry
Tagsanalysis-and-evaluation
SupersedesLIT-031 — Pruned Neural Networks are Surprisingly Modular

Graph View

Filan et al. (2021) — ARXIV-2103.03386

Key takeaways

  • Larger models develop more modular features
  • More modular learning reduces need for exploration via schedule variation
  • Spectral regularization promotes parameter clustering, but can be costly

Standing in the anthology

just doesn’t seem that interesting. Not sure why Claude recommended this. Hallucination?

Backlinks