nucleation

THEORY-1040: The units of the early layers of trained vision networks fall largely into neuron families, sets of variants of one feature across orientation, colour or scale, which reflect structure in the network rather than a convenience of the person sorting them, and the same families form across models, as Olah, Cammarata, Schubert, Goh, Petrov and Carter propose

2 min read

Properties4
StatusProposed
Tagsrepresentation-learning
SourceLIT-1624 — An Overview of Early Vision in InceptionV1
ExtendsTHEORY-1042 — In the Overview of Early Vision's taxonomy of InceptionV1, every one of the 1,056 units of conv2d0 to mixed3b is assigned by the authors to exactly one of 77 groups, and 391 of them (37%), from 7% of conv2d2 to 48% of conv2d1 and 45% of mixed3b, are in groups whose own captions call them catch-all, broad and not well defined, low-confidence, unstudied or not understood; the mixed3b curve family is 10 of that layer's 480 units

Graph View

Source

Olah et al. (2020), An Overview of Early Vision in InceptionV1, LIT-1624, read in full in NOTE-1206: Playing Cards with Neurons, Caveats and the Conclusion.

What is asserted

“Our experience is that many neurons in vision models seem to fall into families of similar features”, “it’s not unusual to see a dozen neurons detecting the same feature in different orientations or colors”, and “the same ‘neuron families’ seem to recur across models!” Some families are suspected “real”: “some can be recovered from factorizing the layer’s weight matrices”. Early vision “seems most likely to be universal”.

What the record holds toward it

  • One model, sorted by eye (THEORY-1042): 77 groups over 1,056 units, 37% flagged by the authors as uncertain or catch-all.
  • Equivariant families in depth for the curve circuit: Curve Circuits (LIT-1629) finds rotated copies of one unit across Gabor, line and early-curve families, which is the kind of structure a family would have if it were real.
  • Another model’s first layer only: the TF-Slim InceptionV1, whose conv2d0 is crisper, shown in a footnote.

Why it stays Proposed

The factorization that would make a family more than a label is mentioned, not shown, and no second model is sorted. The authors list the question as open: “To what extent do these feature families reflect fundamental clusters in features, versus a taxonomy that might be helpful to humans but is ultimately somewhat arbitrary?”

What would confirm or reject it

The promote_when above.

Backlinks