Gang Fang's Blog

It helps IID and OOD generalization because lower level features in smaller quantities can be combined in different way to generalize to something out of distribution of the training data. It has forms such as:

• Distributed representation: http://stanford.edu/~jlmcc/papers/PDP/Chapter3.pdf
• Compositions of layers in an ANN
• Compositions in Language

And the Bengio’s goal is to introduce more forms of compositionality.