no code implementations • 22 Oct 2021 • Yizhang Lou, Chris Mingard, Yoonsoo Nam, Soufiane Hayou
Recent work by Baratin et al. (2021) sheds light on an intriguing pattern that occurs during the training of deep neural networks: some layers align much more with data compared to other layers (where the alignment is defined as the euclidean product of the tangent features matrix and the data labels matrix).
