Bringing Generative Learning to Representation Learning: Self-Supervised Transfer Learning as Distribution Matching
Researchers have proposed a new framework for self-supervised learning called Distribution Matching (DM). DM aims to learn an augmentation-invariant encoder that maps data to a designed reference law. This approach is based on the idea of matching the learned representation distribution to an explicit geometric reference, rather than leaving it unspecified as in previous methods. The authors use Mallows distance to measure deviations from the target representation law and pro
Researchers have proposed a new framework for self-supervised learning called Distribution Matching (DM). DM aims to learn an augmentation-invariant encoder that maps data to a designed reference law. This approach is based on the idea of matching the learned representation distribution to an explicit geometric reference, rather than leaving it unspecified as in previous methods. The authors use Mallows distance to measure deviations from the target representation law and prove a non-asymptotic neural-sieve guarantee. They also demonstrate the effectiveness of DM through simulations and image benchmarks.
---
Why it matters: This work is important for researchers in AI because it provides a new framework for self-supervised learning that can lead to more robust and transferable representations. The ability to learn augmentation-invariant encoders could improve the performance of deep learning models on various tasks, including classification and clustering.
Source: https://arxiv.org/abs/2502.14424
This article was originally published at: https://arxiv.org/abs/2502.14424