microsoft/unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities. Fundamental research to improve modeling generality and capability, as well as training stability and efficiency for Transformers at any scale.