MobileBERT: Task-Agnostic Compression of BERT by Progressive

We develop a task-agnosticlly compressed BERT, which is 4.3x smaller and 4.0x faster than BERT-BASE while achieving competitive performance on GLUE and SQuAD.