pip install -r requirements.txt

You will also need to install updated DeepSpeed version (>0.7.0), which contains the compression library.

The python code is modified based on HuggingFace's PyTorch text_classification. The key added feature is the implementation of knowledge distillation (KD)（--distill_method one_stage). If no KD, run (--distill_method zero_stage).

• config: This folder provides DeepSpeed configuration, including quantization, pruning and layer reduction.
• huggingface_transformer: This folder serves the implementation of knowledge distillation. It's based on HuggingFace's transformer The change is line 383, where we output attention_scores instead of attention_prob.
• bash_script This folder contains many bash scripts for various kinds of compression. See more descriptions and results in our tutorial page.