Commit
·
d94e4de
1
Parent(s):
7fd184b
Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# This repo let's you run the following checkpoint using facebookresearch/metaseq.
|
| 2 |
+
|
| 3 |
+
Do the following:
|
| 4 |
+
|
| 5 |
+
## 1. Install PyTorch
|
| 6 |
+
```
|
| 7 |
+
pip3 install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio==0.10.1+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
|
| 8 |
+
```
|
| 9 |
+
|
| 10 |
+
## 2. Install Megatron
|
| 11 |
+
```
|
| 12 |
+
git clone https://github.com/patrickvonplaten/Megatron-LM.git
|
| 13 |
+
cd Megatron-LM
|
| 14 |
+
pip3 install six regex
|
| 15 |
+
pip3 install -e .
|
| 16 |
+
```
|
| 17 |
+
## 3. Install fairscale
|
| 18 |
+
```
|
| 19 |
+
git clone https://github.com/facebookresearch/fairscale.git
|
| 20 |
+
cd fairscale
|
| 21 |
+
git checkout prefetch_fsdp_params_simple
|
| 22 |
+
pip3 install -e .
|
| 23 |
+
```
|
| 24 |
+
## 4. Install metaseq
|
| 25 |
+
```
|
| 26 |
+
git clone https://github.com/patrickvonplaten/metaseq.git
|
| 27 |
+
cd metaseq
|
| 28 |
+
pip3 install -e .
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
## 5. Clone this repo (click top right on "How to clone")
|
| 32 |
+
|
| 33 |
+
## 6.
|
| 34 |
+
|
| 35 |
+
```bash
|
| 36 |
+
cd <path/to/cloned/repo>
|
| 37 |
+
bash run.sh
|
| 38 |
+
```
|