Skip to content

Commit 49362b1

Browse files
Tutorial combining DDP with Pipeline Parallelism to Train Transformer models (#1347)
* Tutorial combining DDP with Pipeline Parallelism to Train Transformer models. Summary: Tutorial which places a pipe on GPUs 0 and 1 and another Pipe on GPUs 2 and 3. Both pipe replicas are replicated via DDP. One process drives GPUs 0 and 1 and another drives GPUs 2 and 3. * Polish out some of the docs. * Add thumbnail and address some comments. Co-authored-by: pritam <pritam.damania@fb.com>
1 parent 07b6340 commit 49362b1

File tree

3 files changed

+472
-0
lines changed

3 files changed

+472
-0
lines changed
34.9 KB
Loading

0 commit comments

Comments
 (0)