Multi GPU and Multi Node

Jump to bottom Edit New page

afiaka87 edited this page Apr 12, 2021 · 4 revisions

Distributed Backends

We have two backends you may use to run on a multi-GPU setup.

Deepspeed

https://github.com/lucidrains/DALLE-pytorch/wiki/Deepspeed

Horovod

https://github.com/lucidrains/DALLE-pytorch/wiki/Horovod

lord krishna with arjun