Skip to content

Add a prerequisites card #2967

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions _static/css/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -71,3 +71,23 @@
.sd-card:hover:after {
transform: scaleX(1);
}

.card-prerequisites:hover {
transition: none;
border: none;
}

.card-prerequisites:hover:after {
transition: none;
transform: none;
}

.card-prerequisites:after {
display: block;
content: '';
border-bottom: none;
background-color: #fff;
transform: none;
transition: none;
transform-origin: none;
}
10 changes: 6 additions & 4 deletions advanced_source/cpp_custom_ops.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,16 @@ Custom C++ and CUDA Operators
.. grid:: 2

.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
:class-card: card-prerequisites

* How to integrate custom operators written in C++/CUDA with PyTorch
* How to test custom operators using ``torch.library.opcheck``
* How to integrate custom operators written in C++/CUDA with PyTorch
* How to test custom operators using ``torch.library.opcheck``

.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
:class-card: card-prerequisites

* PyTorch 2.4 or later
* Basic understanding of C++ and CUDA programming
* PyTorch 2.4 or later
* Basic understanding of C++ and CUDA programming

PyTorch offers a large library of operators that work on Tensors (e.g. torch.add, torch.sum, etc).
However, you may wish to bring a new custom operator to PyTorch. This tutorial demonstrates the
Expand Down
8 changes: 5 additions & 3 deletions advanced_source/python_custom_ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,15 @@
.. grid:: 2

.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
:class-card: card-prerequisites

* How to integrate custom operators written in Python with PyTorch
* How to test custom operators using ``torch.library.opcheck``
* How to integrate custom operators written in Python with PyTorch
* How to test custom operators using ``torch.library.opcheck``

.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
:class-card: card-prerequisites

* PyTorch 2.4 or later
* PyTorch 2.4 or later

PyTorch offers a large library of operators that work on Tensors (e.g.
``torch.add``, ``torch.sum``, etc). However, you might wish to use a new customized
Expand Down
8 changes: 5 additions & 3 deletions beginner_source/ddp_series_fault_tolerance.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,12 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
.. grid:: 2

.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
:class-card: card-prerequisites
:margin: 0

- Launching multi-GPU training jobs with ``torchrun``
- Saving and loading snapshots of your training job
- Structuring your training script for graceful restarts
- Launching multi-GPU training jobs with ``torchrun``
- Saving and loading snapshots of your training job
- Structuring your training script for graceful restarts

.. grid:: 1

Expand All @@ -27,6 +28,7 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
:octicon:`code-square;1.0em;` View the code used in this tutorial on `GitHub <https://github.com/pytorch/examples/blob/main/distributed/ddp-tutorial-series/multigpu_torchrun.py>`__

.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
:class-card: card-prerequisites
:margin: 0

* High-level `overview <ddp_series_theory.html>`__ of DDP
Expand Down
2 changes: 2 additions & 0 deletions beginner_source/ddp_series_multigpu.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
.. grid:: 2

.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
:class-card: card-prerequisites

- How to migrate a single-GPU training script to multi-GPU via DDP
- Setting up the distributed process group
Expand All @@ -26,6 +27,7 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
:octicon:`code-square;1.0em;` View the code used in this tutorial on `GitHub <https://github.com/pytorch/examples/blob/main/distributed/ddp-tutorial-series/multigpu.py>`__

.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
:class-card: card-prerequisites

* High-level overview of `how DDP works <ddp_series_theory.html>`__
* A machine with multiple GPUs (this tutorial uses an AWS p3.8xlarge instance)
Expand Down
2 changes: 2 additions & 0 deletions beginner_source/ddp_series_theory.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,15 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
.. grid:: 2

.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
:class-card: card-prerequisites

* How DDP works under the hood
* What is ``DistributedSampler``
* How gradients are synchronized across GPUs


.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
:class-card: card-prerequisites

* Familiarity with `basic non-distributed training <https://pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html>`__ in PyTorch

Expand Down
14 changes: 8 additions & 6 deletions beginner_source/template_tutorial.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,18 @@
.. grid:: 2

.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
:class-card: card-prerequisites

* Item 1
* Item 2
* Item 3
* Item 1
* Item 2
* Item 3

.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
:class-card: card-prerequisites

* PyTorch v2.0.0
* GPU ???
* Other items 3
* PyTorch v2.0.0
* GPU ???
* Other items 3

If you have a video, add it here like this:

Expand Down
2 changes: 2 additions & 0 deletions intermediate_source/ddp_series_minGPT.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
.. grid:: 2

.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
:class-card: card-prerequisites

- Best practices when writing a distributed training script
- Increased flexibility with saving/loading artifacts in the cloud
Expand All @@ -23,6 +24,7 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
:octicon:`code-square;1.0em;` View the code used in this tutorial on `GitHub <https://github.com/pytorch/examples/tree/main/distributed/minGPT-ddp>`__

.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
:class-card: card-prerequisites

- Familiarity with `multi-GPU training <../beginner/ddp_series_multigpu.html>`__ and `torchrun <../beginner/ddp_series_fault_tolerance.html>`__
- [Optional] Familiarity with `multinode training <ddp_series_multinode.html>`__
Expand Down
2 changes: 2 additions & 0 deletions intermediate_source/ddp_series_multinode.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
.. grid:: 2

.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
:class-card: card-prerequisites

- Launching multinode training jobs with ``torchrun``
- Code changes (and things to keep in mind) when moving from single-node to multinode training.
Expand All @@ -22,6 +23,7 @@ Authors: `Suraj Subramanian <https://github.com/suraj813>`__
:octicon:`code-square;1.0em;` View the code used in this tutorial on `GitHub <https://github.com/pytorch/examples/blob/main/distributed/ddp-tutorial-series/multinode.py>`__

.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
:class-card: card-prerequisites

- Familiarity with `multi-GPU training <../beginner/ddp_series_multigpu.html>`__ and `torchrun <../beginner/ddp_series_fault_tolerance.html>`__
- 2 or more TCP-reachable GPU machines (this tutorial uses AWS p3.2xlarge instances)
Expand Down
12 changes: 7 additions & 5 deletions intermediate_source/dqn_with_rnn_tutorial.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,17 @@
.. grid:: 2

.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
:class-card: card-prerequisites

* How to incorporating an RNN in an actor in TorchRL
* How to use that memory-based policy with a replay buffer and a loss module
* How to incorporating an RNN in an actor in TorchRL
* How to use that memory-based policy with a replay buffer and a loss module

.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
:class-card: card-prerequisites

* PyTorch v2.0.0
* gym[mujoco]
* tqdm
* PyTorch v2.0.0
* gym[mujoco]
* tqdm
"""

#########################################################################
Expand Down
Loading