Skip to content

Commit 4275c42

Browse files
committed
fix: typos
1 parent dcf02de commit 4275c42

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

prototype_source/context_parallel.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ To better demonstrate the usage of this API, we start with a simple code snippet
8282
)
8383
for _ in range(3)
8484
]
85-
# specify the SDPABackend to use
85+
# specify the SDPBackend to use
8686
with sdpa_kernel(backend):
8787
out = F.scaled_dot_product_attention(*qkv, is_causal=True)
8888
@@ -148,7 +148,7 @@ shard to input and distribute the computation across ranks:
148148
)
149149
for _ in range(3)
150150
]
151-
# specify the SDPABackend to use
151+
# specify the SDPBackend to use
152152
with sdpa_kernel(backend):
153153
out = F.scaled_dot_product_attention(*qkv, is_causal=True)
154154
@@ -191,7 +191,7 @@ shard to input and distribute the computation across ranks:
191191
192192
193193
You can use the command ``torchrun --standalone --nnodes=1 --nproc-per-node=4 cp_sdpa_example.py`` to launch the above context parallel
194-
SDPA on 4 GPUs. We demonstrate the nemuric correctness by comparing the output of Ring Attention to that of SDPA on a single GPU.
194+
SDPA on 4 GPUs. We demonstrate the numeric correctness by comparing the output of Ring Attention to that of SDPA on a single GPU.
195195

196196

197197
Select Rotation Approach

0 commit comments

Comments
 (0)