Skip to content

Commit be4fdfb

Browse files
committed
fix white space
1 parent 30a506e commit be4fdfb

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

doc/api/training/smp_versions/latest/smd_model_parallel_pytorch.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -559,7 +559,7 @@ smdistributed.modelparallel.torch.nn.FlashAttentionLayer
559559
which represents the output of attention computation.
560560

561561
**Parameters**
562-
562+
563563
* ``qkv``: ``torch.Tensor`` in the form of ``(batch_size x seqlen x 3 x num_heads x head_size)``.
564564
* ``attn_mask``: ``torch.Tensor`` in the form of ``(batch_size x 1 x 1 x seqlen)``.
565565
By default it is ``None``, and usage of this mask needs ``triton_flash_attention``
@@ -570,9 +570,9 @@ smdistributed.modelparallel.torch.nn.FlashAttentionLayer
570570

571571
.. code:: python
572572
573-
def generate_alibi_attn_mask(attention_mask, batch_size, seq_length,
573+
def generate_alibi_attn_mask(attention_mask, batch_size, seq_length,
574574
num_attention_heads, alibi_bias_max=8):
575-
575+
576576
device, dtype = attention_mask.device, attention_mask.dtype
577577
alibi_attention_mask = torch.zeros(
578578
1, num_attention_heads, 1, seq_length, dtype=dtype, device=device

0 commit comments

Comments
 (0)