Fix Gemma parity issue

[kunal-vaishnavi]

Description

This PR fixes a parity issue with Google's Gemma models by moving the addition of the unit offset to be after the dtype conversion.

Motivation and Context

The Gemma models from Hugging Face are loaded with torch.bfloat16 precision by default. When a unit add is performed on a torch.bfloat16 tensor, the following behavior occurs.

Example:

> import torch 
> t = torch.tensor(3.141592, dtype=torch.bfloat16)
> t
tensor(3.1406, dtype=torch.bfloat16)
> t + 1
tensor(4.1250, dtype=torch.bfloat16)

The value returned is 4.1250 instead of 4.1406 and it will remain this value even when the returned value is converted to torch.float16 or torch.float32.

> (t + 1).to(torch.float16)
tensor(4.1250, dtype=torch.float16)
> (t + 1).to(torch.float32)
tensor(4.1250)

If the unit add is performed after the dtype conversion, the value returned is the expected value.

> t = torch.tensor(3.141592, dtype=torch.bfloat16)
> t
tensor(3.1406, dtype=torch.bfloat16)
> t = t.to(torch.float32)
> t
tensor(3.1406)
> t + 1
tensor(4.1406)

When comparing the LayerNorm weights from the Hugging Face Gemma 2B model and the GGUF Gemma 2B model produced by convert-hf-to-gguf.py before this change, the tensor values are different. Each tensor below is of size 2048 and in float16 precision.

Tensor Name	Hugging Face	GGUF
Layer 0 Attention Norm	[0. 3.469 1.469 ... 2.727 3.031 2.984]	[0. 3.469 1.469 ... 2.719 3.031 2.984]
Layer 0 FFN Norm	[1.4 1.93 1.723 ... 1.73 1.836 1.73]	[1.398 1.93 1.719 ... 1.734 1.836 1.734]
Layer 1 Attention Norm	[1.566 1.383 1.594 ... 1.471 1.354 1.238]	[1.5625 1.383 1.594 ... 1.469 1.352 1.234]
Layer 1 FFN Norm	[2.016 1.91 2.148 ... 1.918 2.11 1.73]	[2.016 1.906 2.156 ... 1.922 2.11 1.734]
...	...	...
Layer 16 Attention Norm	[1.758 2.523 2.148 ... 1.777 2.305 1.945]	[1.758 2.531 2.156 ... 1.781 2.312 1.945]
Layer 16 FFN Norm	[2.852 2.758 2.984 ... 2.922 2.977 2.914]	[2.844 2.75 2.984 ... 2.922 2.969 2.906]
...	...	...
Layer 18 Attention Norm	[1.305 1.832 1.59 ... 1.875 1.516 1.855]	[1.305 1.828 1.594 ... 1.875 1.516 1.859]

After converting the GGUF model to ONNX and running a parity test with ONNX Runtime, ORT reports a parity mismatch for both prompt processing and token generation.

When comparing the LayerNorm weights from the Hugging Face Gemma 2B model and the GGUF Gemma 2B model produced by convert-hf-to-gguf.py after this change, the tensor values are matching. Each tensor below is of size 2048 and in float16 precision.

Tensor Name	Hugging Face	GGUF
Layer 0 Attention Norm	[0. 3.469 1.469 ... 2.727 3.031 2.984]	[0. 3.469 1.469 ... 2.727 3.031 2.984]
Layer 0 FFN Norm	[1.4 1.93 1.723 ... 1.73 1.836 1.73]	[1.4 1.93 1.723 ... 1.73 1.836 1.73]
Layer 1 Attention Norm	[1.566 1.383 1.594 ... 1.471 1.354 1.238]	[1.566 1.383 1.594 ... 1.471 1.354 1.238]
Layer 1 FFN Norm	[2.016 1.91 2.148 ... 1.918 2.11 1.73]	[2.016 1.91 2.148 ... 1.918 2.11 1.73]
...	...	...
Layer 16 Attention Norm	[1.758 2.523 2.148 ... 1.777 2.305 1.945]	[1.758 2.523 2.148 ... 1.777 2.305 1.945]
Layer 16 FFN Norm	[2.852 2.758 2.984 ... 2.922 2.977 2.914]	[2.852 2.758 2.984 ... 2.922 2.977 2.914]
...	...	...
Layer 18 Attention Norm	[1.305 1.832 1.59 ... 1.875 1.516 1.855]	[1.305 1.832 1.59 ... 1.875 1.516 1.855]

After converting the new GGUF model to ONNX and running the same parity test with ONNX Runtime, ORT reports that parity is achieved.