llamafile: support s390x SIMD instruction set

[taronaeo]

This pull request aims to integrate the SIMD instruction set into Llamafile for the s390x platform.

At current, only F32 and F16 data types are activated. Quantised data types are WIP and will not be part of this PR.

Verification

To ensure that this implementation did not break anything, the SIMD instruction set has been tested on the following models:

• Tested IBM Granite 3.3 (F32, F16, Q4_0, Q4_1, Q8_0, Q3_K, Q4_K, Q5_K, Q6_K, IQ4_NL, IQ4_XS)
• Kindly request additional models for testing in this PR

Performance Results

I will be using IBM Granite 3.3 for the performance tests. We notice an average of 8.14% performance improvement in Prompt Processing.

Before Llamafile SIMD Instruction Set

model	size	params	backend	threads	test	t/s
granite 3B all F32	9.44 GiB	2.53 B	BLAS	16	pp512	57.45 ± 0.28
granite 3B all F32	9.44 GiB	2.53 B	BLAS	16	tg128	2.33 ± 0.09
granite 3B F16	4.72 GiB	2.53 B	BLAS	16	pp512	56.54 ± 0.37
granite 3B F16	4.72 GiB	2.53 B	BLAS	16	tg128	4.19 ± 0.31

After Llamafile SIMD Instruction Set

model	size	params	backend	threads	test	t/s
granite 3B all F32	9.44 GiB	2.53 B	BLAS	16	pp512	62.33 ± 0.45
granite 3B all F32	9.44 GiB	2.53 B	BLAS	16	tg128	2.34 ± 0.04
granite 3B F16	4.72 GiB	2.53 B	BLAS	16	pp512	61.98 ± 0.10
granite 3B F16	4.72 GiB	2.53 B	BLAS	16	tg128	4.46 ± 0.14

Note

Tests were conducted on an IBM z15 Mainframe with 16 IFLs (cores) and 160 GB Memory on a shared R&D LPAR.

Please review this pull request and consider merging into the main repository. Thank you!

[zhouwg]

thanks for your excellent PR. as you did in #14037, we can see the performance of horizontal summation of vector elements is one of bottlenecks of mulmat.

skip horizontal summation of vector elements and the computational result is incorrect.

[taronaeo]

Hi @zhouwg, its been awhile :). While I'm unsure what is going on with your end, I've simplified hsum from 3 element additions to 2 element additions by adding the original vector with itself but reversed, and that simplifies the assembly calls while resulting to the same outcome.

Visualisation:

Original Vector (F32): { 9, 6, 3, 8 }
Reversed Vector (F32): { 8, 3, 6, 9 }

Original + Reversed Vector (F32): { 17, 9, 9, 17 } (we call this Sum Vector)

We finally add Sum Vector's first and second element together to get the hsum scalar value

Have you tried something similar in your implementation? :)

[zhouwg]

not yet, seems better than existing approaches I have tried. thanks so much!

---

this elegant approach works fine on Hexagon cDSP, I'm not sure why is a little slower than other approach(e.g.unroll loop), might be lack of some special SIMD instructions(it seems that extract float data from HVX vector need more cycles and memory to memory transfer is more efficient on Hexagon cDSP, I'm not sure why).

thanks again!