server: args for draft model cache types (#11200)

[aa956]

Should fix the #11200, while keeping the default f16 from #10586.

New command line arguments:

Argument	Explanation
-ctkd, --cache-type-k-draft TYPE	KV cache data type for K for speculative decoding model allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16) (env: LLAMA_ARG_CACHE_TYPE_K_DRAFT)
-ctvd, --cache-type-v-draft TYPE	KV cache data type for V for speculative decoding model allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16) (env: LLAMA_ARG_CACHE_TYPE_V_DRAFT)

[CISC]

@ggerganov forgot to merge?

[aa956]

@ggerganov forgot to merge?

@ngxson was added automatically on pull request creation, so merging probably needs 2 approvals.

I'm not sure what can I do regarding the failed arm64/ppc64/risc64 builds as I have no access to that kind of hardware and looking at the build logs all of these failed initial apt operations (maybe apt caches/mirrors for these architectures were down at the check time or something similar).

E: Failed to fetch https://security.ubuntu.com/ubuntu/dists/noble-security/main/binary-riscv64/Packages  404  Not Found [IP: 52.252.163.49 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libcurl4t64:riscv64 : Depends: libgssapi-krb5-2:riscv64 (>= 1.17) but it is not going to be installed
 libssh-4:riscv64 : Depends: libgssapi-krb5-2:riscv64 (>= 1.17) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

[CISC]

@ggerganov forgot to merge?

@ngxson was added automatically on pull request creation, so merging probably needs 2 approvals.

No, this can be merged now, but just want to make sure it's not on hold for some reason first. :)

I'm not sure what can I do regarding the failed arm64/ppc64/risc64 builds as I have no access to that kind of hardware and looking at the build logs all of these failed initial apt operations (maybe apt caches/mirrors for these architectures were down at the check time or something similar).

Yes, these failures are irrelevant, it has been fixed in CI since.

[ggerganov]

Yes, thanks for the reminder.