Add support for cpu_get_num_physical_cores() on Windows

[Septa2112]

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

Add a function cpu_get_num_cores_win(bool print_physical_core_num) to detect the physical and logical cores on Windows. Now the program will successfully detect all sockets and CPUs and set the default number of threads more accurately on Windows.

Test Platform

• OS
  • Windows 10
  • Windows 11
• CPU
  • Intel Xeon CPU E5-2699 v4 (22 physical core per socket, 2 Sockets in total)
  • Intel i7-1185G7 (4 physical cores)
  • Intel Core Ultra5 125H (4 P-core + 8 E-core + 2 LPE-core)

Before

When I run llama-cli.exe -m <my_model> -p <prompt>, the output is like this:

Intel Xeon CPU E5-2699 v4 (2 Sockets)

system_info: n_threads = 22 / 44 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | 

Intel i7-1185G7

system_info: n_threads = 4 / 8 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | 

Intel MTL Ultra5

system_info: n_threads = 9 / 18 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | 

After Modification

Intel Xeon CPU E5-2699 v4 (2 Sockets)

system_info: n_threads = 44 / 88 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | 

Intel i7-1185G7

system_info: n_threads = 4 / 8 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | 

Intel MTL Ultra5

system_info: n_threads = 14 / 18 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | 

[Septa2112]

@mofosyne Could you have a look at my PR? Or anyone others? Thanks.

[ggerganov]

We will need some reports on Windows that this change works as expected

[Septa2112]

I noticed that the CI CI / windows-latest-cmake (msvc-arm64, -G "Ninja Multi-Config"... failed, indicating that the Windows API I used was not detected.
However, another CI about arm64 CI / windows-latest-cmake (llvm-arm64, -G "Ninja Multi-Config" is successful(tested myself using GitHub Actions).

I noticed that these two CIs were added due to #7191

To solve this, I attempted to skip the ARM64 platform with the following macros:

#elif defined(_WIN32) && (!defined(__ARM_NEON) || !defined(_M_ARM) || !defined(_M_ARM64) || !defined(__aarch64__) || !defined(__arm__) || !defined(__ARM_ARCH))

However, despite my repeated attempts, none of these macros related to the ARM architecture were detected in this CI environment.

I suspect there might be an issue with the toolchain, but unfortunately, I don't have any ARM64+Windows machine for testing. Could you kindly suggest any solutions or approaches to resolve this issue?

[Septa2112]

I carefully reviewed the CI error and found that the issue seems to be with MinGW64+ARM64 on windows. Therefore, I made the following modification

#elif defined(_WIN32) && (_WIN32_WINNT >= 0x0601) && !defined(__MINGW64__) // windows 7 and later

to bypass the issue with the toolchain on ARM64+Windows by detecting __MINGW64__.

[hmartinez82]

@Septa2112 Once this is merged I'll take a look at why ARM64 MinGW64 is not building :-)
I have two Windows on ARM64 machines. Once with Snapdragon 8cx (ARMv8.2-A) and one with Snapdragon Elite X (ARMv8.7-A)

[Septa2112]

@hmartinez82 That's great. Let's see when the PR will be merged. Perhaps need more test on windows.

[hmartinez82]

@Septa2112 Just as I suspected, after glancing over the PR, there's nothing there that would not work. I removed the && !defined(__MINGW64__) in both lines and tried building for ARM64 with both x-compiling from a x64 MSVC (17.10) and natively with MSYS2's Clang in a CLANGARM64 environment.

The build was successful in both cases.

[Septa2112]

@hmartinez82 Thanks for you testing. But if I remove this macro, the corresponding CI checks will fail. Maybe something else is causing the issue?

This related CI was introduced in the issue #7191.

[hmartinez82]

Ahh ok, I thought that the issue was during build time. I'll try to replicate what the CI does. Thank you.