[llvm-foreach] Avoid running llvm-foreach when waiting for child processes

[asudarsa]

This resolves #15177

Currently, we have a 'while' loop that keeps checking for child processes to complete. This causes llvm-foreach to run and consume resources when its child processes are active.
Instead, we can use blocking waits to wait for child processes. During blocking waits. parent process is idle and does not consume resources.

I tested this on a fairly large program compilation (with AOT) and it seems to work as expected.

No new tests are needed.
Thanks

[asudarsa]

@vmaksimo

Can you please take a look as you have worked on this before?

Thanks

[maarquitos14]

@asudarsa did you do any performance measurement? I'm afraid this change might degrade performance, at least in scenarios where some jobs take longer than others. If I'm right --there is performance regression--, what do you think about doing this under an option? This way, user could choose whether they prefer better performance at the expense of consuming more CPU, or worse performance but with CPU idle.

[asudarsa]

@asudarsa did you do any performance measurement? I'm afraid this change might degrade performance, at least in scenarios where some jobs take longer than others. If I'm right --there is performance regression--, what do you think about doing this under an option? This way, user could choose whether they prefer better performance at the expense of consuming more CPU, or worse performance but with CPU idle.

Hi @maarquitos14

Thanks for looking into this. I did run this on one of the time consuming compilation benchmarks that we have and verified that we do not have any regression for both serial and parallel compilation scenarios.
In this PR, I have just replaced several 'while' loops that keep checking if child processes are complete with blocking waits. We still start execution of jobs in parallel. I have only modified the way in which we wait.

Thanks

[maarquitos14]

LGTM.

[fwyzard]

Thanks for the fix !