[build] Add option to limit number of link jobs #29361
Conversation
build-script-impl (and thereby build-script) now supports these additional arguments: --swift-tools-num-parallel-link-jobs <number> --llvm-num-parallel-link-jobs <number>
| @@ -640,6 +642,18 @@ function set_build_options_for_host() { | |||
| -DLLVM_BUILD_EXTERNAL_COMPILER_RT:BOOL="$(false_true ${SKIP_BUILD_COMPILER_RT})" | |||
| ) | |||
|
|
|||
| if [[ "${SWIFT_TOOLS_NUM_PARALLEL_LINK_JOBS}" != "" ]] && [[ $(is_swift_lto_enabled) == "FALSE" ]]; then | |||
| swift_cmake_options+=( | |||
| "-DSWIFT_PARALLEL_LINK_JOBS=${SWIFT_TOOLS_NUM_PARALLEL_LINK_JOBS}" | |||
There was a problem hiding this comment.
I haven't looked at this, but I wonder if it would make sense to just have one option for this rather than have two options. Is that possible? How do we do it in the lldb case?
|
I really think that we should spell this as |
|
@compnerd another thing to consider is why even have different ones for swift and llvm. |
|
The reason for the two options is due to the out-of-tree build. If we could do unified builds, then, the LLVM option alone would be sufficient. |
|
@compnerd I understand that, but why shouldn't we just have a higher level option that defines it for both. I don't see why two nobs are necessary even with the out of tree build. |
|
@gottesmm - oh, those options are for ease. Feel free to use the higher level option: |
|
that is my thought. Using the high level options make more sense! |
|
In that case, this would just be written as |
|
(My opinion is that this is a documentation issue, not a build-script-impl issue) |
|
I didn't know about the
I'd be happy to open a separate PR to document this, but I'm not sure which document this should go into. |
Thank you, that would be wonderful! I think that this belongs in DeveloperTips.md |
When building llvm or Swift, linking uses much more memory than compilation, and therefore, it’s useful to limit the number of parallel jobs during linking only, as opposed to all jobs. For example, on a 4 GB RAM linux machine, building fails when two linking tasks are running in parallel, because of shortage of RAM. This is not a problem when two compiling tasks are running in parallel.
This PR enables us to call
build-scriptlike:to make it run only one link job at a time, while compilations jobs are not restricted.
Work to get parallel link jobs information into CMake / Ninja files has already been done for use with LTO. This PR just exposes that functionality in
build-scriptfor the case when LTO is not enabled.