Skip to content

[Build script] Add compilation script #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 22, 2024
Merged

Conversation

Devjiu
Copy link
Contributor

@Devjiu Devjiu commented May 16, 2024

In some cases, when using different versions of the compiler/libc to build the llvm-project and other parts of the project, linking errors occur. To avoid this, a new compilation script has been added that will build the llvm project in the specified environment.

Also external projects will be saved in the externals folder so that the user can delete the build directory separately from the external projects.

@AndreyPavlenko
Copy link
Contributor

I think, it makes sense to do with cmake:
https://github.com/intel/graph-compiler/blob/main/cmake/gtest.cmake
https://github.com/intel/graph-compiler/blob/main/cmake/onednn.cmake
https://github.com/intel/graph-compiler/blob/main/cmake/functions.cmake#L6

The main advantage of this approach - you may use either a local folder, findpackage or git repo.

@Devjiu Devjiu requested a review from leshikus May 17, 2024 14:11
@leshikus
Copy link
Contributor

In some cases, when using different versions of the compiler/libc to build the llvm-project and other parts of the project, linking errors occur.

Let's have one working script instead of two

Copy link
Contributor

@leshikus leshikus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added two notes to think about. You also mentioned conda in our talk, I cannot see a conda env here

echo $project_dir
mlir_dir=$PWD/build/lib/cmake/mlir
popd
popd
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I consider using pushd / popd bad practice, they are not local. Use cd instead. Another option is to use round brackets to do something in a separate shell, then you return to the default dir

@AndreyPavlenko
Copy link
Contributor

Since this is a dev script, could we use the dynamic linking, introduced in this #66 PR? It will make the build/debug faster.

@Devjiu Devjiu force-pushed the dmitriim/compile-dev-script branch 3 times, most recently from f2aa6cf to 287c1ce Compare May 21, 2024 16:55
@Devjiu
Copy link
Contributor Author

Devjiu commented May 21, 2024

Since this is a dev script, could we use the dynamic linking, introduced in this #66 PR? It will make the build/debug faster.

I've tried, but default built failed with this option. Don't know how to enable it properly. (checked with -DGC_DEV_LINK_LLVM_DYLIB="ON")

@Devjiu Devjiu force-pushed the dmitriim/compile-dev-script branch 2 times, most recently from f053836 to 6d20519 Compare May 21, 2024 17:12
@AndreyPavlenko
Copy link
Contributor

I've tried, but default built failed with this option. Don't know how to enable it properly. (checked with -DGC_DEV_LINK_LLVM_DYLIB="ON")

It fails for me either :(

Copy link
Contributor

@leshikus leshikus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@ZhennanQin
Copy link
Contributor

Since this is a dev script, could we use the dynamic linking, introduced in this #66 PR? It will make the build/debug faster.

I've tried, but default built failed with this option. Don't know how to enable it properly. (checked with -DGC_DEV_LINK_LLVM_DYLIB="ON")

@Menooker would you please help to enable this?

@Menooker
Copy link

Menooker commented May 22, 2024

I've tried, but default built failed with this option. Don't know how to enable it properly. (checked with -DGC_DEV_LINK_LLVM_DYLIB="ON")

It fails for me either :(

GC_DEV_LINK_LLVM_DYLIB requires the LLVM built with cmake option -DLLVM_LINK_LLVM_DYLIB=ON -DLLVM_BUILD_LLVM_DYLIB=ON. You may need to re-build LLVM. Please take the Readme.md in the repo as reference.

It is recommended to add optional options -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON to the command cmake -G Ninja llvm ... above. These will enable the build of LLVM/MLIR dynamic libraries and let MLIR/LLVM tools link to them, to reduce the installed binary size of LLVM/MLIR. These options also enable the GC_DEV_LINK_LLVM_DYLIB option of graph-compiler repo (see below). The option -DLLVM_INSTALL_GTEST=ON is optional, if the tests of graph-compiler are disabled (see GC_TEST_ENABLE below).

In some cases, when using different versions of the compiler/libc to build the llvm-project
and other parts of the project, linking errors occur. To avoid this, a new compilation script
has been added that will build the llvm project in the specified environment.

Also external projects will be saved in the externals folder so that the user can
delete the build directory separately from the external projects.

Signed-off-by: Dmitrii Makarenko <[email protected]>
@Devjiu Devjiu force-pushed the dmitriim/compile-dev-script branch from 6d20519 to 4df9e0f Compare May 22, 2024 12:16
@Devjiu
Copy link
Contributor Author

Devjiu commented May 22, 2024

I've tried, but default built failed with this option. Don't know how to enable it properly. (checked with -DGC_DEV_LINK_LLVM_DYLIB="ON")

It fails for me either :(

GC_DEV_LINK_LLVM_DYLIB requires the LLVM built with cmake option -DLLVM_LINK_LLVM_DYLIB=ON -DLLVM_BUILD_LLVM_DYLIB=ON. You may need to re-build LLVM. Please take the Readme.md in the repo as reference.

It is recommended to add optional options -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON to the command cmake -G Ninja llvm ... above. These will enable the build of LLVM/MLIR dynamic libraries and let MLIR/LLVM tools link to them, to reduce the installed binary size of LLVM/MLIR. These options also enable the GC_DEV_LINK_LLVM_DYLIB option of graph-compiler repo (see below). The option -DLLVM_INSTALL_GTEST=ON is optional, if the tests of graph-compiler are disabled (see GC_TEST_ENABLE below).

thanks for the hint, updated script to handle this option.
In conda environment dynamic linking failed, so it's not enabled by default - please check script in your configuration.
In native ubuntu it succeeded, so I checked that in some cases it works.
@AndreyPavlenko FYI

@Devjiu Devjiu merged commit a971bff into main May 22, 2024
4 checks passed
@Devjiu Devjiu deleted the dmitriim/compile-dev-script branch May 22, 2024 12:26

args=$(getopt -a -o dlh --long dev,dyn,help -- "$@")
if [[ $? -gt 0 ]]; then
echo "first"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this echo here by mistake?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, sorry, thanks for pointing this. It was done for debug

args=$(getopt -a -o dlh --long dev,dyn,help -- "$@")
if [[ $? -gt 0 ]]; then
echo "first"
print_usage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
print_usage
print_usage
exit 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants