Skip to content

[flang][driver] Add support for -isysroot in the frontend #77365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 11, 2024

Conversation

luporl
Copy link
Contributor

@luporl luporl commented Jan 8, 2024

If DEFAULT_SYSROOT is not specfied when building flang, then the
-isysroot flag is needed to link binaries against system libraries
on Darwin. It's also needed when linking against a non-default
sysroot.

If DEFAULT_SYSROOT is not specfied when building flang, then the
-isysroot flag is needed to link binaries against system libraries
on Darwin. It's also needed when linking against a non-default
sysroot.
@llvmbot llvmbot added clang Clang issues not falling into any other category flang:driver flang Flang issues not falling into any other category labels Jan 8, 2024
@llvmbot
Copy link
Member

llvmbot commented Jan 8, 2024

@llvm/pr-subscribers-flang-semantics
@llvm/pr-subscribers-flang-openmp
@llvm/pr-subscribers-flang-driver

@llvm/pr-subscribers-clang

Author: Leandro Lupori (luporl)

Changes

If DEFAULT_SYSROOT is not specfied when building flang, then the
-isysroot flag is needed to link binaries against system libraries
on Darwin. It's also needed when linking against a non-default
sysroot.


Full diff: https://github.com/llvm/llvm-project/pull/77365.diff

4 Files Affected:

  • (modified) clang/include/clang/Driver/Options.td (+1-1)
  • (modified) flang/test/Driver/driver-help-hidden.f90 (+1)
  • (modified) flang/test/Driver/driver-help.f90 (+1)
  • (added) flang/test/Driver/isysroot.f90 (+12)
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 6aff37f1336871..f42e9c7eb92a67 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -4265,7 +4265,7 @@ def iquote : JoinedOrSeparate<["-"], "iquote">, Group<clang_i_Group>,
   Visibility<[ClangOption, CC1Option]>,
   HelpText<"Add directory to QUOTE include search path">, MetaVarName<"<directory>">;
 def isysroot : JoinedOrSeparate<["-"], "isysroot">, Group<clang_i_Group>,
-  Visibility<[ClangOption, CC1Option]>,
+  Visibility<[ClangOption, CC1Option, FlangOption]>,
   HelpText<"Set the system root directory (usually /)">, MetaVarName<"<dir>">,
   MarshallingInfoString<HeaderSearchOpts<"Sysroot">, [{"/"}]>;
 def isystem : JoinedOrSeparate<["-"], "isystem">, Group<clang_i_Group>,
diff --git a/flang/test/Driver/driver-help-hidden.f90 b/flang/test/Driver/driver-help-hidden.f90
index 9a11a7a571ffcc..a4444ca043d0ea 100644
--- a/flang/test/Driver/driver-help-hidden.f90
+++ b/flang/test/Driver/driver-help-hidden.f90
@@ -111,6 +111,7 @@
 ! CHECK-NEXT: -g                      Generate source-level debug information
 ! CHECK-NEXT: --help-hidden           Display help for hidden options
 ! CHECK-NEXT: -help                   Display available options
+! CHECK-NEXT: -isysroot <dir>         Set the system root directory (usually /)
 ! CHECK-NEXT: -I <dir>                Add directory to the end of the list of include search paths
 ! CHECK-NEXT: -L <dir>                Add directory to library search path
 ! CHECK-NEXT: -march=<value>          For a list of available architectures for the target use '-mcpu=help'
diff --git a/flang/test/Driver/driver-help.f90 b/flang/test/Driver/driver-help.f90
index e0e74dc56f331e..07189264104592 100644
--- a/flang/test/Driver/driver-help.f90
+++ b/flang/test/Driver/driver-help.f90
@@ -97,6 +97,7 @@
 ! HELP-NEXT: -g                      Generate source-level debug information
 ! HELP-NEXT: --help-hidden           Display help for hidden options
 ! HELP-NEXT: -help                   Display available options
+! HELP-NEXT: -isysroot <dir>         Set the system root directory (usually /)
 ! HELP-NEXT: -I <dir>                Add directory to the end of the list of include search paths
 ! HELP-NEXT: -L <dir>                Add directory to library search path
 ! HELP-NEXT: -march=<value>          For a list of available architectures for the target use '-mcpu=help'
diff --git a/flang/test/Driver/isysroot.f90 b/flang/test/Driver/isysroot.f90
new file mode 100644
index 00000000000000..70d2fc0345ce50
--- /dev/null
+++ b/flang/test/Driver/isysroot.f90
@@ -0,0 +1,12 @@
+! Verify that the -isysroot flag is known to the frontend and, on Darwin,
+! is passed on to the linker.
+
+! RUN: %flang -### --target=aarch64-apple-darwin -isysroot /path/to/sysroot \
+! RUN:        %s 2>&1 | FileCheck %s --check-prefix=CHECK-DARWIN
+! RUN: %flang -### --target=aarch64-linux-gnu -isysroot /path/to/sysroot \
+! RUN:        %s 2>&1 | FileCheck %s --check-prefix=CHECK-LINUX
+
+! CHECK-DARWIN: "{{.*}}/ld" {{.*}}"-syslibroot" "/path/to/sysroot"
+! Unused on Linux.
+! CHECK-LINUX: warning: argument unused during compilation: '-isysroot /path/to/sysroot'
+! CHECK-LINUX-NOT: /path/to/sysroot

@luporl luporl requested a review from banach-space January 8, 2024 19:55
@banach-space
Copy link
Contributor

Thanks - could you also update the driver docs?

Copy link

github-actions bot commented Jan 9, 2024

✅ With the latest revision this PR passed the Python code formatter.

Copy link
Contributor

@banach-space banach-space left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this is very thoroughly tested and documented now, thank you 🙏🏻👌🏻

Feel free to ignore my final suggestion.

@luporl
Copy link
Contributor Author

luporl commented Jan 10, 2024

Thanks for the review and all the suggestions!

@luporl luporl merged commit 9edcf7a into llvm:main Jan 11, 2024
@luporl luporl deleted the luporl-isysroot branch January 11, 2024 13:38
@luporl luporl restored the luporl-isysroot branch January 11, 2024 15:38
@luporl luporl deleted the luporl-isysroot branch January 11, 2024 16:07
@kkwli
Copy link
Collaborator

kkwli commented Jan 14, 2024

If the compiler is built with DEFAULT_SYSROOT, the -isysroot option is ignored. Is that the expected behavior? My local MacOS build (with -DDEFAULT_SYSROOT="$(xcrun --show-sdk-path)") has isysroot.f90 failed.

@luporl
Copy link
Contributor Author

luporl commented Jan 15, 2024

If the compiler is built with DEFAULT_SYSROOT, the -isysroot option is ignored. Is that the expected behavior? My local MacOS build (with -DDEFAULT_SYSROOT="$(xcrun --show-sdk-path)") has isysroot.f90 failed.

I expected -isysroot to override DEFAULT_SYSROOT. I'll take a look at that, thanks for reporting it.

@luporl
Copy link
Contributor Author

luporl commented Jan 15, 2024

If the compiler is built with DEFAULT_SYSROOT, the -isysroot option is ignored. Is that the expected behavior? My local MacOS build (with -DDEFAULT_SYSROOT="$(xcrun --show-sdk-path)") has isysroot.f90 failed.

I expected -isysroot to override DEFAULT_SYSROOT. I'll take a look at that, thanks for reporting it.

This behavior is inherited from clang: https://github.com/llvm/llvm-project/blob/main/clang/lib/Driver/ToolChains/Darwin.cpp#L430

Only --sysroot overrides DEFAULT_SYSROOT.
It seems to me that, before DEFAULT_SYSROOT support was added, the original intention was to make --sysroot take precedence over --isysroot, if both were specified at command line: 980920a.

I think -isysroot should have preference over DEFAULT_SYSROOT, but:

  • I don't have much knowledge about the driver.
  • Since this has been the behavior for a long time, I fear that changing it may break some use case.
  • Some refactoring would need to be done, to make it possible to find out if what is being overridden is DEFAULT_SYSROOT or --sysroot.

An alternative would be to also support --sysroot on Flang, to be able to change it when DEFAULT_SYSROOT is used.
As for the isysroot.f90 test, I guess it would need to be removed, unless there is some way to run it only when DEFAULT_SYSROOT is not set. But it doesn't seem worth the trouble, asexec.f90 ends up testing -isysroot, when DEFAULT_SYSROOT is not specified at config time.

My preference is to add support for --sysroot on Flang and remove isysroot.f90.
@kkwli, @banach-space, what are your thoughts on this?

@banach-space
Copy link
Contributor

Thank you for this summary, @luporl !

I think -isysroot should have preference over DEFAULT_SYSROOT, but:

  • I don't have much knowledge about the driver.
  • Since this has been the behavior for a long time, I fear that changing it may break some use case.
  • Some refactoring would need to be done, to make it possible to find out if what is being overridden is DEFAULT_SYSROOT or --sysroot.

My biggest concern would be your 2nd point above - I'd rather avoid. Like you said - folks might depend on the current behaviour.

As for the isysroot.f90 test, I guess it would need to be removed, unless there is some way to run it only when DEFAULT_SYSROOT is not set.

You could also use DEFAULT_SYSROOT to define a LIT "feature" and check for that (via e.g. REQUIRES) in the test file. That shouldn't be too difficult.

But it doesn't seem worth the trouble, asexec.f90 ends up testing -isysroot, when DEFAULT_SYSROOT is not specified at config time.

Wouldn't the behaviour of exec.f90 change depending on whether the user sets DEFAULT_SYSROOT?

My preference is to add support for --sysroot on Flang and remove isysroot.f90. @kkwli, @banach-space, what are your thoughts on this?

Can you remind me the benefits of using -isysroot over -sysroot to begin with? I think that switching to -sysroot is fine, but I also want to make sure we're not missing anything.

@luporl
Copy link
Contributor Author

luporl commented Jan 16, 2024

As for the isysroot.f90 test, I guess it would need to be removed, unless there is some way to run it only when DEFAULT_SYSROOT is not set.

You could also use DEFAULT_SYSROOT to define a LIT "feature" and check for that (via e.g. REQUIRES) in the test file. That shouldn't be too difficult.

Ok, I'll take at look at that.

But it doesn't seem worth the trouble, asexec.f90 ends up testing -isysroot, when DEFAULT_SYSROOT is not specified at config time.

Wouldn't the behaviour of exec.f90 change depending on whether the user sets DEFAULT_SYSROOT?

The behavior "shouldn't" change, but the sysroot used when linking exec.f90 may change depending on whether the user sets DEFAULT_SYSROOT.

My preference is to add support for --sysroot on Flang and remove isysroot.f90. @kkwli, @banach-space, what are your thoughts on this?

Can you remind me the benefits of using -isysroot over -sysroot to begin with? I think that switching to -sysroot is fine, but I also want to make sure we're not missing anything.

It seems to me that Apple prefers to use -isysroot to select the SDK: https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/cross_development/Configuring/configuring.html#//apple_ref/doc/uid/10000163i-CH1-SW1

Also, when grepping through CMake generated files (for llvm), I saw that they use -isysroot when compiling sources with the clang binary built from source.

Apart from that, the other difference that I'm aware is that on non-Darwin targets -isysroot only sets the sysroot for the includes, while -sysroot also affects the libraries.

@banach-space
Copy link
Contributor

banach-space commented Jan 16, 2024

The behavior "shouldn't" change, but the sysroot used when linking exec.f90 may change depending on whether the user sets DEFAULT_SYSROOT.

Good, so in practice the env variable won’t be affecting this test.

Can you remind me the benefits of using -isysroot over -sysroot to begin with? I think that switching to -sysroot is fine, but I also want to make sure we're not missing anything.

It seems to me that Apple prefers to use -isysroot to select the SDK: https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/cross_development/Configuring/configuring.html#//apple_ref/doc/uid/10000163i-CH1-SW1

Also, when grepping through CMake generated files (for llvm), I saw that they use -isysroot when compiling sources with the clang binary built from source.

Apart from that, the other difference that I'm aware is that on non-Darwin targets -isysroot only sets the sysroot for the includes, while -sysroot also affects the libraries.

Alignment with the official guidelines would be nice. If possible :)

Apart from that, the other difference that I'm aware is that on non-Darwin targets -isysroot only sets the sysroot for the includes, while -sysroot also affects the libraries.

Sounds like in practice it wouldn’t make much difference whether it’s -isysroot or -sysroot?

@luporl
Copy link
Contributor Author

luporl commented Jan 17, 2024

Apart from that, the other difference that I'm aware is that on non-Darwin targets -isysroot only sets the sysroot for the includes, while -sysroot also affects the libraries.

Sounds like in practice it wouldn’t make much difference whether it’s -isysroot or -sysroot?

Right, but adding support for -sysroot too would have 2 main benefits:

  • Make it possible to override DEFAULT_SYSROOT.
  • Make it possible for other targets (non-Darwin) to set the sysroot.

My plan is the following:

  • Try to fix isysroot.f90 test by making DEFAULT_SYSROOT a LIT "feature".
  • Keep using -isysroot on Flang's tests, on Darwin, as it seems to be the recommended way to select an SDK.
  • Add support for -sysroot.

Does it look good?

@kkwli
Copy link
Collaborator

kkwli commented Jan 17, 2024

My plan is the following:

* Try to fix isysroot.f90 test by making DEFAULT_SYSROOT a LIT "feature".

* Keep using -isysroot on Flang's tests, on Darwin, as it seems to be the recommended way to select an SDK.

* Add support for -sysroot.

Does it look good?

@luporl Thanks a lot. Your proposed plan looks good to me.

justinfargnoli pushed a commit to justinfargnoli/llvm-project that referenced this pull request Jan 28, 2024
If DEFAULT_SYSROOT is not specfied when building flang, then the
-isysroot flag is needed to link binaries against system libraries
on Darwin. It's also needed when linking against a non-default
sysroot.
@luporl
Copy link
Contributor Author

luporl commented Jan 30, 2024

It looks like --sysroot= is already supported by flang-new. It worked fine in my tests. Just like clang, it doesn't show up in --help or --help-hidden, for some reason I'm not aware.

I noticed, however, that using --sysroot doesn't work:
flang-new: error: unknown argument '--sysroot'

Interestingly enough, some other flags also show the same issue, such as --verbose:
flang-new: error: unknown argument: '--verbose'

My hunch is that option aliases are not working properly with flang-new, when Visibility is omitted.

Anyone knows if this is expected and Visibility should be explicitly specified in each alias, to add support to flang-new, or is this an issue and the alias should have the same visibility as the original option?

@banach-space
Copy link
Contributor

Thank you for your continued effort to improve this!

Anyone knows if this is expected and Visibility should be explicitly specified in each alias, to add support to flang-new, or is this an issue and the alias should have the same visibility as the original option?

IIRC, when Visibility is omitted then it defaults to "Clang" (or something to that effect). That would explain what you see. Something to be fixed :)

@luporl
Copy link
Contributor Author

luporl commented Feb 5, 2024

#80728 opened to track this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang Clang issues not falling into any other category flang:driver flang:openmp flang:semantics flang Flang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants