Skip to content

[libc++] Fix filesystem::remove_all() on FreeBSD #79540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 29, 2024
Merged

Conversation

markjdb
Copy link
Contributor

@markjdb markjdb commented Jan 26, 2024

remove_all_impl() opens the target path with O_NOFOLLOW, which fails if
the target is a symbolic link. On FreeBSD, rather than returning ELOOP,
openat() returns EMLINK. This is unlikely to change for compatibility
reasons, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214633 .

Thus, check for EMLINK as well.

@markjdb markjdb requested a review from a team as a code owner January 26, 2024 02:34
@llvmbot llvmbot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Jan 26, 2024
@llvmbot
Copy link
Member

llvmbot commented Jan 26, 2024

@llvm/pr-subscribers-libcxx

Author: Mark Johnston (markjdb)

Changes

See the commit message for an explanation.

I'm not sure if this bug is the reason for the XFAIL: LIBCXX-FREEBSD-FIXME annotation in test/std/input.output/filesystems/fs.op.funcs/fs.op.remove_all/remove_all.pass.cpp. @emaste do you have any idea?


Full diff: https://github.com/llvm/llvm-project/pull/79540.diff

1 Files Affected:

  • (modified) libcxx/src/filesystem/operations.cpp (+4-2)
diff --git a/libcxx/src/filesystem/operations.cpp b/libcxx/src/filesystem/operations.cpp
index 6253d1551b062e..c39a3ee53db439 100644
--- a/libcxx/src/filesystem/operations.cpp
+++ b/libcxx/src/filesystem/operations.cpp
@@ -815,8 +815,10 @@ uintmax_t remove_all_impl(int parent_directory, const path& p, error_code& ec) {
 
   // If opening `p` failed because it wasn't a directory, remove it as
   // a normal file instead. Note that `openat()` can return either ENOTDIR
-  // or ELOOP depending on the exact reason of the failure.
-  if (ec == errc::not_a_directory || ec == errc::too_many_symbolic_link_levels) {
+  // or ELOOP depending on the exact reason of the failure. On FreeBSD it
+  // may return EMLINK instead of ELOOP, contradicting POSIX.
+  if (ec == errc::not_a_directory || ec == errc::too_many_symbolic_link_levels ||
+      ec == errc::too_many_links) {
     ec.clear();
     if (::unlinkat(parent_directory, p.c_str(), /* flags = */ 0) == -1) {
       ec = detail::capture_errno();

Copy link

github-actions bot commented Jan 26, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

@emaste
Copy link
Member

emaste commented Jan 26, 2024

The XFAIL was added in 7ec6c62 and just captured the current state of the tests in order to stand up the FreeBSD CI instance. It is presumably because of this issue, and assuming the test passes with this change the XFAIL should be removed.

@markjdb
Copy link
Contributor Author

markjdb commented Jan 26, 2024

The XFAIL was added in 7ec6c62 and just captured the current state of the tests in order to stand up the FreeBSD CI instance. It is presumably because of this issue, and assuming the test passes with this change the XFAIL should be removed.

Do you happen to have a simple recipe to build and run these tests on FreeBSD? I haven't done that before.

@emaste
Copy link
Member

emaste commented Jan 26, 2024

Do you happen to have a simple recipe to build and run these tests on FreeBSD? I haven't done that before.

I haven't built upstream llvm in a while, but you should just be able to run e.g. bin/llvm-lit ../libcxx/test/std/input.output/filesystems/fs.op.funcs/fs.op.remove_all

It looks a run with this change should have results here in a bit though:
https://buildkite.com/llvm-project/libcxx-ci/builds/33234#018d43a2-514b-40fd-86c4-6e743f60fe21

Copy link
Member

@ldionne ldionne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me but you'll need to remove the XFAIL for the CI to pass. Please also ensure that all the other platforms are green before merging.

remove_all_impl() opens the target path with O_NOFOLLOW, which fails if
the target is a symbolic link.  On FreeBSD, rather than returning ELOOP,
openat() returns EMLINK.  This is unlikely to change for compatibility
reasons, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214633 .

Thus, check for EMLINK as well.
It passes on FreeBSD now that remove_all_impl() handles FreeBSD's
non-standard error value when opening a symlink with O_NOFOLLOW.
@markjdb
Copy link
Contributor Author

markjdb commented Jan 26, 2024

This looks reasonable to me but you'll need to remove the XFAIL for the CI to pass. Please also ensure that all the other platforms are green before merging.

Thank you. I pushed a second commit which does that. I verified that the test now passes on FreeBSD.

Copy link
Collaborator

@DimitryAndric DimitryAndric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@markjdb
Copy link
Contributor Author

markjdb commented Jan 27, 2024

It looks like one of the test runs was cancelled. Should I try to rerun them?

@DimitryAndric
Copy link
Collaborator

All the buildkite stuff at https://buildkite.com/llvm-project/libcxx-ci/builds/33247#_ says "cancelled after 5h" so I think it's probably some VM or container failing to start. No idea what is behind all this infrastructure. :))

@DimitryAndric
Copy link
Collaborator

Not sure about the CI failure, it just seems like a timeout for a VM to start up. Nothing to do with your commit, as usual.

@DimitryAndric DimitryAndric merged commit 4a39d08 into llvm:main Jan 29, 2024
@jwakely
Copy link
Contributor

jwakely commented Apr 10, 2024

FWIW while handling EMLINK in libstdc++ I discovered that NetBSD sets its non-standard EFTYPE error for the "openat with O_NOFOLLOW meets a symlink" case.

DragonFly matches FreeBSD, so the current libc++ code should handle that already.

NetBSD 10.0 openat(2)

DragonFly 6.4 openat(2)

iontzialla pushed a commit to GoogleCloudPlatform/kms-integrations that referenced this pull request Jul 22, 2024
This should address a weird segmentation fault issue we have seen
during logging shutdown with older versions of the glog library.

I'm also disabling symlinks with SetLogSymlink(), to prevent the
duplicate "UNKNOWN.INFO" logfile. Doing so also helps with FreeBSD,
because there was an issue with std::filesystem::remove_all (used in our
unit test) which was fixed only recently, causing it to error out due to
the dangling symlinks: llvm/llvm-project#79540

Bug: b/352597329
Change-Id: Ie4faf2740e8eabc7f9dda9127f571084f1a510b3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants