Skip to content

[Docs] Add documentation how to diagnose LSAN CI failures #69897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 16, 2023

Conversation

ahoppen
Copy link
Member

@ahoppen ahoppen commented Nov 15, 2023

Just a document that describes how I symbolicated LSAN CI failures. I’m pretty sure that there are ways to simplify steps, but it’s better to have something rather than nothing.

@ahoppen
Copy link
Member Author

ahoppen commented Nov 15, 2023

@swift-ci Please smoke test

@@ -0,0 +1,78 @@
# Diagnose LSAN Failures in the Compiler

### Create Ubuntu Container
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how much more effort you want to put into this, but IMO it would be nice to split out the "create a ubuntu docker container" into its own doc (that's useful regardless of LSAN).

### Create Ubuntu Container

1. Clone (or pull) swift-docker: https://github.com/apple/swift-docker
2. Build the Ubuntu 18.04 container: `cd swift-ci/master/ubuntu/18.04; docker build .`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe 20/22 instead? 18 getting pretty old now. You can also provide a tag to the build command - docker build -t swift-ci-ubuntu-18 .

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The LSAN bot is using 18.04, so I’m matching that. https://ci.swift.org/job/oss-swift-RA-lsan-linux-ubuntu-18_04/

Comment on lines +9 to +10
4. Copy the sources to inside the Docker container: `cp /src-on-host/* ~`
- We need to to this because the build needs a case-sensitive file system and your host machine probably has a case-insensitive file system
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We really should just fix this. I'm fairly sure it's just include/swift/Strings.h that causes the problems, which we can easily rename. I don't expect you to do that though.

3. Now, disassemble the failing CMake invocation to a swiftc invocation. I needed to set one environment variable and could the copy the swiftc invocation (but this might change as the build changes)

```
export LD_LIBRARY_PATH=/opt/swift/5.8.1/usr/lib/swift/linux
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth adding a little more information here. The reason LD_LIBRARY_PATH is needed here is because the stdlib failed to build. If that wasn't the case, we'd find the just-built libs and everything would be fine.


### Symbolicating the LSAN report

For reasons that are not clear to me, LSAN does not symbolicate the report. To get the functions at the reported offsets, perform the following steps (there might be easier steps, please update this document if you know any).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I'm not opposed to including this, is it a known problem? @shahmishal


### Making Local Changes Inside the Container

For example, to install vim in the container run
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh. Perhaps we should have a development version of the CI images that contain a bunch of nice-to-have tools (eg. vim). @shahmishal

@ahoppen ahoppen merged commit 3bd764d into swiftlang:main Nov 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants