Skip to content

Reduce memory consumption when compiling flang #124919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

kparzysz
Copy link
Contributor

@kparzysz kparzysz commented Jan 29, 2025

I've been experimenting with various things related to memory usage by template instantiations. This draft will have some commits that I think may be helpful.

cc: @pawosm-arm @tblah

Ignore formatting errors, this is just a draft, but it should work otherwise.

@kparzysz
Copy link
Contributor Author

Moving all template instantiations from Lower/Support/Utils.h to Lower/Support/Utils.cpp reduced peak memory use when compiling OpenMP.cpp (on my machine) from 4.9g to 3.7g.

The change was to only leave template declarations in the header file and explicitly instantiate every case that was previously instantiated implicitly. If a new implicit instantiation appears later on, it will cause a compilation error (and the explicit instantiation will need to be added manually). The change lists all instantiations explicitly, but it could likely be shortened using macros.

Copy link
Contributor

@tblah tblah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea.

If these are generated from a script, it might be worth committing the script as well so the file can be more easily re-generated.

@kparzysz
Copy link
Contributor Author

The way it was generated was a combination of

  1. Annotating clang to print class and function instantiations (with extra info).
  2. Capturing the dump from compiling OpenMP.cpp.
  3. Filtering out everything from the dumps except the instantiations' pretty names, e.g. void foo<T>() [T=Fortran::evaluate::Something].
  4. Using a python script to plug the T from the list in [] into the template name, then print the resulting string.

Essentially this is a list of things that actually got instantiated. There are some patterns in the output, and 99% of it could be auto-generated, but I don't have any scripts to do that.

@tblah
Copy link
Contributor

tblah commented Jan 30, 2025

Okay that's fine without a script then

@kparzysz
Copy link
Contributor Author

I'm finishing a python script to generate these instantiations. Should have something later today or tomorrow.

@kparzysz
Copy link
Contributor Author

The PR with the script: #125216

@kparzysz kparzysz closed this May 19, 2025
@kparzysz kparzysz deleted the users/kparzysz/spr/template-memory branch May 19, 2025 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants