Skip to content

Adding a method to clear the Gimli symbol cache #226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jul 22, 2019
Merged

Adding a method to clear the Gimli symbol cache #226

merged 11 commits into from
Jul 22, 2019

Conversation

Tomcc
Copy link
Contributor

@Tomcc Tomcc commented Jul 11, 2019

As discussed in #222 , in my use case I can't really afford each process to lock hundreds of mb's per thread in the backtrace symbol cache.

I've added a clear_symbol_cache() function to the root of the crate that clears the Gimli mapping cache and uh, does nothing at all if you aren't using Gimli.
I tried to fix libbacktrace as well, but that one doesn't expose a way to free its state even in C.

I'm not that good at Rust still so this PR is more of a prototype, let me know if anything is terrible :)
I know that a few things could be improved:

  • Is it fine to take a dependency on lazy_static?
  • The Mutex on the static is redundant, but I couldn't placate the compiler.
  • Defining fn noop_clear_symbol_cache() {} again for each backend is ugly.

Anyway, there it is.

@alexcrichton
Copy link
Member

Thanks for the PR! Could this perhaps avoid picking up a dependency on lazy_static for now? It's probably best to tackle #225 in a dedicated PR.

Additionally if this didn't actually fix the issue for you I wonder if there's some more investigation to do? There may be more caches that need to be cleared in this crate somewhere I'm not realizing?

(one may be libunwind which may have its own set of internal caches)

@Tomcc
Copy link
Contributor Author

Tomcc commented Jul 12, 2019

Sure, I can avoid lazy_static.

Additionally if this didn't actually fix the issue for you I wonder if there's some more investigation to do? There may be more caches that need to be cleared in this crate somewhere I'm not realizing?

Actually this PR fixed my problem! The extra memory leftover that I mentioned on the other issue was... the println! that I used to debug it.
Interestingly, printing out anything from a background thread seems to allocate a bunch of caches as well... I've already deployed this code though, and it does fix the problem. I caused a few backtraces and it's still sitting at 38mb :)
I could even downsize the VM down to 1gb and the swapping doesn't occur anymore, so I trust what docker stats is telling me.

@alexcrichton alexcrichton merged commit 0a23f48 into rust-lang:master Jul 22, 2019
@alexcrichton
Copy link
Member

Thanks again for this!

@Tomcc
Copy link
Contributor Author

Tomcc commented Jul 23, 2019

Thank you for helping me with this!
Btw, I saw that all my commits with horrible messages went in as-is... I thought I'd have a chance to clean them up before merging them :P
I guess it's too late now!

@alexcrichton
Copy link
Member

Ah no worries I don't tend to be too strict with commit history, it's all good!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants