Skip to content

Add support for the new version of DHAT. #341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 7, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 34 additions & 12 deletions collector/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,26 +234,48 @@ except that `$PROFILER` is one of the following.
text output is also written to files with a `clgann` prefix; this output is
much the same as the `cgann`-prefixed files produced by Cachegrind, but
with extra annotations showing function call counts.
- `dhat`: Profile with [DHAT](http://valgrind.org/docs/manual/dh-manual.html),
a heap profiler.
- **Purpose**. DHAT is good for finding which parts of the code are causing a
lot of allocations. This is relevant if another profiler such as
- `exp-dhat`: Profile with [ExpDHAT](http://valgrind.org/docs/manual/dh-manual.html),
an experimental heap profiler that came with Valgrind (`--tool=exp-dhat`)
prior to version 3.15.
- **Purpose**. ExpDHAT is good for finding which parts of the code are
causing a lot of allocations. This is relevant if another profiler such as
`perf-record` or Cachegrind tell you that `malloc` and `free` are hot
functions (as they often are).
- **Slowdown**. Roughly 5--20x.
- **Prerequisites**. DHAT may require a rustc configured with `use-jemalloc =
false` to work well.
- **Configuration**. DHAT is configured within `profile` to run with the
non-default `--tot-blocks-allocd` option, so that it sorts its
output by the number of blocks allocated rather than the number of bytes
allocated. This is because the number of allocations typically has a
greater effect on speed than the size of those allocations; many small
allocations will typically be slower than a few large allocations.
- **Prerequisites**. ExpDHAT may require a rustc configured with
`use-jemalloc = false` to work well.
- **Configuration**. ExpDHAT is configured within `profile` to run with the
non-default `--tot-blocks-allocd` option, so that it sorts its output by
the number of blocks allocated rather than the number of bytes allocated.
This is because the number of allocations typically has a greater effect on
speed than the size of those allocations; many small allocations will
typically be slower than a few large allocations.
- **Output**. Human-readable text output is written to files with a `dhat`
prefix. This file includes summary statistics followed by numerous records,
each of which aggregates data about all the allocations associated with a
particular stack trace: the number of allocations, their average size, and
how often they are read from and written to.
- `dhat`: Profile with [DHAT](http://valgrind.org/docs/manual/dh-manual.html),
a heap profiler that comes with Valgrind (`--tool=dhat`) in versions 3.15
and later. It has the same purpose as ExpDHAT, but is significantly more
powerful.
- **Purpose**. DHAT is good for finding which parts of the code are causing a
lot of allocations. This is relevant if another profiler such as
`perf-record` or Cachegrind tell you that `malloc` and `free` are hot
functions (as they often are). It also gives insight into peak memory
usage, similar to Massif.
- **Slowdown**. Roughly 5--20x.
- **Prerequisites**. DHAT may require a rustc configured with `use-jemalloc =
false` to work well.
- **Configuration**. DHAT is configured within `profile` to run with the
non-default `--num-callers=4` option, which dictates stack depths. (This
value of 4 does not include inlined stack frames, so in practice the depths
of stack traces are a lot more than 4.) This is almost always enough, but
on the rare occasion it isn't, you can the value in `rustc-fake.rs` and
rebuild `collector`. Note that higher values make DHAT run more slowly and
increase the size of its data files.
- **Output**. Raw output is written to files with a `dhout` prefix. Those
files can be viewed with DHAT's viewer (`dh_view.html`).
- `massif`: Profile with
[Massif](http://valgrind.org/docs/manual/ms-manual.html), a heap profiler.
- **Purpose**. Massif is designed to give insight into a program's peak
Expand Down
15 changes: 14 additions & 1 deletion collector/src/bin/rustc-fake.rs
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ fn main() {
assert!(cmd.status().expect("failed to spawn").success());
}

"dhat" => {
"exp-dhat" => {
let mut cmd = Command::new("valgrind");
let has_valgrind = cmd.output().is_ok();
assert!(has_valgrind);
Expand All @@ -106,6 +106,19 @@ fn main() {
assert!(cmd.status().expect("failed to spawn").success());
}

"dhat" => {
let mut cmd = Command::new("valgrind");
let has_valgrind = cmd.output().is_ok();
assert!(has_valgrind);
cmd.arg("--tool=dhat")
.arg("--num-callers=4")
.arg("--dhat-out-file=dhout")
.arg(&rustc)
.args(&args);

assert!(cmd.status().expect("failed to spawn").success());
}

"massif" => {
let mut cmd = Command::new("valgrind");
let has_valgrind = cmd.output().is_ok();
Expand Down
21 changes: 17 additions & 4 deletions collector/src/bin/rustc-perf-collector/execute.rs
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ pub enum Profiler {
PerfRecord,
Cachegrind,
Callgrind,
ExpDHAT,
DHAT,
Massif,
Eprintln,
Expand All @@ -112,6 +113,7 @@ impl Profiler {
"perf-record" => Ok(Profiler::PerfRecord),
"cachegrind" => Ok(Profiler::Cachegrind),
"callgrind" => Ok(Profiler::Callgrind),
"exp-dhat" => Ok(Profiler::ExpDHAT),
"dhat" => Ok(Profiler::DHAT),
"massif" => Ok(Profiler::Massif),
"eprintln" => Ok(Profiler::Eprintln),
Expand All @@ -126,6 +128,7 @@ impl Profiler {
Profiler::PerfRecord => "perf-record",
Profiler::Cachegrind => "cachegrind",
Profiler::Callgrind => "callgrind",
Profiler::ExpDHAT => "exp-dhat",
Profiler::DHAT => "dhat",
Profiler::Massif => "massif",
Profiler::Eprintln => "eprintln",
Expand Down Expand Up @@ -517,16 +520,26 @@ impl<'a> Processor for ProfileProcessor<'a> {
f.flush()?;
}

// DHAT writes its output to stderr. We copy that output into a
// ExpDHAT writes its output to stderr. We copy that output into a
// file in the output dir.
Profiler::DHAT => {
let dhat_file = filepath(self.output_dir, &out_file("dhat"));
Profiler::ExpDHAT => {
let exp_dhat_file = filepath(self.output_dir, &out_file("exp-dhat"));

let mut f = File::create(dhat_file)?;
let mut f = File::create(exp_dhat_file)?;
f.write_all(&output.stderr)?;
f.flush()?;
}

// DHAT produces (via rustc-fake) a data file called 'dhout'. We
// copy it from the temp dir to the output dir, giving it a new
// name in the process.
Profiler::DHAT => {
let tmp_dhout_file = filepath(data.cwd.as_ref(), "dhout");
let dhout_file = filepath(self.output_dir, &out_file("dhout"));

fs::copy(&tmp_dhout_file, &dhout_file)?;
}

// Massif produces (via rustc-fake) a data file called 'msout'. We
// copy it from the temp dir to the output dir, giving it a new
// name in the process.
Expand Down
2 changes: 1 addition & 1 deletion collector/src/bin/rustc-perf-collector/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,7 @@ fn main_result() -> Result<i32, Error> {
'BaseIncr', 'CleanIncr', 'PatchedIncrs', 'All'")
(@arg PROFILER: +required +takes_value
"One of: 'time-passes', 'perf-record', 'cachegrind',\n\
'callgrind', 'dhat', 'massif', 'eprintln'")
'callgrind', 'exp-dhat', 'dhat', 'massif', 'eprintln'")
(@arg ID: +required +takes_value "Identifier to associate benchmark results with")
)
(@subcommand remove_benchmark =>
Expand Down