Skip to content

perf triage report for this week. #1560

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 22, 2023
Merged

perf triage report for this week. #1560

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
272 changes: 272 additions & 0 deletions triage/2023-03-21.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,272 @@
# 2023-03-21 Triage Log

A mixed week, with some nice wins, but also at least two PR's that were
subsequently reverted, such as the upgrade to LLVM 16. We do want to
note PR #108944, which cut down on crate metadata, binary sizes, and
was an overall win on execution time for many benchmarks.

Triage done by **@pnkfelix**.
Revision range: [00587489..ef03fda3](https://perf.rust-lang.org/?start=0058748944abb3282aba0e0a74823c6411703565&end=ef03fda339923e659d3d3ca3321de887316d2807&absolute=false&stat=instructions%3Au)

**Summary**:

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.8% | [0.2%, 2.3%] | 31 |
| Regressions ❌ <br /> (secondary) | 1.5% | [0.3%, 3.0%] | 40 |
| Improvements ✅ <br /> (primary) | -1.1% | [-6.7%, -0.2%] | 101 |
| Improvements ✅ <br /> (secondary) | -3.9% | [-16.8%, -0.5%] | 45 |
| All ❌✅ (primary) | -0.6% | [-6.7%, 2.3%] | 132 |


1 Regressions, 4 Improvements, 11 Mixed; 2 of them in rollups
37 artifact comparisons made in total

#### Regressions

Rollup of 10 pull requests [#109206](https://github.com/rust-lang/rust/pull/109206) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=cd6c574af3886c41f34086d90df42c3da0144693&end=7ac4b82ddd596a218cac8cd6b88a91b54fcdcf13&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.5% | [0.4%, 0.7%] | 2 |
| Regressions ❌ <br /> (secondary) | 1.1% | [0.9%, 1.3%] | 4 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | 0.5% | [0.4%, 0.7%] | 2 |

* as noted on PR, primary regressions are doc benchmarks, and secondary regressions are tt-muncher ... likely being noisy
* marking as triaged.

#### Improvements

Remove identity_future indirection [#104833](https://github.com/rust-lang/rust/pull/104833) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=0058748944abb3282aba0e0a74823c6411703565&end=669e75163957f8f2408d515ce2da3516cb31f747&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -0.5% | [-0.5%, -0.5%] | 1 |
| Improvements ✅ <br /> (secondary) | -3.9% | [-14.3%, -0.9%] | 19 |
| All ❌✅ (primary) | -0.5% | [-0.5%, -0.5%] | 1 |


fix ignore header in MSVC test [#108809](https://github.com/rust-lang/rust/pull/108809) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=18e305dfca6dfef07478378cb9a673fdba6ac3b3&end=cd6c574af3886c41f34086d90df42c3da0144693&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -0.3% | [-0.3%, -0.3%] | 1 |
| Improvements ✅ <br /> (secondary) | -1.0% | [-1.3%, -0.9%] | 6 |
| All ❌✅ (primary) | -0.3% | [-0.3%, -0.3%] | 1 |


fast path for process_obligations [#108815](https://github.com/rust-lang/rust/pull/108815) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=85123d2504885e4670d80dab96f4f9629f2be98b&end=df61fcaec1fdd3b949a7721abfaf37a8ca6e144c&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:------:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.5% | [0.5%, 0.5%] | 1 |
| Regressions ❌ <br /> (secondary) | 1.5% | [1.5%, 1.5%] | 1 |
| Improvements ✅ <br /> (primary) | -2.6% | [-8.6%, -0.3%] | 9 |
| Improvements ✅ <br /> (secondary) | -12.7% | [-21.9%, -7.8%] | 6 |
| All ❌✅ (primary) | -2.3% | [-8.6%, 0.5%] | 10 |


Update host compiler to LLVM 16 [#108802](https://github.com/rust-lang/rust/pull/108802) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=f177b7cb91acd9debd1ed974e6521edb2ab6b097&end=2d0a7def330ed90f416a3429dbb44c5ee2a914e5&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -0.7% | [-1.4%, -0.3%] | 38 |
| Improvements ✅ <br /> (secondary) | -0.9% | [-1.7%, -0.6%] | 11 |
| All ❌✅ (primary) | -0.7% | [-1.4%, -0.3%] | 38 |


#### Mixed

Rollup of 9 pull requests [#109130](https://github.com/rust-lang/rust/pull/109130) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=2e7034ebf7f57066c260b680c5e9dfcf04ca4cd5&end=1716932743a7b3705cbf0c34db0c4e070ed1930d&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.3% | [0.3%, 0.4%] | 6 |
| Regressions ❌ <br /> (secondary) | 0.6% | [0.3%, 1.0%] | 6 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | -0.5% | [-0.5%, -0.3%] | 5 |
| All ❌✅ (primary) | 0.3% | [0.3%, 0.4%] | 6 |

* the primary regressions are all to stm32f4-0.14.0 incremental (unchanged and patched:negate).
* nothing in the rollup stands out as an obvious culprit. (I was briefly curious about #109101, but I think that indeed should only affect compilation of type-erroneous code, IIUC)
* marking as triaged, because I do not think these results warrant deeper investigation at this time.


remove obsolete givens from regionck [#107376](https://github.com/rust-lang/rust/pull/107376) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1716932743a7b3705cbf0c34db0c4e070ed1930d&end=e84e5ff04a647ce28540300244a26ba120642eea&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.6% | [0.5%, 0.7%] | 3 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | -0.4% | [-0.5%, -0.3%] | 3 |
| All ❌✅ (primary) | 0.6% | [0.5%, 0.7%] | 3 |

* primary benchmark regressions were to bitmaps incr-full { check, debug, opt }
* This was reverted in PR #109183; but the revert PR didn't register analogous reversal of the performance impact here.
* The performance regression to bitmaps incr-full was swallowed by big gains registered by PR #109035
* Marking as triaged.

Ensure ptr::read gets all the same LLVM load metadata that dereferencing does [#109035](https://github.com/rust-lang/rust/pull/109035) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=992d154f3a84cc8abcefcf6e6cf3698e4821b506&end=e4b9f86054c581d931f8bad0c87523c53e1e8e3f&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.8% | [0.3%, 1.9%] | 7 |
| Regressions ❌ <br /> (secondary) | 1.5% | [0.3%, 2.9%] | 5 |
| Improvements ✅ <br /> (primary) | -0.8% | [-1.3%, -0.3%] | 14 |
| Improvements ✅ <br /> (secondary) | -2.2% | [-3.7%, -0.4%] | 20 |
| All ❌✅ (primary) | -0.2% | [-1.3%, 1.9%] | 21 |

* already triaged, this was a big broad win.

Implement checked Shl/Shr at MIR building. [#108282](https://github.com/rust-lang/rust/pull/108282) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=ab654863c3d50482f260cf862647f1fe0ff5e010&end=c90eb4825a9faca0d6317292a452859f00d5b786&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | 1.3% | [1.3%, 1.3%] | 2 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | -0.9% | [-1.3%, -0.5%] | 8 |
| All ❌✅ (primary) | - | - | 0 |

* already triaged

Flatten/inline format_args!() and (string and int) literal arguments into format_args!() [#106824](https://github.com/rust-lang/rust/pull/106824) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=7ac4b82ddd596a218cac8cd6b88a91b54fcdcf13&end=1203e0866e6c3659775efcb8aecad21dc13ef38b&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | 0.7% | [0.3%, 1.1%] | 2 |
| Improvements ✅ <br /> (primary) | -0.7% | [-1.3%, -0.3%] | 4 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | -0.7% | [-1.3%, -0.3%] | 4 |

* already triaged

Replace ZST operands and debuginfo by constants. [#107270](https://github.com/rust-lang/rust/pull/107270) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1203e0866e6c3659775efcb8aecad21dc13ef38b&end=e386217dd996d293b3bde2285f6f5f4f502d7c17&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 1.2% | [0.4%, 1.9%] | 4 |
| Regressions ❌ <br /> (secondary) | 0.7% | [0.7%, 0.7%] | 1 |
| Improvements ✅ <br /> (primary) | -0.5% | [-0.6%, -0.3%] | 3 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | 0.5% | [-0.6%, 1.9%] | 7 |

* already triaged

Wrap the whole LocalInfo in ClearCrossCrate. [#108944](https://github.com/rust-lang/rust/pull/108944) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=e386217dd996d293b3bde2285f6f5f4f502d7c17&end=511364e7874dba9649a264100407e4bffe7b5425&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 1.6% | [0.3%, 2.4%] | 6 |
| Regressions ❌ <br /> (secondary) | 1.7% | [1.0%, 2.9%] | 16 |
| Improvements ✅ <br /> (primary) | -0.8% | [-1.3%, -0.5%] | 45 |
| Improvements ✅ <br /> (secondary) | -0.8% | [-0.8%, -0.7%] | 5 |
| All ❌✅ (primary) | -0.5% | [-1.3%, 2.4%] | 51 |


* primary instruction count regression was to unicode normalization (5 variations in [1.5%, 2.4%] range). Slightly more muted for cycle count and wall time there.
* as noted by @lqd and @nnethercote on PR comments: crate metadata + binary size improved all around.
* overall this is a win
* marking as triaged.

Upgrade to LLVM 16 [#107224](https://github.com/rust-lang/rust/pull/107224) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=2d0a7def330ed90f416a3429dbb44c5ee2a914e5&end=4a04d086cac54a41517d5657b59d5fe2caca2d71&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 1.2% | [0.3%, 3.7%] | 63 |
| Regressions ❌ <br /> (secondary) | 1.1% | [0.2%, 2.7%] | 25 |
| Improvements ✅ <br /> (primary) | -1.1% | [-3.4%, -0.5%] | 49 |
| Improvements ✅ <br /> (secondary) | -1.2% | [-4.3%, -0.3%] | 70 |
| All ❌✅ (primary) | 0.2% | [-3.4%, 3.7%] | 112 |

* instruction count deltas are all over the place
* cycle counts and wall-time seem to paint this PR in a much more positive light
* cycle counts regressions: [ +0.67%, +2.92%] +1.88% 22 (9); improvements: [-10.51%, -1.01%] -2.38% 141 (30)
* wall-time regressions: [ +0.51%, +3.75%] +2.04% 9 (6); improvements: [-10.11%, -0.75%] -2.60% 75 (25)
* marking as triaged, at least with respect to performance impact.
* doesn't really matter, since it was reverted in PR #109326 anyway


Revert "Auto merge of #107224 - nikic:llvm-16, r=cuviper" [#109326](https://github.com/rust-lang/rust/pull/109326) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=4a04d086cac54a41517d5657b59d5fe2caca2d71&end=ab9bb3ea368b2412531a3e8c07ba73d1dd690134&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 1.1% | [0.5%, 3.6%] | 51 |
| Regressions ❌ <br /> (secondary) | 1.2% | [0.3%, 4.5%] | 67 |
| Improvements ✅ <br /> (primary) | -1.2% | [-3.6%, -0.3%] | 64 |
| Improvements ✅ <br /> (secondary) | -1.1% | [-2.4%, -0.2%] | 25 |
| All ❌✅ (primary) | -0.2% | [-3.6%, 3.6%] | 115 |

* already triaged. also, see notes for #107224.

Optimize dep node backtrace and ignore fatal errors [#108524](https://github.com/rust-lang/rust/pull/108524) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=356c651e6d013fe9ca1d47da278ba208a95dbcf9&end=13b7aa4d7faafbed78d195c27f6868f6bd90c2f8&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.4% | [0.4%, 0.4%] | 1 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -0.4% | [-0.6%, -0.2%] | 9 |
| Improvements ✅ <br /> (secondary) | -0.3% | [-0.3%, -0.3%] | 1 |
| All ❌✅ (primary) | -0.4% | [-0.6%, 0.4%] | 10 |

* already triaged

Remove the assume(!is_null) from Vec::as_ptr [#106967](https://github.com/rust-lang/rust/pull/106967) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=84c47b8279b39e165dfebeb529eb6d92592e4f8d&end=ef03fda339923e659d3d3ca3321de887316d2807&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.6% | [0.3%, 1.7%] | 16 |
| Regressions ❌ <br /> (secondary) | 0.7% | [0.2%, 1.4%] | 15 |
| Improvements ✅ <br /> (primary) | -0.8% | [-1.9%, -0.4%] | 4 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | 0.3% | [-1.9%, 1.7%] | 20 |


* results from final PR as it finally landed are a bit different (more broad in number of primary benchmarks effected) than the observations from trial performance run
* we expected to see [~2.0%, 2.2%] opt-full regression to regex and serde-derive
* we actually got a [~.7%, ~1.7%] regression to them instead; but a bunch of other primary benchmarks saw a slight dip (less than 0.7% of a regression for the new cases, many less than 0.4%)
* the cycle-counts and wall-times indicate more of an effect on clap and webrender here, but the effect there shows potentially more benefit than harm
* marking as triaged. Its too hard to make strong connections between the changes made here and actual costs/benefits.

#### Untriaged Pull Requests

- [#109206 Rollup of 10 pull requests](https://github.com/rust-lang/rust/pull/109206)
- [#109130 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/109130)
- [#109097 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/109097)
- [#108944 Wrap the whole LocalInfo in ClearCrossCrate.](https://github.com/rust-lang/rust/pull/108944)
- [#108934 Rollup of 8 pull requests](https://github.com/rust-lang/rust/pull/108934)
- [#108872 Strengthen state tracking in const-prop](https://github.com/rust-lang/rust/pull/108872)
- [#108830 Treat projections with infer as placeholder during fast reject in new solver](https://github.com/rust-lang/rust/pull/108830)
- [#108587 Rollup of 10 pull requests](https://github.com/rust-lang/rust/pull/108587)
- [#108025 rustdoc: add more tooltips to intra-doc links](https://github.com/rust-lang/rust/pull/108025)
- [#107833 Factor query arena allocation out from query caches](https://github.com/rust-lang/rust/pull/107833)
- [#107783 rustdoc: simplify DOM for `.item-table`](https://github.com/rust-lang/rust/pull/107783)
- [#107765 rustc/rustdoc: Perform name resolver cleanups enabled by #94857](https://github.com/rust-lang/rust/pull/107765)
- [#107672 Rollup of 3 pull requests](https://github.com/rust-lang/rust/pull/107672)
- [#107408 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/107408)
- [#107376 remove obsolete `givens` from regionck](https://github.com/rust-lang/rust/pull/107376)
- [#107224 Upgrade to LLVM 16](https://github.com/rust-lang/rust/pull/107224)
- [#107143 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/107143)
- [#107103 Use new solver in `evaluate_obligation` query (when new solver is enabled)](https://github.com/rust-lang/rust/pull/107103)
- [#107101 Filter param-env predicates for errors before calling `to_opt_poly_trait_pred`](https://github.com/rust-lang/rust/pull/107101)
- [#106967 Remove the assume(!is_null) from Vec::as_ptr](https://github.com/rust-lang/rust/pull/106967)
- [#106757 Rollup of 10 pull requests](https://github.com/rust-lang/rust/pull/106757)
- [#105657 Guard ProjectionTy creation against passing the wrong number of substs](https://github.com/rust-lang/rust/pull/105657)
- [#105472 Make encode_info_for_trait_item use queries instead of accessing the HIR](https://github.com/rust-lang/rust/pull/105472)
- [#105426 Catch panics/unwinding in destruction of TLS values](https://github.com/rust-lang/rust/pull/105426)
- [#105378 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/105378)
- [#105323 Perform SimplifyLocals before ConstProp.](https://github.com/rust-lang/rust/pull/105323)
- [#105147 Allow unsafe through inline const](https://github.com/rust-lang/rust/pull/105147)
- [#104566 couple of clippy::perf fixes](https://github.com/rust-lang/rust/pull/104566)
- [#104533 Clean up and harden various methods around trait substs](https://github.com/rust-lang/rust/pull/104533)
- [#104017 Rollup of 5 pull requests](https://github.com/rust-lang/rust/pull/104017)