Allow `enum` and `union` literals to also create SSA values #138759

scottmcm · 2025-03-20T18:41:17Z

Today, Some(x) always goes through an alloca, even in trivial cases where the niching means the constructor doesn't even change the value.

For example, https://rust.godbolt.org/z/6KG6PqoYz

pub fn demo(r: &i32) -> Option<&i32> {
    Some(r)
}

currently emits the IR

define align 4 ptr @demo(ptr align 4 %r) unnamed_addr {
start:
  %_0 = alloca [8 x i8], align 8
  store ptr %r, ptr %_0, align 8
  %0 = load ptr, ptr %_0, align 8
  ret ptr %0
}

but with this PR it becomes just

define align 4 ptr @demo(ptr align 4 %r) unnamed_addr {
start:
  ret ptr %r
}

(Of course the optimizer can clean that up, but it'd be nice if it didn't have to -- especially in debug where it doesn't run. This is like #123886, but that only handled non-simd structs -- this PR generalizes it to all non-simd ADTs.)

Doing this means handing variants other than FIRST_VARIANT, handling the active field for unions, refactoring the discriminant code so the Place and Operand parts can share the calculation, etc.

Other PRs that led up to this one:

rustbot · 2025-03-20T18:41:22Z

r? @Nadrieril

rustbot has assigned @Nadrieril.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2025-03-20T18:41:24Z

Some changes occurred in compiler/rustc_codegen_ssa

cc @WaffleLapkin

rustbot · 2025-03-20T18:58:00Z

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

saethlin · 2025-03-20T20:14:02Z

r? codegen

saethlin · 2025-03-20T20:14:16Z

Spiderman meme

scottmcm · 2025-03-20T20:22:43Z

@bors try @rust-timer queue

bors · 2025-03-20T20:23:55Z

⌛ Trying commit aa8ceb8 with merge 875f416...

Allow `enum` and `union` literals to also create SSA values Today, `Some(x)` always goes through an `alloca`, even in trivial cases where the niching means the constructor doesn't even change the value. For example, <https://rust.godbolt.org/z/6KG6PqoYz> ```rust pub fn demo(r: &i32) -> Option<&i32> { Some(r) } ``` currently emits the IR ```llvm define align 4 ptr `@demo(ptr` align 4 %r) unnamed_addr { start: %_0 = alloca [8 x i8], align 8 store ptr %r, ptr %_0, align 8 %0 = load ptr, ptr %_0, align 8 ret ptr %0 } ``` but with this PR it becomes just ```llvm define align 4 ptr `@demo(ptr` align 4 %r) unnamed_addr { start: ret ptr %r } ``` (Of course the optimizer can clean that up, but it'd be nice if it didn't have to -- especially in debug where it doesn't run. This is like rust-lang#123886, but that only handled non-simd `struct`s -- this PR generalizes it to all non-simd ADTs.) There's two commits you can review independently: 1. The first is simplifying how the aggregate handling works. Past-me wrote something overly complicated, needing arrayvecs and zipping, depending on a careful iteration order of the fields, and fragile enough that even for just structs it needed extra double-checks to make sure it even made the right variant. It's replaced with something far more direct that works just like `extract_field`: use the offset to put it in exactly the correct immediate in the `OperandValue`. This doesn't support anything new, just refactors -- including moving some things off `FunctionCx` that had no reason to be there. (I have no idea why my past self put them there.) 2. The second extends that work to support more ADTs. That means handing variants other than `FIRST_VARIANT`, handling the active field for unions, refactoring the discriminant code so the Place and Operand parts can share the calculation, etc.

bors · 2025-03-20T22:32:13Z

☀️ Try build successful - checks-actions
Build commit: 875f416 (875f416b9aa583f09153eac5b56d36fcd932b274)

rust-timer · 2025-03-20T23:49:26Z

Finished benchmarking commit (875f416): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.0%	[0.0%, 0.0%]	1
Regressions ❌ (secondary)	0.2%	[0.2%, 0.3%]	3
Improvements ✅ (primary)	-0.3%	[-0.5%, -0.2%]	7
Improvements ✅ (secondary)	-0.3%	[-0.4%, -0.3%]	2
All ❌✅ (primary)	-0.3%	[-0.5%, 0.0%]	8

Max RSS (memory usage)

Results (primary -2.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.3%	[-2.3%, -2.3%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-2.3%	[-2.3%, -2.3%]	1

Cycles

Results (secondary -2.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.1%	[-2.3%, -1.6%]	8
All ❌✅ (primary)	-	-	0

Binary size

Results (primary -0.1%, secondary -0.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.1%	[-0.2%, -0.0%]	57
Improvements ✅ (secondary)	-0.1%	[-0.3%, -0.0%]	27
All ❌✅ (primary)	-0.1%	[-0.2%, -0.0%]	57

Bootstrap: 775.297s -> 774.424s (-0.11%)
Artifact size: 365.52 MiB -> 365.52 MiB (0.00%)

rustbot · 2025-03-23T04:48:35Z

Some changes occurred to the CTFE machinery

cc @RalfJung, @oli-obk, @lcnr

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri

Some changes occurred in compiler/rustc_codegen_cranelift

cc @bjorn3

scottmcm · 2025-03-28T21:49:10Z

Ben said I should re-roll this
r? codegen

workingjubilee · 2025-05-01T04:52:21Z

hello.

…gjubilee Remove unneeded `FunctionCx` from some codegen methods No changes; just removing the `self` that wasn't needed. r? workingjubilee cc rust-lang#138759 (comment)

Rollup merge of #142324 - scottmcm:less-functioncx, r=workingjubilee Remove unneeded `FunctionCx` from some codegen methods No changes; just removing the `self` that wasn't needed. r? workingjubilee cc #138759 (comment)

bors · 2025-06-11T21:26:41Z

☔ The latest upstream changes (presumably #142358) made this pull request unmergeable. Please resolve the merge conflicts.

Remove unneeded `FunctionCx` from some codegen methods No changes; just removing the `self` that wasn't needed. r? workingjubilee cc rust-lang/rust#138759 (comment)

…kingjubilee CodeGen: rework Aggregate implemention for rvalue_creates_operand cases A non-trivial refactor pulled out from rust-lang#138759 r? workingjubilee The previous implementation I'd written here based on `index_by_increasing_offset` is complicated to follow and difficult to extend to non-structs. This changes the implementation, without actually changing any codegen (thus no test changes either), to be more like the existing `extract_field` (<https://github.com/rust-lang/rust/blob/2b0274c71dba0e24370ebf65593da450e2e91868/compiler/rustc_codegen_ssa/src/mir/operand.rs#L345-L425>) in that it allows setting a particular field directly. Notably I've found this one much easier to get right, in particular because having the `OperandRef<Result<V, Scalar>>` gives a really useful thing to include in ICE messages if something did happen to go wrong.

Rollup merge of #142383 - scottmcm:operandref-builder, r=workingjubilee CodeGen: rework Aggregate implemention for rvalue_creates_operand cases A non-trivial refactor pulled out from #138759 r? workingjubilee The previous implementation I'd written here based on `index_by_increasing_offset` is complicated to follow and difficult to extend to non-structs. This changes the implementation, without actually changing any codegen (thus no test changes either), to be more like the existing `extract_field` (<https://github.com/rust-lang/rust/blob/2b0274c71dba0e24370ebf65593da450e2e91868/compiler/rustc_codegen_ssa/src/mir/operand.rs#L345-L425>) in that it allows setting a particular field directly. Notably I've found this one much easier to get right, in particular because having the `OperandRef<Result<V, Scalar>>` gives a really useful thing to include in ICE messages if something did happen to go wrong.

CodeGen: rework Aggregate implemention for rvalue_creates_operand cases A non-trivial refactor pulled out from rust-lang/rust#138759 r? workingjubilee The previous implementation I'd written here based on `index_by_increasing_offset` is complicated to follow and difficult to extend to non-structs. This changes the implementation, without actually changing any codegen (thus no test changes either), to be more like the existing `extract_field` (<https://github.com/rust-lang/rust/blob/2b0274c71dba0e24370ebf65593da450e2e91868/compiler/rustc_codegen_ssa/src/mir/operand.rs#L345-L425>) in that it allows setting a particular field directly. Notably I've found this one much easier to get right, in particular because having the `OperandRef<Result<V, Scalar>>` gives a really useful thing to include in ICE messages if something did happen to go wrong.

scottmcm · 2025-06-19T06:46:24Z

Perf was good three months ago, but having redone this a bunch it's probably worth double-checking:
@bors try @rust-timer queue

Allow `enum` and `union` literals to also create SSA values Today, `Some(x)` always goes through an `alloca`, even in trivial cases where the niching means the constructor doesn't even change the value. For example, <https://rust.godbolt.org/z/6KG6PqoYz> ```rust pub fn demo(r: &i32) -> Option<&i32> { Some(r) } ``` currently emits the IR ```llvm define align 4 ptr `@demo(ptr` align 4 %r) unnamed_addr { start: %_0 = alloca [8 x i8], align 8 store ptr %r, ptr %_0, align 8 %0 = load ptr, ptr %_0, align 8 ret ptr %0 } ``` but with this PR it becomes just ```llvm define align 4 ptr `@demo(ptr` align 4 %r) unnamed_addr { start: ret ptr %r } ``` (Of course the optimizer can clean that up, but it'd be nice if it didn't have to -- especially in debug where it doesn't run. This is like #123886, but that only handled non-simd `struct`s -- this PR generalizes it to all non-simd ADTs.) Doing this means handing variants other than `FIRST_VARIANT`, handling the active field for unions, refactoring the discriminant code so the Place and Operand parts can share the calculation, etc. Other PRs that led up to this one: - #142005 - #142103 - #142324 - #142383

bors · 2025-06-19T06:47:36Z

⌛ Trying commit 3c75d48 with merge cf39981...

scottmcm · 2025-06-19T07:50:16Z

compiler/rustc_codegen_ssa/src/mir/rvalue.rs

-            mir::Rvalue::Aggregate(ref kind, _) => {
-                let allowed_kind = match **kind {
-                    // This always produces a `ty::RawPtr`, so will be Immediate or Pair
-                    mir::AggregateKind::RawPtr(..) => true,
-                    mir::AggregateKind::Array(..) => false,
-                    mir::AggregateKind::Tuple => true,
-                    mir::AggregateKind::Adt(def_id, ..) => {
-                        let adt_def = self.cx.tcx().adt_def(def_id);
-                        adt_def.is_struct() && !adt_def.repr().simd()
-                    }
-                    mir::AggregateKind::Closure(..) => true,
-                    // FIXME: Can we do this for simple coroutines too?
-                    mir::AggregateKind::Coroutine(..) | mir::AggregateKind::CoroutineClosure(..) => false,
-                };
-                allowed_kind && {


As an update from previous revisions of this PR, note that we no longer need to check the AggregateKind here at all. The layout check is enough to cover all the possibilities.

bors · 2025-06-19T09:18:41Z

☀️ Try build successful - checks-actions
Build commit: cf39981 (cf39981b11b1dcc0f8260a7d2ea06f6b942a3daf)

rust-timer · 2025-06-19T14:55:16Z

Finished benchmarking commit (cf39981): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.4%, -0.3%]	3
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.3%]	2
All ❌✅ (primary)	-0.4%	[-0.4%, -0.3%]	3

Max RSS (memory usage)

Results (primary -4.6%, secondary -0.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.6%	[3.6%, 3.6%]	1
Improvements ✅ (primary)	-4.6%	[-4.6%, -4.6%]	1
Improvements ✅ (secondary)	-4.6%	[-4.6%, -4.6%]	1
All ❌✅ (primary)	-4.6%	[-4.6%, -4.6%]	1

Cycles

Results (secondary 8.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	8.7%	[8.7%, 8.7%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

Results (primary -0.1%, secondary -0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.1%	[-0.2%, -0.0%]	41
Improvements ✅ (secondary)	-0.1%	[-0.3%, -0.0%]	18
All ❌✅ (primary)	-0.1%	[-0.2%, -0.0%]	41

Bootstrap: 692.997s -> 691.442s (-0.22%)
Artifact size: 372.00 MiB -> 371.99 MiB (-0.00%)

rustbot assigned Nadrieril Mar 20, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 20, 2025

This comment has been minimized.

Sign in to view

rustbot assigned saethlin and unassigned Nadrieril Mar 20, 2025

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 20, 2025

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Mar 20, 2025

This comment has been minimized.

Sign in to view

scottmcm force-pushed the operand-builder branch from 9423566 to a9031aa Compare March 23, 2025 05:04

scottmcm mentioned this pull request Mar 23, 2025

Simplify PartialOrd on tuples containing primitives #138135

Merged

rustbot assigned workingjubilee and unassigned saethlin Mar 28, 2025

scottmcm mentioned this pull request Apr 1, 2025

Rust fails to optimize away useless unwrap check #57166

Open

scottmcm mentioned this pull request Jun 11, 2025

CodeGen: rework Aggregate implemention for rvalue_creates_operand cases #142383

Merged

Allow all MIR Aggregates to take the operand path (if layout permits)

3c75d48

scottmcm force-pushed the operand-builder branch from 4c822e9 to 3c75d48 Compare June 19, 2025 06:38

scottmcm marked this pull request as ready for review June 19, 2025 06:39

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 19, 2025

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 19, 2025

This comment has been minimized.

Sign in to view

scottmcm commented Jun 19, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

rustbot removed S-waiting-on-perf Status: Waiting on a perf run to be completed. perf-regression Performance regression. labels Jun 19, 2025

Allow enum and union literals to also create SSA values #138759

Are you sure you want to change the base?

Allow enum and union literals to also create SSA values #138759

Uh oh!

Conversation

scottmcm commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Mar 20, 2025

Uh oh!

rustbot commented Mar 20, 2025

Uh oh!

This comment has been minimized.

rustbot commented Mar 20, 2025

Uh oh!

saethlin commented Mar 20, 2025

Uh oh!

saethlin commented Mar 20, 2025

Uh oh!

scottmcm commented Mar 20, 2025

Uh oh!

This comment has been minimized.

bors commented Mar 20, 2025

Uh oh!

bors commented Mar 20, 2025

Uh oh!

This comment has been minimized.

rust-timer commented Mar 20, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

rustbot commented Mar 23, 2025

Uh oh!

This comment has been minimized.

scottmcm commented Mar 28, 2025

Uh oh!

workingjubilee commented May 1, 2025

Uh oh!

bors commented Jun 11, 2025

Uh oh!

scottmcm commented Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

bors commented Jun 19, 2025

Uh oh!

This comment has been minimized.

Choose a reason for hiding this comment

Uh oh!

bors commented Jun 19, 2025

Uh oh!

This comment has been minimized.

rust-timer commented Jun 19, 2025

Overall result: ✅ improvements - no action needed

Uh oh!

Uh oh!

Allow `enum` and `union` literals to also create SSA values #138759

Allow `enum` and `union` literals to also create SSA values #138759

scottmcm commented Mar 20, 2025 •

edited

Loading

scottmcm commented Jun 19, 2025 •

edited

Loading