Merge pull request #1218 from nnethercote/demote-futures

Demote `futures` to only run in the "stable" set.

Author: nnethercote

ci/check-profiling.sh

@@ -129,7 +129,7 @@ test   -f results/eprintln-Test-helloworld-Check-Full
 test ! -s results/eprintln-Test-helloworld-Check-Full
 
 # llvm-lines. `Debug` not `Check` because it doesn't support `Check` profiles.
-# Including both `helloworld` and `futures` benchmarks, as they exercise the
+# Including both `helloworld` and `regex` benchmarks, as they exercise the
 # zero dependency and the greater than zero dependency cases, respectively, the
 # latter of which has broken before.
 RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=debug \
@@ -138,12 +138,12 @@ RUST_BACKTRACE=1 RUST_LOG=raw_cargo_messages=trace,collector=debug,rust_sysroot=
         --id Test \
         --profiles Debug \
         --cargo $bindir/cargo \
-        --include helloworld,futures \
+        --include helloworld,regex \
         --scenarios Full
 test -f results/ll-Test-helloworld-Debug-Full
 grep -q "Lines.*Copies.*Function name" results/ll-Test-helloworld-Debug-Full
-test -f results/ll-Test-futures-Debug-Full
-grep -q "Lines.*Copies.*Function name" results/ll-Test-futures-Debug-Full
+test -f results/ll-Test-regex-Debug-Full
+grep -q "Lines.*Copies.*Function name" results/ll-Test-regex-Debug-Full
 
 
 #----------------------------------------------------------------------------

collector/benchmarks/README.md

@@ -25,7 +25,6 @@ They mostly consist of real-world crates.
   ensure a lot of invariants. Stresses anything related to resolving
   trait bounds, by having a lot of trait impls for a large number of different
   types.
-- **futures**: A futures implementation. Used by many Rust programs.
 - **helloworld**: A trivial program. Gives a lower bound on compile time.
 - **html5ever**: An HTML parser. Stresses macro parsing code significantly.
 - **hyper-2**: A fairly large crate. Utilizes async/await, and used by
@@ -124,7 +123,11 @@ Rust code being written today.
 
 - **encoding**: An old crate providing character encoding support. Contains
   some large tables.
-- **futures**: See above.
+- **futures**: v0.1.0 of the popular `futures` crate, which was used by many
+  Rust programs. Newer versions of this crate (e.g. v0.3.21 from February 2021)
+  contain very little code, instead relying on sub-crates. This makes them less
+  interesting as benchmarks, because we only measure final crate compilation.
+  This is why there is no futures crate among the primary benchmarks.
 - **html5ever**: See above.
 - **inflate**: An old implementation of the DEFLATE algorithm. Contains
   a very large function containing many locals and basic blocks, similar to

collector/benchmarks/futures/perf-config.json

@@ -1,3 +1,3 @@
 {
-    "category": "primary-and-stable"
+    "category": "stable"
 }