Better startup times through --dev #179
Conversation
Opposed to what I said in rspec/rspec-core#2444 there is something we can do, I just underestimated its effect. There is `--dev` (see https://github.com/jruby/jruby/wiki/Improving-startup-time#use-the---dev-flag). Locally for the rspec-core --bisect spec this brought the runtime down from 33 to 36 seconds to 21 seconds! It also encompasses disabling invoke dynamic already, so no need for that. It is opposed to server mode that was used however, server mode targets peak performance at the cost of bigger startup and warmup times and is usually intended for running on the server/ benchmarks. It was introduced for no reason apparent to me in bc0b976 and I think --dev fits the use case better and is closer to what most peeps probably run either way. Other changes I did: * collapsed all the envs into one, it shouldn't hurt the other elements in the matrix, we lose the compat version 2.0 but that doesn't seem too important atm if it is please tell me * updated the JRuby version to the most current one
|
Thanks for finding that! I've opened PRs for each of the repos with this update:
Assuming it has the intended effect (as I expect it to), I'll be glad to merge this :). |
|
Thank you! 👍 🎉 |
Startup time is still very much a problem with JRuby, hence feature type aruba cukes that invoke rspec a lot of times are still problematic. The bisect specs do this to the extreme, resulting in long spec run times. Locally a single scenario takes 36 seconds to run for me, on a 4-core desktop computer. An overloaded CI might take longer, hence just double the timeout and hope it's enough. Nothing much else we can do, but try using --dev see rspec/rspec-dev#179 ref: rspec#2442
|
Grrml I just checked the other PRs, and it seems that (once again) my drive by updates of course caused problems, newest JRuby seems to be failing. I'll revert that back to the version you were testing with before and then we can figure out/fix newer JRuby versions in separate PRs. :'( |
|
Thanks, @PragTob. New batch of PRs:
Regarding updating the version of JRuby: a couple of the failures are for pending specs for bugs in JRuby that are apparently fixed, which is good :). For rspec-core, there are some formatter specs that get slightly different output on the new version of JRuby, so we just have to update those. I don't think updating the JRuby version will be too bad, but agree with doing it as a separate step. |
|
@myronmarston the PRs don't seem to have picked up the latest change, they still specify 9.1.12.0 but I changed it back to 9.1.2.0 :) |
|
Thanks, @PragTob! |
Startup time is still very much a problem with JRuby, hence feature type aruba cukes that invoke rspec a lot of times are still problematic. The bisect specs do this to the extreme, resulting in long spec run times. Locally a single scenario takes 36 seconds to run for me, on a 4-core desktop computer. An overloaded CI might take longer, hence just double the timeout and hope it's enough. Nothing much else we can do, but try using --dev see rspec/rspec-dev#179 ref: rspec#2442
Opposed to what I said in rspec/rspec-core#2444 there is something
we can do, I just underestimated its effect. There is
--dev(see https://github.com/jruby/jruby/wiki/Improving-startup-time#use-the---dev-flag).
Locally for the rspec-core --bisect spec this brought the runtime
down from 33 to 36 seconds to 21 seconds!
It also encompasses disabling invoke dynamic already, so no need
for that. It is opposed to server mode that was used however,
server mode targets peak performance at the cost of bigger startup
and warmup times and is usually intended for running on the server/
benchmarks. It was introduced for no reason apparent to me in
bc0b976 and I think --dev fits the use case better and is
closer to what most peeps probably run either way.
Other changes I did:
elements in the matrix, we lose the compat version 2.0 but
that doesn't seem too important atm if it is please tell me
runtime comparison: