Restrict certain gems on certain versions of ruby

[robotdana]

As rspec gitignores its Gemfile.lock it can mean new prs can fail with unrelated changes because bundler will get the most recent versions of gems which might have new and exciting incompatibilities. (as i found with #2749)

Also because the travis agents cache these bundles and don't force an update it can sometimes just work

[robotdana]

Alright I think this probably needs to be a maintainer from here

1. i think there should have an automated travis build that runs without the cached bundler directory (IF ENV or something) forcing the latest resolved versions of all gems so that its easy to tell what version incompatibilities have showed up and new prs don't have to deal with trying to get travis to setup before seeing if their tests pass in all rubies
   (still use the cached bundler directory for builds other than this automated build).
2. make sure all the repos pass this (with changes like i've done here).
3. ???
4. profit

[JonRowe]

This should change to > 13.0.0 in this sort of matrix

[JonRowe]

As rspec gitignores its Gemfile.lock it can mean new prs can fail with unrelated changes because bundler will get the most recent versions of gems which might have new and exciting incompatibilities.

Its considered best practise to do so by the way, it ensures that builds don't go stale locked to some outdated dependency while your users get fresh ones, in fact we have cron builds that check main every week to find these problems (I reran one earlier when you first opened your other PR which passed in fact)

[pirj]

Sorry, I'm a bit out of context. What problem exactly this change is addressing, @robotdana ? Were there build failures?
We have had a fair share of similar kind of fixes on rspec-rails side, and it turns out that Bundler should be resolving versions considering the Ruby version constraint, but in certain situations it doesn't. I'd rather apply the effort there.

[robotdana]

What problem exactly this change is addressing, @robotdana ? Were there build failures?

yeah there were sometimes build failures depending on whether the bundler cache was there or not (if it was there, bundler happily used the working versions and had no issues. if it wasn't there, then bundler would fetch the newest versions, which did have ruby version incompatibilities)

e.g. this build: https://travis-ci.org/github/rspec/rspec-core/jobs/717498044

I've just done a bit of reading on bundler and required_ruby_version, and i'm so much more confused than i was before. it seems like it should work. it demonstrably doesn't, at least, not consistently

in fact we have cron builds that check main every week to find these problems (I reran one earlier when you first opened your other PR which passed in fact)

if you mean the this build https://travis-ci.org/github/rspec/rspec-core/jobs/717567773 its doesn't necessarily consider the latest versions bundler would resolve instead whatever versions were cached, only updating things that it needs to to match the Gemfile.

[JonRowe]

I've noticed an increase in flakey builds due to bundler issues of late, but they are typically resolved with a job restart as I've demonstrated, but if these changes increase the reliability of the build I guess I'm for them

[pirj]

if you mean the this build

More those two changes actually rspec/rspec-rails#2366 rspec/rspec-rails#2360

I understand now, this is to protect from the same flakiness. But I still don't understand the need for the cache removal.

[robotdana]

the intent was to force bundler to not use whatever versions that it has cached that still satisfy the gemfile, and instead fetch the latest version of the gem that satisfies the gemfile

[robotdana]

though if bundler address this: rubygems/rubygems#3594 the problem goes away

[JonRowe]

Thats a very interesting issue, that sounds plausible thats what we're hitting, as with Travis kicking off a large amounts of builds simultaneously the chances of that happening go up.

[pirj]

instead fetch the latest version of the gem that satisfies the gemfile

I wouldn't rely on that

though if bundler address this: rubygems/rubygems#3594 the problem goes away

There's evidence that it's not as simple as that. rspec/rspec-rails#2366 (comment)

[pirj]

We are relying on a more stable Bundler API right now. Also, RubyGems version support DB was retroactively updated.
This allowed us to drop all Ruby version-specific constraints from the Gemfile in upcoming 4.0 release.
Thanks!