add support for --example-matches -E that allows matching examples with regex syntax #2586

tansaku · 2018-12-21T15:54:16Z

worked on this in @AgileVentures mob with @mattwr18 and @okothkongo1

…th regex syntax, fixes rspec#2584 Co-authored-by: Matt Rider <[email protected]>

JonRowe · 2018-12-21T18:29:54Z

Great start, could you add a cucumber scenario!

JonRowe · 2018-12-21T18:32:17Z

lib/rspec/core/option_parser.rb

@@ -226,6 +226,11 @@ def parser(options)
          (options[:full_description] ||= []) << Regexp.compile(Regexp.escape(o))
        end

+        parser.on('-E', '--example-matches STRING', "Run examples whose full nested names match REGEX (may be",


Should probably say REGEX or PATTERN rather than string.

updated as suggested

You're going to need to mark the option parser block with a comment to turn off the rubocop rule, and then turn it back on at the end, sorry!

@JonRowe thanks - not exactly sure what comment to add - can you be more explicit? Or is it even something you can do yourself - you can edit our PR directly in github right?

Hm thats a new feature :P, I've fixed the rubocop failures but your scenario isn't working!

thanks - it seems to be passing on some versions of ruby and not others - do we need to test locally on all the different versions of ruby going back to 1.8.7?

It needs to work on all versions back to 1.8.7 at this point in time, I would suggest seeing what regex or command line differences there maybe in older versions?

tansaku · 2018-12-24T16:23:54Z

sure thing - will get some cukes in in the new year

tansaku · 2019-01-04T17:17:03Z

thanks @mattwr18 for pushing up our interim work on cukes

@JonRowe we're a bit stuck on trying to do a scenario where a regex is being passed through the step definition. It seems like the step definition, or the I run command is escaping the regex, although maybe we don't quite have things hooked up correctly ...

we were trying:

  Scenario: Match only matching regex
    When I run `rspec . --example-matches "first\b" --format d`
    Then the examples should all pass
    And the output should contain all of these:
      |first|
    And the output should not contain any of these:
      |first example in first group|
      |second example in first group|
      |first example in second group|
      |second example in second group|
      |first example in third group|
      |nested group first example in nested group|
      |nested group second example in nested group|

JonRowe · 2019-01-04T17:37:25Z

Yeah you probably have to escape it

fables-tales · 2019-01-05T09:37:17Z

Folks, this looks really good. Thanks so much for the contribution! Can't wait to see it across the line :)

tansaku · 2019-01-07T10:51:15Z

thanks @samphippen :-)

@JonRowe yeah - I think the problem is that it is being escaped and we need to prevent that ... anyway we'll be reviewing the details again this coming Friday as part of our agile development mob

JonRowe · 2019-01-07T10:59:04Z

@tansaku Apologies I don't think I was clear, I believe you need to escape it yourself. So that \b becomes \\b so it then gets run correctly. I think.

tansaku · 2019-01-08T09:39:19Z

thanks @JonRowe - we'll certainly try that :-)

tansaku · 2019-01-11T16:04:13Z

@JonRowe we tried escaping it precisely as you say, but that's not what we want. We need to test the unescaped regex is processed correctly by rspec, but it seems the cucumber automatically escapes regex coming through variables in step definitions. We're currently working on a way to override that default cucumber functionality in order to have the cucumber test correctly test what we want ...

tansaku · 2019-01-25T16:03:15Z

@JonRowe @samphippen - apologies for the delay - we think it's all working now - actually there wasn't a problem with escaping, our tests were just a little whacked - we've fixed them and checked that they fail when the functionality is removed - hope this is now ready to be merged! :-)

tansaku · 2019-01-25T17:19:05Z

green for us locally, but failing the CI builds :-(

JonRowe · 2019-01-25T17:22:49Z

@tansaku See my comment, its a rubocop thing

tansaku · 2019-02-08T16:03:42Z

@JonRowe we've been trying to install 1.8.7 locally in order to replicate the CI failures but that seems like a mountain of errors and issues to get through to even run the test locally (installing older version of bundler etc. etc.).

If you're interested here's the series of errors and fixes so far:

$ gem install bundler
Fetching: bundler-2.0.1.gem (100%)
ERROR:  Error installing bundler:
	bundler requires Ruby version >= 2.3.0.

fixed with

$ gem install bundler -v 1.0
Fetching: bundler-1.0.0.gem (100%)
Successfully installed bundler-1.0.0
Installing ri documentation for bundler-1.0.0
/Users/tansaku/.rvm/rubies/ruby-1.8.7-head/lib/ruby/1.8/rdoc/rdoc.rb:280: warning: conflicting chdir during another chdir block
/Users/tansaku/.rvm/rubies/ruby-1.8.7-head/lib/ruby/1.8/rdoc/rdoc.rb:287: warning: conflicting chdir during another chdir block
Done installing documentation for bundler after 6 seconds
1 gem installed

next error

$ bundle
/Users/tansaku/.rvm/gems/ruby-1.8.7-head/gems/bundler-1.0.0/lib/bundler/shared_helpers.rb:3: undefined method `source_index' for Gem:Module (NoMethodError)

fixed with

$ gem update --system 1.8.25

next error

$ bundle
/Users/tansaku/.rvm/gems/ruby-1.8.7-head/gems/bundler-1.0.0/lib/bundler/ui.rb:46: uninitialized constant Gem::SilentUI (NameError)
	from /Users/tansaku/.rvm/gems/ruby-1.8.7-head/gems/bundler-1.0.0/lib/bundler/cli.rb:17:in `initialize'
	from /Users/tansaku/.rvm/gems/ruby-1.8.7-head/gems/bundler-1.0.0/lib/bundler/vendor/thor.rb:246:in `new'

for which the fix is apparently https://stackoverflow.com/a/4864145/316729 but then we get:

$ bundle
NOTE: Gem.source_index is deprecated, use Specification. It will be removed on or after 2011-11-01.
Gem.source_index called from /Users/tansaku/.rvm/gems/ruby-1.8.7-head/gems/bundler-1.0.10/lib/bundler/shared_helpers.rb:3.
/Users/tansaku/.rvm/rubies/ruby-1.8.7-head/lib/ruby/site_ruby/1.8/rubygems/specification.rb:1496:in `method_missing': undefined method `metadata=' for #<Gem::Specification:0x100a49420> (NoMethodError)
	from /Users/tansaku/Documents/GitHub/AgileVentures/rspec-core/rspec-core.gemspec:16
	from /Users/tansaku/.rvm/rubies/ruby-1.8.7-head/lib/ruby/site_ruby/1.8/rubygems/specification.rb:1369:in `initialize'
	from /Users/tansaku/Documents/GitHub/AgileVentures/rspec-core/rspec-core.gemspec:5:in `new'
	from /Users/tansaku/Documents/GitHub/AgileVentures/rspec-core/rspec-core.gemspec:5
[

so feels like a bit of a rabbit hole here

We've analyzed the failure and it seems that it is related to our test code - the app code we changed does not introduce anything new. Ruby 1.8.7 does support the regex we are using ...

1.8.7-p376 :002 > Regexp.compile('hello\b')
 => /hello\b/ 
1.8.7-p376 :003 > r = _
 => /hello\b/ 
1.8.7-p376 :004 > r.match 'hello'
 => #<MatchData "hello"> 
1.8.7-p376 :005 > r.match 'hello '
 => #<MatchData "hello"> 
1.8.7-p376 :006 > r.match 'hellof'
 => nil

we did notice that the way the existing tests were written did make some assumptions about how the rspec output was formatted, that we saw were not necessarily followed, but difficult to check when we can't run locally and the CI is only giving us progress output ...

We've tried pushing up a few changes, but they still fail - it's feeling like we are on a bit of an endless cycle of yak-shaving here :-(

JonRowe · 2019-02-08T18:38:37Z

A quick investigation shows its how the command line interprets the \b on 1.8.7 \b is just b in the string before compilation. \\\b produces \b on 1.8.7 but not on others.

tansaku · 2019-02-11T09:58:29Z

thanks @JonRowe - we didn't see any difference in how the \b was interpreted. It might have been super quick for you, but perhaps you could share the details of how you made that investigation? Copy and paste of your command prompt input and output would be very revealing and educational for us, since we are currently unable to run rspec from the command line in 1.8.7:

→ bundle exec rspec
NOTE: Gem.source_index is deprecated, use Specification. It will be removed on or after 2011-11-01.
Gem.source_index called from /Users/tansaku/.rvm/gems/ruby-1.8.7-head/gems/bundler-1.0.10/lib/bundler/shared_helpers.rb:3.
/Users/tansaku/.rvm/rubies/ruby-1.8.7-head/lib/ruby/site_ruby/1.8/rubygems/specification.rb:1496:in `method_missing': undefined method `metadata=' for #<Gem::Specification:0x110f32af8> (NoMethodError)
	from /Users/tansaku/Documents/GitHub/AgileVentures/rspec-core/rspec-core.gemspec:16
	from /Users/tansaku/.rvm/rubies/ruby-1.8.7-head/lib/ruby/site_ruby/1.8/rubygems/specification.rb:1369:in `initialize'
	from /Users/tansaku/Documents/GitHub/AgileVentures/rspec-core/rspec-core.gemspec:5:in `new'
	from /Users/tansaku/Documents/GitHub/AgileVentures/rspec-core/rspec-core.gemspec:5

JonRowe · 2019-02-11T10:14:36Z

@tansaku I have a working 1.8.7 installation I dare not delete to debug these sorts of things 😂

It's 1.8.7-p374 which was installed with RVM and these versions of bundler / rubygems.

~$ bundler -v
Bundler version 1.15.3
~$ gem -v
YAML safe loading is not available. Please upgrade psych to a version that supports safe loading (>= 2.0).
2.6.14

I investigated by puts in the option parser with running the specs, after some more prodding I think its actually a cucumber issue. (As in the parser produces different output fed to RSpec). I was attempting to hardcode the step for 1.8.7 only to produce the right input but couldn't get it to work and don't have time to work on it my self atm

tansaku · 2019-02-11T11:05:39Z

@JonRowe thanks so much - that's really helpful - really appreciate you taking the time to share the details

BTW, we'd love to know the motivation for maintaining backwards compatibility with 1.8.7 - are there a lot of folks out there still relying on it?

JonRowe · 2019-02-11T11:08:37Z

Historically the RSpec team has held the opinion that we should be one of the last people to support 1.8.7, as we are a "base" dependency, you need to test your code and we shouldn't exclude people on older Rubies.

We also view dropping support for it as a breaking change in terms of semantic versioning.

That said with RSpec 4 we will be dropping support for a swathe of old Rubies, especially as Rubygems and bundler have now done so.

…x works

tansaku · 2019-03-01T15:58:16Z

thanks so much for all your help @JonRowe - I think as it turns out, we can get all green by using regex other than \b - your change allowed the 1.8.7 build to pass, but then failed on some more recent rubies. Our test is effectively same using some other regex syntax (i.e. "[^_]") which does seem to pass for everything, and I think is sufficient since all we are trying to show is that at least some regex works - there's no need to exhaustively test all regex, particularly those that have variations across ruby versions.

So fingers crossed, this latest will go green and can get merged in

JonRowe · 2019-03-01T18:04:17Z

My pleasure, thank you for your time and your patience, merged!

This commit was imported from rspec/rspec-core@d048c36.

add support for --example-matches -E that allows matching examples wi…

f677950

…th regex syntax, fixes rspec#2584 Co-authored-by: Matt Rider <[email protected]>

tansaku force-pushed the 2584_add_option_to_allow_matching_examples_with_regex_syntax branch from b575a41 to f677950 Compare December 21, 2018 16:00

tansaku mentioned this pull request Dec 21, 2018

Use --example STRING rather than :linenumber to future-proof saasbook/hw-ruby-intro#31

Closed

JonRowe reviewed Dec 21, 2018

View reviewed changes

Update option_parser.rb

a6ef31f

Add cucumber test for --example-matches

da07a79

tansaku added 2 commits January 25, 2019 15:57

tweak tests to correctly check for regex matches

464d3b2

add back example file indentation

724876a

add link to 101 regex description

bdcd83a

JonRowe and others added 3 commits January 28, 2019 10:39

Add rubocop rule supression

9aa7d8f

Remove trailing whitespace

d2f891c

try alternative regex

ab29bf4

Change regex for Ruby 1.8.7

0c9524b

no need to do an example with slash b, we just need to show that rege…

5d66290

…x works

JonRowe merged commit ef17239 into rspec:master Mar 1, 2019

JonRowe added a commit that referenced this pull request Mar 1, 2019

Change log for #2586

d048c36

MatheusRich pushed a commit to MatheusRich/rspec-core that referenced this pull request Oct 30, 2020

Change log for rspec#2586

a6144d6

yujinakayama pushed a commit to yujinakayama/rspec-monorepo that referenced this pull request Oct 6, 2021

[core] Change log for rspec/rspec-core#2586

66e6fd7

This commit was imported from rspec/rspec-core@d048c36.

Uh oh!

add support for --example-matches -E that allows matching examples with regex syntax #2586

add support for --example-matches -E that allows matching examples with regex syntax #2586

Uh oh!

Conversation

tansaku commented Dec 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JonRowe commented Dec 21, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tansaku commented Dec 24, 2018

Uh oh!

tansaku commented Jan 4, 2019

Uh oh!

JonRowe commented Jan 4, 2019

Uh oh!

fables-tales commented Jan 5, 2019

Uh oh!

tansaku commented Jan 7, 2019

Uh oh!

JonRowe commented Jan 7, 2019

Uh oh!

tansaku commented Jan 8, 2019

Uh oh!

tansaku commented Jan 11, 2019

Uh oh!

tansaku commented Jan 25, 2019

Uh oh!

tansaku commented Jan 25, 2019

Uh oh!

JonRowe commented Jan 25, 2019

Uh oh!

tansaku commented Feb 8, 2019

Uh oh!

JonRowe commented Feb 8, 2019

Uh oh!

tansaku commented Feb 11, 2019

Uh oh!

JonRowe commented Feb 11, 2019

Uh oh!

tansaku commented Feb 11, 2019

Uh oh!

JonRowe commented Feb 11, 2019

Uh oh!

tansaku commented Mar 1, 2019

Uh oh!

JonRowe commented Mar 1, 2019

Uh oh!

Uh oh!

tansaku commented Dec 21, 2018 •

edited

Loading