Fix coveralls and a brittle test

[myronmarston]

• #debug? depends on whether or not the debugger is loaded. The tests for it were brittle and could start to fail if you were running the specs with the debugger loaded.
• On the coveralls UI I'm seeing the same file listed many times. I think it's because we run all the specs together and then run each individually and the multiple test runs confuses it.

[coveralls]

Coverage increased (+0%) when pulling 88c1747 on fix_coveralls into badfb72 on master.

[coveralls]

Coverage increased (+0%) when pulling faf4b53 on fix_coveralls into badfb72 on master.

[coveralls]

Coverage increased (+0%) when pulling 24e4d72 on fix_coveralls into badfb72 on master.

[JonRowe]

On mocks I ended up using wear_merged! which is supposed to sorts the multiple files confusion, the downside is it requires an uber-rescue (which mock's needed anyway) to get it to run on 1.9.2. I've tried the same on exceptions to see if it improves the stats.

I'd quite like to keep the cucumber as part of the coverage stats and if something is covered in them which isnt in specs I'd still like that chalked up as a win, but I'm open to removing it.

[myronmarston]

On mocks I ended up using wear_merged! which is supposed to sorts the multiple files confusion, the downside is it requires an uber-rescue (which mock's needed anyway) to get it to run on 1.9.2. I've tried the same on exceptions to see if it improves the stats.

It's not working on rspec-mocks. It's having the same problems:

Under the hood, I believe it's using Simplecov's merging, which generally works w/o additional configuration when the multiple runs are different binaries (e.g. rspec and cucumber) but requires that you configure SimpleCov.command_name when it's the same binary (as it is for the full rspec suite plus running each individual spec).

On top of that, there's no benefit to using coveralls/simplecov while running the individual spec files -- even if we got the merging to work, it's not going to provide us any more or less coverage.

I'd quite like to keep the cucumber as part of the coverage stats and if something is covered in them which isnt in specs I'd still like that chalked up as a win, but I'm open to removing it.

The problem is that there's no way the basic cucumber setup you had can possibly work. Here's why:

• You were loading coveralls within the cucumber process. On a project that runs all the code in that process (e.g. a typical cucumber suite using capybara) that would work fine.
• However, with our setup, all the cucumber process does is create files, shell out to start a new project (by running the rspec command), and then sets expectations about the exit statuses and output of the rspec processes.
• Each rspec process is a fresh memory space that doesn't have simplecov/coveralls loaded (how could it?), and so it's not going to be able to track the code coverage achieved by the rspec processes.

[JonRowe]

Fair enough