Description
When an expectation fails, the reported failure includes an extract from the source code which caused the failure, together with a message what exactly failed. The source code serves to identify the test case and how it works, possibly helping find the reason for the failure.
For example, the expectation (1 + 1).should eq 1
generates the following output:
Failure/Error: (1 + 1).should eq 1
Expected: 1
got: 2
However, this only works well for expectations that fit on a single line.
Granted, this is by far the most common use case, but sometimes expectations cover multiple lines.
Most notably, expect_raises
typically has the raising code in a block on a separate line.
expect_raises RuntimeError, "foobar" do
1 + 1
end
The resulting source excerpt shows only the first line and thus omits the code that's actually being tested.
Failure/Error: expect_raises RuntimeError, "foobar" do
Expected RuntimeError but nothing was raised
The source code does not provide any useful information besides what the failure message already states. It's just duplication noise.
It would be much more helpful to show 1 + 1
as the offending source code.
Additional to expect_raises
there can be other cases where a normal expectation spans multiple lines.
A simplified and extreme example for that:
[
1
].should eq [
] of Int32
The resulting source excerpt is completely useless:
Failure/Error: [
Expected: []
got: [1]
The spec report should handle this better and provide meaningful information in the source excerpt.
We read the source code from the original file, based on the location information associated with the expectation by courtesy of __FILE__
and __LINE__
. Similarly, the magic constant __END_LINE__
would provide the line number of the end location and thus would allow us to read all lines of a multi-line expectation.
This should work at least for the expected value because it's an argument of the should
call. But the start line of the actual value is not trivially available, unfortunately.
There might be some downside if the expectation is extremely long, because that would impact readability of the spec report. Perhaps we'd want to add some upper limit on the number of printed lines?
For expect_raises
in particular, the wrapping call itself often takes a considerable amount of space and does not provide any meaningful information.
Ideally, the report would leave that out and only show the body of the block. I don't think we can easily retrieve the start and end location of the block. But maybe we can use a heuristic to figure out whether the first and last line of an expect_raises
call are safe to strip.