Skip to content

Print additional details when emitting assertion failure messages #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 8, 2016

Conversation

briancroom
Copy link
Contributor

This brings the output in line with what Darwin's XCTest outputs

Take a failing assertion like:

func testAssertEqual() {
    XCTAssertEqual(1, 2, "message", file: "test.swift")
}

The current behavior of swift-corelibs-xctest is to produce a failure message like:

test.swift:36: error: Test.testAssertEqual : message

Darwin's XCTest produces:

test.swift:36: error: -[Test.Test testAssertEqual] : XCTAssertEqual failed: ("Optional(1)") is not equal to ("Optional(2)") - message

After this patch, the output from swift-corelibs-xctest would be:

test.swift:36: error: Test.testAssertEqual : XCTAssertEqual failed: ("Optional(1)") is not equal to ("Optional(2)") - message

Note that this now includes the name of the failing assertion function and includes details about the asserted expressions when appropriate.

This change just screams for test coverage, but I guess that will have to wait until #20 or similar gets merged. In the meantime, though, here is a test fixture that I wrote while working on this, and have been using to manually verify the new behavior.

@ddunbar
Copy link
Contributor

ddunbar commented Dec 24, 2015

It would be great to get this in, the current failure messages are very unfortunate when we see CI failures using a Linux XCTest.

@parkera
Copy link
Contributor

parkera commented Jan 4, 2016

This looks good to me. @mike-ferris-apple should take a look too.

@briancroom briancroom force-pushed the improveAssertionMessages branch from bf3fddf to 712a5db Compare January 8, 2016 00:42
@briancroom
Copy link
Contributor Author

I've just rebased this now.

@mike-ferris
Copy link

This looks good and some testing with Xcode's XCTest shows that the output is not almost the same for these. The one exception seems to be XCTAssertNil where you have the failureDetails returning ""not nil"". To get the same output as Xcode's XCTest, this should return nil. Can you make that tweak? Then I'll definitely merge this! Thanks for providing this, it will make the output a ton more useful.

@briancroom briancroom force-pushed the improveAssertionMessages branch from 712a5db to 78600a0 Compare January 8, 2016 17:36
This brings the output in line with what Darwin's XCTest outputs
* Includes the name of the failing assertion function
* Includes details about the asserted expressions when appropriate
@briancroom briancroom force-pushed the improveAssertionMessages branch from 78600a0 to a4b543c Compare January 8, 2016 17:36
@briancroom
Copy link
Contributor Author

Good catch @mike-ferris-apple! I think I confused myself by choosing "notnil" as the object in the fixture I was working off of. 😁

I went back to closely compare the output from Xcode's XCTest again, and realized that does include the actual value of the expression in the message, e.g.:

test.swift:80: error: -[Test.Test testAssertNil] : XCTAssertNil failed: "MyString" - message

I have updated the PR to properly capture and emit this expression value in the failure message.

@mike-ferris
Copy link

Thanks!

mike-ferris pushed a commit that referenced this pull request Jan 8, 2016
Print additional details when emitting assertion failure messages
@mike-ferris mike-ferris merged commit b0cd443 into swiftlang:master Jan 8, 2016
@briancroom briancroom deleted the improveAssertionMessages branch January 8, 2016 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants