Skip to content

[Tests] Replace %{built_tests_dir} with %T #130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 29, 2016

Conversation

modocache
Copy link
Contributor

When I first added functional tests in 589b23d, I struggled against lit's design: I attempted to prevent test output from being generated alongside the tests themselves. It wasn't until 8181c16 that I saw the error of my ways: lit places test output in the next to the tests in order to make them easier to debug. That commit embraced this design, and excluded the test output from source control.

However, we can do better. The XCTest lit tests produce test executables, which the test suite currently attempts to hide in a tmpdir somewhere. These executables are useful: it's nice to be able to re-run an executable at will.

Change the location of the where the executables are generated. Instead of placing them in %{built_tests_dir}, place them in an output directory next to the test itself (a concept built into lit
and provided by the %T substitution). Since we no longer use %{built_tests_dir}, remove it.

/cc @ddunbar

When I first added functional tests in 589b23d, I struggled against
`lit`'s design: I attempted to prevent test output from being
generated alongside the tests themselves. It wasn't until 8181c16
that I saw the error of my ways: `lit` places test output in the
next to the tests in order to make them easier to debug. That commit
embraced this design, and excluded the test output from source control.

However, we can do better. The XCTest `lit` tests produce test
executables, which the test suite currently attempts to hide in a
tmpdir somewhere. These executables are useful: it's nice to be able to
re-run an executable at will.

Change the location of the where the executables are generated.
Instead of placing them in `%{built_tests_dir}`, place them in an
output directory next to the test itself (a concept built into `lit`
and provided by the `%T` substitution). Since we no longer use
`%{built_tests_dir}`, remove it.
@modocache
Copy link
Contributor Author

This change makes it much easier to see the test output. Now, instead of finding the tmpdir %{built_tests_dir} on your system, you can simply run the executable in the Output dir:

modocache@ubuntu:~/GitHub/apple/swift-corelibs-xctest$ Tests/Functional/SingleFailingTestCase/Output/SingleFailingTestCase 
Test Suite 'All tests' started at 06:39:13.606
Test Suite 'Output.xctest' started at 06:39:13.607
Test Suite 'SingleFailingTestCase' started at 06:39:13.607
Test Case 'SingleFailingTestCase.test_fails' started at 06:39:13.607
/home/modocache/GitHub/apple/swift-corelibs-xctest/Tests/Functional/SingleFailingTestCase/main.swift:26: error: SingleFailingTestCase.test_fails : XCTAssertTrue failed - 
Test Case 'SingleFailingTestCase.test_fails' failed (0.0 seconds).
Test Suite 'SingleFailingTestCase' failed at 06:39:13.607
     Executed 1 test, with 1 failure (0 unexpected) in 0.0 (0.0) seconds
Test Suite 'Output.xctest' failed at 06:39:13.608
     Executed 1 test, with 1 failure (0 unexpected) in 0.0 (0.0) seconds
Test Suite 'All tests' failed at 06:39:13.608
     Executed 1 test, with 1 failure (0 unexpected) in 0.0 (0.0) seconds

It's super handy! 💯

@swift-ci please test

@modocache
Copy link
Contributor Author

Tests pass! I think this is a big improvement. @mike-ferris-apple, @briancroom, please merge if you agree. :)

@mike-ferris
Copy link

I don't know much about lit, but the diff makes things look simpler... "T" isn't exactly as descriptive as the "{built_tests_dir}" it replaces, but it looks like you had to do custom stuff to get that defined while presumably T is a more standard lit thing... Seems like a good change.

@mike-ferris mike-ferris merged commit def5695 into swiftlang:master Jun 29, 2016
@modocache modocache deleted the remove-built-tests-dir branch June 29, 2016 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants