[Tests] Use %t for test output

[modocache]

Addresses a suggestion by @ddunbar in #20, to use lit's built-in %t
substitution for temporary test output. To prevent output from being
checked in to source control, add Output to the .gitignore.

[ddunbar]

I will try and investigate why test_exec_root isn't doing the right thing, at some point, but for now I am ok with the temporary solution we have. It is easy to mechanically update the tests when we get this fixed.

[modocache]

I actually encountered issues when adding multiple tests in #32 -- sorry for not testing this case!! This pull request addresses the issues from #32.

The problem is that %{test_output} is shared across test runs, and that by default lit used multiple threads. The result is that when adding multiple tests, the wrong files are compared. Here's an example of the test output in such a case:

swift-corelibs-xctest $ lit -v Tests/Functional
-- Testing: 2 tests, 2 threads --
FAIL: SwiftXCTestFunctionalTests :: SingleFailingTestCase/main.swift (1 of 2)
******************** TEST 'SwiftXCTestFunctionalTests :: SingleFailingTestCase/main.swift' FAILED ********************
Script:
--
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -target x86_64-apple-macosx10.11 -L /Users/bgesiak/Library/Developer/Xcode/DerivedData/XCTest-bzxmivvccdrmtvctkfgutfkgbsze/Build/Products/Debug -I /Users/bgesiak/Library/Developer/Xcode/DerivedData/XCTest-bzxmivvccdrmtvctkfgutfkgbsze/Build/Products/Debug -F /Users/bgesiak/Library/Developer/Xcode/DerivedData/XCTest-bzxmivvccdrmtvctkfgutfkgbsze/Build/Products/Debug -Xlinker -rpath -Xlinker /Users/bgesiak/Library/Developer/Xcode/DerivedData/XCTest-bzxmivvccdrmtvctkfgutfkgbsze/Build/Products/Debug /Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/SingleFailingTestCase/main.swift -o /var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/tmprnmkQU/SingleFailingTestCase
/var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/tmprnmkQU/SingleFailingTestCase > /var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/tmp8gVmRe || true
/Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/xctest_checker/xctest_checker.py /var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/tmp8gVmRe /Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/SingleFailingTestCase/main.swift
--
Exit Code: 1

Command Output (stdout):
--
Command 0: "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc" "-sdk" "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk" "-target" "x86_64-apple-macosx10.11" "-L" "/Users/bgesiak/Library/Developer/Xcode/DerivedData/XCTest-bzxmivvccdrmtvctkfgutfkgbsze/Build/Products/Debug" "-I" "/Users/bgesiak/Library/Developer/Xcode/DerivedData/XCTest-bzxmivvccdrmtvctkfgutfkgbsze/Build/Products/Debug" "-F" "/Users/bgesiak/Library/Developer/Xcode/DerivedData/XCTest-bzxmivvccdrmtvctkfgutfkgbsze/Build/Products/Debug" "-Xlinker" "-rpath" "-Xlinker" "/Users/bgesiak/Library/Developer/Xcode/DerivedData/XCTest-bzxmivvccdrmtvctkfgutfkgbsze/Build/Products/Debug" "/Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/SingleFailingTestCase/main.swift" "-o" "/var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/tmprnmkQU/SingleFailingTestCase"
Command 0 Result: 0
Command 0 Output:


Command 0 Stderr:
/Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/SingleFailingTestCase/main.swift:10:21: warning: unknown operating system for build configuration 'os'
#if os(Linux) || os(FreeBSD)
                    ^


Command 1: "/var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/tmprnmkQU/SingleFailingTestCase"
Command 1 Result: 1
Command 1 Output:
None

Command 1 Stderr:


Command 2: "true"
Command 2 Result: 0
Command 2 Output:


Command 2 Stderr:


Command 3: "/Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/xctest_checker/xctest_checker.py" "/var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/tmp8gVmRe" "/Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/SingleFailingTestCase/main.swift"
Command 3 Result: 1
Command 3 Output:


Command 3 Stderr:
Traceback (most recent call last):
  File "/Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/xctest_checker/xctest_checker.py", line 6, in <module>
    xctest_checker.main()
  File "/Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/xctest_checker/xctest_checker/main.py", line 24, in main
    compare.compare(args.actual, args.expected, args.check_prefix)
  File "/Users/bgesiak/GitHub/apple/swift-corelibs-xctest/Tests/Functional/xctest_checker/xctest_checker/compare.py", line 46, in compare
    repr(actual_line), repr(expected_line)))
AssertionError: Actual line did not match the expected regular expression.
Actual: "Test Case 'NegativeAccuracyTestCase.test_equalWithAccuracy_passes' started.\n"
Expected: "Test Case 'SingleFailingTestCase.test_fails' started.\n"




--

********************
PASS: SwiftXCTestFunctionalTests :: NegativeAccuracyTestCase/main.swift (2 of 2)
Testing Time: 0.14s
********************
Failing Tests (1):
    SwiftXCTestFunctionalTests :: SingleFailingTestCase/main.swift

  Expected Passes    : 1
  Unexpected Failures: 1

The problem can be fixed either in this pull request, or by modifying the Xcode project and build_script.py to run lit -j 1, thus restricting the number of threads.

[mike-ferris]

OK, since the current state of this doesn't work once a second test is aded, let's take this. I would still love to find a good way to move this output to outside the source tree.