Skip to content

Commit c045a1f

Browse files
benlangmuirbenlangmuir
authored
Merge pull request #35 from benlangmuir/docs1
[docs] Add documentation about writing tests for IndexStoreDB
2 parents 90cafe9 + 0a66164 commit c045a1f

File tree

3 files changed

+256
-0
lines changed

3 files changed

+256
-0
lines changed

‎Documentation/Development.md

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Development
2+
3+
This document contains notes about development and testing of IndexStoreDB.
4+
5+
## Table of Contents
6+
7+
* [Writing Tests](#writing-tests)
8+
* [Tibs, the "Test Index Build System"](Tibs.md)
9+
10+
## Writing Tests
11+
12+
As much as is practical, all code should be covered by tests. New tests can be added under the `Tests` directory and should use `XCTest`. The rest of this section will describe the additional tools available in the `ISDBTestSupport` module to make it easier to write good and efficient tests.
13+
14+
Most indexer tests follow a pattern:
15+
16+
1. Build and index a test project
17+
2. Perform index queries
18+
3. Compare the results against known locations in the test code
19+
4. (Optional) modify the code and repeat.
20+
21+
### Test Projects (Fixtures)
22+
23+
Index test projects should be put in the `Tests/INPUTS` directory, and use the [Tibs](Tibs.md) build system to define their sources and targets. An example test project might look like:
24+
25+
```
26+
Tests/
27+
INPUTS/
28+
MyTestProj/
29+
a.swift
30+
b.swift
31+
c.cpp
32+
```
33+
34+
Where `project.json` describes the project's targets, for example
35+
36+
```
37+
{ "sources": ["a.swift", "b.swift", "c.cpp"] }
38+
```
39+
40+
Tibs supports more advanced project configurations, such as multiple swift modules with dependencies, etc. For much more information about Tibs, including what features it supports and how it works, see [Tibs.md](Tibs.md).
41+
42+
### TibsTestWorkspace
43+
44+
The `TibsTestWorkspace` pulls together the various pieces needed for working with tests, including building the project, creating an IndexStoreDB instance, and keeping it up to date after modifying sources.
45+
46+
To create a `TibsTestWorkspace`, use the `staticTibsTestWorkspace` and `mutableTibsTestWorkspace` methods. Tests that do not mutate sources should use `staticTibsTestWorkspace`.
47+
48+
```swift
49+
func testFoo() {
50+
51+
// Loads an immutable workspace for the test project "MyTestProj" and resolves
52+
// all of its targets, ready to be indexed.
53+
guard let ws = try staticTibsTestWorkspace(name: "MyTestProj") else { return }
54+
55+
// Perform the build and read the produced index data.
56+
try ws.buildAndIndex()
57+
58+
// See the results
59+
ws.index.occurrences(ofUSR: "<some usr>", ...)
60+
61+
...
62+
}
63+
```
64+
65+
#### Source Locations
66+
67+
It is common to want to refer to specific locations in the source code of a test project. This is supported using inline comment syntax.
68+
69+
```swift
70+
Test.swift:
71+
func /*myFuncDef*/myFunc() {
72+
/*myFuncCall*/myFunc()
73+
}
74+
```
75+
76+
In a test, these locations can be referenced by name. The named location is immediately after the comment.
77+
78+
```swift
79+
80+
let loc = ws.testLoc("myFuncDef")
81+
// TestLocation(url: ..., line: 1, column: 19)
82+
```
83+
84+
A common use of a `TestLocation` is to form a `SymbolOccurrence` at that location.
85+
86+
```swift
87+
let occurrence = Symbol(...).at(ws.testLoc("myFuncDef"), roles: .definition)
88+
```
89+
90+
#### Comparing SymbolOccurrences
91+
92+
Rather than attempting to compare `SymbolOccurrences` for equality, prefer using the custom XCTest assertion method `checkOccurrences`. This API avoids accidentally comparing details that are undefined (e.g. the order of occurrences), or likely to change frequently (e.g. `SymbolRole`s are checked to be a superset instead of an exact match, as we often add new roles).
93+
94+
```
95+
checkOccurrences(ws.index.occurrences(ofUSR: sym.usr, roles: .all), expected: [
96+
sym.at(ws.testLoc("c"), roles: .definition),
97+
sym.at(ws.testLoc("c_call"), roles: .call),
98+
])
99+
```
100+
101+
#### Mutating Test Sources
102+
103+
In order to test changes to the index, we create a mutable test workspace. The mutable project takes care of copying the sources to a new location. Afterwards, we can mutate the sources using the `TibsTestWorkspace.edit(rebuild: Bool, block: ...)` method.
104+
105+
```
106+
func testMutating() {
107+
guard let ws = try mutableTibsTestWorkspace(name: "MyTestProj") else { return }
108+
try ws.buildAndIndex()
109+
110+
// Check initial state...
111+
112+
// Perform modifications to "url"!
113+
ws.edit(rebuild: true) { editor, files in
114+
115+
// Read the original contents and add a new line.
116+
let new = try files.get(url).appending("func /*moreCodeDef*/moreCode() {}")
117+
118+
// Provide the new contents for "url".
119+
editor.write(new, to: url)
120+
}
121+
}
122+
```
123+
124+
After we return from `edit()`, the sources are modified and any changes to stored source locations are reflected. We can now `buildAndIndex()` to update the index, or as a convenience we can pass `rebuild: true` to `edit`.

‎Documentation/Tibs.md

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# Tibs
2+
3+
Tibs ("Test Index Build System") is a simple and flexible build system designed for test projects of IndexStoreDB and SourceKit-LSP. Tibs can incrementally build or rebuild the index data and generated module files for a Swift and/or C-family language test project. Tibs can also dump the compiler arguments to a clang-compatible JSON compilation database. It is *not designed to compile to executable code*.
4+
5+
Tibs is implemented using [Ninja](https://ninja-build.org), which introduces a new dependency in IndexStoreDB when running tests.
6+
7+
Tibs projects are described by a `project.json` file containing one or more targets. Typically, a test case will use a project fixture located in the `Tests/INPUTS` directory.
8+
9+
## Project
10+
11+
A Tibs project is described by `project.json` manifest file. The top-level entity is a target, which may depend on other targets.
12+
13+
Example:
14+
15+
```
16+
{
17+
"targets": [
18+
{
19+
"name": "mytarget",
20+
"swift_flags": ["-warnings-as-errors"],
21+
"sources": ["a.swift", "b.swift"],
22+
"dependencies": ["dep1"],
23+
},
24+
{
25+
"name": "dep1",
26+
"sources": ["dep1.swift"],
27+
},
28+
]
29+
}
30+
```
31+
32+
As a convenience, if the project consists of a single target, it can be written at the top level and the name can be omitted, e.g.
33+
34+
```
35+
{ "sources": ["main.swift"] }
36+
```
37+
38+
### Targets
39+
40+
Targets have the following fields:
41+
42+
* "name": The name of the target. If omitted, it will be "main".
43+
* "sources": Array of source file paths.
44+
* "bridging_header": Optional path to a Swift bridging header.
45+
* "dependencies": Optional array of target names that this target depends on. This is useful for Swift module dependencies.
46+
* "swift_flags": Optional array of extra compiler arguments for Swift.
47+
* "clang_flags": Optional array of extra compiler arguments for Clang.
48+
49+
The directory containing the `project.json` manifest file is considered the project's root directory and any relative paths are relative to that.
50+
51+
## Building Tibs Projects
52+
53+
Most tests using Tibs will use the APIs provided by the `TibsTestWorkspace` class in the `ISDBTestSupport` module, which provides a high-level API for working with a complete Tibs project. The following information is for anyone needing to work using the lower-level APIs from `ISDBTibs` or the `tibs` command-line tool.
54+
55+
At a high level, both the library and command-line interfaces for Tibs work similarly: given a project and toolchain (containing executables clang, swiftc, ninja, etc.), produce a Ninja build description file and a compilation database. When built using `ninja`, all of the generated Swift modules are created and the raw index data is produced.
56+
57+
### ISDBTibs Library
58+
59+
The `TibsBuilder` class provides a library interface for building Tibs projects. It is responsible for fully resolving all the dependencies, build outputs, and compiler arguments. It also has APIs for executing the build using the `ninja` command-line tool and outputting a compilation database (`compile_commands.json`).
60+
61+
#### Example
62+
63+
The inputs to the build process are a toolchain, project, and build directory.
64+
65+
```
66+
import ISDBTibs
67+
68+
let toolchain = TibsToolchain(...)
69+
let projectDir: URL = ...
70+
let buildDir: URL = ...
71+
```
72+
73+
We load the `project.json` manifest from the project directory.
74+
75+
```
76+
let manifest = try TibsManifest.load(projectRoot: projectDir)
77+
```
78+
79+
From a manifest, we can create a `TibsBuilder` and examine the fully resolved targets.
80+
81+
```
82+
let builder = try TibsBuilder(
83+
manifest: manifest,
84+
sourceRoot: projectDir,
85+
buildRoot: buildDir,
86+
toolchain: toolchain)
87+
88+
for target in builder.targets {
89+
...
90+
}
91+
```
92+
93+
Finally, we can write the build files and build, or incrementally rebuild the project's index data and modules.
94+
95+
```
96+
try builder.writeBuildFiles()
97+
try builder.build()
98+
// edit sources
99+
try builder.build()
100+
```
101+
102+
### tibs Command-line Tool
103+
104+
As a convenience, we also provide a command-line interface `tibs`. It uses the `ISDBTibs` library to write out build files, which can then be built using `ninja`.
105+
106+
#### Example
107+
108+
Running `tibs` will write out the necessary Ninja build file and compilation database.
109+
110+
```
111+
mkdir build
112+
cd build
113+
tibs <path to project directory, containing project.json>
114+
```
115+
116+
To execute the build, use the `ninja` command
117+
118+
```
119+
ninja
120+
```
121+
122+
The build directory will be populated with the generated swift modules and index data.
123+
124+
## FAQ
125+
126+
### Why not use the Swift Package Manager?
127+
128+
The primary reason not to use the Swift Package Manager (SwiftPM) for our test projects is that SwiftPM's model is stricter than other build systems we need to support, and stricter than we want for our testing support. For example, we want to be able to test mixed language targets (using bridging headers), and to perform only the module-generation and indexing parts of the build without emitting object code. We need to be able to add features to our test support that would break the clean model that SwiftPM provides.

‎README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,7 @@ The C++ code in the index requires `libdispatch`, but unlike Swift code, it cann
2222
```sh
2323
$ swift build -Xcxx -I<path_to_swift_toolchain>/usr/lib/swift -Xcxx -I<path_to_swift_toolchain>/usr/lib/swift/Block
2424
```
25+
26+
## Development
27+
28+
For more information about developing IndexStoreDB, see [Development](Documentation/Development.md).

0 commit comments

Comments
 (0)