|
| 1 | +<!--- |
| 2 | +This file was generated on 2015-07-27T23:49:06-07:00 from the rspec-dev repo. |
| 3 | +DO NOT modify it by hand as your changes will get lost the next time it is generated. |
| 4 | +--> |
| 5 | + |
| 6 | +# The CI build, in detail |
| 7 | + |
| 8 | +The [Travis CI build](https://travis-ci.org/rspec/rspec-rails) |
| 9 | +runs many verification steps to prevent regressions and |
| 10 | +ensure high-quality code. To run the Travis build locally, run: |
| 11 | + |
| 12 | +``` |
| 13 | +$ script/run_build |
| 14 | +``` |
| 15 | + |
| 16 | +It can be useful to run the build steps individually |
| 17 | +to repro a failing part of a Travis build. Let's break |
| 18 | +the build down into the individual steps. |
| 19 | + |
| 20 | +## Specs |
| 21 | + |
| 22 | +RSpec dogfoods itself. Its primary defense against regressions is its spec suite. Run with: |
| 23 | + |
| 24 | +``` |
| 25 | +$ bundle exec rspec |
| 26 | +
|
| 27 | +# or, if you installed your bundle with `--standalone --binstubs`: |
| 28 | +
|
| 29 | +$ bin/rspec |
| 30 | +``` |
| 31 | + |
| 32 | +The spec suite performs a couple extra checks that are worth noting: |
| 33 | + |
| 34 | +* *That all the code is warning-free.* Any individual example that produces output |
| 35 | + to `stderr` will fail. We also have a spec that loads all the `lib` and `spec` |
| 36 | + files in a newly spawned process to detect load-time warnings and fail if there |
| 37 | + are any. RSpec must be warning-free so that users who enable Ruby warnings will |
| 38 | + not get warnings from our code. |
| 39 | +* *That only a minimal set of stdlibs are loaded.* Since Ruby makes loaded libraries |
| 40 | + available for use in any context, we want to minimize how many bits of the standard |
| 41 | + library we load and use. Otherwise, RSpec's use of part of the standard library could |
| 42 | + mask a problem where a gem author forgets to load a part of the standard library they |
| 43 | + rely on. The spec suite contains a spec that defines a whitelist of allowed loaded |
| 44 | + stdlibs. |
| 45 | + |
| 46 | +In addition, we use [SimpleCov](https://github.com/colszowka/simplecov) |
| 47 | +to measure and enforce test coverage. If the coverage falls below a |
| 48 | +project-specific threshold, the build will fail. |
| 49 | + |
| 50 | +## Cukes |
| 51 | + |
| 52 | +RSpec uses [cucumber](https://cucumber.io/) for both acceptance testing |
| 53 | +and [documentation](https://relishapp.com/rspec). Since we publish our cukes |
| 54 | +as documentation, please limit new cucumber scenarios to user-facing examples |
| 55 | +that help demonstrate usage. Any tests that exist purely to prevent regressions |
| 56 | +should be written as specs, even if they are written in an acceptance style. |
| 57 | +Duplication between our YARD API docs and the cucumber documentation is fine. |
| 58 | + |
| 59 | +Run with: |
| 60 | + |
| 61 | +``` |
| 62 | +$ bundle exec cucumber |
| 63 | +
|
| 64 | +# or, if you installed your bundle with `--standalone --binstubs`: |
| 65 | +
|
| 66 | +$ bin/cucumber |
| 67 | +``` |
| 68 | + |
| 69 | +## YARD documentation |
| 70 | + |
| 71 | +RSpec uses [YARD](http://yardoc.org/) for API documentation on the [rspec.info site](http://rspec.info/). |
| 72 | +Our commitment to [SemVer](htp://semver.org) requires that we explicitly |
| 73 | +declare our public API, and our build uses YARD to ensure that every |
| 74 | +class, module and method has either been labeled `@private` or has at |
| 75 | +least some level of documentation. For new APIs, this forces us to make |
| 76 | +an intentional decision about whether or not it should be part of |
| 77 | +RSpec's public API or not. |
| 78 | + |
| 79 | +To run the YARD documentation coverage check, run: |
| 80 | + |
| 81 | +``` |
| 82 | +$ bundle exec yard stats --list-undoc |
| 83 | +
|
| 84 | +# or, if you installed your bundle with `--standalone --binstubs`: |
| 85 | +
|
| 86 | +$ bin/yard stats --list-undoc |
| 87 | +``` |
| 88 | + |
| 89 | +We also want to prevent YARD errors or warnings when actually generating |
| 90 | +the docs. To check for those, run: |
| 91 | + |
| 92 | +``` |
| 93 | +$ bundle exec yard doc --no-cache |
| 94 | +
|
| 95 | +# or, if you installed your bundle with `--standalone --binstubs`: |
| 96 | +
|
| 97 | +$ bin/yard doc --no-cache |
| 98 | +``` |
| 99 | + |
| 100 | +## Rubocop |
| 101 | + |
| 102 | +We use [Rubocop](https://github.com/bbatsov/rubocop) to enforce style conventions on the project so |
| 103 | +that the code has stylistic consistency throughout. Run with: |
| 104 | + |
| 105 | +``` |
| 106 | +$ bundle exec rubocop lib |
| 107 | +
|
| 108 | +# or, if you installed your bundle with `--standalone --binstubs`: |
| 109 | +
|
| 110 | +$ bin/rubocop lib |
| 111 | +``` |
| 112 | + |
| 113 | +Our Rubocop configuration is a work-in-progress, so if you get a failure |
| 114 | +due to a Rubocop default, feel free to ask about changing the |
| 115 | +configuration. Otherwise, you'll need to address the Rubocop failure, |
| 116 | +or, as a measure of last resort, by wrapping the offending code in |
| 117 | +comments like `# rubocop:disable SomeCheck` and `# rubocop:enable SomeCheck`. |
| 118 | + |
| 119 | +## Run spec files one-by-one |
| 120 | + |
| 121 | +A fast TDD cycle depends upon being able to run a single spec file, |
| 122 | +without the rest of the test suite. While rare, it's fairly easy to |
| 123 | +create a situation where a spec passes when the entire suite runs |
| 124 | +but fails when its individual file is run. To guard against this, |
| 125 | +our CI build runs each spec file individually, using a bit of bash like: |
| 126 | + |
| 127 | +``` |
| 128 | +for file in `find spec -iname '*_spec.rb'`; do |
| 129 | + echo "Running $file" |
| 130 | + bin/rspec $file -b --format progress |
| 131 | +done |
| 132 | +``` |
| 133 | + |
| 134 | +Since this step boots RSpec so many times, it runs much, much |
| 135 | +faster when we can avoid the overhead of bundler. This is a main reason our |
| 136 | +CI build installs the bundle with `--standalone --binstubs` and |
| 137 | +runs RSpec via `bin/rspec` rather than `bundle exec rspec`. |
| 138 | + |
| 139 | +## Running the spec suite for each of the other repos |
| 140 | + |
| 141 | +While each of the RSpec repos is an independent gem (generally designed |
| 142 | +to be usable on its own), there are interdependencies between the gems, |
| 143 | +and the specs for each tend to use features from the other gems. We |
| 144 | +don't want to merge a pull request for one repo that might break the |
| 145 | +build for another repo, so our CI build includes a spec that runs the |
| 146 | +spec suite of each of the _other_ project repos. Note that we only run |
| 147 | +the spec suite, not the full build, of the other projects, as the spec |
| 148 | +suite runs very quickly compared to the full build. |
| 149 | + |
0 commit comments