Skip to content

Transit CI to GitHub Actions #1942

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Dec 21, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
216 changes: 216 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
name: CI

on:
push:
branches:
- auto
- try
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On travis we need to include the master branch, because it's (well technically the repo HEAD) cache is used as the fallback for new branches and PRs. That doesn't seem to apply to the caching logic on GH Actions, so I think its okay to drop it here.


pull_request:

jobs:
frontend:
name: Frontend
runs-on: ubuntu-16.04
env:
JOBS: 1 # See https://git.io/vdao3 for details.
PERCY_PARALLEL_TOTAL: 2
# Percy secrets are included here to enable Percy's GitHub integration
# on community-submitted PRs
PERCY_TOKEN: 0d8707a02b19aebbec79bb0bf302b8d2fa95edb33169cfe41b084289596670b1
PERCY_PROJECT: crates-io/crates.io

steps:
- uses: actions/checkout@v2-beta

- uses: actions/cache@v1
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-

- uses: actions/setup-node@v1
with:
node-version: '12.x'

- name: Install node modules
run: npm ci

- name: Lint
run: |
npm run lint:hbs
npm run lint:js
npm run lint:deps

- name: Add nonce for parallel tests
run: echo ::set-env name=PERCY_PARALLEL_NONCE::`date +%s`

- name: Test
run: npm test

backend:
name: Backend
runs-on: ubuntu-16.04
strategy:
# TODO: [ci] should be true if GitHub Actions supports ""allow failures" on matrix
fail-fast: false
matrix:
rust:
- stable
- beta
- nightly

env:
DATABASE_URL: postgres://postgres:@localhost/cargo_registry_test
TEST_DATABASE_URL: postgres://postgres:@localhost/cargo_registry_test
CARGO_INCREMENTAL: 0
RUSTFLAGS: "-C debuginfo=0 -D warnings"

steps:
- uses: actions/checkout@v2-beta

- name: Cleanup pre-installed Rust toolchains
# The pre-installed toolchain is under root permission, and this would
# make tarball fail to extract. Here we remove the symlink and install
# our own Rust toolchain if necessary.
run: |
which rustup
which cargo
if [[ -L "$HOME/.cargo" && -L "$HOME/.rustup" ]]; then
rm -v "$HOME/.rustup"
rm -v "$HOME/.cargo"
fi
echo "::add-path::$HOME/.cargo/bin"

- name: Cache rustup
uses: actions/cache@v1
with:
path: ~/.rustup
key: ${{ runner.os }}-rustup-${{ matrix.rust }}-${{ hashFiles('**/Cargo.lock') }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it is necessary to include the -${{ hashFiles('**/Cargo.lock') }} portion for this directory and for the ~/.cargo/ paths below. (It definitely makes sense for the target/ directory.) My reasoning is that the contents of these directories shouldn't really be affected by the contents of the lock file.

By the same reasoning, -${{ matrix.rust }} may also not be necessary for many of these.

If you're unsure about removing it though, I definitely don't see this as a blocker. This is probably a micro-optimization given the expected size of these directories.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. We need figure out when to invalidate old cache. Maybe some trick like script//ci/cargo-clean-on-new-rustc-version.sh, but I am not sure how to implement that.

restore-key: |
${{ runner.os }}-rustup-${{ matrix.rust }}-
${{ runner.os }}-rustup-

- name: Cache cargo binaries
uses: actions/cache@v1
with:
path: ~/.cargo/bin
key: ${{ runner.os }}-cargo-bin-${{ matrix.rust }}-${{ hashFiles('**/Cargo.lock') }}
restore-key: |
${{ runner.os }}-cargo-bin-${{ matrix.rust }}-
${{ runner.os }}-cargo-bin-

- name: Cache cargo registry cache
uses: actions/cache@v1
with:
path: ~/.cargo/registry/cache
key: ${{ runner.os }}-cargo-registry-cache-${{ matrix.rust }}-${{ hashFiles('**/Cargo.lock') }}
restore-key: |
${{ runner.os }}-cargo-registry-cache-${{ matrix.rust }}-
${{ runner.os }}-cargo-registry-cache-

- name: Cache cargo registry index
uses: actions/cache@v1
with:
path: ~/.cargo/registry/index
key: ${{ runner.os }}-cargo-registry-index-${{ matrix.rust }}-${{ hashFiles('**/Cargo.lock') }}
restore-key: |
${{ runner.os }}-cargo-registry-index-${{ matrix.rust }}-
${{ runner.os }}-cargo-registry-index-

- name: Cache cargo git db
uses: actions/cache@v1
with:
path: ~/.cargo/git/db
key: ${{ runner.os }}-cargo-git-db-${{ matrix.rust }}-${{ hashFiles('**/Cargo.lock') }}
restore-key: |
${{ runner.os }}-cargo-git-db-${{ matrix.rust }}-
${{ runner.os }}-cargo-git-db-

- name: Cache cargo build
uses: actions/cache@v1
with:
path: target
key: ${{ runner.os }}-cargo-build-target-${{ matrix.rust }}-${{ hashFiles('**/Cargo.lock') }}
restore-key: |
${{ runner.os }}-cargo-build-target-${{ matrix.rust }}-
${{ runner.os }}-cargo-build-target-

- name: Install ${{ matrix.rust }} Rust
run: |
if [[ ! -d "$HOME/.cargo" || ! -d "$HOME/.rustup" ]]; then
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs > rustup-init.sh
sh rustup-init.sh -y --default-toolchain none
fi
rustup set profile minimal
rustup update ${{ matrix.rust }}
rustup default ${{ matrix.rust }}

- name: Install lint tools
if: matrix.rust == 'stable'
run: |
rustup component add rustfmt
rustup component add clippy

- name: Cargo clean on new rustc version
run: script/ci/cargo-clean-on-new-rustc-version.sh

- name: Install required system libraries
run: |
sudo tee /etc/apt/sources.list.d/pgdg.list <<END
deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main
END
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install -y libpq-dev postgresql-11
mkdir -p /etc/postgresql/11/main
sudo tee /etc/postgresql/11/main/pg_hba.conf <<END
local all all trust
host all all 127.0.0.1/32 trust
END
sudo systemctl restart postgresql@11-main

- name: Setup database
run: |
which diesel || cargo install diesel_cli --vers $(cat .diesel_version) --no-default-features --features postgres
diesel database setup --locked-schema

- name: Lint
if: matrix.rust == 'stable'
run: |
cargo fmt -- --check
cargo clippy --all-targets --all-features --all

- name: Test
run: cargo test

- name: Prune unnecessary cache
run: script/ci/prune-cache.sh


# These jobs doesn't actually test anything, but they're only used to tell
# bors the build completed, as there is no practical way to detect when a
# workflow is successful listening to webhooks only.
#
# ALL THE PREVIOUS JOBS NEEDS TO BE ADDED TO THE `needs` SECTION OF THIS JOB!

end-success:
name: bors build finished
if: success()
runs-on: ubuntu-latest
needs: [frontend, backend]

steps:
- name: Mark the job as successful
run: exit 0

end-failure:
name: bors build finished
if: "!success()"
runs-on: ubuntu-latest
needs: [frontend, backend]

steps:
- name: Mark the job as a failure
run: exit 1
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# crates.io

[![Build Status](https://travis-ci.com/rust-lang/crates.io.svg?branch=master)](https://travis-ci.com/rust-lang/crates.io)
[![Build Status](https://github.com/rust-lang/crates.io/workflows/CI/badge.svg)](https://github.com/rust-lang/crates.io/actions?query=workflow%3ACI)
[![What's Deployed](https://img.shields.io/badge/whatsdeployed-prod-green.svg)](https://whatsdeployed.io/s-9IG)

Source code for the default [Cargo](http://doc.crates.io) registry. Viewable
Expand Down
4 changes: 2 additions & 2 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ local development environment
writable directory - (ignored in `.gitignore`)
* `tmp/index-bare` - A bare git repository, used as the origin for `tmp/index-co` during
development - (ignored in `.gitignore`)
* `.travis.yml` - Configuration for continous integration at [Travis CI][]
* `.github/workflows/*` - Configuration for continuous integration at [GitHub Actions]
* `.watchmanconfig` - Use by Ember CLI to efficiently watch for file changes if you install watchman

[Travis CI]: https://travis-ci.com/rust-lang/crates.io
[GitHub Actions]: https://github.com/rust-lang/crates.io/actions
13 changes: 7 additions & 6 deletions docs/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,22 +26,23 @@ like to see all new types and functions, public and private, to have documentati
comments on them. If you change an existing type or function, and it doesn't have
a documentation comment on it, it'd be great if you could add one to it too.

When you submit a pull request, it will be automatically tested on TravisCI. In
When you submit a pull request, it will be automatically tested on GitHub Actions. In
addition to running both the frontend and the backend tests described below,
Travis runs [jslint], [clippy], and [rustfmt] on each PR.
GitHub Actions runs [jslint], [clippy], and [rustfmt] on each PR.

If you don't want to run these tools locally, please watch the Travis results
If you don't want to run these tools locally, please watch the GitHub Actions results
and submit additional commits to your pull request to fix any issues they find!

If you do want to run these tools locally in order to fix issues before
submitting, that would be great too! Please consult each tool's installation
instructions and the .travis.yml file in this repository for the latest
installation and running instructions. The logs for recent builds in Travis
instructions and the [.github/workflows/ci.yml] file in this repository for the latest
installation and running instructions. The logs for recent builds in GitHub Actions
may also be helpful to see which versions of these tools we're currently using.

[jslint]: http://jslint.com/
[clippy]: https://github.com/rust-lang-nursery/rust-clippy
[rustfmt]: https://github.com/rust-lang-nursery/rustfmt
[.github/workflows/ci.yml]: /.github/workflows/ci.yml

We will try to review your pull requests as soon as possible!

Expand Down Expand Up @@ -86,7 +87,7 @@ as well.

In order to run the frontend on Windows and macOS, you will need to have installed:

- [node](https://nodejs.org/en/) >= 12.9.1 (see `package.json` and `.travis.yml` for what we currently use)
- [node](https://nodejs.org/en/) >= 12.9.1 (see `package.json` and `.github/workflows/ci.yml` for what we currently use)
- [npm](https://www.npmjs.com/get-npm) >= 6.10.2

Follow the links for each of these tools for their recommended installation
Expand Down
5 changes: 5 additions & 0 deletions script/ci/prune-cache.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

set -e

if [[ ! -d "target/debug" ]]; then
echo "No build artifacts found. Exit immediately."
exit 0
fi

echo "Initial cache size:"
du -hs target/debug

Expand Down