Skip to content

build: add circleci fallback caching #15625

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 21 additions & 14 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,70 +8,77 @@
# http://yaml-online-parser.appspot.com/

var_1: &docker_image angular/ngcontainer:0.10.0
var_2: &cache_key v2-ng-mat-{{ checksum "WORKSPACE" }}-{{ checksum "yarn.lock" }}-0.7.0

# **Note**: When updating the beginning of the cache key, also update the cache key to match
# the new cache key prefix. This allows us to take advantage of CircleCI's fallback caching.
# Read more here: https://circleci.com/docs/2.0/caching/#restoring-cache.
var_2: &cache_key v2-ng-mat-{{ checksum "WORKSPACE" }}-{{ checksum "yarn.lock" }}
var_3: &cache_fallback_key v2-ng-mat-

# Settings common to each job
var_3: &job_defaults
var_4: &job_defaults
working_directory: ~/ng
docker:
- image: *docker_image

# Job step for checking out the source code from GitHub. This also ensures that the source code
# is rebased on top of master.
var_4: &checkout_code
var_5: &checkout_code
checkout:
# After checkout, rebase on top of master. By default, PRs are not rebased on top of master,
# which we want. See https://discuss.circleci.com/t/1662
post: git pull --ff-only origin "refs/pull/${CI_PULL_REQUEST//*pull\//}/merge"

# Restores the cache that could be available for the current Yarn lock file. The cache usually
# includes the node modules and the Bazel repository cache.
var_5: &restore_cache
var_6: &restore_cache
restore_cache:
key: *cache_key
keys:
- *cache_key
- *cache_fallback_key

# Saves the cache for the current Yarn lock file. We store the node modules and the Bazel
# repository cache in order to make subsequent builds faster.
var_6: &save_cache
var_7: &save_cache
save_cache:
key: *cache_key
paths:
- "node_modules"
- "~/bazel_repository_cache"

# Decryption token that is used to decode the GCP credentials file in ".circleci/gcp_token".
var_7: &gcp_decrypt_token "angular"
var_8: &gcp_decrypt_token "angular"

# Job step that ensures that the node module dependencies are installed and up-to-date. We use
# Yarn with the frozen lockfile option in order to make sure that lock file and package.json are
# in sync. Unlike in Travis, we don't need to manually purge the node modules if stale because
# CircleCI automatically discards the cache if the checksum of the lock file has changed.
var_8: &yarn_install
var_9: &yarn_install
run: yarn install --frozen-lockfile --non-interactive

# Copies the Bazel config which is specifically for CircleCI to a location where Bazel picks it
# up and merges it with the project-wide bazel configuration (tools/bazel.rc)
var_9: &copy_bazel_config
var_10: &copy_bazel_config
# Set up the CircleCI specific bazel configuration.
run: sudo cp ./.circleci/bazel.rc /etc/bazel.bazelrc

# Sets up a different Docker image that includes a moe recent Firefox version which
# is needed for headless testing.
var_10: &docker-firefox-image
var_11: &docker-firefox-image
# TODO(devversion): Temporarily use a image that includes Firefox 62 because the
# ngcontainer image does include an old Firefox version that does not support headless.
- image: circleci/node:11.4.0-browsers

# Attaches the release output which has been stored in the workspace to the current job.
# https://circleci.com/docs/2.0/workflows/#using-workspaces-to-share-data-among-jobs
var_11: &attach_release_output
var_12: &attach_release_output
attach_workspace:
at: dist/

# Branch filter that we can specify for jobs that should only run on publish branches. This filter
# is used to ensure that not all upstream branches will be published as Github builds
# (e.g. revert branches, feature branches)
var_12: &publish_branches_filter
var_13: &publish_branches_filter
branches:
only:
- master
Expand All @@ -85,15 +92,15 @@ var_12: &publish_branches_filter
# In order to reduce duplication we use a YAML anchor that just always excludes the "_presubmit"
# branch. We don't want to run Circle for the temporary "_presubmit" branch which is reserved
# for the caretaker.
var_13: &ignore_presubmit_branch_filter
var_14: &ignore_presubmit_branch_filter
branches:
ignore:
- "_presubmit"
- "ivy-2019"

# Runs a script that sets up the Bazel remote execution. This will be used by jobs that run
# Bazel primarily and should benefit from remote caching and execution.
var_14: &setup_bazel_remote_execution
var_15: &setup_bazel_remote_execution
run:
name: "Setup bazel RBE remote execution"
command: ./scripts/circleci/bazel/setup-remote-execution.sh
Expand Down