Buildkite build PR job

[nkammah]

(moved over from #2804 since the previous one was started by a contractor no-longer at Elastic and I took over the work from there).

This PR introduces a Buildkite job to build pull requests. It replaces and combine the following Jenkins job:

• product repositories job Jenkins Job template
• elastic/docs Jenkins PR job

General approach

In the Jenkins world, we setup a dedicated job per repository with the job dsl coming from a template. We took a different approach in Buildkite, and instead setup a single Buildkite job that gets triggered for every PR builds on any of the docs product repo or the elastic/docs repo. This allows us to keep things tidy in Buildkite ( a single job) and limit considerably the amount of CI code to write in any product repository (see discussion on this approach here).

There are some downsides to this approach however:

• going to Buildkite build page looking for a specific build for a PR is not straight forward - it can be done by adding GET params to the builds url. For eg, to view the builds PR 3366 in elastic/observability-docs, we can visit the following URL: https://buildkite.com/elastic/docs-build-pr/builds?meta_data[repo_pr]=observability-docs_3366

• Since the same job is used for any PR in any of the product repositories, we allow concurrent builds of that job (otherwise, we could only run a single doc build at a time for any of the product repo). I have not seen/experienced/validated this yet, but we could run into scenarios where the same PR with 2 docs builds triggered in a short time frame try to push to the same preview branch and .

Docs repo configuration

We use the Buildkite PR bot in a non-conventional manner to achieve the above result:

• we use an "org-wide" configuration file stored in the master branch of elastic/docs - here (That file is not read in other branches).
• for all the products repositories, we leverage the "always_trigger_branch":true option. This is currently set to this branch while under development.
• for the elastic:docs repository, we use the default "always_trigger_branch":false option - since the build logic is stored within the elastic:docs repo - this is what would allow us to test out changes to the build logic itself.

Finally, the Buildkite PR bot allows for changes detections at certain path - this allows us to not trigger a PR build for every change in every docs repo, but only target doc changes.

With the above said, we have 3 configuration blobs in the PR bot config:

• product docs repo that should build for any change in the repo should be added here
• product docs repo that should build only when files within the ./docs folder should be added here
• config changes to elastic/docs is controlled here

The below config options have been configured for all of the above repositories:

• run docs-build
• run docs-build rebuild
• run docs-build warnlinkcheck
• run docs-build skiplinkcheck (takes precedence over the warnlinkcheck option)

PR job setup

When a docs PR job is opened / commented on, the Buildkite PR bot will trigger a docs-build-pr job, passing a few key env variables about the PR. The build script checks out that specific branch of the repository locally and references it in the build argument leveraging the --sub-dir option.

Similarly to Jenkins, PR on the elastic/docs repo always trigger a full rebuild.

PR are annotated with a link to the job preview and diff upon successful build - see example here.

Similar to other Buildkite job, docs preview changes are pushed to the bk branch until the migration is complete.

pre/post command hooks

We're leveraging theses the pre-command hook to setup the build commit status to pending and the post-commit one to success and failure. At this time, we're re-using the GITHUB token from the preview-cleaner job that has sufficient permissions to do such API calls.

All of the perms (to clone repo, commit) are tied to elasticmachine - I suspect this may have to change at some point in the future, but that's the current setup.

Jenkins/Buildkite comparison

Example of jobs that ran, and the differences between Jenkins and Buildkite, and the diff between the built-docs branches.

• [do not merge] testing out doc change for Buildkite observability-docs#3366 - diff
• https://github.com/elastic/tech-content/pull/382 - diff
• [do not merge] validating buildkite builds in docs repo #2833 - diff
• [do not merge] testing out doc change for Buildkite on forks observability-docs#3373 - diff

Note - on the docs changes in the docs repo, we're seeing images conflicts and changes - see convo about it here.

[gtback]

• Since the same job is used for any PR in any of the product repositories, we allow concurrent builds of that job (otherwise, we could only run a single doc build at a time for any of the product repo). I have not seen/experienced/validated this yet, but we could run into scenarios where the same PR with 2 docs builds triggered in a short time frame try to push to the same preview branch and .

This makes sense to me. I can see this causing problems if someone pushes to a PR, and then relatively quickly pushes a followup commit. At a minimum, it's worth documenting, but folks are probably used to Jenkins enough to know to just run a new build if they see anything strange. I imagine we could also check if any other builds are running on the same content repo and PR, and either abort the current one or cancel the other one, but this might require additional credentials to talk to Buildkite about other running builds.

[gtback]

I really like how readable the build_pr.sh script ended up being, and could be used for PRs in both the docs repo and content repos!

I handful of questions and suggestions, but nothing that should block the merge. Some of the suggestions could also be handled in follow-up PRs.

[gtback]

I haven't fully thought through how this works, but it doesn't seem like it's possible for --warnlinkcheck to get set unless --skiplinkcheck is also set, but I thought that --skiplinkcheck overrides --warnlinkcheck. Am I missing something?

[nkammah]

Theses flags will be passed by the Buildkite PR bot based on the comment left on the PR - as configured here:

"trigger_comment_regex": "run docs-build ?(?<rebuild_opt>rebuild)? ?(?<warn_opt>warnlinkcheck)? ?(?<skip_opt>skiplinkcheck)?",

Nothing prevents a GH user from passing all 3 flags - does it make sense?

[gtback]

What flags would you pass to get --warnlinkcheck to run? If you pass GITHUB_PR_COMMENT_VAR_WARN_OPT=warnlink, then build_args won't be modified unless you also pass GITHUB_PR_COMMENT_VAR_SKIP_OPT=skiplinkcheck.

But you can't pass both, or the docs build should raise an error.

[gtback]

I'm still confused about how to pass --warnlinkcheck, and have one other small suggestion, but I'll approve this so you can merge, in case I'm misunderstanding how the logic works.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 68954f7
• Documentation preview
  • 📚 HTML diff
  • 📙 Preview

History

• 💚 Build #78 succeeded 576fcc6
• 💚 Build #77 succeeded 03a384e
• 💚 Build #75 succeeded 3515da7
• 💚 Build #60 succeeded a4b2ff3
• 💚 Build #52 succeeded 460125a
• 💔 Build #50 failed 13fe044