Skip to content

Release #1968

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Apr 25, 2022
Merged

Release #1968

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/workflows/semantic-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: "Semantic Check"

on:
pull_request_target:
types:
- opened
- edited
- synchronize

permissions:
contents: read
pull-requests: read

jobs:
main:
name: Semantic Commit Message Check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- uses: amannn/action-semantic-pull-request@v4
name: Check PR for Semantic Commit Message
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
requireScope: false
validateSingleCommit: true
ignoreLabels: release merge
21 changes: 11 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ For receiving the `check_run` or `workflow_job` event by the webhook (lambda), a

- `workflow_job`: **(preferred option)** create a webhook on enterprise, org or app level. Select this option for ephemeral runners.
- `check_run`: create a webhook on enterprise, org, repo or app level. When using the app option, the app needs to be installed to repo's are using the self-hosted runners.
- a Webhook needs to be created. The webhook hook can be defined on enterprise, org, repo, or app level.
- a Webhook needs to be created. The webhook hook can be defined on enterprise, org, repo, or app level.

In AWS a [API gateway](https://docs.aws.amazon.com/apigateway/index.html) endpoint is created that is able to receive the GitHub webhook events via HTTP post. The gateway triggers the webhook lambda which will verify the signature of the event. This check guarantees the event is sent by the GitHub App. The lambda only handles `workflow_job` or `check_run` events with status `queued` and matching the runner labels (only for `workflow_job`). The accepted events are posted on a SQS queue. Messages on this queue will be delayed for a configurable amount of seconds (default 30 seconds) to give the available runners time to pick up this build.

Expand Down Expand Up @@ -197,17 +197,18 @@ The lambda for syncing the GitHub distribution to S3 is triggered via CloudWatch

### Setup the webhook / GitHub App (part 2)

At this point you have 2 options. Either create a separate webhook (enterprise,
org, or repo), or create webhook in the App.
At this point you have 2 options. Either create a separate webhook (enterprise,
org, or repo), or create webhook in the App.

#### Option 1: Webhook

1. Create a new webhook on repo level for repo level for repo level runner, or org (or enterprise level) for an org level runner.
2. Provide the webhook url, should be part of the output of terraform.
3. Provide the webhook secret (`terraform output -raw <NAME_OUTPUT_VAR>`).
4. In the "Permissions & Events" section and then "Subscribe to Events" subsection, check either "Workflow Job" or "Check Run" (choose only 1 option!!!).
5. In the "Install App" section, install the App in your organization, either in all or in selected repositories.

4. Ensure content type as `application/json`.
5. In the "Permissions & Events" section and then "Subscribe to Events" subsection, check either "Workflow Job" or "Check Run" (choose only 1 option!!!).
6. In the "Install App" section, install the App in your organization, either in all or in selected repositories.

#### Option 2: App

Go back to the GitHub App and update the following settings.
Expand Down Expand Up @@ -252,7 +253,7 @@ module "runners" {

### Pool

The module basically supports two options for keeping a pool of runners. One is via a pool which only supports org-level runners, the second option is [keeping runners idle](#idle-runners).
The module basically supports two options for keeping a pool of runners. One is via a pool which only supports org-level runners, the second option is [keeping runners idle](#idle-runners).

The pool is introduced in combination with the ephemeral runners and is primary meant to ensure if any event is unexpected dropped, and no runner was created the pool can pick up the job. The pool is maintained by a lambda. Each time the lambda is triggered a check is preformed if the number of idler runners managed by the module are meeting the expected pool size. If not, the pool will be adjusted. Keep in mind that the scale down function is still active and will terminate instances that are detected as idle.

Expand Down Expand Up @@ -304,10 +305,10 @@ For time zones please check [TZ database name column](https://en.wikipedia.org/w
Currently a beta feature! You can configure runners to be ephemeral, runners will be used only for one job. The feature should be used in conjunction with listening for the workflow job event. Please consider the following:

- The scale down lambda is still active, and should only remove orphan instances. But there is no strict check in place. So ensure you configure the `minimum_running_time_in_minutes` to a value that is high enough to got your runner booted and connected to avoid it got terminated before executing a job.
- The messages sent from the webhook lambda to scale-up lambda are by default delayed delayed by SQS, to give available runners to option to start the job before the decision is made to scale more runners. For ephemeral runners there is no need to wait. Set `delay_webhook_event` to `0`.
- The messages sent from the webhook lambda to scale-up lambda are by default delayed delayed by SQS, to give available runners to option to start the job before the decision is made to scale more runners. For ephemeral runners there is no need to wait. Set `delay_webhook_event` to `0`.
- All events on the queue will lead to a new runner crated by the lambda. By setting `enable_job_queued_check` to `true` you can enforce only create a runner if the event has a correlated queued job. Setting this can avoid creating useless runners, for example whn jobs got cancelled before a runner is created. We suggest to use this in combination with a pool.
- To ensure runners are created in the same order GitHub sends the events we use by default a FIFO queue, this is mainly relevant for repo level runners. For ephemeral runners you can set `fifo_build_queue` to `false`.
- Error related to scaling should be retried via SQS. You can configure `job_queue_retention_in_seconds` `redrive_build_queue` to tune the behavior. We have no mechanism to avoid events will never processed, which means potential no runner could be created and the job in GitHub can time out in 6 hours.
- Error related to scaling should be retried via SQS. You can configure `job_queue_retention_in_seconds` `redrive_build_queue` to tune the behavior. We have no mechanism to avoid events will never processed, which means potential no runner could be created and the job in GitHub can time out in 6 hours.

The example for [ephemeral runners](./examples/ephemeral) is based on the [default example](./examples/default). Have look on the diff to see the major configuration differences.

Expand Down Expand Up @@ -345,7 +346,7 @@ The following sub modules are optional and are provided as example or utility:

### ARM64 configuration for submodules

When using the top level module configure `runner_architecture = "arm64"` and ensure the list of `instance_types` matches. When not using the top-level, ensure these properties are set on the submodules.
When using the top level module configure `runner_architecture = "arm64"` and ensure the list of `instance_types` matches. When not using the top-level, ensure these properties are set on the submodules.

## Debugging

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@
"@octokit/rest": "^18.12.0",
"@trivago/prettier-plugin-sort-imports": "^3.2.0",
"@types/jest": "^27.4.1",
"@types/node": "^17.0.23",
"@types/node": "^17.0.25",
"@types/request": "^2.48.8",
"@typescript-eslint/eslint-plugin": "^4.33.0",
"@typescript-eslint/parser": "^4.33.0",
"@vercel/ncc": "^0.33.3",
"aws-sdk": "^2.1112.0",
"@vercel/ncc": "^0.33.4",
"aws-sdk": "^2.1116.0",
"eslint": "^7.32.0",
"eslint-plugin-prettier": "4.0.0",
"jest": "^27.5.1",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -828,10 +828,10 @@
resolved "https://registry.yarnpkg.com/@types/json-schema/-/json-schema-7.0.9.tgz#97edc9037ea0c38585320b28964dde3b39e4660d"
integrity sha512-qcUXuemtEu+E5wZSJHNxUXeCZhAfXKQ41D+duX+VYPde7xyEVZci+/oXKJL13tnRs9lR2pr4fod59GT6/X1/yQ==

"@types/node@*", "@types/node@^17.0.23":
version "17.0.23"
resolved "https://registry.yarnpkg.com/@types/node/-/node-17.0.23.tgz#3b41a6e643589ac6442bdbd7a4a3ded62f33f7da"
integrity sha512-UxDxWn7dl97rKVeVS61vErvw086aCYhDLyvRQZ5Rk65rZKepaFdm53GeqXaKBuOhED4e9uWq34IC3TdSdJJ2Gw==
"@types/node@*", "@types/node@^17.0.25":
version "17.0.25"
resolved "https://registry.yarnpkg.com/@types/node/-/node-17.0.25.tgz#527051f3c2f77aa52e5dc74e45a3da5fb2301448"
integrity sha512-wANk6fBrUwdpY4isjWrKTufkrXdu1D2YHCot2fD/DfWxF5sMrVSA+KN7ydckvaTCh0HiqX9IVl0L5/ZoXg5M7w==

"@types/prettier@^2.1.5":
version "2.4.2"
Expand Down Expand Up @@ -950,10 +950,10 @@
"@typescript-eslint/types" "4.33.0"
eslint-visitor-keys "^2.0.0"

"@vercel/ncc@^0.33.3":
version "0.33.3"
resolved "https://registry.yarnpkg.com/@vercel/ncc/-/ncc-0.33.3.tgz#aacc6b3ea9f7b175e0c9a18c9b97e4005a2f4fcc"
integrity sha512-JGZ11QV+/ZcfudW2Cz2JVp54/pJNXbsuWRgSh2ZmmZdQBKXqBtIGrwI1Wyx8nlbzAiEFe7FHi4K1zX4//jxTnQ==
"@vercel/ncc@^0.33.4":
version "0.33.4"
resolved "https://registry.yarnpkg.com/@vercel/ncc/-/ncc-0.33.4.tgz#e44a87511f583b7ba88e4b9ae90eeb7ba252b872"
integrity sha512-ln18hs7dMffelP47tpkaR+V5Tj6coykNyxJrlcmCormPqRQjB/Gv4cu2FfBG+PMzIfdZp2CLDsrrB1NPU22Qhg==

abab@^2.0.3, abab@^2.0.5:
version "2.0.5"
Expand Down Expand Up @@ -1086,10 +1086,10 @@ asynckit@^0.4.0:
resolved "https://registry.yarnpkg.com/asynckit/-/asynckit-0.4.0.tgz#c79ed97f7f34cb8f2ba1bc9790bcc366474b4b79"
integrity sha1-x57Zf380y48robyXkLzDZkdLS3k=

aws-sdk@^2.1112.0:
version "2.1112.0"
resolved "https://registry.yarnpkg.com/aws-sdk/-/aws-sdk-2.1112.0.tgz#7e6f60df562ebd909a9435b01673ed7685e10ffd"
integrity sha512-8FKr3zwD2bHZQLmxJ7d1VwvDkW9kKx4YPdNXawV+CqNUzQC/9ZmSmHyx8mZCv6J722tqxIZh5c567499QawuZw==
aws-sdk@^2.1116.0:
version "2.1116.0"
resolved "https://registry.yarnpkg.com/aws-sdk/-/aws-sdk-2.1116.0.tgz#1187ab943e6bf730db282afe7950dd2af409cb5b"
integrity sha512-36JFrxPPh/fRQWsgGrZZbzTxRu7dq4KyCKKXPxgVMXylEJsG/KEAVMB1f3eq4PiI5eGxYrpt2OkKoMQZQZLjPA==
dependencies:
buffer "4.9.2"
events "1.1.1"
Expand Down
6 changes: 3 additions & 3 deletions modules/runners/lambdas/runners/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,14 @@
"ts-node-dev": "^1.1.6"
},
"dependencies": {
"@aws-sdk/client-ssm": "^3.67.0",
"@aws-sdk/client-ssm": "^3.74.0",
"@octokit/auth-app": "3.6.1",
"@octokit/rest": "^18.12.0",
"@octokit/types": "^6.34.0",
"@types/aws-lambda": "^8.10.93",
"@types/express": "^4.17.11",
"@types/node": "^17.0.23",
"aws-sdk": "^2.1112.0",
"@types/node": "^17.0.26",
"aws-sdk": "^2.1116.0",
"cron-parser": "^4.3.0",
"tslog": "^3.3.3",
"typescript": "^4.6.3"
Expand Down
Loading