|
| 1 | +# Overview |
| 2 | + |
| 3 | +PlanetScale has a fork of MySQL so that we can make changes that would not be acceptable or useful to upstream MySQL, or that we don't want to wait for upstream MySQL to accept. That includes changes related to billing and Insights. |
| 4 | + |
| 5 | +This document describes how we develop, maintain, and deploy our fork of MySQL. |
| 6 | + |
| 7 | +# Open source |
| 8 | + |
| 9 | +Upstream MySQL is distributed under the GPLv2, which means our fork of MySQL also must be. We deploy MySQL images to managed customers' clusters, which counts as a binary distribution, so we must make our modified source available following one of the methods in the GPL. |
| 10 | + |
| 11 | +We use two GitHub repositories to do that: |
| 12 | + |
| 13 | +[`planetscale/mysql-server-private`](https://github.com/planetscale/mysql-server-private) is an independent (non-fork), private repository, containing a full copy of `mysql/mysql-server`. This is where issues and pull requests happen. This is also where PlanetScale-specific CI, docs, and operations code go. |
| 14 | + |
| 15 | +Within the private fork of MySQL, we have two ongoing branches: |
| 16 | + 1. `8.0` is the main branch, which is currently based on the upstream `mysql-8.0.28` tag. Our original development history is here. |
| 17 | + 2. `8.0.23` is based on upstream's `mysql-8.0.23` tag, with our changes squashed onto it. Some further development has happened there, and all useful commits from `8.0.23` have been cherry-picked onto `8.0`. Since deployed images are based on MySQL version 8.0.23, the `8.0.23` branch is where active development currently happens. |
| 18 | + |
| 19 | +We do not need to keep both branches indefinitely. Once we begin deploying images based on `8.0.28` or newer, we can remove, or just ignore, the `8.0.23` branch. |
| 20 | + |
| 21 | +[`planetscale/mysql-server`](https://github.com/planetscale/mysql-server) is a public member of the `mysql/mysql-server` network. This is where GPL releases go. The commits we push to this public fork are squashed and omit meaningful history. This is a "code dumps" approach to open source. Nothing at PlanetScale is built or deployed from this repo. |
| 22 | + |
| 23 | +The public repo has only one branch, `8.0`, which is a fork of the upstream `8.0` with our changes applied as squash merges. Exclude PlanetScale-specific content when creating squash commits for the public repo. For example: |
| 24 | + |
| 25 | +```sh |
| 26 | +git co -- README |
| 27 | +rm -r .buildkite/ .dockerignore Dockerfile.* _planetscale/ build.sh docker-compose.yml |
| 28 | +``` |
| 29 | + |
| 30 | +The public repo has one tag per image that we have deployed broadly, with the format `8.0.23-ps1`. In general, create a commit only when rolling out a new image, which means there will be one tag per commit. The semver in each tag is the upstream version the build is based on. The final number (`1` in `ps1`) in each tag is sequential and has no semantic (i.e., compatibility) meaning. |
| 31 | + |
| 32 | +# Developing |
| 33 | + |
| 34 | +Clone the private repository, which will be `origin`: |
| 35 | + |
| 36 | +```sh |
| 37 | +git clone [email protected]:planetscale/mysql-server-private |
| 38 | +``` |
| 39 | + |
| 40 | +Add remotes for the public and upstream repositories: |
| 41 | + |
| 42 | +```sh |
| 43 | +git remote add public [email protected]:planetscale/mysql-server |
| 44 | +git remote add upstream [email protected]:mysql/mysql-server |
| 45 | +``` |
| 46 | + |
| 47 | +Create your development branches and PRs off of `8.0.23`, for now. |
| 48 | + |
| 49 | +Create a local branch for public releases: |
| 50 | + |
| 51 | +```sh |
| 52 | +git fetch public 8.0:public-8.0 |
| 53 | +``` |
| 54 | + |
| 55 | +Every PR gets built in CI and pushed to the ECR registry at `997601596833.dkr.ecr.us-east-1.amazonaws.com/branch/`. These are candidate builds, and you can deploy them to individual database branches using the [admin](https://admin.planetscale.com/admin/) page. |
| 56 | + |
| 57 | +Every commit to the `8.0` (default) or `8.0.23` branch gets built in CI and pushed to ECR at `997601596833.dkr.ecr.us-east-1.amazonaws.com/main/`. These are release builds. You can deploy them to individual branches using the [admin](https://admin.planetscale.com/admin/) page, or to api-bb `MYSQLD_VERSIONS` update channels in [`database_branch.rb`](https://github.com/planetscale/api-bb/blob/main/app/models/database_branch.rb#L174). |
| 58 | + |
| 59 | +For more about container management, see [`registry.md`](registry.md). |
| 60 | + |
| 61 | +# Deployment |
| 62 | + |
| 63 | +Anytime there are changes worth deploying, do an open-source code dump, and update all the release channels. |
| 64 | + |
| 65 | +For the open-source code dump, cherry-pick or squash-merge commits (minus PlanetScale-specific content) onto your `public-8.0` branch, and then push it and tag it: |
| 66 | + |
| 67 | +```sh |
| 68 | +git push public public-8.0:8.0 |
| 69 | +git tag 8.0.NN-psN |
| 70 | +git push public 8.0.NN-psN |
| 71 | +``` |
| 72 | + |
| 73 | +For the release channels, read the Vitess docs [here](https://coda.io/d/Vitess-Upgrades_dy3xDEfjOx5/Vitess-Upgrades_sui49), and do the MySQL equivalents. In short, create an api-bb PR to update `MYSQLD_VERSIONS` in [`database_branch.rb`](https://github.com/planetscale/api-bb/blob/main/app/models/database_branch.rb#L174), disable the [`update_mysqld_image_version`](https://admin.planetscale.com/admin/feature-flags/update_mysqld_image_version) feature flag, merge the api-bb PR, and slowly increment the feature flag to 100%. Normally, new code goes to the `next` channel only. Repeat the process with `stable` and `oldstable` a day or a week later until all branches are running the desired version of MySQL. |
| 74 | + |
| 75 | +The update process will never downgrade the version of MySQL on a cluster. An image is newer if its MySQL version (e.g., `8.0.28`) is higher, or if the MySQL version is the same and the build date is higher. A branch-deployed image (e.g., `8.0.23-20220601-ps-branchname`) is always pinned and does not get replaced by the update process. |
| 76 | + |
| 77 | +# Feature flags |
| 78 | + |
| 79 | +There are [feature flags](https://admin.planetscale.com/admin/feature-flags) with names like `vitess_flag_something` that get passed to Singularity to affect PSC behavior. Each of those flags has a minimum version of Vitess and/or MySQL that they require; any PSC not running suiteable images ignores the given flag. If a flag requires a particular version of Vitess, add an entry to `VITESS_FLAG_MIN_VERSION` in `database_branch.rb`. If it requires a particular version of MySQL, add an entry to `MYSQL_FLAG_MIN_VERSION`. Each `vitess_flag_something` must be gated on either a Vitess version, a MySQL version, or both. |
| 80 | + |
| 81 | +Note that a few database branches have 8.0.28 builds of MySQL rather than 8.0.23. That is because we initially did our development on 8.0.28, before hedging our bets (minimizing changes and risk) and rebasing on 8.0.23. Those 8.0.28 clusters can't be downgraded to 8.0.23, because MySQL made breaking changes to the on-disk format. Because the 8.0.28 clusters have a higher version number, they are presumed to support all `vitess_flag_something` flags that require 8.0.23 or higher. To avoid compatibility traps when deploying a new MySQL-gated feature flag, make sure to create an 8.0.28 image that supports the flag and deploy it to all 8.0.28 clusters before enabling the feature flag. |
| 82 | + |
| 83 | +The list of 8.0.28 clusters can be found by querying the main database for `WHERE database_branch.mysqld_image_version like '8.0.28%'`. It currently comprises three test clusters and `planetscale/beam/main`. |
0 commit comments