update deprecation template steps (#55163)

Author: rachmari

src/ghes-releases/lib/deprecation-steps.md

@@ -8,7 +8,9 @@ labels:
 
 # Deprecation steps for GHES releases
 
-The day after a GHES version's [deprecation date](https://github.com/github/docs-internal/tree/main/src/ghes-releases/lib/enterprise-dates.json), a banner on the docs will say: `This version was deprecated on <date>.` This is all users need to know. However, we don't want to update those docs anymore or link to them in the nav. Follow the steps in this issue to **archive** the docs.
+The day after a GHES version's [deprecation date](https://github.com/github/docs-internal/tree/main/src/ghes-releases/lib/enterprise-dates.json), a banner on the docs will say: `This version was deprecated on <date>.` This lets users know that the release is deprecated. However, until the release is fully deprecated, it will show up in the Versions dropdown on the docs.github.com site.
+
+When we fully deprecate the release, we remove all any content (YML, JSON, Markdown) versioned for that release or lower. Follow the steps in this issue to fully **deprecate** the docs.
 
 **Note**: Each step below, except step 0, must be done in order. Only move on to the next step after successfully completing the previous step.
 
@@ -17,38 +19,48 @@ The following large repositories are used throughout this checklist, it may be u
 - `github/github`
 - `github/docs-internal`
 
-Additionally, you can download:
-
-- [Azure Storage Explorer](https://aka.ms/portalfx/downloadstorageexplorer)
+## Step 0: Confirm the deprecation date
 
-## Step 0: Before beginning the deprecation, ensure the date of the deprecation is correctly defined
-
-- [ ] Completed step 0 ✅
+Before beginning the deprecation, ensure the date of the deprecation is correctly defined:
 
 1. Check that the deprecation date is correct by looking up the version you are deprecating in the [release date list](https://github.com/github/enterprise-releases/blob/master/releases.json) and finding the corresponding `prp` owner. Send them a slack message to confirm that the date is correct. If the date is being pushed out, you can ask the `prp` to update the date in the release date list. If the release date list does not get updated (it doesn't always) we have to prepare that our version of that file (`src/ghes-releases/lib/enterprise-dates.json`) will also be inaccurate.
 
-   If there is no `prp` defined, reach out to our content friends for help in the #docs-content-enterprise Slack channel.
+   If there is no `prp` defined, reach out to our content friends for help in the #docs-content-enterprise or #ghes-releases Slack channel.
 
 1. If this release is being pushed out, update the target date of this issue and you can wait to proceed with any futher steps.
 
-## Step 1: Remove deprecated version numbers from docs-content issue forms
+1. In the `docs-content` repo, remove the deprecated GHES version number from the `options` list in [`release-tracking.yml`](https://github.com/github/docs-content/blob/main/.github/ISSUE_TEMPLATE/release-tracking.yml).
+
+1. When the PR is approved, merge it in.
 
-- [ ] Completed step 1 ✅
+## Step 1: Create the new archived repository
 
-**Note**: This step can be performed independently of all other steps, and can be done several days before or along with the other steps.
+All previously archived content lives in its own repository. For example, GHES 3.11 archived content is located in https://github.com/github/docs-ghes-3.11.
 
-In the `docs-content` repo, remove the deprecated GHES version number from the `options` list in [`release-tracking.yml`](https://github.com/github/docs-content/blob/main/.github/ISSUE_TEMPLATE/release-tracking.yml).
+1. Create a new repository that will store the scraped deprecated files:
+
+   ```shell
+   npm run deprecate-ghes -- create-repo --version <release to deprecate>
+   ```
 
-When the PR is approved, merge it in.
+   For example, to deprecate GHES 3.11, you would run:
+
+   ```shell
+   npm run deprecate-ghes -- create-repo --version 3.11
+   ```
+
+1. From the new repository's home page, click the gear icon next to the "About" section and deselect the "Releases", "Packages", and "Depployments" checkboxes. Click "Save changes".
 
 ## Step 2: Dry run: Scrape the docs and archive the files
 
-- [ ] Completed step 2 ✅
+**Note:** You may want to perform the following dry run steps on a new temporary branch that you can delete after the dry run is complete.
 
 1. If the release date documented in the [release date list](https://github.com/github/enterprise-releases/blob/master/releases.json) is incorrect or differs from what we have documented in `src/ghes-releases/lib/enterprise-dates.json`, update the date in `src/ghes-releases/lib/enterprise-dates.json` to the correct deprecation date before proceeding with the deprecation. A banner is displayed on each page with a version that will be deprecated soon. The banner uses the dates defined in `src/ghes-releases/lib/enterprise-dates.json`.
+
 1. Ensure you have local clones of the [translation repositories](#configuring-the-translation-repositories).
+
 1. Update all translation directories to the latest `main` branch.
-1. You can do this on the main branch or check out a new temporary branch.
+
 1. Hide search component temporarily while scraping docs in `src/search/components/Search.tsx`, by adding the `visually-hidden` class to the `form` element:
 
     ```javascript
@@ -69,59 +81,78 @@ When the PR is approved, merge it in.
 1. Do a dry run by scraping a small amount of files to test locally on your machine. This command does not overwrite the references to asset files so they will render on your machine.
 
     ```shell
-    npm run archive-version -- --dry-run --local-dev
+    npm run deprecate-ghes -- archive --dry-run --local-dev
     ```
 
 1. Navigate to the scraped files directory (`tmpArchivalDir_<VERSION_TO_DEPRECATE>`) inside your docs-internal checkout. Open a few HTML files and ensure they render and drop-down pickers work correctly.
 
 1. If the dry-run looks good, scrape all content files. This will take about 20-30 minutes. **Note:**  This will overwrite the directory that was previously generated with new files. You can also create a specific output directory using the `--output` flag.
 
     ```shell
-    npm run archive-version
+    npm run deprecate-ghes -- archive
     ```
 
 1. Revert changes to `src/search/components/Search.tsx`.
 
 1. Check in any change to `src/ghes-releases/lib/enterprise-dates.json`.
 
-## Step 3: Upload the scraped content directory to Azure storage
+## Step 3: Commit the scraped docs to the new repository
 
-- [ ] Completed step 3 ✅
+1. Copy the scraped files from the `tmpArchivalDir_<VERSION_TO_DEPRECATE>` directory in `docs-internal` over to the new `github/docs-ghes-<RELEASE_NUM>` repository.
 
-1. Log in to the Azure portal from Okta. You'll first need to use the Azure JIT tile in Okta, and request access. Then either logout after being redirected to Azure and log back in, or log into the normal Azure tile from Okta. That will force a refresh on your access and give you write permissions to upload files.
+1. Commit the files. A GitHub Pages build should automatically begin, creating the static site that serves these docs.
 
-1. Once in Azure, navigate to the [githubdocs Azure Storage Blob resource](https://portal.azure.com/#@githubazure.onmicrosoft.com/resource/subscriptions/fa6134a7-f27e-4972-8e9f-0cedffa328f1/resourceGroups/docs-production/providers/Microsoft.Storage/storageAccounts/githubdocs/overview).
+1. Preview a few pages, by navigating to the full URL checked into the repo. For example, for GHES 3.11, you can view `https://github.github.com/docs-ghes-3.11/en/[email protected]/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/about-notifications/index.html`.
 
-1. Upload the files. You can do this directly from the UI or using the Microsoft Azure Storage Explorer app. The app allows you more insight into the status of the upload and allows you to delete directories.
+1. Remove the `tmpArchivalDir_<VERSION_TO_DEPRECATE>` directory from your `github/docs-internal` checkout.
 
-   UI method:
+## Step 4: Deprecate the GHES release in docs-internal
 
-   1. Click `Containers` in the left sidebar, then click the `enterprise` container.
+1. In your `docs-internal` checkout, create a new branch: `git checkout -b deprecate-<version>`.
 
-   1. Click `Upload` in the top bar. Drag and drop or click `Browse for files` to select the directory inside of `tmpArchivalDir_<VERSION_TO_DEPRECATE>`. For example, at the time of this writing, the directory name was `3.5`. The upload will take several minutes and the UI may not give feedback immediately. Don't close the browser tab or navigate away from the page until the upload is complete.
+1. In your `docs-internal` checkout, edit `src/versions/lib/enterprise-server-releases.js` by removing the version number to be deprecated from the `supported` array and move it to the `deprecatedWithFunctionalRedirects` array.
 
-   App method:
+1. Deprecate the automated pipelines data files:
 
-   1. Open the Microsoft Azure Storage Explorer app from the `Overview` tab [githubdocs Azure Storage Blob resource page](https://portal.azure.com/#@githubazure.onmicrosoft.com/resource/subscriptions/fa6134a7-f27e-4972-8e9f-0cedffa328f1/resourceGroups/docs-production/providers/Microsoft.Storage/storageAccounts/githubdocs/overview).
-   1. You'll need to navigate to the correct subscription, resource, and container.
-   1. To upload, Click "Upload" and select "Upload folder." Click the "Selected folder" input to navigate to the temp directory you just generated. Inside that temp directory, select the `<VERSION_TO_DEPRECATE>` directory (e.g., `3.2`). Leave the destination directory input blank.
+   ```shell
+   npm run deprecate-ghes -- pipelines
+   ```
 
-1. From the UI, click on the newly uploaded directory navigate to an HTML file. Clicking on the file will show you a metadata page for that file. Click the `URL` copy button. Navigate to that URL in your browser to see the rendered HTML page. Ensure that the pages render correctly and that the drop-down pickers work correctly.
+1. Remove deprecated content files and update the versions frontmatter:
 
-1. Remove the temporarily created directory from your `github/docs-internal` checkout.
+    ```shell
+   npm run deprecate-ghes -- content
+    ```
 
-## Step 4: Deprecate the GHES release in docs-internal
+1. Remove deprecated Liquid from content and data files. **Note:** The previous step to update content file frontmatter must have run successfully for this step to work because the updated frontmatter is used to determine file versions.
 
-- [ ] Completed step 4 ✅
+   ```shell
+   npm run lint-content -- --paths content data --rules liquid-unused-conditional --fix
+   ```
 
-This step will remove the version from the drop-down picker, effectively deprecating the version from a user's perspective. The content for the deprecated release will still exist in the Markdown files.
+1. There are some `data/variables/*.yml` files that can't be autofixed. These will show up as errors. You can manually make the changes to these files. For example, this means open file data/variables/code-scanning and find the code_scanning_thread_model_support key. Edit the key’s value to remove the deprecated liquid:
 
-1. In your `docs-internal` checkout, create a new branch: `git checkout -b deprecate-<version>`.
+   ![Output from script that indicates manual fixes to variable files are needed](./variable-example.png)
 
-1. In your `docs-internal` checkout, edit `src/versions/lib/enterprise-server-releases.js` by removing the version number to be deprecated from the `supported` array and move it to the `deprecatedWithFunctionalRedirects` array.
+1. Deprecate any data files that are now empty, remove data resuables references that were deleted:
+
+   ```shell
+   npm run deprecate-ghes -- data
+   ```
 
-1. You can test that the static pages were generated correctly on localhost and on staging. Verify that the static pages are accessible by running `npm run dev` in your local `docs-internal` checkout and navigate to:
-`http://localhost:3000/enterprise/<version>/`.
+1. Run the linter again to remove whitespace and check for any other errors:
+
+   ```shell
+   npm run lint-content -- --fix
+   ```
+
+1. Use VSCode find/replace to remove any remaining table pipes after liquid has been removed. For example lines that only contain 1 or two pipes: ` |` or ` | |`. You can use the following regexes: `^\|\s*\|$` and `^\s?\|\s?$`.
+
+1. Test the changes by running the site locally:
+
+   ```shell
+   npm run start
+   ```
 
 1. Poke around several deprecated pages by navigating to `docs.github.com/enterprise/<DEPRECATED VERSION>`, and ensure that:
    - Stylesheets are working properly
@@ -131,38 +162,18 @@ This step will remove the version from the drop-down picker, effectively depreca
    - You should see a banner on the top of every deprecated page with the date that the version was deprecated.
    - You should see a banner at the top of every page for the oldes currently supported version with the date that it will be deprecated in the ~3 months.
 
-1. If everything looks good, check in the changes to `src/versions/lib/enterprise-server-releases.js` and create a pull request.
+1. If everything looks good, check in all changes and create a pull request.
 
 1. Ensure that CI is passing or make any changes to content needed to get tests to pass.
 
+1. Add the PR to the [docs-content review board](https://github.com/orgs/github/projects/2936/views/2).
+
 1. 🚢 Ship the change.
 
 ## Step 5: Create a tag
 
-- [ ] ✅ Completed step 5
-
 1. Create a new tag for the most recent commit on the `main` branch so that we can keep track of where in commit history we removed the GHES release. Create a tag called `enterprise-<release number>-release`. To create only a tag and not a release, you can [create a new release](https://github.com/github/docs-internal/releases), which allows you to "Choose a tag." Select add a new tag and use the tag name as the release title. After creating the new release, you will see the new tag as well. You can then delete the release.
 
-## Step 6: Remove static files and liquid conditionals for the version
-
-- [ ] Completed step 6 ✅
-
-1. In your `docs-internal` checkout, create a new branch: `git checkout -b remove-<version>-content`.
-
-1. Run `src/ghes-releases/scripts/sync-automated-pipeline-data.js` and commit results.
-
-1. Remove the outdated Liquid markup and frontmatter. **Note:** There are typically a few bugs in the updated Markdown, which will be caught by the content linter or CI. Fix any bugs you find. For example, a liquid end tag may be removed but the start tag still exists. There are typically only a few bugs to fix. The script does a pretty great job of fixing most use cases, so this is typically a lightweight task. If there are several errors, something is likely broken and should be fixed in the script.
-
-   ```shell
-   npm run remove-version-markup -- --release <number>
-   ```
-
-1. If data reusables were deleted automatically, you'll need to remove references to the deleted reusable in the content. Using VSCode to find the occurrences is quick and there are typically only a few to update. If you have any questions, reach out to the docs-content team in #docs-content to ask for help updating the Markdown files.
-
-1. Open a new PR. When the CI is 🍏, add the PR to the [docs-content review board](https://github.com/orgs/github/projects/2936/views/2).
-
-1. When the PR is approved, merge it in. 🚢
-
 ## Step 7: Deprecate the OpenAPI description in `github/github`
 
 1. In `github/github`, edit the release's config file in `app/api/description/config/releases/`, and change `deprecated: false` to `deprecated: true`.
@@ -200,9 +211,9 @@ TRANSLATIONS_ROOT_DE_DE=${TRANSLATIONS}/docs-internal.de-de
 
 ## Re-scraping a page or all pages
 
-Occasionally, a change will need to be added to our archived enterprise versions. If this occurs, you can check out the `enterprise-<release number>-release` branch and re-scrape the page or all pages using `npm run archive-version`. To scrape a single page you can use the `—page <page relative path>` option.
+Occasionally, a change will need to be added to our archived enterprise versions. If this occurs, you can check out the `enterprise-<release number>-release` branch and re-scrape the page or all pages using `npm run deprecate-ghes -- archive`. To scrape a single page you can use the `—page <page relative path>` option.
 
-For each language, upload the new file to Azure blob storage in the `enterprise` container.
+For each language, upload the new file to the `github/docs-ghes-<RELEASE_NUM>` repo.
 
 After uploading the new files, you will need to purge the Fastly cache for the single page. From Okta, go to Fastly and select `docs`. Click `Purge` then `Purge URL`. If you need to purge a whole path, just do a `Purge All`
 

src/ghes-releases/lib/variable-example.png

@@ -0,0 +1,77 @@
+# This script creates a new repository for an archived version of GitHub Enterprise Server documentation.
+# Please update the version variable first.
+# You may wish to run this script a little bit at a time instead of all at once incase there are any errors.
+
+version=$1
+cd ~/Documents/gh/github
+echo "--- Creating repository for github/docs-ghes-$version"
+echo "--- gh repo create"
+gh repo create \
+  "github/docs-ghes-$version" \
+  --add-readme \
+  --clone \
+  --description="Archived docs for GHES $version" \
+  --disable-issues \
+  --disable-wiki \
+  --license="CC-BY-4.0" \
+  --private \
+  --team="docs-engineering" \
+  --homepage="https://github.github.com/docs-ghes-$version/"
+echo "--- gh repo edit"
+gh repo edit \
+  "github/docs-ghes-$version" \
+  --add-topic="docs-ghes-archive" \
+  --allow-update-branch \
+  --delete-branch-on-merge \
+  --enable-auto-merge \
+  --enable-projects=false \
+  --enable-merge-commit=false \
+  --enable-rebase-merge=false
+echo "--- github/docs-engineering as admin"
+gh api -X PUT "/orgs/github/teams/docs-engineering/repos/github/docs-ghes-$version" \
+        -f 'permission=admin' --silent
+echo "--- github/employees as read"
+gh api -X PUT "/orgs/github/teams/employees/repos/github/docs-ghes-$version" --silent
+echo "--- Require a pull request review before merging"
+repositoryId="$(gh api graphql -f query="{repository(owner:\"github\",name:\"docs-ghes-$version\"){id}}" -q .data.repository.id)"
+gh api graphql -f query='
+mutation($repositoryId:ID!,$branch:String!,$requiredReviews:Int!) {
+  createBranchProtectionRule(input: {
+    repositoryId: $repositoryId
+    pattern: $branch
+    requiresApprovingReviews: true
+    requiredApprovingReviewCount: $requiredReviews
+    requiresCodeOwnerReviews: true
+  }) { clientMutationId }
+}' -f repositoryId="$repositoryId" -f branch=main -F requiredReviews=1 --silent
+echo "--- Enable GitHub Pages, set source to main in root directory, and make the pages site public"
+gh api -X POST "/repos/github/docs-ghes-$version/pages" \
+  -f "source[branch]=main" -f "source[path]=/" -f "public=true" --silent
+echo "--- Update custom properties"
+gh api --method PATCH /repos/github/docs-ghes-$version/properties/values \
+  -f "properties[][property_name]=ownership-name" \
+  -f "properties[][value]=@github/docs-engineering" \
+  -f "properties[][property_name]=ownership-type" \
+  -f "properties[][value]=Team" \
+  --silent
+echo "--- FILE UPDATES"
+cd "docs-ghes-$version"
+echo "--- docs engineering as codeowners"
+touch CODEOWNERS
+echo "* @github/docs-engineering" > CODEOWNERS
+echo "--- add index.html file"
+touch index.html
+echo "<h1>GitHub Enterprise Server $version Docs</h1>" > index.html
+echo "--- add .gitignore"
+touch .gitignore
+echo ".DS_Store" > .gitignore
+echo "--- add .nojekyll"
+touch .nojekyll
+echo "--- push"
+git add .
+git commit -am "initial commit"
+git push
+cd ..
+echo "--- END FILE UPDATES"
+echo "--- MANUAL NOTES"
+echo "Manually disable releases, packages, and deployments"