Skip to content

Commit e89544a

Browse files
mansonamansona
committed
Merge branch 'github'
2 parents af4f494 + ce690d8 commit e89544a

15 files changed

+163
-417
lines changed

‎MAINTAINERS.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Maintainers
2+
3+
In order to release a new version of the API docs manually, follow these steps.
4+
5+
1. Make sure you have the latest version of this repository, `ember.js`,
6+
`ember-data`, `ember-api-docs-data`, and `ember-api-docs`.
7+
2. Make sure you have a clean working directory in each repository,
8+
i.e. `git status`
9+
3. Go through the steps in the README to generate the docs and preview
10+
them in the web app
11+
4. If everything looks good, commit the changes to `ember-api-docs-data`
12+
and open a pull request to that repository.
13+
14+
When the pull request is merged in `ember-api-docs-data`, the files
15+
will be synced automatically to S3.
16+
Either you can wait for them to show up in the deployed website,
17+
or you can clear the Fastly cache to force them to show up faster.

‎README.md

Lines changed: 81 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -1,152 +1,127 @@
11
# Ember JSON API Docs [![Build Status](https://travis-ci.org/ember-learn/ember-jsonapi-docs.svg?branch=master)](https://travis-ci.org/ember-learn/ember-jsonapi-docs)
22

3-
This is an internal tool for generating API docs for the Ember.js framework and exposing it through [JSON:API](http://jsonapi.org/) for various applications (e.g. https://api.emberjs.com/). The tool allows you to:
3+
This tool gets the code comments from `ember.js` and `ember-data` libraries,
4+
turns them into JSON files, and then turns those JSON files into a
5+
[JSON:API](http://jsonapi.org/) format.
46

5-
- Generate API docs from [YUIDoc](http://yui.github.io/yuidoc/syntax/index.html) comments in the [emberjs/ember.js](https://github.com/emberjs/ember.js/) and [emberjs/data](https://github.com/emberjs/data) repositories in [YUIDoc](http://yui.github.io/yuidoc/) and [JSON:API](http://jsonapi.org/) formats.
6-
- Publish docs to an Amazon S3 bucket (`s3://api-docs.emberjs.com`).
7-
- Expose API docs in the JSON:API format through API.
7+
The files this tool creates are used to power
8+
[api.emberjs.com](https://api.emberjs.com).
89

9-
All the generated files are stored in the `tmp` folder under the project root:
1010

11-
```
12-
tmp
13-
├── json-docs // JSON:API-comlaint docs generated locally from YUIDoc files in `s3-docs`.
14-
├── rev-index // Rev index files used for generating JSON:API-comlaint docs.
15-
├── s3-docs // YUIDoc docs generated locally or downloaded from s3://api-docs.emberjs.com.
16-
```
17-
18-
> ℹ️ **NOTE:** If you are looking for the app behind https://api.emberjs.com/, visit [ember-api-docs](https://github.com/ember-learn/ember-api-docs) instead.
11+
> ℹ️ **NOTE:** If you are looking for the front end app behind https://api.emberjs.com/, visit [ember-api-docs](https://github.com/ember-learn/ember-api-docs) instead.
1912
2013
## Prerequisites
2114

22-
1. Install the latest [Node.js](https://nodejs.org/) LTS.
23-
2. Install [AWS CLI version 2](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html).
24-
3. Create an account in [AWS Management Console](https://console.aws.amazon.com). [Create a New Access Key](https://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html) (access key ID and secret access key) in *My Security Credentials* under your profile name to be able to access public AWS S3 buckets.
15+
- the latest [Node.js](https://nodejs.org/) LTS
16+
- [yarn v1](https://yarnpkg.com/)
2517

26-
## Quickstart
18+
Clone all of the following repositories into the same directory so they are "siblings" on the file system
2719

28-
### Generate Docs from Ember Source
20+
- This repository, `ember-jsonapi-docs`
21+
- [ember-api-docs-data](https://github.com/ember-learn/ember-api-docs-data)
22+
- [ember.js](https://github.com/emberjs/ember.js)
23+
- [ember-data](https://github.com/emberjs/data/)
2924

30-
You can generate docs from your local copies of [emberjs/ember.js](https://github.com/emberjs/ember.js/) and [emberjs/data](https://github.com/emberjs/data) repositories and serve them locally as [JSON:API](http://jsonapi.org/).
25+
## Generate docs
3126

32-
```bash
33-
# Clone Ember.js and Ember Data repositories into the same root folder.
34-
git clone https://github.com/emberjs/ember.js/
35-
git clone https://github.com/emberjs/data
27+
Decide which version and project you want to build docs for.
28+
Then, in `ember.js` and/or `ember-data` repositories, check out the version
29+
tags you want to generate docs for. For example:
3630

37-
# Clone this repository into the same root folder.
38-
git clone https://github.com/ember-learn/ember-jsonapi-docs
39-
cd ember-jsonapi-docs
40-
41-
# Install the dependencies.
42-
yarn
31+
```sh
32+
get fetch --all
33+
git checkout v5.2.0
34+
```
4335

44-
# Generate docs for a particular project from its local repository.
45-
# Version should match the one in `package.json` of a target project.
46-
yarn gen --project ember --version 3.17.0
47-
yarn gen --project ember-data --version 3.17.0
36+
Generate the JSON docs for `ember` and/or `ember-data`. This will add new JSON
37+
files into the `s3-docs` directory of `ember-api-docs-data`:
4838

49-
# Run API locally.
50-
yarn serve
39+
```sh
40+
yarn
41+
yarn gen --project ember --version "5.2.0"
5142
```
5243

53-
> ℹ️ **NOTE:** `--version` should match the one in the `package.json` of a target Ember project. If `package.json` uses a release name (e.g. `beta` or `canary`), omit it and use only numbers. For example, if the `package.json` says `3.19.0-beta.2`, use `3.19.0`.
54-
55-
> ✅ **TIP:** If you are debugging failed builds, periodically clear out the contents of the `tmp` directory, and run the script again. Past failed runs can cause subsequent runs to fail in unexpected ways.
44+
I would recommend committing at this stage so that you can see that the following steps work.
5645

57-
### Generate Docs from YUIDoc Files Stored in AWS
46+
```sh
47+
cd ../ember-api-docs-data/
48+
git add .
49+
git commit -m "add docs for ember v5.2.0"
50+
```
5851

59-
All [Ember.js](https://github.com/emberjs/ember.js/) and [Ember.js Data](https://github.com/emberjs/data) releases have already generated docs in a public Amazon S3 bucket (`s3://api-docs.emberjs.com`). You can download them and serve locally as [JSON:API](http://jsonapi.org/).
52+
Next we need to fix the generated files. (Note: this step could probably be incorporated into ember-jsonapi-docs
53+
but for now this step works).
6054

61-
> ⚠️ **WARNING:** The app tries to process all Ember.js and Ember Data versions since 1.0 which takes high memory & time to complete.
55+
```sh
56+
node ./clean-urls.js
57+
node ./fix-rev-index.js
58+
git add .
59+
git commit -m "fix urls and rev-index for ember"
60+
```
6261

63-
```bash
64-
# Clone the repository.
65-
git clone https://github.com/ember-learn/ember-jsonapi-docs
66-
cd ember-jsonapi-docs
62+
Lastly we need to make sure that each of the new files are valid. You need to run the test suite in `ember-api-docs-data`
6763

68-
# Install the dependencies.
69-
yarn
70-
71-
# Set environment variables to get access to s3://api-docs.emberjs.com.
72-
# Use Access Keys generated in step 3 in "Prerequisites".
73-
export AWS_ACCESS_KEY_ID=xxxxxx
74-
export AWS_SECRET_ACCESS_KEY=xxxxx
64+
```sh
65+
npm i
66+
npm test
67+
```
7568

76-
# Download YUIDoc docs and index files for all projects/releases from s3://api-docs.emberjs.com.
77-
# Then, generate JSON:API-comlaint docs from the downloaded files.
78-
yarn start --sync --max_old_space_size=8192
69+
If you see any errors e.g. `AssertionError: expected '' not to be empty` you need to manually fix these. These are caused by bugs in the yuidoc generation and can probably be fixed in the source files somehow.
7970

80-
# At this point, you can also generate JSON:API-comlaint docs only for
81-
# a particular project/release.
82-
yarn start --project ember --version 3.17.0
83-
yarn start --project ember-data --version 3.17.0
71+
Once all of the fixes are in place you can commit them:
8472

85-
# Run API locally.
86-
yarn serve
73+
```sh
74+
git add .
75+
git commit -m "fixing tests for ember"
8776
```
8877

89-
> ℹ️ **NOTE:** If docs for a particular version are missing in `s3://api-docs.emberjs.com`, the tool downloads them from npm (e.g. https://unpkg.com/[email protected]/docs/data.json) as a fallback.
90-
91-
## Overriding a specific version of YUIDoc file with a local copy (for core contributors).
78+
---
9279

93-
To proceed, you need AWS Access Keys to publish to [api-docs.emberjs.com](http://api-docs.emberjs.com/) and all necessary environemnt variables set.
80+
Once that is all committed I would recommend doing the same thing for ember-data with new commits:
9481

95-
> ⚠️ **WARNING:** In its present form this should be used only when there aren't new docs out there that are yet to be processed. For example, if Ember 3.17 is released but isn't indexed yet you should wait for this app to finish processing it via the cron job on Heroku before you can proceed to modify any of the existing docs from your machine.
82+
```sh
83+
cd ../ember-jsonapi-docs
84+
yarn gen --project ember-data --version "5.2.0"
9685

97-
```bash
98-
# Set environment variables.
99-
export AWS_ACCESS_KEY_ID=xxxxxx
100-
export AWS_SECRET_ACCESS_KEY=xxxxx
86+
cd ../ember-api-docs-data
10187

102-
# Download YUIDoc docs and index files for all releases from s3://api-docs.emberjs.com.
103-
# This is important so that we don't loose other versions of docs that
104-
# are already out there when the index files are generated.
105-
yarn start --sync --max_old_space_size=8192
88+
git add .
89+
git commit -m "add docs for ember-data v5.2.0"
10690

107-
# Go to the folder and and replace a YUIDoc file that you want to be processed.
108-
# Ensure that the file name is same as the one that's already there.
109-
cd tmp/s3-docs/<VERSION_TO_REPLACE>
91+
node ./clean-urls.js
92+
node ./fix-rev-index.js
93+
git add .
94+
git commit -m "fix urls and rev-index for ember-data"
11095

111-
# Set an environment variable to enable publishing to s3://api-docs.emberjs.com.
112-
export AWS_SHOULD_PUBLISH=yes
96+
npm test
11397

114-
# Regenerate JSON:API-comlaint docs only for a particular project/release and
115-
# publish them to s3://api-docs.emberjs.com...
116-
yarn start --project ember --version <VERSION_TO_REPLACE> --ignorePreviouslyIndexedDoc
117-
yarn start --project ember-data --version <VERSION_TO_REPLACE> --ignorePreviouslyIndexedDoc
118-
119-
# OR
120-
# Regenerate JSON:API-comlaint docs for ALL projects/releases regardless of indexed version and
121-
# publish them to s3://api-docs.emberjs.com.
122-
yarn start --clean --max_old_space_size=8192
98+
# if you needed to fix any files you do that now and then
99+
git add .
100+
git commit -m "fixing tests for ember-data"
123101
```
124102

125-
## Backing Up Docs
126-
127-
If you plan to run a major migration, back up all the content to a timestamped folder in the Amazon S3 bucket.
103+
Once that is all done, push it and open a PR 🎉
128104

129-
```bash
130-
yarn backup
131-
```
105+
> ℹ️ **NOTE:** `--version` should match the one in the `package.json` of a target Ember project. If `package.json` uses a release name (e.g. `beta` or `canary`), omit it and use only numbers. For example, if the `package.json` says `3.19.0-beta.2`, use `3.19.0`.
132106
133-
## FAQ
107+
> ✅ **TIP:** If you are debugging failed builds, periodically discard the changes
108+
made to `ember-api-docs-data`, since changes are made in-place.
134109

135-
### Can I use API from the [ember-api-docs](https://github.com/ember-learn/ember-api-docs) app?
110+
## (Optional) View the generated docs in a web app
136111

137-
Yes, follow one of the quickstarts and then run the `ember-api-docs` application using the following commands.
112+
The Prembered version of the ember-api-docs expects a folder in its root that links to the `ember-api-docs-data` folder, so all you need to do is create a symbolic link to ember-api-docs-data and you can see the app running locally.
138113

139-
```bash
140-
# Start your local server for this repository
141-
yarn serve
114+
Clone the [ember-api-docs](https://github.com/ember-learn/ember-api-docs)
115+
repository, install dependencies, and start the front end in "local" mode:
142116

143-
# Clone the repository with the "ember-api-docs" app.
117+
```sh
144118
git clone https://github.com/ember-learn/ember-api-docs
145119
cd ember-api-docs
146-
147-
# Install the dependencies.
120+
ln -s ../ember-api-docs-data # assuming it's checked out in the same folder
148121
npm install
149-
150-
# Run the application side by side with a locally running API.
151-
npm run start:local
122+
yarn start
152123
```
124+
125+
> Note: at the time of writing this the ember-api-docs app needs to be on the `prember` branch but this will change very soon when we go live with it.
126+
127+
Visit the app in your browser at [http://localhost:4200](http://localhost:4200)

‎app.json

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,4 @@
11
{
2-
"env": {
3-
"AWS_ACCESS_KEY_ID": {
4-
"required": true
5-
},
6-
"AWS_SECRET_ACCESS_KEY": {
7-
"required": true
8-
},
9-
"AWS_SHOULD_PUBLISH": {
10-
"required": true,
11-
"default": "yes"
12-
}
13-
},
142
"buildpacks": [
153
{
164
"url": "heroku/nodejs"

‎backup.js

Lines changed: 0 additions & 7 deletions
This file was deleted.

‎generate-local.js

Lines changed: 27 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
import chalk from 'chalk'
22
import commandExists from 'command-exists'
33
import execa from 'execa'
4-
import { copyFileSync, existsSync, mkdirpSync, removeSync } from 'fs-extra'
4+
import { copyFileSync, ensureDirSync, ensureFileSync, existsSync, mkdirpSync, removeSync } from 'fs-extra'
55
import minimist from 'minimist'
66
import path from 'path'
77
import 'hard-rejection/register'
88

9-
import { apiDocsProcessor } from './main'
9+
const docsPath = '../ember-api-docs-data';
1010

1111
const argv = minimist(process.argv.slice(2))
1212

@@ -17,10 +17,9 @@ const exit = function exit() {
1717
process.exit(1)
1818
}
1919

20-
const runCmd = async (cmd, path) => {
21-
console.log(chalk.underline(`Running '${chalk.green(cmd)}' in '${path}'`))
22-
try {
23-
const executedCmd = await execa(`cd ${path} && ${cmd}`, { shell: true })
20+
const runCmd = async (cmd, path, args = []) => {
21+
console.log(chalk.underline(`Running '${chalk.green(cmd)}' in ${path}`))
22+
const executedCmd = await execa(cmd, args, { cwd: path, shell: true, stdio: 'inherit' })
2423

2524
if (executedCmd.failed) {
2625
console.error(executedCmd.stdout)
@@ -64,20 +63,25 @@ const runCmd = async (cmd, path) => {
6463
let buildDocs = async projDirPath => {
6564
checkIfProjectDirExists(projDirPath)
6665

66+
if (project === 'ember') {
67+
await runCmd('volta', projDirPath, ['run', 'yarn'])
68+
} else {
69+
await runCmd('corepack', projDirPath, ['pnpm', 'install'])
70+
}
71+
72+
6773
if (install) {
6874
await runCmd(project === 'ember' ? 'yarn' : 'pnpm install', projDirPath)
6975
console.log('\n\n')
7076
}
7177

72-
if (build) {
73-
await runCmd(project === 'ember' ? 'yarn docs' : 'pnpm build:docs', projDirPath)
74-
}
78+
await runCmd(project === 'ember' ? 'volta run yarn docs' : 'corepack pnpm run build:docs', projDirPath)
7579

76-
const projYuiDocFile = `tmp/s3-docs/v${version}/${project}-docs.json`
80+
let destination = `${docsPath}/s3-docs/v${version}/${project}-docs.json`
81+
ensureFileSync(destination)
82+
const projYuiDocFile = destination;
7783
removeSync(projYuiDocFile)
78-
removeSync(`tmp/json-docs/${project}/${version}`)
79-
80-
mkdirpSync(`tmp/s3-docs/v${version}`)
84+
removeSync(`${docsPath}/json-docs/${project}/${version}`)
8185

8286
const yuiDocFile = path.join(
8387
projDirPath,
@@ -93,5 +97,14 @@ const runCmd = async (cmd, path) => {
9397

9498
await buildDocs(dirMap[project])
9599

96-
await apiDocsProcessor([project], [version], true, false, true)
100+
await execa('volta', [
101+
'run',
102+
'yarn',
103+
'start',
104+
'--project',
105+
project,
106+
'--version',
107+
version,
108+
'--no-sync'
109+
]).stdout.pipe(process.stdout)
97110
})()

‎index.js

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,8 @@ let projects =
99
argv.project && possibleProjects.includes(argv.project) ? [argv.project] : possibleProjects
1010
let specificDocsVersion = argv.version ? argv.version : ''
1111

12-
let ignorePreviouslyIndexedDoc =
13-
projects.length !== 0 && specificDocsVersion !== '' && argv.ignorePreviouslyIndexedDoc
1412
let runClean = !!argv.clean
1513
let noSync = !argv.sync
1614

1715
const { apiDocsProcessor } = require('./main.js')
18-
apiDocsProcessor(projects, specificDocsVersion, ignorePreviouslyIndexedDoc, runClean, noSync)
16+
apiDocsProcessor(projects, specificDocsVersion, runClean, noSync)

0 commit comments

Comments
 (0)