Add docs README.md and a custom directive (#208)

Summary:
Pull Request resolved: #208

Add a basic README for the documentation folder that gives a basic overview of how to build locally and contribute, as well as provide tips on how to author better pages with MyST parser.
Add a custom Sphinx directive that adds a code block with a nightly PyTorch version from the nightly.txt file.

Reviewed By: mergennachin, dbort

Differential Revision: D48757229

fbshipit-source-id: 6a83d46ae65a694bcc24bced88d4a999b998a94b

Author: svekars

.ci/docker/requirements-ci.txt

@@ -17,3 +17,4 @@ matplotlib
 -e git+https://github.com/pytorch/pytorch_sphinx_theme.git#egg=pytorch_sphinx_theme
 myst-parser
 sphinx_design
+sphinx-copybutton

docs/README.md

@@ -1,126 +1,134 @@
-# Contributing to the Executorch Website
+# ExecuTorch Documentation
 
-# Layout
+Welcome to the ExecuTorch documentation! This README.md will provide an
+overview of the ExecuTorch docs and its features, as well as instructions on
+how to contribute and build locally.
 
-Directory structure:
+<!-- toc -->
+
+- [Toolchain Overview](#toolchain-overview)
+- [Building Locally](#building-locally)
+- [Using Custom Variables](#using-custom-variables)
+- [Including READMEs to the Documentation Build](#including-readmes-to-the-documentation-build)
+- [Contributing](#contributing)
+- [Auto-generated API documentation](#auto-generated-api-documentation)
+  - [Python APIs](#python-apis)
+  - [C++ APIs](#c-apis)
+
+<!-- tocstop -->
+
+
+## Toolchain Overview
+
+We are using [sphinx](https://www.sphinx-doc.org/en/master/) with
+[myst_parser](https://myst-parser.readthedocs.io/en/latest/),
+[sphinx-gallery](https://sphinx-gallery.github.io/stable/index.html), and
+[sphinx_design](https://sphinx-design.readthedocs.io/en/latest/) in this
+documentation set.
+
+We support both `.rst` and `.md` files but prefer the content to be authored in
+`.md` as much as possible.
+
+## Building Locally
+
+Documentation dependancies are stored in
+[.ci/docker/requirements-ci.txt](https://github.com/pytorch/executorch/blob/main/.ci/docker/requirements-ci.txt).
+
+To build the documentation locally:
+
+1. Clone the executorch repo to your machine.
+
+1. Switch to the `docs/` directory.
+
+1. Start a virtual environment:
+
+   ```bash
+   virtualenv venv
+   source venv/bin/activate
+   ```
+
+1. Install dependancies:
+
+   ```bash
+   pip3 install -r ../.ci/docker/requirements-ci.txt
+   ```
+
+1. Build the documentation set:
+
+   ```
+   make html
+   ```
+
+   This should build both documentation and tutorials. The build will be
+   placed in the `_build` directory.
+
+1. You can preview locally by using [sphinx-serve](https://pypi.org/project/sphinx-serve/).
+   To install sphinx-serve, run: `pip3 install sphinx-serve`. To serve
+   your documentation:
+
+   ```
+   sphinx-serve -b _build
+   ```
+
+   Open http://0.0.0.0:8081/ in your browser to preview your updated
+   documentation.
+
+## Using Custom Variables
+
+You can use custom variables in your `.md` and `.rst` files.
+The variables take their values from the files listed in the
+`./.ci/docker/ci_commit_pins/` directory.
+For example, to insert a variable that specifies the latest Buck2 version,
+use the following syntax:
 
 ```
-docs
-│   README.md
-│
-└─── website
-│   │   docusaurus.config.json
-│   │   sidebar.js
-|   └─── fb
-│   |   └─── sidebar.js
-│   │    ...
-|   └─── docs
-│       |    .md files
-│       │   ...
-|       └─── fb
-|           └─── .md files
-│
-└─── source_py
-│   │    conf.py
-│   └─── .rst files
-|
-└─── source_cpp
-    │    Doxyfile
-    │    conf.py
-    └─── index.rst files
+The current version of Buck2 is ${executorch_version:buck2}.
 ```
 
-The Executorch website is hosted on
-https://www.internalfb.com/intern/staticdocs/executorch/.
-
-# General Documentation
-
-General documentation can be written in Markdown and live within the
-`website/docs/` directory. Through Static Docs, the Markdown files will be
-automatatically rendered into a pretty static website on
-https://www.internalfb.com/intern/staticdocs/executorch/.
-
-To add a page to the website, create a .md file under the `website/docs/`
-directory. If the page is internal-only, create the file under
-`website/docs/fb/`. Afterwards, go to `website/sidebars.js` or
-`website/fb/sidebars.js` (depending on if your document is public or
-internal-only) and add your document there so that it will show up on the
-sidebar.
-
-To view your changes:
-1. `ssh -L 3000:localhost:3000 <devserver>` from your laptop to your devserver.
-2. `cd fbcode/executorch/docs/website`
-3. `yarn` to install any packages
-4. `yarn build` to generate static HTML files
-5. `yarn start-fb` to start a local development server and view the internal
-variant of the website at `localhost:3000`. `yarn start` to view the public
-variant of the website. Most changes are reflected live without having to
-restart the server.
-
-Additionally, once you make changes to the website and create a diff containing
-your changes, a Static Docs Bot will automatically add an ephemeral version of
-the Static Docs website containing your changes to the test plan of the diff
-created. It may take around 10-20 minutes for the Static Docs Bot to populate
-the ephemeral site with your changes and update the test plan of the diff on
-Phabricator. It should look something like this:
-
-![static_docs_preview](static_docs_preview.png)
-
-# Auto-generated APIs
-
-Auto-generated API documentation are currently generated into
-https://www.internalfb.com/intern/staticdocs/executorch/py_api/ for Python APIs,
-and https://www.internalfb.com/intern/staticdocs/executorch/cpp_api/ for C++
-APIs.
+This will result in the following output:
 
-On a high level (will go into detail later), we use Sphinx to generate both
-Python and C++ documentation in the form of HTML pages. These pages are
-generated into `website/static` and are linked on the navigation bar (which is
-configured in `website/docusaurus.config.js`).
+<img src="./source/_static/img/s_custom_variables_extension.png" width="300">
+
+We have the following custom variables defined in this docset:
+
+* `${executorch_version:buck2}`
+* `${executorch_version:nightly}`
+* `${executorch_version:pytorch}`
+* `${executorch_version:vision}`
+* `${executorch_version:audio}`
+
+
+You can use the variables in both regular text and code blocks.
 
-## Setup (in fbcode)
 
-Since we are using Sphinx, which is an external library, we need to set up a
-`TARGETS` file within your working repository in order to run sphinx commands
-that you might typically be able to run just on the command line. Specifically
-the ones we used were:
+## Including READMEs to the Documentation Build
 
-* [`sphinx-build`](https://www.sphinx-doc.org/en/master/man/sphinx-build.html) in order to build the Sphinx project and generate the HTML files
-* [`sphinx-apidoc`](https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html) in order to generate an initial set of Sphinx sources
+You might want to include some of the `README.md` files from various
+directories in this repositories in your documentation build.
+To do that, create an `.md` file and use the `{include}` directive to
+insert your `.md` files. Example:
 
-A sample `TARGETS` file looks like:
 ```
-python_binary(
-    # @autodeps-skip
-    name = "sphinx-build",
-    main_module = "sphinx.cmd.build",
-    deps = [
-        "fbsource//third-party/pypi/breathe:breathe",
-        "fbsource//third-party/pypi/exhale:exhale",
-        "fbsource//third-party/pypi/pytorch-sphinx-theme:pytorch-sphinx-theme",
-        "fbsource//third-party/pypi/sphinx:sphinx",
-    ],
-)
-
-python_binary(
-    # @autodeps-skip
-    name = "sphinx-apidoc",
-    main_module = "sphinx.ext.apidoc",
-    deps = [
-        "fbsource//third-party/pypi/pytorch-sphinx-theme:pytorch-sphinx-theme",
-        "fbsource//third-party/pypi/sphinx:sphinx",
-    ],
-)
+```{include} ../README.md
 ```
 
-Any libraries that are used within your doc generation should be placed within
-the dependency list of the `sphinx-build` binary in the `TARGETS` file created.
-If the library you are using does not exist within `fbsource/third-party`,
-follow this
-[link](https://www.internalfb.com/intern/wiki/Third-Party_for_Megarepo/Python_(PyPI)/)
-to fork that library into `fbsource/third-party` so that you can import it here.
+**NOTE:** Many `README.md` files are written as placeholders with limited information
+provided. Some of that content you might want to keep in the repository rather
+than on the website. If you still want to add it, make sure to check
+the content for accuracy, structure, and overall quality.
 
-## Generating the documentation
+## Contributing
+
+Use the [PyTorch contributing guidelines](https://github.com/pytorch/pytorch/blob/main/CONTRIBUTING.md#writing-documentation)
+to contribute to the documentation.
+
+In addition to that, see [Markdown in Sphinx Tips and Tricks](https://pytorch.org/executorch/markdown-sphinx-tips-tricks.html)
+for tips on how to author high-quality markdown pages with Myst Parser.
+
+# Auto-generated API documentation
+
+On a high level (will go into detail later), we use Sphinx to generate both
+Python and C++ documentation in the form of HTML pages.
 
 ### Python APIs
 
@@ -132,8 +140,9 @@ The setup for Python documentation lies within `source_py/`. To set up Sphinx, a
 `conf.py` configuration file is required. We can specify ways to generate
 documentation here. Specifically, the most important/relevant parts are:
 
-* Make sure to add a path to the directory above the directory you're trying to generate documentation for. For example, since we want to generate documenation for the `executorch/` directory, the `sys.path` is `.../fbsource/fbcode/`. This tells Sphinx where to find the code to generate docs for.
-* `extensions` contains extension modules. For auto-generating APIs make sure to include `sphinx.ext.autodoc`.
+* Make sure to add a path to the directory above the directory you're trying to generate documentation for. For example, since we want to generate documenation for the `executorch/` directory. This tells Sphinx where to find the code to generate docs for.
+
+* `extensions` contains extension modules. For auto-generating APIs, make sure to include `sphinx.ext.autodoc`.
 * `autodoc_mock_imports` is where you put imports that Sphinx is unable to access. Sphinx runs your code in order to autogenerate the docs, so for any libraries that it unable to access due to it being outside of the directory, or containing c++ bindings, we need to specify it in the `autodoc_mock_imports` list. You can see what modules Sphinx is confused by when you run into importing errors when generating docs.
 
 Additionally, RST files are needed in order to specify the structure of the
@@ -165,11 +174,10 @@ To view your changes on the executorch website, you can follow the same steps
 listed in the "General Documentation" section.
 
 To view just the auto-generated pages:
-1. `ssh -L 8000:localhost:8000 <devserver>` from your laptop to your devserver.
-2. `cd fbcode/executorch/docs/`
-3. `buck2 run //executorch/docs:sphinx-build -- -M html source_py sphinxbuild_py`
+1. `cd executorch/docs/`
+2. `sphinx-build -M html source_py sphinxbuild_py`
 to build Sphinx and generate APIs any packages.
-4. `python3 -m http.server 8000 --directory sphinxbuild_py/html` to view your HTML
+3. `python3 -m http.server 8000 --directory sphinxbuild_py/html` to view your HTML
 files at `localhost:8000`.
 
 ### C++ APIs
@@ -203,35 +211,9 @@ structure can be found in `source_cpp/`.
 A diagram of how the files work together:
 ![image](cpp_docs.png)
 
-To view your changes on the executorch website, you can follow the same steps
-listed in the "General Documentation" section.
+To view the C++ API documentation locally, run:
 
-To view just the auto-generated pages:
-1. `ssh -L 8000:localhost:8000 <devserver>` from your laptop to your devserver.
-2. `cd fbcode/executorch/docs/`
-3. `buck2 run //executorch/docs:sphinx-build -- -M html source_cpp sphinxbuild_cpp`
-to build Sphinx and generate APIs any packages.
-4. `python3 -m http.server 8000 --directory sphinxbuild_cpp/html` to view your HTML
+1. `cd executorch/docs/`
+2. `sphinx-build -M html source_cpp sphinxbuild_cpp`
+3. `python3 -m http.server 8000 --directory sphinxbuild_py/html` to view your HTML
 files at `localhost:8000`.
-
-## Integration with Static Docs
-
-To integrate the generated files with Static Docs, we created a `gen_docs.sh`
-script under `website/`. The script contains commands to generate HTML files
-into the `website/static/` folder, which are linked on the navigation bar (which
-is configured in `website/docusaurus.config.js`).
-
-To have Static Docs run this script each time the website is built, we inserted
-a command to run this script into `website/package.json` under `scripts/build`:
-```
-{
-  ...
-  "scripts": {
-    "docusaurus": "docusaurus",
-    "start": "docusaurus start",
-    "build": "./gen_docs.sh && docusaurus build",
-    ...
-  }
-  ...
-}
-```

docs/source/_static/example.py

@@ -0,0 +1,27 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""
+A sample python file
+"""
+
+
+class Person:
+    def __init__(self, name, age):
+        self.name = name
+        self.age = age
+
+    # start
+
+    def introduce(self):
+        print("Hello, my name is", self.name)
+        print("I am", self.age, "years old")
+
+    # end
+
+
+person = Person("Alice", 25)
+person.introduce()

docs/source/_static/img/s_custom_variables_extension.png

@@ -49,6 +49,7 @@
     "sphinx.ext.autosummary",
     "sphinx.ext.coverage",
     "sphinx.ext.doctest",
+    "sphinx_copybutton",
     "sphinx.ext.napoleon",
     "sphinx.ext.todo",
     "sphinx.ext.viewcode",
@@ -57,6 +58,7 @@
     "myst_parser",
     "sphinx_design",
     "sphinx_gallery.gen_gallery",
+    "executorch_custom_versions",
 ]
 
 templates_path = ["_templates"]
@@ -128,7 +130,7 @@
     "torch": ("https://pytorch.org/docs/stable/", None),
 }
 
-# Custom directives defintions to create cards on main TorchServe page
+# Custom directives defintions to create cards on main landing page
 
 from custom_directives import (
     CustomCardEnd,