[libc][docgen] regen docgen via cmake

[nickdesaulniers]

Now, ninja docs-libc-html will re-run docgen.

Previously, we would run docgen offline, and commit the result.

Now we no longer need to do that; docgen is invoked from the
dependencies of the docs-libc-html target on demand. This
commit removes the dynamically generated .rst files (keeping
the static ones that haven't been converted to docgen), and
fixes up some mistakes I failed to cleanup recently since I
didn't have such automation in place to catch such bugs.

[llvmbot]

@llvm/pr-subscribers-libc

Author: Nick Desaulniers (nickdesaulniers)

Changes

---

Full diff: https://github.com/llvm/llvm-project/pull/119628.diff

1 Files Affected:

• (modified) libc/docs/CMakeLists.txt (+9)

diff --git a/libc/docs/CMakeLists.txt b/libc/docs/CMakeLists.txt
index be09423d38e8e3..c8e9bcd29f612c 100644
--- a/libc/docs/CMakeLists.txt
+++ b/libc/docs/CMakeLists.txt
@@ -4,6 +4,15 @@ include(AddSphinxTarget)
 if (SPHINX_FOUND)
   if (${SPHINX_OUTPUT_HTML})
     add_sphinx_target(html libc)
+
+    list(APPEND docgen_list string strings)
+
+    foreach(stem IN LISTS docgen_list)
+      add_custom_target(${stem}_rst
+        COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/../utils/docgen/docgen.py ${stem}.h >
+                ${CMAKE_CURRENT_SOURCE_DIR}/headers/${stem}.rst)
+      add_dependencies(docs-libc-html ${stem}_rst)
+    endforeach()
   endif()
 endif()
 endif()

[nickdesaulniers]

I wonder why these are being highlighted? Are they keywords in CMake? Perhaps I should make this a ; separated string?

[nickdesaulniers]

So there's probably a better way to do this. IIUC, it's going to run the command every time, which isn't necessary. We should only need to re-run the command in the input file (libc/utils/docgen/{stem}.yaml) is modified. Not sure yet how to express that in cmake...

[nickdesaulniers]

Maybe the SOURCES argument?
https://cmake.org/cmake/help/latest/command/add_custom_target.html

[nickdesaulniers]

Seems like NEITHER SOURCES or DEPENDS prevents rebuilds upon reinvocations of ninja docs-libc-html...oh well

[nickdesaulniers]

ninja docs-libc-html -d explain prints useful info to debug that, but I don't plan to resolve that in this PR. Nice to have; not required.

[boomanaiden154]

For the most part, we can't have any build targets modify the source directory. If we want to have a build command rerun docgen automatically, it should be a separate target and not under docs-libc-html.

The better way to do this would be to do something similar to #113739. Convert everything in the libc/docs directory to templates and as part of libc-docs-html, run docgen writing all the outputs to a folder in the build directory that we then run sphinx over.

[nickdesaulniers]

So it looks like clang itself copies its rst files into the build dir. I can emulate the approach I think.

$ ninja copy-clang-rst-docs

[nickdesaulniers]

ok, I have that working. One thing I think would be nice to add; I suspect there's a way to say that docgen depends on the yaml files. If those haven't changed, then docgen doesn't need to be rerun. Looking at these docs https://cmake.org/cmake/help/latest/command/add_custom_target.html, I'm not sure if that's DEPENDS or SOURCES.

Looks like I need to rebase this pr...

[boomanaiden154]

LGTM with the latest approach.