Skip to content

Commit f31341c

Browse files
committed
[docs] Expand HowToAddABuilder with guidance on testing locally
Once <llvm/llvm-zorg#289> and <llvm/llvm-zorg#293> land, it's quite reasonable to ask people to test their builder configurations locally. This patch adds documentation on how to do so. I think review at this stage is useful, but of course if there's more review feedback on <llvm/llvm-zorg#289> it's possible some details may change. This won't be committed until those llvm-zorg PRs land of course.
1 parent ba60f6d commit f31341c

File tree

1 file changed

+68
-0
lines changed

1 file changed

+68
-0
lines changed

llvm/docs/HowToAddABuilder.rst

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,74 @@ Here are the steps you can follow to do so:
179179
buildbot.tac file to change the port number from 9994 to 9990 and start it
180180
again.
181181

182+
Testing a Builder Config Locally
183+
================================
184+
185+
It's possible to test a builder running against a local version of LLVM's
186+
buildmaster configuration. This can be helpful to allow quickly identifying
187+
and iterating over fixes to any issues in either the changes that introduce
188+
the new builder, or the machine configuration for your worker (preinstalled
189+
packages etc). A buildmaster launched in this "local testing" mode will bind
190+
only to local interfaces, use SQLite as the database, use a fixed password for
191+
workers, and disable things like GitHub authentication.
192+
193+
* Within a checkout of `llvm-zorg <https://github.com/llvm/llvm-zorg>`_,
194+
create and activate a Python `venv
195+
<https://docs.python.org/3/library/venv.html>`_ and install the necessary
196+
dependencies.
197+
198+
.. code-block:: bash
199+
200+
python -m venv bbenv
201+
source bbenv/bin/activate
202+
pip install buildbot{,-console-view,-grid-view,-waterfall-view,-worker,-www}==3.11.7 urllib3
203+
204+
* Initialise the necessary buildmaster files, link to the configuration in
205+
``llvm-zorg`` and run a litmus check (run in the directory of your choice):
206+
207+
.. code-block:: bash
208+
209+
buildbot create-master llvm-testbbmaster
210+
cd llvm-testbbmaster
211+
ln -s /path/to/checkout/of/llvm-zorg/buildbot/osuosl/master/master.cfg .
212+
ln -s /path/to/checkout/of/llvm-zorg/buildbot/osuosl/master/config/ .
213+
ln -s /path/to/checkout/of/llvm-zorg/zorg/ .
214+
BUILDMASTER_TEST=1 buildbot checkconfig
215+
216+
* Start the buildmaster using the command below. After a few seconds to
217+
startup, you should be able to open the web UI at ``http://localhost:8011``.
218+
If there are any errors or this isn't working, be sure to check
219+
``twistd.log`` for more information.
220+
221+
.. code-block:: bash
222+
223+
BUILDMASTER_TEST=1 buildbot start --nodaemon .
224+
225+
* With the above in place, you can now create and start a buildbot worker.
226+
Ensure you pick the correct name for the worker attached to the build
227+
configuration you want to test in
228+
``buildbot/osuosl/master/config/builders.py``. After doing the below, either
229+
wait until the poller sets off a build, or you can force a build to start in
230+
the web UI (which is also the best place to review the build results).
231+
232+
.. code-block:: bash
233+
234+
buildbot-worker create-worker <buildbot-worker-root-directory> \
235+
localhost:9990 \
236+
<buildbot-worker-name> \
237+
test
238+
buildbot-worker start --nodaemon <buildbot-worker-root-directory>
239+
240+
This local testing configuration defaults to binding only to the loopback
241+
interface for security reasons. If you want to run the test worker on a
242+
different machine, or to run the buildmaster on a remote server, ssh port
243+
forwarding can be used to make connection possible. For instance, if running
244+
the buildmaster on a remote server the following command will suffice to make
245+
the web UI accessible via ``http://localhost:8011`` and make it possible for a
246+
local worker to connect to the remote buildmaster by connecting to
247+
``localhost:9900``: ``ssh -N -L 8011:localhost:8011 -L 9990:localhost:9990
248+
username@server_address``.
249+
182250
Best Practices for Configuring a Fast Builder
183251
=============================================
184252

0 commit comments

Comments
 (0)