Skip to content

Commit 45fbe73

Browse files
fabianvfjmrodri
fabianvf
authored and
jmrodri
committed
Add Ansible Operator testing guide (#922)
* add testing guide * add missing links
1 parent 3a83a0c commit 45fbe73

File tree

1 file changed

+222
-0
lines changed

1 file changed

+222
-0
lines changed

‎doc/ansible/dev/testing_guide.md

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
# Testing Ansible Operators with Molecule
2+
3+
## Getting started
4+
5+
### Requirements
6+
To begin, you sould have:
7+
- The latest version of the [operator-sdk](https://github.com/operator-framework/operator-sdk) installed.
8+
- Docker installed and running
9+
- [Molecule](https://github.com/ansible/molecule) >= v2.20 (currently that will require installation from source, `pip install git+https://github.com/ansible/molecule.git`)
10+
- [Ansible](https://github.com/ansible/ansible) >= v2.7
11+
- [jmespath](https://pypi.org/project/jmespath/)
12+
- [The OpenShift Python client](https://github.com/openshift/openshift-restclient-python) >= v0.8
13+
- An initialized Ansible Operator project, with the molecule directory present. If you initialized a project with a previous
14+
version of operator-sdk, you can generate a new dummy project and copy in the `molecule` directory. Just be sure
15+
to generate the dummy project with the same `api-version` and `kind`, or some of the generated files will not work
16+
without modification. Your top-level project structure should look like this:
17+
```
18+
.
19+
├── build
20+
├── deploy
21+
├── molecule
22+
├── roles
23+
├── playbook.yml (optional)
24+
└── watches.yaml
25+
```
26+
27+
### Molecule scenarios
28+
If you look into the `molecule` directory, you will see three directories (`default`, `test-local`, `test-cluster`).
29+
Each of those directories contains a set of files that together make up what is known as a molecule *scenario*.
30+
31+
Our molecule scenarios have the following basic structure:
32+
33+
```
34+
.
35+
├── molecule.yml
36+
├── prepare.yml
37+
└── playbook.yml
38+
```
39+
40+
`molecule.yml` is a configuration file for molecule. It defines what driver to use to stand up an environment and the associated configuration, linting rules, and a variety of other configuration options. For full documentation on the options available here, see the [molecule configuration documentation](https://molecule.readthedocs.io/en/latest/configuration.html)
41+
42+
`prepare.yml` is an Ansible playbook that is run once during the set up of a scenario. You
43+
can put any arbitrary Ansible in this playbook. It is used for one-time configuration
44+
of your test environment, for example, creating the cluster-wide `CustomResourceDefinition`
45+
that your Operator will watch.
46+
47+
`playbook.yml` is an Ansible playbook that contains your core logic for the scenario. In a
48+
normal molecule scenario, this would import and run the associated role. For Ansible
49+
Operator, we mostly use this to create the Kubernetes resources and then execute a
50+
series of asserts that verify your cluster state.
51+
52+
Below we will walk through the structure and function of each file for each scenario.
53+
54+
#### default
55+
The default scenario is intended for use during the development of your Ansible role or playbook, and will run it
56+
outside of the context of an operator.
57+
You can run this scenario with
58+
`molecule test`
59+
or
60+
`molecule converge`. There is no corresponding `operator-sdk` command for this scenario.
61+
62+
The scenario has the following structure:
63+
64+
```
65+
molecule/default
66+
├── asserts.yml
67+
├── molecule.yml
68+
├── playbook.yml
69+
└── prepare.yml
70+
```
71+
72+
`asserts.yml` is an Ansible playbook contains Ansible assert tasks that will be run by all three scenarios.
73+
If you would like to write specific asserts for individual scenarios, you can instead remove the `asserts.yml`
74+
playbook import from that scenario's `playbook.yml`, or if you only want to add additional asserts, you can
75+
create a new playbook in that scenario and import it at the bottom of that scenario's `playbook.yml`.
76+
77+
`molecule.yml` for this scenario tells molecule to use the docker driver to bring up a Kubernetes-in-Docker container,
78+
and exposes the API on the host's port 9443. It also specifies a few inventory and environment
79+
variables which are used in `prepare.yml` and `playbook.yml`.
80+
81+
`prepare.yml` ensures that a kubeconfig properly configured to connect to the Kubernetes-in-Docker cluster exists and is mapped to the proper port, and also waits for the Kubernetes API to become
82+
available before allowing testing to begin.
83+
84+
`playbook.yml` only imports your role or playbook and then imports the `asserts.yml` playbook.
85+
86+
#### test-local
87+
The test-local scenario is a more full integration test of your operator. It brings up a Kubernetes-in-docker cluster, builds your Operator, deploys it
88+
into the cluster, and then creates an instance of your CustomResource and runs your assertions to make sure the Operator responded properly. You can run
89+
this scenario with
90+
`molecule test -s local`, which is equivalent to `operator-sdk test local`, or with `molecule converge -s test-local`, which will leave the environment up
91+
afterward.
92+
93+
The scenario has the following structure:
94+
95+
```
96+
molecule/test-local
97+
├── molecule.yml
98+
├── playbook.yml
99+
└── prepare.yml
100+
```
101+
102+
`molecule.yml` for this scenario tells molecule to use the docker driver to bring up a Kubernetes-in-Docker container with the project root mounted,
103+
and exposes the API on the host's port 10443. It also specifies a few inventory and environment
104+
variables which are used in `prepare.yml` and `playbook.yml`. It is very similar to the default scenario's configuration.
105+
106+
`prepare.yml` first runs the `prepare.yml` from the default scenario to ensure the kubeconfig is present and the API is up. It then creates the CustomResourceDefinition, namespace, and RBAC
107+
resources specified in the `deploy/` directory.
108+
109+
`playbook.yml` is the most complicated file in this project. First, it connects to your
110+
Kubernetes-in-Docker container, and uses your mounted project root to build your Operator.
111+
This makes your Operator available to the cluster without needing to push it to an external
112+
registry. Next, it will ensure that a fresh deployment of your Operator is present in the
113+
cluster, and once there is it will create an instance of your Custom Resource
114+
(specified in `deploy/crds/`). It will then wait for the CustomResource to report a successful
115+
run, and once it has, will import the `asserts.yml` from the default scenario.
116+
117+
#### test-cluster
118+
The test-cluster scenario is intended as a full integration test against
119+
an existing Kubernetes cluster, and assumes that the cluster is already available, the dependent resources from the `deploy/` directory
120+
are created, the operator image is built with `--enable-tests`, and that the image is available in a container registry. It connects
121+
to the existing Kubernetes cluster and deploys the test Operator, creates a Custom Resource, and runs your asserts. You shouldn't
122+
call this scenario directly, rather you should build your operator with the `--enable-tests` flag, in which case a new entrypoint will
123+
be added that runs this scenario when the container starts up. It is recommended that you only interact with this scenario through
124+
`operator-sdk test cluster`.
125+
126+
The scenario has the following structure:
127+
128+
```
129+
molecule/test-cluster
130+
├── molecule.yml
131+
└── playbook.yml
132+
```
133+
`molecule.yml` for this scenario is very simple, as it assumes an environment is already
134+
present. It essentially is just specifying the metadata of the scenario, and telling molecule
135+
not to try and create or destroy anything when run.
136+
137+
`playbook.yml` is also pretty simple, compared to the previous scenarios. All it does is create
138+
an instance of your Custom Resource (specified in `deploy/crds`), and then import the `asserts.yml` from the `default` scenario.
139+
140+
#### converge vs test
141+
The two most common molecule commands for testing during development are `molecule test` and `molecule converge`.
142+
`molecule test` performs a full loop, bringing a cluster up, preparing it, running your tasks, and tearing it down.
143+
`molecule converge` is more useful for iterative development, as it leaves your environment up between runs. This
144+
can cause unexpected problems if you end up corrupting your environment during testing, but running `molecule destroy`
145+
will reset it.
146+
147+
148+
149+
## operator-sdk test commands
150+
151+
### test local
152+
153+
The `operator-sdk test local` command kicks off an end-to-end test of your Operator. It will bring up a [Kubernetes-in-Docker (kind)](https://github.com/bsycorp/kind) cluster, builds your Operator
154+
image and make it available to that cluster, create all the required resources from the `deploy/` directory, create an instance of your
155+
Custom Resource (specified in the `deploy/crds` directory), and then verify that the Operator has responded appropriately by running
156+
the asserts from `molecule/default/asserts.yml`.
157+
158+
159+
### test cluster
160+
161+
The `operator-sdk test cluster` command does much less than the `test local` command. It is intended as a full integration test against
162+
an existing Kubernetes cluster, and assumes that the cluster is already available, the dependent resources from the `deploy/` directory
163+
are created, the operator image is built with `--enable-tests`, and that the image is available in a container registry. When you run the command, it will connect
164+
to the existing Kubernetes cluster and deploy the test Operator, create a Custom Resource, and run the asserts in `molecule/default/asserts.yml`.
165+
166+
## Writing tests
167+
168+
### Adding a task
169+
The default operator that is generated by `operator-sdk new` doesn't do anything, so first we will need to add an
170+
Ansible task so that the Operator does something we can verify. For this example, we will create a simple ConfigMap
171+
with a single key.
172+
We'll be adding the task to `roles/example/tasks/main.yml`, which should now look like this:
173+
174+
```
175+
---
176+
# tasks file for exampleapp
177+
- name: create Example configmap
178+
k8s:
179+
definition:
180+
apiVersion: v1
181+
kind: ConfigMap
182+
metadata:
183+
name: 'test-data'
184+
namespace: '{{ meta.namespace }}'
185+
data:
186+
hello: world
187+
```
188+
189+
190+
191+
### Adding a test
192+
193+
Now that our Operator actually does some work, we can add a corresponding assert to `molecule/default/asserts.yml`.
194+
We'll also add a debug message so that we can see what the ConfigMap looks like.
195+
The file should now look like this:
196+
197+
```
198+
---
199+
200+
- name: Verify
201+
hosts: localhost
202+
connection: local
203+
vars:
204+
ansible_python_interpreter: '{{ ansible_playbook_python }}'
205+
tasks:
206+
- debug: var=cm
207+
vars:
208+
cm: '{{ lookup("k8s", api_version="v1", kind="ConfigMap", namespace=namespace, resource_name="test-data") }}'
209+
- assert:
210+
that: cm.data.hello == 'world'
211+
vars:
212+
cm: '{{ lookup("k8s", api_version="v1", kind="ConfigMap", namespace=namespace, resource_name="test-data") }}'
213+
```
214+
215+
Now that we have a functional Operator, and an assertion of its behavior, we can verify that everything is working
216+
by running `operator-sdk test local`.
217+
218+
#### The Ansible `assert` and `fail` modules
219+
These modules are handy for adding assertions and failure conditions to your Ansible Operator tests:
220+
221+
- [assert](https://docs.ansible.com/ansible/latest/modules/assert_module.html)
222+
- [fail](https://docs.ansible.com/ansible/latest/modules/fail_module.html)

0 commit comments

Comments
 (0)