Skip to content

Commit d3a37c1

Browse files
fabianvffabianvf
authored
Add APB migration guide (#1762)
* Add APB migration guide * respond to code review
1 parent 66e78cc commit d3a37c1

File tree

1 file changed

+397
-0
lines changed

1 file changed

+397
-0
lines changed

‎doc/ansible/dev/apb_migration_guide.md

Lines changed: 397 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,397 @@
1+
# Migrating an Ansible Playbook Bundle (APB) to an Ansible-based Operator
2+
3+
This guide will walk you through migrating your legacy Ansible Playbook Bundles (APB)
4+
to a more modern Kubernetes Operator architecture. We will cover how to prepare a new
5+
Ansible-based Operator, cover the changes that you will need to make to the Ansible
6+
logic in your existing APB, and at the end you will have a functional Operator.
7+
8+
## Directory structure and metadata
9+
### Generate Operator resources
10+
11+
Before we begin our migration, we need to first generate an Ansible operator using the operator-sdk tool. Run the following command:
12+
13+
```
14+
operator-sdk new <name> --type=ansible --api-version=<group>/<version> --kind=<kind>`
15+
```
16+
where:
17+
18+
* `<name>` is the name for your operator, so if for example you have a `memcached-apb`, you would probably use `memcached-operator`
19+
* `<group>` is the API group for your Kubernetes Custom Resource Definition. For example, if I own the domain `example.com`, I might use the group `apps.example.com`.
20+
* `<version>` is the API version for your Kubernetes Custom Resource Definition. `v1alpha1` is a common starting value, with `v1beta1` implying a fair amount of API stability and `v1` implying no breaking API changes at all.
21+
* `<kind>` is the kind of your resource. For example, if you are creating a `memcached-operator`, your `kind` would likely be `Memcached`
22+
23+
So for the example `memcached-operator`, the command would be:
24+
25+
```
26+
operator-sdk new memcached-operator --type=ansible --api-version=apps.example.com/v1alpha1 --kind=Memcached
27+
```
28+
29+
30+
Once this is generated, take the `build`, `deploy`, and `molecule` directories, as well as the `watches.yaml` and copy them into your APB directory.
31+
32+
### Dockerfile
33+
You now have two Dockerfiles, your original APB `Dockerfile` at the top-level, and a `build/Dockerfile` for your operator.
34+
35+
In your `build/Dockerfile`, ensure that your playbooks and roles are being copied to `${HOME}/roles` and `${HOME}/playbooks`, and that your `watches.yaml` is being copied to `${HOME}/watches.yaml`.
36+
37+
If you are installing any additional dependencies, ensure that those are reflected in your `build/Dockerfile` as well.
38+
39+
As a sample, your `build/Dockerfile` will probably look something like this:
40+
41+
```
42+
FROM quay.io/operator-framework/ansible-operator:v0.9.0
43+
44+
COPY watches.yaml ${HOME}/watches.yaml
45+
46+
COPY roles/ ${HOME}/roles/
47+
COPY playbooks/ ${HOME}/playbooks/
48+
```
49+
50+
Once this is done you may remove your original APB `Dockerfile`.
51+
52+
### watches.yaml
53+
In the `watches.yaml`, ensure the playbook for your `kind` points to your `provision.yml` playbook in the container (likely location for that will be `/opt/ansible/playbooks/provision.yml`).
54+
55+
Next, add a finalizer block with a name of: `finalizer.<name>.<group>/<version>`, and set the playbook to point to your `deprovision.yml` in the container (likely location for that will be `/opt/ansible/playbooks/deprovision.yml`). For the memcached-operator we generated above, the watches.yaml would look like this:
56+
57+
```yaml
58+
---
59+
- version: v1alpha1
60+
group: apps.example.com
61+
kind: Memcached
62+
playbook: /opt/ansible/playbooks/provision.yml
63+
finalizer:
64+
name: finalizer.memcached.apps.example.com/v1alpha1
65+
playbook: /opt/ansible/playbooks/deprovision.yml
66+
```
67+
68+
#### Binding
69+
If you have a `bind` playbook, add a new entry to your `watches.yaml` (you can copy paste the existing entry).
70+
71+
The `version` and `group`, will remain unchanged, but update the `kind` with a `Binding` suffix.
72+
73+
For example, if you have a resource with `kind: Memcached`, the kind of your new entry will be `MemcachedBinding`.
74+
75+
The playbook for this entry should map to your `bind` playbook, (likely location `/opt/ansible/playbooks/bind.yml`), and if you have an `unbind` playbook then set the playbook for the finalizer to point to it (likely location `/opt/ansible/playbooks/unbind.yml`). If you don't have an `unbind` playbook, remove the finalizer block for your `Binding` resource.
76+
77+
For an APB with both `bind` and `unbind` playbooks, the `watches.yaml` would end up looking like this:
78+
79+
```yaml
80+
---
81+
- version: v1alpha1
82+
group: apps.example.com
83+
kind: Memcached
84+
playbook: /opt/ansible/playbooks/provision.yml
85+
finalizer:
86+
name: finalizer.memcached.apps.example.com/v1alpha1
87+
playbook: /opt/ansible/playbooks/deprovision.yml
88+
- version: v1alpha1
89+
group: apps.example.com
90+
kind: MemcachedBinding
91+
playbook: /opt/ansible/playbooks/bind.yml
92+
finalizer:
93+
name: finalizer.memcachedbinding.apps.example.com/v1alpha1
94+
playbook: /opt/ansible/playbooks/unbind.yml
95+
```
96+
97+
98+
You will also need to run `operator-sdk add crd --api-version=<group>/<version> --kind=<kind>` to generate a new CRD and example in `deploy/crds`.
99+
100+
### deploy/crds/
101+
Now that you have all your CRDs created, you can generate the OpenAPI spec for them using your `apb.yml`.
102+
103+
The `convert.py` script included at the bottom of this document can handle the conversion to the OpenAPI spec, at which point you can copy paste everything from `validation:` on into your primary CRD (for the regular `parameters`), or into your `Binding` CRD (for `bind_parameters`).
104+
105+
You may notice that the OpenAPI validation uses `camelCase` parameters, while your `apb.yml` and Ansible playbooks probably assume `snake_case` variables. `Ansible Operator` will automatically convert the `camelCase` parameters from the Kubernetes resource into `snake_case` before passing them to your playbook, so this should not require any change on your part.
106+
107+
## Ansible logic
108+
There will be some changes required to your Ansible playbooks/roles/tasks.
109+
110+
### Idempotence
111+
112+
One major conceptual difference between the APB model and the Operator model, is that APBs are meant to run `provision` once, while operators constantly reconcile to ensure that the state of the cluster matches the state that the user requested.
113+
114+
This means that you will need to ensure that your playbooks are idempotent, and can be run repeatedly with the same parameters without causing an error.
115+
116+
### Service Bundle contract and meta variables
117+
Ansible Operator does not respect the Service Bundle contract that exists between APBs and the Ansible Service Broker. The following variables will not be passed in by the Ansible Operator:
118+
119+
- `cluster`: Operators ideally work on both Kubernetes and OpenShift, so any uses of openshift-specific resources should handle errors and fallback
120+
- `_apb_plan_id`: Operators have no concept of a plan
121+
- `_apb_service_class_id`: This concept is replaced by the group/version/kind specified in your CRD
122+
- `_apb_service_instance_id`: This concept is replaced by `meta.name`, the name of the Custom Resource created by the user requesting the action.
123+
- `_apb_last_requesting_user`: There is no analogue to this.
124+
- `_apb_provision_creds`: There is no analogue to this.
125+
- `_apb_service_binding_id`: This concept is replaced by the `meta.name` of a `<kind>Binding` resource
126+
- `namespace`: This is accessible via the `meta.namespace` variable
127+
128+
Instead, the Ansible Operator will pass in a field called `meta`, which contains the `name` and `namespace` of the Custom Resource that the user created.
129+
130+
131+
### asb_encode_binding
132+
The `asb_encode_binding` module will not be present in the Ansible Operator base image. In order to save credentials after a successful provision, you will need to create a `secret` in Kubernetes, and update the status of your custom resource so that people can find it. For example, if we have the following Custom Resource group/version/kind:
133+
134+
```yaml
135+
version: v1alpha1
136+
group: apps.example.com
137+
kind: PostgreSQL
138+
```
139+
140+
the following task:
141+
142+
```yaml
143+
- name: encode bind credentials
144+
asb_encode_binding:
145+
fields:
146+
DB_TYPE: postgres
147+
DB_HOST: "{{ app_name }}"
148+
DB_PORT: "5432"
149+
DB_USER: "{{ postgresql_user }}"
150+
DB_PASSWORD: "{{ postgresql_password }}"
151+
DB_NAME: "{{ postgresql_database }}"
152+
```
153+
154+
would become:
155+
156+
```yaml
157+
- name: Create bind credential secret
158+
k8s:
159+
definition:
160+
apiVersion: v1
161+
kind: Secret
162+
metadata:
163+
name: '{{ meta.name }}-credentials'
164+
namespace: '{{ meta.namespace }}'
165+
data:
166+
DB_TYPE: "{{ 'postgres' | b64encode }}"
167+
DB_HOST: "{{ app_name | b64encode }}"
168+
DB_PORT: "{{ '5432' | b64encode }}"
169+
DB_USER: "{{ postgresql_user | b64encode }}"
170+
DB_PASSWORD: "{{ postgresql_password | b64encode }}"
171+
DB_NAME: "{{ postgresql_database | b64encode }}"
172+
173+
- name: Attach secret to CR status
174+
k8s_status:
175+
api_version: apps.example.com/v1alpha1
176+
kind: PostgreSQL
177+
name: '{{ meta.name }}'
178+
namespace: '{{ meta.namespace }}'
179+
status:
180+
bind_credentials_secret: '{{ meta.name }}-credentials'
181+
```
182+
183+
### ansible_kubernetes_modules
184+
The `ansible_kubernetes_modules` role and the generated modules are now deprecated.
185+
The `k8s` module was added in Ansible 2.6 and is the supported way to interact with Kubernetes from Ansible.
186+
The `k8s` module takes normal kubernetes manifests, so if you currently rely on the old generated modules some refactoring will be required.
187+
188+
189+
# convert.py
190+
The `convert.py` script should be run from inside the APB directory, next to the `apb.yml`
191+
```python
192+
#!/usr/bin/env python
193+
194+
import yaml
195+
196+
197+
def extract_params(all_params):
198+
properties = {}
199+
required = set()
200+
for param in all_params:
201+
name = param['name']
202+
name_parts = name.split('_')
203+
camel_name = name_parts[0] + ''.join([x.title() for x in name_parts[1:]])
204+
if param.get('required') is True:
205+
if camel_name not in properties:
206+
required.add(camel_name)
207+
elif camel_name in required and param.get('required') is False:
208+
required.remove(camel_name)
209+
properties[camel_name] = {
210+
"type": param["type"],
211+
"description": param.get("description", param.get("title", ""))
212+
}
213+
214+
return {
215+
"validation": {"openAPIv3Schema": {
216+
"properties": {
217+
"spec": {
218+
"required": list(required),
219+
"properties": properties
220+
}
221+
}
222+
}}
223+
}
224+
225+
226+
def main():
227+
with open('apb.yml', 'r') as f:
228+
apb_meta = yaml.safe_load(f.read())
229+
230+
for field in ['parameters', 'bind_parameters']:
231+
print("Converting {0} to OpenAPI spec".format(field))
232+
print(yaml.dump({field: extract_params([
233+
param for x in apb_meta['plans'] for param in x.get(field, [])
234+
])}, default_flow_style=False))
235+
236+
237+
if __name__ == '__main__':
238+
main()
239+
```
240+
241+
It will parse the `parameters` and `bind_parameters` from your `apb.yml`, and output OpenAPI
242+
validation blocks that can be included in your `CustomResourceDefinition`s. For example, when
243+
run through the `convert.py` script, the following `apb.yml`:
244+
245+
```yaml
246+
version: 1.0.0
247+
name: keycloak-apb
248+
description: Keycloak - Open Source Identity and Access Management
249+
bindable: True
250+
async: optional
251+
tags:
252+
- sso
253+
- keycloak
254+
metadata:
255+
displayName: Keycloak (APB)
256+
imageUrl: "https://github.com/ansibleplaybookbundle/keycloak-apb/raw/master/docs/imgs/keycloak_ico.png"
257+
documentationUrl: "http://www.keycloak.org/documentation.html"
258+
providerDisplayName: "Red Hat, Inc."
259+
dependencies:
260+
- 'docker.io/jboss/keycloak-openshift:3.4.3.Final'
261+
- 'centos/postgresql-95-centos7:9.5'
262+
serviceName: keycloak
263+
plans:
264+
- name: ephemeral
265+
description: Deploy keycloak without persistence
266+
free: True
267+
metadata:
268+
displayName: Keycloak ephemeral
269+
parameters:
270+
- name: admin_username
271+
required: True
272+
default: admin
273+
type: string
274+
title: Keycloak admin username
275+
- name: admin_password
276+
required: True
277+
type: string
278+
display_type: password
279+
title: Keycloak admin password
280+
- name: apb_keycloak_uri
281+
required: False
282+
type: string
283+
title: Keycloak URL
284+
description: URL where the applications should redirect to for authentication. Must be resolvable by the browser and pods. Leave empty to use the host generated by the route
285+
- name: keycloak_users
286+
required: False
287+
type: string
288+
display_type: textarea
289+
title: Users
290+
description: JSON defining the users to add to the realm and their memberships
291+
- name: keycloak_roles
292+
required: False
293+
type: string
294+
display_type: textarea
295+
title: Roles
296+
description: JSON defining the roles to add to the realm
297+
bind_parameters:
298+
- name: service_name
299+
display_group: Provision
300+
required: True
301+
title: Name of the service to bind
302+
type: string
303+
- name: redirect_uris
304+
display_group: Provision
305+
required: True
306+
title: Redirect URIs
307+
description: Valid Redirect URIs a browser can redirect to after a successful login/logout. Simple wildcards are allowed. e.g. https://myservice-myproject.apps.example.com/*
308+
type: string
309+
- name: web_origins
310+
display_group: Provision
311+
title: Web Origins
312+
description: Web Origins to allow CORS
313+
type: string
314+
- name: sso_url_name
315+
default: SSO_URL
316+
display_group: Binding
317+
title: Keycloak URL Variable name
318+
description: How the application will refer to the Keycloak URL
319+
type: string
320+
- name: sso_realm_name
321+
default: SSO_REALM
322+
display_group: Binding
323+
title: Keycloak Realm Variable name
324+
description: How the application will refer to the Keycloak Realm
325+
type: string
326+
- name: sso_client_name
327+
default: SSO_CLIENT
328+
display_group: Binding
329+
title: Keycloak Client Variable name
330+
description: How the application will refer to the Keycloak Client name
331+
type: string
332+
```
333+
334+
will produce this output:
335+
336+
```yaml
337+
Converting parameters to OpenAPI spec
338+
parameters:
339+
validation:
340+
openAPIv3Schema:
341+
properties:
342+
spec:
343+
properties:
344+
adminPassword:
345+
description: Keycloak admin password
346+
type: string
347+
adminUsername:
348+
description: Keycloak admin username
349+
type: string
350+
apbKeycloakUri:
351+
description: URL where the applications should redirect to for authentication.
352+
Must be resolvable by the browser and pods. Leave empty to use the
353+
host generated by the route
354+
type: string
355+
keycloakRoles:
356+
description: JSON defining the roles to add to the realm
357+
type: string
358+
keycloakUsers:
359+
description: JSON defining the users to add to the realm and their memberships
360+
type: string
361+
required:
362+
- adminUsername
363+
- adminPassword
364+
365+
Converting bind_parameters to OpenAPI spec
366+
bind_parameters:
367+
validation:
368+
openAPIv3Schema:
369+
properties:
370+
spec:
371+
properties:
372+
redirectUris:
373+
description: Valid Redirect URIs a browser can redirect to after a successful
374+
login/logout. Simple wildcards are allowed. e.g. https://myservice-myproject.apps.example.com/*
375+
type: string
376+
serviceName:
377+
description: Name of the service to bind
378+
type: string
379+
ssoClientName:
380+
description: How the application will refer to the Keycloak Client name
381+
type: string
382+
ssoRealmName:
383+
description: How the application will refer to the Keycloak Realm
384+
type: string
385+
ssoUrlName:
386+
description: How the application will refer to the Keycloak URL
387+
type: string
388+
webOrigins:
389+
description: Web Origins to allow CORS
390+
type: string
391+
required:
392+
- serviceName
393+
- redirectUris
394+
```
395+
396+
The block beneath `parameters` would be put into the `Keycloak` CRD, and the block beneath `bind_parameters` would be put in the `KeycloakBinding` CRD.
397+

0 commit comments

Comments
 (0)