Skip to content

Commit 962bd3c

Browse files
padamstxpadamstx
authored
docs: add update_service.md (#193)
Signed-off-by: Phil Adams <[email protected]>
1 parent edb0719 commit 962bd3c

File tree

2 files changed

+287
-0
lines changed

2 files changed

+287
-0
lines changed

‎CONTRIBUTING.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,7 @@ Before that, please search for similar issues. It's possible that someone has al
1111
# General Information
1212
For general guidance on contributing to this project, please see
1313
[this link](https://github.com/IBM/ibm-cloud-sdk-common/blob/main/CONTRIBUTING_python.md)
14+
15+
# Updating an existing service within the SDK
16+
For instructions on updating an existing service within the SDK, please see
17+
[this link](update_service.md)

‎update_service.md

Lines changed: 283 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,283 @@
1+
# How to update a service
2+
This document describes the steps needed to update a service contained in this SDK project.
3+
4+
## Table of Contents
5+
<!--
6+
The TOC below is generated using the `markdown-toc` node package.
7+
8+
https://github.com/jonschlinkert/markdown-toc
9+
10+
You should regenerate the TOC after making changes to this file.
11+
12+
markdown-toc -i update_service.md
13+
-->
14+
15+
<!-- toc -->
16+
17+
- [Overview](#overview)
18+
- [Prerequisites](#prerequisites)
19+
- [Initial project setup](#initial-project-setup)
20+
- [Steps to update a service](#steps-to-update-a-service)
21+
* [1. Validate the API definition](#1-validate-the-api-definition)
22+
* [2. Create feature branch](#2-create-feature-branch)
23+
* [3. Re-generate the SDK code](#3-re-generate-the-sdk-code)
24+
* [4. Inspect new generated SDK code](#4-inspect-new-generated-sdk-code)
25+
* [5. Re-format code](#5-re-format-code)
26+
* [6. Build/test](#6-buildtest)
27+
* [7. Modify integration tests and examples](#7-modify-integration-tests-and-examples)
28+
* [8. Open PR with your changes](#8-open-pr-with-your-changes)
29+
- [Appendix](#appendix)
30+
* [Running Integration Tests/Examples](#running-integration-testsexamples)
31+
* [Updating Integration Tests/Examples](#updating-integration-testsexamples)
32+
- [References](#references)
33+
34+
<!-- tocstop -->
35+
36+
## Overview
37+
It is a good practice to keep the SDK code for each service updated so that it is in sync
38+
with the most recent production version of its API definition.
39+
So, when a service's API definition is changed, the SDK code for the service should be updated
40+
(re-generated) in each SDK project in which it exists.
41+
This could be a change such as (a) editorial changes made to various descriptions, (b) the addition of
42+
a new parameter to an existing operation, or (c) the addition of one or more new operations.
43+
44+
## Prerequisites
45+
1. If you are an IBMer, make sure that your
46+
[Annual Open Source Training](https://w3.ibm.com/developer/docs/open-source/training/) is current.
47+
2. Make sure that your internal github.ibm.com id is [linked](https://gh-user-map.dal1a.cirrus.ibm.com/)
48+
to your external github.com id. The id linking step will also result in an invitation to join the
49+
`github.com/IBM` org. Accept that invitation.
50+
3. If you do not yet have "push" access to the SDK project, contact the project maintainer to request push access
51+
(you must be a member of the github.com/IBM org).
52+
4. Make sure that your installed version of Python is >= the minimum version supported by the SDK project.
53+
54+
## Initial project setup
55+
1. Clone/fork the repo. If you have push access (see above), you can clone the repo directly (no fork).
56+
Example:
57+
```sh
58+
git clone [email protected]:IBM/platform-services-python-sdk.git
59+
```
60+
2. If you do not have push access, then you'll need to first create a fork and then clone your fork in your
61+
local sandbox environment.
62+
Example:
63+
```sh
64+
git clone [email protected]:my-git-id/platform-services-python-sdk.git
65+
```
66+
3. Make sure that your local sandbox is in sync with the remote, setup a virtual environment (venv),
67+
and then build/test the project. If you're using a fork, you'd need to first make sure that your fork is in sync with the primary repo.
68+
Example:
69+
```sh
70+
# pull the latest from remote
71+
cd <project-root>
72+
git checkout main
73+
git pull
74+
75+
# create a virtual environment using the current python version, store in "./venv"
76+
python -m venv ./venv
77+
78+
# source in the resulting "activate" script to use the virtual environment
79+
. venv/bin/activate
80+
81+
# Install dependencies, run unit-tests and link-check the project
82+
make all
83+
```
84+
4. Make sure that the integration tests and working examples run clean for your service.
85+
See [Running Integration Tests/Examples](#running-integration-testsexamples) for details.
86+
87+
88+
Before proceeding to make any changes, make sure the above steps complete cleanly. This is your "baseline".
89+
90+
## Steps to update a service
91+
92+
### 1. Validate the API definition
93+
Prior to re-generating the SDK code for your service, be sure to validate the updated version of the API definition
94+
using the [IBM OpenAPI Validator](https://github.com/IBM/openapi-validator).
95+
Example:
96+
```sh
97+
lint-openapi example-service.yaml
98+
```
99+
This command will display a list of errors and warnings found in the API definition
100+
as well as a summary at the end.
101+
It's not required that you fix all errors and warnings before trying to use the SDK generator, but
102+
this step should identify any critical errors that will need to be fixed prior to the generation step.
103+
104+
Video: [Getting Started With The OpenAPI Validator](https://secure.video.ibm.com/channel/23887899/playlist/651457/video/131770428)
105+
106+
### 2. Create feature branch
107+
After validating the API definition, you're ready to generate new SDK code for your service.
108+
However, before you do that, you should probably create a new feature branch in which to deliver your updates:
109+
```sh
110+
cd <project-root>
111+
git checkout -b update-example-service
112+
```
113+
114+
### 3. Re-generate the SDK code
115+
Next, run the [IBM OpenAPI SDK Generator](https://github.ibm.com/CloudEngineering/openapi-sdkgen) to
116+
process your API definition and generate new service and unit test code for the service:
117+
```sh
118+
cd <project-root>
119+
120+
openapi-sdkgen.sh generate -g ibm-python -i example-service.json -o .
121+
```
122+
The generated service code is written to the SDK's primary module directory (e.g. ./ibm_platform_services/example_service_v1.py), and the
123+
unit tests are written to the "test/unit" directory.
124+
125+
Video: [Getting Started With The SDK Generator](https://secure.video.ibm.com/channel/23887899/playlist/651457/video/131770438)
126+
127+
### 4. Inspect new generated SDK code
128+
Next, it is recommended that you inspect the differences between the previous and new generated code to
129+
get an overall view of the changes caused by the re-generation step. The changes that you see in the
130+
generated SDK code should align with the API definition changes that have occurred since you last
131+
generated the SDK code.
132+
Example:
133+
```
134+
git diff # alternative: use the "source control" view within vscode
135+
```
136+
137+
### 5. Re-format code
138+
Next, we need to make the lint-checker happy, so build the "lint-fix" target:
139+
```sh
140+
cd <project-root>
141+
142+
make lint-fix
143+
```
144+
145+
If any lint-check problems remain (perhaps you made other manual changes to the code?), you'll need
146+
to fix those manually before proceeding.
147+
148+
### 6. Build/test
149+
Next, build and unit-test the project with the new generated SDK code:
150+
```sh
151+
cd <project-root>
152+
153+
make all
154+
```
155+
The unit tests and link-check should run clean. If not, then any failures should be diagnosed and resolved
156+
before proceeding.
157+
158+
### 7. Modify integration tests and examples
159+
After ensuring that your service's unit tests run clean, the next step would be to modify
160+
your service's integration tests and working examples code to reflect the updated version of
161+
your API definition. See [Updating Integration Tests/Examples](#updating-integration-testsexamples)
162+
for more information on this topic.
163+
164+
Even if no changes are needed (perhaps only very minor updates were made to the generated
165+
SDK code), at a mininum you should make sure that the integration tests and examples run clean after you
166+
re-generate the service and unit test code.
167+
168+
For instructions on running the integration tests and examples code,
169+
see [Running Integration Tests/Examples](#running-integration-testsexamples).
170+
171+
172+
### 8. Open PR with your changes
173+
After completing the previous steps to update the service, unit test, integration test, and working examples
174+
code, commit your changes. This is also a good time to make sure that the entire project builds and tests
175+
cleanly.
176+
Example:
177+
```sh
178+
cd <project-root>
179+
npm ci
180+
npm run all
181+
# Fix any remaining lint-check errors with "npm run lint-fix"
182+
183+
git add .
184+
git commit -s -m "feat(Example Service): re-gen service after recent API changes"
185+
git push
186+
```
187+
Note: be sure to sign off on your commits (git commit `-s` option) as that is a required PR check within the
188+
github.com/IBM org.
189+
190+
Finally, open a pull request (PR) and tag the project maintainer for approval.
191+
192+
## Appendix
193+
### Running Integration Tests/Examples
194+
To run the integration tests and working examples for a particular service, follow these steps. We'll use
195+
the mythical "Example Service" within the examples below, but you can make the necessary adjustments for your
196+
own service.
197+
198+
1. Make sure you have the required .env file in your project root directory. Each service's integration
199+
test and working examples code assumes that external configuration properties (service URL, IAM ApiKey, etc.)
200+
are stored in a .env file located in the project's root directory. The name of the file can be found in
201+
the integration test and examples code.
202+
Example:
203+
```python
204+
# Read config file
205+
configFile = 'example_service_v1.env'
206+
```
207+
The precise set of configuration properties required by each service will vary somewhat among the services,
208+
but there are a minimal set of properties that are commonly required by every service. The integration tests
209+
and examples code for certain services might require additional service-specific configuration properties as well.
210+
Typically these are documented in the working examples code.
211+
212+
2. Make sure that you have built/unit-tested the project successfully before trying to run the integration tests
213+
and/or examples:
214+
```sh
215+
cd <project-root>
216+
217+
make all
218+
```
219+
220+
3. To run the integration tests for a service, follow these steps:
221+
```sh
222+
cd <project-root>
223+
224+
pytest test/integration/test_example_service_v1.py
225+
```
226+
You should see 100% clean test results from the `pytest` command above, with no tests being skipped.
227+
228+
4. To run the examples code for a service, follow these steps:
229+
```sh
230+
cd <project-root>
231+
232+
pytest examples/test_example_service_v1_examples.py
233+
```
234+
You should see 100% clean results from the `pytest` command above, with no tests being skipped.
235+
236+
237+
### Updating Integration Tests/Examples
238+
Certain types of API changes will require that the integration tests and examples code
239+
are also updated along with the re-generated SDK service and unit test code.
240+
For example, perhaps a new operation was introduced or a new parameter was added
241+
to an existing operation and you'd like to incorporate it in the integration tests
242+
and examples.
243+
244+
Keep in mind that the integration tests are used to verify that the
245+
generated SDK code interacts correctly with the service implementation, so any non-trivial changes
246+
made to the API definition (and hence the generated service code) should probably result in updates
247+
to the integration tests. At a minimum, the integration tests for a service should include a
248+
testcase for EACH operation.
249+
250+
While modifying the integration tests, also consider if you should make any changes to the service's
251+
working examples code. We want the working examples to provide a good example for users
252+
to follow when writing their own application code which uses your service, so consider whether or not
253+
the examples code should be updated to reflect the changes made to the API.
254+
255+
The integration tests and examples code for each service were initially
256+
generated by the SDK generator, then (most likely) manual changes were made
257+
so that the tests and examples run cleanly using realistic values for
258+
various parameters and properties. The amount of manual changes required will vary from
259+
one service to the next, but usually depends on the degree to which your API definition:
260+
1. Includes good, realistic example values for operation parameters, request bodies, and responses.
261+
2. Includes links that capture any inter-operation dependencies (e.g. the `create_cloud` operation's `id`
262+
response property's value should be used as the `get_cloud` operation's `cloud_id` path parameter).
263+
264+
Regardless, it is likely that the integration tests and examples code have **some** manual
265+
changes which will need to be retained as you apply updates to them to reflect the current
266+
changes being made to the API.
267+
268+
Therefore, it is not recommended that you simply re-generate the integration tests and examples code
269+
such that the existing files are overwritten. Instead, we recommend that you generate new integration tests
270+
and examples off to the side, then manually copy fragments from the newly-generated files to the existing
271+
files located in the SDK project.
272+
Example:
273+
```sh
274+
openapi-sdkgen.sh generate -g ibm-python -i example-service.json --genITs --genExamples -o /tmp/code
275+
```
276+
The newly-generated integration tests and examples would be found in `/tmp/code/test/integration` and
277+
`/tmp/code/examples`. You could then copy fragments from there as needed to modify the corresponding files
278+
in the SDK project. This is not ideal, but you can minimize the amount of manual changes by improving your API definition
279+
as mentioned above.
280+
281+
## References
282+
- [IBM OpenAPI Validator](https://github.com/IBM/openapi-validator)
283+
- [IBM OpenAPI SDK Generator](https://github.ibm.com/CloudEngineering/openapi-sdkgen)

0 commit comments

Comments
 (0)