Skip to content

Commit 1ae8c7f

Browse files
durrandurran
authored
test(DRIVERS-2384): test deployed faas functions (#1387)
1 parent 935954b commit 1ae8c7f

File tree

1 file changed

+303
-0
lines changed

1 file changed

+303
-0
lines changed

‎source/faas-automated-testing/faas-automated-testing.rst

Lines changed: 303 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,303 @@
1+
======================
2+
FaaS Automated Testing
3+
======================
4+
5+
:Status:
6+
:Minimum Server Version: 3.6
7+
8+
.. contents::
9+
10+
--------
11+
12+
Abstract
13+
========
14+
15+
This specification is about the ability for drivers to automate tests for
16+
"Functions as a Service" from continuous integration.
17+
18+
META
19+
====
20+
21+
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
22+
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
23+
interpreted as described in `RFC 2119 <https://www.ietf.org/rfc/rfc2119.txt>`_.
24+
25+
Specification
26+
=============
27+
28+
Terms
29+
-----
30+
31+
FaaS
32+
~~~~
33+
34+
"Function as a Service", such as AWS Lambda.
35+
36+
Implementing Automated FaaS Tests
37+
---------------------------------
38+
39+
AWS Lambda
40+
~~~~~~~~~~
41+
42+
This section describes the required setup of an AWS Lambda function and the
43+
steps needed to automate the deployment and execution of the function in
44+
Evergreen.
45+
46+
Local Execution
47+
***************
48+
49+
Prerequisites
50+
`````````````
51+
52+
For the initial local setup the following are required:
53+
54+
- The docker daemon running on the local machine.
55+
- The [https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html](AWS SAM CLI)
56+
57+
The following environment variables MUST be present:
58+
59+
- `AWS_ACCESS_KEY_ID`
60+
- `AWS_SECRET_ACCESS_KEY`
61+
- `AWS_REGION` - Set to us-east-1
62+
- `MONGODB_URI` - The local MongoDB instance
63+
64+
Project Initialization
65+
``````````````````````
66+
67+
Create the new project via SAM and follow the prompts:
68+
69+
.. code:: none
70+
71+
sam init
72+
73+
For the template, select "AWS Quick Start Template".
74+
75+
.. code:: none
76+
77+
Which template source would you like to use?
78+
1 - AWS Quick Start Templates
79+
2 - Custom Template Location
80+
Choice: 1
81+
82+
For the quick start template, select "Hello World Example".
83+
84+
.. code:: none
85+
86+
Choose an AWS Quick Start application template
87+
1 - Hello World Example
88+
2 - Multi-step workflow
89+
3 - Serverless API
90+
4 - Scheduled task
91+
5 - Standalone function
92+
6 - Data processing
93+
7 - Infrastructure event management
94+
8 - Hello World Example With Powertools
95+
9 - Serverless Connector Hello World Example
96+
10 - Multi-step workflow with Connectors
97+
11 - Lambda EFS example
98+
12 - DynamoDB Example
99+
13 - Machine Learning
100+
Template: 1
101+
102+
When prompted for language if the driver language is not Python, select "N".
103+
104+
.. code:: none
105+
106+
Use the most popular runtime and package type? (Python and zip) [y/N]: n
107+
108+
Then select the runtime for your driver:
109+
110+
.. code:: none
111+
112+
Which runtime would you like to use?
113+
1 - aot.dotnet7 (provided.al2)
114+
2 - dotnet6
115+
3 - dotnet5.0
116+
4 - dotnetcore3.1
117+
5 - go1.x
118+
6 - go (provided.al2)
119+
7 - graalvm.java11 (provided.al2)
120+
8 - graalvm.java17 (provided.al2)
121+
9 - java11
122+
10 - java8.al2
123+
11 - java8
124+
12 - nodejs18.x
125+
13 - nodejs16.x
126+
14 - nodejs14.x
127+
15 - nodejs12.x
128+
16 - python3.9
129+
17 - python3.8
130+
18 - python3.7
131+
19 - ruby2.7
132+
20 - rust (provided.al2)
133+
Runtime: 12
134+
135+
Select Zip package type:
136+
137+
.. code:: none
138+
139+
What package type would you like to use?
140+
1 - Zip
141+
2 - Image
142+
Package type: 1
143+
144+
Then follow the remaining prompts for the driver language to finish setup. Drivers MAY
145+
choose to also enable X-Ray tracing and CloudWatch Application Insights during these
146+
next steps.
147+
148+
Function Setup
149+
``````````````
150+
151+
In the newly created project directory modify the template.yaml file:
152+
153+
Change default timeout to 30 seconds:
154+
155+
.. code:: yaml
156+
157+
Globals:
158+
Function:
159+
Timeout: 30
160+
161+
Add a root parameter for the MongoDB connection string:
162+
163+
.. code:: yaml
164+
165+
Parameters:
166+
MongoDbUri:
167+
Type: String
168+
Description: The MongoDB connection string.
169+
170+
Replace all instances in the template.yaml of `HelloWorld` with `MongoDB` and then
171+
modify the root `Resources` config to add the MONGODB_URI env variable reference
172+
and change the `CodeUri` to mongodb/ : Then rename the `hello-world` directory to `mongodb`.
173+
Do not change the `Handler` and `Runtime` properties.
174+
175+
.. code:: yaml
176+
177+
Resources:
178+
MongoDBFunction:
179+
Type: AWS::Serverless::Function
180+
Properties:
181+
CodeUri: mongodb/
182+
Environment:
183+
Variables:
184+
MONGODB_URI: !Ref MongoDbUri
185+
186+
Run the function locally from the same directory where the template.yaml resides:
187+
188+
.. code:: none
189+
190+
sam build
191+
sam local invoke --parameter-overrides "MongoDbUri=${MONGODB_URI}"
192+
193+
194+
Implementing the Function
195+
`````````````````````````
196+
197+
Drivers MUST setup the function as would be done in their appropriate language. In
198+
the function implementation the driver MUST:
199+
200+
- Create a MongoClient that points to MONGODB_URI.
201+
- Add listeners for the following monitoring events: ServerHeartbeatStarted,
202+
ServerHeartbeatFailed, CommandSucceeded, CommandFailed, ConnectionCreated,
203+
ConnectionClosed.
204+
- Drivers MUST perform a single insert and then a single delete of the inserted document
205+
to force write operations on the primary node.
206+
- Drivers MUST record the durations and counts of the heartbeats, the durations of the
207+
commands, as well as keep track of the number of open connections, and report this information in
208+
the function response as JSON.
209+
210+
211+
Running in Continuous Integration
212+
`````````````````````````````````
213+
214+
Running in CI requires Evergreen to be setup to assume the appropriate role in AWS
215+
and then execute the script in drivers-evergreen-tools with the required environment
216+
variables. An explanation of the required environment is as follows:
217+
218+
+-------------------------------+----------+--------------------------+
219+
| Name | Description |
220+
+===============================+=====================================+
221+
| LAMBDA_AWS_ROLE_ARN | The role ARN to assume |
222+
+-------------------------------+-------------------------------------+
223+
| TEST_LAMBDA_DIRECTORY | The lambda function directory |
224+
+-------------------------------+-------------------------------------+
225+
| DRIVERS_TOOLS | Location of drivers-evergreen-tools |
226+
+-------------------------------+-------------------------------------+
227+
| DRIVERS_ATLAS_PUBLIC_API_KEY | The Atlas public API key |
228+
+-------------------------------+-------------------------------------+
229+
| DRIVERS_ATLAS_PRIVATE_API_KEY | The Atlas private API key |
230+
+-------------------------------+-------------------------------------+
231+
| DRIVERS_ATLAS_LAMBDA_USER | The Atlas cluster user name |
232+
+-------------------------------+-------------------------------------+
233+
| DRIVERS_ATLAS_LAMBDA_PASSWORD | The Atlas cluster user password |
234+
+-------------------------------+-------------------------------------+
235+
| DRIVERS_ATLAS_GROUP_ID | The driver's Atlas group id |
236+
+-------------------------------+-------------------------------------+
237+
| LAMBDA_STACK_NAME | The driver's Lambda stack name |
238+
+-------------------------------+-------------------------------------+
239+
| AWS_REGION | The function AWS region |
240+
+-------------------------------+-------------------------------------+
241+
| AWS_ACCESS_KEY_ID | Assume role atuomatically sets this |
242+
+-------------------------------+-------------------------------------+
243+
| AWS_SECRET_ACCESS_KEY | Assume role automatically sets this |
244+
+-------------------------------+-------------------------------------+
245+
| AWS_SESSION_TOKEN | Assume role automatically sets this |
246+
+-------------------------------+-------------------------------------+
247+
248+
249+
This is an example function in the Evergreen config that accomplishes this, using
250+
subprocess.exec to execute a script that calls the drivers-evergreen-tools
251+
function inside of it:
252+
253+
.. code:: yaml
254+
255+
run deployed aws lambda tests:
256+
- command: ec2.assume_role
257+
params:
258+
role_arn: ${LAMBDA_AWS_ROLE_ARN}
259+
- command: subprocess.exec
260+
params:
261+
working_dir: src
262+
binary: bash
263+
args:
264+
- .evergreen/run-deployed-lambda-aws-tests.sh
265+
env:
266+
TEST_LAMBDA_DIRECTORY: ${PROJECT_DIRECTORY}/test/lambda
267+
DRIVERS_TOOLS: ${DRIVERS_TOOLS}
268+
DRIVERS_ATLAS_PUBLIC_API_KEY: ${DRIVERS_ATLAS_PUBLIC_API_KEY}
269+
DRIVERS_ATLAS_PRIVATE_API_KEY: ${DRIVERS_ATLAS_PRIVATE_API_KEY}
270+
DRIVERS_ATLAS_LAMBDA_USER: ${DRIVERS_ATLAS_LAMBDA_USER}
271+
DRIVERS_ATLAS_LAMBDA_PASSWORD: ${DRIVERS_ATLAS_LAMBDA_PASSWORD}
272+
DRIVERS_ATLAS_GROUP_ID: ${DRIVERS_ATLAS_GROUP_ID}
273+
LAMBDA_STACK_NAME: dbx-node-lambda
274+
AWS_REGION: us-east-1
275+
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
276+
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
277+
AWS_SESSION_TOKEN: ${AWS_SESSION_TOKEN}
278+
279+
Drivers MUST run the function on a single variant in Evergreen, in order to not
280+
potentially hit the Atlas API rate limit. The variant itself MUST be either a
281+
RHEL8 or Ubuntu 20 variant in order to have the SAM CLI installed.
282+
283+
The script itself:
284+
285+
.. code:: none
286+
287+
#!/bin/bash
288+
set -o errexit # Exit the script with error if any of the commands fail
289+
. ${DRIVERS_TOOLS}/.evergreen/run-deployed-lambda-aws-tests.sh
290+
291+
Description of the behaviour of run-deployed-lambda-aws-tests.sh:
292+
293+
- Creates a new Atlas cluster in a specific driver project
294+
- Polls for the cluster SRV record when cluster creation is complete
295+
- Builds the Lambda function locally
296+
- Deploys the Lambda function to AWS.
297+
- Queries for the Lambda function ARN.
298+
- Invokes the Lambda function cold and frozen.
299+
- Initiates a primary failover of the cluster in Atlas.
300+
- Calls the frozen lambda function again.
301+
- Deletes the Lambda function.
302+
- Deletes the Atlas cluster.
303+

0 commit comments

Comments
 (0)