You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/utilities/idempotency.md
+15-34Lines changed: 15 additions & 34 deletions
Original file line number
Diff line number
Diff line change
@@ -69,18 +69,16 @@ We provide Infrastrucure as Code examples with [AWS Serverless Application Model
69
69
70
70
### Required resources
71
71
72
-
Before getting started, you need to create a persistent storage layer where the idempotency utility can store its state - your lambda functions will need read and write access to it.
72
+
To start, you'll need:
73
73
74
-
We currently support Amazon DynamoDB and Redis as a storage layer. The following example demonstrates how to create a table in DynamoDB. If you prefer to use Redis, refer go to the section [RedisPersistenceLayer](#redispersistencelayer) section.
74
+
1. A persistent storage layer (DynamoDB or [Redis](#redis-as-persistent-storage-layer-provider))
75
+
2. An AWS Lambda function with [permissions](#iam-permissions) to use your persistent storage layer
75
76
76
-
**Default table configuration**
77
+
#### DynamoDB table
77
78
78
-
If you're not [changing the default configuration for the DynamoDB persistence layer](#dynamodbpersistencelayer), this is the expected default configuration:
79
+
!!! tip "You can share a single state table for all functions"
@@ -89,7 +87,7 @@ If you're not [changing the default configuration for the DynamoDB persistence l
89
87
90
88
Note that `fn_qualified_name` means the [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} defined in PEP-3155.
91
89
92
-
##### DynamoDB IaC examples
90
+
##### IaC examples
93
91
94
92
=== "AWS Serverless Application Model (SAM) example"
95
93
@@ -108,38 +106,21 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
???+ warning "Warning: Large responses with DynamoDB persistence layer"
112
-
When using this utility with DynamoDB, your function's responses must be [smaller than 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}.
109
+
##### Limitations
113
110
114
-
Larger items cannot be written to DynamoDB and will cause exceptions. If your response exceeds 400kb, consider using Redis as your persistence layer.
111
+
***DynamoDB restricts [item sizes to 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}**. This means that if your annotated function's response must be smaller than 400KB, otherwise your function will fail. Consider [Redis](#redis-as-persistent-storage-layer-provider) as an alternative.
115
112
116
-
<!-- markdownlint-disable MD013 -->
117
-
???+ info "Info: DynamoDB"
113
+
***Expect 2 WCU per non-idempotent call**. During the first invocation, we use `PutItem` for locking and `UpdateItem` for completion. Consider reviewing [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/){target="_blank"}) to estimate cost.
118
114
119
-
During the first invocation with a payload, the Lambda function executes both a `PutItem` and an `UpdateItem` operations to store the data in DynamoDB. If the result returned by your Lambda is less than 1kb, you can expect 2 WCUs per Lambda invocation.
115
+
***Old boto3 versions can increase costs**. For cost optimization, we use a conditional `PutItem`to always lock a new idempotency record. If locking fails, it means we already have an idempotency record saving us an additional `GetItem` call. However, this is only supported in boto3 `1.26.194` and higher _([June 30th 2023](https://aws.amazon.com/about-aws/whats-new/2023/06/amazon-dynamodb-cost-failed-conditional-writes/){target="_blank"})_.
120
116
121
-
On subsequent invocations with the same payload, you can expect just 1 `PutItem` request to DynamoDB.
117
+
#### Redis cluster
122
118
123
-
We recommend you start with a Redis compatible management services such as [Amazon ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB for Redis](https://aws.amazon.com/memorydb/){target="_blank"}.
119
+
**TODO**: Experiment bringing upfront Redis even at the cost of readability, as setup and usage are disconnected today causing further harm.
124
120
125
-
In both services and self-hosting Redis, you'll need to configure [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} to your AWS Lambda.
121
+
##### Constraints
126
122
127
-
##### Redis IaC examples
128
-
129
-
=== "AWS CloudFormation example"
130
-
131
-
!!! tip inline end "Prefer AWS Console/CLI?"
132
-
133
-
Follow the official tutorials for [Amazon ElastiCache for Redis](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/LambdaRedis.html) or [Amazon MemoryDB for Redis](https://aws.amazon.com/blogs/database/access-amazon-memorydb-for-redis-from-aws-lambda/)
0 commit comments