Skip to content

Commit e65e8bd

Browse files
committed
docs(idempotency): cleanup dynamodb required resource; break subsections and update nav
1 parent 11bc9fb commit e65e8bd

File tree

1 file changed

+15
-34
lines changed

1 file changed

+15
-34
lines changed

docs/utilities/idempotency.md

Lines changed: 15 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -69,18 +69,16 @@ We provide Infrastrucure as Code examples with [AWS Serverless Application Model
6969

7070
### Required resources
7171

72-
Before getting started, you need to create a persistent storage layer where the idempotency utility can store its state - your lambda functions will need read and write access to it.
72+
To start, you'll need:
7373

74-
We currently support Amazon DynamoDB and Redis as a storage layer. The following example demonstrates how to create a table in DynamoDB. If you prefer to use Redis, refer go to the section [RedisPersistenceLayer](#redispersistencelayer) section.
74+
1. A persistent storage layer (DynamoDB or [Redis](#redis-as-persistent-storage-layer-provider))
75+
2. An AWS Lambda function with [permissions](#iam-permissions) to use your persistent storage layer
7576

76-
**Default table configuration**
77+
#### DynamoDB table
7778

78-
If you're not [changing the default configuration for the DynamoDB persistence layer](#dynamodbpersistencelayer), this is the expected default configuration:
79+
!!! tip "You can share a single state table for all functions"
7980

80-
| Configuration | Value | Notes |
81-
| ------------------ | ------------ | ----------------------------------------------------------------------------------- |
82-
| Partition key | `id` | |
83-
| TTL attribute name | `expiration` | This can only be configured after your table is created if you're using AWS Console |
81+
Unless you're looking to use an [existing table or customize each attribute](#dynamodbpersistencelayer), you only need the following:
8482

8583
| Configuration | Value | Notes |
8684
| ------------------ | ------------ | -------------------------------------------------------------------------------------------------------- |
@@ -89,7 +87,7 @@ If you're not [changing the default configuration for the DynamoDB persistence l
8987

9088
Note that `fn_qualified_name` means the [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank" rel="nofollow"} defined in PEP-3155.
9189

92-
##### DynamoDB IaC examples
90+
##### IaC examples
9391

9492
=== "AWS Serverless Application Model (SAM) example"
9593

@@ -108,38 +106,21 @@ Note that `fn_qualified_name` means the [qualified name for classes and function
108106
--8<-- "examples/idempotency/templates/terraform.tf"
109107
```
110108

111-
???+ warning "Warning: Large responses with DynamoDB persistence layer"
112-
When using this utility with DynamoDB, your function's responses must be [smaller than 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}.
109+
##### Limitations
113110

114-
Larger items cannot be written to DynamoDB and will cause exceptions. If your response exceeds 400kb, consider using Redis as your persistence layer.
111+
* **DynamoDB restricts [item sizes to 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}**. This means that if your annotated function's response must be smaller than 400KB, otherwise your function will fail. Consider [Redis](#redis-as-persistent-storage-layer-provider) as an alternative.
115112

116-
<!-- markdownlint-disable MD013 -->
117-
???+ info "Info: DynamoDB"
113+
* **Expect 2 WCU per non-idempotent call**. During the first invocation, we use `PutItem` for locking and `UpdateItem` for completion. Consider reviewing [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/){target="_blank"}) to estimate cost.
118114

119-
During the first invocation with a payload, the Lambda function executes both a `PutItem` and an `UpdateItem` operations to store the data in DynamoDB. If the result returned by your Lambda is less than 1kb, you can expect 2 WCUs per Lambda invocation.
115+
* **Old boto3 versions can increase costs**. For cost optimization, we use a conditional `PutItem` to always lock a new idempotency record. If locking fails, it means we already have an idempotency record saving us an additional `GetItem` call. However, this is only supported in boto3 `1.26.194` and higher _([June 30th 2023](https://aws.amazon.com/about-aws/whats-new/2023/06/amazon-dynamodb-cost-failed-conditional-writes/){target="_blank"})_.
120116

121-
On subsequent invocations with the same payload, you can expect just 1 `PutItem` request to DynamoDB.
117+
#### Redis cluster
122118

123-
We recommend you start with a Redis compatible management services such as [Amazon ElastiCache for Redis](https://aws.amazon.com/elasticache/redis/){target="_blank"} or [Amazon MemoryDB for Redis](https://aws.amazon.com/memorydb/){target="_blank"}.
119+
**TODO**: Experiment bringing upfront Redis even at the cost of readability, as setup and usage are disconnected today causing further harm.
124120

125-
In both services and self-hosting Redis, you'll need to configure [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html){target="_blank"} to your AWS Lambda.
121+
##### Constraints
126122

127-
##### Redis IaC examples
128-
129-
=== "AWS CloudFormation example"
130-
131-
!!! tip inline end "Prefer AWS Console/CLI?"
132-
133-
Follow the official tutorials for [Amazon ElastiCache for Redis](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/LambdaRedis.html) or [Amazon MemoryDB for Redis](https://aws.amazon.com/blogs/database/access-amazon-memorydb-for-redis-from-aws-lambda/)
134-
135-
```yaml hl_lines="5 21"
136-
--8<-- "examples/idempotency/templates/cfn_redis_serverless.yaml"
137-
```
138-
139-
1. Replace the Security Group ID and Subnet ID to match your VPC settings.
140-
2. Replace the Security Group ID and Subnet ID to match your VPC settings.
141-
142-
Once setup, you can find a quick start and advanced examples for Redis in [the persistent layers section](RedisCachePersistenceLayer).
123+
If you'd like to use Redis, please [read here](#redis-as-persistent-storage-layer-provider) on how to setup and access secrets/SSL certs.
143124

144125
<!-- markdownlint-enable MD013 -->
145126

0 commit comments

Comments
 (0)