Merge pull request #526 from aws-powertools/develop

chore: Sync main and develop

Author: hjgraca

README.md

@@ -27,7 +27,7 @@ Powertools for AWS Lambda (.NET) provides three core utilities:
 
 * **[Parameters](https://docs.powertools.aws.dev/lambda/dotnet/utilities/parameters/)** - provides high-level functionality to retrieve one or multiple parameter values from [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html), [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/), or [Amazon DynamoDB](https://aws.amazon.com/dynamodb/). We also provide extensibility to bring your own providers.
 
-* **[Idempotency (developer preview)](https://docs.powertools.aws.dev/lambda/dotnet/utilities/idempotency/)** - The idempotency utility provides a simple solution to convert your Lambda functions into idempotent operations which are safe to retry.
+* **[Idempotency](https://docs.powertools.aws.dev/lambda/dotnet/utilities/idempotency/)** - The idempotency utility provides a simple solution to convert your Lambda functions into idempotent operations which are safe to retry.
 
 * **[Batch Processing (developer preview)](https://docs.powertools.aws.dev/lambda/dotnet/utilities/batch-processing/)** - The batch processing utility handles partial failures when processing batches from Amazon SQS, Amazon Kinesis Data Streams, and Amazon DynamoDB Streams.
 

docs/core/logging.md

@@ -72,14 +72,19 @@ Here is an example using the AWS SAM [Globals section](https://docs.aws.amazon.c
 
 ### Using AWS Lambda Advanced Logging Controls (ALC)
 
-With [AWS Lambda Advanced Logging Controls (ALC)](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-advanced), you can control the output format of your logs as either TEXT or JSON and specify the minimum accepted log level for your application. Regardless of the output format setting in Lambda, Powertools for AWS Lambda will always output JSON formatted logging messages.
+!!! question "When is it useful?"
+    When you want to set a logging policy to drop informational or verbose logs for one or all AWS Lambda functions, regardless of runtime and logger used.
 
-When you have this feature enabled, log messages that don’t meet the configured log level are discarded by Lambda. For example, if you set the minimum log level to WARN, you will only receive WARN and ERROR messages in your AWS CloudWatch Logs, all other log levels will be discarded by Lambda.
+With [AWS Lambda Advanced Logging Controls (ALC)](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-advanced){target="_blank"}, you can enforce a minimum log level that Lambda will accept from your application code.
+
+When enabled, you should keep `Logger` and ALC log level in sync to avoid data loss.
 
 !!! warning "When using AWS Lambda Advanced Logging Controls (ALC)"
     - When Powertools Logger output is set to `PascalCase` **`Level`**  property name will be replaced by **`LogLevel`** as a property name.
     - ALC takes precedence over **`POWERTOOLS_LOG_LEVEL`** and when setting it in code using **`[Logging(LogLevel = )]`**
 
+Here's a sequence diagram to demonstrate how ALC will drop both `Information` and `Debug` logs emitted from `Logger`, when ALC log level is stricter than `Logger`.
+
 ```mermaid
 sequenceDiagram
     title Lambda ALC allows WARN logs only
@@ -88,6 +93,7 @@ sequenceDiagram
     participant Application Logger
     
     Note over Lambda service: AWS_LAMBDA_LOG_LEVEL="WARN"
+    Note over Application Logger: POWERTOOLS_LOG_LEVEL="DEBUG"
     Lambda service->>Lambda function: Invoke (event)
     Lambda function->>Lambda function: Calls handler
     Lambda function->>Application Logger: Logger.Warning("Something happened")
@@ -99,20 +105,18 @@ sequenceDiagram
     Lambda service->>CloudWatch Logs: Ingest error logs
 ```
 
-Logger will automatically listen for the AWS_LAMBDA_LOG_FORMAT and AWS_LAMBDA_LOG_LEVEL environment variables, and change behaviour if they’re found to ensure as much compatibility as possible.
-
 **Priority of log level settings in Powertools for AWS Lambda**
 
-When the Advanced Logging Controls feature is enabled, we are unable to increase the minimum log level below the AWS_LAMBDA_LOG_LEVEL environment variable value, see [AWS Lambda service documentation](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-log-level) for more details.
-
 We prioritise log level settings in this order:
 
 1. AWS_LAMBDA_LOG_LEVEL environment variable
 2. Setting the log level in code using `[Logging(LogLevel = )]`
 3. POWERTOOLS_LOG_LEVEL environment variable
 
-In the event you have set POWERTOOLS_LOG_LEVEL to a level lower than the ACL setting, Powertools for AWS Lambda will output a warning log message informing you that your messages will be discarded by Lambda.
+If you set `Logger` level lower than ALC, we will emit a warning informing you that your messages will be discarded by Lambda.
 
+> **NOTE**
+> With ALC enabled, we are unable to increase the minimum log level below the `AWS_LAMBDA_LOG_LEVEL` environment variable value, see [AWS Lambda service documentation](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html#monitoring-cloudwatchlogs-log-level){target="_blank"} for more details.
 
 ## Standard structured keys
 

docs/index.md

@@ -30,7 +30,7 @@ Core utilities such as Tracing, Logging, and Metrics will be available across al
 [Logger](./core/logging.md) | Structured logging made easier, and decorator to enrich structured logging with key Lambda context details
 [Metrics](./core/metrics.md) | Custom AWS metrics created asynchronously via CloudWatch Embedded Metric Format (EMF)
 [Parameters](./utilities/parameters/) | provides high-level functionality to retrieve one or multiple parameter values from [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html){target="_blank"}, [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/){target="_blank"}, or [Amazon DynamoDB](https://aws.amazon.com/dynamodb/){target="_blank"}. We also provide extensibility to bring your own providers.
-[Idempotency (developer preview)](./utilities/idempotency/) | The idempotency utility provides a simple solution to convert your Lambda functions into idempotent operations which are safe to retry.
+[Idempotency](./utilities/idempotency/) | The idempotency utility provides a simple solution to convert your Lambda functions into idempotent operations which are safe to retry.
 [Batch Processing (developer preview)](./utilities/batch-processing/) | The batch processing utility handles partial failures when processing batches from Amazon SQS, Amazon Kinesis Data Streams, and Amazon DynamoDB Streams.
 
 ## Install
@@ -128,4 +128,4 @@ These are our core principles to guide our decision making.
 * **Keep it lean**. Additional dependencies are carefully considered for security and ease of maintenance, and prevent negatively impacting startup time.
 * **We strive for backwards compatibility**. New features and changes should keep backwards compatibility. If a breaking change cannot be avoided, the deprecation and migration process should be clearly defined.
 * **We work backwards from the community**. We aim to strike a balance of what would work best for 80% of customers. Emerging practices are considered and discussed via Requests for Comment (RFCs)
-* **Idiomatic**. Utilities follow programming language idioms and language-specific best practices.
+* **Idiomatic**. Utilities follow programming language idioms and language-specific best practices.

docs/utilities/idempotency.md

@@ -2,8 +2,6 @@
 title: Idempotency
 description: Utility
 ---
-???+ warning
-	**This utility is currently in developer preview** and is intended strictly for feedback and testing purposes **and not for production workloads**. The version and all future versions tagged with the `-preview` suffix should be treated as not stable. Until this utility is [General Availability](https://github.com/aws-powertools/powertools-lambda-dotnet/milestone/3) we may introduce significant breaking changes and improvements in response to customers feedback.
 
 <!-- markdownlint-disable MD013 -->
 

examples/BatchProcessing/src/HelloWorld/HelloWorld.csproj

@@ -8,7 +8,7 @@
         <PackageReference Include="Amazon.Lambda.Core" Version="2.1.0" />
         <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.1" />
         <PackageReference Include="AWS.Lambda.Powertools.BatchProcessing" Version="0.0.1-preview" />
-        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.3.3" />
+        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.4.4" />
         <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="7.0.0" />
     </ItemGroup>
 </Project>

examples/Idempotency/src/HelloWorld/HelloWorld.csproj

@@ -9,6 +9,6 @@
         <PackageReference Include="Amazon.Lambda.APIGatewayEvents" Version="2.6.0" />
         <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.1" />
         <PackageReference Include="AWS.Lambda.Powertools.Idempotency" Version="0.2.2-preview" />
-        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.3.3" />
+        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.4.4" />
     </ItemGroup>
 </Project>

examples/Logging/src/HelloWorld/HelloWorld.csproj

@@ -8,7 +8,7 @@
         <PackageReference Include="Amazon.Lambda.Core" Version="2.1.0" />
         <PackageReference Include="Amazon.Lambda.APIGatewayEvents" Version="2.6.0" />
         <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.1" />
-        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.3.3" />
+        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.4.4" />
         <PackageReference Include="AWSSDK.DynamoDBv2" Version="3.7.201.13" />
     </ItemGroup>
 </Project>

examples/Metrics/src/HelloWorld/HelloWorld.csproj

@@ -8,8 +8,8 @@
         <PackageReference Include="Amazon.Lambda.Core" Version="2.1.0" />
         <PackageReference Include="Amazon.Lambda.APIGatewayEvents" Version="2.6.0" />
         <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.1" />
-        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.3.3" />
-        <PackageReference Include="AWS.Lambda.Powertools.Metrics" Version="1.4.3" />
+        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.4.4" />
+        <PackageReference Include="AWS.Lambda.Powertools.Metrics" Version="1.5.3" />
         <PackageReference Include="AWSSDK.DynamoDBv2" Version="3.7.201.13" />
     </ItemGroup>
 </Project>

examples/ServerlessApi/src/LambdaPowertoolsAPI/LambdaPowertoolsAPI.csproj

@@ -13,8 +13,8 @@
   </PropertyGroup>
   <ItemGroup>
     <PackageReference Include="Amazon.Lambda.AspNetCoreServer" Version="8.1.0" />
-    <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.3.3" />
-    <PackageReference Include="AWS.Lambda.Powertools.Metrics" Version="1.4.3" />
+    <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.4.4" />
+    <PackageReference Include="AWS.Lambda.Powertools.Metrics" Version="1.5.3" />
     <PackageReference Include="AWS.Lambda.Powertools.Tracing" Version="1.3.2" />
   </ItemGroup>
 </Project>

examples/Tracing/src/HelloWorld/HelloWorld.csproj

@@ -8,7 +8,7 @@
         <PackageReference Include="Amazon.Lambda.Core" Version="2.1.0" />
         <PackageReference Include="Amazon.Lambda.APIGatewayEvents" Version="2.6.0" />
         <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.1" />
-        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.3.3" />
+        <PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.4.4" />
         <PackageReference Include="AWS.Lambda.Powertools.Tracing" Version="1.3.2" />
         <PackageReference Include="AWSSDK.DynamoDBv2" Version="3.7.201.13" />
     </ItemGroup>

libraries/tests/AWS.Lambda.Powertools.Idempotency.Tests/Handlers/IdempotencyFunctionMethodDecorated.cs

@@ -0,0 +1,99 @@
+/*
+ * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License").
+ * You may not use this file except in compliance with the License.
+ * A copy of the License is located at
+ * 
+ *  http://aws.amazon.com/apache2.0
+ * 
+ * or in the "license" file accompanying this file. This file is distributed
+ * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+ * express or implied. See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Net.Http;
+using System.Text.Json;
+using System.Threading.Tasks;
+using Amazon.DynamoDBv2;
+using Amazon.Lambda.APIGatewayEvents;
+using Amazon.Lambda.Core;
+
+namespace AWS.Lambda.Powertools.Idempotency.Tests.Handlers;
+
+public class IdempotencyFunctionMethodDecorated
+{
+    public bool MethodCalled;
+
+    public IdempotencyFunctionMethodDecorated(AmazonDynamoDBClient client)
+    {
+        Idempotency.Configure(builder =>
+            builder
+                .UseDynamoDb(storeBuilder =>
+                    storeBuilder
+                        .WithTableName("idempotency_table")
+                        .WithDynamoDBClient(client)
+                ));
+    }
+
+    
+    public async Task<APIGatewayProxyResponse> Handle(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context)
+    {
+        Idempotency.RegisterLambdaContext(context);
+        var result= await InternalFunctionHandler(apigProxyEvent);
+
+        return result;
+    }
+    
+    private async Task<APIGatewayProxyResponse> InternalFunctionHandler(APIGatewayProxyRequest apigProxyEvent)
+    {
+        Dictionary<string, string> headers = new()
+        {
+            {"Content-Type", "application/json"},
+            {"Access-Control-Allow-Origin", "*"},
+            {"Access-Control-Allow-Methods", "GET, OPTIONS"},
+            {"Access-Control-Allow-Headers", "*"}
+        };
+
+        try
+        {
+            var address = JsonDocument.Parse(apigProxyEvent.Body).RootElement.GetProperty("address").GetString();
+            var pageContents = await GetPageContents(address);
+            var output = $"{{ \"message\": \"hello world\", \"location\": \"{pageContents}\" }}";
+
+            return new APIGatewayProxyResponse
+            {
+                Body = output,
+                StatusCode = 200,
+                Headers = headers
+            };
+
+        }
+        catch (IOException)
+        {
+            return new APIGatewayProxyResponse
+            {
+                Body = "{}",
+                StatusCode = 500,
+                Headers = headers
+            };
+        }
+    }
+    
+    [Idempotent]
+    private async Task<string> GetPageContents(string address)
+    {
+        MethodCalled = true;
+        
+        var client = new HttpClient();
+        using var response = await client.GetAsync(address);
+        using var content = response.Content;
+        var pageContent = await content.ReadAsStringAsync();
+        
+        return pageContent;
+    }
+}

libraries/tests/AWS.Lambda.Powertools.Idempotency.Tests/IdempotencyTest.cs

@@ -1,24 +1,27 @@
 /*
  * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
- * 
+ *
  * Licensed under the Apache License, Version 2.0 (the "License").
  * You may not use this file except in compliance with the License.
  * A copy of the License is located at
- * 
+ *
  *  http://aws.amazon.com/apache2.0
- * 
+ *
  * or in the "license" file accompanying this file. This file is distributed
  * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
  * express or implied. See the License for the specific language governing
  * permissions and limitations under the License.
  */
 
+using System;
+using System.Collections.Generic;
 using System.IO;
 using System.Text.Json;
 using System.Threading.Tasks;
 using Amazon.DynamoDBv2;
 using Amazon.DynamoDBv2.Model;
 using Amazon.Lambda.APIGatewayEvents;
+using Amazon.Lambda.TestUtilities;
 using AWS.Lambda.Powertools.Idempotency.Tests.Handlers;
 using AWS.Lambda.Powertools.Idempotency.Tests.Persistence;
 using FluentAssertions;
@@ -66,5 +69,57 @@ public async Task EndToEndTest()
             TableName = _tableName
         });
         scanResponse.Count.Should().Be(1);
+        
+        // delete row dynamo
+        var key = new Dictionary<string, AttributeValue>
+        {
+            ["id"] = new AttributeValue { S = "testFunction.GetPageContents#ff323c6f0c5ceb97eed49121babcec0f" }
+        };
+        await _client.DeleteItemAsync(new DeleteItemRequest{TableName = _tableName, Key = key});
+    }
+    
+    [Fact]
+    [Trait("Category", "Integration")]
+    public async Task EndToEndTestMethod() 
+    {
+        var function = new IdempotencyFunctionMethodDecorated(_client);
+        
+        var options = new JsonSerializerOptions
+        {
+            PropertyNameCaseInsensitive = true
+        };
+        
+        var context = new TestLambdaContext
+        {
+            RemainingTime = TimeSpan.FromSeconds(30)
+        };
+        
+        var request = JsonSerializer.Deserialize<APIGatewayProxyRequest>(await File.ReadAllTextAsync("./resources/apigw_event2.json"),options);
+        
+        var response = await function.Handle(request, context);
+        function.MethodCalled.Should().BeTrue();
+
+        function.MethodCalled = false;
+
+        var response2 = await function.Handle(request, context);
+        function.MethodCalled.Should().BeFalse();
+
+        // Assert
+        JsonSerializer.Serialize(response).Should().Be(JsonSerializer.Serialize(response2));
+        response.Body.Should().Contain("hello world");
+        response2.Body.Should().Contain("hello world");
+
+        var scanResponse = await _client.ScanAsync(new ScanRequest
+        {
+            TableName = _tableName
+        });
+        scanResponse.Count.Should().Be(1);
+        
+        // delete row dynamo
+        var key = new Dictionary<string, AttributeValue>
+        {
+            ["id"] = new AttributeValue { S = "testFunction.GetPageContents#ff323c6f0c5ceb97eed49121babcec0f" }
+        };
+        await _client.DeleteItemAsync(new DeleteItemRequest{TableName = _tableName, Key = key});
     }
 }

libraries/tests/AWS.Lambda.Powertools.Idempotency.Tests/Internal/IdempotentAspectTests.cs

@@ -311,7 +311,7 @@ public void Handle_WhenIdempotencyOnSubMethodAnnotated_AndSecondCall_AndNotExpir
         // Arrange
         var store = Substitute.For<BasePersistenceStore>();
         store.SaveInProgress(Arg.Any<JsonDocument>(), Arg.Any<DateTimeOffset>(), Arg.Any<double>())
-            .Throws(new IdempotencyItemAlreadyExistsException());
+            .Returns(_ => throw new IdempotencyItemAlreadyExistsException());
 
         Idempotency.Configure(builder => builder.WithPersistenceStore(store));
 
@@ -327,7 +327,7 @@ public void Handle_WhenIdempotencyOnSubMethodAnnotated_AndSecondCall_AndNotExpir
             .Returns(record);
 
         // Act
-        var function = new IdempotencyInternalFunction(false);
+        var function = new IdempotencyInternalFunction(true);
         Basket resultBasket = function.HandleRequest(product, new TestLambdaContext());
 
         // assert

version.json

@@ -6,7 +6,7 @@
 	},
 	"Utilities": {
 		"Parameters": "1.1.2",
-		"Idempotency": "0.2.2-preview",
+		"Idempotency": "1.0.0",
 		"BatchProcessing": "0.0.1-preview"
 	}
 }