Skip to content

Releases: aws-powertools/powertools-lambda-python

v2.15.0

04 May 16:34
@release-drafter release-drafter
v2.15.0
9594b86
Compare
Choose a tag to compare
v2.15.0

Summary

This release is packed with a number of improvements:

  • New external provider formats in Logger
  • New Parser models and JMESPath expressions to naturally process S3 Event Notifications targets
  • Printing Event Source Data Classes now automatically mask sensitive data
  • New flush_metrics in Metrics to support customers using Lambda Web Adapter
  • Added support for the China and GovCloud regions in API Gateway Authorizer event (bugfix)

And… a ton of documentation improvements.

⭐ Huge thanks to new contributors: @theipster (S3 events), @neilramsay (Data Class debug), @arjanschaaf (Batch docs) and @leif-ye (API Gateway Event Source)

External provider formats in the Logger utility

Docs: Observability providers

You can now send logs to the observability provider of your choice via Lambda Extensions. In most cases, you shouldn't need any custom Logger configuration, and logs will be shipped asynchronously with no performance impact.
image

Improved S3 Event Notifications developer experience

Docs: Built-in envelopes, Parser

S3 event notifications can be sometimes be ingested into Lambda via an intermediary such as an SQS queue (i.e. Lambda event source), for various architectural reasons - batching, retries, etc. However, from the Lambda function's perspective, the intermediary might not be too important; what's important is the S3 event notification itself. Now you can easily parse and access the inner payload for SQS-wrapper S3 events.

JMESPath

carbon (2)

Parser

carbon (3)

Printing Event Source Data Classes

Docs: Debugging

You can now print out the fields of a data class instance to obtain more information. All classes come with a __str__ method that generates a dictionary string which can be quite useful for debugging. Sensitive fields such as secret_access_key and session_token, are labeled as [SENSITIVE], to prevent any accidental disclosure of confidential information.

carbon (4)

Flushing metrics manually

Docs: Flushing metrics manually

You can now manually flush the metrics at any time. This is useful when not running within a standard Lambda handler (e.g: Lambda Web Adapter), where the @log_metrics decorator does not work as intended.

image

Changes

🌟New features and non-breaking changes

  • feat(event_source): support custom json_deserializer; add json_body in SQSEvent (#2200) by @leandrodamascena
  • feat(parser): add support for SQS-wrapped S3 event notifications (#2108) by @theipster
  • feat(logger): add DatadogLogFormatter and observability provider (#2183) by @heitorlessa
  • feat(jmespath): new built-in envelopes to unwrap S3 events (#2169) by @leandrodamascena
  • feat(event_source): add support for dynamic partitions in the Api Gateway Authorizer event (#2176) by @leif-ye
  • feat(metrics): add flush_metrics() method to allow manual flushing of metrics (#2171) by @rubenfonseca
  • feat(event_sources): Add str to Data Classes base DictWrapper (#2129) by @neilramsay
  • feat(ci): dispatch GitHub analytics action (#2161) by @rubenfonseca

📜 Documentation updates

🔧 Maintenance

This release was made possible by the following contributors:

@arjanschaaf, @dependabot, @dependabot[bot], @heitorlessa, @leandrodamascena, @leif-ye, @neilramsay, @rubenfonseca, @theipster and Release bot

Contributors

rubenfonseca, neilramsay, and 6 other contributors
Loading

v2.14.1

21 Apr 13:30
@release-drafter release-drafter
v2.14.1
49bd097
Compare
Choose a tag to compare
v2.14.1

Summary

This patch release addresses a regression in Batch for anyone not using Pydantic or installing Powertools via Layers.

Huge thanks to @walmsles ⭐ for spotting it and sending a fix in record time.

Changes

🌟New features and non-breaking changes

🐛 Bug and hot fixes

🔧 Maintenance

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @heitorlessa, @leandrodamascena, @rubenfonseca, @walmsles and Release bot

Contributors

rubenfonseca, walmsles, and 3 other contributors
Loading
0 Join discussion

v2.14.0

21 Apr 13:25
@release-drafter release-drafter
v2.14.0
c44a45c
Compare
Choose a tag to compare
v2.14.0

Summary

Duplicate: due to a GitHub glitch this notification might have appeared earlier.

This release adds support for 3.10.

Changes

🌟New features and non-breaking changes

🔧 Maintenance

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @heitorlessa, @rubenfonseca and Release bot

Contributors

rubenfonseca, heitorlessa, and dependabot
Loading

v2.13.0

14 Apr 10:42
@release-drafter release-drafter
v2.13.0
a76a554
This commit was signed with the committer’s verified signature.
rubenfonseca Ruben Fonseca
GPG key ID: C229A8A73F0A9AEA
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v2.13.0

Summary

This release improves the developer experience using the Parameters utility, fixes a bug on the CodePipeline event source and adds several documentation improvements!

⭐ Huge thanks to our new contributor @neilramsay

Improved DX on the Parameters utility

Docs: Cache TTL, SSMProvider

You can now set the POWERTOOLS_PARAMETERS_MAX_AGE and POWERTOOLS_PARAMETERS_SSM_DECRYPT environment variables to set defaults values for all parameters fetched by this utility. This avoids having to pass max_age and decrypt when you fetch several parameters.

image

Changes

🌟New features and non-breaking changes

  • feat(parameters): Configure max_age and decrypt parameters via environment variables (#2088) by @leandrodamascena

📜 Documentation updates

  • feat(parameters): Configure max_age and decrypt parameters via environment variables (#2088) by @leandrodamascena
  • fix(ci): update layer version on logger, tracer and metrics docs (#2120) by @rubenfonseca

🐛 Bug and hot fixes

  • fix(event_sources): Update CodePipeline event source to include optional encryption_key field and make user_parameters field optional (#2113) by @neilramsay
  • fix(ci): update layer version on logger, tracer and metrics docs (#2120) by @rubenfonseca

🔧 Maintenance

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @djfurman, @leandrodamascena, @neilramsay, @rubenfonseca and Release bot

Contributors

rubenfonseca, neilramsay, and 3 other contributors
Loading

v2.12.0

07 Apr 15:04
@release-drafter release-drafter
v2.12.0
985c1ae
Compare
Choose a tag to compare
v2.12.0

Summary

This release improves the authoring experience for Batch processing (-boilerplate), fix a bug between Batch and Pydantic integration for poison pills, and several documentation and static typing improvements!

⭐ Huge thanks to new contributors: @michael-k (docs) and @LuckIlNe (Batch pydantic bugfix)

New Batch experience

No more decorators for Batch Processing

Docs: Amazon SQS, Amazon Kinesis Data Streams, Amazon DynamoDB Streams

Batch feature is generally used for building more resilient data pipelines ranging from ETL to Change Data Capture to AWS automation.

This release significantly removes boilerplate and makes it less error prone to author batch processors. You can now use process_partial_response and async_process_partial_response within your Lambda handler to kick off processing, handle failures and return the expected response for partial failures in Lambda.

Before

image

After

New Batch experience

Addressing poison pill issue with Batch and Pydantic model validation

Docs

Pydantic provides parsing and deep data validation mechanisms. It's commonly used with Batch feature to author and validate data models.

Thanks to @LuckIlNe, we now correctly handle malformed messages that fail validation early (poison pills), thus not breaking the partial failure mechanism. Previously, any record that doesn't conform with an early model validation would not fail gracefully.

Documentation improvements

Idempotency

Docs: Idempotency new diagrams, terminology, and Idempotency record expiration vs DynamoDB time-to-live (TTL).

Thanks to a discussion with @pmarko1711, we revamped the Idempotency documentation to include more sequence diagrams to visually explain the different idempotent scenarios. We also clarified terminologies like Idempotency Record, and a confusion between Idempotency Record Expiration and DynamoDB TTL.

The latter can be used as a fine-grained concurrency mechanism.

image

Batch

Docs

Pydantic integration examples now use the new Json model from Pydantic to auto-deserialize JSON strings into Pydantic models. This removes excessive boilerplate when working with SQS and Kinesis Data Streams in Batch.

All examples now recommend use of process_partial_response over the legacy decorator, including a new question in the FAQ section to clarify them.

Static typing improvements in Parser

When using Mypy for applications using Pydantic models provided by Parser, you couldn't use nested models for SQS, Kinesis, and DynamoDB Streams.

This quality of life improvement also allows you to use higher-level models like Json to auto-deserialize JSON strings in common Lambda Event Source fields like SQS.body.

from aws_lambda_powertools.utilities.parser import BaseModel, validator
from aws_lambda_powertools.utilities.parser.models import (
    DynamoDBStreamChangedRecordModel,
    DynamoDBStreamRecordModel,
    KinesisDataStreamRecord,
    KinesisDataStreamRecordPayload,
    SqsRecordModel,
)
from aws_lambda_powertools.utilities.parser.types import Json, Literal


class Order(BaseModel):
    item: dict


# SQS example
class OrderSqs(SqsRecordModel):
    body: Json[Order]


# Kinesis example
class OrderKinesisPayloadRecord(KinesisDataStreamRecordPayload):
    data: Json[Order]


class OrderKinesisRecord(KinesisDataStreamRecord):
    kinesis: OrderKinesisPayloadRecord

# DynamoDB Streams example
class OrderDynamoDBChangeRecord(DynamoDBStreamChangedRecordModel):
    NewImage: Optional[OrderDynamoDB]
    OldImage: Optional[OrderDynamoDB]


class OrderDynamoDBRecord(DynamoDBStreamRecordModel):
    dynamodb: OrderDynamoDBChangeRecord

Changes

🌟New features and non-breaking changes

  • feat(batch): reduce boilerplate with process_partial_response (#2090) by @heitorlessa
  • feat(idempotency): allow custom sdk clients in DynamoDBPersistenceLayer (#2087) by @heitorlessa

📜 Documentation updates

🐛 Bug and hot fixes

  • fix(batch): handle early validation errors for pydantic models (poison pill) #2091 (#2099) by @LuckIlNe

🔧 Maintenance

This release was made possible by the following contributors:

@LuckIlNe, @dependabot, @dependabot[bot], @heitorlessa, @leandrodamascena, @michael-k and Release bot

Contributors

michael-k, heitorlessa, and 4 other contributors
Loading

v2.11.0

29 Mar 12:36
@github-actions github-actions
v2.11.0
b572f38
Compare
Choose a tag to compare
v2.11.0

Summary

This release adds support for S3 Event Notifications via EventBridge in Event Source Data Classes, and a bug fix for Feature Flags to handle falsy context values correctly.

⭐ Huge welcome to new contributors: @ajwad-shaikh and @ivica-k

image

S3 Event Notifications via EventBridge

Docs

A new use case is to process S3 Events with the newer and faster EventBridge integration. Previously, EventBridge integration with S3 would use CloudTrail leading up to a few minutes delays.

With the new integration, there is a slightly new payload that we had to support in order to ease that integration.

Thanks to @ivica-k, you can now use S3EventBridgeNotificationEvent!

from aws_lambda_powertools.utilities.data_classes import S3EventBridgeNotificationEvent, event_source
from aws_lambda_powertools.utilities.typing import LambdaContext


@event_source(data_class=S3EventBridgeNotificationEvent)
def lambda_handler(event: S3EventBridgeNotificationEvent, context: LambdaContext):
    bucket_name = event.detail.bucket.name
    file_key = event.detail.object.key
    new_storage_class = event.detail.destination_storage_class
    ...

Fixing falsy values in Feature Flags

Thanks to @ajwad-shaikh for reporting and fixing this bug (incl. @ran-isenberg), you can now handle flag variants when their context values are falsy, e.g., empty lists, empty strings, False, 0, etc.

Example that wasn't possible before this release

Flag

{
  "sales_offer": {
    "default": false,
    "rules": {
      "unpaid_users": {
        "when_match": true,
        "conditions": [ {"action": "EQUALS", "key": "is_paid", "value": 0} ]
      }
   }
}

Context

{
   "is_paid": 0,
}

Changes

🌟New features and non-breaking changes

  • feat(event_sources): support for S3 Event Notifications through EventBridge (#2024) by @ivica-k

📜 Documentation updates

  • fix(feature_flags): make test conditions deterministic (#2059) by @ajwad-shaikh
  • docs(logger): warn append_keys on not being thread-safe (#2046) by @heitorlessa
  • feat(event_sources): support for S3 Event Notifications through EventBridge (#2024) by @ivica-k

🐛 Bug and hot fixes

🔧 Maintenance

  • chore(deps-dev): bump types-requests from 2.28.11.16 to 2.28.11.17 (#2061) by @dependabot
  • chore(deps-dev): bump filelock from 3.10.4 to 3.10.7 (#2055) by @dependabot
  • chore(deps-dev): bump cfn-lint from 0.75.1 to 0.76.1 (#2056) by @dependabot
  • chore(deps-dev): bump types-python-dateutil from 2.8.19.10 to 2.8.19.11 (#2057) by @dependabot
  • chore(deps-dev): bump mypy-boto3-s3 from 1.26.97.post2 to 1.26.99 (#2054) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 9.1.3 to 9.1.4 (#2050) by @dependabot
  • chore(deps-dev): bump filelock from 3.10.2 to 3.10.4 (#2048) by @dependabot
  • chore(deps-dev): bump mypy-boto3-cloudwatch from 1.26.52 to 1.26.99 (#2049) by @dependabot
  • chore(deps-dev): bump filelock from 3.10.1 to 3.10.2 (#2045) by @dependabot
  • chore(deps-dev): bump types-requests from 2.28.11.15 to 2.28.11.16 (#2044) by @dependabot
  • chore(deps-dev): bump mypy-boto3-s3 from 1.26.97 to 1.26.97.post2 (#2043) by @dependabot
  • chore(deps-dev): bump mypy-boto3-dynamodb from 1.26.97 to 1.26.97.post1 (#2042) by @dependabot
  • chore(deps-dev): bump filelock from 3.10.0 to 3.10.1 (#2036) by @dependabot
  • chore(deps-dev): bump aws-cdk from 2.69.0 to 2.70.0 (#2039) by @dependabot
  • chore(deps-dev): bump mypy-boto3-dynamodb from 1.26.87 to 1.26.97 (#2035) by @dependabot
  • chore(deps-dev): bump mypy-boto3-s3 from 1.26.62 to 1.26.97 (#2037) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.69.0 to 2.70.0 (#2038) by @dependabot
  • chore(deps): bump pydantic from 1.10.6 to 1.10.7 (#2034) by @dependabot
  • chore(deps-dev): bump mypy-boto3-ssm from 1.26.77 to 1.26.97 (#2033) by @dependabot
  • chore(deps-dev): bump flake8-comprehensions from 3.11.0 to 3.11.1 (#2029) by @dependabot
  • chore(deps-dev): bump cfn-lint from 0.75.0 to 0.75.1 (#2027) by @dependabot
  • chore(deps-dev): bump pytest-asyncio from 0.20.3 to 0.21.0 (#2026) by @dependabot
  • feat(event_sources): support for S3 Event Notifications through EventBridge (#2024) by @ivica-k

This release was made possible by the following contributors:

@ajwad-shaikh, @dependabot, @dependabot[bot], @heitorlessa, @ivica-k, @ran-isenberg and Release bot

Contributors

heitorlessa, ivica-k, and 3 other contributors
Loading
0 Join discussion

v2.10.0

17 Mar 12:09
@release-drafter release-drafter
v2.10.0
32b6638
Compare
Choose a tag to compare
v2.10.0

Summary

This release brings support for parsing S3 Event Notifications via EventBridge and support for custom properties on the ActiveMQ event source data class.

⭐️ Huge thanks for our first-time contributors @alexaiss and @ivica-k, and also @tibbe for the typing hot-fix!

Parsing S3 event notifications via EventBridge

Docs

Amazon S3 can send event notifications with Amazon EventBridge. We added support for parsing those events in Lambda using the new S3EventNotificationEventBridgeModel.

carbon (43)

Thank you @robert-malai for capturing this feature gap!

Custom properties on ActiveMQ event

When integrating with Active MQ, Lambda supports JMS custom properties. After this change, you can also access them when using the ActiveMQ event source data class.

Changes

🌟New features and non-breaking changes

  • feat(event_sources): support for custom properties in ActiveMQEvent (#1999) by @alexaiss
  • feat(parser): support for S3 Event Notifications via EventBridge (#1982) by @ivica-k

📜 Documentation updates

  • docs(metrics): fix high-resolution metrics announcement link (#2017) by @rubenfonseca
  • feat(parser): support for S3 Event Notifications via EventBridge (#1982) by @ivica-k
  • docs(homepage): revamp install UX & share how we build Lambda Layer (#1978) by @heitorlessa

🐛 Bug and hot fixes

  • fix(build): auto-generate setup.py for legacy build tools (#2013) by @sthulb
  • fix(typing): swap NoReturn with None for methods with no return value (#2004) by @tibbe

🔧 Maintenance

  • chore(deps-dev): bump cfn-lint from 0.74.3 to 0.75.0 (#2020) by @dependabot
  • chore(deps-dev): bump coverage from 7.2.1 to 7.2.2 (#2021) by @dependabot
  • chore(deps-dev): bump filelock from 3.9.1 to 3.10.0 (#2019) by @dependabot
  • fix(build): auto-generate setup.py for legacy build tools (#2013) by @sthulb
  • chore(deps-dev): bump aws-cdk from 2.67.0 to 2.69.0 (#2010) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 9.1.2 to 9.1.3 (#2009) by @dependabot
  • chore(deps-dev): bump cfn-lint from 0.74.2 to 0.74.3 (#2008) by @dependabot
  • chore(deps-dev): bump filelock from 3.9.0 to 3.9.1 (#2006) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.68.0 to 2.69.0 (#2007) by @dependabot
  • chore(deps-dev): bump cfn-lint from 0.74.1 to 0.74.2 (#2005) by @dependabot
  • chore(deps-dev): bump mypy from 0.982 to 1.1.1 (#1985) by @dependabot
  • chore(deps-dev): bump pytest-xdist from 3.2.0 to 3.2.1 (#2000) by @dependabot
  • chore(deps-dev): bump flake8-bugbear from 23.2.13 to 23.3.12 (#2001) by @dependabot
  • chore(deps-dev): bump bandit from 1.7.4 to 1.7.5 (#1997) by @dependabot
  • chore(deps): bump docker/setup-buildx-action from 2.4.1 to 2.5.0 (#1995) by @dependabot
  • chore(deps-dev): bump mypy-boto3-secretsmanager from 1.26.49 to 1.26.89 (#1996) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 9.1.1 to 9.1.2 (#1994) by @dependabot
  • chore(deps-dev): bump mypy-boto3-dynamodb from 1.26.84 to 1.26.87 (#1993) by @dependabot
  • chore(deps): bump pydantic from 1.10.5 to 1.10.6 (#1991) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.67.0 to 2.68.0 (#1992) by @dependabot
  • chore(deps-dev): bump cfn-lint from 0.74.0 to 0.74.1 (#1988) by @dependabot
  • chore(deps): bump aws-actions/configure-aws-credentials from 1 to 2 (#1987) by @dependabot
  • chore(deps-dev): bump pytest from 7.2.1 to 7.2.2 (#1980) by @dependabot
  • chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions from 2.1.1 to 2.1.2 (#1979) by @dependabot
  • chore(deps-dev): bump types-python-dateutil from 2.8.19.9 to 2.8.19.10 (#1973) by @dependabot
  • chore(deps-dev): bump hvac from 1.0.2 to 1.1.0 (#1983) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 9.1.0 to 9.1.1 (#1984) by @dependabot
  • chore(deps-dev): bump mypy-boto3-dynamodb from 1.26.24 to 1.26.84 (#1981) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 9.0.15 to 9.1.0 (#1976) by @dependabot
  • chore(deps-dev): bump cfn-lint from 0.67.0 to 0.74.0 (#1974) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.66.1 to 2.67.0 (#1977) by @dependabot

This release was made possible by the following contributors:

@alexaiss, @dependabot, @dependabot[bot], @heitorlessa, @ivica-k, @rubenfonseca, @sthulb, @tibbe and Release bot

Contributors

rubenfonseca, tibbe, and 6 other contributors
Loading

2.9.1

02 Mar 05:42
@release-drafter release-drafter
v2.9.1
8358af0
Compare
Choose a tag to compare
2.9.1

Summary

This patch release reverts a regression in Idempotency when using static_pk_value for custom composite keys in DynamoDB Persistence Layer - huge thanks to @Tankanow for reporting and fixing it.

⭐ Big thanks to new contributors: @KeltonKarboviak on inconsistencies on Parameters docs (#1966), and @cibinmathew on upgrading GitHub Actions set-output command in our workflows.

Changes

📜 Documentation updates

🐛 Bug and hot fixes

  • fix(idempotency): revert dict mutation that impacted static_pk_value feature (#1970) by @Tankanow

🔧 Maintenance

  • chore(parser): add workaround to make API GW test button work (#1971) by @heitorlessa
  • chore(deps-dev): bump mypy-boto3-lambda from 1.26.55 to 1.26.80 (#1967) by @dependabot
  • chore(ci): replace deprecated set-output commands (#1957) by @cibinmathew
  • chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions from 2.1.0 to 2.1.1 (#1958) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 9.0.14 to 9.0.15 (#1959) by @dependabot
  • chore(deps-dev): bump types-python-dateutil from 2.8.19.8 to 2.8.19.9 (#1960) by @dependabot
  • chore(deps): bump fastjsonschema from 2.16.2 to 2.16.3 (#1961) by @dependabot
  • chore(deps-dev): bump coverage from 7.2.0 to 7.2.1 (#1963) by @dependabot
  • chore(deps-dev): bump types-requests from 2.28.11.14 to 2.28.11.15 (#1962) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.66.0 to 2.66.1 (#1954) by @dependabot
  • chore(deps-dev): bump coverage from 7.1.0 to 7.2.0 (#1951) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 9.0.13 to 9.0.14 (#1952) by @dependabot
  • chore(deps-dev): bump mypy-boto3-ssm from 1.26.43 to 1.26.77 (#1949) by @dependabot
  • chore(deps): bump release-drafter/release-drafter from 5.22.0 to 5.23.0 (#1947) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.65.0 to 2.66.0 (#1948) by @dependabot
  • chore(deps-dev): bump types-requests from 2.28.11.13 to 2.28.11.14 (#1946) by @dependabot
  • chore(deps-dev): bump types-python-dateutil from 2.8.19.7 to 2.8.19.8 (#1945) by @dependabot

This release was made possible by the following contributors:

@KeltonKarboviak, @cibinmathew, @Tankanow, @dependabot, @dependabot[bot], @heitorlessa and Release bot

Contributors

Tankanow, heitorlessa, and 3 other contributors
Loading

v2.9.0

21 Feb 12:49
@release-drafter release-drafter
v2.9.0
051b101
Compare
Choose a tag to compare
v2.9.0

Summary

This release contains a new feature on Batch Processing, along with enhancements and bug fixes to Logger and Feature Flags utilities.

⭐ Huge welcome to the new contributors: @royygael, @iago1460

SQS FIFO Batch Processing

Docs

You can now use SQS FIFO queues with the Batch Processing utility. We stop processing messages after the first failure and return all failed and unprocessed messages natively. This helps preserve the ordering of messages in your queue.

carbon (38)

⭐ Thanks to @whoDoneItAgain for reporting this feature gap.

Logger improvements

We now allow exception and exception_name attributes as extra fields in all logging levels. Previously, we dropped those attributes in non-error logging levels (e.g: INFO).

⭐ Thanks to @iago1460 for reporting this bug and submitting the fix!

New community content

Finally, we have two new community posts on Feature Flags from @ran-isenberg:

  • How CyberArk uses Powertools to implement Feature Flags with AWS AppConfig: link
  • A talk about the added value of using feature flags as part of your CI/CD process and how AWS Lambda Powertools can help: link

Changes

🌟New features and non-breaking changes

  • feat(batch): add support to SQS FIFO queues (SqsFifoPartialProcessor) (#1934) by @rubenfonseca

📜 Documentation updates

🐛 Bug and hot fixes

  • fix(metrics): clarify no-metrics user warning (#1935) by @rubenfonseca
  • fix(logger): support exception and exception_name fields at any log level (#1930) by @iago1460
  • fix(feature-flags): revert RuleAction Enum inheritance on str (#1910) by @royygael

🔧 Maintenance

  • chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions from 2.0.5 to 2.1.0 (#1943) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 9.0.12 to 9.0.13 (#1944) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.64.0 to 2.65.0 (#1938) by @dependabot
  • chore(deps-dev): bump types-python-dateutil from 2.8.19.6 to 2.8.19.7 (#1932) by @dependabot
  • chore(deps-dev): bump types-requests from 2.28.11.12 to 2.28.11.13 (#1933) by @dependabot
  • chore(deps): bump pydantic from 1.10.4 to 1.10.5 (#1931) by @dependabot
  • chore(deps-dev): bump mypy-boto3-appconfig from 1.26.63 to 1.26.71 (#1928) by @dependabot
  • chore(deps-dev): bump flake8-bugbear from 23.1.20 to 23.2.13 (#1924) by @dependabot
  • chore(deps-dev): bump mypy-boto3-appconfigdata from 1.26.0.post1 to 1.26.70 (#1925) by @dependabot

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @heitorlessa, @iago1460, @leandrodamascena, @royygael, @rubenfonseca and Release bot

Contributors

rubenfonseca, heitorlessa, and 6 other contributors
Loading

v2.8.0

10 Feb 17:42
@release-drafter release-drafter
v2.8.0
43feb1e
Compare
Choose a tag to compare
v2.8.0

Summary

This release is packed with new features and enhancements, specifically to Batch Processing, Metrics and Idempotency utility.

⭐ Huge welcome to new contributors: @BakasuraRCE, @prudnikov, @kavichu, @marcraminv

image

High-concurrency async processor

Docs

Thanks to @BakasuraRCE, we’re adding a new specialized processor for high concurrency jobs that can process thousands of messages sub-second (thanks to @sthulb for load testing) - AsyncBatchProcessor and async_batch_processor.

These will use Python’s asyncio.gather to call your record handler with all records at the same time. During load testing, we’ve managed to process 5500 messages in approximately 300ms compared to 1.2 seconds with BatchProcessor synchronous counterpart.

1-second resolution metrics

Docs

You can now create high-resolution (1s) metrics that are common for use cases like IoT, telemetry, time-series, real-time incident, etc. Previously, metrics were only created with 60-second resolution. This is an opt-in feature.

image

Inheriting dimensions in single_metric

Docs

Thanks to @prudnikov, you can now optionally set default dimensions for single metrics. These are individual application or operational metrics that despite being isolated from a larger set of metrics, they do share the same set of dimensions (avoid repetition, sparse metrics).

image

Idempotent_function decorator is now thread-safe

Thanks to @mploski, any Python function decorated with idempotent_function can now safely be called from threads. Previously, we used to create higher-level AWS SDK clients which didn’t support threading - calling functions decorated back then could result in unforeseen side-effects.

Example

import os
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from threading import current_thread

from aws_lambda_powertools.utilities.idempotency import (
    DynamoDBPersistenceLayer,
    idempotent_function,
)

TABLE_NAME = os.getenv("IdempotencyTable", "")
persistence_layer = DynamoDBPersistenceLayer(table_name=TABLE_NAME)
threads_count = 2


@idempotent_function(persistence_store=persistence_layer, data_keyword_argument="record")
def record_handler(record):
    time_now = time.time()
    return {"thread_name": current_thread().name, "time": str(time_now)}


def lambda_handler(event, context):
    with ThreadPoolExecutor(max_workers=threads_count) as executor:
        futures = [executor.submit(record_handler, **{"record": event}) for _ in range(threads_count)]

    return [
        {"state": future._state, "exception": future.exception(), "output": future.result()}
        for future in as_completed(futures)
    ]

Changes

🌟New features and non-breaking changes

  • feat(batch): add async_batch_processor for concurrent processing (#1724) by @BakasuraRCE
  • feat(metrics): add default_dimensions to single_metric (#1880) by @prudnikov

📜 Documentation updates

  • docs(homepage): Replace poetry command to add group parameter (#1917) by @marcraminv
  • feat(metrics) - add support for high resolution metrics (#1915) by @leandrodamascena
  • feat(batch): add async_batch_processor for concurrent processing (#1724) by @BakasuraRCE
  • fix(idempotency): make idempotent_function decorator thread safe (#1899) by @mploski
  • docs(idempotency): add IAM permissions section (#1902) by @leandrodamascena
  • docs(homepage): set url for end-of-support in announce block (#1893) by @kavichu
  • feat(metrics): add default_dimensions to single_metric (#1880) by @prudnikov

🐛 Bug and hot fixes

  • fix(idempotency): make idempotent_function decorator thread safe (#1899) by @mploski

🔧 Maintenance

  • feat(batch): add async_batch_processor for concurrent processing (#1724) by @BakasuraRCE
  • chore(deps-dev): bump mkdocs-material from 9.0.11 to 9.0.12 (#1919) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.63.2 to 2.64.0 (#1918) by @dependabot
  • chore(pypi): add new links to Pypi package homepage (#1912) by @leandrodamascena
  • chore(deps-dev): bump types-requests from 2.28.11.8 to 2.28.11.12 (#1906) by @dependabot
  • chore(deps-dev): bump pytest-xdist from 3.1.0 to 3.2.0 (#1905) by @dependabot
  • chore(deps): bump docker/setup-buildx-action from 2.4.0 to 2.4.1 (#1903) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.63.0 to 2.63.2 (#1904) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 9.0.10 to 9.0.11 (#1896) by @dependabot
  • chore(deps-dev): bump mypy-boto3-appconfig from 1.26.0.post1 to 1.26.63 (#1895) by @dependabot
  • docs(homepage): set url for end-of-support in announce block (#1893) by @kavichu
  • chore(deps-dev): bump mkdocs-material from 9.0.9 to 9.0.10 (#1888) by @dependabot
  • chore(deps-dev): bump mypy-boto3-s3 from 1.26.58 to 1.26.62 (#1889) by @dependabot
  • chore(deps-dev): bump black from 22.12.0 to 23.1.0 (#1886) by @dependabot
  • chore(deps-dev): bump aws-cdk-lib from 2.62.2 to 2.63.0 (#1887) by @dependabot

This release was made possible by the following contributors:

@BakasuraRCE, @dependabot, @dependabot[bot], @heitorlessa, @kavichu, @leandrodamascena, @marcraminv, @mploski, @prudnikov and Release bot

Contributors

mploski, prudnikov, and 6 other contributors
Loading
0 Join discussion