Skip to content

improv(idempotency): expose record status & expiry config + make DynamoDB Client optional #1679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/utilities/idempotency.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,10 @@ classDiagram
## Getting started

### Installation

Install the library in your project
```shell
npm i @aws-lambda-powertools/idempotency @aws-sdk/client-dynamodb
npm i @aws-lambda-powertools/idempotency @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb
```

While we support Amazon DynamoDB as a persistance layer out of the box, you need to bring your own AWS SDK for JavaScript v3 DynamoDB client.
Expand Down
20 changes: 9 additions & 11 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion packages/idempotency/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The current implementation provides a persistence layer for Amazon DynamoDB, whi
To get started, install the library by running:

```sh
npm install @aws-lambda-powertools/idempotency
npm i @aws-lambda-powertools/idempotency @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb
```

Next, review the IAM permissions attached to your AWS Lambda function and make sure you allow the [actions detailed](https://docs.powertools.aws.dev/lambda/typescript/latest/utilities/idempotency/#iam-permissions) in the documentation of the utility.
Expand Down
20 changes: 18 additions & 2 deletions packages/idempotency/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -80,10 +80,25 @@
},
"dependencies": {
"@aws-lambda-powertools/commons": "^1.12.1",
"@aws-sdk/lib-dynamodb": "^3.231.0",
"@aws-sdk/util-base64-node": "^3.209.0",
"jmespath": "^0.16.0"
},
"peerDependencies": {
"@aws-sdk/client-dynamodb": ">=3.360.0",
"@aws-sdk/lib-dynamodb": ">=3.410.0",
"@middy/core": ">=3.x <4.x"
},
"peerDependenciesMeta": {
"@aws-sdk/client-dynamodb": {
"optional": true
},
"@aws-sdk/lib-dynamodb": {
"optional": true
},
"@middy/core": {
"optional": true
}
},
"keywords": [
"aws",
"lambda",
Expand All @@ -95,8 +110,9 @@
"devDependencies": {
"@aws-lambda-powertools/testing-utils": "file:../testing",
"@aws-sdk/client-dynamodb": "^3.360.0",
"@aws-sdk/lib-dynamodb": "^3.410.0",
"@types/jmespath": "^0.15.0",
"aws-sdk-client-mock": "^2.2.0",
"aws-sdk-client-mock-jest": "^2.2.0"
}
}
}
2 changes: 1 addition & 1 deletion packages/idempotency/src/IdempotencyHandler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import type {
MiddyLikeRequest,
} from '@aws-lambda-powertools/commons';
import type { AnyFunction, IdempotencyHandlerOptions } from './types';
import { IdempotencyRecordStatus } from './types';
import { IdempotencyRecordStatus } from './constants';
import {
IdempotencyAlreadyInProgressError,
IdempotencyInconsistentStateError,
Expand Down
15 changes: 14 additions & 1 deletion packages/idempotency/src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,17 @@
*/
const MAX_RETRIES = 2;

export { MAX_RETRIES };
/**
* Idempotency record status.
*
* A record is created when a request is received. The status is set to `INPROGRESS` and the request is processed.
* After the request is processed, the status is set to `COMPLETED`. If the request is not processed within the
* `inProgressExpiryTimestamp`, the status is set to `EXPIRED`.
*/
const IdempotencyRecordStatus = {
INPROGRESS: 'INPROGRESS',
COMPLETED: 'COMPLETED',
EXPIRED: 'EXPIRED',
} as const;

export { IdempotencyRecordStatus, MAX_RETRIES };
1 change: 1 addition & 0 deletions packages/idempotency/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
export * from './errors';
export * from './IdempotencyConfig';
export * from './makeIdempotent';
export { IdempotencyRecordStatus } from './constants';
12 changes: 11 additions & 1 deletion packages/idempotency/src/persistence/BasePersistenceLayer.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { createHash, Hash } from 'node:crypto';
import { search } from 'jmespath';
import type { BasePersistenceLayerOptions } from '../types';
import { IdempotencyRecordStatus } from '../types';
import { IdempotencyRecordStatus } from '../constants';
import { EnvironmentVariablesService } from '../config';
import { IdempotencyRecord } from './IdempotencyRecord';
import { BasePersistenceLayerInterface } from './BasePersistenceLayerInterface';
Expand Down Expand Up @@ -91,6 +91,13 @@ abstract class BasePersistenceLayer implements BasePersistenceLayerInterface {
this.deleteFromCache(idempotencyRecord.idempotencyKey);
}

/**
* Retrieve the number of seconds that records will be kept in the persistence store
*/
public getExpiresAfterSeconds(): number {
return this.expiresAfterSeconds;
}

/**
* Retrieves idempotency key for the provided data and fetches data for that key from the persistence store
*
Expand All @@ -113,6 +120,9 @@ abstract class BasePersistenceLayer implements BasePersistenceLayerInterface {
return record;
}

/**
* Check whether payload validation is enabled or not
*/
public isPayloadValidationEnabled(): boolean {
return this.payloadValidationEnabled;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import {
IdempotencyItemAlreadyExistsError,
IdempotencyItemNotFoundError,
} from '../errors';
import { IdempotencyRecordStatus } from '../types';
import { IdempotencyRecordStatus } from '../constants';
import type { DynamoDBPersistenceOptions } from '../types';
import {
AttributeValue,
Expand Down
13 changes: 8 additions & 5 deletions packages/idempotency/src/persistence/IdempotencyRecord.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import type { JSONValue } from '@aws-lambda-powertools/commons';
import type { IdempotencyRecordOptions } from '../types';
import { IdempotencyRecordStatus } from '../types';
import type {
IdempotencyRecordOptions,
IdempotencyRecordStatusValue,
} from '../types';
import { IdempotencyRecordStatus } from '../constants';
import { IdempotencyInvalidStatusError } from '../errors';

/**
Expand Down Expand Up @@ -31,10 +34,10 @@ class IdempotencyRecord {
/**
* The idempotency record status can be COMPLETED, IN_PROGRESS or EXPIRED.
* We check the status during idempotency processing to make sure we don't process an expired record and handle concurrent requests.
* @link {IdempotencyRecordStatus}
* @link {IdempotencyRecordStatusValue}
* @private
*/
private status: IdempotencyRecordStatus;
private status: IdempotencyRecordStatusValue;

public constructor(config: IdempotencyRecordOptions) {
this.idempotencyKey = config.idempotencyKey;
Expand All @@ -56,7 +59,7 @@ class IdempotencyRecord {
* Get the status of the record.
* @throws {IdempotencyInvalidStatusError} If the status is not a valid status.
*/
public getStatus(): IdempotencyRecordStatus {
public getStatus(): IdempotencyRecordStatusValue {
if (this.isExpired()) {
return IdempotencyRecordStatus.EXPIRED;
} else if (Object.values(IdempotencyRecordStatus).includes(this.status)) {
Expand Down
13 changes: 4 additions & 9 deletions packages/idempotency/src/types/IdempotencyRecord.ts
Original file line number Diff line number Diff line change
@@ -1,21 +1,16 @@
import type { JSONValue } from '@aws-lambda-powertools/commons';
import { IdempotencyRecordStatus } from '../constants';

const IdempotencyRecordStatus = {
INPROGRESS: 'INPROGRESS',
COMPLETED: 'COMPLETED',
EXPIRED: 'EXPIRED',
} as const;

type IdempotencyRecordStatus =
type IdempotencyRecordStatusValue =
(typeof IdempotencyRecordStatus)[keyof typeof IdempotencyRecordStatus];

type IdempotencyRecordOptions = {
idempotencyKey: string;
status: IdempotencyRecordStatus;
status: IdempotencyRecordStatusValue;
expiryTimestamp?: number;
inProgressExpiryTimestamp?: number;
responseData?: JSONValue;
payloadHash?: string;
};

export { IdempotencyRecordStatus, IdempotencyRecordOptions };
export { IdempotencyRecordStatusValue, IdempotencyRecordOptions };
3 changes: 1 addition & 2 deletions packages/idempotency/tests/unit/IdempotencyHandler.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,10 @@ import {
IdempotencyItemAlreadyExistsError,
IdempotencyPersistenceLayerError,
} from '../../src/errors';
import { IdempotencyRecordStatus } from '../../src/types';
import { IdempotencyRecord } from '../../src/persistence';
import { IdempotencyHandler } from '../../src/IdempotencyHandler';
import { IdempotencyConfig } from '../../src/';
import { MAX_RETRIES } from '../../src/constants';
import { MAX_RETRIES, IdempotencyRecordStatus } from '../../src/constants';
import { PersistenceLayerTestClass } from '../helpers/idempotencyUtils';

const mockFunctionToMakeIdempotent = jest.fn();
Expand Down
7 changes: 3 additions & 4 deletions packages/idempotency/tests/unit/makeHandlerIdempotent.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,8 @@
* @group unit/idempotency/makeHandlerIdempotent
*/
import { makeHandlerIdempotent } from '../../src/middleware';
import { helloworldContext as dummyContext } from '../../../commons/src/samples/resources/contexts';
import { Custom as dummyEvent } from '../../../commons/src/samples/resources/events';
import { IdempotencyRecordStatus } from '../../src/types';
import { helloworldContext as dummyContext } from '@aws-lambda-powertools/commons/lib/samples/resources/contexts';
import { Custom as dummyEvent } from '@aws-lambda-powertools/commons/lib/samples/resources/events';
import { IdempotencyRecord } from '../../src/persistence';
import {
IdempotencyInconsistentStateError,
Expand All @@ -15,7 +14,7 @@ import {
} from '../../src/errors';
import { IdempotencyConfig } from '../../src/';
import middy from '@middy/core';
import { MAX_RETRIES } from '../../src/constants';
import { MAX_RETRIES, IdempotencyRecordStatus } from '../../src/constants';
import { PersistenceLayerTestClass } from '../helpers/idempotencyUtils';
import type { Context } from 'aws-lambda';

Expand Down
7 changes: 3 additions & 4 deletions packages/idempotency/tests/unit/makeIdempotent.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,15 @@
*/
import { IdempotencyRecord } from '../../src/persistence';
import { makeIdempotent } from '../../src';
import { IdempotencyRecordStatus } from '../../src/types';
import {
IdempotencyInconsistentStateError,
IdempotencyItemAlreadyExistsError,
IdempotencyPersistenceLayerError,
} from '../../src/errors';
import { IdempotencyConfig } from '../../src';
import { helloworldContext as dummyContext } from '../../../commons/src/samples/resources/contexts';
import { Custom as dummyEvent } from '../../../commons/src/samples/resources/events';
import { MAX_RETRIES } from '../../src/constants';
import { helloworldContext as dummyContext } from '@aws-lambda-powertools/commons/lib/samples/resources/contexts';
import { Custom as dummyEvent } from '@aws-lambda-powertools/commons/lib/samples/resources/events';
import { MAX_RETRIES, IdempotencyRecordStatus } from '../../src/constants';
import { PersistenceLayerTestClass } from '../helpers/idempotencyUtils';
import type { Context } from 'aws-lambda';

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
IdempotencyValidationError,
} from '../../../src/errors';
import type { IdempotencyConfigOptions } from '../../../src/types';
import { IdempotencyRecordStatus } from '../../../src/types';
import { IdempotencyRecordStatus } from '../../../src';

jest.mock('node:crypto', () => ({
createHash: jest.fn().mockReturnValue({
Expand Down Expand Up @@ -462,4 +462,17 @@ describe('Class: BasePersistenceLayer', () => {
);
});
});

describe('Method: getExpiresAfterSeconds', () => {
it('returns the configured value', () => {
// Prepare
const persistenceLayer = new PersistenceLayerTestClass();

// Act
const result = persistenceLayer.getExpiresAfterSeconds();

// Assess
expect(result).toBe(3600);
});
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
} from '../../../src/errors';
import { IdempotencyRecord } from '../../../src/persistence';
import type { DynamoDBPersistenceOptions } from '../../../src/types';
import { IdempotencyRecordStatus } from '../../../src/types';
import { IdempotencyRecordStatus } from '../../../src';
import {
DynamoDBClient,
DynamoDBServiceException,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
*/
import { IdempotencyInvalidStatusError } from '../../../src/errors';
import { IdempotencyRecord } from '../../../src/persistence';
import { IdempotencyRecordStatus } from '../../../src/types';
import { IdempotencyRecordStatus } from '../../../src';
import type { IdempotencyRecordStatusValue } from '../../../src/types';

const mockIdempotencyKey = '123';
const mockData = undefined;
Expand All @@ -28,7 +29,7 @@ describe('Given an INPROGRESS record that has already expired', () => {
});
});
describe('When checking the status of the idempotency record', () => {
let resultingStatus: IdempotencyRecordStatus;
let resultingStatus: IdempotencyRecordStatusValue;
beforeEach(() => {
resultingStatus = idempotencyRecord.getStatus();
});
Expand Down Expand Up @@ -75,7 +76,7 @@ describe('Given an idempotency record that has a status not in the IdempotencyRe
Date.now = jest.fn(() => mockNowBeforeExiryTime);
idempotencyRecord = new IdempotencyRecord({
idempotencyKey: mockIdempotencyKey,
status: 'NOT_A_STATUS' as IdempotencyRecordStatus,
status: 'NOT_A_STATUS' as IdempotencyRecordStatusValue,
expiryTimestamp: expiryTimeAfterNow,
inProgressExpiryTimestamp: mockInProgressExpiry,
responseData: mockData,
Expand Down