Skip to content

Make monomorphic APIs the default by swapping the changing convention. #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 12, 2024
Merged

Make monomorphic APIs the default by swapping the changing convention. #22

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 82 additions & 60 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,43 @@ The `deleteItem` operation will succeed even if the specified row doesn't exist

## Queries and Batch

All or a subset of the rows from a partition can be retrieved using a query-
All or a subset of the rows from a partition can be retrieved using a query.

```swift
let (queryItems, nextPageToken): ([StandardTypedDatabaseItem<TestTypeA>], String?) =
try await table.query(forPartitionKey: "partitionId",
sortKeyCondition: nil,
limit: 100,
exclusiveStartKey: exclusiveStartKey)

for databaseItem in queryItems {
...
}
```

1. The sort key condition can restrict the query to a subset of the partition rows. A nil condition will return all rows in the partition.
2. The `query` operation will fail if any of the rows being returned are not of type `TestTypeA`.
3. The optional String returned by the `query` operation can be used as the `exclusiveStartKey` in another request to retrieve the next "page" of results from DynamoDB.
4. There is an overload of the `query` operation that doesn't accept a `limit` or `exclusiveStartKey`. This overload will internally handle the API pagination, making multiple calls to DynamoDB if necessary.

There is also an equivalent `getItems` call that uses DynamoDB's BatchGetItem API-

```swift
let batch: [StandardCompositePrimaryKey: StandardTypedDatabaseItem<TestTypeA>]
= try await table.getItems(forKeys: [key1, key2])

guard let retrievedDatabaseItem1 = batch[key1] else {
...
}

guard let retrievedDatabaseItem2 = batch[key2] else {
...
}
```

## Polymorphic Queries and Batch

In addition to the `query` operation, there is a more complex API that allows retrieval of multiple rows that have different types.

```swift
enum TestPolymorphicOperationReturnType: PolymorphicOperationReturnType {
Expand All @@ -186,10 +222,10 @@ enum TestPolymorphicOperationReturnType: PolymorphicOperationReturnType {
}

let (queryItems, nextPageToken): ([TestPolymorphicOperationReturnType], String?) =
try await table.query(forPartitionKey: partitionId,
sortKeyCondition: nil,
limit: 100,
exclusiveStartKey: exclusiveStartKey)
try await table.polymorphicQuery(forPartitionKey: partitionId,
sortKeyCondition: nil,
limit: 100,
exclusiveStartKey: exclusiveStartKey)

for item in queryItems {
switch item {
Expand All @@ -200,15 +236,12 @@ for item in queryItems {
}
```

1. The sort key condition can restrict the query to a subset of the partition rows. A nil condition will return all rows in the partition.
2. The `query` operation will fail if the partition contains rows that are not specified in the output `PolymorphicOperationReturnType` type.
3. The optional String returned by the `query` operation can be used as the `exclusiveStartKey` in another request to retrieve the next "page" of results from DynamoDB.
4. There is an overload of the `query` operation that doesn't accept a `limit` or `exclusiveStartKey`. This overload will internally handle the API pagination, making multiple calls to DynamoDB if necessary.
1. The `polymorphicQuery` operation will fail if any of the rows being returned are not specified in the output `PolymorphicOperationReturnType` type.

A similar operation utilises DynamoDB's BatchGetItem API, returning items in a dictionary keyed by the provided `CompositePrimaryKey` instance-

```swift
let batch: [StandardCompositePrimaryKey: TestPolymorphicOperationReturnType] = try await table.getItems(forKeys: [key1, key2])
let batch: [StandardCompositePrimaryKey: TestPolymorphicOperationReturnType] = try await table.polymorphicGetItems(forKeys: [key1, key2])

guard case .testTypeA(let retrievedDatabaseItem1) = batch[key1] else {
...
Expand All @@ -221,37 +254,6 @@ guard case .testTypeB(let retrievedDatabaseItem2) = batch[key2] else {

This operation will automatically handle retrying unprocessed items (with exponential backoff) if the table doesn't have the capacity during the initial request.

## Monomorphic Queries

In addition to the `query` operation, there is a seperate set of operations that provide a simpler API when a query will only retrieve rows of the same type.

```swift
let (queryItems, nextPageToken): ([StandardTypedDatabaseItem<TestTypeA>], String?) =
try await table.monomorphicQuery(forPartitionKey: "partitionId",
sortKeyCondition: nil,
limit: 100,
exclusiveStartKey: exclusiveStartKey)

for databaseItem in queryItems {
...
}
```

There is also an equivalent `monomorphicGetItems` DynamoDB's BatchGetItem API-

```swift
let batch: [StandardCompositePrimaryKey: StandardTypedDatabaseItem<TestTypeA>]
= try await table.monomorphicGetItems(forKeys: [key1, key2])

guard let retrievedDatabaseItem1 = batch[key1] else {
...
}

guard let retrievedDatabaseItem2 = batch[key2] else {
...
}
```

## Queries on Indices

There are two mechanisms for querying on indices depending on if you have any projected attributes.
Expand All @@ -273,6 +275,20 @@ public struct GSI1PrimaryKeyAttributes: PrimaryKeyAttributes {
}
}

let (queryItems, nextPageToken): ([TypedDatabaseItem<GSI1PrimaryKeyAttributes, TestTypeA>], String?) =
try await table.query(forPartitionKey: "partitionId",
sortKeyCondition: nil,
limit: 100,
exclusiveStartKey: exclusiveStartKey)

for databaseItem in queryItems {
...
}
```

and similarly for polymorphic queries-

```swift
enum TestPolymorphicOperationReturnType: PolymorphicOperationReturnType {
typealias AttributesType = GSI1PrimaryKeyAttributes

Expand All @@ -286,7 +302,7 @@ enum TestPolymorphicOperationReturnType: PolymorphicOperationReturnType {
}

let (queryItems, nextPageToken): ([TestPolymorphicOperationReturnType], String?) =
try await table.query(forPartitionKey: partitionId,
try await table.polymorphicQuery(forPartitionKey: partitionId,
sortKeyCondition: nil,
limit: 100,
exclusiveStartKey: exclusiveStartKey)
Expand All @@ -300,20 +316,6 @@ for item in queryItems {
}
```

and similarly for monomorphic queries-

```swift
let (queryItems, nextPageToken): ([TypedDatabaseItem<GSI1PrimaryKeyAttributes, TestTypeA>], String?) =
try await table.monomorphicQuery(forPartitionKey: "partitionId",
sortKeyCondition: nil,
limit: 100,
exclusiveStartKey: exclusiveStartKey)

for databaseItem in queryItems {
...
}
```

### Using No Projected Attributes

To simply query a partition on an index that has no projected attributes, you can use the `DynamoDBCompositePrimaryKeysProjection` protocol and conforming types like ` AWSDynamoDBCompositePrimaryKeysProjection`. This type is created using a generator class in the same way as the primary table type-
Expand Down Expand Up @@ -347,6 +349,19 @@ You can write multiple database rows using either a bulk or [transaction](https:

```swift
typealias TestTypeAWriteEntry = StandardWriteEntry<TestTypeA>

let entryList: [TestTypeAWriteEntry] = [
.insert(new: databaseItem1),
.insert(new: databaseItem2)
]

try await table.bulkWrite(entryList)
//try await table.transactWrite(entryList) <<-- When implemented
```

and similarly for polymorphic queries-

```swift
typealias TestTypeBWriteEntry = StandardWriteEntry<TestTypeB>

enum TestPolymorphicWriteEntry: PolymorphicWriteEntry {
Expand All @@ -365,17 +380,24 @@ enum TestPolymorphicWriteEntry: PolymorphicWriteEntry {

let entryList: [TestPolymorphicWriteEntry] = [
.testTypeA(.insert(new: databaseItem1)),
.testTypeB(.insert(new: databaseItem2))
.testTypeB(.insert(new: databaseItem3))
]

try await table.bulkWrite(entryList)
try await table.transactWrite(entryList)
try await table.polymorphicBulkWrite(entryList)
try await table.polymorphicTransactWrite(entryList)
```

For transactions, you can additionally specify a set of constraints to be part of the transaction-

```swift
typealias TestTypeAStandardTransactionConstraintEntry = StandardTransactionConstraintEntry<TestTypeA>

// Update when `transactWrite` API implemented
```

and similarly for polymorphic queries-

```swift
typealias TestTypeBStandardTransactionConstraintEntry = StandardTransactionConstraintEntry<TestTypeB>

enum TestPolymorphicTransactionConstraintEntry: PolymorphicTransactionConstraintEntry {
Expand All @@ -397,7 +419,7 @@ let constraintList: [TestPolymorphicTransactionConstraintEntry] = [
.testTypeB(.required(existing: databaseItem4))
]

try await table.transactWrite(entryList, constraints: constraintList)
try await table.polymorphicTransactWrite(entryList, constraints: constraintList)
```

Both the `PolymorphicWriteEntry` and `PolymorphicTransactionConstraintEntry` conforming types can
Expand Down
106 changes: 53 additions & 53 deletions Sources/DynamoDBTables/AWSDynamoDBCompositePrimaryKeyTable+DynamoDBTableAsync.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,36 +96,36 @@ public extension AWSDynamoDBCompositePrimaryKeyTable {
_ = try await self.dynamodb.deleteItem(input: deleteItemInput)
}

func query<ReturnedType: PolymorphicOperationReturnType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
consistentRead: Bool) async throws
func polymorphicQuery<ReturnedType: PolymorphicOperationReturnType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
consistentRead: Bool) async throws
-> [ReturnedType]
{
try await self.partialQuery(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
exclusiveStartKey: nil,
consistentRead: consistentRead)
try await self.polymorphicPartialQuery(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
exclusiveStartKey: nil,
consistentRead: consistentRead)
}

// function to return a future with the results of a query call and all future paginated calls
private func partialQuery<ReturnedType: PolymorphicOperationReturnType>(
private func polymorphicPartialQuery<ReturnedType: PolymorphicOperationReturnType>(
forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
exclusiveStartKey: String?,
consistentRead: Bool) async throws -> [ReturnedType]
{
let paginatedItems: ([ReturnedType], String?) =
try await query(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
limit: nil,
scanIndexForward: true,
exclusiveStartKey: exclusiveStartKey,
consistentRead: consistentRead)
try await polymorphicQuery(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
limit: nil,
scanIndexForward: true,
exclusiveStartKey: exclusiveStartKey,
consistentRead: consistentRead)

// if there are more items
if let lastEvaluatedKey = paginatedItems.1 {
// returns a future with all the results from all later paginated calls
let partialResult: [ReturnedType] = try await self.partialQuery(
let partialResult: [ReturnedType] = try await self.polymorphicPartialQuery(
forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
exclusiveStartKey: lastEvaluatedKey,
Expand All @@ -139,27 +139,27 @@ public extension AWSDynamoDBCompositePrimaryKeyTable {
}
}

func query<ReturnedType: PolymorphicOperationReturnType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
limit: Int?,
exclusiveStartKey: String?,
consistentRead: Bool) async throws
func polymorphicQuery<ReturnedType: PolymorphicOperationReturnType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
limit: Int?,
exclusiveStartKey: String?,
consistentRead: Bool) async throws
-> (items: [ReturnedType], lastEvaluatedKey: String?)
{
try await self.query(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
limit: limit,
scanIndexForward: true,
exclusiveStartKey: exclusiveStartKey,
consistentRead: consistentRead)
try await self.polymorphicQuery(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
limit: limit,
scanIndexForward: true,
exclusiveStartKey: exclusiveStartKey,
consistentRead: consistentRead)
}

func query<ReturnedType: PolymorphicOperationReturnType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
limit: Int?,
scanIndexForward: Bool,
exclusiveStartKey: String?,
consistentRead: Bool) async throws
func polymorphicQuery<ReturnedType: PolymorphicOperationReturnType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
limit: Int?,
scanIndexForward: Bool,
exclusiveStartKey: String?,
consistentRead: Bool) async throws
-> (items: [ReturnedType], lastEvaluatedKey: String?)
{
let queryInput = try AWSDynamoDB.QueryInput.forSortKeyCondition(partitionKey: partitionKey, targetTableName: targetTableName,
Expand Down Expand Up @@ -230,36 +230,36 @@ public extension AWSDynamoDBCompositePrimaryKeyTable {
}
}

func monomorphicQuery<AttributesType, ItemType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
consistentRead: Bool) async throws
func query<AttributesType, ItemType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
consistentRead: Bool) async throws
-> [TypedDatabaseItem<AttributesType, ItemType>]
{
try await self.monomorphicPartialQuery(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
exclusiveStartKey: nil,
consistentRead: consistentRead)
try await self.partialQuery(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
exclusiveStartKey: nil,
consistentRead: consistentRead)
}

// function to return a future with the results of a query call and all future paginated calls
private func monomorphicPartialQuery<AttributesType, ItemType>(
private func partialQuery<AttributesType, ItemType>(
forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
exclusiveStartKey: String?,
consistentRead: Bool) async throws -> [TypedDatabaseItem<AttributesType, ItemType>]
{
let paginatedItems: ([TypedDatabaseItem<AttributesType, ItemType>], String?) =
try await monomorphicQuery(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
limit: nil,
scanIndexForward: true,
exclusiveStartKey: nil,
consistentRead: consistentRead)
try await query(forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
limit: nil,
scanIndexForward: true,
exclusiveStartKey: nil,
consistentRead: consistentRead)

// if there are more items
if let lastEvaluatedKey = paginatedItems.1 {
// returns a future with all the results from all later paginated calls
let partialResult: [TypedDatabaseItem<AttributesType, ItemType>] = try await self.monomorphicPartialQuery(
let partialResult: [TypedDatabaseItem<AttributesType, ItemType>] = try await self.partialQuery(
forPartitionKey: partitionKey,
sortKeyCondition: sortKeyCondition,
exclusiveStartKey: lastEvaluatedKey,
Expand All @@ -273,12 +273,12 @@ public extension AWSDynamoDBCompositePrimaryKeyTable {
}
}

func monomorphicQuery<AttributesType, ItemType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
limit: Int?,
scanIndexForward: Bool,
exclusiveStartKey: String?,
consistentRead: Bool) async throws
func query<AttributesType, ItemType>(forPartitionKey partitionKey: String,
sortKeyCondition: AttributeCondition?,
limit: Int?,
scanIndexForward: Bool,
exclusiveStartKey: String?,
consistentRead: Bool) async throws
-> (items: [TypedDatabaseItem<AttributesType, ItemType>], lastEvaluatedKey: String?)
{
let queryInput = try AWSDynamoDB.QueryInput.forSortKeyCondition(
Expand Down
Loading
Loading