Skip to content

Document API for DynamoDB usung EnhancedDynamoDB impementation issue #36 #3849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Mar 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
75cb67a
Adding new Interface EnhancedDocument (#3702)
joviegas Jan 21, 2023
92ca159
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Jan 25, 2023
01436d3
DefaultEnhancedDocument implementation (#3718)
joviegas Feb 1, 2023
aa3444a
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Feb 8, 2023
d957736
Implement Static factory methods of EnhancedDocument (#3752)
joviegas Feb 8, 2023
27d1948
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Feb 8, 2023
764fc02
DocumentTableSchema Implementation (#3758)
joviegas Feb 16, 2023
fcbc4ae
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Feb 16, 2023
a1f7443
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Feb 20, 2023
7e936ac
The builder for EnhancedDocument should not rely on the order in whic…
joviegas Feb 22, 2023
a9407dd
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Feb 23, 2023
5c683e5
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Mar 6, 2023
0c074c0
Handled surface api comments of removing Generic access as Objects (#…
joviegas Mar 9, 2023
80fe9c1
Remove extra spaces in Json and make it same as Items as in V1 (#3835)
joviegas Mar 15, 2023
b60c4fc
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Mar 15, 2023
5e15af7
Delete unwanted class
joviegas Mar 16, 2023
fc88544
Functional Test Cases for Document DDB API and Surface API Review 2 c…
joviegas Mar 20, 2023
6db71c2
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Mar 20, 2023
3557d15
Handled PR comments, \n 1. Moved DocumentTableSchema from mapper to d…
joviegas Mar 22, 2023
73e9459
Removed @inherit wherever not required
joviegas Mar 23, 2023
5da0e01
Merge branch 'master' into feature/master/joviegas_document_db_impl
joviegas Mar 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ providers.

// New API in TableSchema to create a DocumentTableSchema
DocumentTableSchema documentTableSchema =
TableSchema.fromDocumentSchemaBuilder()
TableSchema.documentSchemaBuilder()
.addIndexPartitionKey(primaryIndexName(), "sample_hash_name", AttributeValueType.S)
.addIndexSortKey("gsi_index", "sample_sort_name", AttributeValueType.N)
.addAttributeConverterProviders(cutomAttributeConverters)
Expand Down Expand Up @@ -78,7 +78,7 @@ EnhancedDocument documentTableItem = documentTable.getItem(
Number sampleSortvalue = documentTableItem.get("sample_sort_name", EnhancedType.of(Number.class));

// Accessing an attribute from document using specific getters.
sampleSortvalue = documentTableItem.getSdkNumber("sample_sort_name");
sampleSortvalue = documentTableItem.getNumber("sample_sort_name");

// Accessing an attribute of custom class using custom converters.
CustomClass customClass = documentTableItem.get("custom_nested_map", new CustomAttributeConverter()));
Expand Down
5 changes: 5 additions & 0 deletions services-custom/dynamodb-enhanced/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,11 @@
<artifactId>aws-core</artifactId>
<version>${awsjavasdk.version}</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>json-utils</artifactId>
<version>${awsjavasdk.version}</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>http-client-spi</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.OptionalLongAttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.PeriodAttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.SdkBytesAttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.SdkNumberAttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.SetAttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.ShortAttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.StringAttributeConverter;
Expand Down Expand Up @@ -88,6 +89,8 @@
@ThreadSafe
@Immutable
public final class DefaultAttributeConverterProvider implements AttributeConverterProvider {
private static final DefaultAttributeConverterProvider INSTANCE = getDefaultBuilder().build();

private static final Logger log = Logger.loggerFor(DefaultAttributeConverterProvider.class);

private final ConcurrentHashMap<EnhancedType<?>, AttributeConverter<?>> converterCache =
Expand Down Expand Up @@ -117,10 +120,9 @@ public DefaultAttributeConverterProvider() {
* Returns an attribute converter provider with all default converters set.
*/
public static DefaultAttributeConverterProvider create() {
return getDefaultBuilder().build();
return INSTANCE;
}


/**
* Equivalent to {@code builder(EnhancedType.of(Object.class))}.
*/
Expand Down Expand Up @@ -246,7 +248,8 @@ private static Builder getDefaultBuilder() {
.addConverter(UuidAttributeConverter.create())
.addConverter(ZonedDateTimeAsStringAttributeConverter.create())
.addConverter(ZoneIdAttributeConverter.create())
.addConverter(ZoneOffsetAttributeConverter.create());
.addConverter(ZoneOffsetAttributeConverter.create())
.addConverter(SdkNumberAttributeConverter.create());
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
import java.util.Map;
import software.amazon.awssdk.annotations.SdkPublicApi;
import software.amazon.awssdk.annotations.ThreadSafe;
import software.amazon.awssdk.enhanced.dynamodb.document.DocumentTableSchema;
import software.amazon.awssdk.enhanced.dynamodb.document.EnhancedDocument;
import software.amazon.awssdk.enhanced.dynamodb.mapper.BeanTableSchema;
import software.amazon.awssdk.enhanced.dynamodb.mapper.ImmutableTableSchema;
import software.amazon.awssdk.enhanced.dynamodb.mapper.StaticImmutableTableSchema;
Expand Down Expand Up @@ -84,6 +86,16 @@ static <T> BeanTableSchema<T> fromBean(Class<T> beanClass) {
return BeanTableSchema.create(beanClass);
}

/**
* Provides interfaces to interact with DynamoDB tables as {@link EnhancedDocument} where the complete Schema of the table is
* not required.
*
* @return A {@link DocumentTableSchema.Builder} for instantiating DocumentTableSchema.
*/
static DocumentTableSchema.Builder documentSchemaBuilder() {
return DocumentTableSchema.builder();
}

/**
* Scans an immutable class that has been annotated with DynamoDb immutable annotations and then returns a
* {@link ImmutableTableSchema} implementation of this interface that can map records to and from items of that
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,257 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License").
* You may not use this file except in compliance with the License.
* A copy of the License is located at
*
* http://aws.amazon.com/apache2.0
*
* or in the "license" file accompanying this file. This file is distributed
* on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
* express or implied. See the License for the specific language governing
* permissions and limitations under the License.
*/

package software.amazon.awssdk.enhanced.dynamodb.document;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import software.amazon.awssdk.annotations.NotThreadSafe;
import software.amazon.awssdk.annotations.SdkPublicApi;
import software.amazon.awssdk.enhanced.dynamodb.AttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.AttributeConverterProvider;
import software.amazon.awssdk.enhanced.dynamodb.AttributeValueType;
import software.amazon.awssdk.enhanced.dynamodb.DefaultAttributeConverterProvider;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbTable;
import software.amazon.awssdk.enhanced.dynamodb.EnhancedType;
import software.amazon.awssdk.enhanced.dynamodb.TableMetadata;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.ConverterProviderResolver;
import software.amazon.awssdk.enhanced.dynamodb.internal.document.DefaultEnhancedDocument;
import software.amazon.awssdk.enhanced.dynamodb.mapper.StaticImmutableTableSchema;
import software.amazon.awssdk.enhanced.dynamodb.mapper.StaticTableMetadata;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;


/**
* Implementation of {@link TableSchema} that builds a table schema based on DynamoDB Items.
* <p>
* In Amazon DynamoDB, an item is a collection of attributes. Each attribute has a name and a value. An attribute value can be a
* scalar, a set, or a document type
* <p>
* A DocumentTableSchema is used to create a {@link DynamoDbTable} which provides read and writes access to DynamoDB table as
* {@link EnhancedDocument}.
* <p> DocumentTableSchema specifying primaryKey, sortKey and a customAttributeConverter can be created as below
* {@snippet :
* DocumentTableSchema documentTableSchema = DocumentTableSchema.builder()
* .primaryKey("sampleHashKey", AttributeValueType.S)
* .sortKey("sampleSortKey", AttributeValueType.S)
* .attributeConverterProviders(customAttributeConverter, AttributeConverterProvider.defaultProvider())
* .build();
*}
* <p> DocumentTableSchema can also be created without specifying primaryKey and sortKey in which cases the
* {@link TableMetadata} of DocumentTableSchema will error if we try to access attributes from metaData. Also if
* attributeConverterProviders are not provided then {@link DefaultAttributeConverterProvider} will be used
* {@snippet :
* DocumentTableSchema documentTableSchema = DocumentTableSchema.builder().build();
*}
*
* @see <a href="https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithItems.html" target="_top">Working
* with items and attributes</a>
*/
@SdkPublicApi
public final class DocumentTableSchema implements TableSchema<EnhancedDocument> {
private final TableMetadata tableMetadata;
private final List<AttributeConverterProvider> attributeConverterProviders;

private DocumentTableSchema(Builder builder) {
this.attributeConverterProviders = builder.attributeConverterProviders;
this.tableMetadata = builder.staticTableMetaDataBuilder.build();
}

public static Builder builder() {
return new Builder();
}

@Override
public EnhancedDocument mapToItem(Map<String, AttributeValue> attributeMap) {
if (attributeMap == null) {
return null;
}
DefaultEnhancedDocument.DefaultBuilder builder =
(DefaultEnhancedDocument.DefaultBuilder) DefaultEnhancedDocument.builder();
attributeMap.forEach(builder::putObject);
return builder.attributeConverterProviders(attributeConverterProviders)
.build();
}

/**
* {@inheritDoc}
*
* This flag does not have significance for the Document API, unlike Java objects where the default value of an undefined
* Object is null.In contrast to mapped classes, where a schema is present, the DocumentSchema is unaware of the entire
* schema.Therefore, if an attribute is not present, it signifies that it is null, and there is no need to handle it in a
* separate way.However, if the user explicitly wants to nullify certain attributes, then the user needs to set those
* attributes as null in the Document that needs to be updated.
*
*/
@Override
public Map<String, AttributeValue> itemToMap(EnhancedDocument item, boolean ignoreNulls) {
if (item == null) {
return null;
}
List<AttributeConverterProvider> providers = mergeAttributeConverterProviders(item);
return item.toBuilder().attributeConverterProviders(providers).build().toMap();
}

private List<AttributeConverterProvider> mergeAttributeConverterProviders(EnhancedDocument item) {
if (item.attributeConverterProviders() != null && !item.attributeConverterProviders().isEmpty()) {
Set<AttributeConverterProvider> providers = new LinkedHashSet<>();
providers.addAll(item.attributeConverterProviders());
providers.addAll(attributeConverterProviders);
return providers.stream().collect(Collectors.toList());
}
return attributeConverterProviders;
}

@Override
public Map<String, AttributeValue> itemToMap(EnhancedDocument item, Collection<String> attributes) {
if (item.toMap() == null) {
return null;
}

List<AttributeConverterProvider> providers = mergeAttributeConverterProviders(item);
return item.toBuilder().attributeConverterProviders(providers).build().toMap().entrySet()
.stream()
.filter(entry -> attributes.contains(entry.getKey()))
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
}

@Override
public AttributeValue attributeValue(EnhancedDocument item, String attributeName) {
if (item == null || item.toMap() == null) {
return null;
}
List<AttributeConverterProvider> providers = mergeAttributeConverterProviders(item);
return item.toBuilder()
.attributeConverterProviders(providers)
.build()
.toMap()
.get(attributeName);
}

@Override
public TableMetadata tableMetadata() {
return tableMetadata;
}

@Override
public EnhancedType<EnhancedDocument> itemType() {
return EnhancedType.of(EnhancedDocument.class);
}

@Override
public List<String> attributeNames() {
return tableMetadata.keyAttributes().stream().map(key -> key.name()).collect(Collectors.toList());
}

@Override
public boolean isAbstract() {
return false;
}

@NotThreadSafe
public static final class Builder {

private final StaticTableMetadata.Builder staticTableMetaDataBuilder = StaticTableMetadata.builder();

/**
* By Default the defaultConverterProvider is used for converting AttributeValue to primitive types.
*/
private List<AttributeConverterProvider> attributeConverterProviders =
Collections.singletonList(ConverterProviderResolver.defaultConverterProvider());

/**
* Adds information about a partition key associated with a specific index.
*
* @param indexName the name of the index to associate the partition key with
* @param attributeName the name of the attribute that represents the partition key
* @param attributeValueType the {@link AttributeValueType} of the partition key
* @throws IllegalArgumentException if a partition key has already been defined for this index
*/
public Builder addIndexPartitionKey(String indexName, String attributeName, AttributeValueType attributeValueType) {
staticTableMetaDataBuilder.addIndexPartitionKey(indexName, attributeName, attributeValueType);
return this;
}

/**
* Adds information about a sort key associated with a specific index.
*
* @param indexName the name of the index to associate the sort key with
* @param attributeName the name of the attribute that represents the sort key
* @param attributeValueType the {@link AttributeValueType} of the sort key
* @throws IllegalArgumentException if a sort key has already been defined for this index
*/
public Builder addIndexSortKey(String indexName, String attributeName, AttributeValueType attributeValueType) {
staticTableMetaDataBuilder.addIndexSortKey(indexName, attributeName, attributeValueType);
return this;
}

/**
* Specifies the {@link AttributeConverterProvider}s to use with the table schema. The list of attribute converter
* providers must provide {@link AttributeConverter}s for Custom types. The attribute converter providers will be loaded
* in the strict order they are supplied here.
* <p>
* By default, {@link DefaultAttributeConverterProvider} will be used, and it will provide standard converters for most
* primitive and common Java types. Configuring this will override the default behavior, so it is recommended to always
* append `DefaultAttributeConverterProvider` when you configure the custom attribute converter providers.
* <p>
* {@snippet :
* builder.attributeConverterProviders(customAttributeConverter, AttributeConverterProvider.defaultProvider());
*}
*
* @param attributeConverterProviders a list of attribute converter providers to use with the table schema
*/
public Builder attributeConverterProviders(AttributeConverterProvider... attributeConverterProviders) {
this.attributeConverterProviders = Arrays.asList(attributeConverterProviders);
return this;
}

/**
* Specifies the {@link AttributeConverterProvider}s to use with the table schema. The list of attribute converter
* providers must provide {@link AttributeConverter}s for all types used in the schema. The attribute converter providers
* will be loaded in the strict order they are supplied here.
* <p>
* By default, {@link DefaultAttributeConverterProvider} will be used, and it will provide standard converters for most
* primitive and common Java types. Configuring this will override the default behavior, so it is recommended to always
* append `DefaultAttributeConverterProvider` when you configure the custom attribute converter providers.
* <p>
* {@snippet :
* List<AttributeConverterProvider> providers = new ArrayList<>( customAttributeConverter,
* AttributeConverterProvider.defaultProvider());
* builder.attributeConverterProviders(providers);
*}
*
* @param attributeConverterProviders a list of attribute converter providers to use with the table schema
*/
public Builder attributeConverterProviders(List<AttributeConverterProvider> attributeConverterProviders) {
this.attributeConverterProviders = new ArrayList<>(attributeConverterProviders);
return this;
}

/**
* Builds a {@link StaticImmutableTableSchema} based on the values this builder has been configured with
*/
public DocumentTableSchema build() {
return new DocumentTableSchema(this);
}
}
}
Loading