Skip to content

Version 1.0.0 of the Large payload offloading common library for AWS #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 14, 2020
Merged

Version 1.0.0 of the Large payload offloading common library for AWS #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
feb4620
Version 1.0.0 of the Large payload offloading common library for AWS
aws-rizi Jul 8, 2020
c5772cf
Addressing feedback
aws-rizi Jul 13, 2020
a9e8842
Constants.java removed and README.md updated
aws-rizi Jul 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 26 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,32 @@
## My Project
## Large Payload Offloading Java Common Library For AWS

TODO: Fill this README out!
The **Large Payload Offloading Java Common Library For AWS** enables you to manage payloads with Amazon S3.
For example, this is useful for storing and retrieving payloads with size greater than the current SQS/SNS limit of 256 KB, up to a maximum of 2 GB.

You can use this library to:

Be sure to:
* Specify whether payloads are always stored in Amazon S3 or only when a payload's size exceeds 256 KB.

* Change the title in this README
* Edit your repository description on GitHub
* Get the corresponding payload object from an Amazon S3 bucket.

* Delete the corresponding payload object from an Amazon S3 bucket.

You can download release builds through the [releases section of this](https://github.com/awslabs/large-payload-offloading-java-common-lib-for-aws/releases) project.

## Getting Started

* **Sign up for AWS** -- Before using this library with Amazon services, you need an AWS account. For more information about creating an AWS account and retrieving your AWS credentials, see [AWS Account and Credentials](http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-setup.html) in the AWS SDK for Java Developer Guide.
* **Minimum requirements** -- You'll need Java 8 (or later) and [Maven 3](http://maven.apache.org/).
* **Download** -- Download the [latest preview release](https://github.com/awslabs/large-payload-offloading-java-common-lib-for-aws/releases) or pick it up from Maven:

```xml
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>large-payload-offloading-java-common-lib-for-aws</artifactId>
<version>1.0.0</version>
<type>jar</type>
</dependency>
```

## Security

Expand Down
141 changes: 141 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.amazonaws</groupId>
<artifactId>large-payload-offloading-java-common-lib-for-aws</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<name>Large payload offloading common library for AWS</name>
<description>Common library between extended Amazon AWS clients to save payloads up to 2GB on Amazon S3.</description>
<url>https://github.com/awslabs/large-payload-offloading-java-common-lib-for-aws</url>

<scm>
<url>https://github.com/awslabs/large-payload-offloading-java-common-lib-for-aws</url>
</scm>

<licenses>
<license>
<name>Apache License, Version 2.0</name>
<url>https://aws.amazon.com/apache2.0</url>
<distribution>repo</distribution>
</license>
</licenses>

<developers>
<developer>
<id>amazonwebservices</id>
<organization>Amazon Web Services</organization>
<organizationUrl>https://aws.amazon.com</organizationUrl>
<roles>
<role>developer</role>
</roles>
</developer>
</developers>

<properties>
<aws-java-sdk.version>1.11.817</aws-java-sdk.version>
</properties>

<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>${aws-java-sdk.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>3.3.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>pl.pragmatists</groupId>
<artifactId>JUnitParams</artifactId>
<version>1.1.1</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
<plugin>
<groupId>org.sonatype.plugins</groupId>
<artifactId>nexus-staging-maven-plugin</artifactId>
<version>1.6.7</version>
<extensions>true</extensions>
<configuration>
<serverId>ossrh</serverId>
<nexusUrl>https://aws.oss.sonatype.org</nexusUrl>
<autoReleaseAfterClose>true</autoReleaseAfterClose>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>

<profiles>
<profile>
<id>publishing</id>
<distributionManagement>
<snapshotRepository>
<id>ossrh</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
<repository>
<id>ossrh</id>
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
</repository>
</distributionManagement>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<id>sign-artifacts</id>
<phase>verify</phase>
<goals>
<goal>sign</goal>
</goals>
<configuration>
<gpgArguments>
<gpgArgument>--pinentry-mode</gpgArgument>
<gpgArgument>loopback</gpgArgument>
</gpgArguments>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
32 changes: 32 additions & 0 deletions src/main/java/com/amazonaws/largepayloadoffloading/CountingOutputStream.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package com.amazonaws.largepayloadoffloading;

import java.io.OutputStream;

/**
* This class is used for checking the size of a string without copying the
* whole string into memory and converting it to bytes array. Compared to
* String.getBytes().length, it is more efficient and reliable for large
* strings.
*/
class CountingOutputStream extends OutputStream {
private long totalSize;

@Override
public void write(int b) {
++totalSize;
}

@Override
public void write(byte[] b) {
totalSize += b.length;
}

@Override
public void write(byte[] b, int offset, int len) {
totalSize += len;
}

public long getTotalSize() {
return totalSize;
}
}
37 changes: 37 additions & 0 deletions src/main/java/com/amazonaws/largepayloadoffloading/JsonDataConverter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
package com.amazonaws.largepayloadoffloading;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping;
import com.fasterxml.jackson.databind.ObjectWriter;
import com.fasterxml.jackson.databind.SerializationFeature;

/**
* This class implements conversion through Jackson JSON processor. Methods are
* provided for serializing an object to JSON and deserializing from JSON to an
* object.
*/
class JsonDataConverter {
protected final ObjectMapper objectMapper;

public JsonDataConverter() {
this(new ObjectMapper());
objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
objectMapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
objectMapper.enableDefaultTyping(DefaultTyping.NON_FINAL);
}

public JsonDataConverter(ObjectMapper objectMapper) {
this.objectMapper = objectMapper;
}

public String serializeToJson(Object obj) throws JsonProcessingException {
ObjectWriter objectWriter = objectMapper.writer();
return objectWriter.writeValueAsString(obj);
}

public <T> T deserializeFromJson(String jsonText, Class<T> objectType) throws Exception {
return objectMapper.readValue(jsonText, objectType);
}
}
Loading