Skip to content

Commit f466b49

Browse files
lesvlesv
authored
Get full IT working again (#523)
* Spanner Works 1. remove comments from pom.xml 2. explicitly pass the SystemPropertyValues in local pom — not sure how it worked earlier. 3. Fix a comment in the QuickStart sample 4. Restore stdOut in both IT samples * Minor README tweak storage-transfer * ErrorProne 1. Turn on ErrorProne - `mvn clean verify -DskipTests` passes 2. Fixes for ErrorProne - mostly removing compiler plugin, java 1.8 for most samples. (No more Java 5) 3. Contributing now mentions gradle and testing 4. Fix storage-transfer README 5. back out some changes to MAVEN_OPTS * Debugging Magic 1. What project are these tests being run on? 2. Skip spanner for now. * change project_ID 1. enabled cloud-samples-tests 2. change the project 3. remove the maven memory hack. 4. remove the debugging code. * revert to j-d-s-t for projectID that BQ uses
1 parent f9ea543 commit f466b49

File tree

14 files changed

+96
-112
lines changed

14 files changed

+96
-112
lines changed

‎.travis.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,8 +42,8 @@ before_install:
4242
- openssl aes-256-cbc -K $encrypted_eb858daba67b_key -iv $encrypted_eb858daba67b_iv -in secrets.env.enc -out secrets.env -d
4343
&& set +x && source secrets.env && set -x
4444
|| true
45+
# Cross project - GOOGLE_APPLICATION_CREDENTIALS uses cloud-docs-tests, but BQ uses G_C_P (argh!)
4546
- export GOOGLE_CLOUD_PROJECT=java-docs-samples-tests
46-
- export MAVEN_OPTS='-Xmx6g -Xms1g -XX:-UseGCOverheadLimit -XX:+UseG1GC -XX:+PrintFlagsInitial -XX:+PrintFlagsFinal'
4747
# Skip the install step, since Maven will download the dependencies we need
4848
# when the test build runs.
4949
# http://stackoverflow.com/q/31945809/101923

‎CONTRIBUTING.md

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# How to become a contributor and submit your own code
22

3+
* [Contributor License Agreements](#Contributor-License-Agreements)
4+
* [Contributing a Patch](#Contributing-a-Patch)
5+
* [Build Tools](#build-tools)
6+
* [Integration Testing](#testing)
7+
* [Style](#Style)
8+
39
## Contributor License Agreements
410

511
We'd love to accept your sample apps and patches! Before we can take them, we
@@ -31,6 +37,35 @@ accept your pull requests.
3137
1. Ensure that your code has an appropriate set of unit tests which all pass.
3238
1. Submit a pull request.
3339

40+
## Build Tools
41+
42+
All new samples should build and run integration tests with both [Maven](https://maven.apache.org/) and [Gradle](https://gradle.org/).
43+
44+
## Testing
45+
46+
All samples must have Integration Tests (ie. They need to run against a real service) that run with
47+
`mvn verify` & `gradle build test`. If we need to enable an API, let us know.
48+
49+
Your `build.gradle` should have the following section:
50+
51+
```groovy
52+
53+
test {
54+
useJUnit()
55+
testLogging.showStandardStreams = true
56+
beforeTest { descriptor ->
57+
logger.lifecycle("test: " + descriptor + " Running")
58+
}
59+
60+
onOutput { descriptor, event ->
61+
logger.lifecycle("test: " + descriptor + ": " + event.message )
62+
}
63+
afterTest { descriptor, result ->
64+
logger.lifecycle("test: " + descriptor + ": " + result )
65+
}
66+
}
67+
```
68+
3469
## Style
3570

3671
Samples in this repository follow the [Google Java Style Guide][java-style].
@@ -47,7 +82,7 @@ tool or IntelliJ plugin.
4782

4883
### Running the Linter
4984

50-
To run the checkstyle plugin on an existing sample, run
85+
To run the checkstyle & ErrorProne plugins on an existing sample, run
5186

5287
```shell
5388
mvn clean verify -DskipTests
@@ -73,3 +108,5 @@ uses the common Checkstyle configuration.
73108
```
74109

75110
This is just used for testing. The sample should build without a parent defined.
111+
112+

‎datastore/pom.xml

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,12 @@
3131
<description>
3232
Example snippets for Datastore concepts and getting started documentation.
3333
</description>
34+
35+
<properties>
36+
<maven.compiler.target>1.8</maven.compiler.target>
37+
<maven.compiler.source>1.8</maven.compiler.source>
38+
</properties>
39+
3440
<dependencies>
3541
<dependency>
3642
<groupId>com.google.cloud</groupId>
@@ -49,15 +55,7 @@
4955
<build>
5056
<plugins>
5157
<!-- // [START maven]-->
52-
<plugin>
53-
<groupId>org.apache.maven.plugins</groupId>
54-
<artifactId>maven-compiler-plugin</artifactId>
55-
<version>2.5.1</version>
56-
<configuration>
57-
<source>1.7</source>
58-
<target>1.7</target>
59-
</configuration>
60-
</plugin>
58+
6159
<!-- // [END maven]-->
6260
<!-- // [START exec] -->
6361
<plugin>

‎logging/pom.xml

Lines changed: 5 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,11 @@
1212
<relativePath>..</relativePath>
1313
</parent>
1414

15+
<properties>
16+
<maven.compiler.target>1.8</maven.compiler.target>
17+
<maven.compiler.source>1.8</maven.compiler.source>
18+
</properties>
19+
1520
<dependencies>
1621
<dependency>
1722
<groupId>com.google.apis</groupId>
@@ -43,19 +48,4 @@
4348
</dependency>
4449
</dependencies>
4550

46-
<build>
47-
<sourceDirectory>src/main/java</sourceDirectory>
48-
<plugins>
49-
<plugin>
50-
<groupId>org.apache.maven.plugins</groupId>
51-
<artifactId>maven-compiler-plugin</artifactId>
52-
<version>3.2</version>
53-
<configuration>
54-
<source>5</source>
55-
<target>5</target>
56-
</configuration>
57-
</plugin>
58-
</plugins>
59-
</build>
60-
6151
</project>

‎monitoring/v2/pom.xml

Lines changed: 5 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,11 @@
2020
</repository>
2121
</repositories>
2222

23+
<properties>
24+
<maven.compiler.target>1.8</maven.compiler.target>
25+
<maven.compiler.source>1.8</maven.compiler.source>
26+
</properties>
27+
2328
<dependencies>
2429
<dependency>
2530
<groupId>com.google.apis</groupId>
@@ -68,19 +73,4 @@
6873
</dependency>
6974
</dependencies>
7075

71-
<build>
72-
<sourceDirectory>src/main/java</sourceDirectory>
73-
<plugins>
74-
<plugin>
75-
<groupId>org.apache.maven.plugins</groupId>
76-
<artifactId>maven-compiler-plugin</artifactId>
77-
<version>3.2</version>
78-
<configuration>
79-
<source>5</source>
80-
<target>5</target>
81-
</configuration>
82-
</plugin>
83-
</plugins>
84-
</build>
85-
8676
</project>

‎pom.xml

Lines changed: 2 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
<parent>
2626
<groupId>com.google.cloud.samples</groupId>
2727
<artifactId>shared-configuration</artifactId>
28-
<version>1.0.1</version>
28+
<version>1.0.2</version>
2929
</parent>
3030

3131
<properties>
@@ -142,14 +142,7 @@
142142
<artifactId>appengine-api-1.0-sdk</artifactId>
143143
<version>${appengine.sdk.version}</version>
144144
</dependency>
145-
<!--
146-
<dependency>
147-
<groupId>javax.servlet</groupId>
148-
<artifactId>servlet-api</artifactId>
149-
<version>2.5</version>
150-
<scope>provided</scope>
151-
</dependency>
152-
-->
145+
153146
<dependency>
154147
<groupId>jstl</groupId>
155148
<artifactId>jstl</artifactId>

‎spanner/cloud-client/pom.xml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,18 @@
9595
</execution>
9696
</executions>
9797
</plugin>
98+
<plugin>
99+
<groupId>org.apache.maven.plugins</groupId>
100+
<artifactId>maven-failsafe-plugin</artifactId>
101+
<version>2.19.1</version>
102+
<configuration>
103+
<systemPropertyVariables>
104+
<spanner.test.instance>${spanner.test.instance}</spanner.test.instance>
105+
<spanner.sample.database>${spanner.sample.database}</spanner.sample.database>
106+
<spanner.quickstart.database>${spanner.quickstart.database}</spanner.quickstart.database>
107+
</systemPropertyVariables>
108+
</configuration>
109+
</plugin>
98110
</plugins>
99111
</build>
100112

‎spanner/cloud-client/src/main/java/com/example/spanner/QuickstartSample.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ public static void main(String... args) throws Exception {
4040
SpannerOptions options = SpannerOptions.newBuilder().build();
4141
Spanner spanner = options.getService();
4242

43-
// Name of your database. Eg: projects/my-project/instances/instanceId/databases/databaseId
43+
// Name of your instance & database.
4444
String instanceId = args[0];
4545
String databaseId = args[1];
4646
try {

‎spanner/cloud-client/src/test/java/com/example/spanner/QuickstartSampleIT.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ public class QuickstartSampleIT {
3737
// This database needs to exist for test to pass.
3838
private String dbId = System.getProperty("spanner.quickstart.database");
3939
private ByteArrayOutputStream bout;
40+
private PrintStream stdOut = System.out;
4041
private PrintStream out;
4142

4243
@Before
@@ -48,7 +49,7 @@ public void setUp() {
4849

4950
@After
5051
public void tearDown() {
51-
System.setOut(null);
52+
System.setOut(stdOut);
5253
}
5354

5455
@Test

‎spanner/cloud-client/src/test/java/com/example/spanner/SpannerSampleIT.java

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,16 +39,18 @@
3939
@SuppressWarnings("checkstyle:abbreviationaswordinname")
4040
public class SpannerSampleIT {
4141
// The instance needs to exist for tests to pass.
42-
String instanceId = System.getProperty("spanner.test.instance");
43-
String databaseId = System.getProperty("spanner.sample.database");
42+
private final String instanceId = System.getProperty("spanner.test.instance");
43+
private final String databaseId = System.getProperty("spanner.sample.database");
4444
DatabaseId dbId;
4545
DatabaseAdminClient dbClient;
4646

4747
private String runSample(String command) throws Exception {
48+
PrintStream stdOut = System.out;
4849
ByteArrayOutputStream bout = new ByteArrayOutputStream();
4950
PrintStream out = new PrintStream(bout);
5051
System.setOut(out);
5152
SpannerSample.main(new String[]{command, instanceId, databaseId});
53+
System.setOut(stdOut);
5254
return bout.toString();
5355
}
5456

‎speech/grpc/pom.xml

Lines changed: 3 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -52,28 +52,10 @@ limitations under the License.
5252
<grpc-protobuf-version>1.0.3</grpc-protobuf-version>
5353
<maven-compiler-plugin-version>3.6.0</maven-compiler-plugin-version>
5454
<xolstice-protobuf-maven-plugin-version>0.5.0</xolstice-protobuf-maven-plugin-version>
55-
</properties>
5655

57-
<profiles>
58-
<profile>
59-
<id>jdk7</id>
60-
<activation>
61-
<jdk>1.7</jdk>
62-
</activation>
63-
<properties>
64-
<jdk.version>1.7</jdk.version>
65-
</properties>
66-
</profile>
67-
<profile>
68-
<id>jdk8</id>
69-
<activation>
70-
<jdk>1.8</jdk>
71-
</activation>
72-
<properties>
73-
<jdk.version>1.8</jdk.version>
74-
</properties>
75-
</profile>
76-
</profiles>
56+
<maven.compiler.target>1.8</maven.compiler.target>
57+
<maven.compiler.source>1.8</maven.compiler.source>
58+
</properties>
7759

7860
<!-- // [START dependency] -->
7961
<dependencies>
@@ -156,34 +138,10 @@ limitations under the License.
156138
</dependencies>
157139
<!-- // [END dependency] -->
158140

159-
<pluginRepositories>
160-
<pluginRepository>
161-
<releases>
162-
<updatePolicy>never</updatePolicy>
163-
</releases>
164-
<snapshots>
165-
<enabled>false</enabled>
166-
</snapshots>
167-
<id>central</id>
168-
<name>Central Repository</name>
169-
<url>https://repo.maven.apache.org/maven2</url>
170-
</pluginRepository>
171-
</pluginRepositories>
172141
<!-- // [START os-maven-plugin] -->
173142
<build>
174143
<!-- // [END os-maven-plugin] -->
175144
<plugins>
176-
<plugin>
177-
<artifactId>maven-compiler-plugin</artifactId>
178-
<version>${maven-compiler-plugin-version}</version>
179-
<configuration>
180-
<source>${jdk.version}</source>
181-
<target>${jdk.version}</target>
182-
<showWarnings>true</showWarnings>
183-
<showDeprecation>false</showDeprecation>
184-
<compilerArgument>-Xlint:-options</compilerArgument>
185-
</configuration>
186-
</plugin>
187145
<plugin>
188146
<artifactId>maven-assembly-plugin</artifactId>
189147
<configuration>

‎speech/grpc/src/main/java/com/examples/cloud/speech/StreamingRecognizeClient.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ private TargetDataLine getAudioInputLine() {
124124
DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
125125
if (!AudioSystem.isLineSupported(info)) {
126126
throw new RuntimeException(String.format(
127-
"Device doesn't support LINEAR16 mono raw audio format at {}Hz", samplingRate));
127+
"Device doesn't support LINEAR16 mono raw audio format at {%d}Hz", samplingRate));
128128
}
129129
try {
130130
TargetDataLine line = (TargetDataLine) AudioSystem.getLine(info);

‎storage/storage-transfer/README.md

Lines changed: 14 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -54,18 +54,21 @@ Creating a one-time transfer from Amazon S3 to Google Cloud Storage.
5454
1. Set the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables.
5555
1. In AwsRequester.java, fill in the user-provided constants.
5656
1. Compile the package with
57-
```
58-
mvn compile
59-
```
57+
58+
```bash
59+
mvn compile
60+
```
61+
6062
1. Run the transfer job with
61-
```
62-
mvn exec:java \
63-
-Dexec.mainClass="com.google.cloud.storage.storagetransfer.samples.AwsRequester" \
64-
-DprojectId=your-google-cloud-project-id \
65-
-DjobDescription="Sample transfer job from S3 to GCS." \
66-
-DawsSourceBucket=your-s3-bucket-name \
67-
-DgcsSinkBucket=your-gcs-bucket-name
68-
```
63+
64+
```bash
65+
mvn exec:java \
66+
-Dexec.mainClass="com.google.cloud.storage.storagetransfer.samples.AwsRequester" \
67+
-DprojectId=your-google-cloud-project-id \
68+
-DjobDescription="Sample transfer job from S3 to GCS." \
69+
-DawsSourceBucket=your-s3-bucket-name \
70+
-DgcsSinkBucket=your-gcs-bucket-name
71+
```
6972
1. Note the job ID in the returned Transfer Job.
7073

7174
## Transfer data from a standard Cloud Storage bucket to a Cloud Storage Nearline bucket

‎travis.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ common_travis_dir="$(travis_changed_files_parent)"
6464

6565
[ -z "$common_travis_dir" ] || pushd "$common_travis_dir"
6666

67-
./mvnw --batch-mode clean verify -DskipTests=$SKIP_TESTS | egrep -v "(^\[INFO\] Download|^\[INFO\].*skipping)"
67+
./mvnw --batch-mode clean verify -e -DskipTests=$SKIP_TESTS | egrep -v "(^\[INFO\] Download|^\[INFO\].*skipping)"
6868

6969
[ -z "$common_travis_dir" ] || popd
7070

0 commit comments

Comments
 (0)