Add new smoke tests.

This commit adds the first batch of new tests. These tests replace the
test apps for Database, Firestore, and Storage.

Author: allisonbm92

smoke-tests/README.md

@@ -0,0 +1,81 @@
+# Firebase Smoke Test Suite
+
+This directory contains smoke tests for Firebase on Android. These tests are
+intended to verify the integrations between different components and versions of
+Firebase. The tests should not include additional overhead, such as user
+interfaces. However, the tests should strive to remain similar to real use
+cases. As such, these tests run on devices or emulators (no Robolectric). This
+is a work in progress, and the following list shows what is complete:
+
+- [x] Create first set of tests to replace old test apps.
+- [ ] Reliably run smoke tests on CI.
+- [ ] Support version matrices.
+- [ ] Extend to collect system health metrics.
+
+# Test Synchronization
+
+Tests on devices usually involve at least two threads: the main thread and the
+testing thread. Either thread should be sufficient for testing, but most users
+will likely invoke Firebase methods from the main thread. As a result, this
+framework provides a simple solution to easily share state between threads. Most
+tests will consist of two methods. Both may be present in the same file for
+simplicity and ease of understanding. One method will be executed by the test
+runner on the testing thread, while the other, is invoked via a lambda on the
+main thread:
+
+```java
+@Test
+public void foo() throws Exception {
+  TaskChannel<Foo> channel = new TaskChannel<>();
+  MainThread.run(() -> test_foo(channel));
+
+  Foo foo = channel.waitForSuccess();
+  assertThat(foo)...
+}
+
+private void test_foo(TaskChannel<Foo> channel) {
+  ...
+}
+```
+
+Channels should be used to send results back to the testing thread. The
+`AbstractChannel` class provides the core implementation to send a failure or
+success value back to the test method. Failures are always exceptions, and
+multiple exceptions may be sent per test case. Only one success value may be
+sent. Failures dominate over successful values.
+
+Firebase methods are often difficult to test due to their asynchronous nature.
+This framework provides a few subclasses of `AbstractChannel` to eliminate some
+boilerplate, concentrate the synchronization logic in one place, and shorten the
+tests to focus on the actual Firebase methods under test. The most common
+subclass is `TaskChannel`, and it simplifies chaining and adding listeners to
+Google Play Services tasks. Below is an example (Firebase methods truncated for
+brevity):
+
+```java
+private void test_readAfterSignIn(TaskChannel<Document> channel) {
+  FirebaseAuth auth = FirebaseAuth.getInstance();
+  FirebaseDatabase db = FirebaseDatabase.getInstance();
+
+  channel.trapFailure(auth.signIn()).andThen(a -> {
+    channel.sendOutcome(db.get("path/to/document"));
+  });
+}
+```
+
+Methods like `trapFailure` and `sendOutcome` automatically direct failures from
+tasks to the testing channel. There are also specialized channels for Firestore
+and Database, because they rely on custom listeners. See the source code for
+more details.
+
+# Building the Tests
+
+This Gradle project is split into flavors for each Firebase product. The Android
+plugin adds additional, compound flavors, such as `androidTestDatabase`. All
+test code lives in one of these testing variants. Common infrastructure belongs
+directly in `androidTest`. Do not add any source code to `main` or a non-testing
+variant, such as `firestore`.
+
+As an exception, there is a single activity in the `main` variant. This is used
+to build the APK under test. However, all test logic belongs in the testing
+variants.

smoke-tests/build.gradle

@@ -0,0 +1,105 @@
+// Copyright 2018 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+buildscript {
+  repositories {
+    google()
+    jcenter()
+  }
+
+  dependencies {
+    classpath "com.android.tools.build:gradle:3.3.2"
+    classpath "com.google.gms:google-services:4.1.0"
+  }
+}
+
+apply plugin: "com.android.application"
+
+android {
+  compileSdkVersion 24
+
+  compileOptions {
+    sourceCompatibility JavaVersion.VERSION_1_8
+    targetCompatibility JavaVersion.VERSION_1_8
+  }
+
+  defaultConfig {
+    minSdkVersion 16
+    multiDexEnabled true
+//    applicationId "com.google.firebase.testing.common"
+    testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
+  }
+
+  flavorDimensions "systemUnderTest"
+
+  productFlavors {
+    database {
+      //applicationId "com.google.firebase.testing.database"
+//      applicationId "com.google.firebase.testapps.database"
+      dimension "systemUnderTest"
+    }
+
+    firestore {
+//      applicationId "com.google.firebase.testing.firestore"
+      dimension "systemUnderTest"
+    }
+
+    storage {
+//      applicationId "com.google.firebase.testing.storage"
+      dimension "systemUnderTest"
+    }
+  }
+}
+
+repositories {
+  google()
+  jcenter()
+}
+
+dependencies {
+  // Common
+  implementation "com.android.support.test:runner:1.0.2"
+  implementation "com.google.android.gms:play-services-tasks:16.0.1"
+  implementation "com.google.firebase:firebase-core:16.0.7"
+  implementation "com.google.truth:truth:0.43"
+
+  androidTestImplementation "com.android.support.test:runner:1.0.2"
+  androidTestImplementation "com.google.android.gms:play-services-tasks:16.0.1"
+  androidTestImplementation "com.google.firebase:firebase-core:16.0.7"
+  androidTestImplementation "com.google.truth:truth:0.43"
+  androidTestImplementation "junit:junit:4.12"
+
+  // Database
+  databaseImplementation "com.google.firebase:firebase-auth:16.1.0"
+  databaseImplementation "com.google.firebase:firebase-database:16.1.0"
+
+  androidTestDatabaseImplementation "com.google.firebase:firebase-auth:16.1.0"
+  androidTestDatabaseImplementation "com.google.firebase:firebase-database:16.1.0"
+
+  // Firestore
+  firestoreImplementation "com.google.firebase:firebase-auth:16.1.0"
+  firestoreImplementation "com.google.firebase:firebase-firestore:18.1.0"
+
+  androidTestFirestoreImplementation "com.google.firebase:firebase-auth:16.1.0"
+  androidTestFirestoreImplementation "com.google.firebase:firebase-firestore:18.1.0"
+
+  // Storage
+  storageImplementation "com.google.firebase:firebase-auth:16.1.0"
+  storageImplementation "com.google.firebase:firebase-storage:16.1.0"
+
+  androidTestStorageImplementation "com.google.firebase:firebase-auth:16.1.0"
+  androidTestStorageImplementation "com.google.firebase:firebase-storage:16.1.0"
+}
+
+apply plugin: "com.google.gms.google-services"

smoke-tests/src/androidTest/java/com/google/firebase/testing/common/AbstractChannel.java

@@ -0,0 +1,114 @@
+// Copyright 2018 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.firebase.testing.common;
+
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.locks.ReentrantLock;
+
+/**
+ * An abstract channel for sending test results across threads.
+ *
+ * <p>This enables test code to run on the main thread and signal the test thread when to stop
+ * blocking. Tests may send multiple errors that will then be thrown on the test thread. However, a
+ * test may only send success once. After this is done, nothing else can be sent.
+ */
+public abstract class AbstractChannel<T> {
+
+  private final CountDownLatch latch = new CountDownLatch(1);
+  private final ReentrantLock lock = new ReentrantLock();
+
+  private Exception error = null;
+  private T value = null;
+
+  /**
+   * Sends a failure back to the testing thread.
+   *
+   * <p>This method will always send an exception to the testing thread. If an exception has already
+   * been sent, this method will chain the new exception to the previous. If a successful value has
+   * already been sent, this method will override it with the failure. Note, it is recommended to
+   * send only one value through the channel. This method is safe to invoke from any thread.
+   */
+  protected void fail(Exception err) {
+    // This is explicitly synchronized in case multiple threads are trying to send values.
+    try {
+      lock.lock();
+
+      if (error == null) {
+        error = err;
+      } else {
+        error.addSuppressed(err);
+      }
+
+      latch.countDown();
+    } finally {
+      lock.unlock();
+    }
+  }
+
+  /**
+   * Sends a successful value back to the testing thread.
+   *
+   * <p>This method will only send the value if no value has been sent. It is an error to invoke
+   * this method multiple times or after invoking {@link #fail}. This method is safe to invoke from
+   * any thread.
+   */
+  protected void succeed(T val) {
+    // This is explicitly synchronized in case multiple threads are trying to send values.
+    try {
+      lock.lock();
+
+      if (latch.getCount() == 0) {
+        // Only a single, successful value is supported. This throws the exception on both the
+        // testing thread and the main thread.
+        IllegalStateException error = new IllegalStateException("Result already completed");
+        fail(error);
+        throw error;
+      }
+
+      value = val;
+      latch.countDown();
+    } finally {
+      lock.unlock();
+    }
+  }
+
+  /** Waits 30 seconds to receive the successful value. */
+  public T waitForSuccess() throws InterruptedException {
+    return waitForSuccess(30, TimeUnit.SECONDS);
+  }
+
+  /**
+   * Waits for up to the request time for the sending thread to send a successful value.
+   *
+   * <p>If the sender does not send success within the specified time, this method throws an {@link
+   * AssertionError} and chains any received errors to it. This method is safe to invoke from any
+   * thread.
+   */
+  public T waitForSuccess(long duration, TimeUnit unit) throws InterruptedException {
+    boolean completed = latch.await(duration, unit);
+
+    if (!completed) {
+      String message = String.format("Test did not complete within %s %s", duration, unit);
+      throw new AssertionError(message, error);
+    }
+
+    if (error != null) {
+      throw new AssertionError("Test completed with errors", error);
+    }
+
+    return value;
+  }
+}

smoke-tests/src/androidTest/java/com/google/firebase/testing/common/MainThread.java

@@ -0,0 +1,39 @@
+// Copyright 2018 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.firebase.testing.common;
+
+import android.os.Handler;
+import android.os.Looper;
+
+/**
+ * Convenience class for interacting with Android.
+ *
+ * <p>For now, this only consists of the {@link #run} method.
+ */
+public final class MainThread {
+
+  private static Handler handler = null;
+
+  private MainThread() {}
+
+  /** Runs the {@link Runnable} on the main thread. */
+  public static void run(Runnable r) throws InterruptedException {
+    if (handler == null) {
+      handler = new Handler(Looper.getMainLooper());
+    }
+
+    handler.post(r);
+  }
+}