Writing Performance Tests

Follow the steps below to set up a performance test harness for your library:

1. Create a maven module under sdk/<service>/azure-<service>-perf/ (if one doesn't exist)
   1. Tests will live under sdk/<service>/azure-<service>-perf/src
   2. Pom file will live under sdk/<service>/azure-<service>-perf/pom.xml
2. Pom file structure:

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<parent>
  <groupId>com.azure</groupId>
  <artifactId>azure-client-sdk-parent</artifactId>
  <version>1.7.0</version> <!-- {x-version-update;com.azure:azure-client-sdk-parent;current} -->
  <relativePath>../../../pom.client.xml</relativePath>
</parent>

<modelVersion>4.0.0</modelVersion>

<groupId>com.azure</groupId>
<artifactId>azure-<service-name>-perf</artifactId>
<version>1.0.0-beta.1</version> <!-- {x-version-update;com.azure:azure-storage-perf;current} -->
<packaging>jar</packaging>

<dependencies>
  <dependency>
    <groupId>com.azure</groupId>
    <artifactId><sdk-artifact-id></artifactId>
    <version><sdk-version></version> <!-- {x-version-update;com.azure:azure-storage-blob;current} -->
  </dependency>
  <dependency>
    <groupId>com.azure</groupId>
    <artifactId>perf-test-core</artifactId>
    <version>1.0.0-beta.1</version> <!-- {x-version-update;com.azure:perf-test-core;current} -->
  </dependency>
</dependencies>

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-assembly-plugin</artifactId>
      <version>3.2.0</version> <!-- {x-version-update;org.apache.maven.plugins:maven-assembly-plugin;external_dependency} -->
      <executions>
        <execution>
          <phase>package</phase>
          <goals>
            <goal>single</goal>
          </goals>
          <configuration>
            <archive>
              <manifest>
                <mainClass>
                  com.azure.<service>.<sub-service>.<your-main-class-from-step-3>
                </mainClass>
              </manifest>
            </archive>
            <descriptorRefs>
              <descriptorRef>jar-with-dependencies</descriptorRef>
            </descriptorRefs>
          </configuration>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>
</project>

1. Main Class: The project's main class should follow the following structure:

/**
 * Runs the <Service-Name> performance test.
 *
 * <p>To run from command line. Package the project into a jar with dependencies via mvn clean package.
 * Then run the program via java -jar 'compiled-jar-with-dependencies-path' </p>
 *
 * <p> To run from IDE, set all the required environment variables in IntelliJ via Run -&gt; EditConfigurations section.
 * Then run the App's main method via IDE.</p>
 */
public class App {
    public static void main(String[] args) {
        Class<?>[] testClasses;

        try {
            testClasses = new Class<?>[] {
                Class.forName("com.azure.<service>.<sub-package>.perf.<APIName>Test"),
                Class.forName("com.azure.<service>.<sub-package>.perf.<APIName>Test")
            };
        } catch (ClassNotFoundException e) {
            throw new RuntimeException(e);
        }

        PerfStressProgram.run(testClasses, args);
    }
}

3. Performance Test Class Structure: Create your performance Test classes under the com.azure.<service>.<sub-package>.perf package.

Create abstract test classes for doing client setup and clean up that is shared across all performance test classes.

The leaf nodes will be the performance test classes. Here is the sample structure of abstract Test class:

public abstract class ServiceTest<TOptions extends PerfStressOptions> extends PerfStressTest<TOptions> {
    protected final <ServiceClientName> serviceClientName;
    protected final <ServiceClientAsyncName> serviceClientAsyncName;
    public ServiceTest(TOptions options) {
        super(options);

        // Setup the service client
    }
}

Here is the sample structure of Performance Test class:

public class <API-NAME>Test extends ServiceTest<{OptionsClass}> {
    public <API-NAME>Test({OptionsClass} options) {
        super(options);
    // Optional Client setup is any child client needs to be setup specifically for this test.
    }

    // Required resource setup goes here.
    public Mono<Void> globalSetupAsync() {
        return super.globalSetupAsync()
                    .then(<service-call-to-perform-setup>)
                    .then();
    }


    // Perform the API call to be tested here
    @Override
    public void run() {
        serviceClient.apiCall();
    }

    // Perform the Async API call to be tested here
    @Override
    public Mono<Void> runAsync() {
        return serviceAsyncClient.apiCall()
            .map(<process-output>
            }).then();
    }

}

For Reference, look at Storage Performance Tests setup structure here.