Skip to content

Commit 3b8c031

Browse files
millemsmillems
committed
Updated the metrics design to include details on how metrics can be enabled at the request level, client level and global level.
1 parent fc38805 commit 3b8c031

File tree

2 files changed

+154
-157
lines changed

2 files changed

+154
-157
lines changed

‎docs/design/core/metrics/Design.md

Lines changed: 154 additions & 88 deletions
Original file line numberDiff line numberDiff line change
@@ -60,107 +60,174 @@ standard metrics collected by the SDK.
6060

6161
## Enabling Metrics
6262

63-
Metrics feature is disabled by default. Metrics can be enabled at client level in the following ways.
64-
65-
### Feature Flags (Metrics Provider)
66-
67-
* SDK exposes an [interface](prototype/MetricConfigurationProvider.java) to enable the metrics feature and specify
68-
options to configure the metrics behavior.
69-
* SDK provides an implementation of this interface based on system properties.
70-
* Here are the system properties SDK supports:
71-
- **aws.javasdk2x.metrics.enabled** - Metrics feature is enabled if this system property is set
72-
- **aws.javasdk2x.metrics.category** - Comma separated set of MetricCategory that are enabled for collection
73-
* SDK calls the methods in this interface for each request ie, enabled() method is called for every request to determine
74-
if the metrics feature is enabled or not (similarly for other configuration options).
75-
- This allows customers to control metrics behavior in a more flexible manner; for example using an external database
76-
like DynamoDB to dynamically control metrics collection. This is useful to enable/disable metrics feature and
77-
control metrics options at runtime without the need to make code changes or re-deploy the application.
78-
* As the interface methods are called for each request, it is recommended for the implementations to run expensive tasks
79-
asynchronously in the background, cache the results and periodically refresh the results.
63+
The metrics feature is disabled by default. Metrics can be enabled and configured in the following ways:
64+
65+
### Option 1: Configuring MetricPublishers on a request
66+
67+
A publisher can be configured directly on the `RequestOverrideConfiguration`:
68+
69+
```java
70+
MetricPublisher metricPublisher = CloudWatchMetricPublisher.create();
71+
DynamoDbClient dynamoDb = DynamoDbClient.create();
72+
dynamoDb.listTables(ListTablesRequest.builder()
73+
.overrideConfiguration(c -> c.addMetricPublisher(metricPublisher))
74+
.build());
75+
```
76+
77+
The methods exposed for setting metric publishers follow the pattern established by `ExecutionInterceptor`s:
78+
79+
```java
80+
class RequestOverrideConfiguration {
81+
// ...
82+
class Builder {
83+
// ...
84+
Builder metricPublishers(List<MetricPublisher> metricsPublishers);
85+
Builder addMetricPublisher(MetricPublisher metricsPublisher);
86+
}
87+
}
88+
```
89+
90+
### Option 2: Configuring MetricPublishers on a client
91+
92+
A publisher can be configured directly on the `ClientOverrideConfiguration`. A publisher specified in this way is used
93+
with lower priority than **Option 1** above.
8094

8195
```java
82-
ClientOverrideConfiguration config = ClientOverrideConfiguration
83-
.builder()
84-
// If this is not set, SDK uses the default chain with system property
85-
.metricConfigurationProvider(new SystemSettingsMetricConfigurationProvider())
86-
.build();
87-
88-
// Set the ClientOverrideConfiguration instance on the client builder
89-
CodePipelineAsyncClient asyncClient =
90-
CodePipelineAsyncClient
91-
.builder()
92-
.overrideConfiguration(config)
93-
.build();
96+
MetricPublisher metricPublisher = CloudWatchMetricPublisher.create();
97+
DynamoDbClient dynamoDb = DynamoDbClient.builder()
98+
.overrideConfiguration(c -> c.addMetricPublisher(metricPublisher))
99+
.build();
94100
```
95101

96-
### Metrics Provider Chain
102+
The methods exposed for setting metric publishers follow the pattern established by `ExecutionInterceptor`s:
103+
104+
```java
105+
class ClientOverrideConfiguration {
106+
// ...
107+
class Builder {
108+
// ...
109+
Builder metricPublishers(List<MetricPublisher> metricsPublishers);
110+
Builder addMetricPublisher(MetricPublisher metricsPublisher);
111+
}
112+
}
113+
```
114+
115+
**Note:** As with the `httpClient` setting, calling `close()` on the `DynamoDbClient` *will not* close the configured
116+
`metricPublishers`. You must close the `metricPublishers` yourself when you're done using them.
117+
118+
### Option 3: Configuring MetricPublishers using System Properties or Environment Variables
119+
120+
This option allows the customer to enable metric publishing by default, without needing to enable it via **Option 1**
121+
or **Option 2** above. This means that a customer can enable metrics without needing to make a change to their runtime
122+
code.
123+
124+
This option is enabled using an environment variable or system property. If both are specified, the system property
125+
will be used. If metrics are enabled at the client level using **Option 2** above, this option is ignored. Overriding
126+
the metric publisher at request time using **Option 1** overrides any publishers that have been enabled globally.
127+
128+
**System Property:** `aws.metricPublishingEnabled=true`
129+
130+
**Environment Variable:** `AWS_METRIC_PUBLISHING_ENABLED=true`
131+
132+
The value specified must be one of `"true"` or `"false"`. Specifying any other string values will result in
133+
a value of `"false"` being used, and a warning being logged each time an SDK client is created.
97134

98-
* Customers might want to have different ways of enabling the metrics feature. For example: use SystemProperties by
99-
default. If not use implementation based on Amazon DynamoDB.
100-
* To support multiple providers, SDK allows setting chain of providers (similar to the CredentialsProviderChain to
101-
resolve credentials). As provider has multiple configuration options, a single provider is resolved at chain
102-
construction time and it is used throughout the lifecycle of the application to keep the behavior intuitive.
103-
* If no custom chain is provided, SDK will use a default chain while looks for the System properties defined in above
104-
section. SDK can add more providers in the default chain in the future without breaking customers.
135+
When the value is `"false"`, no metrics will be published by a client.
136+
137+
When the value is `"true"`, metrics will be published by every client to a set of "global metric publishers". The set
138+
of global metric publishers is loaded automatically using the same mechanism currently used to discover HTTP
139+
clients. This means that including the `cloudwatch-metric-publisher` module and enabling the system property or
140+
environment variable above is sufficient to enable metric publishing to CloudWatch on all AWS clients.
141+
142+
The set of "Global Metric Publishers" is static and is used for *all* AWS SDK clients instantiated by the application
143+
(while **Option 3** remains enabled). A JVM shutdown hook will be registered to invoke `MetricPublisher.close()` on
144+
every publisher (in case the publishers use non-daemon threads that would otherwise block JVM shutdown).
145+
146+
#### Updating a MetricPublisher to work as a global metric publisher
147+
148+
**Option 3** above references the concept of "Global Metric Publishers", which are a set of publishers that are
149+
discovered automatically by the SDK. This section outlines how global metric publishers are discovered and created.
150+
151+
Each `MetricPublisher` that supports loading when **Option 3** is enabled must:
152+
1. Provide an `SdkMetricPublisherService` implementation. An `SdkMetricPublisherService` implementation is a class with
153+
a zero-arg constructor, used to instantiate a specific type of `MetricPublisher` (e.g. a
154+
`CloudWatchMetricPublisherService` that is a factory for `CloudWatchMetricPublisher`s).
155+
2. Provide a resource file: `META-INF/services/software.amazon.awssdk.metrics.SdkMetricPublisherService`. This file
156+
contains the list of fully-qualified `SdkMetricPublisherService` implementation class names.
157+
158+
The `software.amazon.awssdk.metrics.SdkMetricPublisherService` interface that must be implemented by all global metric
159+
publisher candidates is defined as:
105160

106161
```java
107-
MetricConfigurationProvider chain = new MetricConfigurationProviderChain(
108-
new SystemSettingsMetricConfigurationProvider(),
109-
// example custom implementation (not provided by the SDK)
110-
DynamoDBMetricConfigurationProvider.builder()
111-
.tableName(TABLE_NAME)
112-
.enabledKey(ENABLE_KEY_NAME)
113-
...
114-
.build(),
115-
);
116-
117-
ClientOverrideConfiguration config = ClientOverrideConfiguration
118-
.builder()
119-
// If this is not set, SDK uses the default chain with system property
120-
.metricConfigurationProvider(chain)
121-
.build();
122-
123-
// Set the ClientOverrideConfiguration instance on the client builder
124-
CodePipelineAsyncClient asyncClient =
125-
CodePipelineAsyncClient
126-
.builder()
127-
.overrideConfiguration(config)
128-
.build();
162+
public interface SdkMetricPublisherService {
163+
MetricPublisher createMetricPublisher();
164+
}
129165
```
130166

131-
### Metric Publishers Configuration
167+
**`SdkMetricPublisherService` Example**
132168

133-
* If metrics are enabled, SDK by default uses a single publisher that uploads metrics to CloudWatch using default
134-
credentials and region.
135-
* Customers might want to use different configuration for the CloudWatch publisher or even use a different publisher to
136-
publish to a different source. To provide this flexibility, SDK exposes an option to set
137-
[MetricPublisherConfiguration](prototype/MetricPublisherConfiguration.java) which can be used to configure custom
138-
publishers.
139-
* SDK publishes the collected metrics to each of the configured publishers in the MetricPublisherConfiguration.
169+
Enabling the `CloudWatchMetricPublisher` as a global metric publisher can be done by implementing the
170+
`SdkMetricPublisherService` interface:
140171

141172
```java
142-
ClientOverrideConfiguration config = ClientOverrideConfiguration
143-
.builder()
144-
.metricPublisherConfiguration(MetricPublisherConfiguration
145-
.builder()
146-
.addPublisher(
147-
CloudWatchPublisher.builder()
148-
.credentialsProvider(...)
149-
.region(Region.AP_SOUTH_1)
150-
.publishFrequency(5, TimeUnit.MINUTES)
151-
.build(),
152-
CsmPublisher.create()).bu
153-
.build())
154-
.build();
155-
156-
// Set the ClientOverrideConfiguration instance on the client builder
157-
CodePipelineAsyncClient asyncClient =
158-
CodePipelineAsyncClient
159-
.builder()
160-
.overrideConfiguration(config)
161-
.build();
173+
package software.amazon.awssdk.metrics.publishers.cloudwatch;
174+
175+
public final class CloudWatchSdkMetricPublisherService implements SdkMetricPublisherService {
176+
@Override
177+
public MetricPublisher createMetricPublisher() {
178+
return CloudWatchMetricPublisher.create();
179+
}
180+
}
162181
```
163182

183+
And creating a `META-INF/services/software.amazon.awssdk.metrics.SdkMetricPublisherService` resource file in the
184+
`cloudwatch-metric-publisher` module with the following contents:
185+
186+
```
187+
software.amazon.awssdk.metrics.publishers.cloudwatch.CloudWatchSdkMetricPublisherService
188+
```
189+
190+
#### Option 3 Implementation Details and Edge Cases
191+
192+
**How the SDK loads `MetricPublisher`s when Option 3 is enabled**
193+
194+
When a client is created with **Option 3** enabled (and **Option 2** "not specified"), the client retrieves the list of
195+
global metric publishers to use via a static "global metric publisher list" singleton. This singleton is initialized
196+
exactly once using the following process:
197+
1. The singleton uses `java.util.ServiceLoader` to locate all `SdkMetricPublisherService` implementations configured
198+
as described above. The classloader used with the service loader is chosen in the same manner as the one chosen for the
199+
HTTP client service loader (`software.amazon.awssdk.core.internal.http.loader.SdkServiceLoader`). That is, the first
200+
classloader present in the following list: (1) the classloader that loaded the SDK, (2) the current thread's classloader,
201+
then (3) the system classloader.
202+
2. The singleton creates an instance of every `SdkMetricPublisherService` located in this manner.
203+
3. The singleton creates an instance of each `MetricPublisher` instance using the metrics publisher services.
204+
205+
**How Option 3 and Option 1 behave when Option 2 is "not specified"**
206+
207+
The SDK treats **Option 3** as the default set of client-level metric publishers to be
208+
used when **Option 2** is "not specified". This means that if a customer: (1) enables global metric publishing using
209+
**Option 3**, (2) does not specify client-level publishers using **Option 2**, and (3) specifies metric publishers at
210+
the request level with **Option 1**, then the global metric publishers are still *instantiated* but will not be used.
211+
This nuance prevents the SDK from needing to consult the global metric configuration with every request.
212+
213+
**How Option 2 is considered "not specified" for the purposes of considering Option 3**
214+
215+
Global metric publishers (**Option 3**) are only considered for use when **Option 2** is "not specified".
216+
217+
"Not specified" is defined to be when the customer either: (1) does not invoke
218+
`ClientOverrideConfiguration.Builder.addMetricPublisher()` / `ClientOverrideConfiguration.Builder.metricPublishers()`,
219+
or (2) invokes `ClientOverrideConfiguration.Builder.metricPublishers(null)` as the last `metricPublisher`-mutating
220+
action on the client override configuration builder.
221+
222+
This definition purposefully excludes `ClientOverrideConfiguration.Builder.metricPublishers(emptyList())`. Setting
223+
the `metricPublishers` to an empty list is equivalent to setting the `metricPublishers` to the `NoOpMetricPublisher`.
224+
225+
**Implementing an SdkMetricPublisherService that depends on an AWS clients**
226+
227+
Any `MetricPublisher`s that supports creation via a `SdkMetricPublisherService` and depends on an AWS service client
228+
**must** disable metric publishing on those AWS service clients using **Option 2** when they are created via the
229+
`SdkMetricPublisherService`. This is to prevent a scenario where the global metric publisher singleton's initialization
230+
process depends on the global metric publishers singleton already being initialized.
164231

165232
## Modules
166233
New modules are created to support metrics feature.
@@ -175,7 +242,6 @@ New modules are created to support metrics feature.
175242
* Under this module, a new sub-module is created for each publisher (`cloudwatch-publisher`, `csm-publisher`)
176243
* Customers has to **explicitly add dependency** on these modules to use the sdk provided publishers
177244

178-
179245
## Performance
180246
One of the main tenets for metrics is “Enabling default metrics should have
181247
minimal impact on the application performance". The following design choices are

‎docs/design/core/metrics/README.md

Lines changed: 0 additions & 69 deletions
This file was deleted.

0 commit comments

Comments
 (0)