Skip to content

samples: query profile #1385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
9d0a267
feat: query profiling
kolea2 Dec 13, 2023
674850b
samples: query profiling
kolea2 Dec 13, 2023
b09775e
Merge branch 'main' of github.com:googleapis/java-datastore into qp
kolea2 Jan 4, 2024
1a47dee
Merge branch 'main' of github.com:googleapis/java-datastore into qp-s…
kolea2 Jan 4, 2024
c8b5ae3
Merge branch 'main' of github.com:googleapis/java-datastore into qp
kolea2 Jan 18, 2024
ca6a85e
fix clirr, run linter
kolea2 Jan 18, 2024
f23e1ec
review feedback
kolea2 Jan 18, 2024
151ce49
lint again
kolea2 Jan 18, 2024
4a6dc23
switch return type to optional for ResultSetStats
kolea2 Jan 18, 2024
7a8ee8e
Merge branch 'qp' of github.com:kolea2/java-datastore into qp-samples
kolea2 Jan 18, 2024
dd270b8
update API to use optional
kolea2 Jan 18, 2024
8b92953
Merge branch 'qp-samples' of github.com:kolea2/java-datastore into qp…
kolea2 Jan 18, 2024
31aa956
Merge branch 'main' of github.com:googleapis/java-datastore into qp
kolea2 Feb 13, 2024
d7bbb36
new api updates
kolea2 Feb 14, 2024
61f2e50
Merge branch 'main' of github.com:googleapis/java-datastore into qp
kolea2 Mar 9, 2024
058bc93
update clirr
kolea2 Mar 10, 2024
08d3563
Merge branch 'main' of github.com:googleapis/java-datastore into qp
kolea2 Mar 13, 2024
a439264
cleanup, add docs
kolea2 Mar 13, 2024
6c041b2
javadoc fixes
kolea2 Mar 13, 2024
e2c6927
Merge branch 'qp' of github.com:kolea2/java-datastore into qp-samples
kolea2 Mar 14, 2024
6ddd783
update samples to latest api
kolea2 Mar 14, 2024
c1cd702
small update
kolea2 Mar 22, 2024
6d5c8a7
merge upstream
kolea2 Mar 25, 2024
f5cc431
fix tests
kolea2 Mar 25, 2024
c19d1a8
checkstyle
kolea2 Mar 25, 2024
27e9107
rename variables
kolea2 Mar 27, 2024
950f9c1
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 27, 2024
7f17d56
Merge branch 'main' of github.com:googleapis/java-datastore into qp-s…
kolea2 Mar 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,10 @@ Samples are in the [`samples/`](https://github.com/googleapis/java-datastore/tre
| Sum Aggregation With Order By | [source code](https://github.com/googleapis/java-datastore/blob/main/samples/snippets/src/main/java/com/example/datastore/aggregation/SumAggregationWithOrderBy.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-datastore&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/datastore/aggregation/SumAggregationWithOrderBy.java) |
| Sum Aggregation With Property Filter | [source code](https://github.com/googleapis/java-datastore/blob/main/samples/snippets/src/main/java/com/example/datastore/aggregation/SumAggregationWithPropertyFilter.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-datastore&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/datastore/aggregation/SumAggregationWithPropertyFilter.java) |
| Create a union between two filters | [source code](https://github.com/googleapis/java-datastore/blob/main/samples/snippets/src/main/java/com/example/datastore/filters/OrFilterQuery.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-datastore&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/datastore/filters/OrFilterQuery.java) |
| Query Profile Explain | [source code](https://github.com/googleapis/java-datastore/blob/main/samples/snippets/src/main/java/com/example/datastore/queryprofile/QueryProfileExplain.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-datastore&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/datastore/queryprofile/QueryProfileExplain.java) |
| Query Profile Explain Aggregation | [source code](https://github.com/googleapis/java-datastore/blob/main/samples/snippets/src/main/java/com/example/datastore/queryprofile/QueryProfileExplainAggregation.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-datastore&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/datastore/queryprofile/QueryProfileExplainAggregation.java) |
| Query Profile Explain Analyze | [source code](https://github.com/googleapis/java-datastore/blob/main/samples/snippets/src/main/java/com/example/datastore/queryprofile/QueryProfileExplainAnalyze.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-datastore&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/datastore/queryprofile/QueryProfileExplainAnalyze.java) |
| Query Profile Explain Analyze Aggregation | [source code](https://github.com/googleapis/java-datastore/blob/main/samples/snippets/src/main/java/com/example/datastore/queryprofile/QueryProfileExplainAnalyzeAggregation.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-datastore&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/datastore/queryprofile/QueryProfileExplainAnalyzeAggregation.java) |
| Task List | [source code](https://github.com/googleapis/java-datastore/blob/main/samples/snippets/src/main/java/com/google/datastore/snippets/TaskList.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-datastore&page=editor&open_in_editor=samples/snippets/src/main/java/com/google/datastore/snippets/TaskList.java) |


Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
/*
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.datastore.queryprofile;

// [START datastore_query_explain_entity]

import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.Entity;
import com.google.cloud.datastore.Query;
import com.google.cloud.datastore.QueryResults;
import com.google.cloud.datastore.models.ExplainMetrics;
import com.google.cloud.datastore.models.ExplainOptions;
import com.google.cloud.datastore.models.PlanSummary;
import java.util.List;
import java.util.Map;
import java.util.Optional;

public class QueryProfileExplain {
public static void invoke() throws Exception {
// Instantiates a client
Datastore datastore = DatastoreOptions.getDefaultInstance().getService();

// Build the query
Query<Entity> query = Query.newEntityQueryBuilder().setKind("Task").build();

// Set the explain options to get back *only* the plan summary
QueryResults<Entity> results = datastore.run(query, ExplainOptions.newBuilder().build());

// Get the explain metrics
Optional<ExplainMetrics> explainMetrics = results.getExplainMetrics();
if (!explainMetrics.isPresent()) {
throw new Exception("No explain metrics returned");
}
PlanSummary planSummary = explainMetrics.get().getPlanSummary();
List<Map<String, Object>> indexesUsed = planSummary.getIndexesUsed();
System.out.println("----- Indexes Used -----");
indexesUsed.forEach(map -> map.forEach((key, val) -> System.out.println(key + ": " + val)));
}
}
// [END datastore_query_explain_entity]
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
/*
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.datastore.queryprofile;

// [START datastore_query_explain_aggregation]

import static com.google.cloud.datastore.aggregation.Aggregation.count;

import com.google.cloud.datastore.AggregationQuery;
import com.google.cloud.datastore.AggregationResults;
import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.Entity;
import com.google.cloud.datastore.Query;
import com.google.cloud.datastore.StructuredQuery;
import com.google.cloud.datastore.models.ExplainMetrics;
import com.google.cloud.datastore.models.ExplainOptions;
import com.google.cloud.datastore.models.PlanSummary;
import java.util.List;
import java.util.Map;
import java.util.Optional;

public class QueryProfileExplainAggregation {
public static void invoke() throws Exception {
// Instantiates a client
Datastore datastore = DatastoreOptions.getDefaultInstance().getService();

// Build the query
StructuredQuery<Entity> query = Query.newEntityQueryBuilder().setKind("Task").build();

// Build the aggregation query
AggregationQuery aggregationQuery =
Query.newAggregationQueryBuilder().over(query).addAggregation(count()).build();

// Set the explain options to get back *only* the plan summary
AggregationResults results =
datastore.runAggregation(aggregationQuery, ExplainOptions.newBuilder().build());

// Get the explain metrics
Optional<ExplainMetrics> explainMetrics = results.getExplainMetrics();
if (!explainMetrics.isPresent()) {
throw new Exception("No explain metrics returned");
}
PlanSummary planSummary = explainMetrics.get().getPlanSummary();
List<Map<String, Object>> indexesUsed = planSummary.getIndexesUsed();
System.out.println("----- Indexes Used -----");
indexesUsed.forEach(map -> map.forEach((key, val) -> System.out.println(key + ": " + val)));
}
}
// [END datastore_query_explain_aggregation]
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
/*
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.datastore.queryprofile;

// [START datastore_query_explain_analyze_entity]
import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.Entity;
import com.google.cloud.datastore.Query;
import com.google.cloud.datastore.QueryResults;
import com.google.cloud.datastore.models.ExecutionStats;
import com.google.cloud.datastore.models.ExplainMetrics;
import com.google.cloud.datastore.models.ExplainOptions;
import com.google.cloud.datastore.models.PlanSummary;
import java.util.List;
import java.util.Map;

public class QueryProfileExplainAnalyze {
public static void invoke() throws Exception {
// Instantiates a client
Datastore datastore = DatastoreOptions.getDefaultInstance().getService();

// Build the query
Query<Entity> query = Query.newEntityQueryBuilder().setKind("Task").build();

// Set explain options with analzye = true to get back the query stats, plan info, and query
// results
QueryResults<Entity> results =
datastore.run(query, ExplainOptions.newBuilder().setAnalyze(true).build());

// Get the result set stats
if (!results.getExplainMetrics().isPresent()) {
throw new Exception("No explain metrics returned");
}
ExplainMetrics explainMetrics = results.getExplainMetrics().get();

// Get the execution stats
if (!explainMetrics.getExecutionStats().isPresent()) {
throw new Exception("No execution stats returned");
}

ExecutionStats executionStats = explainMetrics.getExecutionStats().get();
Map<String, Object> debugStats = executionStats.getDebugStats();
System.out.println("----- Debug Stats -----");
debugStats.forEach((key, val) -> System.out.println(key + ": " + val));
System.out.println("----------");

long resultsReturned = executionStats.getResultsReturned();
System.out.println("Results returned: " + resultsReturned);

// Get the plan summary
PlanSummary planSummary = explainMetrics.getPlanSummary();
List<Map<String, Object>> indexesUsed = planSummary.getIndexesUsed();
System.out.println("----- Indexes Used -----");
indexesUsed.forEach(map -> map.forEach((key, val) -> System.out.println(key + ": " + val)));

if (!results.hasNext()) {
throw new Exception("query yielded no results");
}

// Get the query results
System.out.println("----- Query Results -----");
while (results.hasNext()) {
Entity entity = results.next();
System.out.printf("Entity: %s%n", entity);
}
}
}
// [END datastore_query_explain_analyze_entity]
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
/*
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.datastore.queryprofile;

// [START datastore_query_explain_analyze_aggregation]

import static com.google.cloud.datastore.aggregation.Aggregation.count;

import com.google.cloud.datastore.AggregationQuery;
import com.google.cloud.datastore.AggregationResult;
import com.google.cloud.datastore.AggregationResults;
import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.Entity;
import com.google.cloud.datastore.Query;
import com.google.cloud.datastore.StructuredQuery;
import com.google.cloud.datastore.models.ExecutionStats;
import com.google.cloud.datastore.models.ExplainMetrics;
import com.google.cloud.datastore.models.ExplainOptions;
import com.google.cloud.datastore.models.PlanSummary;
import com.google.common.collect.Iterables;
import java.util.List;
import java.util.Map;

public class QueryProfileExplainAnalyzeAggregation {
public static void invoke() throws Exception {
// Instantiates a client
Datastore datastore = DatastoreOptions.getDefaultInstance().getService();

// Build the query
StructuredQuery<Entity> query = Query.newEntityQueryBuilder().setKind("Task").build();

// Build the aggregation query
AggregationQuery aggregationQuery =
Query.newAggregationQueryBuilder()
.over(query)
.addAggregation(count().as("countVal"))
.build();

// Set the explain options with analyze = true to get back the query stats, plan info, and
// aggregation results
AggregationResults results =
datastore.runAggregation(
aggregationQuery, ExplainOptions.newBuilder().setAnalyze(true).build());

if (!results.getExplainMetrics().isPresent()) {
throw new Exception("No explain metrics returned");
}
ExplainMetrics explainMetrics = results.getExplainMetrics().get();

if (!explainMetrics.getExecutionStats().isPresent()) {
throw new Exception("No execution stats returned");
}
ExecutionStats executionStats = explainMetrics.getExecutionStats().get();
Map<String, Object> debugStats = executionStats.getDebugStats();
System.out.println("----- Debug Stats -----");
debugStats.forEach((key, val) -> System.out.println(key + ": " + val));
System.out.println("----------");

long resultsReturned = executionStats.getResultsReturned();
System.out.println("Results returned: " + resultsReturned);

// Get the plan summary
PlanSummary planSummary = explainMetrics.getPlanSummary();
List<Map<String, Object>> indexesUsed = planSummary.getIndexesUsed();
System.out.println("----- Indexes Used -----");
indexesUsed.forEach(map -> map.forEach((key, val) -> System.out.println(key + ": " + val)));

System.out.println("----- Aggregation Results -----");
AggregationResult result = Iterables.getOnlyElement(results);
System.out.println("Count: " + result.getLong("countVal"));
}
}
// [END datastore_query_explain_analyze_aggregation]
Original file line number Diff line number Diff line change
Expand Up @@ -14,27 +14,10 @@
* limitations under the License.
*/

package com.example.datastore;
package com.example.datastore.aggregation;

import static org.junit.Assert.assertThrows;

import com.example.datastore.aggregation.AvgAggregationOnKind;
import com.example.datastore.aggregation.AvgAggregationWithLimit;
import com.example.datastore.aggregation.AvgAggregationWithOrderBy;
import com.example.datastore.aggregation.AvgAggregationWithPropertyFilter;
import com.example.datastore.aggregation.CountAggregationInTransaction;
import com.example.datastore.aggregation.CountAggregationOnKind;
import com.example.datastore.aggregation.CountAggregationWithGqlQuery;
import com.example.datastore.aggregation.CountAggregationWithLimit;
import com.example.datastore.aggregation.CountAggregationWithOrderBy;
import com.example.datastore.aggregation.CountAggregationWithPropertyFilter;
import com.example.datastore.aggregation.CountAggregationWithStaleRead;
import com.example.datastore.aggregation.MultipleAggregationsInGqlQuery;
import com.example.datastore.aggregation.MultipleAggregationsInStructuredQuery;
import com.example.datastore.aggregation.SumAggregationOnKind;
import com.example.datastore.aggregation.SumAggregationWithLimit;
import com.example.datastore.aggregation.SumAggregationWithOrderBy;
import com.example.datastore.aggregation.SumAggregationWithPropertyFilter;
import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.Key;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,8 @@
* limitations under the License.
*/

package com.example.datastore;
package com.example.datastore.filters;

import com.example.datastore.filters.OrFilterQuery;
import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.Entity;
Expand Down
Loading