Deprecate validateAndCollect in favor of explicitly calling loadCollectors

Author: justin-tay

doc/collector-context.md

@@ -1,6 +1,6 @@
 ### CollectorContext
 
-There could be use cases where we want collect the information while we are validating the data. A simple example could be fetching some value from a database or from a microservice based on the data (which could be a text or a JSON object. It should be noted that this should be a simple operation or validation might take more time to complete.) in a given JSON node and the schema keyword we are using. 
+There could be use cases where we want collect the information while we are validating the data. A simple example could be fetching some value from a database or from a microservice based on the data (which could be a text or a JSON object. It should be noted that this should be a simple operation or validation might take more time to complete.) in a given JSON node and the schema keyword we are using.
 
 The fetched data can be stored somewhere so that it can be used later after the validation is done. Since the current validation logic already parses the data and schema, both validation and collecting the required information can be done in one go.
 
@@ -10,6 +10,12 @@ The `CollectorContext` and `Collector` classes are designed to work with this us
 
 The `CollectorContext` is stored as a variable on the `ExecutionContext` that is used during the validation. This allows users to add objects to context at many points in the framework like Formats and Validators where the `ExecutionContext` is available as a parameter.
 
+By default the `CollectorContext` created by the library contains maps backed by `HashMap`. If the `CollectorContext` needs to be shared by multiple threads then a `ConcurrentHashMap` needs to be used.
+
+```java
+CollectorContext collectorContext = new CollectorContext(new ConcurrentHashMap<>(), new ConcurrentHashMap<>());
+```
+
 Collectors are added to `CollectorContext`. Collectors allow to collect the objects. A `Collector` is added to `CollectorContext` with a name and corresponding `Collector` instance.
 
 ```java
@@ -28,29 +34,44 @@ However there might be use cases where we want to add a simple Object like Strin
 
 ```java
 CollectorContext collectorContext = executionContext.getCollectorContext();
-collectorContext.add(SAMPLE_COLLECTOR, "sample-string")
+collectorContext.add(SAMPLE_COLLECTOR, "sample-string");
 ```
 
-To use the `CollectorContext` while validating, the `validateAndCollect` method has to be invoked on the `JsonSchema` class.
-This method returns a `ValidationResult` that contains the errors encountered during validation and a `ExecutionContext` instance that contains the `CollectorContext`.
-Objects constructed by collectors or directly added to `CollectorContext` can be retrieved from `CollectorContext` by using the name they were added with.
-
-To collect across multiple validation runs, the `CollectorContext` needs to be explicitly reused by passing the `ExecutionContext` as a parameter to the validation.
+Implementations that need to modify values the `CollectorContext` should do so in a thread-safe manner.
 
 ```java
-ValidationResult validationResult = jsonSchema.validateAndCollect(jsonNode);
-ExecutionContext executionContext = validationResult.getExecutionContext();
 CollectorContext collectorContext = executionContext.getCollectorContext();
-List<String> contextValue = (List<String>) collectorContext.get(SAMPLE_COLLECTOR);
+AtomicInteger count = (AtomicInteger) collectorContext.getCollectorMap().computeIfAbsent(SAMPLE_COLLECTOR,
+        (key) -> new AtomicInteger(0));
+count.incrementAndGet();
+```
+
+To use the `CollectorContext` while validating, the `CollectorContext` should be instantiated outside and set for every validation execution.
 
-// Do something with contextValue
-...
+At the end of all the runs the `CollectorContext.loadCollectors()` method can be called if needed for the `Collector` implementations to aggregate values.
 
-// To collect more information for subsequent runs reuse the context
-validationResult = jsonSchema.validateAndCollect(executionContext, jsonNode);
+```java
+// This creates a CollectorContext that can be used by multiple threads although this is not neccessary in this example
+CollectorContext collectorContext = new CollectorContext(new ConcurrentHashMap<>(), new ConcurrentHashMap<>());
+// This adds a custom collect keyword that sets values in the CollectorContext whenever it gets processed
+JsonMetaSchema metaSchema = JsonMetaSchema.builder(JsonMetaSchema.getV202012()).keyword(new CollectKeyword()).build();
+JsonSchemaFactory factory = JsonSchemaFactory.getInstance(VersionFlag.V202012, builder -> builder.metaSchema(metaSchema));
+JsonSchema schema = factory.getSchema("{\n"
+        + "  \"collect\": true\n"
+        + "}");
+for (int i = 0; i < 50; i++) {
+    // The shared CollectorContext is set on the ExecutionContext for every run to aggregate data from all the runs
+    schema.validate("1", InputFormat.JSON, executionContext -> {
+        executionContext.setCollectorContext(collectorContext);
+    });
+}
+// This is called for Collector implementations to aggregate data
+collectorContext.loadCollectors();
+AtomicInteger result = (AtomicInteger) collectorContext.get("collect");
+assertEquals(50, result.get());
 ```
 
-There might be use cases where a collector needs to collect the data at multiple touch points. For example one use case might be collecting data in a validator and a formatter. If you are using a `Collector` rather than a `Object`, the combine method of the `Collector` allows to define how we want to combine the data into existing `Collector`. `CollectorContext` `combineWithCollector` method calls the combine method on the `Collector`. User just needs to call the `CollectorContext` `combineWithCollector` method every time some data needs to merged into existing `Collector`. The `collect` method on the `Collector` is called by the framework at the end of validation to return the data that was collected.
+There might be use cases where a collector needs to collect the data at multiple touch points. For example one use case might be collecting data in a validator and a formatter. If you are using a `Collector` rather than a `Object`, the combine method of the `Collector` allows to define how we want to combine the data into existing `Collector`. `CollectorContext` `combineWithCollector` method calls the combine method on the `Collector`. User just needs to call the `CollectorContext` `combineWithCollector` method every time some data needs to merged into existing `Collector`. The `collect` method on the `Collector` is called by explicitly calling `CollectorContext.loadCollectors()` at the end of processing.
 
 ```java
 class CustomCollector implements Collector<List<String>> {
@@ -70,16 +91,25 @@ class CustomCollector implements Collector<List<String>> {
 
     @Override
     public void combine(Object object) {
-        returnList.add(referenceMap.get((String) object));
+        synchronized(returnList) {
+            returnList.add(referenceMap.get((String) object));
+        }
     }
 }
+```
 
-CollectorContext collectorContext = executionContext.getCollectorContext();
-if (collectorContext.get(SAMPLE_COLLECTOR) == null) {
-    collectorContext.add(SAMPLE_COLLECTOR, new CustomCollector());
+```java
+private class CustomValidator extends AbstractJsonValidator {
+    @Override
+    public Set<ValidationMessage> validate(ExecutionContext executionContext, JsonNode node, JsonNode rootNode,
+            JsonNodePath instanceLocation) {
+        CollectorContext collectorContext = executionContext.getCollectorContext();
+        CustomCollector customCollector = (CustomCollector) collectorContext.getCollectorMap().computeIfAbsent(SAMPLE_COLLECTOR,
+                key -> new CustomCollector());
+        customCollector.combine(node.textValue());
+        return Collections.emptySet();
+    }
 }
-collectorContext.combineWithCollector(SAMPLE_COLLECTOR, node.textValue());
-
 ```
 
 One important thing to note when using Collectors is if we call get method on `CollectorContext` before the validation is complete, we would get back a `Collector` instance that was added to `CollectorContext`.
@@ -96,12 +126,9 @@ List<String> data = collectorContext.get(SAMPLE_COLLECTOR);
 If you are using simple objects and if the data needs to be collected from multiple touch points, logic is straightforward as shown.
 
 ```java
-CollectorContext collectorContext = executionContext.getCollectorContext();
-// If collector name is not added to context add one.
-if (collectorContext.get(SAMPLE_COLLECTOR) == null) {
-    collectorContext.add(SAMPLE_COLLECTOR, new ArrayList<String>());
+List<String> returnList = (List<String>) collectorContext.getCollectorMap()
+        .computeIfAbsent(SAMPLE_COLLECTOR, key -> new ArrayList<String>());
+synchronized(returnList) {
+    returnList.add(node.textValue());
 }
-// In this case we are adding a list to CollectorContext.
-List<String> returnList = (List<String>) collectorContext.get(SAMPLE_COLLECTOR);
-
-```
+```

src/main/java/com/networknt/schema/CollectorContext.java

@@ -140,7 +140,7 @@ public void combineWithCollector(String name, Object data) {
     /**
      * Loads data from all collectors.
      */
-    void loadCollectors() {
+    public void loadCollectors() {
         Set<Entry<String, Object>> entrySet = this.collectorMap.entrySet();
         for (Entry<String, Object> entry : entrySet) {
             if (entry.getValue() instanceof Collector<?>) {

src/main/java/com/networknt/schema/JsonSchema.java

@@ -942,11 +942,21 @@ private JsonNode deserialize(String input, InputFormat inputFormat) {
         }
     }
 
+    /**
+     * Deprecated. Initialize the CollectorContext externally and call loadCollectors when done.
+     * <p>
+     * @param executionContext ExecutionContext
+     * @param node JsonNode
+     * @return ValidationResult
+     */
+    @Deprecated
     public ValidationResult validateAndCollect(ExecutionContext executionContext, JsonNode node) {
         return validateAndCollect(executionContext, node, node, atRoot());
     }
 
     /**
+     * Deprecated. Initialize the CollectorContext externally and call loadCollectors when done.
+     * <p>
      * This method both validates and collects the data in a CollectorContext.
      * Unlike others this methods cleans and removes everything from collector
      * context before returning.
@@ -957,6 +967,7 @@ public ValidationResult validateAndCollect(ExecutionContext executionContext, Js
      *
      * @return ValidationResult
      */
+    @Deprecated
     private ValidationResult validateAndCollect(ExecutionContext executionContext, JsonNode jsonNode, JsonNode rootNode, JsonNodePath instanceLocation) {
         // Validate.
         Set<ValidationMessage> errors = validate(executionContext, jsonNode, rootNode, instanceLocation);
@@ -976,6 +987,13 @@ private ValidationResult validateAndCollect(ExecutionContext executionContext, J
         return new ValidationResult(errors, executionContext);
     }
 
+    /**
+     * Deprecated. Initialize the CollectorContext externally and call loadCollectors when done.
+     *
+     * @param node JsonNode
+     * @return ValidationResult
+     */
+    @Deprecated
     public ValidationResult validateAndCollect(JsonNode node) {
         return validateAndCollect(createExecutionContext(), node, node, atRoot());
     }
@@ -984,6 +1002,38 @@ public ValidationResult validateAndCollect(JsonNode node) {
 
     /*********************** START OF WALK METHODS **********************************/
 
+    /**
+     * Walk the JSON node.
+     * 
+     * @param executionContext the execution context
+     * @param node             the input
+     * @param validate         true to validate the input against the schema
+     * @param executionCustomizer the customizer
+     *
+     * @return the validation result
+     */
+    public ValidationResult walk(ExecutionContext executionContext, JsonNode node, boolean validate,
+            ExecutionContextCustomizer executionCustomizer) {
+        return walkAtNodeInternal(executionContext, node, node, atRoot(), validate, OutputFormat.RESULT,
+                executionCustomizer);
+    }
+
+    /**
+     * Walk the JSON node.
+     * 
+     * @param executionContext the execution context
+     * @param node             the input
+     * @param validate         true to validate the input against the schema
+     * @param executionCustomizer the customizer
+     *
+     * @return the validation result
+     */
+    public ValidationResult walk(ExecutionContext executionContext, JsonNode node, boolean validate,
+            Consumer<ExecutionContext> executionCustomizer) {
+        return walkAtNodeInternal(executionContext, node, node, atRoot(), validate, OutputFormat.RESULT,
+                executionCustomizer);
+    }
+
     /**
      * Walk the JSON node.
      * 
@@ -994,9 +1044,11 @@ public ValidationResult validateAndCollect(JsonNode node) {
      * @return the validation result
      */
     public ValidationResult walk(ExecutionContext executionContext, JsonNode node, boolean validate) {
-        return walkAtNodeInternal(executionContext, node, node, atRoot(), validate);
+        return walkAtNodeInternal(executionContext, node, node, atRoot(), validate, OutputFormat.RESULT,
+                (ExecutionContextCustomizer) null);
     }
 
+
     /**
      * Walk the input.
      * 
@@ -1009,7 +1061,41 @@ public ValidationResult walk(ExecutionContext executionContext, JsonNode node, b
     public ValidationResult walk(ExecutionContext executionContext, String input, InputFormat inputFormat,
             boolean validate) {
         JsonNode node = deserialize(input, inputFormat);
-        return walkAtNodeInternal(executionContext, node, node, atRoot(), validate);
+        return walkAtNodeInternal(executionContext, node, node, atRoot(), validate, OutputFormat.RESULT,
+                (ExecutionContextCustomizer) null);
+    }
+
+    /**
+     * Walk the input.
+     * 
+     * @param executionContext the execution context
+     * @param input            the input
+     * @param inputFormat      the input format
+     * @param validate         true to validate the input against the schema
+     * @param executionCustomizer the customizer
+     * @return the validation result
+     */
+    public ValidationResult walk(ExecutionContext executionContext, String input, InputFormat inputFormat,
+            boolean validate, ExecutionContextCustomizer executionCustomizer) {
+        JsonNode node = deserialize(input, inputFormat);
+        return walkAtNodeInternal(executionContext, node, node, atRoot(), validate, OutputFormat.RESULT, executionCustomizer);
+    }
+
+    /**
+     * Walk the input.
+     * 
+     * @param executionContext the execution context
+     * @param input            the input
+     * @param inputFormat      the input format
+     * @param outputFormat      the output format
+     * @param validate         true to validate the input against the schema
+     * @param executionCustomizer the customizer
+     * @return the validation result
+     */
+    public <T> T walk(ExecutionContext executionContext, String input, InputFormat inputFormat,
+            OutputFormat<T> outputFormat, boolean validate, ExecutionContextCustomizer executionCustomizer) {
+        JsonNode node = deserialize(input, inputFormat);
+        return walkAtNodeInternal(executionContext, node, node, atRoot(), validate, outputFormat, executionCustomizer);
     }
 
     /**
@@ -1035,6 +1121,34 @@ public ValidationResult walk(String input, InputFormat inputFormat, boolean vali
         return walk(createExecutionContext(), deserialize(input, inputFormat), validate);
     }
 
+    /**
+     * Walk the input.
+     * 
+     * @param input       the input
+     * @param inputFormat the input format
+     * @param validate    true to validate the input against the schema
+     * @param executionCustomizer the customizer
+     * @return the validation result
+     */
+    public ValidationResult walk(String input, InputFormat inputFormat, boolean validate,
+            ExecutionContextCustomizer executionCustomizer) {
+        return walk(createExecutionContext(), deserialize(input, inputFormat), validate, executionCustomizer);
+    }
+
+    /**
+     * Walk the input.
+     * 
+     * @param input       the input
+     * @param inputFormat the input format
+     * @param validate    true to validate the input against the schema
+     * @param executionCustomizer the customizer
+     * @return the validation result
+     */
+    public ValidationResult walk(String input, InputFormat inputFormat, boolean validate,
+            Consumer<ExecutionContext> executionCustomizer) {
+        return walk(createExecutionContext(), deserialize(input, inputFormat), validate, executionCustomizer);
+    }
+
     /**
      * Walk at the node.
      * 
@@ -1047,25 +1161,39 @@ public ValidationResult walk(String input, InputFormat inputFormat, boolean vali
      */
     public ValidationResult walkAtNode(ExecutionContext executionContext, JsonNode node, JsonNode rootNode,
             JsonNodePath instanceLocation, boolean validate) {
-        return walkAtNodeInternal(executionContext, node, rootNode, instanceLocation, validate);
+        return walkAtNodeInternal(executionContext, node, rootNode, instanceLocation, validate, OutputFormat.RESULT,
+                (ExecutionContextCustomizer) null);
     }
 
-    private ValidationResult walkAtNodeInternal(ExecutionContext executionContext, JsonNode node, JsonNode rootNode,
-            JsonNodePath instanceLocation, boolean shouldValidateSchema) {
+    private <T> T walkAtNodeInternal(ExecutionContext executionContext, JsonNode node, JsonNode rootNode,
+            JsonNodePath instanceLocation, boolean validate, OutputFormat<T> format, Consumer<ExecutionContext> executionCustomizer) {
+        return walkAtNodeInternal(executionContext, node, rootNode, instanceLocation, validate, format,
+                (executeContext, validationContext) -> {
+                    executionCustomizer.accept(executeContext);
+                });
+    }
+
+    private <T> T walkAtNodeInternal(ExecutionContext executionContext, JsonNode node, JsonNode rootNode,
+            JsonNodePath instanceLocation, boolean validate, OutputFormat<T> format,
+            ExecutionContextCustomizer executionCustomizer) {
+        if (executionCustomizer != null) {
+            executionCustomizer.customize(executionContext, this.validationContext);
+        }
         // Walk through the schema.
-        Set<ValidationMessage> errors = walk(executionContext, node, rootNode, instanceLocation, shouldValidateSchema);
+        Set<ValidationMessage> errors = walk(executionContext, node, rootNode, instanceLocation, validate);
 
         // Get the config.
         SchemaValidatorsConfig config = this.validationContext.getConfig();
         // When walk is called in series of nested call we don't want to load the collectors every time. Leave to the API to decide when to call collectors.
+        /* When doLoadCollectors is removed after the deprecation period the following block should be removed */
         if (config.doLoadCollectors()) {
             // Get the collector context.
             CollectorContext collectorContext = executionContext.getCollectorContext();
 
             // Load all the data from collectors into the context.
             collectorContext.loadCollectors();
         }
-        return new ValidationResult(errors, executionContext);
+        return format.format(this, errors, executionContext, this.validationContext);
     }
 
     @Override