Add new parameters to set Spark custom configurations #4172
Conversation
|
@jeffersonezra, |
There was a problem hiding this comment.
@jeffersonezra one comment about the markdown help
Also, can you provide tests for the new parameters?
| @@ -232,6 +234,58 @@ Accept pipeline input: False | |||
| Accept wildcard characters: False | |||
| ``` | |||
|
|
|||
| ### -Spark2Defaults | |||
| Specifies the Spark2 Defaults configurations of this HDInsight cluster.```yaml | |||
There was a problem hiding this comment.
@jeffersonezra can you fix this file so that the ```yaml is on a line by itself. For example:
### -RServer
Specifies the RServer configurations. Valid only for RServer clusters.
```yaml
Text
```
|
@azuresdkci test this please |
| @@ -82,6 +82,18 @@ public class AddAzureHDInsightConfigValuesCommand : HDInsightCmdletBase | |||
| [Parameter(HelpMessage = "Gets the RServer configurations.")] | |||
| public Hashtable RServer { get; set; } | |||
|
|
|||
| [Parameter(HelpMessage = "Gets the Spark Defaults configurations of this HDInsight cluster.")] | |||
| public Hashtable SparkDefaults { get; set; } | |||
There was a problem hiding this comment.
@jeffersonezra out of curiosity, is it possible for a configuration to have values for both Spark and Spark2 properties? For example, should it be valid for a user to provide values for SparkDefaults and Spark2Defaults?
| { | ||
| AzureHDInsightConfig config = new AzureHDInsightConfig(); | ||
|
|
||
| var addConfigValuesCmdlet = new AddAzureHDInsightConfigValuesCommand |
There was a problem hiding this comment.
@jeffersonezra should this test be updated since it is using Spark and Spark2 in the same cmdlet call? Or at least have this test split into two, one for each version?
There was a problem hiding this comment.
Not really required, it still tests the whether the configs are correctly set. But I've split it into two.
Description
This change adds new parameters SparkDefaults, SparkThriftConf, Spark2Defaults, Spark2ThriftConf to the Add-AzureHDInsightConfigValuesCommand cmdlet. The parameters are used to set the Spark-Defaults, Spark-Thrift-SparkConf, Spark2-Defaults and Spark2-Thrift-SparkConf Ambari sections of the configurations.
This checklist is used to make sure that common guidelines for a pull request are followed. You can find a more complete discussion of PowerShell cmdlet best practices here.
General Guidelines
Testing Guidelines
Cmdlet Signature Guidelines
ShouldProcessand haveSupportShouldProcess=truespecified in the cmdlet attribute. You can find more information onShouldProcesshere.OutputTypeattribute if any output is produced - if the cmdlet produces no output, it should implement aPassThruparameter.Cmdlet Parameter Guidelines