Merge branch 'clu' of https://github.com/amarzavery/azure-powershell into clu

Author: amarzavery

clu-getstart.md

@@ -2,10 +2,12 @@
 
 ### Prerequsites
 
-Visual Studio 2015 RTM with ASP.NET. For details, check out the [installation doc](http://docs.asp.net/en/latest/getting-started/installing-on-windows.html). 
+* Visual Studio 2015 RTM with ASP.NET. For details, check out the [installation doc](http://docs.asp.net/en/latest/getting-started/installing-on-windows.html). 
 
 Note, after done, run `dnvm list` command to check the 'coreclr' runtime is installed with right version of `1.0.0-rc1-final`. If not, run `dnvm install 1.0.0-rc1-final -r coreclr -a x64 -p`. Remember always use `-p` flag, so the selection can persist. 
 
+* Get the latest dotnet from "https://azureclu.blob.core.windows.net/tools/dotnet-win-x64.latest.zip", unzip, then add its bin folder to the PATH
+
 ### Project Artifacts
 
 CLUPackages require some additional files to direct generation of indexing, and to provide shortcuts when files are installed.  These files can be copied from the Profile project and updated for each package.   
@@ -92,6 +94,26 @@ Testing will consist of scenario tests and unit tests. Scenario tests should be
 #### Scenario Tests
 - Scenario tests should be saved under `./examples` directory with one directory per package. Each scenario tests should (eventually) consist of both `.ps1` and `.sh` files and should cover "P0" scenarios.
 
+##### Environment Variables for Authentication
+Please set the environment variables for either Username/Password (no 2FA) or ServicePrincipal authentication:
+
+**Username/Password (without 2-factor auth):**
+
+|  Field (case sensitive) |  Description  |
+| ------------- |:-------------|
+| azureUser | an OrgId user name |
+| password | a service principal name |
+| userSubscription | (optional) Selects a particular subscription by id.  If not provided, the first listed subscription will be selected |
+
+**Service Principal:**
+
+|  Field (case sensitive) |  Description  |
+| ------------- |:-------------|
+| spn | The tenant guid to authenticate against |
+| secret | the password or application secret to sue for authentication |
+| tenant | The tenant guid to authenticate against |
+| spnSubscription | (optional) Selects a particular subscription by id.  If not provided, the first listed subscription will be selected |
+
 ##### XUnit Automation For Bash Scenario Tests
 - The ```Commands.Common.ScenarioTest``` project contains classes that enable executing bash scenario tests in Visual Studio, or cross-platform using dnx.
 
@@ -116,20 +138,22 @@ Testing will consist of scenario tests and unit tests. Scenario tests should be
         _fixture.GetRunner("resource-management").RunScript("01-ResourceGroups");
     }
     ```
-    - Set the environment variable 'TestCredentials' to a connection string providing the credentials to use during test execution. Possible fields include:
-    
-      |  Field        |  Description  |
-      | ------------- |:-------------|
-      |  Username     | an OrgId user name |
-      | ServicePrincipal | a service principal name |
-      | Password      | the password or application secret to sue for authentication |
-      | TenantId      | (required for Service authentication) The tenant guid to authenticate against |
-      | SubscriptionID | (optional) Selects a particular subscription by id.  If not provided, the first listed subscription will be selected |
-    - The infrastructure automatically generates a resource group name and assigns the value to the bash variable ```"$resourceGroupName"```.  If your scripts require additional variables, you can add these to your environment before running tests, or you can generate values using the ScriptRunner (for the tests using that runner).
+    - Set the [environment variables](#Environment_Variables_for_Authentication) for either Username/Password (no 2FA) or ServicePrincipal authentication
+    - Update PATH to include location of CLU bin drop.
+    ```bash
+   export PATH=/<path-to-drop>/clurun/win7-x64/:$PATH
+    ```
+    - The infrastructure automatically generates the following environment variables:
+	   - `BASEDIR` - directory path where test script is located
+	   - `location` - default "WestUS" location 
+	   - `groupName` - randomly generated resource group name (note: the test guarantees that this resource group is deleted at the end of a test run; any other resource groups generated as part of the test run need to be deleted by the test)
+	   - `storageAccountType` - default "Standard_GRS" storage account type 
+	   - `storageAccountName` - randomly generated storage account name 
+   - If the script require additional variables, you can add these to your environment before running tests, or you can generate values using the ScriptRunner (for the tests using that runner).
     ```C#
-        runner.EnvironmentVariables.Add("myVariableName", runner.GenerateName("myres"));
+runner.EnvironmentVariables.Add("myVariableName", runner.GenerateName("myres"));
     ```
-    - Tests can be executed in vs, or by runnign ```dnx test project.json```.  If you execute dnx test from the project directory, it will work without modification and a log file for each script will be written to the test results directory ```..\TestResults```.  If you execute dnx test from a different directory, you must set the following environment variables to provide the path to the examples directory and where to write log files:
+    - Tests can be executed in Visual Studio, or by running ```dnx test project.json```.  If you execute dnx test from the project directory, it will work without modification and a log file for each script will be written to the test results directory ```..\TestResults```.  If you execute dnx test from a different directory, you must set the following environment variables to provide the path to the examples directory and where to write log files:
 
       |  Environment Variable  |  Description  |
       | ------------- |:-------------|
@@ -138,14 +162,14 @@ Testing will consist of scenario tests and unit tests. Scenario tests should be
 
 ##### Running Bash Tests using Bash shell
 - Bash tests should be runnable from bash shell in windows/linux/mac environments.
-- To manually run the tests; please set the following envt. variables for authentication and run `./examples/lib/testrunner.sh`
+- To manually run the tests; please set [environment variables](#Environment_Variables_for_Authentication) for authentication as well as update PATH and run `./examples/lib/testrunner.sh`
+
    ```bash
-   export azureuser=<username@contosocorp.com>
-   export azurepassword=<your_password>
-   export PATH=$PATH:/<path-to-drop>/clurun/win7-x64/
-   . /examples/lib/testrunner.sh
+   export azureUser=<[email protected]>
+   export password=<your_password>
+   export PATH=/<path-to-drop>/clurun/win7-x64/:$PATH
    ```
-- All the parameters to the cmdlets should be passed in as envt. variables
+- All the parameters to the cmdlets should be passed in as environment variables
 - The current test runners will provide a unique resource group name via `$groupName` but may not remove it at the end if the test fails.
 - The location for ARM will be provided via variable `$location`.
 - "jq" package and BASH assert (e.g. `[ "foo" == "bar" ]`) should be used to validate the responses.

examples/compute-management/01-VirtualMachineSizes.ps1

@@ -0,0 +1,6 @@
+Param(
+  [string]$resourceGroupName,
+  [string]$resourceGroupLocation
+)
+
+Write-Host "Skip"

examples/compute-management/01-VirtualMachineSizes.sh

@@ -0,0 +1,40 @@
+#!/bin/bash
+set -e
+printf "\n=== Managing Virtual Machine Sizes in Azure Compute ===\n"
+
+printf "\n1. Showing VM size results in location: %s.\n" "$location"
+azure vmsize get --location "$location"
+
+printf "\n2. Checking VM size results in location: %s.\n" "$location"
+vmSizeResult=`azure vmsize get --location "$location"`
+
+if [[ $vmSizeResult == "" ]]; then
+    echo "Failure: No VM sizes!" 1>&2
+    exit 1
+else
+    echo "Success: Non-empty Results."
+fi
+
+filterResult=`azure vmsize get --location "$location" | cat | jq 'select(.name | contains("Standard_A0"))' --raw-output`
+if [[ "$filterResult" == "" ]]; then
+    echo "Failure: Standard_A0 vm size not found." 1>&2
+    exit 1
+else
+    echo "Success: Standard_A0 vm size found."
+fi
+
+filterResult=`azure vmsize get --location "$location" | cat | jq 'select(.name | contains("Standard_G1"))' --raw-output`
+if [[ "$filterResult" == "" ]]; then
+    echo "Failure: Standard_G1 vm size not found." 1>&2
+    exit 1
+else
+    echo "Success: Standard_G1 vm size found."
+fi
+
+filterResult=`azure vmsize get --location "$location" | cat | jq 'select(.name | contains("NonStandard_A1"))' --raw-output`
+if [[ "$filterResult" == "" ]]; then
+    echo "Success: NonStandard_A1 vm size not found."
+else
+    echo "Failure: NonStandard_A1 vm size found." 1>&2
+    exit 1
+fi 

examples/compute-management/02-VirtualMachineCreation.ps1

@@ -0,0 +1,6 @@
+Param(
+  [string]$resourceGroupName,
+  [string]$resourceGroupLocation
+)
+
+Write-Host "Skip"

examples/compute-management/02-VirtualMachineCreation.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+set -e
+printf "\n=== Managing Virtual Machine Creation in Azure Compute ===\n"
+
+printf "\n1. Creating a new resource group: %s and location: %s.\n" "$groupName" "$location"
+azure group create -n "$groupName" --location "$location"
+
+printf "\n2. Creating a new storage account '%s' in type '%s'.\n" "$storageAccountName" "$storageAccountType"
+azure storage account new --resourcegroupname "$groupName" --name "$storageAccountName" --location "$location" --type "$storageAccountType"
+
+printf "\n3. Create virtual network.\n"
+result=`azure virtual network new --resourcegroupname "$groupName" --name test --location "$location" --addressprefix "[\"10.0.0.0/16\"]" --subnet "[{\"Name\":\"test\",\"AddressPrefix\":\"10.0.0.0/24\"}]" --force`
+
+contextResult=`azure context get`
+
+subId=`echo $contextResult | jq '.Subscription.SubscriptionId' --raw-output`
+
+subnetId="/subscriptions/$subId/resourceGroups/$groupName/providers/Microsoft.Network/virtualNetworks/test/subnets/test"
+
+printf "\n4. Create network interface with:\r\nsubId='%s' \r\n& \r\nsubnetId='$subnetId'.\n" "$subId"
+export MSYS_NO_PATHCONV=1
+azure network interface new --name test --resourcegroupname "$groupName" --location "$location" --subnetid "$subnetId"
+export MSYS_NO_PATHCONV=
+
+nicId="/subscriptions/$subId/resourceGroups/$groupName/providers/Microsoft.Network/networkInterfaces/test"
+
+vhdUri="https://$storageAccountName.blob.core.windows.net/$storageAccountName/$storageAccountName.vhd"
+
+vmStr="{\"Name\":\"test\",\"HardwareProfile\":{\"VmSize\":\"Standard_A1\"},\"NetworkProfile\":{\"NetworkInterfaces\":[{\"Id\":\"$nicId\"}]},\"OSProfile\":{\"ComputerName\":\"test\",\"AdminPassword\":\"BaR@1234\",\"AdminUsername\":\"Foo12\"},\"StorageProfile\":{\"ImageReference\":{\"Offer\":\"WindowsServer\",\"Publisher\":\"MicrosoftWindowsServer\",\"Sku\":\"2008-R2-SP1\",\"Version\":\"latest\"},\"OSDisk\":{\"Caching\":\"ReadWrite\",\"CreateOption\":\"FromImage\",\"Name\":\"osDisk\",\"Vhd\":{\"Uri\":\"$vhdUri\"}}}}"
+
+printf "\n5. Create virtual machine with\r\nnicId='%s'\r\nvhdUri='%s'\r\nvmStr='%s'\n" "$nicId" "$vhdUri" "$vmStr"
+
+azure vm new --resourcegroupname "$groupName" --location "$location" --vmprofile "$vmStr"
+
+printf "\n6. Removing resource group: %s.\n" "$groupName"
+azure group remove -n "$groupName" -f

examples/lib/loginService.sh

@@ -1,2 +1,2 @@
 #!/bin/bash
-azure account add --spn --appid "$spn" --secret "$secret" -t "$tenant" -s "$subscription"
+azure account add --spn --appid "$spn" --secret "$secret" -t "$tenant" -s "$spnSubscription"

examples/lib/loginUser.sh

@@ -1,2 +1,2 @@
 #!/bin/bash
-azure account add -u "$azureUser" -p "$password" -s "$subscription"
+azure account add -u "$azureUser" -p "$password" -s "$userSubscription"

examples/lib/testrunner.sh

@@ -1,17 +1,24 @@
 #!/bin/bash
-export BASEDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
-. $BASEDIR/helper.sh
-. $BASEDIR/setup.sh
+export TESTDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+. $TESTDIR/helper.sh
 export groupName=`randomName testrg`
 export location="westus"
+export MSYS_NO_PATHCONV=1
 
-login
+echo "Logging in as user"
+. $TESTDIR/loginUser.sh
 
-for d in $( ls $BASEDIR/.. --ignore=lib ); do
-    for f in $( ls $BASEDIR/../$d/*.sh ); do
+for d in $( ls $TESTDIR/.. --ignore=lib ); do
+    for f in $( ls $TESTDIR/../$d/*.sh ); do
         echo "running: $f"
+        BASEDIR=$(cd "$(dirname "$f")" && pwd)
         . $f
-        cleanup
+        set +e
+        printf "\nCleanup: removing resource group: %s\n" $groupName
+        azure group remove --name "$groupName" --force
+        set -e
         echo "success: $f"
     done
-done
+done
+
+export MSYS_NO_PATHCONV=

examples/resource-management/01-ResourceGroups.ps1

@@ -1,25 +1,25 @@
 Param(
-  [string]$resourceGroupName,
-  [string]$resourceGroupLocation
+  [string]$groupName,
+  [string]$location
 )
 
 Write-Host "=== Managing Resource Groups in Azure ==="
 
 Write-Host "1. Create a new resource group"
-New-AzureRmResourceGroup -Name $resourceGroupName -Location $resourceGroupLocation
+New-AzureRmResourceGroup -Name $groupName -Location $location
 
 Write-Host "2. Update group tags"
-Set-AzureRmResourceGroup -Name $resourceGroupName -Tags @{Name = "testtag"; Value = "testval"} 
+Set-AzureRmResourceGroup -Name $groupName -Tags @{Name = "testtag"; Value = "testval"} 
 
 Write-Host "3. Get information about resource group"
-$resourceGroup = Get-AzureRmResourceGroup -Name $resourceGroupName
+$resourceGroup = Get-AzureRmResourceGroup -Name $groupName
 Write-Host $resourceGroup
 
 Write-Host "4. List all resource groups in the subscription"
 Get-AzureRmResourceGroup
 
 Write-Host "5. Remove resource group"
-Remove-AzureRmResourceGroup -Name $resourceGroupName -Force
+Remove-AzureRmResourceGroup -Name $groupName -Force
 
 Write-Host "6. Validations"
-Assert-AreEqual $resourceGroup.ResourceGroupName $resourceGroupName
+Assert-AreEqual $resourceGroup.ResourceGroupName $groupName

examples/resource-management/02-Resource.ps1

@@ -0,0 +1,41 @@
+Param(
+  [string]$groupName,
+  [string]$location,
+  [string]$rName,
+)
+
+Write-Host "=== Managing Resources in Azure ==="
+
+Write-Host "1. Creating a new resource group"
+New-AzureRmResourceGroup -Name $groupName -Location $location
+$destinationGroupName = $groupName + "Destination"
+
+Write-Host "2. Registering Resource Provider Namespace."
+$providerNamespace="Providers.Test"
+Register-AzureRmResourceProvider -ProviderNamespace $providerNamespace -Force
+
+Write-Host "3. Creating a new Resource"
+$resourceType = $providerNamespace + "/statefulResources"
+$apiversion="2014-04-01"
+New-AzureRmResource -Name $rName -Location $location -Tags @{Name = "testtag"; Value = "testval"} -ResourceGroupName $groupName -ResourceType $resourceType -PropertyObject @{"administratorLogin" = "adminuser"; "administratorLoginPassword" = "P@ssword1"} -ApiVersion $apiversion -Force
+
+Write-Host "4. Get information about Resource"
+$resourceInfo = Get-AzureRmResource -ResourceGroupName $groupName
+Write-Host "Validating Resource name"
+Assert-AreEqual $rName $resourceInfo.Name
+
+Write-Host "5. Find Resource with name"
+$foundResource = Find-AzureRmResource -ResourceType $resourceType -ResourceNameContains $rName
+Write-Host "Validating Resource name"
+Assert-AreEqual $rName $foundResource.Name
+
+Write-Host "6. Update Resource" 
+Set-AzureRmResource -ResourceGroupName $groupName -ResourceName $rName -ResourceType $resourceType -Tags @{Name = "testtagUpdated"; Value = "testvalueUpdated"} -Force
+
+Write-Host "7. Move Resource to resource group"
+New-AzureRmResourceGroup -Name $destinationGroupName -Location $location
+Move-AzureRmResource -DestinationResourceGroupName  $destinationGroupName -ResourceId $resourceInfo.ResourceId  -Force
+
+Write-Host "8. Removing resource"
+$foundResource = Find-AzureRmResource -ResourceType $resourceType -ResourceNameContains $rName
+Remove-AzureRmResource -ResourceId $foundResource.ResourceId -Force

examples/resource-management/02-Resource.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+set -e
+printf "\n=== Managing Resources in Azure ===\n"
+
+printf "\n1. Creating a new resource group: %s and location: %s.\n" "$groupName" "$location"
+azure group create --name "$groupName" --location "$location"
+destinationGroupName=$groupName"Destination"
+
+printf "\n2. Registering Resource Provider Namespace.\n"
+providerNamespace="Providers.Test"
+azure resource provider register --ProviderNamespace $providerNamespace --Force
+
+printf "\n3. Creating a new Resource: %s.\n" "$resourceName"
+resourceType="$providerNamespace/statefulResources"
+tags='[{"Name": "testtag", "Value": "testvalue"}]'
+properties='{"administratorLogin": "adminuser", "administratorLoginPassword": "P@ssword1"}'
+apiversion="2014-04-01"
+azure resource create --Name $resourceName --Location $location --Tags "$tags" --ResourceGroupName $groupName --ResourceType $resourceType --PropertyObject "$properties" --ApiVersion $apiversion --Force
+
+printf "\n4. Get information about Resource : %s.\n" "$resourceName"
+resourceInfo=$(azure resource get -n $resourceName)
+printf "\nValidating Resource name is: %s\n" "$resourceName"
+[ $(echo $resourceInfo | jq '.Name' --raw-output) == "$resourceName" ]
+
+printf "\n5. Find Resource with name '%s' and type '%s'.\n" "$resourceName" "$resourceType"
+foundResource=$(azure resource find -n "$resourceName" -t $resourceType)
+printf "\nValidating Resource name is: %s.\n" "$resourceName"
+[ $(echo $foundResource | jq '.Name' --raw-output) == "$resourceName" ]
+
+printf "\n6. Update Resource.\n" 
+tagsUpdate='[{"Name": "testtagUpdated", "Value": "testvalueUpdated"}]'
+azure resource set --ResourceGroupName $groupName --ResourceName $resourceName --ResourceType $resourceType --Tags "$tagsUpdate" -f
+
+printf "\n7. Move Resource to resource group: %s.\n" "$destinationGroupName"
+azure group create --name "$destinationGroupName" --location "$location"
+resourceId=$(echo $resourceInfo | jq '.Id')
+arrayId="[$resourceId]"
+azure resource move -g "$destinationGroupName" --ResourceId "$arrayId" -f
+
+printf "\n8. Removing resource: %s.\n" "$resourceName"
+foundResource=$(azure resource find -n "$resourceName" -t $resourceType)
+resourceId=$(echo $foundResource | jq '.Id' --raw-output)
+echo $resourceId
+export MSYS_NO_PATHCONV=1
+azure resource remove --Id "$resourceId" -f
+export MSYS_NO_PATHCONV=

examples/resource-management/03-Deployments.ps1

@@ -0,0 +1,24 @@
+Param(
+  [string]$groupName,
+  [string]$location
+)
+
+Write-Host "=== Provisioning Deployments in Azure ==="
+
+Write-Host "1. Create a new resource group"
+New-AzureRmResourceGroup -Name $groupName -Location $location
+
+Write-Host "2. Test template"
+$siteName = Get-ResourceName
+$list = Test-AzureResourceGroupTemplate -ResourceGroupName $groupName -TemplateFile sampleTemplate.json -siteName $siteName -hostingPlanName $siteName -siteLocation $location -sku Free -workerSize 0
+Write-Host $list
+# Assert
+Assert-AreEqual 0 @($list).Count
+
+Write-Host "3. Provisioning Deployment"
+$deployment = New-AzureRmResourceGroupDeployment -Name $siteName -ResourceGroupName $groupName -TemplateFile sampleTemplate.json -TemplateParameterFile sampleTemplateParams.json
+Write-Host $deployment
+# Assert
+Assert-AreEqual Succeeded $deployment.ProvisioningState
+
+