Merge remote-tracking branch 'origin/clu' into DynamicParameters

Author: johanste

clu-getstart.md

@@ -46,15 +46,17 @@ CLUPackages require some additional files to direct generation of indexing, and
 ``` 
 
 ### Package Creation and Testing
-   2 options
-   * Run `<repo-root>\tools\CLU\SetupEnv.bat` which build and generate all cmdlet packages and deploy to under `<repo root>\drop` folder. When you have a clean environment, you should always do this first.
-   * Run `<repo-root>\tools\CLU\BuildCmdlet` <name like: Microsoft.Azure.Commands.Profile>", this will build and refresh an individual cmdlet package.
+   Two options
+   1. Run `<repo-root>\tools\CLU\BuildAndInstallClu.bat` which build and generate all cmdlet packages and deploy to under `<repo root>\drop\clurun` folder, with 3 flavors `win7-x64`, `osx.10.10-x64` and `ubuntu.14.04-x64`. When you have a clean environment or just pull from upstream, you should clean temporary bits such as `git clean -xdf`, and run this command.
+   2. Run `<repo-root>\tools\CLU\BuildCmdlet <package name like Microsoft.Azure.Commands.Profile>` <name like: Microsoft.Azure.Commands.Profile>", this will build and refresh an individual cmdlet package.
 
-Once you are done with #1, in the same command window, you can type "azure help" to explore and run cmdlets. 
+After #1 above is finished, you can run `drop\clurun\<platform>\azure.bat help` to explore.
 
-To debug, set environment variable of `DebugCLU` to "1"(#1 should set it up already). When you run any command, you will see a prompt telling you to attach debugger. 
+To debug, set environment variable of `DebugCLU` to "1". Then on running any command, you will be prompted to attach a debugger.
 
-To test on osx/linux boxes, do #1, open `<repo-root>\drop\clurun`, you should see subfolders for "osx" and "ubuntu", copy the folder to your target machine, and run the "azure.sh" inside. Make sure set execution permission using `chmod +x azure.sh clurun`
+There is also `<repo-root>\tools\CLU\SetupEnv.bat` which is a windows batch wrapping around the `BuildAndInstallClu.bat`, plus set the `DebugCLU` for you, and add the `drop\clurun\win7-x64\azure.bat` to the PATH environment variable.
+
+To test on osx/linux boxes, do #1, open `<repo-root>\drop\clurun`, copy the flavor folder to your target machine, and run the "azure.sh" inside. Make sure set execution permission using `chmod +x azure.sh clurun`
 
 (All of those are subject to change, contact yugangw or adxsdkdev for any questions)
 
@@ -88,9 +90,53 @@ To test on osx/linux boxes, do #1, open `<repo-root>\drop\clurun`, you should se
 Testing will consist of scenario tests and unit tests. Scenario tests should be written in a form of an example and be available in `.ps1` and `.sh` formats.
 
 #### Scenario Tests
-- Scenario tests should be saved under `./examples` directory and grouped by the package or service area. Each scenario tests should consist of both `.ps1` and `.sh` files and should cover "P0" scenarios.
-
-##### Bash Tests
+- Scenario tests should be saved under `./examples` directory with one directory per package. Each scenario tests should (eventually) consist of both `.ps1` and `.sh` files and should cover "P0" scenarios.
+
+##### XUnit Automation For Bash Scenario Tests
+- The ```Commands.Common.ScenarioTest``` project contains classes that enable executing bash scenario tests in Visual Studio, or cross-platform using dnx.
+
+- To implement an xunit bash scenario test you must
+  - Add a ```[Collection("SampleCollection")]``` attribute to your test class
+  - Add a field to your class of type ```ScenarioTestFixture``` and add a constructor that initializes it
+    ```C#
+    [Collection("SampleCollection")]
+    public class SampleTestClass
+    {
+        ScenarioTestFixture _fixture;
+        public SampleTestClass(ScenarioTestFixture fixture)
+        {
+            _fixture = fixture;
+        }
+    ```
+    - Use the fixture in your test method to create a script runner for your directory and to run your test script:
+    ```C#
+    [Fact]
+    public void RunSampleTest()
+    {
+        _fixture.GetRunner("resource-management").RunScript("01-ResourceGroups");
+    }
+    ```
+    - Set the environment variable 'TestCredentials' to a connection string providing the credentials to use during test execution. Possible fields include:
+    
+      |  Field (case sensitive) |  Description  |
+      | ------------- |:-------------|
+      |  Username     | an OrgId user name |
+      | ServicePrincipal | a service principal name |
+      | Password      | the password or application secret to sue for authentication |
+      | TenantId      | (required for Service authentication) The tenant guid to authenticate against |
+      | SubscriptionId | (optional) Selects a particular subscription by id.  If not provided, the first listed subscription will be selected |
+    - The infrastructure automatically generates a resource group name and assigns the value to the bash variable ```"$resourceGroupName"```.  If your scripts require additional variables, you can add these to your environment before running tests, or you can generate values using the ScriptRunner (for the tests using that runner).
+    ```C#
+        runner.EnvironmentVariables.Add("myVariableName", runner.GenerateName("myres"));
+    ```
+    - Tests can be executed in vs, or by runnign ```dnx test project.json```.  If you execute dnx test from the project directory, it will work without modification and a log file for each script will be written to the test results directory ```..\TestResults```.  If you execute dnx test from a different directory, you must set the following environment variables to provide the path to the examples directory and where to write log files:
+    
+      |  Environment Variable  |  Description  |
+      | ------------- |:-------------|
+      |  ExamplesDirectory     | The path to the 'examples' directory ($pshome/examples) |
+      | TestDirectory | The path to the directory where logs will be written |
+      
+##### Running Bash Tests using Bash shell
 - Bash tests should be runnable from bash shell in windows/linux/mac environments.
 - To manually run the tests; please set the following envt. variables for authentication and run `./examples/lib/testrunner.sh`
    ```bash

examples/lib/loginUser.sh

@@ -1,2 +1,3 @@
 #!/bin/bash
+echo "Logging in as user"
 azure account add -u "$azureUser" -p "$password" -s "$subscription"

examples/lib/setup.sh

@@ -1,18 +1,23 @@
 #!/bin/bash
 export BASEDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
-. $BASEDIR/assert.sh
 . $BASEDIR/helper.sh
-. $BASEDIR/setup.sh
 export groupName=`randomName testrg`
 export location="westus"
+export CmdletSessionID=1010
+export MSYS_NO_PATHCONV=1
 
-login
+. $BASEDIR/loginUser.sh
 
 for d in $( ls $BASEDIR/.. --ignore=lib ); do
     for f in $( ls $BASEDIR/../$d/*.sh ); do
         echo "running: $f"
         . $f
-        cleanup
+        set +e
+        printf "\nCleanup: removing resource group: %s\n" $groupName
+        azure group remove --name "$groupName" --force
+        set -e
         echo "success: $f"
     done
-done
+done
+
+export MSYS_NO_PATHCONV=

examples/lib/testrunner.sh

@@ -0,0 +1,6 @@
+Param(
+  [string]$resourceGroupName,
+  [string]$resourceGroupLocation
+)
+
+Write-Host "Skip"

examples/virtual-hard-disk/01-VirtualHardDisks.ps1

@@ -0,0 +1,27 @@
+#!/bin/bash
+set -e
+printf "\n=== Managing Virtual Hard Disks in Azure Compute ===\n"
+
+printf "\n1. Creating a new resource group: %s and location: %s.\n" "$groupName" "$location"
+azure group create -n "$groupName" --location "$location"
+
+printf "\n2. Creating a new storage account '%s' in type '%s'.\n" "$storageAccountName" "$storageAccountType"
+azure storage account new --resourcegroupname "$groupName" --name "$storageAccountName" --location "$location" --type "$storageAccountType"
+
+printf "\n3. Uploading a virtual hard disk to: %s.\n" "$storageAccountName"
+azure vhd add -o --resourcegroupname "$groupName" --destination https://"$storageAccountName".blob.core.windows.net/test/test.vhd --localfilepath $BASEDIR/test.vhd
+
+printf "\n4. Downloading a virtual hard disk"
+azure vhd save -o --resourcegroupname "$groupName" --sourceuri https://"$storageAccountName".blob.core.windows.net/test/test.vhd --localfilepath ./test_downloaded_by_clu.vhd
+
+printf "\n5. Validating the downloaded file is the same.\n"
+diffResult=`diff ./test_downloaded_by_clu.vhd $BASEDIR/test_uploaded_byps.vhd`
+if [ "$diffResult" = "" ]; then
+    echo "Checked"
+else
+    echo "Different!" 1>&2
+    exit 1
+fi
+
+printf "\n6. Removing resource group: %s.\n" "$groupName"
+azure group remove -n "$groupName" -f

examples/virtual-hard-disk/01-VirtualHardDisks.sh

@@ -57,6 +57,13 @@ Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.Azure.Commands.Co
 EndProject
 Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.ScenarioTest.CLU", "Microsoft.ScenarioTests.CLU\Microsoft.ScenarioTest.CLU.xproj", "{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}"
 EndProject
+Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "VhdManagement", "VhdManagement\VhdManagement.xproj", "{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}"
+EndProject
+Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Sync", "Sync\Sync.xproj", "{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}"
+	ProjectSection(ProjectDependencies) = postProject
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9} = {094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}
+	EndProjectSection
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Any CPU = Debug|Any CPU
@@ -147,6 +154,14 @@ Global
 		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Release|Any CPU.Build.0 = Release|Any CPU
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}.Release|Any CPU.Build.0 = Release|Any CPU
+		{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}.Release|Any CPU.Build.0 = Release|Any CPU
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE

examples/virtual-hard-disk/test.vhd

@@ -12,9 +12,9 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Microsoft.Azure.Commands.Common.Authentication.Models;
 using System;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication
 {

examples/virtual-hard-disk/test_uploaded_byps.vhd

@@ -12,11 +12,11 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Microsoft.Azure.Commands.Common.Authentication.Models;
 using Microsoft.IdentityModel.Clients.ActiveDirectory;
 using Microsoft.Rest;
 using System;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication
 {

src/CLU/CLUCoreCLR.sln

@@ -12,7 +12,6 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Microsoft.Azure.Commands.Common.Authentication.Models;
 using Microsoft.Rest;
 using System;
@@ -23,6 +22,7 @@
 using System.Net.Http;
 using System.Net.Http.Headers;
 using System.Reflection;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication.Factories
 {

src/CLU/Commands.Common.Authentication/Authentication/AdalTokenProvider.cs

@@ -7,7 +7,7 @@
   <Import Project="$(VSToolsPath)\DNX\Microsoft.DNX.Props" Condition="'$(VSToolsPath)' != ''" />
   <PropertyGroup Label="Globals">
     <ProjectGuid>4ce82310-d016-497d-93a0-0323a3e62064</ProjectGuid>
-    <RootNamespace>Microsoft.Azure.Commands.Common.Authentication</RootNamespace>
+    <RootNamespace>Commands.Common.Authentication</RootNamespace>
     <BaseIntermediateOutputPath Condition="'$(BaseIntermediateOutputPath)'=='' ">..\artifacts\obj\$(MSBuildProjectName)</BaseIntermediateOutputPath>
     <OutputPath Condition="'$(OutputPath)'=='' ">..\artifacts\bin\$(MSBuildProjectName)\</OutputPath>
   </PropertyGroup>

src/CLU/Commands.Common.Authentication/Authentication/ServicePrincipalTokenProvider.cs

@@ -12,12 +12,12 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Newtonsoft.Json;
 using Newtonsoft.Json.Linq;
 using System;
 using System.Collections.Generic;
 using System.Diagnostics.CodeAnalysis;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication
 {

src/CLU/Commands.Common.Authentication/ClientFactory.cs

@@ -12,9 +12,9 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using System;
 using System.Collections.Generic;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication.Models
 {

src/CLU/Commands.Common.Authentication/Commands.Common.Authentication.xproj

@@ -12,12 +12,12 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Microsoft.Rest;
 using Newtonsoft.Json;
 using System;
 using System.Collections.Generic;
 using System.Linq;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication.Models
 {

src/CLU/Commands.Common.Authentication/JsonUtilities.cs

@@ -12,11 +12,11 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using System;
 using System.IO;
 using System.Security.Cryptography.X509Certificates;
 using System.Text;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication.Models
 {