Clu

clu-getstart.md

@@ -46,15 +46,17 @@ CLUPackages require some additional files to direct generation of indexing, and
 ``` 
 
 ### Package Creation and Testing
-   2 options
-   * Run `<repo-root>\tools\CLU\SetupEnv.bat` which build and generate all cmdlet packages and deploy to under `<repo root>\drop` folder. When you have a clean environment, you should always do this first.
-   * Run `<repo-root>\tools\CLU\BuildCmdlet` <name like: Microsoft.Azure.Commands.Profile>", this will build and refresh an individual cmdlet package.
+   Two options
+   1. Run `<repo-root>\tools\CLU\BuildAndInstallClu.bat` which build and generate all cmdlet packages and deploy to under `<repo root>\drop\clurun` folder, with 3 flavors `win7-x64`, `osx.10.10-x64` and `ubuntu.14.04-x64`. When you have a clean environment or just pull from upstream, you should clean temporary bits such as `git clean -xdf`, and run this command.
+   2. Run `<repo-root>\tools\CLU\BuildCmdlet <package name like Microsoft.Azure.Commands.Profile>` <name like: Microsoft.Azure.Commands.Profile>", this will build and refresh an individual cmdlet package.
 
-Once you are done with #1, in the same command window, you can type "azure help" to explore and run cmdlets. 
+After #1 above is finished, you can run `drop\clurun\<platform>\azure.bat help` to explore.
 
-To debug, set environment variable of `DebugCLU` to "1"(#1 should set it up already). When you run any command, you will see a prompt telling you to attach debugger. 
+To debug, set environment variable of `DebugCLU` to "1". Then on running any command, you will be prompted to attach a debugger.
 
-To test on osx/linux boxes, do #1, open `<repo-root>\drop\clurun`, you should see subfolders for "osx" and "ubuntu", copy the folder to your target machine, and run the "azure.sh" inside. Make sure set execution permission using `chmod +x azure.sh clurun`
+There is also `<repo-root>\tools\CLU\SetupEnv.bat` which is a windows batch wrapping around the `BuildAndInstallClu.bat`, plus set the `DebugCLU` for you, and add the `drop\clurun\win7-x64\azure.bat` to the PATH environment variable.
+
+To test on osx/linux boxes, do #1, open `<repo-root>\drop\clurun`, copy the flavor folder to your target machine, and run the "azure.sh" inside. Make sure set execution permission using `chmod +x azure.sh clurun`
 
 (All of those are subject to change, contact yugangw or adxsdkdev for any questions)
 
@@ -88,9 +90,53 @@ To test on osx/linux boxes, do #1, open `<repo-root>\drop\clurun`, you should se
 Testing will consist of scenario tests and unit tests. Scenario tests should be written in a form of an example and be available in `.ps1` and `.sh` formats.
 
 #### Scenario Tests
-- Scenario tests should be saved under `./examples` directory and grouped by the package or service area. Each scenario tests should consist of both `.ps1` and `.sh` files and should cover "P0" scenarios.
-
-##### Bash Tests
+- Scenario tests should be saved under `./examples` directory with one directory per package. Each scenario tests should (eventually) consist of both `.ps1` and `.sh` files and should cover "P0" scenarios.
+
+##### XUnit Automation For Bash Scenario Tests
+- The ```Commands.Common.ScenarioTest``` project contains classes that enable executing bash scenario tests in Visual Studio, or cross-platform using dnx.
+
+- To implement an xunit bash scenario test you must
+  - Add a ```[Collection("SampleCollection")]``` attribute to your test class
+  - Add a field to your class of type ```ScenarioTestFixture``` and add a constructor that initializes it
+    ```C#
+    [Collection("SampleCollection")]
+    public class SampleTestClass
+    {
+        ScenarioTestFixture _fixture;
+        public SampleTestClass(ScenarioTestFixture fixture)
+        {
+            _fixture = fixture;
+        }
+    ```
+    - Use the fixture in your test method to create a script runner for your directory and to run your test script:
+    ```C#
+    [Fact]
+    public void RunSampleTest()
+    {
+        _fixture.GetRunner("resource-management").RunScript("01-ResourceGroups");
+    }
+    ```
+    - Set the environment variable 'TestCredentials' to a connection string providing the credentials to use during test execution. Possible fields include:
+
+      |  Field        |  Description  |
+      | ------------- |:-------------|
+      |  Username     | an OrgId user name |
+      | ServicePrincipal | a service principal name |
+      | Password      | the password or application secret to sue for authentication |
+      | TenantId      | (required for Service authentication) The tenant guid to authenticate against |
+      | SubscriptionID | (optional) Selects a particular subscription by id.  If not provided, the first listed subscription will be selected |
+    - The infrastructure automatically generates a resource group name and assigns the value to the bash variable ```"$resourceGroupName"```.  If your scripts require additional variables, you can add these to your environment before running tests, or you can generate values using the ScriptRunner (for the tests using that runner).
+    ```C#
+        runner.EnvironmentVariables.Add("myVariableName", runner.GenerateName("myres"));
+    ```
+    - Tests can be executed in vs, or by runnign ```dnx test project.json```.  If you execute dnx test from the project directory, it will work without modification and a log file for each script will be written to the test results directory ```..\TestResults```.  If you execute dnx test from a different directory, you must set the following environment variables to provide the path to the examples directory and where to write log files:
+
+      |  Environment Variable  |  Description  |
+      | ------------- |:-------------|
+      |  ExamplesDirectory     | The path to the 'examples' directory ($pshome/examples) |
+      | TestDirectory | The path to the directory where logs will be written |
+
+##### Running Bash Tests using Bash shell
 - Bash tests should be runnable from bash shell in windows/linux/mac environments.
 - To manually run the tests; please set the following envt. variables for authentication and run `./examples/lib/testrunner.sh`
    ```bash

examples/lib/loginService.sh

@@ -0,0 +1,2 @@
+#!/bin/bash
+azure account add --spn --appid "$spn" --secret "$secret" -t "$tenant" -s "$subscription"

examples/lib/loginUser.sh

@@ -0,0 +1,2 @@
+#!/bin/bash
+azure account add -u "$azureUser" -p "$password" -s "$subscription"

examples/resource-management/01-ResourceGroups.sh

@@ -3,13 +3,13 @@ set -e
 printf "\n=== Managing Resource Groups in Azure ===\n"
 
 printf "\n1. Creating a new resource group: %s and location: %s.\n" "$groupName" "$location"
-azure group create --name "$groupName" --location "$location"
+azure group create -n "$groupName" --location "$location"
 
 printf "\n2. Updating the group %s with tags.\n" "$groupName"
-azure group set --name "$groupName" --tags "[{\"Value\":\"testval\",\"Name\":\"testtag\"}]"
+azure group set -n "$groupName" --tags "[{\"Value\":\"testval\",\"Name\":\"testtag\"}]"
 
 printf "\n3. Get information about resource group : %s.\n" "$groupName"
-resourceGroupInfo=`azure group get --name $groupName`
+resourceGroupInfo=`azure group get -n $groupName`
 
 printf "\nValidating resource group name is: %s\n" "$groupName"
 [ $(echo $resourceGroupInfo | jq '.ResourceGroupName' --raw-output) == "$groupName" ]
@@ -18,4 +18,4 @@ printf "\n4. Listing all resource groups in the subscription.\n"
 azure group get
 
 printf "\n5. Removing resource group: %s.\n" "$groupName"
-azure group remove --name "$groupName" --force
+azure group remove -n "$groupName" -f

src/CLU/CLUCoreCLR.sln

@@ -1,7 +1,7 @@
 
 Microsoft Visual Studio Solution File, Format Version 12.00
 # Visual Studio 14
-VisualStudioVersion = 14.0.23107.0
+VisualStudioVersion = 14.0.24720.0
 MinimumVisualStudioVersion = 10.0.40219.1
 Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Commands.Common", "Commands.Common\Commands.Common.xproj", "{5F567ACA-595E-436D-83DB-A21E08F82DF6}"
 EndProject
@@ -37,8 +37,6 @@ Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.CLU.Run", "Micros
 EndProject
 Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.CLU", "Microsoft.CLU\Microsoft.CLU.xproj", "{D0A59671-088D-463B-B060-2ADAFFB9C3F6}"
 EndProject
-Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.CLU.Test", "Microsoft.CLU.Test\Microsoft.CLU.Test.xproj", "{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}"
-EndProject
 Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.Azure.Commands.Compute", "Microsoft.Azure.Commands.Compute\Microsoft.Azure.Commands.Compute.xproj", "{04F9968A-5662-4508-BEE2-31F56848FCBA}"
 	ProjectSection(ProjectDependencies) = postProject
 		{99B1290D-A073-4907-8018-51C714431778} = {99B1290D-A073-4907-8018-51C714431778}
@@ -47,6 +45,8 @@ Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.Azure.Commands.Co
 		{45B05B68-516F-4D74-897F-56D12894946C} = {45B05B68-516F-4D74-897F-56D12894946C}
 	EndProjectSection
 EndProject
+Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Commands.Common.ScenarioTest", "Commands.Common.ScenarioTest\Commands.Common.ScenarioTest.xproj", "{B1D3CB1F-C0CA-401F-8146-B2E9C1EF460F}"
+EndProject
 Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.Azure.Commands.Compute.Test", "Microsoft.Azure.Commands.Compute.Test\Microsoft.Azure.Commands.Compute.Test.xproj", "{13C34370-51A4-4726-81B8-BE0996FC9CFF}"
 	ProjectSection(ProjectDependencies) = postProject
 		{3910613E-4ED2-49E2-8CCF-966D586665AC} = {3910613E-4ED2-49E2-8CCF-966D586665AC}
@@ -55,6 +55,15 @@ Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.Azure.Commands.Co
 		{04F9968A-5662-4508-BEE2-31F56848FCBA} = {04F9968A-5662-4508-BEE2-31F56848FCBA}
 	EndProjectSection
 EndProject
+Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Microsoft.ScenarioTest.CLU", "Microsoft.ScenarioTests.CLU\Microsoft.ScenarioTest.CLU.xproj", "{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}"
+EndProject
+Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "VhdManagement", "VhdManagement\VhdManagement.xproj", "{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}"
+EndProject
+Project("{8BB2217D-0F2D-49D1-97BC-3654ED321F3B}") = "Sync", "Sync\Sync.xproj", "{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}"
+	ProjectSection(ProjectDependencies) = postProject
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9} = {094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}
+	EndProjectSection
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Any CPU = Debug|Any CPU
@@ -129,18 +138,30 @@ Global
 		{D0A59671-088D-463B-B060-2ADAFFB9C3F6}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{D0A59671-088D-463B-B060-2ADAFFB9C3F6}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{D0A59671-088D-463B-B060-2ADAFFB9C3F6}.Release|Any CPU.Build.0 = Release|Any CPU
-		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
-		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Debug|Any CPU.Build.0 = Debug|Any CPU
-		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Release|Any CPU.ActiveCfg = Release|Any CPU
-		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Release|Any CPU.Build.0 = Release|Any CPU
 		{04F9968A-5662-4508-BEE2-31F56848FCBA}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
 		{04F9968A-5662-4508-BEE2-31F56848FCBA}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{04F9968A-5662-4508-BEE2-31F56848FCBA}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{04F9968A-5662-4508-BEE2-31F56848FCBA}.Release|Any CPU.Build.0 = Release|Any CPU
+		{B1D3CB1F-C0CA-401F-8146-B2E9C1EF460F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{B1D3CB1F-C0CA-401F-8146-B2E9C1EF460F}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{B1D3CB1F-C0CA-401F-8146-B2E9C1EF460F}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{B1D3CB1F-C0CA-401F-8146-B2E9C1EF460F}.Release|Any CPU.Build.0 = Release|Any CPU
 		{13C34370-51A4-4726-81B8-BE0996FC9CFF}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
 		{13C34370-51A4-4726-81B8-BE0996FC9CFF}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{13C34370-51A4-4726-81B8-BE0996FC9CFF}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{13C34370-51A4-4726-81B8-BE0996FC9CFF}.Release|Any CPU.Build.0 = Release|Any CPU
+		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{91422B55-28A5-48DE-BCA0-30C3E30FFB1C}.Release|Any CPU.Build.0 = Release|Any CPU
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{094A32EA-BABC-4A0C-9B6C-3CF7F6EABEC9}.Release|Any CPU.Build.0 = Release|Any CPU
+		{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{6EDCB32A-8420-48FC-99CE-94BEA12D2FD2}.Release|Any CPU.Build.0 = Release|Any CPU
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE

src/CLU/Commands.Common.Authentication/Authentication/AdalTokenProvider.cs

@@ -12,9 +12,9 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Microsoft.Azure.Commands.Common.Authentication.Models;
 using System;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication
 {

src/CLU/Commands.Common.Authentication/Authentication/ServicePrincipalTokenProvider.cs

@@ -12,11 +12,11 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Microsoft.Azure.Commands.Common.Authentication.Models;
 using Microsoft.IdentityModel.Clients.ActiveDirectory;
 using Microsoft.Rest;
 using System;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication
 {

src/CLU/Commands.Common.Authentication/ClientFactory.cs

@@ -12,7 +12,6 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Microsoft.Azure.Commands.Common.Authentication.Models;
 using Microsoft.Rest;
 using System;
@@ -23,6 +22,7 @@
 using System.Net.Http;
 using System.Net.Http.Headers;
 using System.Reflection;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication.Factories
 {

src/CLU/Commands.Common.Authentication/Commands.Common.Authentication.xproj

@@ -7,7 +7,7 @@
   <Import Project="$(VSToolsPath)\DNX\Microsoft.DNX.Props" Condition="'$(VSToolsPath)' != ''" />
   <PropertyGroup Label="Globals">
     <ProjectGuid>4ce82310-d016-497d-93a0-0323a3e62064</ProjectGuid>
-    <RootNamespace>Microsoft.Azure.Commands.Common.Authentication</RootNamespace>
+    <RootNamespace>Commands.Common.Authentication</RootNamespace>
     <BaseIntermediateOutputPath Condition="'$(BaseIntermediateOutputPath)'=='' ">..\artifacts\obj\$(MSBuildProjectName)</BaseIntermediateOutputPath>
     <OutputPath Condition="'$(OutputPath)'=='' ">..\artifacts\bin\$(MSBuildProjectName)\</OutputPath>
   </PropertyGroup>

src/CLU/Commands.Common.Authentication/JsonUtilities.cs

@@ -12,12 +12,12 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Newtonsoft.Json;
 using Newtonsoft.Json.Linq;
 using System;
 using System.Collections.Generic;
 using System.Diagnostics.CodeAnalysis;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication
 {

src/CLU/Commands.Common.Authentication/Models/AzureEnvironment.Methods.cs

@@ -12,9 +12,9 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using System;
 using System.Collections.Generic;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication.Models
 {

src/CLU/Commands.Common.Authentication/Models/AzureSMProfile.cs

@@ -12,12 +12,12 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using Microsoft.Rest;
 using Newtonsoft.Json;
 using System;
 using System.Collections.Generic;
 using System.Linq;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication.Models
 {

src/CLU/Commands.Common.Authentication/Models/DiskDataStore.cs

@@ -12,11 +12,11 @@
 // limitations under the License.
 // ----------------------------------------------------------------------------------
 
-using Commands.Common.Authentication.Properties;
 using System;
 using System.IO;
 using System.Security.Cryptography.X509Certificates;
 using System.Text;
+using Commands.Common.Authentication.Properties;
 
 namespace Microsoft.Azure.Commands.Common.Authentication.Models
 {