qMerge branch 'preview' of github.com:Azure/azure-powershell into group_fix

Author: twitchax

.dockerignore

@@ -3,4 +3,5 @@
 
 # Include the build.
 !src/Package/Release/ResourceManager/**/*
-!src/Package/Debug/ResourceManager/**/*
+!src/Package/Debug/ResourceManager/**/*
+!tools/InstallationTests/NetcoreTests/**/*

.gitignore

@@ -211,4 +211,5 @@ FakesAssemblies/
 /tools/*.dll
 *.GhostDoc.xml
 pingme.txt
-groupMapping*.json
+groupMapping*.json
+.vscode/

.travis.yml

@@ -13,6 +13,11 @@ services:
 before_install:
  - sudo apt-get update
  - sudo apt-get install docker-ce
+ - curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
+ - curl https://packages.microsoft.com/config/ubuntu/14.04/prod.list | sudo tee /etc/apt/sources.list.d/microsoft.list
+ - sudo apt-get update
+ - sudo apt-get install -y powershell
+ - sudo pwsh -NonInteractive -NoLogo -NoProfile -Command "Install-Module platyPS -Force -Confirm:\$false -Scope CurrentUser"
 
 script:
  - dotnet msbuild build.proj /t:BuildNetcore /p:Configuration=$CONFIG

ChangeLog.md

@@ -1,4 +1,96 @@
-## 2017.11.10 Version 5.0.1
+## 2017.12.8 Version 5.1.1
+* AnalysisServices
+    - Change validate set of location to dynamic lookup so that all clouds are supported.
+* Automation
+    - Update to Import-AzureRMAutomationRunbook
+        - Support is now being provided for Python2 runbooks
+* Batch
+    - Fixed a bug where account operations without a resource group failed to auto-detect the resource group
+* Compute
+    - Get-AzureRmComputeResourceSku shows zone information.
+    - Update Disable-AzureRmVmssDiskEncryption to fix issue https://github.com/Azure/azure-powershell/issues/5038
+    - Added -AsJob support for long-running Compute cmdlets. Allows selected cmdlets to run in the background and return a job to track and control progress.
+        - Affected cmdlets include: New-, Update-, Set-, Remove-, Start-, Restart-, Stop- cmdlets for Virtual Machines and Virtual Machine Scale Sets
+    - Added simplified parameter set to New-AzureRmVM, which creates a Virtual Machine and all required resources using smart defaults
+* ContainerInstance
+    - Apply Azure Container Instance SDK 2017-10-01
+        - Support container run-to-completion
+        - Support Azure File volume mount
+        - Support opening multiple ports for public IP
+* ContainerRegistry
+    - New cmdlets for geo-replication and webhooks
+        - Get/New/Remove-AzureRmContainerRegistryReplication
+        - Get/New/Remove/Test/Update-AzureRmContainerRegistryWebhook
+* DataFactories
+    - Credential encryption functionality now works with both "Remote Access" enabled (Over Network) and "Remote Access" disabled (Local Machine).
+* DataFactoryV2
+    - Added two new cmdlets: Update-AzureRmDataFactoryV2 and Stop-AzureRmDataFactoryV2PipelineRun
+* DataLakeAnalytics
+    - Added a parameter called ScriptParameter to Submit-AzureRmDataLakeAnalyticsJob
+        - Detailed information about ScriptParameter can be found using Get-Help on Submit-AzureRmDataLakeAnalyticsJob
+    - For New-AzureRmDataLakeAnalyticsAccount, changed the parameter MaxDegreeOfParallelism to MaxAnalyticsUnits
+        - Added an alias for the parameter MaxAnalyticsUnits: MaxDegreeOfParallelism
+    - For New-AzureRmDataLakeAnalyticsComputePolicy, changed the parameter MaxDegreeOfParallelismPerJob to MaxAnalyticsUnitsPerJob
+        - Added an alias for the parameter MaxAnalyticsUnitsPerJob: MaxDegreeOfParallelismPerJob
+    - For Set-AzureRmDataLakeAnalyticsAccount, changed the parameter MaxDegreeOfParallelism to MaxAnalyticsUnits
+        - Added an alias for the parameter MaxAnalyticsUnits: MaxDegreeOfParallelism
+    - For Submit-AzureRmDataLakeAnalyticsJob, changed the parameter DegreeOfParallelism to AnalyticsUnits
+        - Added an alias for the parameter AnalyticsUnits: DegreeOfParallelism
+    - For Update-AzureRmDataLakeAnalyticsComputePolicy, changed the parameter MaxDegreeOfParallelismPerJob to MaxAnalyticsUnitsPerJob
+        - Added an alias for the parameter MaxAnalyticsUnitsPerJob: MaxDegreeOfParallelismPerJob
+* MachineLearningCompute
+    - Add Set-AzureRmMlOpCluster
+        - Update a cluster's agent count or SSL configuration
+    - Orchestrator properties are optional
+        - The service will create a service principal if not provided, so the orchestrator
+        properties are now optional
+* PowerBIEmbedded
+    - Add support for Power BI Embedded Capacity cmdlets
+    - New Cmdlet Get-AzureRmPowerBIEmbeddedCapacity - Gets the details of a PowerBI Embedded Capacity.
+    - New Cmdlet New-AzureRmPowerBIEmbeddedCapacity - Creates a new PowerBI Embedded Capacity
+    - New Cmdlet Remove-AzureRmPowerBIEmbeddedCapacity - Deletes an instance of PowerBI Embedded Capacity
+    - New Cmdlet Resume-AzureRmPowerBIEmbeddedCapacity - Resumes an instance of PowerBI Embedded Capacity
+    - New Cmdlet Suspend-AzureRmPowerBIEmbeddedCapacity - Suspends an instance of PowerBI Embedded Capacity
+    - New Cmdlet Test-AzureRmPowerBIEmbeddedCapacity - Tests the existence of an instance of PowerBI Embedded Capacity
+    - New Cmdlet Update-AzureRmPowerBIEmbeddedCapacity - Modifies an instance of PowerBI Embedded Capacity
+* Profile
+    - Updated USGovernmentActiveDirectoryEndpoint to https://login.microsoftonline.us/
+        - For more information about the Azure Government endpoint mappings, please see the following: https://docs.microsoft.com/en-us/azure/azure-government/documentation-government-developer-guide#endpoint-mapping
+    - Added -AsJob support for cmdlets, enabling selected cmdlets to execute in the background and return a job to track and control progress
+    - Added -AsJob parameter to Get-AzureRmSubscription cmdlet
+* RecoveryServices.Backup
+    - Fixed bug - Get-AzureRmRecoveryServicesBackupItem should do case insensitive comparison for container name filter.
+    - Fixed bug - AzureVmItem now has a property that shows the last time a backup operation has happened - LastBackupTime.
+* Resources
+    - Fixed issue where Get-AzureRMRoleAssignment would result in a assignments without roledefiniton name for custom roles
+        - Users can now use Get-AzureRMRoleAssignment with assignments having roledefinition names irrespective of the type of role
+    - Fixed issue where Set-AzureRMRoleRoleDefinition used to throw RD not found error when there was a new scope in assignablescopes
+        - Users can now use Set-AzureRMRoleRoleDefinition with assignable scopes including new scopes irrespective of the position of the scope
+    - Allow scopes to end with "/"
+        - Users can now use RoleDefinition and RoleAssignment commandlets with scopes ending with "/" ,consistent with API and CLI
+    - Allow users to create RoleAssignment using delegation flag
+        - Users can now use New-AzureRMRoleAssignment with an option of adding the delegation flag
+    - Fix RoleAssignment get to respect the scope parameter
+    - Add an alias for ServicePrincipalName in the New-AzureRmRoleAssignment Commandlet
+        - Users can now use the ApplicationId instead of the ServicePrincipalName when using the New-AzureRmRoleAssignment commandlet
+* SiteRecovery
+    - Add deprecation warnings for all cmdlets in this module in preparation for the next breaking change release.
+        - Please see the upcoming breaking changes guide for more information on how to migrate your cmdlets from AzureRM.
+* Sql
+    - Added ability to rename database using Set-AzureRmSqlDatabase
+    - Fixed issue https://github.com/Azure/azure-powershell/issues/4974
+        - Providing invalid AUDIT_CHANGED_GROUP value for auditing cmdlets no longer throws an error and will be removed in an upcoming release.
+    - Fixed issue https://github.com/Azure/azure-powershell/issues/5046
+        - AuditAction parameter in auditing cmdlets is no longer being ignored
+    - Fixed an issue in Auditing cmdlets when 'Secondary' StorageKeyType is provided
+        - When setting blob auditing, the primary storage account key was used instead of the secondary key when providing 'Secondary' value for StorageKeyType parameter.
+    - Changing the wording for confirmation message from Set-AzureRmSqlServerTransparentDataEncryptionProtector
+* Azure (RDFE)
+    - Removed all RemoteApp Cmdles
+* Azure.Storage
+    - Upgrade to Azure Storage Client Library 8.6.0 and Azure Storage DataMovement Library 0.6.5
+
+## 2017.11.10 Version 5.0.1
 * Fixed assembly loading issue that caused some cmdlets to fail when executing in the following modules:
     - AzureRM.ApiManagement
     - AzureRM.Backup

Dockerfile

@@ -2,4 +2,5 @@ FROM microsoft/powershell
 
 ARG CONFIG=Release
 
-COPY src/Package/${CONFIG}/ResourceManager /usr/local/share/powershell/Modules
+COPY src/Package/${CONFIG}/ResourceManager/AzureResourceManager /usr/local/share/powershell/Modules
+COPY tools/InstallationTests/NetcoreTests /azpstests

TestMappings.json

@@ -44,6 +44,9 @@
   "src/ResourceManager/DataLakeStore/": [
     ".\\src\\ResourceManager\\DataLakeStore\\Commands.DataLakeStore.Test\\bin\\Debug\\Microsoft.Azure.Commands.DataLake.Test.dll"
   ],
+  "src/ResourceManager/DataMigration/": [
+    ".\\src\\ResourceManager\\DataMigration\\Commands.DataMigration.Test\\bin\\Debug\\Microsoft.Azure.Commands.DataMigration.Test.dll"
+  ],
   "src/ResourceManager/DevTestLabs/": [],
   "src/ResourceManager/Dns/": [
     ".\\src\\ResourceManager\\Dns\\Commands.Dns.Test\\bin\\Debug\\Microsoft.Azure.Commands.Dns.Test.dll"
@@ -101,6 +104,9 @@
   "src/ResourceManager/Resources/": [
     ".\\src\\ResourceManager\\Resources\\Commands.Resources.Test\\bin\\Debug\\Microsoft.Azure.Commands.Resources.Test.dll"
   ],
+  "src/ResourceManager/Reservations/": [
+    ".\\src\\ResourceManager\\Reservations\\Commands.Reservations.Test\\bin\\Debug\\Microsoft.Azure.Commands.Reservations.Test.dll"
+  ],
   "src/ResourceManager/Scheduler/": [
     ".\\src\\ResourceManager\\Scheduler\\Commands.Scheduler.Test\\bin\\Debug\\Microsoft.Azure.Commands.Scheduler.Test.dll"
   ],
@@ -123,6 +129,9 @@
   "src/ResourceManager/StreamAnalytics/": [
     ".\\src\\ResourceManager\\StreamAnalytics\\Commands.StreamAnalytics.Test\\bin\\Debug\\Microsoft.Azure.Commands.StreamAnalytics.Test.dll"
   ],
+  "src/ResourceManager/Subscription/": [
+    ".\\src\\ResourceManager\\Subscription\\Commands.Subscription.Test\\bin\\Debug\\Microsoft.Azure.Commands.Subscription.Test.dll"
+  ],
   "src/ResourceManager/Tags/": [],
   "src/ResourceManager/TrafficManager/": [
     ".\\src\\ResourceManager\\TrafficManager\\Commands.TrafficManager2.Test\\bin\\Debug\\Microsoft.Azure.Commands.TrafficManager.Test.ScenarioTests.dll"

build.proj

@@ -149,6 +149,7 @@
     <NuGetLocalBuildTaskPackages>-PackagesDirectory $(LibraryToolsFolder)\BuildPackagesTask\packages</NuGetLocalBuildTaskPackages>
 
     <PowerShellCommand Condition=" '$(PowerShellCommand)' == '' ">C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe</PowerShellCommand>
+    <PowerShellCoreCommand Condition=" '$(PowerShellCoreCommand)' == '' ">pwsh</PowerShellCoreCommand>
   </PropertyGroup>
 
   <!--
@@ -271,54 +272,54 @@
 
     <!-- Delete powershell runtime files -->
     <ItemGroup>
-      <RedundantDlls Include="$(PackageDirectory)\**\netcoreapp2.0\publish\Microsoft.Powershell.*.dll" />
-      <RedundantDlls Include="$(PackageDirectory)\**\netcoreapp2.0\publish\System*.dll" 
-                     Exclude="$(PackageDirectory)\**\netcoreapp2.0\publish\System.Security.Cryptography.ProtectedData.dll" />
-      <RedundantDlls Include="$(PackageDirectory)\**\netcoreapp2.0\publish\Microsoft.VisualBasic.dll" />
-      <RedundantDlls Include="$(PackageDirectory)\**\netcoreapp2.0\publish\Microsoft.CSharp.dll" />
-      <RedundantDlls Include="$(PackageDirectory)\**\netcoreapp2.0\publish\Microsoft.CodeAnalysis.dll" />
-      <RedundantDlls Include="$(PackageDirectory)\**\netcoreapp2.0\publish\Microsoft.CodeAnalysis.CSharp.dll" />
+      <RedundantDlls Include="$(PackageDirectory)\**\publish\Microsoft.Powershell.*.dll" />
+      <RedundantDlls Include="$(PackageDirectory)\**\publish\System*.dll" 
+                     Exclude="$(PackageDirectory)\**\publish\System.Security.Cryptography.ProtectedData.dll" />
+      <RedundantDlls Include="$(PackageDirectory)\**\publish\Microsoft.VisualBasic.dll" />
+      <RedundantDlls Include="$(PackageDirectory)\**\publish\Microsoft.CSharp.dll" />
+      <RedundantDlls Include="$(PackageDirectory)\**\publish\Microsoft.CodeAnalysis.dll" />
+      <RedundantDlls Include="$(PackageDirectory)\**\publish\Microsoft.CodeAnalysis.CSharp.dll" />
     </ItemGroup>
 
     <Delete Files="@(RedundantDlls->'%(FullPath)')"/>
 
-    <!-- Copy content of the publish folder one folders up -->
+    <!-- Copy content of the publish folder one folder up. -->
     <ItemGroup>
-      <PackageContent Include="$(PackageDirectory)\**\netcoreapp2.0\publish\*.*" />
+      <PackageContent Include="$(PackageDirectory)\**\publish\*.*" />
       <SourceFile Include="@(PackageContent)">
         <DestinationFolder>$([System.IO.Path]::Combine('%(RelativeDir)', '..', '%(FileName)%(Extension)'))</DestinationFolder>
       </SourceFile>
     </ItemGroup>
 
     <Copy SourceFiles="@(SourceFile->'%(FullPath)')" DestinationFiles="@(SourceFile->'%(DestinationFolder)')" SkipUnchangedFiles="true"/>
 
-    <!-- Move PSD1 files one folders up -->
-    <ItemGroup>
-      <psdFiles Include="$(PackageDirectory)\**\netcoreapp2.0\*.psd1" />
-      <psdFilesTarget Include="@(psdFiles)">
-        <DestinationFolder>$([System.IO.Path]::Combine('%(RelativeDir)', '..', '%(FileName)%(Extension)'))</DestinationFolder>
-      </psdFilesTarget>
-    </ItemGroup>
-
-    <Move SourceFiles="@(psdFilesTarget->'%(FullPath)')" DestinationFiles="@(psdFilesTarget->'%(DestinationFolder)')"/>
-
     <!-- Delete build artifacts -->
     <ItemGroup>
-      <GetAllFiles Include="$(PackageDirectory)\**\netcoreapp2.0\publish\*.*" />
+      <GetAllFiles Include="$(PackageDirectory)\**\publish\*.*" />
       <Artifacts Include="@(GetAllFiles->'%(RootDir)%(Directory)'->Distinct())" />
     </ItemGroup>
 
     <RemoveDir Directories="%(Artifacts.Identity)" ContinueOnError="true" />
 
     <ItemGroup>
-      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureRM.Resources.Netcore\AzureRM.Tags.Netcore.psd1" />
-      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureRM.Websites.Netcore\AzureRM.Resources.Netcore.psd1" />
-      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureRM.Websites.Netcore\AzureRM.Tags.Netcore.psd1" />
-      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureRM.Network.Netcore\AzureRM.Resources.Netcore.psd1" />
-      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureRM.Network.Netcore\AzureRM.Tags.Netcore.psd1" />
+      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureResourceManager\AzureRM.Resources.Netcore\AzureRM.Tags.Netcore.psd1" />
+      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureResourceManager\AzureRM.Websites.Netcore\AzureRM.Resources.Netcore.psd1" />
+      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureResourceManager\AzureRM.Websites.Netcore\AzureRM.Tags.Netcore.psd1" />
+      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureResourceManager\AzureRM.Network.Netcore\AzureRM.Resources.Netcore.psd1" />
+      <ExtraPsdFiles Include="$(PackageDirectory)\$(Configuration)\ResourceManager\AzureResourceManager\AzureRM.Network.Netcore\AzureRM.Tags.Netcore.psd1" />
     </ItemGroup>
 
     <Delete Files="@(ExtraPsdFiles->'%(FullPath)')"/>
+
+    <!-- Update module manifests. -->
+    <Exec Command="&quot;$(PowerShellCoreCommand)&quot; -NonInteractive -NoLogo -NoProfile -Command &quot;. $(LibraryToolsFolder)\UpdateModules.ps1 $(Configuration) AzureRM.Netcore -Profile Latest &quot; " Condition="'$(Latest)' == 'true' and '$(CodeSign)' == 'false'"/>
+
+    <!-- Generate the help. -->
+    <Exec Command="&quot;$(PowerShellCoreCommand)&quot; -NonInteractive -NoLogo -NoProfile -Command &quot;. $(LibraryToolsFolder)\GenerateHelp.ps1 -GenerateMamlHelp -BuildConfig $(Configuration) &quot;" ContinueOnError="true" Condition=" '$(SkipHelp)' == 'false' and '$(Latest)' == 'true'"/>
+
+    <!-- Cleanup extraneous files. -->
+    <Exec Command="&quot;$(PowerShellCoreCommand)&quot; -NonInteractive -NoLogo -NoProfile -Command &quot;. $(LibraryToolsFolder)\CleanupBuild.ps1 -BuildConfig $(Configuration) &quot;" />
+
     <CallTarget Targets="CodeSignBinaries" Condition=" '$(CodeSign)' == 'true' " /> 
   </Target>
 
@@ -368,7 +369,6 @@
       <Output TaskParameter="Path" PropertyName="WindowsSdkPath"/>
     </GetFrameworkSdkPath>
 
-
     <Copy SourceFiles="$(LibraryRoot)tools\AzureRM.Netcore\AzureRM.Netcore.psd1"
           DestinationFolder="$(LibrarySourceFolder)\Package\$(Configuration)" Condition= " '$(NetCore)' == 'true' "/>
 

documentation/Using-Azure-TestFramework.md

@@ -39,7 +39,7 @@ In order to Record/Playback a test, you need to setup a connection string that c
 
 ### New-TestCredential
 
-This cmdlet, located in Repo-Tasks, will allow you to create a credentials file (located in C:/Users/\<currentuser\>/.azure/testcredentials.json) that will be used to set the environment variable when scenario tests are run. This credentials file will be used in all future sessions unless it is deleted or the environment variables are manually set.  This cmdlet is not currently available for .NET SDK development.
+This cmdlet, located in [Repo-Tasks.psd1](/tools/Repo-Tasks.psd1), which pulls in [TestFx-Tasks.psd1](/tools/Modules/TestFx-Tasks.psd1) and [Build-Tasks.psd1](/tools/Modules/Build-Tasks.psd1), will allow you to create a credentials file (located in C:/Users/\<currentuser\>/.azure/testcredentials.json) that will be used to set the environment variable when scenario tests are run. This credentials file will be used in all future sessions unless it is deleted or the environment variables are manually set.  This cmdlet is not currently available for .NET SDK development.
 
 #### Create New Service Principal
 

setup/azurecmd.wxs

@@ -1,11 +1,11 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <Wix xmlns="http://schemas.microsoft.com/wix/2006/wi" xmlns:util="http://schemas.microsoft.com/wix/UtilExtension" xmlns:netfx="http://schemas.microsoft.com/wix/NetFxExtension">
 
-  <?define productName="Microsoft Azure PowerShell - November 2017" ?>
+  <?define productName="Microsoft Azure PowerShell - December 2017" ?>
   <?define sourceDir="$(var.SolutionDir)..\src\Package\$(var.Configuration)" ?>
   <?define caSourceDir="$(var.SolutionDir)setup\bin\$(var.Configuration)" ?>
 
-  <?define version="5.0.0" ?>
+  <?define version="5.1.1" ?>
 
   <Product Id="*"
            Name="$(var.productName)"