Merge branch 'preview' of https://github.com/Azure/azure-powershell into preview

Author: Andrey Fedyashov

.gitignore

@@ -237,7 +237,6 @@ Package
 launchSettings.json
 /src/StackAdmin/AzureRM/AzureRM.psm1
 /src/StackAdmin/AzureStack/AzureStack.psm1
-/tools/AzureRM.Netcore/AzureRM.Netcore.psm1
 /ModuleMappings.json
 /SolutionMappings.json
 /TestMappings.json

.travis.yml

@@ -25,7 +25,7 @@ before_install:
  # https://github.com/travis-ci/travis-ci/issues/1066#issuecomment-383489298
 script:
  - sudo dotnet msbuild build.proj /t:BuildNetcore /p:Configuration=$CONFIG || travis_terminate 1
- - sudo dotnet tools/StaticAnalysis/bin/$CONFIG/netcoreapp2.1/StaticAnalysis.Netcore.dll -p src/Package/$CONFIG -r src/Package -u || travis_terminate 1
+ - sudo dotnet src/Package/StaticAnalysis/StaticAnalysis.Netcore.dll -p $TRAVIS_BUILD_DIR/src/Package/$CONFIG -r $TRAVIS_BUILD_DIR/src/Package/StaticAnalysis -u || travis_terminate 1
  - sudo pwsh -NonInteractive -NoLogo -NoProfile -File tools/TestModuleLoading.ps1 || travis_terminate 1
  - sudo dotnet pack src/TestPrerequisites/Common.TestPrerequisites.sln --configuration $CONFIG || travis_terminate 1
  - sudo dotnet test src/Azure.PowerShell.Netcore.Test.sln --filter "AcceptanceType=CheckIn&RunType!=DesktopOnly" --configuration $CONFIG --framework netcoreapp2.0

ChangeLog.md

@@ -1,4 +1,30 @@
-## 6.10.0 - October 2018
+## 6.11.0 - October 2018
+#### AzureRM.Profile
+* Fix issue with Get-AzureRmSubscription in CloudShell
+* Update common code to use latest version of ClientRuntime
+
+#### AzureRM.Backup
+* Deprecated Azure Backup cmdlets.
+
+#### AzureRM.Compute
+* Added new sizes to the whitelist of VM sizes for which accelerated networking will be turned on when using the simple param set for 'New-AzureRmVm'
+* Added ResourceName argument completer to all cmdlets.
+
+#### AzureRM.DataLakeStore
+* Adding support for Virtual Network Rules
+    - Get-AzureRmDataLakeStoreVirtualNetworkRule: Gets or Lists Azure Data Lake Store virtual network rule.
+    - Add-AzureRmDataLakeStoreVirtualNetworkRule: Adds a virtual network rule to the specified Data Lake Store account.
+    - Set-AzureRmDataLakeStoreVirtualNetworkRule: Modifies the specified virtual network rule in the specified Data Lake Store account.
+    - Remove-AzureRmDataLakeStoreVirtualNetworkRule: Deletes an Azure Data Lake Store virtual network rule.
+
+#### AzureRM.Network
+* Update cmdlet Test-AzureRmNetworkWatcherConnectivity, pass the protocol value to backend.
+* Added ResourceName argument completer to all cmdlets.
+
+#### AzureRM.Resources
+* Fix isssue where Get-AzureRMRoleDefinition throws an unintelligible exception (when the default profile has no subscription in it and no scope is specified) by adding a meaningful exception in the scenario. Also set the default param set to 'RoleDefinitionNameParameterSet'.
+
+## 6.10.0 - October 2018
 #### Azure.Storage
 * Fix Copy Blob/File won't copy metadata when destination has metadata issue
     - Start-AzureStorageBlobCopy

build.proj

@@ -1,7 +1,6 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 
-
   <!--
     Available Targets:
 
@@ -68,7 +67,7 @@
   </PropertyGroup>
 
   <PropertyGroup Condition="$(NetCore) == 'true' ">
-    <Scope>NetCore</Scope>
+    <Scope>Netcore</Scope>
   </PropertyGroup>
 
   <!-- Set Skips -->
@@ -85,7 +84,6 @@
     <SkipDependencyAnalysis>true</SkipDependencyAnalysis>
   </PropertyGroup>
 
-
   <ItemGroup>
     <LocalBuildTasks  Include="$(LibraryToolsFolder)\RepoTasks\RepoTasks.sln" Condition="'$(NetCore)' == 'false'" />
     <StackSolution    Include="$(LibrarySourceFolder)\StackAdmin\Stack.sln" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
@@ -112,9 +110,9 @@
             <![CDATA[System.Environment.SetEnvironmentVariable(EnvName, EnvValue);]]>
       </Code>
     </Task>
-  </UsingTask>  <!--
-  CI build related
-  -->
+  </UsingTask>
+
+  <!-- CI build related -->
   <PropertyGroup>
     <!--OnPremiseBuildTasks is not a good name, but CI server is using that, will update across soon-->
     <CIToolsPath>$(OnPremiseBuildTasks)</CIToolsPath>
@@ -201,11 +199,6 @@
     <!-- Restore packages for local build tasks-->
     <Exec Command="$(NuGetCommand) restore %(LocalBuildTasks.FullPath) $(NuGetRestoreConfigSwitch) $(NuGetRestorePackageSetting)"
             ContinueOnError="false" />
-    <!-- Restore packages for repo tasks tasks-->
-    <Exec Command="$(NuGetCommand) restore %(LocalBuildTasks.FullPath) $(NuGetRestoreConfigSwitch) $(NuGetRestorePackageSetting)"
-            ContinueOnError="false" />
-    <Exec Command="$(NuGetCommand) restore %(SetupSln.FullPath) $(NuGetRestoreConfigSwitch) $(NuGetRestorePackageSetting)"
-            ContinueOnError="false" />
     <!--Restore the xunit runner needed to run unit tests-->
     <Exec Command="$(NuGetCommand) restore $(MSBuildProjectDirectory)\packages.config -PackagesDirectory $(MSBuildProjectDirectory)\packages" />
   </Target>
@@ -282,60 +275,12 @@
     <Exec Command="$(PowerShellCommandPrefix) &quot;. $(LibraryToolsFolder)\UpdateModules.ps1 -BuildConfig $(Configuration) -Scope $(Scope) &quot;"
           Condition="'$(CodeSign)' == 'false'"/>
 
-    <!-- Copying modules to be signed -->
-
-    <!-- AzureRM -->
-    <Copy SourceFiles="$(LibraryToolsFolder)\AzureRM\AzureRM.psd1"
-          DestinationFolder="$(PackageDirectory)\$(Configuration)"
-          ContinueOnError="false"
-          Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Latest'" />
-    <Copy SourceFiles="$(LibraryToolsFolder)\AzureRM\AzureRM.psm1"
-          DestinationFolder="$(PackageDirectory)\$(Configuration)"
-          ContinueOnError="false"
-          Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Latest'" />
-
-    <!-- Stack Specific -->
-    <Copy SourceFiles="$(LibrarySourceFolder)\StackAdmin\AzureRM\AzureRM.psd1"
-          DestinationFolder="$(StackPackageFolder)\$(Configuration)"
-          ContinueOnError="false"
-          Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
-    <Copy SourceFiles="$(LibrarySourceFolder)\StackAdmin\AzureRM\AzureRM.psm1"
-          DestinationFolder="$(StackPackageFolder)\$(Configuration)"
-          ContinueOnError="false"
-          Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
-
-    <Copy SourceFiles="$(LibrarySourceFolder)\StackAdmin\AzureStack\AzureStack.psd1"
-          DestinationFolder="$(StackPackageFolder)\$(Configuration)"
-          ContinueOnError="false"
-          Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
-    <Copy SourceFiles="$(LibrarySourceFolder)\StackAdmin\AzureStack\AzureStack.psm1"
-          DestinationFolder="$(StackPackageFolder)\$(Configuration)"
-          ContinueOnError="false"
-          Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
-
-    <CallTarget Targets="CodeSignBinaries" Condition=" '$(CodeSign)' == 'true' " />
+    <CallTarget Targets="CodeSignBinaries" Condition="'$(CodeSign)' == 'true'" />
 
     <Exec ContinueOnError="false"
       Command="&quot;$(PowerShellCommand)&quot; -NonInteractive -NoLogo -NoProfile -Command &quot; . $(LibraryToolsFolder)\NewOutputTypeIndex.ps1 -OutputFile $(PackageDirectory)\outputtypes.json -BuildConfig $(Configuration) &quot;"
       Condition="('$(Scope)' == 'All' or '$(Scope)' == 'Latest') and $(CodeSign) == 'true'" />
 
-    <!-- AzureRM -->
-    <Copy SourceFiles="$(PackageDirectory)\$(Configuration)\AzureRM.psd1"
-          DestinationFolder="$(LibraryToolsFolder)\AzureRM" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Latest'" />
-    <Copy SourceFiles="$(PackageDirectory)\$(Configuration)\AzureRM.psm1"
-          DestinationFolder="$(LibraryToolsFolder)\AzureRM" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Latest'" />
-
-    <!-- Stack -->
-    <Copy SourceFiles="$(StackPackageFolder)\$(Configuration)\AzureRM.psd1"
-          DestinationFolder="$(LibrarySourceFolder)\StackAdmin\AzureRM" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
-    <Copy SourceFiles="$(StackPackageFolder)\$(Configuration)\AzureRM.psm1"
-          DestinationFolder="$(LibrarySourceFolder)\StackAdmin\AzureRM" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
-
-    <Copy SourceFiles="$(StackPackageFolder)\$(Configuration)\AzureStack.psd1"
-          DestinationFolder="$(LibrarySourceFolder)\StackAdmin\AzureStack" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
-    <Copy SourceFiles="$(StackPackageFolder)\$(Configuration)\AzureStack.psm1"
-          DestinationFolder="$(LibrarySourceFolder)\StackAdmin\AzureStack" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
-
     <Message Importance="high" Text="Running Static Analyser" />
     <CallTarget targets="DependencyAnalysis" ContinueOnError="ErrorAndContinue" />
     <Exec Command="$(PowerShellCommandPrefix) &quot;. $(LibraryToolsFolder)\CheckAssemblies.ps1 -BuildConfig $(Configuration) &quot;" />
@@ -346,14 +291,15 @@
           Condition="('$(Scope)' == 'All' or '$(Scope)' == 'Latest') and $(CodeSign) == 'true'" />
   </Target>
 
-
-
   <!--
     Build .Net Core Cmdlets
     Fully self contained
   -->
   <Target Name="BuildNetCore">
     <Message Importance="high" Text="Building Cmdlets..." />
+    <PropertyGroup>
+      <Scope>Netcore</Scope>
+    </PropertyGroup>
 
     <Exec Command="$(PowerShellCoreCommandPrefix) &quot;$ProgressPreference = 'SilentlyContinue';. $(LibraryToolsFolder)\CreateAliasMapping.ps1 &quot;"/>
 
@@ -404,7 +350,8 @@
     <Delete Files="@(ExtraPsdFiles->'%(FullPath)')"/>
 
     <!-- Update module manifests. -->
-    <Exec Command="$(PowerShellCoreCommandPrefix) &quot;. $(LibraryToolsFolder)\UpdateModules.ps1 -BuildConfig $(Configuration) -Scope Netcore &quot; "
+    <Exec Command="$(PowerShellCoreCommandPrefix) &quot;. $(LibraryToolsFolder)\UpdateModules.ps1 -BuildConfig $(Configuration) -Scope $(Scope) &quot; "
+          Condition="'$(CodeSign)' == 'false'"
           ContinueOnError="false" />
     <!-- Generate the Help -->
     <Exec Command="$(PowerShellCoreCommandPrefix) &quot;Set-Variable -Name ProgressPreference -Value 'SilentlyContinue';. $(LibraryToolsFolder)\GenerateHelp.ps1 -ValidateMarkdownHelp -GenerateMamlHelp -BuildConfig $(Configuration) &quot;"
@@ -413,15 +360,13 @@
 
     <!-- Cleanup extraneous files. -->
     <Exec Command="$(PowerShellCoreCommandPrefix) &quot;. $(LibraryToolsFolder)\CleanupBuild.ps1 -BuildConfig $(Configuration) &quot;" />
-
-    <CallTarget Targets="CodeSignBinaries" Condition=" '$(CodeSign)' == 'true' " />
+    
+    <CallTarget Targets="CodeSignBinaries" Condition="'$(CodeSign)' == 'true'" />
   </Target>
 
-
   <!-- Do everything possible except Publish -->
   <Target Name="Full" DependsOnTargets="Clean;Build;Test" />
 
-
   <Target Name="BuildMsBuildTask" DependsOnTargets="RestoreNugetPackages">
     <Message Importance="high" Text="Building RepoTasks" />
     <MSBuild Projects="@(LocalBuildTasks)"
@@ -438,9 +383,6 @@
       <Output TaskParameter="Path" PropertyName="WindowsSdkPath"/>
     </GetFrameworkSdkPath>
 
-    <Copy SourceFiles="$(LibraryRoot)tools\Az\Az.psd1"
-          DestinationFolder="$(PackageDirectory)\$(Configuration)" Condition= " '$(Scope)' == 'Netcore' "/>
-
     <!-- Azure -->
     <ItemGroup Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Latest' or '$(Scope)' == 'Netcore'">
       <DelaySignedAssembliesToSign Include="$(PackageDirectory)\$(Configuration)\**\Microsoft*Azure*Commands*.dll" Exclude="$(PackageDirectory)\$(Configuration)\**\Microsoft*Azure*Commands*Common*.dll" />
@@ -486,18 +428,18 @@
         ContinueOnError="false"
         Condition="!$(DelaySign) and '@(DelaySignedAssembliesToSign)' != ''"/>
 
-    <Exec Command="$(PowerShellCommandPrefix) &quot;. $(LibraryToolsFolder)\UpdateModules.ps1 -BuildConfig $(Configuration) -Scope $(Scope) &quot;"
-          Condition="'$(Scope)' != 'Netcore'"/>
+    <Exec Command="$(PowerShellCommandPrefix) &quot;. $(LibraryToolsFolder)\UpdateModules.ps1 -BuildConfig $(Configuration) -Scope $(Scope) &quot;"/>
 
     <!-- Copying shortcut to be signed -->
+    <Copy SourceFiles="$(LibraryRoot)tools\Az\Az.psm1"
+          DestinationFolder="$(PackageDirectory)\$(Configuration)" Condition="'$(Scope)' == 'Netcore'" />
     <Copy SourceFiles="$(LibraryRoot)tools\AzureRM\AzureRM.psm1"
           DestinationFolder="$(PackageDirectory)\$(Configuration)" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Latest'" />
     <Copy SourceFiles="$(LibrarySourceFolder)\StackAdmin\AzureRM\AzureRM.psm1"
           DestinationFolder="$(StackPackageFolder)\$(Configuration)" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
     <Copy SourceFiles="$(LibrarySourceFolder)\StackAdmin\AzureStack\AzureStack.psm1"
           DestinationFolder="$(StackPackageFolder)\$(Configuration)" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
 
-
     <!-- Azure -->
     <ItemGroup Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Latest' or '$(Scope)' == 'Netcore'">
       <ScriptsToSign Include="$(PackageDirectory)\$(Configuration)\**\*.ps1"/>
@@ -533,22 +475,24 @@
           Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'"
           ContinueOnError="ErrorAndContinue" />
 
-    <!-- Copying signed shortcut back -->
-    <Copy SourceFiles="$(PackageDirectory)\$(Configuration)\Az.psd1"
-          DestinationFolder="$(LibraryRoot)tools\Az" Condition= " '$(Scope)' == 'Latest' "/>
+    <!-- Copy files back after signing -->
+      <Copy SourceFiles="$(PackageDirectory)\$(Configuration)\Az.psm1"
+          DestinationFolder="$(LibraryRoot)tools\Az" Condition="'$(Scope)' == 'Netcore'" />
+    <Copy SourceFiles="$(PackageDirectory)\$(Configuration)\AzureRM.psm1"
+          DestinationFolder="$(LibraryRoot)tools\AzureRM" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Latest'" />
+    <Copy SourceFiles="$(StackPackageFolder)\$(Configuration)\AzureRM.psm1"
+          DestinationFolder="$(LibrarySourceFolder)\StackAdmin\AzureRM" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
+    <Copy SourceFiles="$(StackPackageFolder)\$(Configuration)\AzureStack.psm1"
+          DestinationFolder="$(LibrarySourceFolder)\StackAdmin\AzureStack" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Stack'" />
   </Target>
 
-
-
   <Target Name="BuildInstaller" Condition="'$(Scope)' == 'All' or '$(Scope)' == 'Latest'">
     <Exec Command="$(PowerShellCommandPrefix) &quot;. Register-PSRepository -Name MSIcreationrepository -SourceLocation $(PackageDirectory) -InstallationPolicy Trusted &quot; "/>
     <Exec Command="$(PowerShellCommandPrefix) &quot;. $(LibraryRoot)\setup\generate.ps1 -repository MSIcreationrepository &quot; "/>
     <Exec Command="$(PowerShellCommandPrefix) &quot;. Unregister-PSRepository -Name MSIcreationrepository &quot; "/>
     <CallTarget Targets="CodeSignInstaller" Condition=" '$(CodeSign)' == 'true'" />
   </Target>
 
-
-
   <Target Name="CodeSignInstaller">
     <PropertyGroup>
       <!--public token associated with MSSharedLibKey.snk-->
@@ -580,8 +524,6 @@
     <SetEnvVar EnvName="SignedMsiDir" EnvValue="$(SignedOutputRootDir)" />
   </Target>
 
-
-
   <!-- Run Validation -->
   <Target Name="DependencyAnalysis"
           Condition="'$(SkipDependencyAnalysis)' == 'false'">
@@ -606,8 +548,7 @@
     <OnError ExecuteTargets="StaticAnalysisErrorMessage"/>
   </Target>
 
-
- <Target Name="StaticAnalysisErrorMessage">
+  <Target Name="StaticAnalysisErrorMessage">
     <Error Text="StaticAnalysis has failed.  Please follow the instructions on this doc: https://github.com/Azure/azure-powershell/blob/preview/documentation/Debugging-StaticAnalysis-Errors.md"/>
   </Target>
 
@@ -637,7 +578,7 @@
           ContinueOnError="false"/>
 
     <CallTarget Targets="BuildInstaller" Condition="'$(Scope)' =='All'" />
- </Target>
+  </Target>
 
   <PropertyGroup>
     <RunTestLive Condition="'$(RunTestLive)' == ''">false</RunTestLive>
@@ -697,10 +638,7 @@
     DependsOnTargets="Clean;Build;Test;RunOneSDKCIT" />
 
   <!-- Run live tests and record mocks -->
-  <Target
-  Name="RunLiveTests"
-  DependsOnTargets="Clean;Build;LiveTestsFilter" />
-
+  <Target Name="RunLiveTests" DependsOnTargets="Clean;Build;LiveTestsFilter" />
 
   <Target Name="LiveTestsFilter" DependsOnTargets="Build;BeforeRunTests">
     <Message Importance="high" Text="Running live tests..." />

documentation/announcing-az-module.md

@@ -29,10 +29,27 @@ Az is a new module, and reorganizing and simplifying cmdlet names involves break
 ## AzureRM Module Support
 AzureRM will continue to be supported, and important bugs will be fixed, but new development and new Azure capabilities will be shipped only in Az starting December 2018.
 
+## Authentication Changes
+   - ADAL has removed support for the "not recommended" user credential auth flow.
+     - Connect-AzAccount will no longer accept PSCredential from Get-Credential
+     - Here are a couple issues that describe why ADAL has removed this support   
+        - [Azure ActiveDirectory UserPasswordCredential doesn't support .NET core](https://github.com/AzureAD/azure-activedirectory-library-for-dotnet/issues/482#issuecomment-262256236)
+        - [Connect-AzureRmAccount with user credential login does not work in Az](https://github.com/Azure/azure-powershell/issues/7430#issuecomment-426480499)
+
+  - Service Principal improvements
+    - Self-renewing Service Principal Authentication
+    - Service Principal Certificate Authentication (in the future)
+  - Device Auth Flow
+  - Future Improvements
+    - [Interactive user credential login in az](https://github.com/Azure/azure-powershell/issues/7358)
+
+
 ## Migrating From AzureRM
 
 To make it easier for existing scripts to migrate from AzureRM to Az, we have provided cmdlets to create aliases that map the cmdlet names in AzureRM into the appropriate cmdlets in Az.  When the aliases are enabled, any script that uses AzureRM cmdlet names should run against Az without modification.
 
+Note: Connect-AzAccount aka Connect-AzureRmAccount no longer supports $pscredential. See [Authentication Changes](#authentication-changes) for more
+
 ```powershell
 PS C:\> Enable-AzureRmAlias
 ```

documentation/breaking-changes/breaking-changes-attribute-help.md

@@ -35,6 +35,7 @@ All of the attributes have the following common traits:
 
 The properties "deprecatedByVersion" and "changeInEfectByDate" are infotmation only (for the end user) as of now.
 
+#### Supress the breaking change messages at runtime
 - If the user desires to not see the warning messages generated by the attirbutes at runtime, they can set the env variable "SuppressAzurePowerShellBreakingChangeWarnings" to "true".
 
 NOTE :
@@ -485,4 +486,4 @@ Get-SomeObjectA -Param1=...
 ...
 Breaking changes in the cmdlet : Get-SomeObjectA
  - The parameter 'Param1' is becoming mandatory
-```
+```