Skip to content

Commit de907d1

Browse files
cormacpaynecormacpayne
authored
Merge pull request Azure#4718 from ro-joowan/object-refactor-2
[ADLA - ADLS] - Object refactoring
2 parents 90a36a0 + f161741 commit de907d1

File tree

63 files changed

+124438
-49293
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

63 files changed

+124438
-49293
lines changed

‎src/ResourceManager/DataLakeAnalytics/ChangeLog.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,16 @@
1818
- Additional information about change #1
1919
-->
2020
## Current Release
21+
* [BREAKING CHANGE] Removed the Obsolete Properties fields in PSDataLakeAnalyticsAccount.cs and its assoicated files
22+
* Changed one of the two OutputTypes of Get-AzureRmDataLakeAnalyticsAccount
23+
- List<DataLakeAnalyticsAccount> to List<PSDataLakeAnalyticsAccountBasic>
24+
- The properties of PSDataLakeAnalyticsAccountBasic is a strict subset of the properties of DataLakeAnalyticsAccount
25+
- The additional properties that are in DataLakeAnalyticsAccount are not returned by the service. Therefore, this change is to reflect this accurately. These additional properties are still in PSDataLakeAnalyticsAccountBasic, but they are tagged as Obsolete.
26+
* Changed one of the two OutputTypes of Get-AzureRmDataLakeAnalyticsJob
27+
- List<JobInformation> to List<PSJobInformationBasic>
28+
- The properties of PSJobInformationBasic is a strict subset of the properties of JobInformation
29+
- The additional properties that are in JobInformation are not returned by the service. Therefore, this change is to reflect this accurately. These additional properties are still in PSJobInformationBasic, but they are tagged as Obsolete.
30+
* Updated the cmdlet logic of Submit-AzureRmDataLakeAnalyticsJob because SubmitJob and BuildJob now require an explicit parameter object rather than the generic JobInformation object
2131

2232
## Version 3.4.1
2333

@@ -95,4 +105,4 @@
95105
- Better error messaging and support for invalid input
96106
* General help improvements
97107
- Clearer help for job operations
98-
- Fixed typos and incorrect examples
108+
- Fixed typos and incorrect examples

‎src/ResourceManager/DataLakeAnalytics/Commands.DataLakeAnalytics.Test/Commands.DataLakeAnalytics.Test.csproj

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -64,12 +64,11 @@
6464
</Reference>
6565
<Reference Include="Microsoft.Azure.Management.DataLake.Analytics, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
6666
<SpecificVersion>False</SpecificVersion>
67-
<HintPath>..\..\..\packages\Microsoft.Azure.Management.DataLake.Analytics.3.0.0\lib\net452\Microsoft.Azure.Management.DataLake.Analytics.dll</HintPath>
67+
<HintPath>..\..\..\packages\Microsoft.Azure.Management.DataLake.Analytics.3.1.2-preview\lib\net452\Microsoft.Azure.Management.DataLake.Analytics.dll</HintPath>
6868
<Private>True</Private>
6969
</Reference>
70-
<Reference Include="Microsoft.Azure.Management.DataLake.Store, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
71-
<SpecificVersion>False</SpecificVersion>
72-
<HintPath>..\..\..\packages\Microsoft.Azure.Management.DataLake.Store.2.2.0\lib\net452\Microsoft.Azure.Management.DataLake.Store.dll</HintPath>
70+
<Reference Include="Microsoft.Azure.Management.DataLake.Store">
71+
<HintPath>..\..\..\packages\Microsoft.Azure.Management.DataLake.Store.2.3.0-preview\lib\net452\Microsoft.Azure.Management.DataLake.Store.dll</HintPath>
7372
<Private>True</Private>
7473
</Reference>
7574
<Reference Include="Microsoft.Azure.Management.ResourceManager, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
@@ -104,12 +103,12 @@
104103
</Reference>
105104
<Reference Include="Microsoft.Rest.ClientRuntime, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
106105
<SpecificVersion>False</SpecificVersion>
107-
<HintPath>..\..\..\packages\Microsoft.Rest.ClientRuntime.2.3.8\lib\net452\Microsoft.Rest.ClientRuntime.dll</HintPath>
106+
<HintPath>..\..\..\packages\Microsoft.Rest.ClientRuntime.2.3.10\lib\net452\Microsoft.Rest.ClientRuntime.dll</HintPath>
108107
<Private>True</Private>
109108
</Reference>
110109
<Reference Include="Microsoft.Rest.ClientRuntime.Azure, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
111110
<SpecificVersion>False</SpecificVersion>
112-
<HintPath>..\..\..\packages\Microsoft.Rest.ClientRuntime.Azure.3.3.7\lib\net452\Microsoft.Rest.ClientRuntime.Azure.dll</HintPath>
111+
<HintPath>..\..\..\packages\Microsoft.Rest.ClientRuntime.Azure.3.3.10\lib\net452\Microsoft.Rest.ClientRuntime.Azure.dll</HintPath>
113112
<Private>True</Private>
114113
</Reference>
115114
<Reference Include="Microsoft.Rest.ClientRuntime.Azure.Authentication, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">

‎src/ResourceManager/DataLakeAnalytics/Commands.DataLakeAnalytics.Test/ScenarioTests/AdlaAliasTests.ps1

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,10 @@ function Test-DataLakeAnalyticsJobRelationships
4242
Assert-False {$i -eq 60} "dataLakeAnalytics accounts not in succeeded state even after 30 min."
4343
}
4444

45+
# Wait for 5 minutes for the server to restore the account cache
46+
# Without this, the test will pass non-deterministically
47+
[Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::Wait(300000)
48+
4549
# submit a job
4650
$guidForJob = [Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::GenerateGuid("relationTest01")
4751

@@ -576,6 +580,10 @@ function Test-DataLakeAnalyticsJob
576580
Assert-False {$i -eq 60} "dataLakeAnalytics accounts not in succeeded state even after 30 min."
577581
}
578582

583+
# Wait for 5 minutes for the server to restore the account cache
584+
# Without this, the test will pass non-deterministically
585+
[Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::Wait(300000)
586+
579587
# submit a job
580588
$guidForJob = [Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::GenerateGuid("jobTest02")
581589
[Microsoft.Azure.Commands.DataLakeAnalytics.Models.DataLakeAnalyticsClient]::JobIdQueue.Enqueue($guidForJob)
@@ -595,12 +603,12 @@ function Test-DataLakeAnalyticsJob
595603

596604
Assert-NotNull {Get-AdlJob -AccountName $accountName}
597605

598-
$jobsWithDateOffset = Get-AdlJob -AccountName $accountName -SubmittedAfter $([DateTimeOffset]($nowTime).AddMinutes(-5))
606+
$jobsWithDateOffset = Get-AdlJob -AccountName $accountName -SubmittedAfter $([DateTimeOffset]($nowTime).AddMinutes(-10))
599607

600-
Assert-True {$jobsWithDateOffset.Count -gt 0} "Failed to retrieve jobs submitted after five miuntes ago"
608+
Assert-True {$jobsWithDateOffset.Count -gt 0} "Failed to retrieve jobs submitted after ten miuntes ago"
601609

602-
# we add five minutes to ensure that the timing is right, since we are using the account creation time, and not truly "now"
603-
$jobsWithDateOffset = Get-AdlJob -AccountName $accountName -SubmittedBefore $([DateTimeOffset]($nowTime).AddMinutes(5))
610+
# we add ten minutes to ensure that the timing is right, since we are using the account creation time, and not truly "now"
611+
$jobsWithDateOffset = Get-AdlJob -AccountName $accountName -SubmittedBefore $([DateTimeOffset]($nowTime).AddMinutes(10))
604612

605613
Assert-True {$jobsWithDateOffset.Count -gt 0} "Failed to retrieve jobs submitted before right now"
606614

@@ -817,6 +825,10 @@ function Test-DataLakeAnalyticsCatalog
817825
[Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::Wait(30000)
818826
Assert-False {$i -eq 60} "dataLakeAnalytics accounts not in succeeded state even after 30 min."
819827
}
828+
829+
# Wait for 5 minutes for the server to restore the account cache
830+
# Without this, the test will pass non-deterministically
831+
[Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::Wait(300000)
820832

821833
# Run a job to create the catalog items (except secret and credential)
822834
$scriptTemplate = @"

‎src/ResourceManager/DataLakeAnalytics/Commands.DataLakeAnalytics.Test/ScenarioTests/AdlaTests.ps1

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -467,6 +467,10 @@ function Test-DataLakeAnalyticsJob
467467
Assert-False {$i -eq 60} "dataLakeAnalytics accounts not in succeeded state even after 30 min."
468468
}
469469

470+
# Wait for 5 minutes for the server to restore the account cache
471+
# Without this, the test will pass non-deterministically
472+
[Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::Wait(300000)
473+
470474
# submit a job
471475
$guidForJob = [Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::GenerateGuid("jobTest01")
472476
[Microsoft.Azure.Commands.DataLakeAnalytics.Models.DataLakeAnalyticsClient]::JobIdQueue.Enqueue($guidForJob)
@@ -485,11 +489,12 @@ function Test-DataLakeAnalyticsJob
485489

486490
Assert-NotNull {Get-AzureRmDataLakeAnalyticsJob -AccountName $accountName}
487491

488-
$jobsWithDateOffset = Get-AzureRmDataLakeAnalyticsJob -AccountName $accountName -SubmittedAfter $([DateTimeOffset]($nowTime).AddMinutes(-5))
492+
$jobsWithDateOffset = Get-AzureRmDataLakeAnalyticsJob -AccountName $accountName -SubmittedAfter $([DateTimeOffset]($nowTime).AddMinutes(-10))
493+
494+
Assert-True {$jobsWithDateOffset.Count -gt 0} "Failed to retrieve jobs submitted after ten miuntes ago"
489495

490-
Assert-True {$jobsWithDateOffset.Count -gt 0} "Failed to retrieve jobs submitted after five miuntes ago"
491-
# we add five minutes to ensure that the timing is right, since we are using the account creation time, and not truly "now"
492-
$jobsWithDateOffset = Get-AzureRmDataLakeAnalyticsJob -AccountName $accountName -SubmittedBefore $([DateTimeOffset]($nowTime).AddMinutes(5))
496+
# we add ten minutes to ensure that the timing is right, since we are using the account creation time, and not truly "now"
497+
$jobsWithDateOffset = Get-AzureRmDataLakeAnalyticsJob -AccountName $accountName -SubmittedBefore $([DateTimeOffset]($nowTime).AddMinutes(10))
493498

494499
Assert-True {$jobsWithDateOffset.Count -gt 0} "Failed to retrieve jobs submitted before right now"
495500

@@ -553,6 +558,10 @@ function Test-DataLakeAnalyticsJobRelationships
553558
Assert-False {$i -eq 60} "dataLakeAnalytics accounts not in succeeded state even after 30 min."
554559
}
555560

561+
# Wait for 5 minutes for the server to restore the account cache
562+
# Without this, the test will pass non-deterministically
563+
[Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::Wait(300000)
564+
556565
# submit a job
557566
$guidForJob = [Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::GenerateGuid("relationTest01")
558567

@@ -815,6 +824,10 @@ function Test-DataLakeAnalyticsCatalog
815824
[Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::Wait(30000)
816825
Assert-False {$i -eq 60} "dataLakeAnalytics accounts not in succeeded state even after 30 min."
817826
}
827+
828+
# Wait for 5 minutes for the server to restore the account cache
829+
# Without this, the test will pass non-deterministically
830+
[Microsoft.Rest.ClientRuntime.Azure.TestFramework.TestUtilities]::Wait(300000)
818831

819832
# Run a job to create the catalog items (except secret and credential)
820833
$scriptTemplate = @"

0 commit comments

Comments
 (0)