Add SecurityTools (CredScan, BinSkim, PoliCheck) to PR builds in ADO #9035

MiYanni · 2019-04-18T22:50:52Z

Description

This allows our PR build process to run CredScan, BinSkim, and PoliCheck. This only happens in the Windows PowerShell pipeline, since we only need to run it once and it can only run on Windows. I also removed the travis yml since we shouldn't need to use Travis anymore.

Checklist

I have read the Submitting Changes section of CONTRIBUTING.md
The title of the PR is clear and informative
The appropriate change log has been updated
The PR does not introduce breaking changes
If applicable, the changes made in the PR have proper test coverage
For public API changes to cmdlets:
- the changes have gone through a cmdlet design review (Microsoft internal partners only)
- the cmdlet markdown files were generated using the platyPS module

…over CodeSign, since CodeSign should be specific for signing. While, those processes were actual specific for configuration.

…urityTools.

…te or paths that no longer exist.

…to Windows PowerShell pipeline.

…use local feed.

markcowl · 2019-04-22T17:54:21Z

tools/SecurityToolsScripts/RunBinSkim.ps1

+$repoRoot = "$PSScriptRoot/../.."
+$binSkim = (Get-Item -Path "$repoRoot/tools/SecurityTools/Microsoft.CodeAnalysis.BinSkim*/tools/*/win-x64/BinSkim.exe" | Select-Object -First 1).FullName
+$dllBaseFolder = "$repoRoot/artifacts/Debug"
+$suppressions = @('Microsoft.Azure.DataLake.Store.dll','Microsoft.WindowsAzure.Storage.DataMovement.PowerShell.dll')


Do these suppressions come from failures or from previous suppression?

These are new failures. It seems these are signed with Sha1+Sha2. So, to avoid these suppressions, we would need resigned DLLs. I did, however, want to have a suppression strategy for BinSkim since it natively doesn't allow it.

azure-pipelines · 2019-04-26T02:23:27Z

Azure Pipelines successfully started running 2 pipeline(s).

MiYanni · 2019-04-26T02:24:42Z

/azp run azure-powershell - security-tools

azure-pipelines · 2019-04-26T02:24:47Z

For the Azure DevOps organization azure-sdk, no matching pipelines using the Azure Pipelines app were found for this pull request.

MiYanni · 2019-04-26T02:25:26Z

/azp run "azure-powershell - security-tools"

azure-pipelines · 2019-04-26T02:25:32Z

For the Azure DevOps organization azure-sdk, no matching pipelines using the Azure Pipelines app were found for this pull request.

MiYanni · 2019-04-26T02:40:34Z

/azp help

azure-pipelines · 2019-04-26T02:40:35Z

Supported commands
     help:
          Get descriptions, examples and documentation about supported commands
          Example: help "command_name"
     run:
          Run all pipelines or a specific pipeline for this repository using a comment. Use
          this command by itself to trigger all related pipelines, or specify a pipeline
          to run.
          Example: "run" or "run pipeline_name"

See additional documentation.

MiYanni · 2019-04-26T02:41:14Z

/azp "run azure-powershell - security-tools"

azure-pipelines · 2019-04-26T02:41:15Z

Command '"run' is not supported by Azure Pipelines.

Supported commands
     help:
          Get descriptions, examples and documentation about supported commands
          Example: help "command_name"
     run:
          Run all pipelines or a specific pipeline for this repository using a comment. Use
          this command by itself to trigger all related pipelines, or specify a pipeline
          to run.
          Example: "run" or "run pipeline_name"

See additional documentation.

MiYanni · 2019-04-26T03:09:36Z

/azp run azure-powershell - powershell-core

azure-pipelines · 2019-04-26T03:09:50Z

Azure Pipelines successfully started running 1 pipeline(s).

… SecurityTools.

MiYanni · 2019-04-27T00:04:35Z

Build: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=23012

MiYanni · 2019-04-27T00:28:39Z

src/HDInsight/HDInsight.Test/UnitTests/DataLakeStoreTests.cs

@@ -29,9 +29,6 @@ namespace Commands.HDInsight.Test.UnitTests
    public class DataLakeStoreTests : HDInsightTestBase
    {
        private NewAzureHDInsightClusterCommand cmdlet;
-        private const string StorageName = "dummystorage.blob.core.windows.net";
-        private const string StorageKey = "O9EQvp3A3AjXq/W27rst1GQfLllhp01qlJMJfSU1hVW2K42gUeiUUn2D8zX2lU3taiXSSfqkZlcPv+nQcYYwUx==";
-        private const int ClusterSize = 4;


These variables were unused in this class and were flagged by CredScan. Made more sense to remove unused creds than to suppress them.

MiYanni · 2019-04-27T00:29:52Z

tools/PolicheckSuppressions.json

@@ -1,3 +0,0 @@
-{


This was an unused example suppressions file.

MiYanni · 2019-04-27T00:54:48Z

Build: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=23055

MiYanni · 2019-04-30T23:30:59Z

Manual build using new service connection: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=24263

MiYanni added 22 commits April 9, 2019 12:43

Added release build job.

4894222

Initial attempt at release process.

2a94ba7

Update release pipeline to (hopefully) work.

e910017

Merge branch 'master' into ado-v2-release

7a862e7

Changed condition for certain processes to use Release configuration …

9142477

…over CodeSign, since CodeSign should be specific for signing. While, those processes were actual specific for configuration.

Space fix

f8d1372

Removed release yaml since it is not used to do releases.

246c04f

Merge branch 'master' into ado-v2-release

c7531d2

Added package config for getting security tools.

8cd180d

Merge branch 'master' into ado-v2-release

ab54daf

Merge branch 'ado-v2-release' into securitytools-changes

3112aa7

Updated packages config for SecurityTools. Added nuget config for Sec…

1b12d25

…urityTools.

Apparently, the names need to be in a specific format.

3d14471

Added PoliCheck list of file extensions.

5a6158f

Added CredScan suppressions for non-test files.

7db9822

Trying back slashes for suppressions.

eba02f3

Fixed suppressions files to have correct paths and removal of duplica…

ef9d894

…te or paths that no longer exist.

Added verified suppressions (or adjusted code) for CredScan.

f6bc4b1

Added missing suppression.

e188450

Added scripts for running security tools. Added security tools tasks …

a67dc44

…to Windows PowerShell pipeline.

Removed travis yml.

b61428f

Change to retrigger CI.

a991036

azuresdkci added the needs-review label Apr 18, 2019

MiYanni added 3 commits April 18, 2019 16:52

Cannot download the SecurityTools from the nuget feed. Attempting to …

c2857c1

…use local feed.

Changed package info to correct GUIDs.

89405cd

Changed feed for accessing tools.

cbe183a

markcowl added needs-revision and removed needs-review labels Apr 22, 2019

markcowl requested changes Apr 22, 2019

View reviewed changes

markcowl assigned MiYanni Apr 22, 2019

Test file to retrigger CI

e728ea6

MiYanni added 4 commits April 26, 2019 13:06

Changed folder from SecurityToolsScripts to SecurityTools.

91711cc

Removed legacy security tools scripts. Moved security tools assets to…

adc4506

… SecurityTools.

Removed unused suppressions file. Additional suppression for BinSkim.

7a97c79

Updated BinSkim script path.

3f22cf5

MiYanni marked this pull request as ready for review April 26, 2019 22:55

MiYanni added needs-review and removed needs-revision labels Apr 26, 2019

MiYanni removed their assignment Apr 26, 2019

MiYanni added 2 commits April 26, 2019 15:58

Merge branch 'master' into securitytools-changes

299d85d

Added suppression for generated test resource group.

54dfed1

MiYanni commented Apr 27, 2019

View reviewed changes

tools/PolicheckSuppressions.json

@@ -1,3 +0,0 @@

{

Copy link

Contributor Author

MiYanni Apr 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an unused example suppressions file.

Removed temp file from CI testing.

b2bf167

markcowl approved these changes May 1, 2019

View reviewed changes

markcowl merged commit 6350c76 into Azure:master May 1, 2019

Add SecurityTools (CredScan, BinSkim, PoliCheck) to PR builds in ADO #9035

Add SecurityTools (CredScan, BinSkim, PoliCheck) to PR builds in ADO #9035

Uh oh!

Conversation

MiYanni commented Apr 18, 2019 • edited by msimecek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

markcowl Apr 22, 2019

Choose a reason for hiding this comment

Uh oh!

MiYanni Apr 22, 2019

Choose a reason for hiding this comment

Uh oh!

azure-pipelines bot commented Apr 26, 2019

Uh oh!

MiYanni commented Apr 26, 2019

Uh oh!

azure-pipelines bot commented Apr 26, 2019

Uh oh!

MiYanni commented Apr 26, 2019

Uh oh!

azure-pipelines bot commented Apr 26, 2019

Uh oh!

MiYanni commented Apr 26, 2019

Uh oh!

azure-pipelines bot commented Apr 26, 2019

Uh oh!

MiYanni commented Apr 26, 2019

Uh oh!

azure-pipelines bot commented Apr 26, 2019

Uh oh!

MiYanni commented Apr 26, 2019

Uh oh!

azure-pipelines bot commented Apr 26, 2019

Uh oh!

MiYanni commented Apr 27, 2019

Uh oh!

MiYanni Apr 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MiYanni Apr 27, 2019

Choose a reason for hiding this comment

Uh oh!

MiYanni commented Apr 27, 2019

Uh oh!

MiYanni commented Apr 30, 2019

Uh oh!

Uh oh!

MiYanni commented Apr 18, 2019 •

edited by msimecek

Loading

MiYanni Apr 27, 2019 •

edited

Loading