Fix broken urls (#8653)

Fix broken URL - Also, move other static analysis tests into that CI. - Also, make the link analysis a pester test (partly to make sure the step fails in case of an error)
2019-01-15 16:20:45 -08:00 · 2019-01-15 16:20:45 -08:00 · 7bf4641444
parent d61013e94d
commit 7bf4641444
15 changed files with 196 additions and 78 deletions
--- a/.vsts-ci/misc-analysis.yml
+++ b/.vsts-ci/misc-analysis.yml
@ -30,23 +30,33 @@ phases:
      Get-ChildItem -Path env:
    displayName: Capture environment
    condition: succeededOrFailed()
+
+  - powershell:  Install-module pester -Scope CurrentUser -Force -SkipPublisherCheck
+    displayName: Install Pester
+    condition: succeededOrFailed()
+
  - bash: |
      sudo npm install -g markdown-spellcheck@0.11.0
    displayName: Install mdspell
    condition: succeededOrFailed()
-  - bash: |
-      sudo npm install -g markdown-link-check@3.7.2
-    # WARNING: Keep markdown-link-check pinned at 3.7.2 OR ELSE...
-    displayName: Install markdown-link-check
-    condition: succeededOrFailed()
+
  - powershell:  Write-Host "##vso[build.updatebuildnumber]$env:BUILD_SOURCEBRANCHNAME-$env:BUILD_SOURCEVERSION-$((get-date).ToString("yyyyMMddhhmmss"))"
    displayName: Set Build Name for Non-PR
    condition: ne(variables['Build.Reason'], 'PullRequest')
+
  - bash: |
      mdspell '**/*.md' '!**/Pester/**/*.md' --ignore-numbers --ignore-acronyms --report --en-us;
    displayName: Test Spelling
    condition: succeededOrFailed()
-  - bash: |
-      find . -name \*.md -exec markdown-link-check {} \;
-    displayName: Test Links
+
+  - powershell: |
+      Import-module ./build.psm1
+      $path = Join-Path -Path $pwd -ChildPath './commonTestResults.xml'
+      $results = invoke-pester -Script ./test/common -OutputFile $path -OutputFormat NUnitXml -PassThru
+      Write-Host "##vso[results.publish type=NUnit;mergeResults=true;runTitle=Common Tests;publishRunAttachments=true;resultFiles=$path;]"
+      if($results.TotalCount -eq 0 -or $results.FailedCount -gt 0)
+      {
+        throw "markdown link tests failed"
+      }
+    displayName: Run Common Tests
    condition: succeededOrFailed()
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -1826,7 +1826,7 @@ Many modules and cmdlets that didn't work in the past may now work on .NET Core,

 If you want to opt-out of this telemetry, simply delete `$PSHome\DELETE_ME_TO_DISABLE_CONSOLEHOST_TELEMETRY`.
 Even before the first run of Powershell, deleting this file will bypass all telemetry.
-In the future, we plan on also enabling a configuration value for whatever is approved as part of [RFC0015](https://github.com/PowerShell/PowerShell-RFC/blob/master/1-Draft/RFC0015-PowerShell-StartupConfig.md).
+In the future, we plan on also enabling a configuration value for whatever is approved as part of [RFC0015](https://github.com/PowerShell/PowerShell-RFC/blob/master/X-Rejected/RFC0015-PowerShell-StartupConfig.md).
 We also plan on exposing this telemetry data (as well as whatever insights we leverage from the telemetry) in [our community dashboard](https://blogs.msdn.microsoft.com/powershell/2017/01/31/powershell-open-source-community-dashboard/).

 If you have any questions or comments about our telemetry, please file an issue.
--- a/README.md
+++ b/README.md
@ -79,12 +79,12 @@ You can also download the PowerShell binary archives for Windows, macOS and Linu

 [pv-windows-64]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/PowerShell-6.2.0-preview.3-win-x64.msi
 [pv-windows-86]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/PowerShell-6.2.0-preview.3-win-x86.msi
-[pv-ubuntu18]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell_6.2.0-preview.3-1.ubuntu.18.04_amd64.deb
-[pv-ubuntu16]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell_6.2.0-preview.3-1.ubuntu.16.04_amd64.deb
-[pv-ubuntu14]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell_6.2.0-preview.3-1.ubuntu.14.04_amd64.deb
-[pv-debian8]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell_6.2.0-preview.3-1.debian.8_amd64.deb
-[pv-debian9]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell_6.2.0-preview.3-1.debian.9_amd64.deb
-[pv-centos]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell-6.2.0-preview.3-1.rhel.7.x86_64.rpm
+[pv-ubuntu18]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell-preview_6.2.0-preview.3-1.ubuntu.18.04_amd64.deb
+[pv-ubuntu16]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell-preview_6.2.0-preview.3-1.ubuntu.16.04_amd64.deb
+[pv-ubuntu14]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell-preview_6.2.0-preview.3-1.ubuntu.14.04_amd64.deb
+[pv-debian8]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell-preview_6.2.0-preview.3-1.debian.8_amd64.deb
+[pv-debian9]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell-preview_6.2.0-preview.3-1.debian.9_amd64.deb
+[pv-centos]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell-preview-6.2.0_preview.3-1.rhel.7.x86_64.rpm
 [pv-macos]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/powershell-6.2.0-preview.3-osx-x64.pkg
 [pv-winarm]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/PowerShell-6.2.0-preview.3-win-arm32.zip
 [pv-winarm64]: https://github.com/PowerShell/PowerShell/releases/download/v6.2.0-preview.3/PowerShell-6.2.0-preview.3-win-arm64.zip
--- a/build.psm1
+++ b/build.psm1
@ -866,7 +866,7 @@ function Start-PSPester {
    [CmdletBinding(DefaultParameterSetName='default')]
    param(
        [Parameter(Position=0)]
-        [string[]]$Path = @("$PSScriptRoot/test/common","$PSScriptRoot/test/powershell"),
+        [string[]]$Path = @("$PSScriptRoot/test/powershell"),
        [string]$OutputFormat = "NUnitXml",
        [string]$OutputFile = "pester-tests.xml",
        [string[]]$ExcludeTag = 'Slow',
@ -884,6 +884,7 @@ function Start-PSPester {
        [Parameter(ParameterSetName='PassThru',HelpMessage='Run commands on Linux with sudo.')]
        [switch]$Sudo,
        [switch]$IncludeFailingTest,
+        [switch]$IncludeCommonTests,
        [string]$ExperimentalFeatureName,
        [Parameter(HelpMessage='Title to publish the results as.')]
        [string]$Title = 'PowerShell Core Tests'
@ -899,6 +900,11 @@ function Start-PSPester {
        $Path += "$PSScriptRoot/tools/failingTests"
    }

+    if($IncludeCommonTests.IsPresent)
+    {
+        $path = += "$PSScriptRoot/test/common"
+    }
+
    # we need to do few checks and if user didn't provide $ExcludeTag explicitly, we should alternate the default
    if ($Unelevate)
    {
--- a/docs/dev-process/breaking-change-contract.md
+++ b/docs/dev-process/breaking-change-contract.md
@ -31,7 +31,7 @@ Any change that is a clear violation of the public contract.
 + Any existing behavior that results in an error message generally may be changed to provide new functionality.
 + A new instance field is added to a type (this impacts .NET serialization but not PowerShell serialization and so is considered acceptable.)
 + Adding new types, new type members and new cmdlets
-+ Making changes to the protocols with a protocol version increment. Older versions of the protocol would still need to be maintained to allow communication with earlier systems. This would require that protocol negotiation take place between the two systems. In addition to any protocol code changes, the Microsoft Open Specification program requires that the formal protocol specification for a protocol be updated in a timely manner.  An example of a MS protocol specification document (MS-PSRP) can be found at: https://msdn.microsoft.com/library/dd357801.aspx
+ Making changes to the protocols with a protocol version increment. Older versions of the protocol would still need to be maintained to allow communication with earlier systems. This would require that protocol negotiation take place between the two systems. In addition to any protocol code changes, the Microsoft Open Specification program requires that the formal protocol specification for a protocol be updated in a timely manner.  An example of a MS protocol specification document (MS-PSRP) can be found at: https://msdn.microsoft.com/library/mt242417.aspx

 ## Bucket 2: Reasonable Grey Area

--- a/docs/learning-powershell/debugging-from-commandline.md
+++ b/docs/learning-powershell/debugging-from-commandline.md
@ -1,7 +1,7 @@
 Debugging in PowerShell Command-line
 =====

-As we know, we can debug PowerShell code via GUI tools like [Visual Studio Code](./using-vscode.md#debugging-with-vs-code). In addition, we can directly perform debugging within the PowerShell command-line session by using the PowerShell debugger cmdlets. This document demonstrates how to use the cmdlets for the PowerShell command-line debugging. We will cover the following topics: setting a debug breakpoint on a line of code and on a variable.
+As we know, we can debug PowerShell code via GUI tools like [Visual Studio Code](https://docs.microsoft.com/en-us/powershell/scripting/components/vscode/using-vscode?view=powershell-6#debugging-with-visual-studio-code). In addition, we can directly perform debugging within the PowerShell command-line session by using the PowerShell debugger cmdlets. This document demonstrates how to use the cmdlets for the PowerShell command-line debugging. We will cover the following topics: setting a debug breakpoint on a line of code and on a variable.

 Let's use the following code snippet as our sample script.

--- a/docs/learning-powershell/powershell-beginners-guide.md
+++ b/docs/learning-powershell/powershell-beginners-guide.md
@ -302,7 +302,7 @@ For more details, see [Create and Run PowerShell Script Guide][create-run-script
 ## Recommended Training and Reading

 - Video: [Get Started with PowerShell][remoting] from Channel9
- [eBooks from PowerShell.org](https://powershell.org/ebooks/)
+- [eBooks from PowerShell.org](https://leanpub.com/u/devopscollective)
 - [eBooks from PowerShell.com][ebooks-powershell.com]
 - [eBooks List][ebook-list] by Martin Schvartzman
 - [Tutorial from MVP][tutorial]
@ -322,9 +322,9 @@ For more details, see [Create and Run PowerShell Script Guide][create-run-script
 - [Windows PowerShell in Action][in-action] by Bruce Payette
 - [Windows PowerShell Cookbook][cookbook] by Lee Holmes

-[in-action]: https://www.amazon.com/Windows-PowerShell-Action-Second-Payette/dp/1935182137
+[in-action]: https://www.amazon.com/Windows-PowerShell-Action-Bruce-Payette/dp/1633430294
 [cookbook]: http://shop.oreilly.com/product/9780596801519.do
-[ebook-list]: https://blogs.technet.microsoft.com/pstips/2014/05/26/free-powershell-ebooks/
+[ebook-list]: https://martin77s.wordpress.com/2014/05/26/free-powershell-ebooks/
 [ebooks-powershell.com]: http://powershell.com/cs/blogs/ebookv2/default.aspx
 [tutorial]: http://www.computerperformance.co.uk/powershell/index.htm
 [to-learn]:https://blogs.technet.microsoft.com/heyscriptingguy/2015/01/04/weekend-scripter-the-best-ways-to-learn-powershell/
--- a/docs/testing-guidelines/getting-code-coverage.md
+++ b/docs/testing-guidelines/getting-code-coverage.md
@ -160,11 +160,11 @@ Invoke-Item C:\temp\Coverage\index.htm
 ```

 This should open a screen in the browser like this:
-![Coverage report browser page](Images/CoverageReportTop.png)
+![Coverage report browser page](Images/CoverageReportTop.PNG)

 The main report, which is below the summary and risk hot spots, has
 a filter functionality as well (when "Enable Filtering" is clicked on):
-![Coverage report with filter on](Images/CoverageReportFilter.png)
+![Coverage report with filter on](Images/CoverageReportFilter.PNG)

 [OpenCover]: https://github.com/OpenCover/opencover
 [codecov.io]: https://codecov.io
--- a/test/common/markdown/.gitignore
+++ b/test/common/markdown/.gitignore
@ -0,0 +1 @@
+node_modules/
--- a/test/common/markdown/cleanup.ps1
+++ b/test/common/markdown/cleanup.ps1
@ -0,0 +1,15 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+Start-Process `
+-FilePath "npm" `
+-ArgumentList @('install','rimraf','-g','--silent') `
+-Wait `
+-WorkingDirectory $PSScriptRoot `
+-NoNewWindow
+Start-Process `
+-FilePath "rimraf" `
+-ArgumentList @(Join-Path -Path $PSScriptRoot -ChildPath 'node_modules') `
+-Wait `
+-WorkingDirectory $PSScriptRoot `
+-NoNewWindow
--- a/test/common/markdown/gulpfile.js
+++ b/test/common/markdown/gulpfile.js
@ -37,7 +37,7 @@ function RunTest() {
        var rootJsonFile = rootpath + "/.markdownlint.json"
        var fs = require('fs');
        fs.appendFileSync('markdownissues.txt', '--EMPTY--\r\n');
-        gulp.src(paths, { "read": false })
+        return gulp.src(paths, { "read": false })
            .pipe(through2.obj(function obj(file, enc, next) {
                markdownlint(
                    {
--- a/test/common/markdown/markdown-link.tests.ps1
+++ b/test/common/markdown/markdown-link.tests.ps1
@ -0,0 +1,111 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+Describe "Verify Markdown Links" {
+    BeforeAll {
+        # WARNING: Keep markdown-link-check pinned at 3.7.2 OR ELSE...
+        start-nativeExecution { sudo npm install -g markdown-link-check@3.7.2 }
+
+        # Cleanup jobs for reliability
+        get-job | remove-job -force
+    }
+
+    AfterAll {
+        # Cleanup jobs to leave the process the same
+        get-job | remove-job -force
+    }
+
+    $groups = Get-ChildItem -Path "$PSScriptRoot\..\..\..\*.md" -Recurse | Group-Object -Property directory
+
+    $jobs = @{}
+    # start all link verification in parallel
+    Write-Verbose -verbose "starting jobs for performance ..."
+    Foreach($group in $groups)
+    {
+        $job = start-job {
+            param([object] $group)
+            foreach($file in $group.Group)
+            {
+                $results = markdown-link-check $file 2>&1
+                Write-Output ([PSCustomObject]@{
+                    file = $file
+                    results = $results
+                })
+            }
+        } -ArgumentList @($group)
+        $jobs.add($group.name,$job)
+    }
+
+    # Get the results and verify
+    foreach($key in $jobs.keys)
+    {
+        $job = $jobs.$key
+        $results = Receive-Job -Job $job -Wait
+        Remove-job -job $Job
+        foreach($jobResult in $results)
+        {
+            $file = $jobResult.file
+            $result = $jobResult.results
+            Context "Verify links in $file" {
+                # failures look like `[✖] https://someurl` (perhaps without the https://)
+                # passes look like `[✓] https://someurl` (perhaps without the https://)
+                $failures = $result -like '*[✖]*' | ForEach-Object { $_.Substring(4) }
+                $passes = $result -like '*[✓]*' | ForEach-Object {
+                    @{url=$_.Substring(4)}
+                }
+                $trueFailures = @()
+                $verifyFailures = @()
+                foreach ($failure in $failures) {
+                    if($failure -like 'https://www.amazon.com*')
+                    {
+                        # In testing amazon links often failed when they are valid
+                        # Verify manually
+                        $verifyFailures += @{url = $failure}
+                    }
+                    else
+                    {
+                        $trueFailures += @{url = $failure}
+                    }
+                }
+
+                # must have some code in the test for it to pass
+                function noop {
+                }
+
+                if($passes)
+                {
+                    it "<url> should work" -TestCases $passes {
+                        noop
+                    }
+                }
+
+                if($trueFailures)
+                {
+                    it "<url> should work" -TestCases $trueFailures  {
+                        if($url -match '^http(s)?:')
+                        {
+                            # If invoke-WebRequest can handle the URL, re-verify, with 5 retries
+                            $null = Invoke-WebRequest -uri $url -RetryIntervalSec 2 -MaximumRetryCount 5
+                        }
+                        else {
+                            throw "Tool reported Url as unreachable"
+                        }
+                    }
+                }
+
+                if($verifyFailures)
+                {
+                    it "<url> should work" -TestCases $verifyFailures -Pending  {
+                    }
+                }
+
+                if(!$passes -and !$trueFailures -and !$verifyFailures)
+                {
+                    It "has no links" {
+                        noop
+                    }
+                }
+            }
+        }
+    }
+}
--- a/test/common/markdown/markdown.tests.ps1
+++ b/test/common/markdown/markdown.tests.ps1
@ -10,69 +10,44 @@ $repoRootPathFound = $false

 Describe 'Common Tests - Validate Markdown Files' -Tag 'CI' {
    BeforeAll {
-        # Skip if not windows, We don't need these tests to run on linux (the tests run fine in travis-ci)
-        $skip = !$IsWindows
-        if ( !$skip )
+        Push-Location $psscriptroot
+        $skip = $false
+        $NpmInstalled = "not installed"
+        if (Get-Command -Name 'npm' -ErrorAction SilentlyContinue)
        {
-            $NpmInstalled = "not installed"
-            if (Get-Command -Name 'npm' -ErrorAction SilentlyContinue)
+            $NpmInstalled = "Installed"
+            Write-Verbose -Message "NPM is checking Gulp is installed. This may take a few moments." -Verbose
+            start-nativeExecution { npm install --silent }
+            start-nativeExecution { npm install 'gulp@4.0.0' --silent }
+            if(!(Get-Command -Name 'gulp' -ErrorAction SilentlyContinue))
            {
-                $NpmInstalled = "Installed"
-                Write-Verbose -Message "NPM is checking Gulp is installed. This may take a few moments." -Verbose
-                Start-Process `
-                    -FilePath "npm" `
-                    -ArgumentList @('install','--silent') `
-                    -Wait `
-                    -WorkingDirectory $PSScriptRoot `
-                    -NoNewWindow
-                Start-Process `
-                    -FilePath "npm" `
-                    -ArgumentList @('install','-g','gulp@4.0.0','--silent') `
-                    -Wait `
-                    -WorkingDirectory $PSScriptRoot `
-                    -NoNewWindow
+                start-nativeExecution { sudo npm install -g 'gulp@4.0.0' --silent }
            }
-            elseif( -not $env:AppVeyor)
+            if(!(Get-Command -Name 'node' -ErrorAction SilentlyContinue))
            {
-                <#
-                    On Windows, but not an AppVeyor and pre-requisites are missing
-                    For now we will skip, and write a warning.  Work to resolve this is tracked in:
-                    https://github.com/PowerShell/PowerShell/issues/3429
-                #>
-                Write-Warning "Node and npm are required to run this test"
-                $skip = $true
+                throw "node not found"
            }
-
-            $mdIssuesPath = Join-Path -Path $PSScriptRoot -ChildPath "markdownissues.txt"
-            Remove-Item -Path $mdIssuesPath -Force -ErrorAction SilentlyContinue
        }
+        elseif( -not $env:AppVeyor)
+        {
+            <#
+                On Windows, but not an AppVeyor and pre-requisites are missing
+                For now we will skip, and write a warning.  Work to resolve this is tracked in:
+                https://github.com/PowerShell/PowerShell/issues/3429
+            #>
+            Write-Warning "Node and npm are required to run this test"
+            $skip = $true
+        }
+
+        $mdIssuesPath = Join-Path -Path $PSScriptRoot -ChildPath "markdownissues.txt"
+        Remove-Item -Path $mdIssuesPath -Force -ErrorAction SilentlyContinue
    }

    AfterAll {
-        if ( !$skip )
-        {
-            <#
-                NPM install all the tools needed to run this test in the test folder.
-                We will now clean these up.
-                We're using this tool to delete the node_modules folder because it gets too long
-                for PowerShell to remove.
-            #>
-            Start-Process `
-                -FilePath "npm" `
-                -ArgumentList @('install','rimraf','-g','--silent') `
-                -Wait `
-                -WorkingDirectory $PSScriptRoot `
-                -NoNewWindow
-            Start-Process `
-                -FilePath "rimraf" `
-                -ArgumentList @(Join-Path -Path $PSScriptRoot -ChildPath 'node_modules') `
-                -Wait `
-                -WorkingDirectory $PSScriptRoot `
-                -NoNewWindow
-        }
+        Pop-Location
    }

-    It "Should not have errors in any markdown files" -Skip:$skip {
+    It "Should not have errors in any markdown files" -skip:$skip {
        $NpmInstalled | should BeExactly "Installed"
        $mdErrors = 0
        Push-Location -Path $PSScriptRoot
--- a/test/tools/CodeCoverageAutomation/README.md
+++ b/test/tools/CodeCoverageAutomation/README.md
@ -13,7 +13,7 @@ The script follows the steps below:
 1. Download the code coverage binaries package from Azure DevOps Windows nightly builds artifacts (CodeCoverage.zip).
 2. Download the OpenCover powershell module from Azure DevOps Windows nightly builds artifacts. (OpenCover.zip)
 3. Download the tests from Azure DevOps Windows nightly builds artifacts (tests.zip)
-4. Download Coveralls.net from 'https://github.com/csMACnz/coveralls.net/releases/download/0.7.0/coveralls.net.0.7.0.nupkg'
+4. Download Coveralls.net from [here](https://github.com/csMACnz/coveralls.net/releases/download/0.7.0/coveralls.net.0.7.0.nupkg)
 5. Invoke 'Install-OpenCover' to install OpenCover toolset.
 6. Invoke 'Invoke-OpenCover' to execute tests.
 7. Invoke powershell to get the git commit ID of the downloaded daily build package.
--- a/test/xUnit/README.md
+++ b/test/xUnit/README.md
@ -9,7 +9,7 @@ Go to the top level of the PowerShell repository and run full set of tests:

 Go to the test project folder and run `dotnet test -c Release`.

-Use [`filter`](xunit-filter) parameter to run only needed tests:
+Use [`filter`][xunit-filter] parameter to run only needed tests:
 ```powershell
 dotnet test -c Release --filter "FullyQualifiedName~UnitTest1   # Runs tests which have UnitTest1 in FullyQualifiedName
 dotnet test --filter Name~TestMethod1   # Runs tests whose name contains TestMethod1