PowerShell/test/powershell
Andrew Schwartzmeyer 7e65fa448e Mark as pending Travis CI OS X failing tests
These tests *do not* fail locally, nor on the Linux test runner. They do
not fail when accessing the OS X runner under debug mode. The *only*
cause is the lack of a TTY on the OS X runner, which is a Travis CI
regression. Moreover, the formatting tests do not fail when the TTY is
removed locally.

These absolutely should be fixed at some point, but it is not worth
spending any more time on it.
2016-05-17 13:28:44 -07:00
..
assets New pester test for Import-LocalizedData 2016-02-11 10:13:56 -08:00
Add-Content.Tests.ps1 Incorporating feedback from code-review 2016-05-04 17:09:01 -07:00
Add-Member.Tests.ps1 Fix the issues of CR for Add-Member 2016-04-22 01:44:22 -07:00
Add-Type.Tests.ps1 Make Add-Type work without a flat folder structure 2016-04-08 21:41:30 -07:00
Clear-Content.Tests.ps1 Incorporating feedback from code-review 2016-05-04 17:09:01 -07:00
Clear-EventLog.Tests.ps1 removing debug code 2016-05-04 17:10:16 -07:00
Clear-Variable.Tests.ps1 Whitespace cleanup 2016-04-20 14:09:16 -07:00
CmdletExample.Tests.ps1 Add tests for cmdlet example 2016-05-04 12:13:00 -07:00
Compare-Object.Tests.ps1 Fix tests broken by removal of aliases 2016-04-08 13:36:54 -07:00
ComparisonOperator.Tests.ps1 Add comparison operator tests for #219 2016-03-22 01:17:22 -07:00
ConsoleHost.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
ConvertFrom-Csv.Tests.ps1 Fixed CR issues 2016-05-12 21:46:34 -07:00
ConvertFrom-StringData.Tests.ps1 Add Unit Test for ConvertFrom-StringData 2016-05-06 04:09:57 -07:00
ConvertTo-Csv.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
ConvertTo-SecureString.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Default-Aliases.Tests.ps1 Add tests for removal of aliases 2016-04-08 15:52:37 -07:00
DotNetAPI.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
DotNetInterop.Tests.ps1 Add regression test .NET class interop 2016-03-22 21:41:46 -07:00
Environment-Variables.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
ExecutionPolicy.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Export-Alias.Tests.ps1 Fix tests broken by removal of aliases 2016-04-08 13:36:54 -07:00
Export-Csv.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Export-FormatData.Tests.ps1 Fixed CR issues 2016-05-12 21:51:55 -07:00
Format-Custom.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
Format-List.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
Format-Table.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
Format-Wide.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
Get-Alias.Tests.ps1 Fix Pester tests 2016-04-25 15:27:24 -07:00
Get-ChildItem.Tests.ps1 Fix tests broken by removal of aliases 2016-04-08 13:36:54 -07:00
Get-Content.Tests.ps1 Incorporating feedback from code-review 2016-05-04 17:09:01 -07:00
Get-Culture.Tests.ps1 Fix the issue of CR fot Get-Culture 2016-04-21 23:46:21 -07:00
Get-Date.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
Get-Event.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-EventLog.Tests.ps1 Incorporating feedback from code-review 2016-05-04 17:09:01 -07:00
Get-EventSubscriber.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-FileHash.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-FormatData.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-Host.Tests.ps1 Add Unit Test for Get-Host 2016-04-20 02:16:36 -07:00
Get-Item.Tests.ps1 Fix tests broken by removal of aliases 2016-04-08 13:36:54 -07:00
Get-ItemProperty.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-Location.Tests.ps1 Fixed Pester Get-Location test 2016-03-08 14:55:30 +08:00
Get-Member.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-Process.Tests.ps1 Fix Pester tests 2016-04-25 15:27:24 -07:00
Get-PSBreakpoint.Tests.ps1 Modify Pester Test for Get-PSBreakpoint 2016-03-20 22:04:08 -07:00
Get-PSDrive.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-PSProvider.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-Random.Tests.ps1 Fix the CR issues for Get-Random 2016-05-05 14:13:55 -07:00
Get-RunspaceDebug.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-UICulture.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Get-Unique.Tests.ps1 Fix the CR issue for Get-Unique 2016-05-05 14:14:39 -07:00
Get-Variable.Tests.ps1 Update Get-Variable Pester Test 2016-04-22 02:02:22 -07:00
Group-Object.Tests.ps1 Add Unit Test for Group-Object 2016-05-07 02:24:08 -07:00
Import-Alias.Tests.ps1 Fix Import-Alias tests 2016-05-17 13:28:44 -07:00
Import-Csv.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Import-LocalizedData.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Import-Module.Tests.ps1 Fix tests for removal of Platform module 2016-04-08 15:13:01 -07:00
Invoke-Expression.Tests.ps1 Add Invoke-Expression Pester Unit Test 2016-04-20 21:25:16 -07:00
Join-Path.Tests.ps1 Incorporating feedback from code-review 2016-05-04 17:09:01 -07:00
Json.Tests.ps1 Don't load Netwonsoft.Json from path 2016-04-01 22:35:00 -07:00
Measure-Command.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Measure-Object.Tests.ps1 Fix Pester tests 2016-04-25 15:27:24 -07:00
NativeLinuxCommands.Tests.ps1 Disable broken OS X Pester tests 2016-04-14 10:05:51 -07:00
New-Alias.Tests.ps1 Fix Pester tests 2016-04-25 15:27:24 -07:00
New-Event.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
New-EventLog.Tests.ps1 Incorporating feedback from code-review 2016-05-04 17:09:01 -07:00
New-Item.Tests.ps1 Allow creation of symlink to non-existent target 2016-04-09 18:45:23 -07:00
New-Object.Tests.ps1 Modify Pester test for New-Object 2016-03-24 20:20:06 -07:00
New-TimeSpan.Tests.ps1 Add Unit Test for New-TimeSpan 2016-05-08 21:45:04 -07:00
New-Variable.Tests.ps1 Update fixes based on comments for New Variable Pester Unit Test 2016-04-19 16:00:39 -07:00
Out-File.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
Out-String.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
Pop-Location.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
PSReadLine.Tests.ps1 Add PSReadLine regression tests 2016-05-04 16:27:51 -07:00
PSVersionTable.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Push-Location.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
README.md Update Pester tests readme 2016-04-21 15:50:30 -07:00
Register-EngineEvent.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Register-ObjectEvent.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Remove-Event.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Remove-EventLog.Tests.ps1 Incorporating feedback from code-review 2016-05-04 17:09:01 -07:00
Remove-Item.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Remove-Module.Tests.ps1 Fix tests for removal of Platform module 2016-04-08 15:13:01 -07:00
Remove-PSBreakpoint.Tests.ps1 Fix Pester tests 2016-04-25 15:27:24 -07:00
Remove-TypeData.Tests.ps1 Fixed CR issues for Remove-TypeData 2016-05-11 19:46:56 -07:00
Remove-Variable.Tests.ps1 Add Remove-Variable Pester Test 2016-04-24 22:33:29 -07:00
Select-Object.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Select-String.Tests.ps1 Fix Pester test 2016-04-08 15:23:38 -07:00
Set-Alias.Tests.ps1 Add New-Alias Pester and update Set-Alias Pester test 2016-04-06 01:24:00 -07:00
Set-Content.Tests.ps1 Incorporating feedback from code-review 2016-05-04 17:09:01 -07:00
Set-Location.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Set-PSBreakpoint.Tests.ps1 Mark as pending tests not working with ConsoleHost 2016-05-17 13:28:44 -07:00
Set-PSDebug.Tests.ps1 Disable Set-PSDebug tests on AppVeyor 2016-05-17 13:28:44 -07:00
Set-Variable.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
Sort-Object.Tests.ps1 Add Pester Test for Get-PSBreakpoint, Import-Module, Sort-Object 2016-03-18 02:30:26 -07:00
Split-Path.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Start-Process.Tests.ps1 Disable broken OS X Pester tests 2016-04-14 10:05:51 -07:00
Start-Sleep.Tests.ps1 Fix tests broken by removal of aliases 2016-04-08 13:36:54 -07:00
Tee-Object.Tests.ps1 Fix tests broken by removal of aliases 2016-04-08 13:36:54 -07:00
Test-Mocks.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Test-Path.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Update-FormatData.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Update-TypeData.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Wait-Event.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00
Write-Error.Tests.ps1 Use Pending for tests that shouldn't be Skipped 2016-05-17 13:28:44 -07:00
Write-Output.Tests.ps1 Remove alias tests for Write-Output 2016-04-11 10:46:39 -07:00
Write-Progress.Tests.ps1 Mark as pending tests not working with ConsoleHost 2016-05-17 13:28:44 -07:00
Write-Stream.Tests.ps1 Mark as pending Travis CI OS X failing tests 2016-05-17 13:28:44 -07:00
Write-Verbose.Tests.ps1 Programmatic indentation and whitespace-cleanup of tests 2016-03-04 14:52:27 -08:00

Pester Testing Test Guide

Running Pester Tests

Go to the top level of the PowerShell repository and run: Start-PSPester inside a self-hosted copy of PowerShell.

You can use Start-PSPester -Tests SomeTestSuite* to limit the tests run.

Testing new powershell processes

Any launch of a new powershell process must include -noprofile so that modified user and system profiles do not causes tests to fail. You also must take care to call the development copy of PowerShell, which is not the first one on the path.

Example:

    $powershell = Join-Path -Path $PsHome -ChildPath "powershell"
    & $powershell -noprofile -command "ExampleCommand" | Should Be "ExampleOutput"

Portability

Some tests simply must be tied to certain platforms. Use Pester's -Skip directive on an It statement to do this. For instance to run the test only on Windows:

It "Should do something on Windows" -Skip:($IsLinux -Or $IsOSX) { ... }

Or only on Linux and OS X:

It "Should do something on Linux" -Skip:$IsWindows { ... }

Pending

When writing a test that should pass, but does not, please do not skip or delete the test, but use It "Should Pass" -Pending to mark the test as pending, and file an issue on GitHub.

Who this is for

Cmdlet behavior is validated using the Pester testing framework. The purpose of this document is to create a single standard to maximize unit test coverage while minimizing confusion on expectations. What follows is a working document intended to guide those writing Pester unit tests for PowerShell.

Unit testing is done not only to validate that the block of code works as expected, but also to assist the developer to know precisely where in the code to look; in some cases, seeing the source code may inspire better unit tests. In many cases, a unit test is the only documented specification. Fortunately, the MSDN is a great source of information about Cmdlets.

Test suites need to be created and many cmdlets added and unit-tested. The following list is to be used to guide the thought process of the developer in writing a suite in minimal time, while enhancing quality.

Test suites should proceed as functional and system tests of the cmdlets, and the code treated as a black box for the purpose of test suite design.

Testing Standards

Readability

Every effort should be made to maximize readability of code. Code is written for the developer in the future to debug- not for the developer writing the code.

  1. When assertions are on consecutive lines, the pipes should line up:
MyFirstCondition | Should Be 0
MySecondCondition | Should Be 1

This is less readable than:

MyFirstCondition  | Should Be 0
MySecondCondition | Should Be 1

So the second section of code should instead be used. The same style should be followed for assignments of variables on consecutive lines:

$var1 = <expression 1>
$variable2 = <expression 2>
$var3 = <expression 3>
$typeCollection1 = <expression 4>
$object1 = <expression>
... etc

is much less readable than

$var1            = <expression 1>
$variable2       = <expression 2>
$var3            = <expression 3>
$typeCollection1 = <expression 4>
$object1         = <expression 5>
... etc

So all assignment statements must be aligned.

Other style standards are no less important to readability of the code:

  • Use readable and meaningful variable name when assigning variables.

  • Do not make large functions. Tests should be simple: define -> manipulate -> assert

  • Do not use tabs. Tabs are rendered differently depending upon the machine. This greatly affects readability.

  • Remove the first 3 auto-generated lines of each .Tests.ps1 file. This is created automatically by Pester and is unnecessary. Each .Test.ps1 file should begin with a Describe block.

  • Discard the auto-generated function file that is generated in tandem with the .Tests.ps1 file

  • Name the test file "Test- when you create a new test fixture.

  • Each test describes a behavior- use the word "Should" at the beginning of each test description- so it reads "It 'Should..."