From 21494be11badbf787acb8bc08d45d7927053e1b6 Mon Sep 17 00:00:00 2001 From: James Truher Date: Wed, 27 Jul 2016 18:37:50 -0700 Subject: [PATCH 1/4] updates from previous PR to incorporate feedback --- docs/testing-guidelines/PesterDoAndDont.md | 28 ++-- docs/testing-guidelines/Testing.md | 106 -------------- docs/testing-guidelines/WritingPesterTests.md | 66 +++++---- docs/testing-guidelines/testing-guidelines.md | 138 +++++++++++++++++- 4 files changed, 186 insertions(+), 152 deletions(-) mode change 100644 => 100755 docs/testing-guidelines/PesterDoAndDont.md delete mode 100644 docs/testing-guidelines/Testing.md mode change 100644 => 100755 docs/testing-guidelines/WritingPesterTests.md mode change 100644 => 100755 docs/testing-guidelines/testing-guidelines.md diff --git a/docs/testing-guidelines/PesterDoAndDont.md b/docs/testing-guidelines/PesterDoAndDont.md old mode 100644 new mode 100755 index 56bcfe857..250e5b6f0 --- a/docs/testing-guidelines/PesterDoAndDont.md +++ b/docs/testing-guidelines/PesterDoAndDont.md @@ -1,23 +1,25 @@ ## Do -1. Name your files .tests.ps1 +1. Name your files .tests.ps1 2. Keep tests simple 1. Test only what you need 2. Reduce dependencies -3. Be sure to tag your Describe blocks with "inner" and "outer" -4. Make sure that Describe/Context/It descriptions are useful +3. Be sure to tag your `Describe` blocks based on their purpose + 1. Tag `CI` indicates that it will be run as part of the continuous integration process. These should be unit test like, and fast. + 2. Tag `Feature` indicates a higher level feature test (we will run these on a regular basis), for example, tests which go to remote resources, or test broader functionality + 3. Tag `Scenario` indicates tests of integration with other features (these will be run on a less regular basis and test even broader functionality than feature tests. +4. Make sure that `Describe`/`Context`/`It` descriptions are useful 1. The error message should not be the place where you describe the test -5. Use "Context" to group tests - 1. Multiple Contexts can help you group your test suite into logical sections -6. Use BeforeAll/AfterAll/BeforeEach/AfterEach instead of custom initiators -7. Use Try-Catch for expected errors and check $_.fullyQualifiedErrorId -8. Loop It blocks for checking multiple properties +5. Use `Context` to group tests + 1. Multiple `Context` blocks can help you group your test suite into logical sections +6. Use `BeforeAll`/`AfterAll`/`BeforeEach`/`AfterEach` instead of custom initiators +7. Use Try-Catch for expected errors and check $_.fullyQualifiedErrorId (don't use `should throw`) +8. Use `-testcases` when iterating over multiple `It` blocks 9. Use code coverage functionality where appropriate -10. Use Mock functionality when you don't have your entire environment -11. Avoid free code in a Describe block +10. Use `Mock` functionality when you don't have your entire environment +11. Avoid free code in a `Describe` block 1. Use `[Before|After][Each|All]` _see Free Code in a Describe block_ ## Don't 1. Have too many evaluations in a single It block - 1. The first "Should" failure will stop that block -2. Don't use "Should" anywhere but within an "It" Block - + 1. The first `Should` failure will stop that block +2. `Should` be used in an `It` Block diff --git a/docs/testing-guidelines/Testing.md b/docs/testing-guidelines/Testing.md deleted file mode 100644 index d322cee69..000000000 --- a/docs/testing-guidelines/Testing.md +++ /dev/null @@ -1,106 +0,0 @@ -# DRAFT - -_I have more questions than answers_ - - -#### Current Test Infrastructure -We currently rely heavily on STEX environment for our testing, and we will continue to do so through the Server2016 release. We -need to use that current infrastructure to continue to test full PowerShell builds; it should be possible to build automation -which takes a full PowerShell build and lay it on an existing lab system, update the appropriate test files and execute -a test pass in the same way that we do with official builds. - -The test artifacts which are applicable to full PowerShell are not universally applicable to Core/Nano/OtherPlatform, we will need -to create tooling which allows us to apply a set of test artifacts to a configuration, and then execute tests. Eventually, we need -to have our CI environment test all the flavors of PowerShell we create. -**Question**: Can AppVeyor/Travis service that need? - - -#### Organization -**Proposal**: Create 3 tiers of testing: - -* Checkin - * These are run as part of the CI process, and should run quickly. How quickly is an open question. We need to determine - the right amount of coverage without spending too much time. It may be that we can improve our coverage here through parallelization - but we have not investigated enough to determine whether it's possible. -* Feature - * the tests which look at corner cases, and stand-alone modules (for example, the archive module tests could fall into this - category) -* Scenario - * these are tests which span features, and determine whether the whole product is working correctly. The current P3 tests fall - largely here - -**Actions**: Decide what goes where. My initial thoughts are to migrate our current TTEST unittests into tier 1 (Checkin) - -**Current Migration Activity** - -We have teams working on migrating tests which are in non-portable frameworks (TTest, Lite1, Lite3, etc) to portable frameworks. -The first effort is to migrate our TTEST cmdlet unit tests to Pester, we should be taking those migrated tests and get them into -SD - -##### Layout -We need to have a reasonable layout of our tests, not sure what that looks like yet. We need to make it -easy to find both feature and test code to reduce our maintainance burden. - -##### Self Hosting -Self-Hosting remains problematic while are still so early in the development phase, but it is _imperative_ -that we dog food as early as possible. This is especially true on the non-Windows platforms where we have made -assumptions about the working environment with regard to a number of issues: -* removal of well known aliases -* case sensitivity of some operations -* coverage -We should be using these non-windows platforms as much as possible to - -======= -# DRAFT - -_I have more questions than answers_ - - -#### Current Test Infrastructure -We currently rely heavily on STEX environment for our testing, and we will continue to do so through the Server2016 release. We -need to use that current infrastructure to continue to test full PowerShell builds; it should be possible to build automation -which takes a full PowerShell build and lay it on an existing lab system, update the appropriate test files and execute -a test pass in the same way that we do with official builds. - -The test artifacts which are applicable to full PowerShell are not universally applicable to Core/Nano/OtherPlatform, we will need -to create tooling which allows us to apply a set of test artifacts to a configuration, and then execute tests. Eventually, we need -to have our CI environment test all the flavors of PowerShell we create. -**Question**: Can AppVeyor/Travis service that need? - - -#### Organization -**Proposal**: Create 3 tiers of testing: - -* Checkin - * These are run as part of the CI process, and should run quickly. How quickly is an open question. We need to determine - the right amount of coverage without spending too much time. It may be that we can improve our coverage here through parallelization - but we have not investigated enough to determine whether it's possible. -* Feature - * the tests which look at corner cases, and stand-alone modules (for example, the archive module tests could fall into this - category) -* Scenario - * these are tests which span features, and determine whether the whole product is working correctly. The current P3 tests fall - largely here - -**Actions**: Decide what goes where. My initial thoughts are to migrate our current TTEST unittests into tier 1 (Checkin) - -**Current Migration Activity** - -We have teams working on migrating tests which are in non-portable frameworks (TTest, Lite1, Lite3, etc) to portable frameworks. -The first effort is to migrate our TTEST cmdlet unit tests to Pester, we should be taking those migrated tests and get them into -SD - -##### Layout -We need to have a reasonable layout of our tests, not sure what that looks like yet. We need to make it -easy to find both feature and test code to reduce our maintainance burden. - -##### Self Hosting -Self-Hosting remains problematic while are still so early in the development phase, but it is _imperative_ -that we dog food as early as possible. This is especially true on the non-Windows platforms where we have made -assumptions about the working environment with regard to a number of issues: -* removal of well known aliases -* case sensitivity of some operations -* coverage -We should be using these non-windows platforms as much as possible to - ->>>>>>> Create Testing.md diff --git a/docs/testing-guidelines/WritingPesterTests.md b/docs/testing-guidelines/WritingPesterTests.md old mode 100644 new mode 100755 index 88c532e78..a60c3a36e --- a/docs/testing-guidelines/WritingPesterTests.md +++ b/docs/testing-guidelines/WritingPesterTests.md @@ -1,9 +1,13 @@ -Because we are planning to continue to extend Pester to support remote/parallel execution, it's important to keep in mind the following when create tests: +### Writing Pester Tests +Note that this document does not replace the documents found in the [Pester](https://github.com/pester/pester "Pester") project. This is just +some quick tips and suggestions for creating Pester tests for this project. The Pester community is vibrant and active, if you have questions +about Pester or creating tests, the [Pester Wiki](https://github.com/pester/pester/wiki) has a lot of great information. + +When creating tests, keep the following in mind: * Tests should not be overly complicated and test too many things * boil down your tests to their essence, test only what you need * Tests should be as simple as they can * Tests should generally not rely on any other test - Examples: Here's the simplest of tests @@ -27,14 +31,28 @@ Describe "One is really one" { } It "1 is really an int" { $i = 1 - $i.GetType().FullName | Should Be System.Int32 + $i.GetType() | Should Be "int" } } ``` -If you are checking for proper errors, do that in a try catch, and then check fully qualified error id +alternatively, you could do the following: +``` +Describe "One is really one" { + It "Compare 1 to 1" { + $a = 1 + $a | Should be 1 + } + It "1 is really an int" { + $i = 1 + $i.GetType() | Should Be ([System.Int32]) + } +} +``` + +If you are checking for proper errors, do that in a `try/catch`, and then check `FullyQualifiedErrorId`. Checking against `FullyQualifiedErrorId` is recommended because it does not change based on culture as an error message might. ``` ... -it "Error should be PathNotFound" { +it "Get-Item on a nonexisting file should have error PathNotFound" { try { get-item "ThisFileCannotPossiblyExist" -ErrorAction Stop @@ -42,8 +60,7 @@ it "Error should be PathNotFound" { } catch { - $_.FullyQualifiedErrorId | - should be "PathNotFound,Microsoft.PowerShell.Commands.GetItemCommand" + $_.FullyQualifiedErrorId | should be "PathNotFound,Microsoft.PowerShell.Commands.GetItemCommand" } } ``` @@ -51,12 +68,10 @@ it "Error should be PathNotFound" { Note that if get-item were to succeed, a different FullyQualifiedErrorId would be thrown and the test will fail because the FQErrorId is wrong. This is the suggested path because Pester wants to check the error message, which will likely not work here because of localized builds, but the FullyQualifiedErrorId is constant regardless of the locale. ### Describe/Context/It - -From an organizational standpoint, a Describe block roughly compares to a LITE3 Test Suite and an It block roughly compares to a LITE3 testcase. Unlike LITE3, individual tests are not prioritized (remember that a test name in Pester is really a descriptive sentence, rather than a name), but a Describe block (TestSuite) may be tagged with a string. -For creation of PowerShell tests, the Describe block is the level of granularity suggested and one of two tags should be used: "Inner" or "Outer". If the tag is not provided, tests in that describe block will be run any time tests are executed. +For creation of PowerShell tests, the Describe block is the level of granularity suggested and one of three tags should be used: "CI", "Feature", or "Scenario". If the tag is not provided, tests in that describe block will be run any time tests are executed. #### Describe -Creates a logical group of tests. All Mocks and TestDrive contents defined within a Describe block are scoped to that Describe ; they will no longer be present when the Describe block exits. A Describe block may contain any number of Context and It blocks. +Creates a logical group of tests. All Mocks and TestDrive contents defined within a Describe block are scoped to that Describe; they will no longer be present when the Describe block exits. A `Describe block may contain any number of Context and It blocks. #### Context Provides logical grouping of It blocks within a single Describe block. Any Mocks defined inside a Context are removed at the end of the Context scope, as are any files or folders added to the TestDrive during the Context block's execution. Any BeforeEach or AfterEach blocks defined inside a Context also only apply to tests within that Context . @@ -92,16 +107,7 @@ Describe "Add-Footer" { When this test completes, the contents of the TestDrive PSDrive will be removed. #### Parameter Generation -This is quite a bit different from LITE3, but still possible, see the following: - ``` -Describe "A test" { -function Test-Xor { - param ($a, $b, $ExpectedResult) - It ("Applies XOR to inputs {0} and {1}" -f $a, $b) { - $a -xor $b | Should be $ExpectedResult - } -} $testCases = @( @{ a = 0; b = 1; ExpectedResult = 1 } @@ -109,13 +115,16 @@ $testCases = @( @{ a = 1; b = 1; ExpectedResult = 0 } @{ a = 0; b = 0; ExpectedResult = 0 } ) - foreach ($testCase in $testCases) { - Test-Xor @testCase - } + +Describe "A test" { + It " -xor should be " -testcase $testcases { + param ($a, $b, $ExpectedResult) + $a -xor $b | Should be $ExpectedResult + } } ``` -You can construct the values to pass as parameters, including the expected value, and use that iteratively in a loop. Note the location of the `It` block +You can also construct loops and pass values as parameters, including the expected value, but Pester does this for you. #### Mocking Mocks the behavior of an existing command with an alternate implementation. This creates new behavior for any existing command within the scope of a Describe or Context block. The function allows you to specify a script block that will become the command's new behavior. @@ -128,7 +137,7 @@ Context "Get-Random is not random" { } } ``` -More information may be found here: https://github.com/pester/Pester/wiki/Mock +More information may be found on the [wiki](https://github.com/pester/Pester/wiki/Mock) ### Free Code in a Describe block Code execution in Pester can be very subtle and can cause issues when executing test code. The execution of code which lays outside of the usual code blocks may not happen as you expect. Consider the following: ``` @@ -138,14 +147,14 @@ Describe it { Write-Host -for DarkRed "Before BeforeAll" BeforeAll { write-host -for Blue "In Context BeforeAll" } Write-Host -for DarkRed "After BeforeAll" -  + Write-Host -for DarkRed "Before AfterAll" AfterAll { Write-Host -for Blue "In Context AfterAll" } Write-Host -for DarkRed "After AfterAll" -  + BeforeEach { Write-Host -for Blue "In BeforeEach" } AfterEach { Write-Host -for Blue "In AfterEach" } -  + Write-Host -for DarkRed "Before It" It "should not be a surprise" { 1 | should be 1 @@ -189,3 +198,4 @@ The DESCRIBE BeforeAll block is executed before any other code even though it wa Generally, you should have code reside in one of the code block elements of `[Before|After][All|Each]`, especially if those block rely on state set by free code elsewhere in the block. + diff --git a/docs/testing-guidelines/testing-guidelines.md b/docs/testing-guidelines/testing-guidelines.md old mode 100644 new mode 100755 index 85a9b17e6..0d431f921 --- a/docs/testing-guidelines/testing-guidelines.md +++ b/docs/testing-guidelines/testing-guidelines.md @@ -1,7 +1,135 @@ -author: Jim + +# Testing Guidelines + +Testing is a critical part of the PowerShell project. + +The PowerShell team created nearly 100,000 tests over the last 12 years which we run as part of the release process for PowerShell in Windows. +Having all of those tests available for the initial release of OPS was not feasible, and we have targeted those tests which +we believe will provide us the ability to catch regressions in the areas which have had the largest changes for OPS. It is our +intent to continue to release more and more of our tests until we have the coverage we need. + +For creating new tests, please review the +[documents](https://github.com/PowerShell/PowerShell/tree/master/docs/testing-guidelines) on how to +create tests for OpenPowerShell + +## CI System + +We use [AppVeyor](http://www.appveyor.com/) as a continuous integration (CI) system for Windows +and [Travis CI](http://www.travis-ci.com) for non-Windows platforms. + +### AppVeyor + +In the `README.md` at the top of the repo, you can see AppVeyor badge. +It indicates the last build status of `master` branch. +Hopefully, it's green: + +![AppVeyor-Badge-Green.png](Images/AppVeyor-Badge-Green.png) + +This badge is **clickable**; you can open corresponding build page with logs, artifacts and tests results. +From there you can easily navigate to the build history. + +### Travis CI + +Travis CI works similarly to AppVeyor. For Travis CI there will be multiple badges: +The badges indicate the last build status of `master` branch for different platforms. +Hopefully, each badge is green: + +![Travis-CI-Badge-Green.png](Images/Travis-CI-Badge-Green.png) + +This badge is **clickable**; you can open corresponding build page with logs, artifacts and tests results. +From there you can easily navigate to the build history. + +### Getting CI Results + +CI System builds (Appveyor and Travis CI) and runs tests on every pull request and provides quick feedback about it. + +![AppVeyor-Github](Images/AppVeyor-Github.png) + +These green check boxes and red crosses are **clickable** as well. +They will bring you to the corresponding page with details. + +## Test Frameworks +### Pester +Our script based test framework is [Pester](https://github.com/Pester/Pester). This is the framework which we are using internally +at Microsoft for new script based tests, and a large number of the tests which are part of the OPS project have been migrated from that test base. Pester tests can be used to test most of PowerShell behavior (even some API operations can +easily be tested in Pester) + +Substantial changes were required, to get Pester executing on Non-Windows systems. These changes are not yet in the +official Pester code base. Some features of Pester may not be available or may have incorrect behavior. Please make sure +to create issues in [PowerShell/PowerShell](https://github.com/PowerShell/PowerShell/issues) (not Pester) for anything that you find. + +### xUnit +For those tests which are not easily run via Pester, we have decided to use [xUnit](https://xunit.github.io/) as the test framework. +Currently, we have a minuscule number of tests which are run via xUnit, + +## Running Tests outside of CI +When working on new features or fixes, it is natural to want to run those tests locally before +making a PR. Two helper functions are part of the build.psm1 module to help with that: +* `Start-PSPester` will execute all Pester tests which are run by the CI system +* `Start-PSxUnit` will execute the available xUnit tests run by the CI system +Our CI system runs these as well; there should be no difference between running these on your dev system, versus in CI. + +When running tests in this way, be sure that you have started PowerShell with `-noprofile` as some tests will fail if the +environment is not the default or has any customization. + +for example, to run all the Pester tests for CI (assuming you are at the root the PowerShell repo): +``` +import-module ./build.psm1 +Start-PSPester +``` +if you wish to run specific tests, that is possible as well: +``` +Start-PSPester -Directory test/powershell/engine/Api +``` +or a specific Pester test file: +``` +Start-PSPester -Directory test/powershell/engine/Api -Test XmlAdapter.Tests.Api +``` + +### What happens after your PR? +When your PR has successfully passed the CI test gates, your changes will be used to create PowerShell binaries which can be run +in Microsoft's internal test frameworks. The tests that you created for your change and the library of historical tests will be +run to determine if any regressions are present. If these tests find regressions, you'll be notified that your PR is not ready, and provide +you enough information for you to investigate why the failure happened. -> Jason/Jim: Testing requirements (code coverage, required for incoming features/enhancements) - > local build/test execution instructions - > tools needed for build/test execution - > our CI explained + +## Test Layout +We have taken a functional approach to the layout of our Pester tests. You should place new tests in their appropriate +location. If you are making a fix to a cmdlet in a module, the test belongs in the module directory. +If you are unsure; you can make it part of your PR, or create an issue. The current layout of tests is: +* test/powershell/engine +* test/powershell/engine/Api +* test/powershell/engine/Basic +* test/powershell/engine/ETS +* test/powershell/engine/Help +* test/powershell/engine/Logging +* test/powershell/engine/Module +* test/powershell/engine/ParameterBinding +* test/powershell/engine/Runspace +* test/powershell/engine/Logging/MessageAnalyzer +* test/powershell/Host +* test/powershell/Host/ConsoleHost +* test/powershell/Host/TabCompletion +* test/powershell/Language +* test/powershell/Modules +* test/powershell/Provider +* test/powershell/Scripting +* test/powershell/Scripting/Debugging +* test/powershell/Scripting/NativeExecution +* test/powershell/SDK +* test/powershell/Security +* test/powershell/Language/Classes +* test/powershell/Language/Interop +* test/powershell/Language/Operators +* test/powershell/Language/Parser +* test/powershell/Language/Interop/DotNet +* test/powershell/Modules/Microsoft.PowerShell.Archive +* test/powershell/Modules/Microsoft.PowerShell.Core +* test/powershell/Modules/Microsoft.PowerShell.Diagnostics +* test/powershell/Modules/Microsoft.PowerShell.Management +* test/powershell/Modules/Microsoft.PowerShell.Security +* test/powershell/Modules/Microsoft.PowerShell.Utility +* test/powershell/Modules/Microsoft.PowerShell.Security +* test/powershell/Modules/PSReadLine + From 2920a3db665c6a5c09c4bd05d0c8ad1bfa20cd82 Mon Sep 17 00:00:00 2001 From: James Truher Date: Wed, 27 Jul 2016 18:43:03 -0700 Subject: [PATCH 2/4] Add images to this branch --- .../Images/AppVeyor-Badge-Green.png | Bin 0 -> 2334 bytes .../Images/AppVeyor-Github.png | Bin 0 -> 13910 bytes 2 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 docs/testing-guidelines/Images/AppVeyor-Badge-Green.png create mode 100644 docs/testing-guidelines/Images/AppVeyor-Github.png diff --git a/docs/testing-guidelines/Images/AppVeyor-Badge-Green.png b/docs/testing-guidelines/Images/AppVeyor-Badge-Green.png new file mode 100644 index 0000000000000000000000000000000000000000..777ca2c4b443fa82f5739a17c50eee5e1cf4e82d GIT binary patch literal 2334 zcmb7G`9Bkm8=twBnJdN|IitvtqsftD#1y5R<)~O$D@RtyM9Nt#rE=t);X}#R&>T6! zmMgawmTOtDq*#Rc?(_W*K0iFq`}I7p*YmvIulMtMUeA+u+RX_frYHsg03b+bhckS9 zz&BG6knhbF{Xl#Wh&Bf4uGqQS0RSXCc>AI-Ki(7W>>CLHNWA~o1ZZKE z!Th8G*3k!hHtaGs>f#L)Ap9aa6l)NQ!m61X7#WzMl8jLR00@tCu=9ux;N*hO%6da- zf53E%5P=b_s@m4+nVO*XpgGyGyHdIj6Xzj3#l8f~XmrN`fLUtSys|yojdVM6K%|Go z9kmJy4=sQun#Y-aKS63U&UmmU=|^GgmfHwKZ`6a6lV8Of`-l5t*E%lcXna%t_!l); zeU{246^8owe}f7YGk3kdwQ|<=F&KYLW!5=aweU|0#nD} z@w7W{drG1%*P?o%8OI6FOGmzu$>b0kSf`xl$(=v>vEtu`6hznw1~u}EzrRWyN@PZv z^CX-gS@KSal9O-Je4tG=1-g@j3ic8Sd)p8N0c6X`|l;tuI@wHJsp(4-7wZ1rP2?97O@M{$5>MG1y5pwQ4rAyBgZJ znBg+PZx4lQ_r>>sI!HYRN}HtRBO7NaQj%W+6$5>x6lcGS7G`l*zmGJ;#O-){ft|Rv z^v}#)3F-G>CzZK{8GK6%Jk7A6+ z9A|Y?=9HJ2E%|W3GJ_N(dVokIHUlT1*~P#2TUF9wz;oerCPB}FZ2e`n!a%&;&^FS7t8FJJvYFAfcuu?R1@{70-8{{B99cr5Azl#M?_jH zo6vuBvUVaAhyHxuxAZPn3c4QW)2$A+;v8r%Cg8%UhVCEfENNiV!>`i9iH>C<`i(t5EqoqXvh|mrKG; zqjwX2v4;rFs#a=Z$pYI4B`80;4tbK?rPbTp)pf*;c4GrUjn62L5>2Wi%C);~b+v3I z6oO5@Ce7$;{K_kb8fC$uF7r6J{q*G`IpK67O|EDSQi}99@D81KNPWJ_6QDjTYw zg%m*0U}nd&FRT_qa$g_+oiCdTHIgTy$jZ0N%nPF!xZBCkc{k6Rse3vEcK zBXNkSOG*Rz{cH7His$6dk}^|+2O*jgb~=pu>VnYohts>0CFhzU2IObGi!nR4n&zes zSTukA)+M+rUnFbVq{ZodefQyBv-P*Wj~)z>a!Ii1vXi4Ghq!}AREDn1dz2#DVc>{l zOJ}}7NS2tN3eS2cqmZj z>5l59w7ohKs>j5uVK7)T+!l%46Q3@)+?mSzjURt?qOXQiN8mNybJ$%QFd|3!@jwgw zS*;R?QoUGSsSqKw(AKZl_p|t^AR180i=&w#qrmz(lX?GY(fuvr*CO}836YmJZBg~f zf4kl>^v;~`3x}9d{@!0->VKw*ykh-q;X2hqWTGE|L~j|td}=OK5Ih+Q-jSUf)X}oJ zn&&VT;$pu(>~Uh+NCu0P%{5{BTF$&)t3sq_Lhs>bSNeFec!yFuTJY~xyV-#IB8cBqHjYkS!N`CRe(M^&bE?Im&MpV9OE3SX> zxA#KqMqI|nzcT@2Mv9fKuCbl7GAVg^dCY9!1f(u8M_@&R!C+9vm+H>xAE4ailS)V! z92_hzPDm&&K8-r`^sdi-FS~~i-IImy>&AG(_{7vtn(1lo2Ztql^*CaJ z3ga23t6ldOm3)8P9dYCrIj^@SJnvjsSeToe3j^t#Dm>}zT#RWIQlDrkn{fq8!EUna7a0G2 O0U#aS9BS+@B>oS!xoHsq literal 0 HcmV?d00001 diff --git a/docs/testing-guidelines/Images/AppVeyor-Github.png b/docs/testing-guidelines/Images/AppVeyor-Github.png new file mode 100644 index 0000000000000000000000000000000000000000..e467d28accf4e66a6eab7abdc34d9075dee9b5fc GIT binary patch literal 13910 zcmdtIWmFtr+veMZ;FbgkjRs55KqJ8k1W1BIaEHbT5TtP@(6~DU5AGgZ8g~os?jGDu z<^P;#<~?iXJ@d|*^Xc?jRMXYfReSHMz3=<_-B+Nz>}MQI5=;;Xgd-s?rT_vxK?nXH zM1KNo6=+|P0B=Yzh0h|Ok|DAk;KNgs4>BJ>pz?66J17e98N*s!9R>p7w*UP@>bA-= z1a^Y$KC0O%S{d6pLT!ye)=;x=b}ZkF>?k=|AS~~U;Os^q(7SF4u@6d4ItM9pilpkP zTU#yDp89E%c%%Z?M)lNOv#cB2f%GlYc#4`F+Wktu`QE6k9bb&F`R4AM))#gS+P!6& zVJy@%DrnVu3MMYAi>V_d@&;*s0^39*NpXGB3ONWWxlfVGIWGJ*tA7VMc5)>60AaL?LwfR#07`(W+;Hn>4SZHc)W|28tewa{H-HV@IT=rCoyYH6C!EC9xyT9kw z79au|nn9lcnQw3`ZD|pTl@zoutFM<-*Vfh^wxj^h^AsP_=u+^jPV*qSHy3 zgO=MY@EEQ-gc*5tn{zvMHJEVxc$$#a$j^ko0Iuu6VGZpuAUvAC>ydUub8j*87{H^L zavxxPsFfvacH^DW@wg#+&3tfmn|66LY3>-KvWOX`_~lFGr9isXQ9Vc=n?M>&LP8Q3 z7dN(J_a=Ju8^gDV9Ov>s?ww|DKt>O@!!sw08hCcxGr(}D)C1abo;(Dam!F?u-GT9_rB%f z@#H8P>-3q|PvJJ-l^<5k5^tg_D=WXvk#c_%AHO?3eh|_He_}bgPB=Z>*E%<=ah%<> zs2m@Re{USk^}2;eTZen0n2+*Ei!}W9KCRg^Su^Ht)kp6B9&wpiQr0rlUBKsYgZCIu zN*obgRV!6zPOY6MwUOL?{~?X{0C!%!S zEVy?59840xJoRbm`xMe_7nHH)?wianZT(<_`cX1k&)Ir&R1CP-EVTryu?(TflPBXF zgD_aw*$lrl;X_ST_(M9*GW{2EDCN-+5;4%5X*p=m&J$kp4D44*a{bcZ-JWnm=oy~- zBRZY73Jh;q)pLrT9-*l#j>!?j^SP%UwGT0RXw>Jk=ifoi4r5!~63s3dQfMXL8wX>` zLPrv!K40#pa^77T%n>qd?XhY=%Pw1M>3YJn zTy%4+B^N!@@Jcq^F2rRCeIWzORD9M-^v2Mhi+I=)Ge^(BP=a~_yl(F zjHQcz$}g{>gD%dQr;*NP4n^&&e;LhHCcq7PV180LXN z``SAsV>gAuN_@swklXm1wHOSvXG1={*sVT^H3dD=TCebPm$X~lrh+}}u_Kub%wLiG zu9lr7vmm*tGRL&9uXd0zx}t1#w5gsB=dGni@GhK+dl_c`s^Y22Y@=UYA(ZzW-b;$# zctWwa@+j878FZi!EiZgtB)VfNSlZi!P75?B?<%KgdDeO{}gsy33#s3#6?1^i|)vzM7_ z`i{{~Du(@oBh8)B$g66_^MO_2xV+W;tH4@MsE}%UYTn+0Oz|0nyrh`VL3d1-o7=2E>QP1m~Zs`acVQ)u5ZLZBW zGdl4f!{JiK-htzCY!%~5m1RQOf|ix#L^h?#(Zu;f?qMtKr+gkv179>Vd}XiCp=p8(l6#`X{nW|sp;3RUtL`_bLt~XA?xONAdpq) zDxsfh1DlYL5D!mnRh1Y*I6}4un;d9CD?hUDhvA_i+`zVhfq`Hm*2$?UR~MJK`1p_Y zkTN7F2-L_yPzvc0B;w-Y`t7XOCqF@l{2)mOB8`0TdDS`dz(*F>eQ?^Fy3ktdzoU^(c{Cqa8FLAx@bDZEG_Lc?@qUq#rZf=17a3l^)19wZR zf2}*5!TpCQzwgB2%6279ya$7)Wyn3|w9VYnNdtQOr>dc)r1X?r_lY)^C?v_e0?Y-8 zvZWWYRq!W+C!!}PYdT>tnnf~7Vjl>|+7MXRo;@toWP1r)~6QN(2I5#Z;h?x+ZKYqI;b4Vb|W+)&JuS3NstWK_vvc48Ct?n zQB?&7iU&bZ4XI`0=H`Y#D3!cx2>3em*=BV$HGhJ>E0~(VV2T>B{VV>v3JS0PA{n>Rb4DUY9dRmTlV6XjTaqq zkYv|qEx>-t*-0)9o1c&B#3s^9s*~8`CK0}89(3$UjyFKwo-8Db=I(RzW^<0HyZpRl za=xkxQuVXCJ8TT+d2C3MSe#VoK62AVebACM^IjP+3>{#3EHFTAVR?d!1Uw!DFB6JN zONCY}EG(E=SQNhTrR0c3dBfrG)KrQwj4@OslxH#QY;06mFfxR-#C_d&kGXPDFi&|` z!6wf=9UZN4jLv)Vl6-0+&K8-#Gi^JXL+e^Qe_86*tt91?%hcGjBx?E#knE;;@hX&pY0TgWG)3&! zq?weABXQ~8=^ygPt4j*|Cg~--XY#V?c>P&NanhSrRhau)?N(qXu9dL!v}+Gl{M`Ei z>-2%kZr7@A-?mG2ocz|!&Gj_*jdwOVXo=pXZY_H}%zNVL3EmKk$V;LUPea43!ouLY zhrYhPv#ntU=Jb@46PrG54Grp)XMqH49cw3(lPc<|iqD|dB?9$MXgcCSJzlS8H?4pmIM99Ugkc7eO zHLJtWVHFd*kdT!kV|${DbN1sg@y3=4H6%M{*e%Vm$Qnvtnzmx5)N?tL*02g2e$MRG zE^>m2{P_0kCr*;rM{*u%$sUZc)NId4gptJ96C3UKR1_8Q%}{4TY;iA4YNMe2xlt&M2$EXlTc`VUtJ z$3r}&sduVaEh+u4@lt>A&*_9!PXr`p>J5@_-G$dMW`9|;s18)ALe4vieUdLc6UsA% zVxVBgyxcDXVf&!R=6#Y@S;ww*);&CJICgz1G#Om?280`v`O+^m6ld1K(UI;&AQ7uh z0Ks=9Z^V-q7*XC0jzx zDz0b!0*PyW@#=hWC4_jUS5n2yuAdGMwjLlskrUo{yyI6EGaLIEYgE3RsYn{is za11OEn)C^#DA$kI{<3pJLqng#DJ)N7QNJsQgV_@YR##U?Mk21*u_Hy~!5^qkmfMgq zNd`zb5tJ0FZ;y<}%DTIidrME~loE5<$qDg&^yc+B>$ultXLY$7nR9!t zJ6bx~xhqQuQH!&!8!Wm=s1QS{v(Hc)um<#e$zuFZR)mNM8=CXo@&q zC7oT2oBlLr`K%jj=_dBe-pBYOcDj}e8(Ku!Wgn8FUj@6tOd?d%VA)0Qx!*t&t9y%i zlEJw6k3?@T1p+VR%slu9>#d=bnHhB|>{SWc?;TzuzNVZmw-=UC{H^(f+MZHub55Hy(JHMuc&m(h9SlZYcAxaa6Nz~sH znP}K5a+zchp{Bfs3KXp4RaZyF4{3s{-j!Xyiu_F$T%#dExamWPCk-z&O8b*`4$JRz zW6VF1Ep)CK`cR4Re|@PIpUw8UbRUw&8lQp_k{dM?@2oBR#WRAJSqco2w8W+s7PNwtk?HOJc(1zFY!m|g z{QQ1!TP3}#MfV}Pp55|aJq%}*yD7LYhNq}N6~)7NEAFQKb&#yL>AymsiBU)H?CcB< z%3u<0Zf!;P8uF#+sH+)Njn1bM0;vTs4E;8fBG$e;6bBX;63pWfcyhX;=rUO-e=coOC9dU?1|YqNH7 zFz4gr)6~@Dc(iC~Zhm{SSGpq~w;N#;1lS)wQ zFU}D1StUe0-Fy;6xP6hPUi}LzKL>HaRw`1l^s-V=R+uiEfLm%gFTf z^w=2t`}ZtZq{9CsAwZTjYHDY1A3GH6#%F2f>EYoKR#{e7R$D7RXUNXW!ND;oLHh!b z(lON7V>?yqDi&wsicZg%YGi#^i#rsYf z=f-z)Hal8dK>|$yI@f0&NDntS*baBg}66ns<# z_`}14665*rw%uJuBYS&$iE#n7`fG`e?9ICKiNMnLMR()Nr>(C_Ac8#S8`rwJyi8)N z?Qv#i$wWKWJ8W;WSN_NebuwOge3aJM>V`1gf8jCOPet$e=x0dhaBWC2(Labv%VgwoJ*D^NGhxe|ZcGvLeVdj|Rdz-6ACXUVBG=yzlv_AD~RR zm)xSI--95vPXDZr76 zJFVOQH+WSoZkR@cwK7oM%7TZoD@jsqII5nBLv515;HiL1LSL6irZ1nK-2{wqG&anP zCAij`O?G@gwYCA;fo#Dk#i$>*!XAQ6|2F2qenYzQO!#`BkL>%n~rBBACd|o_IaNNt@kmSjd-N^FNbmiA;OGUz2b} z!tHvd^rLPkIjXBKrV?(Vs;Tyr*&P+w^&|56%Q1sjtz}iv)98caV~!OgVq#L?Du=OR z!`AH9i=41D=&4HE&n!}6jt&*)30>`kJDm$A3F=B6;&>GM8|yKVYi@I zM0BcHb=CnXU-YXgoJw;%x!AY@WExay z*rJ~uqalrndYqd~KbBOFCKVe^-_SU_R)rD1WOQA2_O37(*$_I6Q;!QjvmL=2y~RA3 znwUokW`sa%>#h`x$&7<^z!rI|N%O(25q=yd(nmBfi-0K}PoH4YT`est0V0?R@T5jg zl1R$-lIq*|*k@*{{$ZT!Uy?!af;$=c3`k*Qp5|5%vzBshsGcDkCkKE@B884jB6QhnMrbY+|t@g4F#&o)>MXU zfLUqX@dlGZqVZ8Hx=$wgKXuJm$O=JXf;iY@yp4sGy{Cq_a;@SZNx-!3j=xtQi=t%BuB7;Kg6Gf0Hz z3%zAIV8V^bJ;jyqkog78$g$S4d2sNpoVE#BJYtgnTz)oaYmg^y2W2MU^*C5|&?xRa zNKZ=7Zj3BGMX8aStN^mv5<=#V_jYLUD`oEEG;MEP&}@%&a)U5=CPC?^D={0a4j1|( z&{ZJ<=w|yo?=%kce))$m5BlVI(P+aIkb?ivI9F+*nbG;TxqZKRp^5@wJw^%F6#G(> z`d=R5xy@T4A-ok4VPnhOMZ5-GnyO9wV;YUAgRcEi97r%|Yq8NoeGT`fM^UrQ)2~U` z4C2HD+Q)}pL6y(VgG>%7G$PqW%;NMqJ-SpZoNF#7L`?+a2xMpu5BTI1B_ux%Nf4QS zAaT8j$jHg^N7m2pN?%t}&t1!WCL+51{?j~W@?MC|k{XFDW;lH&?i_>js~3l&W!0u$c#T<(9h9=cJ963mys-0CU=!H)JW}J|k8b_VJ7JwysNTn$9ioHjK z@BIz;PAQq%;^?;?7a1v7=0fsF)S>*EAI5E6?W-dawuJ6%Z%{)%2VJ-ze>yeVVn~*) zOxfAlA6P?8RG8W(97g4HcERdz$!T#==zkHZsIW+@UxxktwSENmz-tqh&-%Tbnh>b} zCprPw9_t)=)r$Np>#PgJXE!7XjxD(W++_N*&=VGgqpy?F& z%3{6TAXntB+FNS_LnV2-z0b6c>b$76EmeBt7JQ!6fh;E7>=iYehG^Qo$2TmQ{JN;Uo<#3&?q@U=xb|+x%~dmE@@ozQ zP9}!!uRFL!+l5lpOu|}tVbA9#F8mb_$=re3J(0dd+ zUe6`G@0~bwD(_f-(PdNr3(!XwBkA4$?aq~R_Onkywj{Y-%6~$?Y2PYXn@0m zPMy97KFk$SV*97Km25Wjd;xG{7^+3m9u2Q0Y+&!!iVFL)m!R&T+80*w(RWJFHpbRi5PEt zK|r8IVoGcm2L}gnY-3|%okpkf=H|um@yf!>N z(5q}89_EaAk4ez^{`-(Txc9fmg_<8feypvn5wKZUT1IjK3j!-MvpgS$&m;w)hMY|3 z0RqXgXep%+=w5by{sy4f>hYC2QCfj-$9Ga^Yg=)1iYDG~KzAjk7NcEK4)VL<6Mt_& zi;mmir3N$EAJyj%@}KpCv&jT@QstHNKHq1iIVISSNlJ+rn~h9ZJMqjnA?vjOm~-UD z5q7)Y8{OR8ys@z%{f2{!YjSMt^y!K~#`gG{;xcJ8d+HPgZ zQWF!s!{O^!#z^tMYHP^d+PbWzMc3Tw+d)|B{ZQOS1BU_O>s_L{26P`{I$Dhagd#J1 z!QX$wYEbPG@1<3D9nmjLC}{au;WuKrYkq#dwbe@=WmaMPS#YPRr=?}+6LiUp6hI`K z4VLrYOm9R*_17)A=<8M%-7WqQ1t;UPJCe^PMJ07J*{LI07R-9-1kO%*Hi%@3T$Fa%Q7tWd$ z$OK&K2pkfy*)O9%fwKAB+4oNjU6qY9>5a^O^O~w$l~vhA@v?TKjufl%t}ex1m&o<` zSa@b1W!~}DnHVf7Ctp6_V%lAuV>YU;M5x*eD#@Bnbx*F@vp3v*-4gK7eoFQPkm~fz z5OrN$q86e0^73*(&erYNCB2Y9efjdGi;IgmojB=J0%7^Sjt*bu0ULBt_>IwR`-OpJ z2ZZqD+39A;3xl^WaUqXlEJd)k9o(5BnLHYX7#d19qBC>l^*V)8PnVQTch7`kn{{Sq z{j%u!m%p%mbDi|*)K87=<1YM?!@jhV<9 z7qzciUNP!oxtCO65gJ915USy_ns7e$->GTvk-DmfB%JkzahIaJrmH0Wn(@{w9K+{G zpFF>{Q1v9DwDIuyxX=EvvhK$%My98&-cO*@)JLpZk3Bu&s;WfS_p!0DTf=FJYQp@$ zjEQLgN*#PuFBtR#y4Q+}iz@_jDG}-F&WjRN5&j0aySWw>K7&?ea5SuZ+0Ch1v{gcD z$DaEn-W@U9_CJ2=n@?~hu_JDct(7984#meP7}v2Oh0r2E`Q&C zBspuNj{by@C!Wo2xbj}_i`$ZWGh3i9s?XKu(y9=f3idIE<4DOSZGgVIW*=`iR8$;I zm$gt6dD@38-3|Nq!@&?0EA2n-qt_u=qRvjKxnHQ3I!7+U73R=iwnBk*0y<7f5*l?$ z*CHnim?I*==s4*hV1h5!+ZX2N2XgvYeBw2H6)5&?v5}!4z~b`h`Lu_H*ZRn90*XW0 z7}zF5(1(yZmNie7*#9RytwUvAPtL`6$FYMA9}eAn@7~0HUHJv8TW^z@x!JLIDfp=} zK!9S-pK>Pj7e9tpPQ1|=#RV8QX4VsTr`H>J0>_|nqP)TH(9wi;cyR8*ZK zW!|K)mSX=Oqf@cq>nu7)s172%>DnlDz^1*QnOuq>2#xbDds4xy=N(IW$(*ZcX0i82 zT<+dcO2$Bk9+X@tcYg zNdEj)GVw60o;!`$miI|Ov&dk-00^oIGEc#7QD#^RnIPBojs~GNDNm_w|JsnfO_T zdhdCE1XtWEHwS4laE1P1pn-!FQGdP4i+gT6482q#IC`z=>* z`~B@D-J3V`@?c`(>lOc3K(yQ4+cR?SnhQS>un~> z&|ij1T6&u)Ef8mS^0nn{2xyt|Z`^A`vgGIEqt{Pwc&M&^H-`Kj@m-1WV<0VZXlQ5_ z8tn_atK*e+CjnW^(UB4BvrU;wR3hjxweIbm{QUhFBg^#JgZwFOxC~Vt8mFuND;Jd3 zjGq8ueikT}lbZ_weFm+Bv^2Rl(K~i7Mn-UluI_G4O--OWN+IAR3{+%=Gmu|%adZFr z^$VcP=%N|wI~arrHJ?f_UbE`l$dnB@{>``VHB$(4wQG@9*zv zL0DK=Sx;$^0fyROv(|Mm_Y0^R0#xm2v1x8{^5T3-cRYi6*c(FdV)Bu30d08EH&#vU z|2GiTuu^>tfVgN+@)0*T_5YthR7(B-J^1?nj%lrF;ZSpXXYIqkBopGV3)V5Az{Q(y zXsKlg|7Fb2lCq~2`5Cw%<^UCuU&^hr>ig?+-R7J2^UB>u%F0S=Tp$5Axqh;?@8T09lc%P7x6+DgTnk>|(TNuD z3pk(+gbPNNmeM4%Q~A`iwX=%=T5gtvCdDn3-}qbLS2N$qgnz6ji5A6G`~U5U+ha0; zi@Qy;zYRU3RjFn@dTl>6={*{`5m&wueh`H$PfEx|{C91vUj97pI&N zm&aQlxd5lb-x2zsIoaTZsMO>v)mg>{)muZ2-Px+C}H&t!uj zqQ%^VyspXQsYL;S`&#Ay;+T~J%6O{qmbx?D9oAeNX+pKWrFn1QS{MHRf$eGH`=!`1 zTYnP6etuee(kk%{nz1fYnrtX3fl@B7FxlJKn7z1fjZ%tGf~O>im+5VqImP0mX5g6f zD*sLQFGBT+P;n5d{mUzcltV?m-D!IMwowc!ZBN&wDZ8w$)-n{COV`=)iABLK+33m4 zdz?=~oxCP&0A%KiF{7iim8fA(so(yBGtW~V1=QM%1nsf8ThFibUW{pMc-5>YORbF# zcQxp(I+*mJO9}b-l~tF=HEQ@}@3!cqrUJF3Cjr+&Gn<@|e>dHsX%%3x0Zq$6Vrj!u z_G#ORhQ!lYc^5FTu7g{ zVb3e5qJ;8sf0zLYaGo;!FZo4FOyHC7BEhm?NKax`l)`n@Xqc8b&+~xCm@sszXN7|V zY61RQgwSW_Z_J0wd^oBl91OR6;W}K;W3ux^4PDUT^|Nk`n(D!Re{U!JKZga)IV@F= z<{L*x>Me>BH0G@PBy3sOo>ty=-_;P>F~Px>)pPD9r2+R;y)X)c~dH!Y^< zP`H54B;pvu(`F8Cxg97%9@{X(A!(zZsO6Q2=wEw)BG-%s7fC4Yt7m|~+}peM zI0hE7#TUT-EX*t%p@*xry{S^Hir0>;g^Q$FIJSUI2$vZ6cD=hd!&S6zbZUy5lM|?+ zRyQ@hRn+cL1)o-DgIoOH(T=T{X3I zyBDa%g1$(g<01J*1f^EvTwI5qp`jt5wB6mc?|uVTf-Ji+jN+!D(Ol})+z(QVOtC^3 zN$oOV6_@WBIv4#ZfbdQARJ>Hl{5h-Ji9U%}Wc|~JZUMgl^PFJaKX(o4VXK0$Ni;EG z9W3cYe0Z<(R~t*cNg^m)12r}PaoqI2q0aquT|!rP>Gah0O*Annskxb%^W7C&wiV3O zzigJD>|VBLwWBu(`fw|7GCQ_blXmC$+%H9~NeJO__?BOue4mQA=aYG%ND6~Oz*PQ_ zymGq12QUQ$>#}m!%KRZcWk^&ybXXRCuU!4*+Gh?GxTP1uz`!3L)x1S{s?Y!K zU9325>U(#|t@k`U(@RUG)!e@vKBpZBI2LhQ@TIGL>}5E)O?^QGg{=9CNhv!?W+htZ zcC`m0Z;fF3RC8$${&PaWvPBBmwSY3K2%)J|hA8vPpZ7o(g-J;dcL36w?04OV+XnGc zk}O*uIVJ=RW*BS)un0 zA9jBnLkZ;H%6L8azboUsv(lT??cL!O7HpO#%blz)jDF`0HPXDAPuL| z0MT6QQ9td_qjTo|MvN|Z(ji;1KDf(#fWv#AYc5B2u365M`K8$0ihL`(;-x_f&0WS2 z%IA_2)IxUoo}A|b!R!$2t5w%s!gGk*sOp!?fRv-@b9P-Lp#wJPv33y+q0>1d#cAx6 zpuS54dGl`Jww4hb#DggeuIahOni67XADfu?=;3K(v~hj5MeuTD=<14RgXQhpNZA0f zAn@vmleu{z^DsjIghw@%P^BjlMdQU+mxAazy~8{w6t{}N2B9;qVc^HEim_yhE#S&wGIIP|Mr2YRt&qOX;30qfR%|qFwV>RmK~wL$Ei~F4r27oL=>4- zX{77tQD0xPL5kZ)mWN$%%XJ!FJQkZ?(PI7fZRQyBaDfZR`!=HQH@SJ^dy6^I5{ zL)vpb{BIe7mK7ziLz$HB7-6mh?Zl6;e8W`>?-ztpKUn||%`^TAEFb;!gFp?q*pKZo zv;$}Ll~`hPj)A2)8iII|JTj@*@7R^4;Y#WI4WG4$47ZQR*X?z`eGkNV5{wb@n5;V7 zwads_(VKO)5esFTdfNIW9q=a@82HpIEGz_ytpYh4HJnm8z06DOyWujBt`o3CXbb@6 ziGcxub$J&}%^&K>mIq-EGB4ALnTEM8S$lAmw!XEzF1g$ZyVXCZ(8}bIZ2$b%ex)X7 z`=Go$i~so^b5<<=ec7xn^75PL1`MAPR7UWA$hohUo%aoXv)aYN`n2+M9*7nVYIoLX zp-DJG_bQmz1!Gc(uQqd(s^j|t0TVu-$pR@c%ZBO!UA>qn)4j!7d< z`&XlU@KN{q1nyx{PUdm;E+srVv6-k|eC8<<%d5=xQ|vhLg5sf;F|F3IJNZn=Y2@`TGlfh92{<|MNV_;=cufw)x&t5YpkxW&duEd_yR2us|onz^?_B|)wSWT7$mb%aS<6o z0nq|h$zdFEFyNbEPsHA7TVFTy^c2EqMIfFqAoaF3xn6#oueG_oH~`S};^HD;Qmd)1 zcE3NLvZL>ZQDT1vb7?8p<1Hb{X8~5zqT*tVv_-%R0jwt^^18a72ea=op3;J`pl`n~i%zgkq24mwK-as*Z2ZuNX-w%8#T>U|o&0(qE@&1kI|Du{%;l(>)N@X=& z-FR&zkeBGENX)+$8t#AcBZB|>_JZ+mLKXDgWcWddDI{Xx&jr)p9UzI1vSKA7df)#q D<@!M> literal 0 HcmV?d00001 From 8d758b63f2f69a292dde16db0b8176361bb393d7 Mon Sep 17 00:00:00 2001 From: James Truher Date: Tue, 2 Aug 2016 11:50:52 -0700 Subject: [PATCH 3/4] updates based on PR feedback --- docs/testing-guidelines/PesterDoAndDont.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/docs/testing-guidelines/PesterDoAndDont.md b/docs/testing-guidelines/PesterDoAndDont.md index 250e5b6f0..1dd7c91c2 100755 --- a/docs/testing-guidelines/PesterDoAndDont.md +++ b/docs/testing-guidelines/PesterDoAndDont.md @@ -4,7 +4,7 @@ 1. Test only what you need 2. Reduce dependencies 3. Be sure to tag your `Describe` blocks based on their purpose - 1. Tag `CI` indicates that it will be run as part of the continuous integration process. These should be unit test like, and fast. + 1. Tag `CI` indicates that it will be run as part of the continuous integration process. These should be unit test like, and generally take less than a second. 2. Tag `Feature` indicates a higher level feature test (we will run these on a regular basis), for example, tests which go to remote resources, or test broader functionality 3. Tag `Scenario` indicates tests of integration with other features (these will be run on a less regular basis and test even broader functionality than feature tests. 4. Make sure that `Describe`/`Context`/`It` descriptions are useful @@ -12,14 +12,22 @@ 5. Use `Context` to group tests 1. Multiple `Context` blocks can help you group your test suite into logical sections 6. Use `BeforeAll`/`AfterAll`/`BeforeEach`/`AfterEach` instead of custom initiators -7. Use Try-Catch for expected errors and check $_.fullyQualifiedErrorId (don't use `should throw`) +7. Prefer Try-Catch for expected errors and check $_.fullyQualifiedErrorId (don't use `should throw`) 8. Use `-testcases` when iterating over multiple `It` blocks 9. Use code coverage functionality where appropriate 10. Use `Mock` functionality when you don't have your entire environment 11. Avoid free code in a `Describe` block - 1. Use `[Before|After][Each|All]` _see Free Code in a Describe block_ - + 1. Use `[Before|After][Each|All]` see [Free Code in a Describe block](WritingPesterTests.md#free-code-in-a-describe-block) +12. Avoid creating or using test files outside of TESTDRIVE: + 1. TESTDRIVE: has automatic clean-up +13. Keep in mind that we are creating cross platform tests + 1. Avoid using the registry + 2. Avoid using COM +14. Avoid being too specific about the _count_ of a resource as these can change platform to platform + 1. ex: checking for the count of loaded format files, check rather for format data for a specific type ## Don't -1. Have too many evaluations in a single It block +1. Don't have too many evaluations in a single It block 1. The first `Should` failure will stop that block -2. `Should` be used in an `It` Block +2. Don't use `Should` outside of an `It` Block +3. Don't use the word "Error" or "Fail" to test a positive case + 1. ex: "Get-Childitem TESTDRIVE: shouldn't fail", rather "Get-ChildItem should be able to retrieve file listing from TESTDRIVE" From 935185cf6b6c91e688de03dbedee73978ae9a279 Mon Sep 17 00:00:00 2001 From: "James Truher [MSFT]" Date: Tue, 2 Aug 2016 11:55:50 -0700 Subject: [PATCH 4/4] fix don't section so it's visible --- docs/testing-guidelines/PesterDoAndDont.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/testing-guidelines/PesterDoAndDont.md b/docs/testing-guidelines/PesterDoAndDont.md index 1dd7c91c2..da4dbae4c 100755 --- a/docs/testing-guidelines/PesterDoAndDont.md +++ b/docs/testing-guidelines/PesterDoAndDont.md @@ -25,6 +25,7 @@ 2. Avoid using COM 14. Avoid being too specific about the _count_ of a resource as these can change platform to platform 1. ex: checking for the count of loaded format files, check rather for format data for a specific type + ## Don't 1. Don't have too many evaluations in a single It block 1. The first `Should` failure will stop that block