Split Windows CI into the following Jobs
1. Combine Build and xUnit, which published build artifact
1. Elevated CI tests
1. Unelevated CI test
1. Elevated everything else tests
1. Unelevated everything else tests
1. Package build and test
1. Verify xUnit result
## PR Context
Before this test:
- A CI run would take ~14 minutes
- A Feature run would take ~30 minutes
After this change, it always runs all tests and takes ~20 minutes.
Job `Unelevated everything else tests` is the long poll, taking ~4 minutes longer than the next shortest job. We should be able to get this back down to ~15 if we split that test job somehow.
## PR Summary
Related: #8699
## PR Context
Because `PowerShellGet` does not support publishing/saving module on a per platform basis (see [this](https://github.com/PowerShell/PowerShellGet/issues/273) issue), PowerShell currently also ships the fullclr binaries of `PackageManagement`, which it does not need. Therefore removing it to minimise the package size, this saves 1.19 MB.
Fix broken URL
- Also, move other static analysis tests into that CI.
- Also, make the link analysis a pester test (partly to make sure the step fails in case of an error)
## Motivation
I have a PR where there are many new xUnit tests.
It would also be useful to create new xUnit tests for public APIs.
The number of xUnit tests will increase and their ordering is required.
## PR Summary
- Move C# xUnit tests in new folder. This allows to put new xUnit tests in directory structure in accordance with directory structure where cs files are.
- Use an xUnit TestCaseOrderer attribute to sequentially process tests for `powershell.config.json`.
- Update README.md
- A race condition was fixed which allowed to run all XUnit tests in single batch job.
Major changes are as follows:
- Avoid `SecuritySupport.IsProductBinary` and unnecessary AMSI/suspicious code scan at startup time
- Update `CompiledScriptBlockData.IsProductCode` to avoid unnecessary calls to `IsProductBinary`, which attempts to retrieve catalog signature of the target file.
- Update `PerformSecurityChecks` to skip AMSI and suspicious code scan for the `.psd1` file that contains a safe `HashtableAst` only.
- Use customized `ReadOnlyBag` instead of `ImmutableHashSet` so that we can avoid loading the `System.Collections.Immutable.dll` completely.
- Replace `SHA1` with `CRC32` when generating module analysis cache file name
- This remove the loading of `System.Security.Cryptography.Algorithms.dll` at startup
- Move `ConvertFrom-SddlString` to C# to remove the `Utility.psm1` file.
- Crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation
- Even pwsh with crossgen assemblies spends a lot time in jitting at the startup, about `191.6ms` comparing with `24.7ms` for Windows PowerShell.
- Jitting `Microsoft.ApplicationInsights.dll` takes about `51.6ms`.
- By crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation, the jitting time drops to about `98.9ms`.
Add a function to create a NuGet.config file to consume packages from the specified feed URL using the specified credentials.
- Also, exclude the `nuget.config` file under `src/Modules` from being clean by `Start-PSBuild -clean`
As of #7892, the PowerShell repository no longer uses Git submodules.
This is fantastic from a workflow standpoint, and so all the notes about
how to deal with submodules (and all the build steps explicitly
initializing and updating submodules) can be safely removed.
Native code for PowerShell has been moved to repository PowerShell/PowerShell-Native.
Hence, removing all the native code and related build components.
Add the functionality to build a framework dependent (shared framework) package for PowerShell.
The changes create two packages, one for Windows and other for Linux, due to #if code.
Add daily build non-windows platforms
- Also, make the [Feature] tag work in VSTS for non-windows
- Also, add a way to force feature tests to run
- Also, fix an issue where `-workingdirectory` didn't work when running async
On Debian 9, libmi cannot resolve libssl and libcrypto. This change adds symbolic links to packaging and build to the $PSHOME directory to resolve the issue.
The fix was verified interactively connecting to office 365, importing the session, and ensuring Get-MailBox and Get-User succeed.
Fix#7598
- Add VSTS CI for Windows
- Disable `Access-denied test for Get-Item C:\windows\appcompat\Programs\Install -ErrorAction Stop`, because the path does not always exist
- https://github.com/PowerShell/PowerShell/issues/7553
- Disable `Should give .sys file if the fullpath is specified with hidden and force parameter`, because pagefile.sys doesn't always exist and other files don't meet test's requirement.
- https://github.com/PowerShell/PowerShell/issues/7554
- Disable some `Test-Connection` tests for same reasons they failed on VSTS Linux
- https://github.com/PowerShell/PowerShell/issues/7555
- Disable `Test-FileCatalog should pass when catalog is in the same folder as files being tested`, because the CmdLet does not work in that scenario
- Also, give details needed to investigate when the test fails
- https://github.com/PowerShell/PowerShell/issues/7556
- Update `appveyor.psm1` to work with VSTS
- Update `HelpersRemoting.psm1` `New-RemoteSession` to work for CimSession (discovered an issue during the investigation)
- Update `Test wildcard with drive relative directory path` to work when there are multiple drives
- Disable on non-windows machines since the test is assuming drive letters
- Update `New-CimSession` Tests to requireAdmin
- Also, make sure session name is a string
- Add functions to save and restore psoptions
- update `.gitatttributes` so files clone like they do on appveyor
Update Start-PSPester and Start-PSxUnit to upload test results when running in VSTS
Disable tests which require non-root using when running as root
remove verifications in Test-Connection tests which verify buggy behavior
Skip tests which don't behave correctly in VSTS (due to running in a container?)
- Test-Connection - #7528
- Set-Date - #7529
When trying to build on RHEL, the environment is not recognized as RedHatFamily.
Added detection logic so that appropriate symbolic links are generated.
Remove code from build.psm1 that save the modules to a versioned folder. For servicing reasons after release it is preferred and easier to not have the versioned folder so that we can directly replace modules that need to be serviced (MSI specifically makes it difficult to service a module if the folder path changes).
Fix man page conflict between stable and preview Linux package
- rename preview man page pwsh-preview
Fix /etc/shell entries not being created correctly for preview packages
The main purpose of this was to enable full symbols for windows release build.
Also makes explicit where we are optimizing and where we are not optimizing due to https://github.com/dotnet/corefx/issues/29700
Ubuntu 17.04 is EOL, so we need to update our packaging for Ubuntu 17.10. This updates the package script and build.psm1 to use 17.10 rather than 17.04.
* Build Update
- Change `TargetFramework` to `netcoreapp2.1` and removed unnecessary `RuntimeFrameworkVersion` from `PowerShell.Common.props`
- Update dotnet SDK to 2.1.300-rc1-008662
- Update `TypeGen` target in `Build.psm1` to work with 2.1
- Rename macOS runtime to `osx-x64` as the old build logic expects 10.12 and breaks running on 10.13 system.
- Remove `PackageReference` to `System.Memory` as it's part of dotnetcore 2.1
- Update search for `crossgen` executable to find the matching version
* Test Update
- Update test tools `WebListener` to latest `asp.net core`
- Marked `AuthHeader Redirect` tests as `Pending` due to change in CoreFX
Since a PR added support to opt out of telemetry via an environment variable, we can remove the,
always intended to be a temporary, solution of deleting a file to opt out of telemetry since the
environment variable can be defined at the system level and exist before even installing PowerShell Core.
Because the variable is defined as opt out, a value of true, yes, or 1 means no telemetry is sent.
The release packaging build needs to have 'dotnet' in path, fixed that by adding Find-DotNet
Refactored restore logic so it can be used from both Start-PSBuild and Copy-PSGalleryModules
The release packaging build needs to have 'dotnet' in path, fixed that by adding Find-DotNet
Refactored restore logic so it can be used from both Start-PSBuild and Copy-PSGalleryModules
set Debian front end to non-interactive during apt-get commands, to avoid error with prompting
add libffi-dev, which is required for ruby/fpm
let fpm update to the latest
Fix#6332
This change renames log and logerror functions to a single Write-Log [$message] [-error] function to avoid conflicting with the log command on MacOS.
Attempt to fix the macOS build.
In #6263, I tried flushing the cache and it worked in the PR. However, after merging the PR, it turns out the master CI build failed again with the same Pester package restore error. So I'm reverting the change that got Pester from NuGet cache.
If `-sudo` is specified, make sure to include 'RequireSudoOnUnix' tag on Unix if the 'Tag' is not specified.
If `-sudo` is not specified, make sure to exclude `RequireSudoOnUnix` tag on Unix if the 'ExcludeTag' is not specified.
- add `wixpdb` output when creating `MSI` package
- capture `wixpdb` in official build
- clean up anything left behind from previous MSI builds before starting MSI build to prevent using dirty files.
- make sure MSI creation fails if there is an error
- ignore `.wixpdb` files in git
- Add functionality to `Start-NativeExecution` to
- only display output if there is an error
- log caller information
- WXS validation error fixes
- Remove unused `ExitDialog` to fix ICE82
- Add KeyPath to `SetPath` to fix ICE18
- Use `HKMU` which translates to `HKLM` to runtime to fix various validation errors about creating the shortcut
- Suppress Validation errors
- suppress ICE61, which is about same version upgrades being allowed
- suppress ICE57, caused by the shortcut not being installed per user
Based on standard practices, we need to have a copyright and license notice at the top of each source file. Removed existing copyrights and updated/added copyright notices for .h, .cpp, .cs, .ps1, and .psm1 files.
Updated module manifests for consistency to have Author = "PowerShell" and Company = "Microsoft Corporation". Removed multiple line breaks.
Separate PR coming to update contribution document for new source files: #6140
Manually reviewed each change.
Fix#6073
* Test fixes and changes needed to support Pester 4.0.8
* Replace 'Should Contain' with new 'Should FileContentMatch' assertion
Explicitly check for string creation with write-output
* Use the current version of pester and install it in modulesDir
* Simplify logic for relative path test.
Multiple '..' is not needed for a relative path, a single one will do. Also, on multi-drive systems using split-path -noqualifier will probably do the wrong thing with regard to constructing a correct path.
Remove extraneous Should Not Throw test, if this throws, the test will fail, we don't need to explicitly assert the not throw
* In some environments it is possible that computer name is 'localhost', so that should be allowed
* [feature] Add link for migrating tests from Pester v3 to v4
Fix up capitalization and white space issues
Change one test to check FullyQualifiedErrorId rather than just `Should Throw`
* [feature] update invoke-item test to handle the case where multiple notepad processes are running
* Fix spelling issue with Pester 4x, calling it Pester 4 should be sufficient
