Refactor the `ci.psm1` to publish the code coverage and test packages.
Allow `CodeCoverage` configuration on non-windows.
## PR Context
We plan to run code coverage on Windows, Linux and macOS. These changes are needed to unblock those runs.
Add support in packaging.psm1 to produce a .msix AppX package. Update the docker image to use the new msix package type. Update the associated yml files so AzDevOps performs the build.
## PR Context
Enable publishing PSCore6 to Microsoft Store
Fixes#8919
Preserve user shortcuts pinned to Taskbar during MSI upgrade by not removing shortcuts in this case (assuming the user has not changed the installation directory), see https://stackoverflow.com/a/33402698/1810304
This also requires the Guid to not always be re-generated, which PR #7701 originally added to ensure shortcuts get removed when RTM and preview are installed, the underlying problem was rather that RTM and preview shared the same GUIDs, therefore the GUIDs are hard-coded again but different for RTM and preview, therefore the shortcuts will still always get removed on uninstall. But this also means those GUIDs should change when the default installation directory changes, i.e. in PowerShell 7. Should we write the code to already take this into account that it does not get forgotten?
Tested by first reproducing the issue by building installers locally (and bumping the patch version. Then the fix was applied to verify the solution, it. For this to take effect the version from which an MSI is being upgraded must have this fix already, i.e. if this fix got shipped in `6.2.1`, then on upgrading to it, the issue would still occur but when upgrading `6.2.1` to `6.2.2` the shortcut would start being preserved. I am wondering if we could maybe improve this to show effect earlier by trying to extract the used (auto-generated) GUIDs in the `6.2.0` and `6.2.0-rc` packages out and use them...
Please not that we probably need to take this out for `7.0` because the base installation directory will change. This also assumes that the user has not specified a different installation directory on upgrade but this is a bit of an edge case where I think other things might break as well.
Moved check if able to write to $PSHome as way to skip test to `BeforeAll` which already contained a check if running on Windows.
## PR Context
As part https://github.com/PowerShell/PowerShell/pull/9279, tests were updated to be skipped if the test requires writing to `$PSHome` but is not able to. However, these tests already had a skip mechanism in place so the additional check caused the test to run when it should have skipped.
Co-authored-by: Travis Plunk <github@ez13.net>
This PR optimizes the download tasks to reduce build times. After the change we see a order of magnitude reduction in download times for Windows package signing and also the compliance tasks.
In addition to that the PR also makes changes to organize the build artifacts in folders.
Removes appveyor stuff
## PR Context
<!-- Provide a little reasoning as to why this Pull Request helps and why you have opened it. -->
#8686 , appveyor removed
The `f` in Dockerfile is not uppercase, this breaks syntax highlighting for vscode on my client
<!-- Summarize your PR between here and the checklist. -->
## PR Context
`DockerFile` breaks syntax highlighting as mentioned and described above.
<!-- Provide a little reasoning as to why this Pull Request helps and why you have opened it. -->
The NuGet build task should wait on uploads for DEB job to complete.
## PR Context
There is a race condition when the build for DEB is completed but the upload has not. If the NuGet build job starts it may not get all the packages needed for building nuget packages.
Refactor macOS CI so that tests run in parallel
- Also:
- Fix bootstrap issue where curl was failing for a month (so obviously not needed)
- Change Windows test stages to use PowerShell Core (where we can) to avoid compat issues
## PR Context
This allows running all tests in about the same about of clock time.
I noticed a couple of inconsistencies when reading through the install bash scripts.
- Make documentation for switches consistent over all files.
- Replace all `sed` implementations of `lowercase` with a more maintainable `tr` implementation.
- Set the `OS` variable in every install script, making it so previously unused checks are actually used.
- Exit with a non-zero exit code when the script reaches an illegal state.
## PR Context
A lot of people, including myself, read the install scripts before executing them. While doing so I noticed they contained inconsistencies. For example, certain flags you can pass to the install script were either undocumented or had an incorrect description. This PR fixes some of these inconsistencies, which should make them easier to maintain and easier to read.
Co-authored-by: Travis Plunk <github@ez13.net>
* Add shim dll to redirect to appropriate runtime
* Build updates for creating global tool package
* Add building global tool to nuget.yml
* Add azure upload of nuget packages
Make test file result names unique
- Also, remove dead code that I found during investigation
Fixes#8978
## PR Context
Making the filenames unique, ensures that we can download the results from the artifacts and do our own processing of the results.
This removes some of the last mentions of the AppVeyor name. Don't change the title, we don't want the word to appear in the CLs
Also problem: the last references are in the changelogs but I don't want to modify them without committee approval.
## PR Context
it was removed, #8686
Split Windows CI into the following Jobs
1. Combine Build and xUnit, which published build artifact
1. Elevated CI tests
1. Unelevated CI test
1. Elevated everything else tests
1. Unelevated everything else tests
1. Package build and test
1. Verify xUnit result
## PR Context
Before this test:
- A CI run would take ~14 minutes
- A Feature run would take ~30 minutes
After this change, it always runs all tests and takes ~20 minutes.
Job `Unelevated everything else tests` is the long poll, taking ~4 minutes longer than the next shortest job. We should be able to get this back down to ~15 if we split that test job somehow.
Remove references to uninitialized variable
regressed in #8822
## PR Context
Some artifacts never got published because the conditions would never be true
Refactors `travis.ps1`. Important for maintainers: I removed the webhook and badge stuff as neither exist anymore. I also merged the build and test phase with the bootstrap phase.
Added feature to classify experimental features in change log. Also fixed an issue with reporting all PRs with missing tags together.
## PR Context
Updates to the script which generates the change log.
Add binskim to coordinated build and increase timout
## PR Context
Total timeout for the old build was 220 minutes.
The portions before the compliance take ~30 minutes.
So, I went with 180.
I also found I missed binskim when doing this work
Refactor Unified Release Build
- Make injecting the nuget.config a template
- Make setting the version variable a template
- Add parameters to templates where needed
- Run defender scan before uploading linux packages (and rename step to plain english)
Add compliance to Coordinated build
- Also switch to mac internal pool for release build
- Also turn some duplicate tasks into templates
- Also fix issue with vscode configuration which causes yaml files not to be recogized as yaml
Update the universal build to also build the framework package needed for the dotnet sdk container image.
## PR Context
We build the package in individual builds. This change brings over the steps in the universal build.
Update `New-ReferenceAssembly` and `New-UnifiedNugetPackage` to generate reference assembly for `Microsoft.PowerShell.Commands.Utility.dll` and properly deploy it for `Microsoft.PowerShell.Commands.Utility` NuGet package and `Microsoft.PowerShell.SDK` NuGet package.
An incremental step to fix, eventually, #8121
Update `New-ReferenceAssembly` and `New-UnifiedNugetPackage` to generate reference assembly for `Microsoft.PowerShell.Commands.Utility.dll` and properly deploy it for `Microsoft.PowerShell.Commands.Utility` NuGet package and `Microsoft.PowerShell.SDK` NuGet package.
An incremental step to fix, eventually, #8121
This PR is a re-implementation of the -AddToPath switch in tools/install-powershell.ps1, without the side-effects the current implementation has. The changes only affects windows users.
Add script to generate macOS icon file. It does not attempt to integrate with the current build process as the macOS icon file is mostly static.
`libsvg` was chosen because of its low dependency count over other solutions that use backends like Chromium, PhontomJS, etc. ImageMagick's `convert` was tested but resulting PNGs where not satisfactory and would have required an overcomplicated script having to calculate the `density` argument.
* Has a dependency on librsvg (can be installed via Brew)
* Uses SVG file as input
* Can be integrated in later to build process if desired
Closes#7455
When installing (or upgrading) Preview builds the component ids for the files are the same as the installed files from the stable MSI. MSI sees this and assumes the file is already installed so skips installing files that haven't changed (been updated) which results in missing files in the preview install folder and pwsh fails to start. Fix is to dynamically generate new component ids (and compoentrefs) in `files.wxs` using `_Preview` suffix and use that when building a preview package.
Tested manually.
cc @bergmeister if you can help validate different scenarios.
Fix https://github.com/PowerShell/PowerShell/issues/8289
## Motivation
I have a PR where there are many new xUnit tests.
It would also be useful to create new xUnit tests for public APIs.
The number of xUnit tests will increase and their ordering is required.
## PR Summary
- Move C# xUnit tests in new folder. This allows to put new xUnit tests in directory structure in accordance with directory structure where cs files are.
- Use an xUnit TestCaseOrderer attribute to sequentially process tests for `powershell.config.json`.
- Update README.md
- A race condition was fixed which allowed to run all XUnit tests in single batch job.
Create unified release build for macOS and Linux packages
- Also, updated the definition with latest definitions from the definitions in use.
I will work on these in future PRs:
- Add compliance Job
- Add macOS Signing step
- Merging Windows into the YAML
Major changes are as follows:
- Avoid `SecuritySupport.IsProductBinary` and unnecessary AMSI/suspicious code scan at startup time
- Update `CompiledScriptBlockData.IsProductCode` to avoid unnecessary calls to `IsProductBinary`, which attempts to retrieve catalog signature of the target file.
- Update `PerformSecurityChecks` to skip AMSI and suspicious code scan for the `.psd1` file that contains a safe `HashtableAst` only.
- Use customized `ReadOnlyBag` instead of `ImmutableHashSet` so that we can avoid loading the `System.Collections.Immutable.dll` completely.
- Replace `SHA1` with `CRC32` when generating module analysis cache file name
- This remove the loading of `System.Security.Cryptography.Algorithms.dll` at startup
- Move `ConvertFrom-SddlString` to C# to remove the `Utility.psm1` file.
- Crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation
- Even pwsh with crossgen assemblies spends a lot time in jitting at the startup, about `191.6ms` comparing with `24.7ms` for Windows PowerShell.
- Jitting `Microsoft.ApplicationInsights.dll` takes about `51.6ms`.
- By crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation, the jitting time drops to about `98.9ms`.
Compare the PowerShell modules dependencies on PowerShell Gallery and sync them to AzDevOps artifacts feed if a newer is available. The release builds pick up modules from AzDevOps feed.
Remove tilde from Linux preview packages.
- Since we have a separate package for preview, we don't need a second indicator that these packages are preview.
- Also, many system we use to release don't like the `~` character, which leads to manual interventions and errors.
As of #7892, the PowerShell repository no longer uses Git submodules.
This is fantastic from a workflow standpoint, and so all the notes about
how to deal with submodules (and all the build steps explicitly
initializing and updating submodules) can be safely removed.
The variable was set to empty (meaning to delete the variable) in the non-preview case and the build fails.
The fix avoids setting the variable to empty
Add the functionality to build a framework dependent (shared framework) package for PowerShell.
The changes create two packages, one for Windows and other for Linux, due to #if code.
Add daily build non-windows platforms
- Also, make the [Feature] tag work in VSTS for non-windows
- Also, add a way to force feature tests to run
- Also, fix an issue where `-workingdirectory` didn't work when running async
On Debian 9, libmi cannot resolve libssl and libcrypto. This change adds symbolic links to packaging and build to the $PSHOME directory to resolve the issue.
The fix was verified interactively connecting to office 365, importing the session, and ensuring Get-MailBox and Get-User succeed.
Fix#7598
- Add VSTS CI for Windows
- Disable `Access-denied test for Get-Item C:\windows\appcompat\Programs\Install -ErrorAction Stop`, because the path does not always exist
- https://github.com/PowerShell/PowerShell/issues/7553
- Disable `Should give .sys file if the fullpath is specified with hidden and force parameter`, because pagefile.sys doesn't always exist and other files don't meet test's requirement.
- https://github.com/PowerShell/PowerShell/issues/7554
- Disable some `Test-Connection` tests for same reasons they failed on VSTS Linux
- https://github.com/PowerShell/PowerShell/issues/7555
- Disable `Test-FileCatalog should pass when catalog is in the same folder as files being tested`, because the CmdLet does not work in that scenario
- Also, give details needed to investigate when the test fails
- https://github.com/PowerShell/PowerShell/issues/7556
- Update `appveyor.psm1` to work with VSTS
- Update `HelpersRemoting.psm1` `New-RemoteSession` to work for CimSession (discovered an issue during the investigation)
- Update `Test wildcard with drive relative directory path` to work when there are multiple drives
- Disable on non-windows machines since the test is assuming drive letters
- Update `New-CimSession` Tests to requireAdmin
- Also, make sure session name is a string
- Add functions to save and restore psoptions
- update `.gitatttributes` so files clone like they do on appveyor
Update Start-PSPester and Start-PSxUnit to upload test results when running in VSTS
Disable tests which require non-root using when running as root
remove verifications in Test-Connection tests which verify buggy behavior
Skip tests which don't behave correctly in VSTS (due to running in a container?)
- Test-Connection - #7528
- Set-Date - #7529
