When installing (or upgrading) Preview builds the component ids for the files are the same as the installed files from the stable MSI. MSI sees this and assumes the file is already installed so skips installing files that haven't changed (been updated) which results in missing files in the preview install folder and pwsh fails to start. Fix is to dynamically generate new component ids (and compoentrefs) in `files.wxs` using `_Preview` suffix and use that when building a preview package.
Tested manually.
cc @bergmeister if you can help validate different scenarios.
Fix https://github.com/PowerShell/PowerShell/issues/8289
## Motivation
I have a PR where there are many new xUnit tests.
It would also be useful to create new xUnit tests for public APIs.
The number of xUnit tests will increase and their ordering is required.
## PR Summary
- Move C# xUnit tests in new folder. This allows to put new xUnit tests in directory structure in accordance with directory structure where cs files are.
- Use an xUnit TestCaseOrderer attribute to sequentially process tests for `powershell.config.json`.
- Update README.md
- A race condition was fixed which allowed to run all XUnit tests in single batch job.
Create unified release build for macOS and Linux packages
- Also, updated the definition with latest definitions from the definitions in use.
I will work on these in future PRs:
- Add compliance Job
- Add macOS Signing step
- Merging Windows into the YAML
Major changes are as follows:
- Avoid `SecuritySupport.IsProductBinary` and unnecessary AMSI/suspicious code scan at startup time
- Update `CompiledScriptBlockData.IsProductCode` to avoid unnecessary calls to `IsProductBinary`, which attempts to retrieve catalog signature of the target file.
- Update `PerformSecurityChecks` to skip AMSI and suspicious code scan for the `.psd1` file that contains a safe `HashtableAst` only.
- Use customized `ReadOnlyBag` instead of `ImmutableHashSet` so that we can avoid loading the `System.Collections.Immutable.dll` completely.
- Replace `SHA1` with `CRC32` when generating module analysis cache file name
- This remove the loading of `System.Security.Cryptography.Algorithms.dll` at startup
- Move `ConvertFrom-SddlString` to C# to remove the `Utility.psm1` file.
- Crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation
- Even pwsh with crossgen assemblies spends a lot time in jitting at the startup, about `191.6ms` comparing with `24.7ms` for Windows PowerShell.
- Jitting `Microsoft.ApplicationInsights.dll` takes about `51.6ms`.
- By crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation, the jitting time drops to about `98.9ms`.
Compare the PowerShell modules dependencies on PowerShell Gallery and sync them to AzDevOps artifacts feed if a newer is available. The release builds pick up modules from AzDevOps feed.
Remove tilde from Linux preview packages.
- Since we have a separate package for preview, we don't need a second indicator that these packages are preview.
- Also, many system we use to release don't like the `~` character, which leads to manual interventions and errors.
As of #7892, the PowerShell repository no longer uses Git submodules.
This is fantastic from a workflow standpoint, and so all the notes about
how to deal with submodules (and all the build steps explicitly
initializing and updating submodules) can be safely removed.
The variable was set to empty (meaning to delete the variable) in the non-preview case and the build fails.
The fix avoids setting the variable to empty
Add the functionality to build a framework dependent (shared framework) package for PowerShell.
The changes create two packages, one for Windows and other for Linux, due to #if code.
Add daily build non-windows platforms
- Also, make the [Feature] tag work in VSTS for non-windows
- Also, add a way to force feature tests to run
- Also, fix an issue where `-workingdirectory` didn't work when running async
On Debian 9, libmi cannot resolve libssl and libcrypto. This change adds symbolic links to packaging and build to the $PSHOME directory to resolve the issue.
The fix was verified interactively connecting to office 365, importing the session, and ensuring Get-MailBox and Get-User succeed.
Fix#7598
- Add VSTS CI for Windows
- Disable `Access-denied test for Get-Item C:\windows\appcompat\Programs\Install -ErrorAction Stop`, because the path does not always exist
- https://github.com/PowerShell/PowerShell/issues/7553
- Disable `Should give .sys file if the fullpath is specified with hidden and force parameter`, because pagefile.sys doesn't always exist and other files don't meet test's requirement.
- https://github.com/PowerShell/PowerShell/issues/7554
- Disable some `Test-Connection` tests for same reasons they failed on VSTS Linux
- https://github.com/PowerShell/PowerShell/issues/7555
- Disable `Test-FileCatalog should pass when catalog is in the same folder as files being tested`, because the CmdLet does not work in that scenario
- Also, give details needed to investigate when the test fails
- https://github.com/PowerShell/PowerShell/issues/7556
- Update `appveyor.psm1` to work with VSTS
- Update `HelpersRemoting.psm1` `New-RemoteSession` to work for CimSession (discovered an issue during the investigation)
- Update `Test wildcard with drive relative directory path` to work when there are multiple drives
- Disable on non-windows machines since the test is assuming drive letters
- Update `New-CimSession` Tests to requireAdmin
- Also, make sure session name is a string
- Add functions to save and restore psoptions
- update `.gitatttributes` so files clone like they do on appveyor
Update Start-PSPester and Start-PSxUnit to upload test results when running in VSTS
Disable tests which require non-root using when running as root
remove verifications in Test-Connection tests which verify buggy behavior
Skip tests which don't behave correctly in VSTS (due to running in a container?)
- Test-Connection - #7528
- Set-Date - #7529
Fixes#7074. This PR allows concurrent versions of the macOS launcher app to exist. The application name will be either PowerShell or PowerShell-preview. It is now dynamically built and removed post fpm to avoid post build conflicts, see #5262 .
* update to latest package references
* update runtime framework
* update sdk
* automatically read NuGet package dependency info from csproj, where version info is fully qualified
* update file.wxs
-preview switch:
- installs pwsh-preview packages when a package manager is being used.
- uses link name pwsh-preview when tar ball install is being done
- results in preview release being available for calling manually, but does not become systemwide version
Issue #7177
setdate.cpp was incorrectly assigning an int to a struct, fixed to initialize struct zero'd
update the typegen project in the bash script
added some help text at top of script for someone trying to build it
updated runtime to linux-musl-x64 instead of linux-x64 as Alpine distro uses the musl libc libraries instead of glibc which is used by most distros.
Fixed#6815 by adding -allowprereleases parameter
Fixed#6405 by adding -allowprereleases parameter
Added parameters to documentation
Ready for prerelease repositories if Microsoft starts providing them
Added -skip-sudo-check for all distros
Fixed -interactivetesting should do nothing if -includeide was not used
update the Windows release build script with the ability to capture all the project.assets.json files
add a Windows build to use the new feature in the release build script
update the Linux release build script with the ability to capture all the project.assets.json files
-no new build is needed because signing is not currently automated
Closes#6944
Following PR #6945, further, reduce the total build time (without any disadvantages) by around 5 minutes by making sure there is a more even split between the 2 build jobs (the 2nd build job used to be much shorter).
Therefore this PR moves also the xUnit and Pester-Admin tests into the 2nd build job. If it is a daily/feature test commit, then the feature tests will also happen (only) in the 2nd build job. Because both jobs now run tests, the failfast option was removed. The final question from my side is whether running tests in 2 build jobs is OK for the daily build, which uploads code coverage results?
The time to wait for the AppVeyor build results is now 15 +/- 2 minutes, which is a huge improvement to what used to be around 28 minutes before the build matrix was introduced.
Fix man page conflict between stable and preview Linux package
- rename preview man page pwsh-preview
Fix /etc/shell entries not being created correctly for preview packages
Update links that contain 'en-us' culture to remove 'en-us' culture (if possible) and in some cases update to newer re-directed link to docs.microsoft.com
Address mac portions of PowerShell/PowerShell-RFC#115 (comment)
Make path when stable /usr/local/microsoft/powershell/6
when not stable (preview) /usr/local/microsoft/powershell/6-preview and symlink name pwsh-preview
allow side-by-side install of stable and preview
Also:
fix an issue where the utime work around for mac requires sudo
refactor some code into common functions
Change the *nix packaging over with the following changes:
Package name (as used by e.g. apt):
Non-preview releases are namedpowershell
Preview releases are named powershell-preview
Installation path:
No longer looks like /opt/microsoft/powershell/6.1.0/ or /opt/microsoft/powershell/6.1.0-preview.1/
Non-previews go to a path like /opt/microsoft/powershell/6/
Previews go to a path like /opt/microsoft/powershell/6-preview/
Path to executable symlink:
Allows SxS with preview
Non-previews linked to /usr/bin/pwsh or /usr/local/bin/pwsh on macOS
Previews linked to /usr/bin/pwsh-preview or /usr/local/bin/pwsh-preview on macOS
Implements PowerShell/PowerShell-RFC#115 (comment)
Related: #6944
Reduce PR build time by 5 minutes by:
Having Packaging as a separate build job in a matrix -> runs in parallel in PR builds because the Microsoft account is a paid account that allows that (at no additional costs)
Not caching the dotnet folder anymore, which is too large and the overhead of zipping/unzipping/upload/download does not pay off (and fails in forked builds that are on a free AppVeyor account due to the size).
Setting the environment variable DOTNET_SKIP_FIRST_TIME_EXPERIENCE to 1 because the initialization of the dotnet CLI cache (1 minute) does not pay off for the whole build.
The total build time of builds on a fork that is on a free AppVeyor account and therefore does not have parallelism, remains the same due to the time saving of redundant caching.
This is just a simple example of what we can easily achieve, we could continue this pattern and split the test runs as per the referenced issue to bring PR builds down to 10 minutes (but this will incur an increase for fork builds on free AppVeyor accounts)
To support PowerShell modules built with .NET Windows Compatibility Pack, we decided that it was best to ship the WCP assemblies with PS Core. This also adds many new .NET APIs be default while only adding ~3.5 MB additional disk footprint (to a ~137 MB install currently).
Also update the build to adopt the official .NET Core 2.1.
The main purpose of this was to enable full symbols for windows release build.
Also makes explicit where we are optimizing and where we are not optimizing due to https://github.com/dotnet/corefx/issues/29700
