This removes some of the last mentions of the AppVeyor name. Don't change the title, we don't want the word to appear in the CLs
Also problem: the last references are in the changelogs but I don't want to modify them without committee approval.
## PR Context
it was removed, #8686
Split Windows CI into the following Jobs
1. Combine Build and xUnit, which published build artifact
1. Elevated CI tests
1. Unelevated CI test
1. Elevated everything else tests
1. Unelevated everything else tests
1. Package build and test
1. Verify xUnit result
## PR Context
Before this test:
- A CI run would take ~14 minutes
- A Feature run would take ~30 minutes
After this change, it always runs all tests and takes ~20 minutes.
Job `Unelevated everything else tests` is the long poll, taking ~4 minutes longer than the next shortest job. We should be able to get this back down to ~15 if we split that test job somehow.
Remove references to uninitialized variable
regressed in #8822
## PR Context
Some artifacts never got published because the conditions would never be true
Refactors `travis.ps1`. Important for maintainers: I removed the webhook and badge stuff as neither exist anymore. I also merged the build and test phase with the bootstrap phase.
Added feature to classify experimental features in change log. Also fixed an issue with reporting all PRs with missing tags together.
## PR Context
Updates to the script which generates the change log.
Add binskim to coordinated build and increase timout
## PR Context
Total timeout for the old build was 220 minutes.
The portions before the compliance take ~30 minutes.
So, I went with 180.
I also found I missed binskim when doing this work
Refactor Unified Release Build
- Make injecting the nuget.config a template
- Make setting the version variable a template
- Add parameters to templates where needed
- Run defender scan before uploading linux packages (and rename step to plain english)
Add compliance to Coordinated build
- Also switch to mac internal pool for release build
- Also turn some duplicate tasks into templates
- Also fix issue with vscode configuration which causes yaml files not to be recogized as yaml
Update the universal build to also build the framework package needed for the dotnet sdk container image.
## PR Context
We build the package in individual builds. This change brings over the steps in the universal build.
Update `New-ReferenceAssembly` and `New-UnifiedNugetPackage` to generate reference assembly for `Microsoft.PowerShell.Commands.Utility.dll` and properly deploy it for `Microsoft.PowerShell.Commands.Utility` NuGet package and `Microsoft.PowerShell.SDK` NuGet package.
An incremental step to fix, eventually, #8121
Update `New-ReferenceAssembly` and `New-UnifiedNugetPackage` to generate reference assembly for `Microsoft.PowerShell.Commands.Utility.dll` and properly deploy it for `Microsoft.PowerShell.Commands.Utility` NuGet package and `Microsoft.PowerShell.SDK` NuGet package.
An incremental step to fix, eventually, #8121
This PR is a re-implementation of the -AddToPath switch in tools/install-powershell.ps1, without the side-effects the current implementation has. The changes only affects windows users.
Add script to generate macOS icon file. It does not attempt to integrate with the current build process as the macOS icon file is mostly static.
`libsvg` was chosen because of its low dependency count over other solutions that use backends like Chromium, PhontomJS, etc. ImageMagick's `convert` was tested but resulting PNGs where not satisfactory and would have required an overcomplicated script having to calculate the `density` argument.
* Has a dependency on librsvg (can be installed via Brew)
* Uses SVG file as input
* Can be integrated in later to build process if desired
Closes#7455
When installing (or upgrading) Preview builds the component ids for the files are the same as the installed files from the stable MSI. MSI sees this and assumes the file is already installed so skips installing files that haven't changed (been updated) which results in missing files in the preview install folder and pwsh fails to start. Fix is to dynamically generate new component ids (and compoentrefs) in `files.wxs` using `_Preview` suffix and use that when building a preview package.
Tested manually.
cc @bergmeister if you can help validate different scenarios.
Fix https://github.com/PowerShell/PowerShell/issues/8289
## Motivation
I have a PR where there are many new xUnit tests.
It would also be useful to create new xUnit tests for public APIs.
The number of xUnit tests will increase and their ordering is required.
## PR Summary
- Move C# xUnit tests in new folder. This allows to put new xUnit tests in directory structure in accordance with directory structure where cs files are.
- Use an xUnit TestCaseOrderer attribute to sequentially process tests for `powershell.config.json`.
- Update README.md
- A race condition was fixed which allowed to run all XUnit tests in single batch job.
Create unified release build for macOS and Linux packages
- Also, updated the definition with latest definitions from the definitions in use.
I will work on these in future PRs:
- Add compliance Job
- Add macOS Signing step
- Merging Windows into the YAML
Major changes are as follows:
- Avoid `SecuritySupport.IsProductBinary` and unnecessary AMSI/suspicious code scan at startup time
- Update `CompiledScriptBlockData.IsProductCode` to avoid unnecessary calls to `IsProductBinary`, which attempts to retrieve catalog signature of the target file.
- Update `PerformSecurityChecks` to skip AMSI and suspicious code scan for the `.psd1` file that contains a safe `HashtableAst` only.
- Use customized `ReadOnlyBag` instead of `ImmutableHashSet` so that we can avoid loading the `System.Collections.Immutable.dll` completely.
- Replace `SHA1` with `CRC32` when generating module analysis cache file name
- This remove the loading of `System.Security.Cryptography.Algorithms.dll` at startup
- Move `ConvertFrom-SddlString` to C# to remove the `Utility.psm1` file.
- Crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation
- Even pwsh with crossgen assemblies spends a lot time in jitting at the startup, about `191.6ms` comparing with `24.7ms` for Windows PowerShell.
- Jitting `Microsoft.ApplicationInsights.dll` takes about `51.6ms`.
- By crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation, the jitting time drops to about `98.9ms`.
Compare the PowerShell modules dependencies on PowerShell Gallery and sync them to AzDevOps artifacts feed if a newer is available. The release builds pick up modules from AzDevOps feed.
Remove tilde from Linux preview packages.
- Since we have a separate package for preview, we don't need a second indicator that these packages are preview.
- Also, many system we use to release don't like the `~` character, which leads to manual interventions and errors.
As of #7892, the PowerShell repository no longer uses Git submodules.
This is fantastic from a workflow standpoint, and so all the notes about
how to deal with submodules (and all the build steps explicitly
initializing and updating submodules) can be safely removed.
The variable was set to empty (meaning to delete the variable) in the non-preview case and the build fails.
The fix avoids setting the variable to empty
Add the functionality to build a framework dependent (shared framework) package for PowerShell.
The changes create two packages, one for Windows and other for Linux, due to #if code.
Add daily build non-windows platforms
- Also, make the [Feature] tag work in VSTS for non-windows
- Also, add a way to force feature tests to run
- Also, fix an issue where `-workingdirectory` didn't work when running async
On Debian 9, libmi cannot resolve libssl and libcrypto. This change adds symbolic links to packaging and build to the $PSHOME directory to resolve the issue.
The fix was verified interactively connecting to office 365, importing the session, and ensuring Get-MailBox and Get-User succeed.
Fix#7598
- Add VSTS CI for Windows
- Disable `Access-denied test for Get-Item C:\windows\appcompat\Programs\Install -ErrorAction Stop`, because the path does not always exist
- https://github.com/PowerShell/PowerShell/issues/7553
- Disable `Should give .sys file if the fullpath is specified with hidden and force parameter`, because pagefile.sys doesn't always exist and other files don't meet test's requirement.
- https://github.com/PowerShell/PowerShell/issues/7554
- Disable some `Test-Connection` tests for same reasons they failed on VSTS Linux
- https://github.com/PowerShell/PowerShell/issues/7555
- Disable `Test-FileCatalog should pass when catalog is in the same folder as files being tested`, because the CmdLet does not work in that scenario
- Also, give details needed to investigate when the test fails
- https://github.com/PowerShell/PowerShell/issues/7556
- Update `appveyor.psm1` to work with VSTS
- Update `HelpersRemoting.psm1` `New-RemoteSession` to work for CimSession (discovered an issue during the investigation)
- Update `Test wildcard with drive relative directory path` to work when there are multiple drives
- Disable on non-windows machines since the test is assuming drive letters
- Update `New-CimSession` Tests to requireAdmin
- Also, make sure session name is a string
- Add functions to save and restore psoptions
- update `.gitatttributes` so files clone like they do on appveyor
Update Start-PSPester and Start-PSxUnit to upload test results when running in VSTS
Disable tests which require non-root using when running as root
remove verifications in Test-Connection tests which verify buggy behavior
Skip tests which don't behave correctly in VSTS (due to running in a container?)
- Test-Connection - #7528
- Set-Date - #7529
Fixes#7074. This PR allows concurrent versions of the macOS launcher app to exist. The application name will be either PowerShell or PowerShell-preview. It is now dynamically built and removed post fpm to avoid post build conflicts, see #5262 .
* update to latest package references
* update runtime framework
* update sdk
* automatically read NuGet package dependency info from csproj, where version info is fully qualified
* update file.wxs
-preview switch:
- installs pwsh-preview packages when a package manager is being used.
- uses link name pwsh-preview when tar ball install is being done
- results in preview release being available for calling manually, but does not become systemwide version
Issue #7177
setdate.cpp was incorrectly assigning an int to a struct, fixed to initialize struct zero'd
update the typegen project in the bash script
added some help text at top of script for someone trying to build it
updated runtime to linux-musl-x64 instead of linux-x64 as Alpine distro uses the musl libc libraries instead of glibc which is used by most distros.
Fixed#6815 by adding -allowprereleases parameter
Fixed#6405 by adding -allowprereleases parameter
Added parameters to documentation
Ready for prerelease repositories if Microsoft starts providing them
Added -skip-sudo-check for all distros
Fixed -interactivetesting should do nothing if -includeide was not used
update the Windows release build script with the ability to capture all the project.assets.json files
add a Windows build to use the new feature in the release build script
update the Linux release build script with the ability to capture all the project.assets.json files
-no new build is needed because signing is not currently automated
Closes#6944
Following PR #6945, further, reduce the total build time (without any disadvantages) by around 5 minutes by making sure there is a more even split between the 2 build jobs (the 2nd build job used to be much shorter).
Therefore this PR moves also the xUnit and Pester-Admin tests into the 2nd build job. If it is a daily/feature test commit, then the feature tests will also happen (only) in the 2nd build job. Because both jobs now run tests, the failfast option was removed. The final question from my side is whether running tests in 2 build jobs is OK for the daily build, which uploads code coverage results?
The time to wait for the AppVeyor build results is now 15 +/- 2 minutes, which is a huge improvement to what used to be around 28 minutes before the build matrix was introduced.
Fix man page conflict between stable and preview Linux package
- rename preview man page pwsh-preview
Fix /etc/shell entries not being created correctly for preview packages
Update links that contain 'en-us' culture to remove 'en-us' culture (if possible) and in some cases update to newer re-directed link to docs.microsoft.com
Address mac portions of PowerShell/PowerShell-RFC#115 (comment)
Make path when stable /usr/local/microsoft/powershell/6
when not stable (preview) /usr/local/microsoft/powershell/6-preview and symlink name pwsh-preview
allow side-by-side install of stable and preview
Also:
fix an issue where the utime work around for mac requires sudo
refactor some code into common functions
Change the *nix packaging over with the following changes:
Package name (as used by e.g. apt):
Non-preview releases are namedpowershell
Preview releases are named powershell-preview
Installation path:
No longer looks like /opt/microsoft/powershell/6.1.0/ or /opt/microsoft/powershell/6.1.0-preview.1/
Non-previews go to a path like /opt/microsoft/powershell/6/
Previews go to a path like /opt/microsoft/powershell/6-preview/
Path to executable symlink:
Allows SxS with preview
Non-previews linked to /usr/bin/pwsh or /usr/local/bin/pwsh on macOS
Previews linked to /usr/bin/pwsh-preview or /usr/local/bin/pwsh-preview on macOS
Implements PowerShell/PowerShell-RFC#115 (comment)
Related: #6944
Reduce PR build time by 5 minutes by:
Having Packaging as a separate build job in a matrix -> runs in parallel in PR builds because the Microsoft account is a paid account that allows that (at no additional costs)
Not caching the dotnet folder anymore, which is too large and the overhead of zipping/unzipping/upload/download does not pay off (and fails in forked builds that are on a free AppVeyor account due to the size).
Setting the environment variable DOTNET_SKIP_FIRST_TIME_EXPERIENCE to 1 because the initialization of the dotnet CLI cache (1 minute) does not pay off for the whole build.
The total build time of builds on a fork that is on a free AppVeyor account and therefore does not have parallelism, remains the same due to the time saving of redundant caching.
This is just a simple example of what we can easily achieve, we could continue this pattern and split the test runs as per the referenced issue to bring PR builds down to 10 minutes (but this will incur an increase for fork builds on free AppVeyor accounts)
To support PowerShell modules built with .NET Windows Compatibility Pack, we decided that it was best to ship the WCP assemblies with PS Core. This also adds many new .NET APIs be default while only adding ~3.5 MB additional disk footprint (to a ~137 MB install currently).
Also update the build to adopt the official .NET Core 2.1.
The main purpose of this was to enable full symbols for windows release build.
Also makes explicit where we are optimizing and where we are not optimizing due to https://github.com/dotnet/corefx/issues/29700
Ubuntu 17.04 is EOL, so we need to update our packaging for Ubuntu 17.10. This updates the package script and build.psm1 to use 17.10 rather than 17.04.
* Build Update
- Change `TargetFramework` to `netcoreapp2.1` and removed unnecessary `RuntimeFrameworkVersion` from `PowerShell.Common.props`
- Update dotnet SDK to 2.1.300-rc1-008662
- Update `TypeGen` target in `Build.psm1` to work with 2.1
- Rename macOS runtime to `osx-x64` as the old build logic expects 10.12 and breaks running on 10.13 system.
- Remove `PackageReference` to `System.Memory` as it's part of dotnetcore 2.1
- Update search for `crossgen` executable to find the matching version
* Test Update
- Update test tools `WebListener` to latest `asp.net core`
- Marked `AuthHeader Redirect` tests as `Pending` due to change in CoreFX
The comparison to test if we were using sudo was failing when running as root. We would then run -v, and get a bad error message. fixing the comparison.
Update 'Get-ChangeLog' to make it more accurate
Use "||" as the delimiter instead of "|", so it works for commit message that has the pipeline operator in it.
Handle merges that create a merge node in history.
Do not treat powershell team members as external contributors.
Add meta properties to mac VSTS YAML
- Add a property to clean the build machine
- Add a property to set the format of the build name
- add a property to set which queue to run the build in
Since a PR added support to opt out of telemetry via an environment variable, we can remove the,
always intended to be a temporary, solution of deleting a file to opt out of telemetry since the
environment variable can be defined at the system level and exist before even installing PowerShell Core.
Because the variable is defined as opt out, a value of true, yes, or 1 means no telemetry is sent.
- Implementation of PowerShell/PowerShell-RFC#115 (If anything changes in the RFC, we will treat it as a bug, and fix it later)
- Update registry and directory paths to use 6 for the version for stable and 6-preview for a preview release
- Add checkbox to set path
- default checkbox to off for preview builds and on for stable builds
create a PowerShellPackage...ps1 that works with these VMs (vs our custom VM)
create a script to create the \powershell folder with correct permissions
a YML VSTS build definition
Fix#6332
This change renames log and logerror functions to a single Write-Log [$message] [-error] function to avoid conflicting with the log command on MacOS.
Clean build during the daily build to ensure MSI package is generated correctly
- Do another clean build directly before packaging to clean up files that test has added
Fixes#6315
make Linux packages use correct version scheme for preview releases
- Now uses <Major>.<Minor>.<Patch>~<PreviewName> instead of <Major>.<Minor>.<Patch>-<PreviewName> as the - was interpretted as an iteration of the release not a preview.
The script installpsh-debian only works for Ubuntu distributions. This PR adds support for Debian as well.
Related issue: #5700
* Remove support for Ubuntu 17.04 in installpsh-debian.sh
MSI: Make sure that file components are patchable
- avoid changing names and guids of components between builds as this prevents patch generation
- This required submitting the file generated by heat
- add code to make sure the generated file is not out of date
- add `wixpdb` output when creating `MSI` package
- capture `wixpdb` in official build
- clean up anything left behind from previous MSI builds before starting MSI build to prevent using dirty files.
- make sure MSI creation fails if there is an error
- ignore `.wixpdb` files in git
- Add functionality to `Start-NativeExecution` to
- only display output if there is an error
- log caller information
- WXS validation error fixes
- Remove unused `ExitDialog` to fix ICE82
- Add KeyPath to `SetPath` to fix ICE18
- Use `HKMU` which translates to `HKLM` to runtime to fix various validation errors about creating the shortcut
- Suppress Validation errors
- suppress ICE61, which is about same version upgrades being allowed
- suppress ICE57, caused by the shortcut not being installed per user
Remove the functions which generated Nuget packages for Windows.
Add function New-UnifiedNugetPackage to generate nuget packages for each assembly with unix and windows runtimes.
Add function New-NuSpec and New-ReferenceAssembly for creating the required items forNew-UnifiedNugetPackage.
Add a sample for cross platform project with conditional compilation for Linux.
Add function Publish-NugetToMyGet to publish nuget packages to powershell.myget.or
Based on standard practices, we need to have a copyright and license notice at the top of each source file. Removed existing copyrights and updated/added copyright notices for .h, .cpp, .cs, .ps1, and .psm1 files.
Updated module manifests for consistency to have Author = "PowerShell" and Company = "Microsoft Corporation". Removed multiple line breaks.
Separate PR coming to update contribution document for new source files: #6140
Manually reviewed each change.
Fix#6073
PR 6043 broke the installer (issue #6095). To prevent this from happening in the future, add a smoke test that installs the msi in appveyor build and make build fail if installation failed.
It uses the exit code to determine the success. The reason why it does not fail in the current state is because as I pointed out here, the failing custom action is not returning its exit code.
This is the continuation of PR 5499 that had to be abandoned due to a fatal merge conflict and I did not want to risk accidentally reverting recent fixes.
Remove unnecessary/unused default for productGuid because it always gets a new Guid when being called from Start-PSPackage
Add defaults for required files but also add extra path validation attribute
Rename ProductGuid to ProductCode
Make PowerShell Core reads group policy settings from different registry keys (Windows only) and the configuration files (both Windows and Unix).
- On Windows, move to different GPO registry keys.
- On both Windows and Unix, read GPO related settings from the configuration file `powershell.config.json`.
- On Windows, the policy settings in registry take precedence over the configuration file.
- Enable policy controlled logging and transcription on Unix.
Make PowerShell Core reads group policy settings from different registry keys (Windows only) and the configuration files (both Windows and Unix).
- On Windows, move to different GPO registry keys.
- On both Windows and Unix, read GPO related settings from the configuration file `powershell.config.json`.
- On Windows, the policy settings in registry take precedence over the configuration file.
- Enable policy controlled logging and transcription on Unix.
Create the module 'releaseTools.psm1' under 'tools' folder, which expose the function 'Get-ChangeLog' to generate a release change log based on the local commit history.
