This pull request implements a "feature flagging" system that will let
us turn Terminal and conhost features on/off by branch, "release" status
or branding (Dev, Preview, etc.).
It's loosely modelled after the Windows OS concept of "Velocity," but
only insofar as it is driven by an XML document and there's a tool that
emits a header file for you to include.
It only supports toggling features at compile time, and the feature flag
evaluators are intended to be fully constant expressions.
Features are added to `src\features.xml` and marked with a "stage". For
now, the only stages available are `AlwaysDisabled` and `AlwaysEnabled`.
Features can be toggled to different states using branch and branding
tokens, as documented in the included feature flag docs.
For a given feature Feature_XYZ, we will emit two fixtures visible to
the compiler:
1. A preprocessor define `TIL_FEATURE_XYZ_ENABLED` (usable from MIDL,
C++ and C)
2. A feature class type `Feature_XYZ` with a static constexpr member
`IsEnabled()` (usable from C++, designed for `if constexpr()`).
Like Velocity, we rely on the compiler to eliminate dead code caused by
things that compile down to `if constexpr (false)`. :)
Michael suggested that we could use `WindowsInbox` as a branding to
determine when we were being built inside Windows to supplant our use of
the `__INSIDE_WINDOWS` preprocessor token. It was brilliant.
Design Decisions
----------------
* Emitting the header as part of an MSBuild project
* WHY: This allows the MSBuild engine to ensure that the build is
only run once, even in a parallel build situation.
* Only having one feature flag document for the entire project
* WHY: Ease.
* Forcibly including `TilFeatureStaging` with `/FI` for all CL compiler
invocations.
* WHY: If this is a project-wide feature system, we should make it as
easy as possible to use.
* Emitting preprocessor definitions instead of constexpr/consteval
* WHY: Removing entire functions/includes is impossible with `if
constexpr`.
* WHY: MIDL cannot use a `static constexpr bool`, but it can rely on
the C preprocessor to remove text.
* Using MSBuild to emit the text instead of PowerShell
* WHY: This allows us to leverage MSBuild's `WriteOnlyWhenDifferent`
task parameter to avoid changing the file's modification time when
it would have resulted in the same contents. This lets us use the
same FeatureStaging header across multiple builds and multiple
branches and brandings _assuming that they do not result in a
feature flag change_.
* The risk in using a force-include is always that it, for some
reason, determines that the entire project is out of date. We've
gone to great lengths to make sure that it only does so if the
features _actually materially changed_.
Implement PGO in pipelines for AMD64 architecture; supply training test scenarios
## References
- #3075 - Relevant to speed interests there and other linked issues.
## PR Checklist
* [x] Closes#6963
* [x] I work here.
* [x] New UIA Tests added and passed. Manual build runs also tested.
## Detailed Description of the Pull Request / Additional comments
- Creates a new pipeline run for creating instrumented binaries for Profile Guided Optimization (PGO).
- Creates a new suite of UIA tests on the full Windows Terminal app to run PGO training scenarios on instrumented binaries (and incidentally can be used to write other UIA tests later for the full Terminal app.)
- Creates a new NuGet artifact to store trained PGO databases (PGD files) at `Microsoft.Internal.Windows.Terminal.PGODatabase`
- Creates a new NuGet artifact to supply large-scale test content for automated tests at `Microsoft.Internal.Windows.Terminal.TestContent`
- Adjusts the release pipeline to run binaries in PGO optimized mode where content from PGO databases is leveraged at link time to optimize the final release build
The following binaries are trained:
- OpenConsole.exe
- WindowsTerminal.exe
- TerminalApp.dll
- TerminalConnection.dll
- Microsoft.Terminal.Control.dll
- Microsoft.Terminal.Remoting.dll
- Microsoft.Terminal.Settings.Editor.dll
- Microsoft.Terminal.Settings.Model.dll
In the future, adding `<PgoTarget>true</PgoTarget>` to a new `vcxproj` file will automatically enroll the DLL/EXE for PGO instrumentation and optimization going forward.
Two training test scenarios are implemented:
- Smoke test the Terminal by just opening it and typing a bit of text then exiting. (Should help focus on the standard launch path.)
- Optimize bulk text output by launching terminal, outputting `big.txt`, then exiting.
Additional scenarios can be contributed to the `WindowsTerminal_UIATests` project with the `[TestProperty("IsPGO", "true")]` annotation to add them to the suite of scenarios for PGO.
**NOTE:** There are currently no weights applied to the various test scenarios. We will revisit that in the future when/if necessary.
## Validation Steps Performed
- [x] - Training run completed at https://dev.azure.com/ms/terminal/_build?definitionId=492&_a=summary
- [x] - Optimization run completed locally (by forcing `PGOBuildMode` to `Optimize` on my local machine, manually retrieving the databases with NuGet, and building).
- [x] - Validated locally that x86 and ARM64 do not get trained and automatically skip optimization as databases are not present for them.
- [x] - Smoke tested optimized binary versus latest releases. `big.txt` output through CMD is ~11-12seconds prior to PGO and just over 8 seconds with PGO.
This pull request switches us to the new WinDev scaleset agent pool. It
should be faster than the hosted pool, and the larger disks allow us to
get rid of our PCH cleanup step.
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
Cool retro terminal effects
- glow
- scan lines
- cool
- will make terminal competitive with iterm2
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
## References
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [ ] Closes #xxx
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
* Change to use App CRT in preparation for universal.
* Try to make project build again by setting winconpty to static lib so it'll use the CRT inside TerminalConnection (or its other consumers) instead of linking its own.
* Remove test for conpty dll, it's a lib now. Add additional commentary on how CRT linking works for future reference. I'm sure this will come up again.
* use the _apiset variant until proven otherwise to match the existing one.
* Clarification in the comments for linking.
This commit cleans up and deduplicates all of the common build
preamble/postamble across exe, dll, lib and c++/winrt projects.
The following specific changes have been made:
* All projects now define their ConfigurationType
* All projects now set all their properties *before* including a common
build file (or any other build files)
* cppwinrt.pre and cppwinrt.post now delegate most of their
configuration to common.pre and common.post
* (becuase of the above,) all build options are conserved between
console and c++/winrt components, including specific warnings and
preprocessor definitions.
* More properties that are configurable per-project are now
conditioned so the common props don't override them.
* The exe, dll, exe.or.dll, and lib postincludes have been merged into
pre or post and switched based on condition as required
* Shared items (-shared, -common) are now explicitly vcxitems instead of
vcxproj files.
* The link line is now manipulated after Microsoft.Cpp sets it, so the
libraries we specify "win". All console things link first against
onecore_apiset.lib.
* Fix all compilation errors caused by build unification
* Move CascadiaPackage's resources into a separate item file
Fixes#922.
The WAP packaging project in VS <= 16.3.7 produces a couple global
properties as part of its normal operation that cause MSBuild to flag
our projects as out-of-date and requiring a rebuild. By forcing those
properties to match the WAP values, we can get consistent builds.
One of those properties, however, is "GenerateAppxPackageOnBuild", and
WAP sets it to *false*. When we set that, of course, we don't get an
MSIX out of our build pipeline. Therefore, we have to break our build
into two phases -- build, then package.
This required us to change our approach to PCH deletion. A project
without a PCH is *also* considered out-of-date. Now, we keep all PCH
files but truncate them to 0 bytes.
TerminalApp, however, is re-linked during packaging because the Xaml
compiler emits a new generated C++ file on every build. We have to keep
those PCHs around.
* Remove WpfTerminalControl AnyCPU from Arch-specific builds
This removes another source of build nondeterminism: that WpfTerminalControl was propagating TargetFramework into architecture-specific C++ builds. Its "Any CPU" platform has been removed from architecture builds at the solution level.
This also cleans up some new projects that were added and build for "Any
CPU".
* Cleanup PCHs as the build rolls along to leave enough space on CI agents.
* Attempt to restrict pch cleanup to only CI agents.
* Write message when objects are deleted.
