terminal/tools
Michael Niksa 7dadde5dd6
Implement PGO in pipelines for AMD64 architecture; supply training test scenarios (#10071)
Implement PGO in pipelines for AMD64 architecture; supply training test scenarios

## References
- #3075 - Relevant to speed interests there and other linked issues.

## PR Checklist
* [x] Closes #6963
* [x] I work here.
* [x] New UIA Tests added and passed. Manual build runs also tested.

## Detailed Description of the Pull Request / Additional comments
- Creates a new pipeline run for creating instrumented binaries for Profile Guided Optimization (PGO).
- Creates a new suite of UIA tests on the full Windows Terminal app to run PGO training scenarios on instrumented binaries (and incidentally can be used to write other UIA tests later for the full Terminal app.)
- Creates a new NuGet artifact to store trained PGO databases (PGD files) at `Microsoft.Internal.Windows.Terminal.PGODatabase`
- Creates a new NuGet artifact to supply large-scale test content for automated tests at `Microsoft.Internal.Windows.Terminal.TestContent`
- Adjusts the release pipeline to run binaries in PGO optimized mode where content from PGO databases is leveraged at link time to optimize the final release build

The following binaries are trained:
- OpenConsole.exe
- WindowsTerminal.exe
- TerminalApp.dll
- TerminalConnection.dll
- Microsoft.Terminal.Control.dll
- Microsoft.Terminal.Remoting.dll
- Microsoft.Terminal.Settings.Editor.dll
- Microsoft.Terminal.Settings.Model.dll

In the future, adding `<PgoTarget>true</PgoTarget>` to a new `vcxproj` file will automatically enroll the DLL/EXE for PGO instrumentation and optimization going forward.

Two training test scenarios are implemented:
- Smoke test the Terminal by just opening it and typing a bit of text then exiting. (Should help focus on the standard launch path.)
- Optimize bulk text output by launching terminal, outputting `big.txt`, then exiting.

Additional scenarios can be contributed to the `WindowsTerminal_UIATests` project with the `[TestProperty("IsPGO", "true")]` annotation to add them to the suite of scenarios for PGO.

**NOTE:** There are currently no weights applied to the various test scenarios. We will revisit that in the future when/if necessary.

## Validation Steps Performed
- [x] - Training run completed at https://dev.azure.com/ms/terminal/_build?definitionId=492&_a=summary
- [x] - Optimization run completed locally (by forcing `PGOBuildMode` to `Optimize` on my local machine, manually retrieving the databases with NuGet, and building).
- [x] - Validated locally that x86 and ARM64 do not get trained and automatically skip optimization as databases are not present for them.
- [x] - Smoke tested optimized binary versus latest releases. `big.txt` output through CMD is ~11-12seconds prior to PGO and just over 8 seconds with PGO.
2021-05-13 21:12:30 +00:00
..
PGODatabase Implement PGO in pipelines for AMD64 architecture; supply training test scenarios (#10071) 2021-05-13 21:12:30 +00:00
bcx.cmd Create bx.cmd (#2168) 2019-08-05 20:18:40 -05:00
bcz.cmd tools: update bcz.cmd to use the progress bar VT sequence (#8335) 2020-11-20 09:26:50 -08:00
bx.cmd Create bx.cmd (#2168) 2019-08-05 20:18:40 -05:00
bx.ps1 Prevent the v1 propsheet from zeroing colors, causing black text on black background. (#2651) 2019-10-02 16:04:59 -07:00
bz.cmd Create bx.cmd (#2168) 2019-08-05 20:18:40 -05:00
ConsoleTypes.natvis Fix SGR indexed colors to distinguish Indexed256 color (and more) (#5834) 2020-05-27 22:34:45 +00:00
echokey.cmd make copying of files windows localization agnostic (#741) 2019-05-21 16:25:54 +00:00
Generate-CodepointWidthsFromUCD.ps1 Fully regenerate CodepointWidthDetector from Unicode 13.0 (#8035) 2020-10-27 17:36:28 +00:00
GenerateAppxFromManifest.ps1 Fix a bunch of spelling errors across the project (#4295) 2020-02-10 20:40:01 +00:00
GenerateHeaderForJson.ps1 Add an action for identifying windows (#9523) 2021-03-30 16:08:03 +00:00
Get-OSSConhostLog.ps1 tools: add Get-OSSConhostLog (#7250) 2020-08-11 20:08:03 +00:00
openbash.cmd Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
opencon.cmd Prevent the v1 propsheet from zeroing colors, causing black text on black background. (#2651) 2019-10-02 16:04:59 -07:00
OpenConsole.psm1 doc, tools: Improve docs around using clang-format with VS (#9782) 2021-04-21 10:51:58 -05:00
openps.cmd Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
openvt.cmd make copying of files windows localization agnostic (#741) 2019-05-21 16:25:54 +00:00
packages.config Update clang-format to 10.0 (#7389) 2020-08-25 17:15:43 +00:00
razzle.cmd Change TAEF nuget package to use new Microsoft.Taef name; Update to 10.58 release build version. (#9656) 2021-03-30 10:58:11 +00:00
README.md Add support for renaming windows (#9662) 2021-04-02 16:00:04 +00:00
runformat.cmd add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
runft.cmd Correct paths in the the runut and runft test scripts (#8488) 2020-12-03 10:38:18 -08:00
runuia.cmd Turn on Text Buffer unit tests in Azure DevOps CI build (#1057) 2019-05-29 19:51:17 -07:00
runut.cmd Split TermControl into a Core, Interactivity, and Control layer (#9820) 2021-04-27 15:50:45 +00:00
runxamlformat.cmd Auto-format our XAML files and enforce in CI (#9589) 2021-03-29 17:09:38 -05:00
testcon.cmd Fix a bunch of spelling errors across the project (#4295) 2020-02-10 20:40:01 +00:00
tests.xml Split TermControl into a Core, Interactivity, and Control layer (#9820) 2021-04-27 15:50:45 +00:00
vso_ut.cmd Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
WindbgExtension.js Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00

OpenConsole Tools

These are a collection of tools and scripts to make your life building the OpenConsole project easier. Many of them are designed to be functional clones of tools that we used to use when developing inside the Windows build system.

Razzle

This is a script that quickly sets up your environment variables so that these tools can run easily. It's named after another script used by Windows developers to similar effect.

  • It adds msbuild to your path.
  • It adds the tools directory to your path as well, so all these scripts are easily available.
  • It executes \tools\.razzlerc.cmd to add any other personal configuration to your environment as well, or creates one if it doesn't exist.
  • It sets up the default build configuration to be 'Debug'. If you'd like to manually specify a build configuration, pass the parameter dbg for Debug, and rel for Release.

bcz

bcz can quick be used to clean and build the project. By default, it builds the %DEFAULT_CONFIGURATION% configuration, which is Debug if you use razzle.cmd.

  • bcz dbg can be used to manually build the Debug configuration.
  • bcz rel can be used to manually build the Release configuration.

opencon (and openbash, openps)

opencon can be used to launch the last built OpenConsole binary. If given an argument, it will try and run that program in the launched window. Otherwise it will default to cmd.exe.

openbash is similar, it immediately launches bash.exe (the Windows Subsystem for Linux entrypoint) in your ~ directory.

Likewise, openps launches powershell.

runformat & runxamlformat

runxamlformat will format .xaml files to match our coding style. runformat will format the c++ code (and will also call runxamlformat). runformat should be called before making a new PR, to ensure that code is formatted correctly. If it isn't, the CI will prevent your PR from merging.

The C++ code is formatted with clang-format. Many editors have built-in support for automatically running clang-format on save.

Our XAML code is formatted with XamlStyler. I don't have a good way of running this on save, but you can add a git hook to format before committing .xaml files. To do so, add the following to your .git/hooks/pre-commit file:

# XAML Styler - xstyler.exe pre-commit Git Hook
# Documentation: https://github.com/Xavalon/XamlStyler/wiki
# Originally from https://github.com/Xavalon/XamlStyler/wiki/Git-Hook

# Define path to xstyler.exe
XSTYLER_PATH="dotnet tool run xstyler --"

# Define path to XAML Styler configuration
XSTYLER_CONFIG="XamlStyler.json"

echo "Running XAML Styler on committed XAML files"
git diff --cached --name-only --diff-filter=ACM  | grep -e '\.xaml$' | \
# Wrap in brackets to preserve variable through loop
{
    files=""
    # Build list of files to pass to xstyler.exe
    while read FILE; do
        if [ "$files" == "" ]; then
            files="$FILE";
        else
            files="$files,$FILE";
        fi
    done

    if [ "$files" != "" ]; then
        # Check if external configuration is specified
        [ -z "$XSTYLER_CONFIG" ] && configParam="" || configParam="-c $XSTYLER_CONFIG"

        # Format XAML files
        $XSTYLER_PATH -f "$files" $configParam

        for i in $(echo $files | sed "s/,/ /g")
        do
            #strip BOM
            sed -i '1s/^\xEF\xBB\xBF//' $i
            unix2dos $i
            # stage updated file
            git add -u $i
        done
    else
        echo "No XAML files detected in commit"
    fi

    exit 0
}

testcon, runut, runft

runut will automatically run all of the unit tests through TAEF. runft will run the feature tests, and testcon runs all of them. They'll pass any arguments through to TAEF, so you can more finely control the testing.

A recommended workflow is the following command:

bcz dbg && runut /name:*<name of test>*

Where <name of test> is the name of the test testing the relevant feature area you're working on. For example, if I was working on the VT Mouse input support, I would use MouseInputTest as that string, to isolate the mouse input tests. If you'd like to run all the tests, just ignore the /name param: bcz dbg && runut

To make sure your code is ready for a pull request, run the build, then launch the built console, then run the tests in it. The built console will inherit all of the razzle environment, so you can immediately start using the macros:

  1. bcz
  2. opencon
  3. testcon (in the new console window)
  4. runformat

If they all come out green, then you're ready for a pull request!