Commit Graph

69 Commits

Author SHA1 Message Date
SaintMalik 1acfef60f6
Fix typos found in terminal/oss (#11048) 2021-08-26 16:40:26 -05:00
Dustin L. Howett 92437d718f
build: propagate PGOBuildMode into final MSBuild command (#11054) 2021-08-26 11:30:49 -05:00
Michael Niksa 817f598e20
Move PGO Helix pools (#11028)
Moves PGO runs to supported Helix pools. We need to match Microsoft-UI-XAML on which Helix pools we used for each type of activities.

## PR Checklist
* [x] Closes #10850
* [x] I work here
* [x] If it builds, it sits.

## Validation Steps Performed
* [x] Run PGO build against this branch
2021-08-25 22:58:06 +00:00
Michael Niksa 2eb659717c
Move to 1ES engineering pools (#10854)
Move to 1ES engineering pools

## PR Checklist
* [x] Closes #10734
* [x] I work here
* [x] If the builds still work, the tests pass. (release and PR builds...)

## Validation Steps Performed
- [x] Run the builds associated with this PR
- [x] Force run a release build off this branch
- [x] Force run a PGO training build off this branch
2021-08-04 17:00:41 +00:00
Dustin L. Howett d43a14c63f
Replace the placeholder release build with our real one (#10778)
This pull request ports our old release pipeline from Azure DevOps' editor to real YAML.

It includes the following changes on top of a straight-up "export" from Azure:

- Converts all queue-time variables into form-based parameters
- Adds a "matrix" build strategy for Configs * Platforms
- Renames all jobs to have reasonable names
- The YAML generator has a bug where it inlines scripts *and* file paths if a task had both; remove old inlines
- Removes dead rules
- Fixes the WPF build to include the apiset impostor
- Migrates the access token into the environment for the one build stage that needs it
- Cleans up some of the online script logic
- Removes all of the "!is pull request?" checks
2021-07-27 01:14:59 +00:00
Leonard Hecker 32fbd4cbb6
Enable /Zc:preprocessor (#10593)
This commit is a preparation for upcoming changes to KeyChordSerialization for #7539 and #10203.
In order to support variadic macros, /Zc:preprocessor was enabled, which required changing unrelated parts of the project.

## PR Checklist
* [x] I work here
* [x] Tests added/passed

## Validation Steps Performed

* Project still compiles ✔️
2021-07-13 23:00:11 +00:00
Dustin L. Howett 92f0700d5d
Force the execution policy on the til::feature powershell script (#10411)
It was not set to `bypass`, which gave @carlos-zamora some trouble.

This commit brings it in line with the Windows build rule.
2021-06-11 20:14:18 +00:00
Dustin L. Howett 31a39b3b12
Add support for branch- and branding-based feature flagging (#10361)
This pull request implements a "feature flagging" system that will let
us turn Terminal and conhost features on/off by branch, "release" status
or branding (Dev, Preview, etc.).

It's loosely modelled after the Windows OS concept of "Velocity," but
only insofar as it is driven by an XML document and there's a tool that
emits a header file for you to include.

It only supports toggling features at compile time, and the feature flag
evaluators are intended to be fully constant expressions.

Features are added to `src\features.xml` and marked with a "stage". For
now, the only stages available are `AlwaysDisabled` and `AlwaysEnabled`.
Features can be toggled to different states using branch and branding
tokens, as documented in the included feature flag docs.

For a given feature Feature_XYZ, we will emit two fixtures visible to
the compiler:

1. A preprocessor define `TIL_FEATURE_XYZ_ENABLED` (usable from MIDL,
   C++ and C)
2. A feature class type `Feature_XYZ` with a static constexpr member
   `IsEnabled()` (usable from C++, designed for `if constexpr()`).

Like Velocity, we rely on the compiler to eliminate dead code caused by
things that compile down to `if constexpr (false)`. :)

Michael suggested that we could use `WindowsInbox` as a branding to
determine when we were being built inside Windows to supplant our use of
the `__INSIDE_WINDOWS` preprocessor token. It was brilliant.

Design Decisions
----------------

* Emitting the header as part of an MSBuild project
   * WHY: This allows the MSBuild engine to ensure that the build is
     only run once, even in a parallel build situation.
* Only having one feature flag document for the entire project
   * WHY: Ease.
* Forcibly including `TilFeatureStaging` with `/FI` for all CL compiler
  invocations.
   * WHY: If this is a project-wide feature system, we should make it as
     easy as possible to use.
* Emitting preprocessor definitions instead of constexpr/consteval
   * WHY: Removing entire functions/includes is impossible with `if
     constexpr`.
   * WHY: MIDL cannot use a `static constexpr bool`, but it can rely on
     the C preprocessor to remove text.
* Using MSBuild to emit the text instead of PowerShell
   * WHY: This allows us to leverage MSBuild's `WriteOnlyWhenDifferent`
     task parameter to avoid changing the file's modification time when
     it would have resulted in the same contents. This lets us use the
     same FeatureStaging header across multiple builds and multiple
     branches and brandings _assuming that they do not result in a
     feature flag change_.
   * The risk in using a force-include is always that it, for some
     reason, determines that the entire project is out of date. We've
     gone to great lengths to make sure that it only does so if the
     features _actually materially changed_.
2021-06-10 23:09:52 +00:00
Michael Niksa 235f011d6c
Restore helix runs by passing parameters (#10367)
Restore helix runs by passing parameters

## PR Checklist
* [x] Closes #10266 
* [x] I work here
* [x] Tests added/passed

## Detailed Description of the Pull Request / Additional comments
It looks like at some point the configuration and platform variables got messed up passing into the Helix steps and preventing the Powershell scripts from setting up the Helix payload correctly. This restores them to functionality.

## Validation Steps Performed
- Ran it in the pipeline
2021-06-08 18:57:30 +00:00
Luan Vitor Simião Oliveira 504e610841
openConsole.psm1: fix unapproved verb warning (#10145)
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request

<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> 
## References

<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes #10097
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments

<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
2021-05-21 16:11:54 +00:00
Leonard Hecker eaeab7a807
Upgrade Windows SDK to 19041 (#10118)
## Summary of the Pull Request

Upgrade the Windows SDK to 19041 by setting `WindowsTargetPlatformMinVersion` to 17763 and `WindowsTargetPlatformVersion` to 19041.

## PR Checklist
* [x] I work here
* [x] Tests added/passed

## Validation Steps Performed

General usage of the Windows Terminal application appears fine.
2021-05-20 16:04:25 +00:00
Michael Niksa 7dadde5dd6
Implement PGO in pipelines for AMD64 architecture; supply training test scenarios (#10071)
Implement PGO in pipelines for AMD64 architecture; supply training test scenarios

## References
- #3075 - Relevant to speed interests there and other linked issues.

## PR Checklist
* [x] Closes #6963
* [x] I work here.
* [x] New UIA Tests added and passed. Manual build runs also tested.

## Detailed Description of the Pull Request / Additional comments
- Creates a new pipeline run for creating instrumented binaries for Profile Guided Optimization (PGO).
- Creates a new suite of UIA tests on the full Windows Terminal app to run PGO training scenarios on instrumented binaries (and incidentally can be used to write other UIA tests later for the full Terminal app.)
- Creates a new NuGet artifact to store trained PGO databases (PGD files) at `Microsoft.Internal.Windows.Terminal.PGODatabase`
- Creates a new NuGet artifact to supply large-scale test content for automated tests at `Microsoft.Internal.Windows.Terminal.TestContent`
- Adjusts the release pipeline to run binaries in PGO optimized mode where content from PGO databases is leveraged at link time to optimize the final release build

The following binaries are trained:
- OpenConsole.exe
- WindowsTerminal.exe
- TerminalApp.dll
- TerminalConnection.dll
- Microsoft.Terminal.Control.dll
- Microsoft.Terminal.Remoting.dll
- Microsoft.Terminal.Settings.Editor.dll
- Microsoft.Terminal.Settings.Model.dll

In the future, adding `<PgoTarget>true</PgoTarget>` to a new `vcxproj` file will automatically enroll the DLL/EXE for PGO instrumentation and optimization going forward.

Two training test scenarios are implemented:
- Smoke test the Terminal by just opening it and typing a bit of text then exiting. (Should help focus on the standard launch path.)
- Optimize bulk text output by launching terminal, outputting `big.txt`, then exiting.

Additional scenarios can be contributed to the `WindowsTerminal_UIATests` project with the `[TestProperty("IsPGO", "true")]` annotation to add them to the suite of scenarios for PGO.

**NOTE:** There are currently no weights applied to the various test scenarios. We will revisit that in the future when/if necessary.

## Validation Steps Performed
- [x] - Training run completed at https://dev.azure.com/ms/terminal/_build?definitionId=492&_a=summary
- [x] - Optimization run completed locally (by forcing `PGOBuildMode` to `Optimize` on my local machine, manually retrieving the databases with NuGet, and building).
- [x] - Validated locally that x86 and ARM64 do not get trained and automatically skip optimization as databases are not present for them.
- [x] - Smoke tested optimized binary versus latest releases. `big.txt` output through CMD is ~11-12seconds prior to PGO and just over 8 seconds with PGO.
2021-05-13 21:12:30 +00:00
Dustin L. Howett e0bd76b30d
Make sure we run CG component detection on main (#10078)
I believe that there is a hidden "default branch" setting in Azure
DevOps, and ours is set to "master". CG only attempts to automatically
inject to builds off the default branch... and because we're a GitHub
repo running builds in AzDO we _can't change what the default branch
is_. Oops.
2021-05-12 02:47:45 +00:00
Mike Griese 8910a16fd0
Split `TermControl` into a Core, Interactivity, and Control layer (#9820)
## Summary of the Pull Request

Brace yourselves, it's finally here. This PR does the dirty work of splitting the monolithic `TermControl` into three components. These components are: 

* `ControlCore`: This encapsulates the `Terminal` instance, the `DxEngine` and `Renderer`, and the `Connection`. This is intended to everything that someone might need to stand up a terminal instance in a control, but without any regard for how the UX works.
* `ControlInteractivity`: This is a wrapper for the `ControlCore`, which holds the logic for things like double-click, right click copy/paste, selection, etc. This is intended to be a UI framework-independent abstraction. The methods this layer exposes can be called the same from both the WinUI TermControl and the WPF control.
* `TermControl`: This is the UWP control. It's got a Core and Interactivity inside it, which it uses for the actual logic of the terminal itself. TermControl's main responsibility is now 

By splitting into smaller pieces, it will enable us to
* write unit tests for the `Core` and `Interactivity` bits, which we desparately need
* Combine `ControlCore` and `ControlInteractivity` in an out-of-proc core process in the future, to enable tab tearout.

However, we're not doing that work quite yet. There's still lots of work to be done to enable that, thought this is likely the biggest portion.

Ideally, this would just be methods moved wholesale from one file to another. Unfortunately, there are a bunch of cases where that didn't work as well as expected. Especially when trying to better enforce the boundary between the classes. 

We've got a couple tests here that I've added. These are partially examples, and partially things I ran into while implementing this. A bunch of things from #7001 can go in now that we have this.

This PR is gonna be a huge pain to review - 38 files with 3,730 additions and 1,661 deletions is nothing to scoff at. It will also conflict 100% with anything that's targeting `TermControl`. I'm hoping we can review this over the course of the next week and just be done with it, and leave plenty of runway for 1.9 bugs in post.

## References

* In pursuit of #1256
* Proc Model: #5000
* https://github.com/microsoft/terminal/projects/5

## PR Checklist
* [x] Closes #6842
* [x] Closes https://github.com/microsoft/terminal/projects/5#card-50760249
* [x] Closes https://github.com/microsoft/terminal/projects/5#card-50760258
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated

## Detailed Description of the Pull Request / Additional comments

* I don't love the names `ControlCore` and `ControlInteractivity`. Open to other names.
* I added a `ICoreState` interface for "properties that come from the `ControlCore`, but consumers of the `TermControl` need to know". In the future, these will all need to be handled specially, because they might involve an RPC call to retrieve the info from the core (or cache it) in the window process.
* I've added more `EventArgs` to make more events proper `TypedEvent`s.
* I've changed how the TerminalApp layer requests updated TaskbarProgress state. It doesn't need to pump TermControl to raise a new event anymore.
* ~~Something that snuck into this branch in the very long history is the switch to `DCompositionCreateSurfaceHandle` for the `DxEngine`. @miniksa wrote this originally in 30b8335, I'm just finally committing it here. We'll need that in the future for the out-of-proc stuff.~~
  * I reverted this in c113b65d9. We can revert _that_ commit when we want to come back to it.
* I've changed the acrylic handler a decent amount. But added tests!
* All the `ThrottledFunc` things are left in `TermControl`. Some might be able to move down into core/interactivity, but once we figure out how to use a different kind of Dispatcher (because a UI thread won't necessarily exist for those components).
* I've undoubtably messed up the merging of the locking around the appearance config stuff recently

## Validation Steps Performed

I've got a rolling list in https://github.com/microsoft/terminal/issues/6842#issuecomment-810990460 that I'm updating as I go.
2021-04-27 15:50:45 +00:00
Michael Niksa c7d2a818b0
Change TAEF nuget package to use new Microsoft.Taef name; Update to 10.58 release build version. (#9656)
Change TAEF nuget package to use new Microsoft.Taef name; Update to 10.58 release build version.

## PR Checklist
* [x] Closes email from Phil letting us know TAEF has a new release build and a rename.
* [x] Closes annoying duplicate TAEF import warning in `Parser.UnitTests.vcxproj`
* [x] I work here.
* [ ] Need to see the tests run off CI to confirm this is fine for those environments and Helix

## Validation Steps Performed
* [x] Build/run tests locally
* [ ] Build/run unit and feature tests in CI
* [ ] Build/run Helix-lab tests in CI
2021-03-30 10:58:11 +00:00
Mike Griese 3323dc5724
Auto-format our XAML files and enforce in CI (#9589)
This adds [`XamlStyler.Console`] to our solution, and calls it when we
format the code, to also format
our .xaml files. 
* `XamlStyler.Console` is a dotnet tool so it needs to be restored with
  `dotnet tool restore`
* I've added a set of rules to approximately follow [@cmaneu's XAML guidelines].
  Those guidelines also recommend things based on the code-behind, which
  this tool can't figure out, but also _don't matter that much_.
* There's an extra step to strip BOMs from the output, since Xaml Styler
  adds a BOM by default. Some had them before and others didn't. BOMs
  have been nothing but trouble though.

[`XamlStyler.Console`]: https://github.com/Xavalon/XamlStyler
[@cmaneu's XAML guidelines]: https://github.com/cmaneu/xaml-coding-guidelines
2021-03-29 17:09:38 -05:00
Mike Griese d749df70ed
Rename `Microsoft.Terminal.TerminalControl` to `.Control`; Split into dll & lib (#9472)
**BE NOT AFRAID**. I know that there's 107 files in this PR, but almost
all of it is just find/replacing `TerminalControl` with `Control`.

This is the start of the work to move TermControl into multiple pieces,
for #5000. The PR starts this work by:
* Splits `TerminalControl` into separate lib and dll projects. We'll
  want control tests in the future, and for that, we'll need a lib.
* Moves `ICoreSettings` back into the `Microsoft.Terminal.Core`
  namespace. We'll have other types in there soon too. 
  * I could not tell you why this works suddenly. New VS versions? New
    cppwinrt version? Maybe we're just better at dealing with mdmerge
    bugs these days.
* RENAMES  `Microsoft.Terminal.TerminalControl` to
  `Microsoft.Terminal.Control`. This touches pretty much every file in
  the sln. Sorry about that (not sorry). 

An upcoming PR will move much of the logic in TermControl into a new
`ControlCore` class that we'll add in `Microsoft.Terminal.Core`.
`ControlCore` will then be unittest-able in the
`UnitTests_TerminalCore`, which will help prevent regressions like #9455 

## Detailed Description of the Pull Request / Additional comments
You're really gonna want to clean the sln first, then merge this into
your branch, then rebuild. It's very likely that old winmds will get
left behind. If you see something like 

```
Error    MDM2007    Cannot create type
Microsoft.Terminal.TerminalControl.KeyModifiers in read-only metadata
file Microsoft.Terminal.TerminalControl.
```

then that's what happened to you.
2021-03-17 20:47:24 +00:00
Mike Griese 7d503a4352
Add `Microsoft.Terminal.Remoting.dll` (#8607)
Adds a `Microsoft.Terminal.Remoting.dll` to our solution. This DLL will
be responsible for all the Monarch/Peasant work that's been described in
#7240 & #8135. 

This PR does _not_ implement the Monarch/Peasant architecture in any
significant way. The goal of this PR is to just to establish the project
layout, and the most basic connections. This should make reviewing the
actual meat of the implementation (in a later PR) easier. It will also
give us the opportunity to include some of the basic weird things we're
doing (with `CoRegisterClass`) in the Terminal _now_, and get them
selfhosted, before building on them too much.

This PR does have windows registering the `Monarch` class with COM. When
windows are created, they'll as the Monarch if they should create a new
window or not. In this PR, the Monarch will always reply "yes, please
make a new window".

Similar to other projects in our solution, we're adding 3 projects here:
* `Microsoft.Terminal.Remoting.lib`: the actual implementation, as a
  static lib.
* `Microsoft.Terminal.Remoting.dll`: The implementation linked as a DLL,
  for use in `WindowsTerminal.exe`.
* `Remoting.UnitTests.dll`: A unit test dll that links with the static
  lib. 

There are plenty of TODOs scattered about the code. Clearly, most of
this isn't implemented yet, but I do have more WIP branches. I'm using
[`projects/5`](https://github.com/microsoft/terminal/projects/5) as my
notation for TODOs that are too small for an issue, but are part of the
whole Process Model 2.0 work.

## References

* #5000 - this is the process model megathread
* #7240 - The process model 2.0 spec.
* #8135 - the window management spec. (please review me, I have 0/3
  signoffs even after the discussion we had 😢)
* #8171 - the Monarch/peasant sample. (please review me, I have 1/2)

## PR Checklist
* [x] Closes nothing, this is just infrastructure
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
2021-01-07 22:59:37 +00:00
Dustin L. Howett 0bf9dcb63e
Delete the old PackageES code signing configs (#8536)
We've moved to the recommended internal code signing service
and no longer need these configurations.
2020-12-09 12:50:42 -08:00
Dustin L. Howett 4daed9d946
Move CI and audit build to the WinDev scale set pool (#8080)
This pull request switches us to the new WinDev scaleset agent pool. It
should be faster than the hosted pool, and the larger disks allow us to
get rid of our PCH cleanup step.
2020-10-28 20:49:13 +00:00
Dustin L. Howett 403b793179
Prepare for the primary branch name to change to main (#7985) 2020-10-21 17:29:36 -07:00
Dustin L. Howett ae550e0969
Expose WindowsTerminalBranding to the preprocessor (optionally) (#7986)
Our build pipeline was originally set up such that we could take any
binaries from the Terminal build and seamlessly re-package them with the
release or preview livery. My initial plan was to stamp a stable and
preview build at the same time, out of the same bits, to make ring
promotion easier.

I've never done that. For the last five releases, we've just re-cut a
new stable build along with the new preview build, usually because we
want to backport some fixes to stable.

This commit introduces preprocessor defines, detectable through CL and
RC, for any project that wants them. Right now, that's just going to be
WindowsTerminal.vcxproj (since it hosts the icons and the app entry
point). This list may be extended to include wt (the shim executable)
and the shell extension at some future date.

This will greatly simplify the logic in #7971, as we'll no longer need
to detect if we're dev or preview at runtime. It may also simplify the
logic in the shell extension for determining whether we're Dev or not.
2020-10-21 21:43:53 +00:00
Dustin L. Howett e996fadbb6
conpty: fall back to conhost if OpenConsole is missing (#7741)
This commit is in support of WTU.

I initially added support for a new flag, `PSEUDOCONSOLE_UNDOCKED_PREFER_INBOX_CONHOST`,
which I liked because it was more explicit. We chose not to go that route.

### Automatic fallback
#### Pros
* It's easier on the consumer
* We can eventually expand it to support `$ARCH/openconsole.exe`
#### Cons
* Packaging the project wrong will result in a working-but-somewhat-broken experience (old conhost)
   * We ameliorated this by checking it in the packaging script.
* Implicit behavior may be bad
2020-10-15 11:50:27 -07:00
Dustin Howett 91ccbb79f0
BUILD: Disable parallel build
The build agents can't handle the size of our PCH files.

Signed-off-by: Dustin Howett <duhowett@microsoft.com>
2020-10-08 17:17:55 -07:00
Dustin L. Howett da4ca86680
Fix parallel builds by specifying the application type for WAP (#7783)
The WAP packaging project is sensitive to including applications that it
thinks are UWPs. The changes we made to separate WindowsStoreApp and
WindowsAppContainer weren't comprehensive enough to convince WAP that we
were not still UWPs.

Because of that, it would run sub-builds of each of these projects (and
all their dependencies) with an additional `GenerateAppxPackageOnBuild`
property set. The existence of this property caused MSBuild to think the
projects needed to be built *again*.
2020-09-30 13:25:50 -07:00
Dustin Howett 6c0e6d94cd
ci: run CI and triggered builds on feature/* 2020-08-27 17:29:14 -07:00
Michael Niksa 5a0deca3d8
Set ProcessTestResults job to use conditions specified in parent (#7347)
Activating a template doesn't actually process conditions. Only jobs, stages, and tasks can process a condition. So specify the full condition in the parent template call as a parameter and ask the child job (who can actually evaluate the condition) to use that parameter to determine if it should run.
2020-08-19 19:31:03 +00:00
Michael Niksa 5d082ffe67
Helix Testing (#6992)
Use the Helix testing orchestration framework to run our Terminal LocalTests and Console Host UIA tests.

## References
#### Creates the following new issues:
- #7281 - re-enable local tests that were disabled to turn on Helix
- #7282 - re-enable UIA tests that were disabled to turn on Helix
- #7286 - investigate and implement appropriate compromise solution to how Skipped is handled by MUX Helix scripts

#### Consumes from:
- #7164 - The update to TAEF includes wttlog.dll. The WTT logs are what MUX's Helix scripts use to track the run state, convert to XUnit format, and notify both Helix and AzDO of what's going on.

#### Produces for:
- #671 - Making Terminal UIA tests is now possible
- #6963 - MUX's Helix scripts are already ready to capture PGO data on the Helix machines as certain tests run. Presuming we can author some reasonable scenarios, turning on the Helix environment gets us a good way toward automated PGO.

#### Related:
- #4490 - We lost the AzDO integration of our test data when I moved from the TAEF/VSTest adapter directly back to TE. Thanks to the WTTLog + Helix conversion scripts to XUnit + new upload phase, we have it back!

## PR Checklist
* [x] Closes #3838
* [x] I work here.
* [x] Literally adds tests.
* [ ] Should I update a testing doc in this repo?
* [x] Am core contributor. Hear me roar.
* [ ] Correct spell-checking the right way before merge.

## Detailed Description of the Pull Request / Additional comments
We have had two classes of tests that don't work in our usual build-machine testing environment:
1. Tests that require interactive UI automation or input injection (a.k.a. require a logged in user)
2. Tests that require the entire Windows Terminal to stand up (because our Xaml Islands dependency requires 1903 or later and the Windows Server instance for the build is based on 1809.)

The Helix testing environment solves both of these and is brought to us by our friends over in https://github.com/microsoft/microsoft-ui-xaml.

This PR takes a large portion of scripts and pipeline configuration steps from the Microsoft-UI-XAML repository and adjusts them for Terminal needs.
You can see the source of most of the files in either https://github.com/microsoft/microsoft-ui-xaml/tree/master/build/Helix or https://github.com/microsoft/microsoft-ui-xaml/tree/master/build/AzurePipelinesTemplates

Some of the modifications in the files include (but are not limited to) reasons like:
- Our test binaries are named differently than MUX's test binaries
- We don't need certain types of testing that MUX does.
- We use C++ and C# tests while MUX was using only C# tests (so the naming pattern and some of the parsing of those names is different e.g. :: separators in C++ and . separators in C#)
- Our pipeline phases work a bit differently than MUX and/or we need significantly fewer pieces to the testing matrix (like we don't test a wide variety of OS versions).

The build now runs in a few stages:
1. The usual build and run of unit tests/feature tests, packaging verification, and whatnot. This phase now also picks up and packs anything required for running tests in Helix into an artifact. (It also unifies the artifact name between the things Helix needs and the existing build outputs into the single `drop` artifact to make life a little easier.)
2. The Helix preparation build runs that picks up those artifacts, generates all the scripts required for Helix to understand the test modules/functions from our existing TAEF tests, packs it all up, and queues it on the Helix pool.
3. Helix generates a VM for our testing environment and runs all the TAEF tests that require it. The orchestrator at helix.dot.net watches over this and tracks the success/fail and progress of each module and function. The scripts from our MUX friends handle installing dependencies, making the system quiet for better reliability, detecting flaky tests and rerunning them, and coordinating all the log uploads (including for the subruns of tests that are re-run.)
4. A final build phase is run to look through the results with the Helix API and clean up the marking of tests that are flaky, link all the screenshots and console output logs into the AzDO tests panel, and other such niceities.

We are set to run Helix tests on the Feature test policy of only x64 for now. 

Additionally, because the set up of the Helix VMs takes so long, we are *NOT* running these in PR trigger right now as I believe we all very much value our 15ish minute PR turnaround (and the VM takes another 15 minutes to just get going for whatever reason.) For now, they will only run as a rolling build on master after PRs are merged. We should still know when there's an issue within about an hour of something merging and multiple PRs merging fast will be done on the rolling build as a batch run (not one per).

In addition to setting up the entire Helix testing pipeline for the tests that require it, I've preserved our classic way of running unit and feature tests (that don't require an elaborate environment) directly on the build machines. But with one bonus feature... They now use some of the scripts from MUX to transform their log data and report it to AzDO so it shows up beautifully in the build report. (We used to have this before I removed the MStest/VStest wrapper for performance reasons, but now we can have reporting AND performance!) See https://dev.azure.com/ms/terminal/_build/results?buildId=101654&view=ms.vss-test-web.build-test-results-tab for an example. 

I explored running all of the tests on Helix but.... the Helix setup time is long and the resources are more expensive. I felt it was better to preserve the "quick signal" by continuing to run these directly on the build machine (and skipping the more expensive/slow Helix setup if they fail.) It also works well with the split between PR builds not running Helix and the rolling build running Helix. PR builds will get a good chunk of tests for a quick turn around and the rolling build will finish the more thorough job a bit more slowly.

## Validation Steps Performed
- [x] Ran the updated pipelines with Pull Request configuration ensuring that Helix tests don't run in the usual CI
- [x] Ran with simulation of the rolling build to ensure that the tests now running in Helix will pass. All failures marked for follow on in reference issues.
2020-08-18 18:23:24 +00:00
Dustin L. Howett c4a9752be1
Disable parallel build (again) and keep TerminalApp PCHs (#7322)
The build now builds every project multiple times, so I figure, why not
try to fix it.
2020-08-18 00:01:50 +00:00
Dustin L. Howett 592c634577
Build and ship an actual binary named wt that just launches WT (#6860)
Due to a shell limitation, Ctrl+Shift+Enter will not launch Windows
Terminal as Administrator. This is caused by the app execution alias and
the actual targeted executable not having the same name.

In addition, PowerShell has an issue detecting app execution aliases as
GUI/TUI applications. When you run wt from PowerShell, the shell will
wait for WT to exit before returning to the prompt. Having a shim that
immediately re-executes WindowsTerminal and then returns handily knocks
this issue out (as the process that PS was waiting for exits
immediately.)

This could cause a regression for anybody who tries to capture the PID
of wt.exe. Our process tree is not an API, and we have offered no
consistency guarantee on it.

VALIDATION
----------

Tested manual launch in a number of different scenarios:

* [x] start menu "wtd"
* [x] start menu tile
* [x] powertoys run
* [x] powertoys run ctrl+shift (admin)
* [x] powershell inbox, "core"
* [x] cmd
* [x] run dialog
* [x] run dialog ctrl+shift (admin)
* [x] run from a lnk with window mode=maximized

Fixes #4645 (PowerShell waits for wt)
Fixes #6625 (Can't launch as admin using C-S-enter)
2020-07-10 22:41:37 +00:00
Dustin L. Howett 60630cf7c4
Fix the x86 build and re-enable x86 CI (#6467)
This was a miss.
2020-06-11 17:04:42 +00:00
Dustin L. Howett e3ee5838a7
Move all wildcards into targets or expand them (#6406)
Wildcards are not allowed in toplevel ItemGroups in vcxproj; they must
be generated by targets.

We mostly use wildcards for pulling in PRI files that are dumped on disk
by the translation tool. We don't want to check those in, so we can't
expand references to them.

To that end, I've introduced a new target that will take a list of
folders containing resw files and expand wildcards under them.

All[1] other wildcards have been moved into their respective targets
_or_ simply expanded.

[1]: Nothing has complained about the resource wildcards in
CascadiaResources.build.items, so I haven't exploded it yet.

Fixes #6214.
2020-06-08 14:01:47 -07:00
Dustin L. Howett (MSFT) e3fdb1a1a3
build: merge the signing steps and wildcard them (#5817)
This allows me to make the build pipeline, instead of the release
engineer, put the version number in the package name.

It also lets us sign multiple packages (if we ever produce more than
one.)
2020-05-08 14:11:10 -07:00
Carlos Zamora 9a927f3a0f
Add a new appxmanifest for preview (#5774)
## Summary of the Pull Request
This adds a new appxmanifest for 'Windows Terminal (Preview)' and links the resources.

Code-wise, split up `WindowsTerminalReleaseBuild` into...
- WindowsTerminalOfficialBuild: [true, false]
- WindowsTerminalBranding: [Dev, Preview, Release]

Added a comment about that in release.yml

## Validation Steps Performed
used msbuild to build...
- [X] Dev
- [X] Preview
- [X] Release
then checked the msix for the correct name/icon.
2020-05-07 16:00:56 +00:00
Dustin L. Howett (MSFT) d57ef135cc
On second thought, embed the third-party notices in the package (#5673)
This commit introduces a NOTICE.html file that will be embedded into the
package. It will be stamped down with the real notices during a branded
release build (as part of the build pipeline.)

It, in part, reverts some of the really good work in determining the
commit hash at build time. That work will be preserved in history.

This is more compliant with our duties to the OSS we consume.
2020-04-30 15:06:13 -07:00
Dustin L. Howett (MSFT) 3246c4250a
Bundle the entire CRT with our branded Release packages (#5661)
For our release builds, we're just going to integrate the UWPDesktop CRT
into our package and delete the package dependencies.  It's very
difficult for users who do not have access to the store to get our
dependency packages, and we want to be robust and deployable everywhere.
Since these libraries can be redistributed, it's easiest if we simply
redistribute them.

Our package grows by ~550kb per architecture (compressed) because of
this. I've added validation that we don't have both the libs _and_ the
dependencies in the same package.

Fixes #3097.

## Validation

The script does it!
2020-04-30 07:08:43 +00:00
Michael Niksa b1a5604b55
Split rolling and PR builds. Drop ARM64, X86 from PR. (#5256)
## Summary of the Pull Request
Edits the definition file to distinguish further between the rolling build (the one that happens in master as it's updated) and the PR builds (that happen on every push to a pull request to master.) We will build less in PR since it rolls so often by removing the lines that reveal very few to no bugs at PR time. We'll leave them on in rolling so stuff can still be caught.

## PR Checklist
* [x] Closes a desire to not waste builds.
* [x] I work here.
* [ ] We'll see if the build still works.
* [x] No specific docs.
* [x] I talked about this with @DHowett-MSFT already.

## Validation Steps Performed
* [x] This PR itself should validate that the definition still works in PRs. I think we have to wait for it to go to master to see if the trigger still works there.
2020-04-06 22:11:03 +00:00
Dustin L. Howett (MSFT) 0be070f340
Prepare for automated localization (#5119)
This pull request includes a localization config file that identifies
the modules we need to localize. It also moves us back to the
`Resources\LANGUAGE\Resources.resw` resource layout, but using wildcards
so that the build system can pick up any number of languages.
2020-03-25 21:06:59 +00:00
Dustin L. Howett (MSFT) deccf7e12b
Batch up to 64 files per clang-format to speed it up (#4639)
This will make `Invoke-CodeFormat` less bad.
2020-02-19 13:47:56 +00:00
Michael Niksa 86706d7698
Move tests to invoke `te.exe` directly instead of using VSTest runner (#4490)
Moves the tests from using the `vstest.console.exe` route to just using `te.exe`.

PROs:
- `te.exe` is significantly faster for running tests because the TAEF/VSTest adapter isn't great.
- Running through `te.exe` is closer to what our developers are doing on their dev boxes
- `te.exe` is how they run in the Windows gates.
- `te.exe` doesn't seem to have the sporadic `0x6` error code thrown during the tests where somehow the console handles get lost
- `te.exe` doesn't seem to repro the other intermittent issues that we have been having that are inscrutable. 
- Fewer processes in the tree (te is running anyway under `vstest.console.exe`, just indirected a lot
- The log outputs scroll live with all our logging messages instead of suppressing everything until there's a failure
- The log output is actually in the order things are happening versus vstest.

CONs:
- No more code coverage.
- No more test records in the ADO build/test panel.
- Tests really won't work inside Visual Studio at all.
- The log files are really big now
- Testing is not a test task anymore, just another script.

Refuting each CON:
- We didn't read the code coverage numbers
- We didn't look at the ADO test panel results or build-over-build velocities
- Tests didn't really work inside Visual Studio anyway unless you did the right incantations under the full moon.
- We could tone down the logging if we wanted at either the te.exe execution time (with a switch) or by declaring properties in the tests/classes/modules that are very verbose to not log unless it fails.
- I don't think anyone cares how they get run as long as they do.
2020-02-10 19:14:06 +00:00
Dustin L. Howett (MSFT) 790277c909
Update TAEF to 10.51 and remove the private dep on Taef.TestAdapter (#4450)
This removes some longstanding debt we've been carrying around.
2020-02-03 22:14:43 +00:00
Michael Niksa e8658cd71e
Disable auto-injected codesign validation task on CI (#4387)
Sets flag in CI YML that will turn off the auto-injected codesign validation
task since CI is a non-production pipeline.
2020-01-28 15:54:08 -08:00
Dustin L. Howett (MSFT) 1e2f203395
ci: return to the original oneshot build config (#3918) 2019-12-11 13:53:11 -08:00
Dustin L. Howett (MSFT) c0111a706d
Add a signing config for NuGet packages (#3860) 2019-12-05 17:52:01 -08:00
Michael Niksa 7bcb06079e Enable signing and ARM64 for Universal Terminal (#3716) 2019-11-26 14:25:54 -08:00
Dustin L. Howett (MSFT) 58b52ef69e
replace vcpkg-cpprestsdk with a +ARM64 -ssl/boost +UWP version (#3489)
This new cpprestsdk package, 2.10.14, switches us to the app CRT.
cpprestsdk turns fof a bunch of boost and openssl dependencies when it's
built for the Windows Store subplatform, so we got a bunch of stuff for
free.

Incidentally, I fixed #2338 the real/correct way -- the build rules in
the package now make sure they're not using the system vcpkg root.
2019-11-08 14:17:11 -08:00
Michael Niksa ddcc06e911
Move project to app CRTs in preparation to run Universal (#3474)
* Change to use App CRT in preparation for universal.
* Try to make project build again by setting winconpty to static lib so it'll use the CRT inside TerminalConnection (or its other consumers) instead of linking its own.
* Remove test for conpty dll, it's a lib now. Add additional commentary on how CRT linking works for future reference. I'm sure this will come up again.
* use the _apiset variant until proven otherwise to match the existing one.
* Clarification in the comments for linking.
2019-11-08 14:09:39 -08:00
Dustin L. Howett (MSFT) 357e835f5d
Replace ConhostConnection with ConptyConnection (#3461)
This commit deletes ConhostConnection and replaces it with
ConptyConnection. The ConptyConnection uses CreatePseudoConsole and
depends on winconpty to override the one from kernel32.

* winconpty must be packageable, so I've added GetPackagingOutputs.
   * To validate this, I added conpty.dll to the MSIX regression script.
* I moved the code from conpty-universal that deals with environment
  strings into the types library.

This puts us in a way better place to implement #2563, as we can now
separately detect a failure to launch a pseudoconsole, a failure to
CreateProcess, and an unexpected termination of the launched process.

Fixes #1131.
2019-11-06 15:09:01 -08:00
Dustin L. Howett (MSFT) a34c47a493
Fix our parallel (and repeating) builds (#3412)
The WAP packaging project in VS <= 16.3.7 produces a couple global
properties as part of its normal operation that cause MSBuild to flag
our projects as out-of-date and requiring a rebuild. By forcing those
properties to match the WAP values, we can get consistent builds.

One of those properties, however, is "GenerateAppxPackageOnBuild", and
WAP sets it to *false*. When we set that, of course, we don't get an
MSIX out of our build pipeline. Therefore, we have to break our build
into two phases -- build, then package.

This required us to change our approach to PCH deletion. A project
without a PCH is *also* considered out-of-date. Now, we keep all PCH
files but truncate them to 0 bytes.

TerminalApp, however, is re-linked during packaging because the Xaml
compiler emits a new generated C++ file on every build. We have to keep
those PCHs around.

* Remove WpfTerminalControl AnyCPU from Arch-specific builds

This removes another source of build nondeterminism: that WpfTerminalControl was propagating TargetFramework into architecture-specific C++ builds. Its "Any CPU" platform has been removed from architecture builds at the solution level.

This also cleans up some new projects that were added and build for "Any
CPU".
2019-11-01 14:38:13 -07:00
Dustin L. Howett (MSFT) df26c677ef
Upgrade to Microsoft.UI.Xaml 2.2 (#3027)
* We had to move to the final API:
   * Items -> TabItems
   * Items.VectorChanged -> TabItemsChanged
   * TabClose -> TabCloseRequested
   * TabViewItem.Icon -> TabViewItem.IconSource
* TabRowControl has been converted to a ContentPresenter, which
  simplifies its logic a little bit.
* TerminalPage now differentiates MUX and WUX a little better
* Because of the change from Icon to IconSource in TabViewItem,
  Utils::GetColoredIcon needed to be augmented to support MUX IconSources.
  It was still necessary to use for WUX, so it's been templatized.
* I moved us from WUX SplitButton to MUX SplitButton and brought the
  style in line with the one typically provided by TabView.
* Some of our local controls have had their backgrounds removed so
  they're more amenable to being placed on other surfaces.
* I'm suppressing the TabView's padding.
* I removed a number of apparently dead methods from App.
* I've simplified the dragbar's sizing logic and eventing.
* The winmd harvester needed to be taught to not try to copy winmds for
  framework packages.
* We now only initialize the terminal once we know the size

Closes #1896.
Closes #444.
Closes #857.
Closes #771.
Closes #760.
2019-10-14 22:41:43 -07:00