terminal/build/Helix
Mike Griese 7d503a4352
Add Microsoft.Terminal.Remoting.dll (#8607)
Adds a `Microsoft.Terminal.Remoting.dll` to our solution. This DLL will
be responsible for all the Monarch/Peasant work that's been described in
#7240 & #8135. 

This PR does _not_ implement the Monarch/Peasant architecture in any
significant way. The goal of this PR is to just to establish the project
layout, and the most basic connections. This should make reviewing the
actual meat of the implementation (in a later PR) easier. It will also
give us the opportunity to include some of the basic weird things we're
doing (with `CoRegisterClass`) in the Terminal _now_, and get them
selfhosted, before building on them too much.

This PR does have windows registering the `Monarch` class with COM. When
windows are created, they'll as the Monarch if they should create a new
window or not. In this PR, the Monarch will always reply "yes, please
make a new window".

Similar to other projects in our solution, we're adding 3 projects here:
* `Microsoft.Terminal.Remoting.lib`: the actual implementation, as a
  static lib.
* `Microsoft.Terminal.Remoting.dll`: The implementation linked as a DLL,
  for use in `WindowsTerminal.exe`.
* `Remoting.UnitTests.dll`: A unit test dll that links with the static
  lib. 

There are plenty of TODOs scattered about the code. Clearly, most of
this isn't implemented yet, but I do have more WIP branches. I'm using
[`projects/5`](https://github.com/microsoft/terminal/projects/5) as my
notation for TODOs that are too small for an issue, but are part of the
whole Process Model 2.0 work.

## References

* #5000 - this is the process model megathread
* #7240 - The process model 2.0 spec.
* #8135 - the window management spec. (please review me, I have 0/3
  signoffs even after the discussion we had 😢)
* #8171 - the Monarch/peasant sample. (please review me, I have 1/2)

## PR Checklist
* [x] Closes nothing, this is just infrastructure
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
2021-01-07 22:59:37 +00:00
..
AzurePipelinesHelperScripts.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
ConvertWttLogToXUnit.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
EnsureMachineState.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
GenerateTestProjFile.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
global.json Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
HelixTestHelpers.cs Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
InstallTestAppDependencies.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
OutputFailedTestQuery.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
OutputSubResultsJsonFiles.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
OutputTestResults.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
packages.config Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
PrepareHelixPayload.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
ProcessHelixFiles.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
readme.md Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
runtests.cmd Add Microsoft.Terminal.Remoting.dll (#8607) 2021-01-07 22:59:37 +00:00
RunTestsInHelix.proj Helix Testing (#6992) 2020-08-18 18:23:24 +00:00
UpdateUnreliableTests.ps1 Helix Testing (#6992) 2020-08-18 18:23:24 +00:00

This directory contains code and configuration files to run WinUI tests in Helix.

Helix is a cloud hosted test execution environment which is accessed via the Arcade SDK. More details:

WinUI tests are scheduled in Helix by the Azure DevOps Pipeline: RunHelixTests.yml.

The workflow is as follows:

  1. NuGet Restore is called on the packages.config in this directory. This downloads any runtime dependencies that are needed to run tests.
  2. PrepareHelixPayload.ps1 is called. This copies the necessary files from various locations into a Helix payload directory. This directory is what will get sent to the Helix machines.
  3. RunTestsInHelix.proj is executed. This proj has a dependency on Microsoft.DotNet.Helix.Sdk which it uses to publish the Helix payload directory and to schedule the Helix Work Items. The WinUI tests are parallelized into multiple Helix Work Items.
  4. Each Helix Work Item calls runtests.cmd with a specific query to pass to TAEF which runs the tests.
  5. If a test is detected to have failed, we run it again, first once, then eight more times if it fails again. If it fails all ten times, we report the test as failed; otherwise, we report it as unreliable, which will show up as a warning, but which will not fail the build. When a test is reported as unreliable, we include the results for each individual run via a JSON string in the original test's errorMessage field.
  6. TAEF produces logs in WTT format. Helix is able to process logs in XUnit format. We run ConvertWttLogToXUnit.ps1 to convert the logs into the necessary format.
  7. RunTestsInHelix.proj has EnableAzurePipelinesReporter set to true. This allows the XUnit formatted test results to be reported back to the Azure DevOps Pipeline.
  8. We process unreliable tests once all tests have been reported by reading the JSON string from the errorMessage field and calling the Azure DevOps REST API to modify the unreliable tests to have sub-results added to the test and to mark the test as "warning", which will enable people to see exactly how the test failed in runs where it did.