terminal/doc/TAEF.md

65 lines
3.7 KiB
Markdown
Raw Permalink Normal View History

### TAEF Overview ###
TAEF, the Test Authoring and Execution Framework, is used extensively within the Windows organization to test the operating system code in a unified manner for system, driver, and application code. As the console is a Windows OS Component, we strive to continue using the same system such that tests can be ran in a unified manner both externally to Microsoft as well as inside the official OS Build/Test system.
The [official documentation](https://docs.microsoft.com/en-us/windows-hardware/drivers/taef/) for TAEF describes the basic architecture, usage, and functionality of the test system. It is similar to Visual Studio test, but a bit more comprehensive and flexible.
### Writing Tests
You may want to read the section [Authoring Tests in C++](https://docs.microsoft.com/en-us/windows-hardware/drivers/taef/authoring-tests-in-c--) before getting your hands dirty. Note that the quoted header name in `#include "WexTestClass.h"` might be a bit confusing. You are not required to copy TAEF headers into the project folder.
Use the [TAEF Verify Macros for C++](https://docs.microsoft.com/en-us/windows-hardware/drivers/taef/verify) in your test code to perform verifications.
### Running Tests
If you have Visual Studio and related C++ components installed, and you have successfully restored NuGets, you should have the TAEF test runner `te.exe` available locally as part of the `Microsoft.Taef` package.
> Note that you cannot easily run TAEF tests directly through Visual Studio. The `Microsoft.Taef` NuGet package comes with an adapter that will let you browse and execute TAEF tests inside of Visual Studio, but its performance and reliability prevent us from recommending it here.
In a "normal" CMD environment, `te.exe` may not be directly available. Try the following command to set up the development environment first:
```shell
.\tools\razzle.cmd
```
Then you should be able to use `%TAEF%` as an alias of the actual `te.exe`.
For the purposes of the OpenConsole project, you can run the tests using the `te.exe` that matches the architecture for which the test was built (x86/x64):
te.exe Console.Unit.Tests.dll
Replacing the binary name with any other test binary name that might need running. Wildcard patterns or multiple binaries can be specified and all found tests will be executed.
Limiting the tests to be run is also useful with:
te.exe Console.Unit.Tests.dll /name:*BufferTests*
Any pattern of class/method names can be specified after the */name:* flag with wildcard patterns.
For any further details on the functionality of the TAEF test runner, please see the [Executing Tests](https://docs.microsoft.com/en-us/windows-hardware/drivers/taef/executing-tests) section in the official documentation. Or run the embedded help with
te.exe /!
If you use PowerShell, try the following command:
```powershell
Import-Module .\tools\OpenConsole.psm1
Invoke-OpenConsoleTests
```
`Invoke-OpenConsoleTests` supports a number of options, which you can enumerate by running `Invoke-OpenConsoleTests -?`.
Greatly reduce allocations in the conhost/OpenConsole startup path (#8489) I was looking at conhost/OpenConsole and noticed it was being pretty inefficient with allocations due to some usages of std::deque and std::vector that didn't need to be done quite that way. So this uses std::vector for the TextBuffer's storage of ROW objects, which allows one allocation to contiguously reserve space for all the ROWs - on Desktop this is 9001 ROW objects which means it saves 9000 allocations that the std::deque would have done. Plus it has the benefit of increasing locality of the ROW objects since deque is going to chase pointers more often with its data structure. Then, within each ROW there are CharRow and ATTR_ROW objects that use std::vector today. This changes them to use Boost's small_vector, which is a variation of vector that allows for the so-called "small string optimization." Since we know the typical size of these vectors, we can pre-reserve the right number of elements directly in the CharRow/ATTR_ROW instances, avoiding any heap allocations at all for constructing these objects. There are a ton of variations on this "small_vector" concept out there in the world - this one in Boost, LLVM has one called SmallVector, Electronic Arts' STL has a small_vector, Facebook's folly library has one...there are a silly number of these out there. But Boost seems like it's by far the easiest to consume in terms of integration into this repo, the CI/CD pipeline, licensing, and stuff like that, so I went with the boost version. In terms of numbers, I measured the startup path of OpenConsole.exe on my dev box for Release x64 configuration. My box is an i7-6700k @ 4 Ghz, with 32 GB RAM, not that I think machine config matters much here: | | Allocation count | Allocated bytes | CPU usage (ms) | | ------ | ------------------- | ------------------ | -------------- | | Before | 29,461 | 4,984,640 | 103 | | After | 2,459 (-91%) | 4,853,931 (-2.6%) | 96 (-7%) | Along the way, I also fixed a dynamic initializer I happened to spot in the registry code, and updated some docs. ## Validation Steps Performed - Ran "runut", "runft" and "runuia" locally and confirmed results are the same as the main branch - Profiled the before/after numbers in the Visual Studio profiler, for the numbers shown in the table Co-authored-by: Austin Lamb <austinl@microsoft.com>
2020-12-16 19:40:30 +01:00
### Debugging Tests
If you want to debug a test, you can do so by using the TAEF /waitForDebugger flag, such as:
runut *Tests.dll /name:TextBufferTests::TestInsertCharacter /waitForDebugger
Replace the test name with the one you want to debug. Then, TAEF will begin executing the test and output something like this:
TAEF: Waiting for debugger - PID <some PID> @ IP <some IP address>
You can then attach to that PID in your debugger of choice. In Visual Studio, you can use Debug -> Attach To Process, or you could use WinDbg or whatever you want.
Once the debugger attaches, the test will execute and your breakpoints will be hit.