Considering the number of reports of "defterm isn't working (mysteriously)", I figured more logging current hurt. I also added a wprp profile for the defterm logging as well, which should capture conhost side things as well.
From an elevated conhost:
```
wpr -start path\to\Terminal.wprp!Defterm.Verbose
wpr -stop %USERPROFILE%\defterm-trace.etl
```
* [x] I work here
* [x] relevant to: #10594, #11529, #11524.
Up until this commit PSReadline caused OutputDebugString to be called
with a complex log message to on every newline. At the time of writing,
Visual Studio's Output window is fairly slow and after this change newlines
feel a fair bit snappier when running under Visual Studio's debugger.
## Validation Steps Performed
* pwsh.exe continues to work correctly ✔️
This implements command line matching for `CascadiaSettings::GetProfileForArgs`.
The command lines for all user profiles are resolved to absolute file paths,
argument quotes are standardized ("canonicalized") and the results are cached.
When `GetProfileForArgs` is called with a Commandline() value, we "canonicalize"
the argument as well and find the profile that is the longest prefix.
If none could be found the default profile is returned.
## PR Checklist
* [x] Closes#9458
* [x] Closes#10952
* [x] I work here
* [ ] Tests added/passed
## Validation Steps Performed
* Open a `cmd.exe` tab in the store-version of WT
* Run `start cmd`
--> A tab with the `cmd.exe` profile opens
* Run `start pwsh.exe`
--> A tab with the PowerShell 7 profile opens
* Run PowerShell 7 from the start menu
--> A tab with the PowerShell 7 profile opens
* Create a symlink for PowerShell 7 and launch `pwsh.exe` from there
--> A tab with the PowerShell 7 profile opens
Adds a check before every UIA function call to ensure the terminal (specifically the buffer) is initialized before doing work. Both the `ScreenInfoUiaProvider` and the `UiaTextRange` are now covered.
## References
Closes#11135#10971 & #11042
## Detailed Description of the Pull Request / Additional comments
Originally, I tried applying this heuristic to all the `RuntimeClassInitialize` on `UiaTextRangeBase` with the philosophy of "a range pointing to an invalid buffer is invalid itself", but that caused a regression on [MSFT 33353327](https://microsoft.visualstudio.com/OS/_workitems/edit/33353327).
`IUiaData` also has `GetTextBuffer()` return a `TextBuffer&`, which cannot be checked for nullness. Instead, I decided to add a function to `IUiaData` that checks if we have a valid state. Since this is shared with Conhost and Conhost doesn't have this issue, I simply make that function say that it's always in a valid state.
## Validation Steps Performed
- [X] Narrator can detect newly created terminals
- [X] (On Windows Server 2022) Windows Terminal does not hang on launch
Implements the following keyboard selection non-configurable key bindings:
- shift+arrow --> move endpoint by character
- ctrl+shift+left/right --> move endpoint by word
- shift+home/end --> move to beginning/end of line
- ctrl+shift+home/end --> move to beginning/end of buffer
This was purposefully done in the ControlCore layer to make keyboard selection an innate part of how the terminal functions (aka a shared component across terminal consumers).
## References
#715 - Keyboard Selection
#2840 - Spec
## Detailed Description of the Pull Request / Additional comment
The most relevant section is `TerminalSelection.cpp`, where we define how each movement operates. It's basically a giant embedded switch-case statement. We leverage a lot of the work done in a11y to perform the movements.
## Validation Steps Performed
- General cases:
- test all of the key bindings added
- Corner cases:
- `char`: wide glyph support
- `word`: move towards, away, and across the selection pivot
- automatically scroll viewport
- ESC (and other key combos) are still clearing the selection properly
## Summary of the Pull Request
This adds a new action, `clearBuffer`. It accepts 3 values for the `clear` type:
* `"clear": "screen"`: Clear the terminal viewport content. Leaves the scrollback untouched. Moves the cursor row to the top of the viewport (unmodified).
* `"clear": "scrollback"`: Clear the scrollback. Leaves the viewport untouched.
* `"clear": "all"`: (**default**) Clear the scrollback and the visible viewport. Moves the cursor row to the top of the viewport (unmodified).
"Clear Buffer" has also been added to `defaults.json`.
## References
* From microsoft/vscode#75141 originally
## PR Checklist
* [x] Closes#1193
* [x] Closes#1882
* [x] I work here
* [x] Tests added/passed
* [ ] Requires documentation to be updated
## Detailed Description of the Pull Request / Additional comments
This is a bit tricky, because we need to plumb it all the way through conpty to clear the buffer. If we don't, then conpty will immediately just redraw the screen. So this sends a signal to the attached conpty, and then waits for conpty to draw the updated, cleared, screen back to us.
## Validation Steps Performed
* works for each of the three clear types as expected
* tests pass.
* works even with `ping -t 8.8.8.8` as you'd hope.
Improve WriteCharsLegacy performance by increasing LocalBuffer size, allowing
longer runs of characters to be submitted to the remaining parts of conhost.
References #10563 -- vtebench tracking issue
## Validation Steps Performed
* Ran `cat big.txt`, vtebench and termbench and
noted ~5% performance improvements
This PR adds conhost support for downloadable soft fonts - also known as
dynamically redefinable character sets (DRCS) - using the `DECDLD`
escape sequence.
These fonts are typically designed to work on a specific terminal model,
and each model tends to have a different character cell size. So in
order to support as many models as possible, the code attempts to detect
the original target size of the font, and then scale the glyphs to fit
our current cell size.
Once a font has been downloaded to the terminal, it can be designated in
the same way you would a standard character set, using an `SCS` escape
sequence. The identification string for the set is defined by the
`DECDLD` sequence. Internally we map the characters in this set to code
points `U+EF20` to `U+EF7F` in the Unicode private use are (PUA).
Then in the renderer, any characters in that range are split off into
separate runs, which get painted with a special font. The font itself is
dynamically generated as an in-memory resource, constructed from the
downloaded character bitmaps which have been scaled to the appropriate
size.
If no soft fonts are in use, then no mapping of the PUA code points will
take place, so this shouldn't interfere with anyone using those code
points for something else, as along as they aren't also trying to use
soft fonts. I also tried to pick a PUA range that hadn't already been
snatched up by Nerd Fonts, but if we do receive reports of a conflict,
it's easy enough to change.
## Validation Steps Performed
I added an adapter test that runs through a bunch of parameter
variations for the `DECDLD` sequence, to make sure we're correctly
detecting the font sizes for most of the known DEC terminal models.
I've also tested manually on a wide range of existing fonts, of varying
dimensions, and from multiple sources, and made sure they all worked
reasonably well.
Closes#9164
## Summary of the Pull Request
When switching from the alt buffer back to the main buffer, we need to copy certain cursor attributes from the one to the other. However, this copying was taking place after the alt buffer had been freed, and thus could result in the app crashing. This PR simply moves that code up a bit so it's prior to the buffer being freed.
## References
PR #10843 added the code that introduced this problem.
## PR Checklist
* [ ] Closes #xxx
* [x] CLA signed.
* [ ] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Validation Steps Performed
I was able to reproduce the crash when using a debug build, and confirmed that the crash no longer occurred after this PR was applied. I also checked that the cursor attributes were still being correctly copied back when returning from the alt buffer.
When the `SetContoleTitle` API is called with a title containing control
characters, we need to filter out those characters before we can forward
the title change over conpty as an escape sequence. If we don't do that,
the receiving terminal will end up executing the control characters
instead of updating the title. We were already filtering out the C0
control characters, but with this PR we're now filtering out C1 controls
characters as well.
I've simply updated the sanitizing routine in `DoSrvSetConsoleTitleW` to
filter our characters in the range `0x80` to `0x9F`. This is in addition
to the C0 range (`0x00` to `0x1F`) that was already excluded.
## Validation Steps Performed
I've added a conpty unit test that calls `DoSrvSetConsoleTitleW` with
titles containing a variety of C0 and C1 controls characters, and which
verifies that those characters are stripped from the title forwarded to
conpty.
I've also confirmed that the test case in issue #10312 is now working
correctly in Windows Terminal.
Closes#10312
When switching to the alt buffer, the starting cursor position, style,
and visibility is meant to be inherited from the main buffer. Similarly,
when returning to the main buffer, any changes made to those attributes
should be copied back (with the exception of the cursor position, which
is restored to its original state). This PR makes sure we handle that
cursor state correctly.
At some point I'd like to move the cursor state out of the
`SCREEN_INFORMATION` class, which would make this inheritance problem a
non-issue. For now, though, I've just made it copy the state from the
main buffer when creating the alt buffer, and copy it back when
returning to the main buffer.
## Validation Steps Performed
I've added some unit tests to verify the cursor state is inherited
correctly when switching to the alt buffer and back again. I also had to
make a small change to one of the existing alt buffer test that relied
on the initial cursor position being at 0;0, which is no longer the
case.
I've verified that the test case in issue #3545 is now working
correctly. I've also confirmed that this fixes a problem in the
_notcurses_ demo, where the cursor was showing when it should have been
hidden.
Closes#3545
I was watching a video about vectorized instructions and I wanted to
try out some new things, as I had never written AVX code before.
This commit is the result of this tiny Thursday morning detour into
AVX land. It improves performance of `TextColor::GetColor` by about 3x.
## Validation Steps Performed
* Default colors are still properly shifted +8 ✔️
Pass inbound handoff message via heap so it cannot race out of scope by the time it reaches the ConsoleIoThread
## PR Checklist
* [x] Closes#10251
* [x] I work here.
* [x] Manually verified somewhat
## Detailed Description of the Pull Request / Additional comments
- `OpenConsole.exe` is started in response to the OS `conhost.exe` request for a handoff and prepares an Out Of Proc Multithreaded COM server.
- A COM thread from the pool inside `OpenConsole.exe` picks up the inbound message and allocates some stack space for the `CONSOLE_API_MSG` coming in
- That COM thread calls down to set up the I/O thread that will pump the console driver handle and passes a pointer to the stack-allocated `CONSOLE_API_MSG` as the `LPVOID` parameter for starting the thread.
Now one of two things happen:
1. The I/O thread is scheduled pretty much immediately (or soon enough that the COM thread hasn't messed with the stack space), picks up the pointer to the COM thread's stack with `CONSOLE_API_MSG`, and processes the initial message correctly.
2. The COM thread continues and finalizes the handoff message to `conhost.exe` declaring success. It then pops stack and "frees" the memory space. If it doesn't manage to overwrite it, we're still good. If it does, then things go crazy.
This fix changes it so that the `CONSOLE_API_MSG` is sent into the heap before being passed to the other thread so it's in a known location that won't be freed or overwritten unexpectedly.
## Validation Steps Performed
- [x] - Confirmed that many handoffs from the run box seem to work alright on my system after this change.
- [x] - Confirmed that many tab creations/splits seem to work alright on my system after this change.
- [x] - Would prefer if @ianjoneill could try to F5 this branch to build/deploy it, set it as default, and see if it makes it go away completely... but I'm pretty confident it is this based on the dumps provided either way.
Visual Studio 2022 Preview recently released the v143 toolchain.
C4189 is now flagging several unused variables, which breaks our build.
## PR Checklist
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
* CascadiaPackage builds ✔️
* All tests build ✔️
When you use the size parameter to WideCharToMultiByte, it only
null-terminates the output string if the input string was
null-terminated within the specified range.
Burned in for 1k runs-
BEFORE
Summary: Total=1000, Passed=997, Failed=3
AFTER
Summary: Total=1000, Passed=1000, Failed=0
Fixes MSFT-34656993
This commit is a preparation for upcoming changes to KeyChordSerialization for #7539 and #10203.
In order to support variadic macros, /Zc:preprocessor was enabled, which required changing unrelated parts of the project.
## PR Checklist
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
* Project still compiles ✔️
## Summary of the Pull Request
This implements `GetAttributeValue` and `FindAttribute` for `UiaTextRangeBase` (the shared `ITextRangeProvider` for Conhost and Windows Terminal). This also updates `UiaTracing` to collect more useful information on these function calls.
## References
#7000 - Epic
[Text Attribute Identifiers](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-textattribute-ids)
[ITextRangeProvider::GetAttributeValue](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-itextrangeprovider-getattributevalue)
[ITextRangeProvider::FindAttribute](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-itextrangeprovider-findattribute)
## PR Checklist
* [X] Closes#2161
* [X] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
- `TextBuffer`:
- Exposes a new `TextBufferCellIterator` that takes in an end position. This simplifies the logic drastically as we can now use this iterator to navigate through the text buffer. The iterator can also expose the position in the buffer.
- `UiaTextRangeBase`:
- Shared logic & helper functions:
- Most of the text attributes are stored as `TextAttribute`s in the text buffer. To extract them, we generate an attribute verification function via `_getAttrVerificationFn()`, then use that to verify if a given cell has the desired attribute.
- A few attributes are special (i.e. font name, font size, and "is read only"), in that they are (1) acquired differently and (2) consistent across the entire text buffer. These are handled separate from the attribute verification function.
- `GetAttributeValue`: Retrieve the attribute verification of the first cell in the range. Then, verify that the entire range has that attribute by iterating through the text range. If a cell does not have that attribute, return the "reserved mixed attribute value".
- `FindAttribute`: Iterate through the text range and leverage the attribute verification function to find the first contiguous range with that attribute. Then, make the end exclusive and output a `UiaTextRangeBase`. This function must be able to perform a search backwards, so we abstract the "start" and "end" into `resultFirstAnchor` and `resultSecondAnchor`, then perform post processing to output a valid `UiaTextRangeBase`.
- `UiaTracing`:
- `GetAttributeValue`: Log uia text range, desired attribute, resulting attribute metadata, and the type of the result.
- `FindAttribute`: Log uia text range, desired attribute and attribute metadata, if we were searching backwards, the type of the result, and the resulting text range.
- `AttributeType` is a nice way to understand/record if the result was either of the reserved UIA values, a normal result, or an error.
- `UiaTextRangeTests`:
- `GetAttributeValue`:
- verify that we know which attributes we support
- test each of the known text attributes (expecting 100% code coverage for `_getAttrVerificationFn()`)
- `FindAttribute`:
- test each of the known _special_ text attributes
- test `IsItalic`. NOTE: I'm explicitly only testing one of the standard text attributes because the logic is largely the same between all of them and they leverage `_getAttrVerificationFn()`.
## Validation Steps Performed
- @codeofdusk has been testing this Conhost build
- Tests added for Conhost and shared implementation
- Windows Terminal changes were manually verified using accessibility insights and NVDA
Change accessibility notifier creation so we do not create one when we're in PTY mode. (Guard all call sites to skip math/event work when the notifier is null.) MSAA events are legacy events that are registered for globally and used by some screen readers to find content in the conhost window. The PTY mode is not responsible for hosting the display content or input window, so it makes sense for it to not broadcast these events and delegate the accessibility requirement to the connected terminal.
## References
- #10537
## PR Checklist
* [x] Closes#10568
* [x] I work here
* [x] Manual test launches passed.
Currently, when the user changes the console codepage (manually or via a script) the GDI engine tries to find and set the "best possible" font. The "best possible" here is charset-wise, it doesn't mean that the font is actually better or more eye-candy for the user.
Example:
- Open cmd
- Set the font to Consolas
- Enter `chcp 932`
- Suddenly, a wild MS Gothic appears!
This kind of makes sense (*"if I'm changing the codepage I probably want to see the national characters"*) but it doesn't happen anywhere else - all other apps just substitute the missing glyphs.
After #10472 / #10478 this magic should finally work here as well. So, do we still need to change the whole font? Terminal doesn't do that after all.
## Validation Steps Performed
Download [932.cmd.txt](https://github.com/microsoft/terminal/files/6697577/932.cmd.txt),
rename to 932.cmd, run it, check if the output is still readable.
Closes#10497
Try to throttle the cursor redrawing in the conhost world.
The motivation of this is the high CPU usage of `TriggerRedrawCursor` (#10393).
This can be seen as the conhost version of #2960.
This saves 5%~8% of the CPU time.
Supports #10462.
Fix startup race of resizing ConPTY
- Depending on what the timing and ordering is of the message coming in
from the signal thread, it may be applied to the startup structure
after the I/O thread has begun initializing the console buffer
structures but before it has signaled that it is done and the signal
thread is ready to make changes directly. This likely happens because
the end of the I/O thread setup has a weird unlock/lock jog for the
input thread and the signal thread might have been scheduled in the
middle of it.
- My resolution here is to ensure that the signal thread just keeps
storing the latest resize message until it is told that everything is
initialized. Whomever comes in to tell the signal thread this
information (under lock) will pickup and run the resize if one came in
before everything was ready. This should resolve the race.
## Validation Steps Performed
- o-sdn-o confirms this resolves their issue
Closes#10400
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 40df712b33a40e76aeeac87f823bbb028d2e3972
Related work items: MSFT-32007459
Persist inbox conhost; delegate control activities to it via a pipe
## PR Checklist
* [x] Closes#10194 - WSL Debug Tap doesn't work
* [x] Closes#10134 - WSL Parameter is Incorrect
* [x] Closes#10413 - Ctrl+C not passed to client
* [x] Closes#10414 - Leftover processes on abrupt termination
* [x] Might help #10251 - Win+X Powershell sometimes fails to attach
* [x] I work here
* [x] Manually tested with assorted launch scenarios
## Detailed Description of the Pull Request / Additional comments
It turns out that there's a bit of ownership that goes on with the original inbox `conhost.exe` and the operating system/driver. The PID of that original `conhost.exe` is stowed when the initial connection is established and it is verified for several activities. This means that the plan of letting it go completely away and having the `OpenConsole.exe` take over all of its activities must be slightly revised.
I have tested the following two alternatives to keeping `conhost.exe` around and they do not work:
1. Replacing the original owner `conhost.exe` with `OpenConsole.exe` - A.) The driver does not allow this. Once the owner is registered, it cannot be replaced. B.) There's no way of updating this information inside the client process space and it is kept there too in the `kernelbase`/`conclnt` data from its initial connection.
2. Attempting to pick up the first packet (to determine headed/headless and other initial connection information that we use to determine whether handoff is appropriate or not) prior to registering any owner at all. - The driver doesn't allow this either. The owner must be registered prior to a packet coming through.
Put this mental model in your head:
CMD --> Conhost (inbox) --> OpenConsole (WT Package) --> Terminal (WT Package)
So since the `conhost.exe` needs to stick around, here's what I'm doing in this PR:
- `conhost.exe` in the OS will receive back the `OpenConsole.exe` process handle on a successful handoff and is expected to remain alive until the `OpenConsole.exe` exits. It's now waiting on that before it terminates itself.
- `conhost.exe` in the OS will establish a signal channel pipe and listen for control commands from `OpenConsole.exe` in a very similar fashion to how the `ConPTY` signal pipe operates between the Terminal and the PTY (provided by `OpenConsole.exe` in this particular example.) When `OpenConsole.exe` needs to do something that would be verified by the OS and rejected... it will instead signal the original `conhost.exe` to do that thing and it will go through.
- `conhost.exe` will give its own handle through to `OpenConsole.exe` so it can monitor its lifetime and cleanup. If the owner is gone, the session should end.
- Assorted handle cleanup that was leading to improper exits. I was confused between `.reset()` and `.release()` for some of the `wil::unique_any<T>` handling and it lead to leaked handles. The leaked handles meant that threads weren't aware of the other sides collapsing and wouldn't cleanup/terminate appropriately.
How does this fix things?
- For the WSL cases... WSL was specifically looking up the owner PID of the console session from the driver. That was the `conhost.exe` PID. If it exits, that PID isn't valid and is recycled. Thus the parameter is incorrect or other inappropriate WSL setup behaviors.
- Ctrl+C not passed... this is a signal the operating system rejects from a PID that is not the owner. This is now relayed through the original owner and it works.
- Leftover processes... I believe I explained this was both not-enough-monitoring of each others' process lifetimes coupled with mishandling of release/resetting handles and leaking them.
- Powershell sometimes fails to attach... my theory on this one is that it's a race that became upset when the `conhost.exe` disappeared while something about Powershell/.NET was still starting, much like the WSL one. I believe now that it is sticking around, it will be fine.
Also, this WILL require an OS update to complete improvement of functionality and I have revised the interface ID. This is considered an acceptable breaking change with no mitigation because we said this feature was an alpha preview.
## Validation Steps Performed
- Launched WSL with defapp set, it works
- Launched WSL with defapp set and the debug tap on, it works and opens in two tabs
- Launched CMD, ran ping, did Ctrl+C, it now receives it
- Launched Win+X powershell a ton of times. It seems fine now
- Launched cmd, powershell, wsl, etc. Killed assorted processes in the chain (client/conhost/openconsole/windowsterminal) and observed in Process Explorer (with a long delta timer so I could see it) that they all successfully tear down now without leftovers.
This pull request converts four of our existing `#ifdef` (or `#ifndef`)
`INSIDE_WINDOWS` blocks to til::features:
* Attempting to establish a handoff session (inside Windows only)
* The ability to *receive* a handoff session (outside Windows only)
* The DX engine (outside Windows only) and shaders (also outside only)
* Whether we use numpad event synthesis for clipboard/conpty (inside
Windows only)
Most of these are using the preprocessor verison of til::feature, only
because it is more difficult to gate the inclusion of headers on
constant expressions. I'd love to prefer the compile time version.
[Defapp] Use real HPCON for PTY management; Have Monarch always listen for connections
## PR Checklist
* [x] Closes#9464
* [x] Related to #9475 - incomplete fix
* [x] I work here.
* [x] Manual test
## Detailed Description of the Pull Request / Additional comments
- Sometimes peasants can't manage to accept a connection appropriately because I wrote defterm before @zadjii-msft's monarch/peasant architecture. The simple solution here is to just make the monarch always be listening for inbound connections. Then COM won't start a peasant with -Embedding just to ask the monarch where it should go. It'll just join the active window. I didn't close 9475 because it should follow monarch policies on which window to join... and it doesn't yet.
- A lot of interesting things are happening because this didn't have a real HPCON. So I passed through the remaining handles (and re-GUID-ed the interface) that made it possible for me to pack the right process handles and such into an HPCON on the inbound connection and monitor that like any other ConptyConnection. This should resolve some of the process exit behaviors and signal channel things like resizing.
Added trace to conhost to instrument buffers that are cooked prior to being passed to the console.
* [x] I've discussed this with core contributors already (internal)
VALIDATION
- Ensured trace is correctly logged in ETL (via TraceLog)
## Summary of the Pull Request
Introduces `til::rle`, a vector-like container which stores elements of
type T in a run length encoded format. This allows efficient compaction
of repeated elements within the vector.
## References
* #8000 - Supports buffer rewrite work. A re-use of `til::rle` will be
useful as a column counter as we pursue NxM storage and presentation.
* #3075 - The new iterators allow skipping forward by multiple units,
which wasn't possible under `TextBuffer-/OutputCellIterator`.
Additionally it also allows a bulk insertions.
* #8787 and #410 - High probability this should be `pmr`-ified
like `bitmap` for things like `chafa` and `cacafire`
which are changing the run length frequently.
## PR Checklist
* [x] Closes#8741
* [x] I work here.
* [x] Tests added.
* [x] Tests passed.
## Validation Steps Performed
* [x] Ran `cacafire` in `OpenConsole.exe` and it looked beautiful
* [x] Ran new suite of `RunLengthEncodingTests.cpp`
Co-authored-by: Michael Niksa <miniksa@microsoft.com>
Set keyword flags on all events so those sharing a provider with
telemetry do not fire unless tracing is enabled
## PR Checklist
* [x] Closes#10093
* [x] I work here
* [x] Tests passed
* [x] Documentation added in `til.h` about how keywords work and at the
only other site of keywords we define in the Host project tracing
files.
## Detailed Description of the Pull Request / Additional comments
I initially thought that we would need to split providers here to
accomplish this... but @DHowett helped me realize that might be a lot of
additional metadata and bloat binary size. So with help from a friend
from fundamentals, I realized that we could use Keywords to
differentiate here. We can no longer define 0 keywords as that
represents an any/all scenario. Every `TraceLoggingWrite` event now
needs a keyword. When our events have a keyword, they're not included in
any trace. Additionally, when we have an explicit keyword to check that
is different from the ones used for the telemetry pipeline, we can
ensure that we only do "hard work" to generate debug trace data when an
"ALL" type listener like TraceView or Windows Performance Recorder with
our profiles is listening to these providers for ALL keyworded events.
## Validation Steps Performed
- [x] - Built with full release build config to confirm performance is
worse than dev builds BECAUSE of the telemetry event collector camping
our provider and triggering full trace event generation on shared
providers.
- [x] - Built with full release build config to enable statistics
collection and validated trace event collection is excluded and trace
event short-circuits work with this change.
- [x] - Checked that TraceView still sees both telemetry and tracing
events
- [x] - Checked that WPR with our .wprp profile sees both telemetry and
tracing events
Implement PGO in pipelines for AMD64 architecture; supply training test scenarios
## References
- #3075 - Relevant to speed interests there and other linked issues.
## PR Checklist
* [x] Closes#6963
* [x] I work here.
* [x] New UIA Tests added and passed. Manual build runs also tested.
## Detailed Description of the Pull Request / Additional comments
- Creates a new pipeline run for creating instrumented binaries for Profile Guided Optimization (PGO).
- Creates a new suite of UIA tests on the full Windows Terminal app to run PGO training scenarios on instrumented binaries (and incidentally can be used to write other UIA tests later for the full Terminal app.)
- Creates a new NuGet artifact to store trained PGO databases (PGD files) at `Microsoft.Internal.Windows.Terminal.PGODatabase`
- Creates a new NuGet artifact to supply large-scale test content for automated tests at `Microsoft.Internal.Windows.Terminal.TestContent`
- Adjusts the release pipeline to run binaries in PGO optimized mode where content from PGO databases is leveraged at link time to optimize the final release build
The following binaries are trained:
- OpenConsole.exe
- WindowsTerminal.exe
- TerminalApp.dll
- TerminalConnection.dll
- Microsoft.Terminal.Control.dll
- Microsoft.Terminal.Remoting.dll
- Microsoft.Terminal.Settings.Editor.dll
- Microsoft.Terminal.Settings.Model.dll
In the future, adding `<PgoTarget>true</PgoTarget>` to a new `vcxproj` file will automatically enroll the DLL/EXE for PGO instrumentation and optimization going forward.
Two training test scenarios are implemented:
- Smoke test the Terminal by just opening it and typing a bit of text then exiting. (Should help focus on the standard launch path.)
- Optimize bulk text output by launching terminal, outputting `big.txt`, then exiting.
Additional scenarios can be contributed to the `WindowsTerminal_UIATests` project with the `[TestProperty("IsPGO", "true")]` annotation to add them to the suite of scenarios for PGO.
**NOTE:** There are currently no weights applied to the various test scenarios. We will revisit that in the future when/if necessary.
## Validation Steps Performed
- [x] - Training run completed at https://dev.azure.com/ms/terminal/_build?definitionId=492&_a=summary
- [x] - Optimization run completed locally (by forcing `PGOBuildMode` to `Optimize` on my local machine, manually retrieving the databases with NuGet, and building).
- [x] - Validated locally that x86 and ARM64 do not get trained and automatically skip optimization as databases are not present for them.
- [x] - Smoke tested optimized binary versus latest releases. `big.txt` output through CMD is ~11-12seconds prior to PGO and just over 8 seconds with PGO.
This test has some load bearing specifics in it.
In short, it must trigger multiple boost::small_vector growths. The
vector is reused for the lifetime of the console (!) so it must be
forced to act the way the repro requires.
Oh, and the test needs to be run with Application Verifier +Heap.
This validates the fix in c0ab9cb5b5.
Until there's a "Wait", there's usually only one API message inflight at
a time.
In our quest for performance, we put that single API message in charge
of its own buffer management: instead of allocating buffers on the heap
and deleting them later (storing pointers to them at the far corners of
the earth), it would instead allocate them from small internal pools (if
possible) and only heap allocate (transparently) if necessary. The
pointers flung to the corners of the earth would be pointers (1) back
into the API_MSG or (2) to a heap block owned by boost::small_vector.
It took us months to realize that those bare pointers were being held by
COOKED_READ and RAW_READ and not actually being updated when the API
message was _copied_ as it was shuffled off to the background to become
a "Wait" message.
It turns out that it's trivially possible to crash the console by
sending two API calls--one that waits and one that completes
immediately--when the waiting message or the "wait completer" has a
bunch of dangling pointers in it. It further turns out that some
accessibility software (like JAWS) attaches directly to the console
session, much like winpty and ConEmu and friends. They're trying to read
out the buffer (API call!) and sometimes there's a shell waiting for
input (API call!). Oops.
In this commit, we fix up the message's internal pointers (in lieu of
giving it a proper copy constructor; see GH-10076) and then tell the
wait completion routine (which is going to be a COOKED_READ, RAW_READ,
DirectRead or WriteData) about the new buffer location.
This is a scoped fix that should be replaced (TODO GH-10076) with a
final one after Ask mode.
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev eca0875950fd3a9735662474613405e2dc06f485
References GH-10076
Fixes MSFT-33127449
Fixes GH-9692
The ES folks removed support for implicit binary dependencies in test projects, which required our testmd to change to include its dependency on testnetv5.
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev a2a74747feda018c347ba787e166e8385bb17e47
Related work items: MSFT-31480819
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
When the client application sets the console input mode with the `ENABLE_MOUSE_INPUT` flag, we send along the appropriate VT sequences to the connected terminal (if there is one).
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
## References
#376#6859
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [ ] Closes #xxx
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
* [ ] Schema updated.
* [x] I work here
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
Far manager works
## Summary of the Pull Request
ControlCore's _renderer (IRenderTarget) is allocated as std::unique_ptr,
but is given to Terminal::CreateFromSettings as a reference.
ControlCore::Close deallocates the _renderer, but if ThrottledFuncs
are still scheduled to call ControlCore::UpdatePatternLocations
it'll cause Terminal::UpdatePatterns to be called, which in turn ends up
accessing the deallocated IRenderTarget reference and lead to a crash.
A proper solution with shared pointers is nontrivial and should be
attempted at a later point in time. This solution moves the teardown of
the _renderer into ControlCore::~ControlCore, where we can be certain
that no further strong references are held by ThrottledFuncs.
## PR Checklist
* [x] Closes#9910
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
The crash is a race condition and inherently hard to reproduce.
During validation this PR didn't appear to introduce new crashes.
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
Fixes code health failure since 66b9b9d6f1
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
## References
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [ ] Closes #xxx
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
* [ ] Schema updated.
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
