When we encounter clipboard input that cannot be mapped to a keyboard
key, we try our best to map it to an event. If if is an alphanumeric
character or a wide glyph (per the old width algorithm), we will pass it
through as the UnicodeChar of a key event. If it's *not* wide and *not*
alphanumeric, we will synthesize a set of Alt+NumPad events.
Those events comprise a set containing...
1. Alt Down (modifiers = LAlt, UnicodeChar = 0)
2. Numpad 0 Down/Up (modifiers = LAlt, UnicodeChar = 0)
3. Numpad 1 Down/Up (modifiers = LAlt, UnicodeChar = 0)
4. Numpad 2 Down/Up (modifiers = LAlt, UnicodeChar = 0)
5. Alt Up (modifiers = 0, UnicodeChar = [THE CHARACTER])
Because of event group 5, application developers seem to have taken a
dependency on receiving Alt Up + Character and don't seek to recompose
the original character from its numpad representation.
In pull request GH-8035 (!5394370), we stripped the old width algorithm
out and replaced it with a lookup table (finally!). Unfortunately, that
broke clipboard input for Chinese text as it was no longer considered
"Wide" for the purposes of detecting whether we should use numpad
events.
This commit introduces a version of GetQuickCharWidth that fulfills the
exact contract CharToKeyEvents needs, and doesn't answer for a codepoint
more. We'll use it in Windows to fix MSFT-32901370.
The Terminal analogue of this bug, GH-9052, is fixed by *never emitting
numpad events again.* We think this is okay because it looks like nobody
was ever handling numpad events... and that we were only using them as a
way to communicate within conhost (which we're *also* not using) and any
public exposition of event 5 as a contract was unintended.
VALIDATION
----------
I took this new implementation (with an early return) and the old
implementation and compared whether they would yield a numpad event or a
key event over the entire supported codepoint space [0000..FFFF].
They matched.
Fixes MSFT-32901370
Fixes GH-9052
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 224751bbb061f5f59d794c2c9bdac5a9674ebde6
When we resize the text buffer, initialize the buffer with the
_default_¹ attributes, not the _current_ ones. If we use the current
attributes, then we can get into scenarios where something like `vim` is
running, and left the attributes set to something other than the
defaults, and when we resized the buffer, we'd fill it up with color, as
opposed to whatever the default would be.
This PR instead initializes the buffers with the default colors. It also
makes sure to set the active attributes of the newly created buffers
back to whatever the current attributes of the old buffer were.
[1]: For the Terminal, the default attributes are "default on default".
For conhost, the default attributes are whatever the result of
`Settings::GetDefaultAttributes` is, which could be any combo of the
legacy indices and the default color.
## PR Checklist
* [x] Closes#3848
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
## Validation Steps Performed
* ran tests
## Summary of the Pull Request
The "virtual bottom" marks the last line of the mutable viewport area, which is the part of the buffer that VT sequences can write to. This region should typically only move downwards, as new lines are added to the buffer, but there were a number of cases where it was incorrectly being moved up. This PR attempts to fix that.
## PR Checklist
* [x] Closes#9754
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #9754
## Detailed Description of the Pull Request / Additional comments
When a call is made to `UpdateBottom`, we now clamp the value so it's at least as low as the current viewport bottom (i.e. if the viewport has moved down, we want the virtual bottom to move down too), but no lower than the bottom of the buffer (we don't want it to be out of range).
There is one special case where we do actually want the virtual bottom to move up - when the scrollback has been cleared with an `ED3` escape sequence. So in that case we needed a new `ConGetSet` API (`ResetBottom`) to reset the virtual bottom to the top of the buffer (essentially one less than the viewport height, since the virtual bottom points to the last line of the viewport).
## Validation Steps Performed
I had to reset the virtual bottom manually in some parts of the `ScreenBufferTests`, since some of the tests were relying on the virtual bottom being automatically reset when the viewport was reset, which is no longer the case.
I've also added a new test to verify that the virtual bottom doesn't move upwards if an update is triggered while the visible viewport is scrolled up. This essentially reproduces the test case from issue #9754, which I've also manually confirmed is fixed.
Add flag that will ensure we do not handoff to a registered default console. Also for a bonus, allow double-click launches or explicit command line launches of conhost.exe with a binary argument to open with the inbox one.
## PR Checklist
* [x] Closes#9791
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
* [x] Automated tests for parsing
* [x] Enable defapp, double click conhost.exe, opens as itself
* [x] Enable defapp, run conhost.exe runbox, opens as itself
* [x] Enable defapp, run conhost.exe powershell.exe runbox, opens as itself
* [x] Runbox cmd.exe/pwsh.exe trigger defapp handoff to Terminal
* [x] Shortcut to cmd.exe/pwsh.exe trigger defapp handoff to Terminal
* [x] Use CHOP tool to launch conhost with a server handle triggers handoff to Terminal
* [x] Use CHOP tool to launch conhost with a server handle and -ForceNoHandoff opens as itself
Add names to threads to make debugging a slight bit easier.
## PR Checklist
* [x] Closes personal todo item.
* [x] I work here.
* [x] Tested manually.
## Detailed Description of the Pull Request / Additional comments
Thread descriptions show up as names in both the Visual Studio debugger, WinDBG debugger, and Windows Performance Analyzer. This makes it faster and easier to identify threads of interest in our processes.
## Validation Steps Performed
* [x] Checked threads were named in OpenConsole.exe running in classic conhost window mode under VS debug
* [x] Checked threads were named in OpenConsole.exe running in conpty mode under VS debug
* [x] Checked threads were named in WindowsTerminal.exe (for a few of the threads around connections)
* [x] Checked that we could also see it in WinDBG
This PR removes the `GetConsoleCursorInfo` and `SetConsoleCursorInfo`
methods from the `ConGetSet` interface, and the `GetScrollingRegion`
method from the `SCREEN_INFORMATION` class. None of these methods are
used anymore.
PR #2764 removed the last usage of `GetScrollingRegion`.
The `Get/SetConsoleCursorInfo` methods don't seem to have ever been used
in the time that the terminal code has been open source, so whatever
purpose they originally served must have been replaced a long time ago.
The `GetScrollingRegion` method was originally called from the
`ScrollRegion` function, but that usage was removed in PR #2764 when the
margin handling was moved to a higher level.
I've checked that the code still compiles.
Closes#9771
Change TAEF nuget package to use new Microsoft.Taef name; Update to 10.58 release build version.
## PR Checklist
* [x] Closes email from Phil letting us know TAEF has a new release build and a rename.
* [x] Closes annoying duplicate TAEF import warning in `Parser.UnitTests.vcxproj`
* [x] I work here.
* [ ] Need to see the tests run off CI to confirm this is fine for those environments and Helix
## Validation Steps Performed
* [x] Build/run tests locally
* [ ] Build/run unit and feature tests in CI
* [ ] Build/run Helix-lab tests in CI
By default from ARM64 architecture projects, `WIN32` is not defined. It
is supposed to be for this proxy stub to work. So I've set it with the
preprocessor for this project.
## PR Checklist
* [x] Closes release build failure after #7489
* [x] I work here.
* [x] Built on my machine.
## Detailed Description of the Pull Request / Additional comments
`WIN32` appears to convey two meanings depending on who you are:
- To most of Windows, `WIN32` appears to mean the Win32 API surface and
sometimes the major OS version that goes with it. (Specifically in
contrast to 16-bit Windows.)
- To others, `WIN32` appears to mean a 32-bit processor or a synonym of
`x86`.
This is generally not a problem for a few reasons:
- VS defines `WIN32` in the default targets/props only for the `x86`
processor type. **BUT**
- Windows defines `WIN32` if it's not already defined in both
`minwinbase.h` and `ole2.h` which generally speaking manage to get
compiled into practically everything especially since `minwinbase.h`
tends to sneak itself in somehow through `windows.h` and that's
**THE** include to use the Windows API surface.
- Windows also defines `WIN32` for itself unconditionally and relatively
globally when building itself.
However, it's a problem here because:
- `rpcproxy.h` is the only header included in `dlldata.c`, a file
generated automatically by `midl.exe` in the SDK when making a proxy
stub.
- `rpcproxy.h` only defines its contents for a proxy when `WIN32` or
`_M_AMD64` are found.
- Therefore, it's defined pretty naturally for x86 and AMD64 targets
from VS, but not for ARM64.
- ARM64 support is pretty new and those who are attempting to build for
ARM64 and against the public SDK with Visual Studio for a classic COM
proxy... seems like a relatively unlikely combination.
I will follow up with the Visual Studio, Windows SDK, and MIDL/COM teams
to try to remove this pitfall from the public tooling. But for now, this
is the fix.
This commit introduces a new build configuration, "Fuzzing", which
enables the new address sanitizer (shipped in VS 16.9) and code
coverage over the entire solution. Only a small subset of projects
(those comprising original conhost, right now) are selected to build in
this configuration, and even then only in Fuzzing|x64.
It also adds a fuzzing-adapted build of conhost, which makes no server
connections and handles no client applications. To do this, I've
replicated a bit of the console startup routine into fuzzmain.cpp and
made up some fake data. This is the bare minimum required to boot up
Win32 interactivity (or VT interactivity!) and pretend that a process
has connected.
If we don't pretend that a process has connected, "conhost" will exit
immediately. If we don't forge the process list, conhost will exit. If
we can't provide a server handle, we can't provide a "device comm".
Minor changes were necessary to server/host such that they would accept
a preexisting "device comm". We use this new behavior to provide a
"null" one that only hangs up threads and otherwise responds to requests
successfully.
This fuzzing-adapted build links LLVM's libFuzzer, which is an excellent
coverage-based fuzzer that will produce a corpus of inputs that exercise
unique codepaths. Eventually, we can use this to generate known-"good"
inputs for anything.
I've gone ahead and added a fuzz function that yeets bytes directly into
WriteCharsLegacy, which was the original reason I went down this path.
The implementation of LLVMFuzzerTestOneInput should be replaced with
whatever you want to fuzz.
- Implements the default application behavior and handoff mechanisms
between console and terminal. The inbox portion is done already. This
adds the ability for our OpenConsole.exe to accept the incoming server
connection from the Windows OS, stand up a PTY session, start the
Windows Terminal as a listener for an incoming connection, and then
send it the incoming PTY connection for it to launch a tab.
- The tab is launched with default settings at the moment.
- You must configure the default application using the `conhost.exe`
propsheet or with the registry keys. Finishing the setting inside
Windows Terminal will be a todo after this is complete. The OS
Settings panel work to surface this setting is a dependency delivered
by another team and you will not see it here.
## Validation Steps Performed
- [x] Manual adjust of registry keys to the delegation conhost/terminal
behavior
- [x] Adjustment of the delegation options with the propsheet
- [x] Launching things from the run box manually and watching them show
in Terminal
- [x] Launching things from shortcuts and watching them show in the
Terminal
Documentation on how it works will be a TODO post completion in #9462
References #7414 - Default Terminal spec
Closes#492
The PlaySound functions were removed from OneCoreUAP_apiset.Lib in Windows 10 SDK 19041 because they did not actually belong there. Link to WinMM.Lib for PlaySoundW.
### Validation Steps Performed
* Built for x64 from repository root with: `MSBuild.exe -property:TargetPlatformVersion=10.0.19041.0`
* Installed CascadiaPackage_0.0.1.0_x64_Debug.msix and launched on 19042.867
This commit clarifies some things inside WriteCharsLegacy by adding
comments and renaming parameter and enum names. It does not change any
logic.
`WC_ECHO` was used extensively to mean only one thing: whether to print
a control character like `\x18` (Ctrl+X>) as `^X`. It's been renamed
to make that abundantly clear.
Parenthesis in incorrect spot for default app policy check.
Related work items: MSFT-32071839
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 537ab5dfd9cc6af60b570697e77bba4b45822e05
[Git2Git] Merged PR 5760120: Add propsheet chooser to Windows
Now the inbox console propsheet can choose which terminal is default
Related work items: MSFT-32007202 #492
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev d80f506858bd990c267de6cefae7ff55707b3a57
This PR adds support for the VT line rendition attributes, which allow
for double-width and double-height line renditions. These renditions are
enabled with the `DECDWL` (double-width line) and `DECDHL`
(double-height line) escape sequences. Both reset to the default
rendition with the `DECSWL` (single-width line) escape sequence. For now
this functionality is only supported by the GDI renderer in conhost.
There are a lot of changes, so this is just a general overview of the
main areas affected.
Previously it was safe to assume that the screen had a fixed width, at
least for a given point in time. But now we need to deal with the
possibility of different lines have different widths, so all the
functions that are constrained by the right border (text wrapping,
cursor movement operations, and sequences like `EL` and `ICH`) now need
to lookup the width of the active line in order to behave correctly.
Similarly it used to be safe to assume that buffer and screen
coordinates were the same thing, but that is no longer true. Lots of
places now need to translate back and forth between coordinate systems
dependent on the line rendition. This includes clipboard handling, the
conhost color selection and search, accessibility location tracking and
screen reading, IME editor positioning, "snapping" the viewport, and of
course all the rendering calculations.
For the rendering itself, I've had to introduce a new
`PrepareLineTransform` method that the render engines can use to setup
the necessary transform matrix for a given line rendition. This is also
now used to handle the horizontal viewport offset, since that could no
longer be achieved just by changing the target coordinates (on a double
width line, the viewport offset may be halfway through a character).
I've also had to change the renderer's existing `InvalidateCursor`
method to take a `SMALL_RECT` rather than a `COORD`, to allow for the
cursor being a variable width. Technically this was already a problem,
because the cursor could occupy two screen cells when over a
double-width character, but now it can be anything between one and four
screen cells (e.g. a double-width character on the double-width line).
In terms of architectural changes, there is now a new `lineRendition`
field in the `ROW` class that keeps track of the line rendition for each
row, and several new methods in the `ROW` and `TextBuffer` classes for
manipulating that state. This includes a few helper methods for handling
the various issues discussed above, e.g. position clamping and
translating between coordinate systems.
## Validation Steps Performed
I've manually confirmed all the double-width and double-height tests in
_Vttest_ are now working as expected, and the _VT100 Torture Test_ now
renders correctly (at least the line rendition aspects). I've also got
my own test scripts that check many of the line rendition boundary cases
and have confirmed that those are now passing.
I've manually tested as many areas of the conhost UI that I could think
of, that might be affected by line rendition, including things like
searching, selection, copying, and color highlighting. For
accessibility, I've confirmed that the _Magnifier_ and _Narrator_
correctly handle double-width lines. And I've also tested the Japanese
IME, which while not perfect, is at least useable.
Closes#7865
## References
* See also #8617
## PR Checklist
* [x] Supports #3075
* [x] I work here.
* [x] Manual test.
## Detailed Description of the Pull Request / Additional comments
### Window Title Generation
Every time the renderer checks the title, it's doing two bad things that
I've fixed:
1. It's assembling the prefix to the full title doing a concatenation.
No one ever gets just the prefix ever after it is set besides the
concat. So instead of storing prefix and the title, I store the
assembled prefix + title and the bare title.
2. A copy must be made because it was returning `std::wstring` instead
of `std::wstring&`. Now it returns the ref.
### Dirty Area Return
Every time the renderer checks the dirty area, which is sometimes
multiple times per pass (regular text printing, again for selection,
etc.), a vector is created off the heap to return the rectangles. The
consumers only ever iterate this data. Now we return a span over a
rectangle or rectangles that the engine must store itself.
1. For some renderers, it's always a constant 1 element. They update
that 1 element when dirty is queried and return it in the span with a
span size of 1.
2. For other renderers with more complex behavior, they're already
holding a cached vector of rectangles. Now it's effectively giving
out the ref to those in the span for iteration.
### Bitmap Runs
The `til::bitmap` used a `std::optional<std::vector<til::rectangle>>`
inside itself to cache its runs and would clear the optional when the
runs became invalidated. Unfortunately doing `.reset()` to clear the
optional will destroy the underlying vector and have it release its
memory. We know it's about to get reallocated again, so we're just going
to make it a `std::pmr::vector` and give it a memory pool.
The alternative solution here was to use a `bool` and
`std::vector<til::rectangle>` and just flag when the vector was invalid,
but that was honestly more code changes and I love excuses to try out
PMR now.
Also, instead of returning the ref to the vector... I'm just returning a
span now. Everyone just iterates it anyway, may as well not share the
implementation detail.
### UTF-8 conversions
When testing with Terminal and looking at the `conhost.exe`'s PTY
renderer, it spends a TON of allocation time on converting all the
UTF-16 stuff inside to UTF-8 before it sends it out the PTY. This was
because `ConvertToA` was allocating a string inside itself and returning
it just to have it freed after printing and looping back around again...
as a PTY does.
The change here is to use `til::u16u8` that accepts a buffer out
parameter so the caller can just hold onto it.
## Validation Steps Performed
- [x] `big.txt` in conhost.exe (GDI renderer)
- [x] `big.txt` in Terminal (DX, PTY renderer)
- [x] Ensure WDDM and BGFX build under Razzle with this change.
Contains:
- Delegation Configurator that can lookup/edit/save configuration information to registry
- Conhost can lookup the CLSID of a registered default
- Conhost has the ability to handoff a starting visible-window interactive session to the registered default
- Velocity key since this is a big deal and we want to be careful
- IDL for the interface
Related work items: MSFT-16458099
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 0ca55027d8180fbbaa145f2fe7a15005856c0f7c
Moving things out of CharRow into ROW helps us hide it as an implementation detail.
This is part one of many.
### CharRow: Hide ClearCell, use ROW::ClearColumn
### CharRow: Hide GetText, use ROW::GetText
### CharRowBaseTests: remove dead file (never used!)
### CharRow: Move DoubleBytePadded into ROW
### CharRow: Move WrapForced into ROW
### Char/AttrRow: Hide Reset, use ROW::Reset
### Remove RowCellIterator (dead code)
RCI was unused; it was replaced by TextBufferCellIterator shortly after its creation
### Move AttrRowTests to ut_textbuffer from ut_host
It had no reliance on the host.
Make a few changes to memory usage throughout the application to reduce transient allocations from the `big.txt` test from ~213,000 to ~53,000.
## PR Checklist
* [x] Supports #3075
* [x] I work here.
* [x] Tested manually and WPR'd. Test suite should still pass.
* [x] Am core contributor
## Detailed Description of the Pull Request / Additional comments
Transient allocations are those that are new'd, used, then delete'd. Going back and forth to the system allocator for things we're just going to throw away or use rapidly again is a performance detriment. Not only is it a bunch of time to go ask the system with a syscall, it also hits a whole bunch of locks on the allocators. This PR identifies a few places where we were accidentally allocating and didn't mean to or were allocating and freeing just to turn around and allocate again. I chose other strategies to avoid this.
## Validation Steps Performed
- Ran `big.txt` sample (~6MB file) before and after. Observed heap allocations with WPR.
I was looking at conhost/OpenConsole and noticed it was being pretty
inefficient with allocations due to some usages of std::deque and
std::vector that didn't need to be done quite that way.
So this uses std::vector for the TextBuffer's storage of ROW objects,
which allows one allocation to contiguously reserve space for all the
ROWs - on Desktop this is 9001 ROW objects which means it saves 9000
allocations that the std::deque would have done. Plus it has the
benefit of increasing locality of the ROW objects since deque is going
to chase pointers more often with its data structure.
Then, within each ROW there are CharRow and ATTR_ROW objects that use
std::vector today. This changes them to use Boost's small_vector, which
is a variation of vector that allows for the so-called "small string
optimization." Since we know the typical size of these vectors, we can
pre-reserve the right number of elements directly in the
CharRow/ATTR_ROW instances, avoiding any heap allocations at all for
constructing these objects.
There are a ton of variations on this "small_vector" concept out there
in the world - this one in Boost, LLVM has one called SmallVector,
Electronic Arts' STL has a small_vector, Facebook's folly library has
one...there are a silly number of these out there. But Boost seems like
it's by far the easiest to consume in terms of integration into this
repo, the CI/CD pipeline, licensing, and stuff like that, so I went with
the boost version.
In terms of numbers, I measured the startup path of OpenConsole.exe on
my dev box for Release x64 configuration. My box is an i7-6700k @ 4
Ghz, with 32 GB RAM, not that I think machine config matters much here:
| | Allocation count | Allocated bytes | CPU usage (ms) |
| ------ | ------------------- | ------------------ | -------------- |
| Before | 29,461 | 4,984,640 | 103 |
| After | 2,459 (-91%) | 4,853,931 (-2.6%) | 96 (-7%) |
Along the way, I also fixed a dynamic initializer I happened to spot in
the registry code, and updated some docs.
## Validation Steps Performed
- Ran "runut", "runft" and "runuia" locally and confirmed results are
the same as the main branch
- Profiled the before/after numbers in the Visual Studio profiler, for
the numbers shown in the table
Co-authored-by: Austin Lamb <austinl@microsoft.com>
N
* Change backslashes in include statements to forward slashes (CC-8205)
Dustin L. Howett
* Refactor DEC/ANSI modes to avoid duplication when we add SM/RM (GH-8469)
* Fix the xterm and SGR mouse encodings for CTRL, ALT, SHIFT (GH-8379)
James Holderness
* Fix rendering of DBCS characters when partially off screen (CC-8438)
* Correct horizontal coordinates in viewport overflow test (CC-8456)
* Correct paths in the the runut and runft test scripts (CC-8488)
* Retain horizontal viewport offset when moving to bottom (CC-8434)
* Fix out-of-bounds exceptions in Set...{Buffer,Screen}Size (CC-8309)
PankajBhojwani (1)
* Implement ConEmu's OSC 9;4 to set the taskbar progress indicator (CC-8055)
Chester Liu (1)
* Improve OSC 8 Hyperlink parsing logic (CC-7962)
Related work items: #31090459, #31090805, #31090808, #31090810
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_onecore_dep_uxp 91dff13f6953441afe9e28bfe3da9d5bff27bb05
Related: MSFT-26196390
Related work items: #26196390
This commit replaces DeviceComm with the interface IDeviceComm and the
concrete implementation type ConDrvDeviceComm. This work is done in
preparation for different device backends.
In addition to separating out ConDrv-specific behavior, I've introduced
a "handle exchange" interface.
HANDLE EXCHANGE
---------------
There are points where we give ConDrv opaque handle identifiers to our
input buffer, output buffer and process data. The exact content of the
opaque identifier is meaningless to ConDrv: the driver's only
interaction with these identifiers is to hold onto them and send back
whichever are pertinent for each API call.
Because of that, we used the raw register-width pointer value of the
input buffer, output buffer or process data _as_ the opaque handle
value.
This works very well for ConDrv <-> conhost using the ConDrvDeviceComm.
It works less well for something like the "logging" DeviceComm that will
log packets to a file: those packets *cannot* contain pointer values (!)
To address this, and to afford flexibility to DeviceComm implementers,
I've introduced a two-member complement of handle management functions:
* `ULONG_PTR PutHandle(void*)` registers an object with the DeviceComm
and returns an opaque identifier.
* `void* GetHandle(ULONG_PTR)` takes an opaque identifier and returns
the original object.
ConDrvDeviceComm implements PutHandle and GetHandle by casting the
object pointer to the "opaque handle value", which maintains wire format
compatibility[1] with revisions of conhost prior to this change.
Simpler DeviceComm implementations that require handle tracking but
cannot bear raw pointers can implement these functions by returning an
autoincrementing ID (or similar) and registering the raw object pointer
in a mapping.
I've audited all existing handle exchanges with the driver and updated
them to use Put/GetHandle.
(I intended to add DestroyHandle, but we are not equipped for handle
removal at the moment. ConsoleHandleData/ConsoleProcessHandle are
destroyed during wait routine completion, on client disconnect, etc.
This does mean that an id<->pointer mapping will grow without bound, but
at a cost of ~8 bytes per entry and a short-lived console session I am
not too concerned about the cost.)
[1] Wire format compatibility is not required, and later we may want to
switch ConDrvDeviceComm to `EncodePointer` and `DecodePointer` just to
insulate us against a spurious ConDrv packet allowing for an arbitrary
4/8-byte read and subsequent liftoff into space.
There are two issue with copy to clipboard when block is selected:
* We don't add new lines for lines that were wrapped
* We remove trailing whitespaces which is not intuitive in block selection.
Fixed the copy logic to always add newlines and not to remove
whitespaces when block is selected.
Even if shift is pressed!
## Detailed Description of the Pull Request / Additional comments
* Added optional parameter to `TextBuffer::GetText`
that allows to apply formatting (includeCRLF / trimming)
to lines that were wrapped
* Changed `Terminal::RetrieveSelectedTextFromBuffer`
to apply the following parameters when block is selected:
* includeCRLF = true
* trimTrailingWhitespaces = false
* apply the formatting above to all rows, including the ones
that were wrapped
## Validation Steps Performed
* Manual tests for both block and standard selection
* Copy with both right-click and command
* Added UT
Closes#6740
In conhost there is a keyboard shortcut that applies colors to the
selected range of text, and another shortcut that searches for the
selected text, and applies colors to any matching content. The former
operation doesn't work correctly when the selection is wrapped, and both
have problems when the selected text spans DBCS characters. This PR
attempts to fix those issues.
The problem with the color section was that it applied the color to a
simple rect spanning the start and end points of the selection. I've now
updated it to use the `Selection::GetSelectionRects` method, which
correctly handles a wrapped range of lines, and makes sure that double
width characters are fully covered.
The problem with the "find-and-color" operation was in the way it
obtained the search text from the selected screen cells. Since it
retrieved one cell at a time, and a DBCS character can span two cells,
that resulted in some characters being repeated in the search text. I've
now corrected that code to take the width of the characters into
account.
## Validation Steps Performed
I've manually verified that the test cases described in #8572 and #8574
are now working correctly.
Closes#8572Closes#8574
A bunch of our feature tests don't work reliably in CI. They rely on
creating a new `OpenConsole.exe` window, then running the test _in that
console_. As a part of that test setup, the test runner used to wait a
second to attach to the newly created console. Then the test goes on
it's merry way, assuming the console is ready to go. However, in CI,
that might take more than a second. If it does, then the test would fail
pretty immediately, as soon as it tries to get at the buffer of the new
console.
This PR introduces a little retry loop to the test init. After attaching
to the new console, we'll try and get at the screen buffer. If that
fails, we'll wait a second and try again. We'll try 5 times total,
before bailing entirely. Hopefully, this should mitigate most of the
random CI failures we get in the feature tests.
Closes#8495
When resizing the buffer in the `SetConsoleScreenBufferSize` and
`SetConsoleScreenBufferInfoEx` APIs, we have tests in place to make sure
that the resize doesn't result in the viewport extending past the bottom
or right of the buffer (since that can eventually result in exceptions
being thrown). Unfortunately these tests were using the wrong X
coordinate, so they failed to detect an overflow along the horizontal
axis. This PR corrects that mistake.
PR #8309 was where the overflow detection was initially added.
The original code was using the `Viewport::EndExclusive` method to
determine the extent of the viewport, mistakenly thinking that it
returned the bottom right coordinates (it actually returns the left
coordinate). So I've now added a `BottomRightExclusive` method to the
`Viewport` class, that actually does return the coordinates we need, and
have updated the overflow tests to use that method instead.
## Validation Steps Performed
I've manually confirmed that the test case is issue #8453 is no longer
throwing an exception.
Closes#8453
When the viewport is moved to the "virtual bottom" of the buffer (via
the `MoveToBottom` method), it is important that the horizontal viewport
offset be left as it is, otherwise that can result in some undesirable
side effects.
Since the VT coordinate system is relative to the top of the viewport,
many VT operations call the `MoveToBottom` method to make sure the
viewport is correctly positioned before proceeding. There is no need for
the horizontal position to be adjusted, though, since the X coordinates
are not constrained by the viewport, but are instead relative to the
underlying buffer.
Setting the viewport X coordinate to 0 in `MoveToBottom` (as we were
previously doing) could result in the cursor being pushed off screen.
And if the operation itself was moving the cursor, that would then
trigger another viewport move to bring the cursor back into view. These
conflicting movements meant the viewport was always forced as far left
as possible, and could also result in cursor "droppings" as the cursor
lost track of where it had been.
I've now fixed this by updating the `GetVirtualViewport` method to match
the horizontal offset of the active viewport, instead of having the X
coordinate hardcoded to 0.
## Validation Steps Performed
I've manually confirmed that this fixes the cursor "droppings" test case
reported in issue #8213.
I've also added a screen buffer test that makes sure the `MoveToBottom`
method is working as expected, and not changing the horizontal viewport
offset when it moves down.
Closes#8213
Many include statements use forward slashes, while others use backwards
slashes. This is inconsistent formatting. For this reason, I changed the
backward slashes to forward slashes since that is the standard.
[Git2Git] Git Train: Merge of building/rs_onecore_dep_uxp/201117-2002 into official/rs_onecore_dep_uxp Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_onecore_dep_uxp 526043ab6050397506b3cdb77e6a43b5ca9a2e9a
Related work items: MSFT-29990377
This fixes a number of exceptions that can cause conhost to crash when
the buffer is resized in such a way that the viewport or cursor position
end up out of bounds.
Technically this is a fix for issue #256, although that has been closed
as "needs-repro".
The main fix was to add checks in the `SetConsoleScreenBufferSizeImpl`
and `SetConsoleScreenBufferInfoExImpl` methods, to make sure the
viewport doesn't extend past the bottom or right of the buffer after a
resize. If it has overflowed, we move the viewport back up or left until
it's back within the buffer boundaries. We also check if the cursor
position has ended up out of bounds, and if so, clamp it back inside the
buffer.
The `SCREEN_INFORMATION::SetViewport` was also a source of viewport
overflow problems, because it was mistakenly using inclusive coordinates
in its range checks, which resulted in them being off by one. That has
now been corrected to use exclusive coordinates.
Finally, the `IsCursorDoubleWidth` method was incorrectly marked as
`noexcept`, which was preventing its exceptions from being caught.
Ideally it shouldn't be throwing exceptions at all any more, but I've
removed the `noexcept` specifier, so if it does throw an exception,
it'll at least have more chance of recovering without a crash.
## Validation Steps Performed
I put together a few test cases (based on the reports in issues #276 and
#1976) which consistently caused conhost to crash, or to generate an
exception visible in the debug output. With this PR applied, those test
cases are no longer crashing or triggering exceptions.
Closes#1976
Fixed potential errors caused by overflow or underfow in
SectionInput.cpp
## PR Checklist
* [x] CLA signed
* [x] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
In selectionInput.cpp, there is both a potential overflow and potential
underflow. To address this issue, I casted the calculation up to int,
which is then checked because of integer promotion. Underflow and
underflow is therefore impossible because now if the calculation exceeds
SHORT_MAX, it will have exceeded bufferSize.BottomInclusive() or
bufferSize.Top() anyway, and be set to them.
## Validation Steps Performed
Passed Unit Testing
Manual Validation
This pull request is the initial implementation of hyperlink auto
detection
Overall design:
- Upon startup, TerminalCore gives the TextBuffer some patterns it
should know about
- Whenever something in the viewport changes (i.e. text
output/scrolling), TerminalControl tells TerminalCore (through a
throttled function for performance) to retrieve the visible pattern
locations from the TextBuffer
- When the renderer encounters a region that is associated with a
pattern, it paints that region differently
References #5001Closes#574
It turns out that we missed part of the OSC 8 spec which indicated that
_hyperlinks with the same ID but different URIs are logically distinct._
> Character cells that have the same target URI and the same nonempty id
> are always underlined together on mouseover.
> The same id is only used for connecting character cells whose URIs is
> also the same. Character cells pointing to different URIs should never
> be underlined together when hovering over.
This pull request fixes that oversight by appending the (hashed) URI to
the generated ID.
When Terminal receives one of these links over ConPTY, it will hash the
URL a second time and therefore append a second hashed ID. This is taken
as an acceptable cost.
Fixes#7698
This PR introduces a pair of classes for managing VT parameters that
automatically handle range checking and default fallback values, so the
individual operations don't have to do that validation themselves. In
addition to simplifying the code, this fixes a few cases where we were
mishandling missing or extraneous parameters, and adds support for
parameter sequences on commands that couldn't previously handle them.
This PR also sets a limit on the number of parameters allowed, to help
thwart DoS memory consumption attacks.
## References
* The new parameter class also introduces the concept of an
omitted/default parameter which is not necessarily zero, which is a
prerequisite for addressing issue #4417.
## Detailed Description of the Pull Request / Additional comments
There are two new classes provide by this PR: a `VTParameter` class,
similar in function to a `std::optional<size_t>`, which holds an
individual parameter (which may be an omitted/default value); and a
`VTParameters` class, similar in function to `gsl:span<VTParameter>`,
which holds a sequence of those parameters.
Where `VTParameter` differs from `std::optional` is with the inclusion
of two cast operators. There is a `size_t` cast that interprets omitted
and zero values as 1 (the expected behaviour for most numeric
parameters). And there is a generic cast, for use with the enum
parameter types, which interprets omitted values as 0 (the expected
behaviour for most selective parameters).
The advantage of `VTParameters` class is that it has an `at` method that
can never fail - out of range values simply return the a default
`VTParameter` instance (this is standard behaviour in VT terminals). It
also has a `size` method that will always return a minimum count of 1,
since an empty parameter list is typically the equivalent of a single
"default" parameter, so this guarantees you'll get at least one value
when iterating over the list with `size()`.
For cases where we just need to call the same dispatch method for every
parameter, there is a helper `for_each` method, which repeatedly calls a
given predicate function with each value in the sequence. It also
collates the returned success values to determine the overall result of
the sequence. As with the `size` method, this will always make at least
one call, so it correctly handles empty sequences.
With those two classes in place, we could get rid of all the parameter
validation and default handling code in the `OutputStateMachineEngine`.
We now just use the `VTParameters::at` method to grab a parameter and
typically pass it straight to the appropriate dispatch method, letting
the cast operators automatically handle the assignment of default
values. Occasionally we might need a `value_or` call to specify a
non-standard default value, but those cases are fairly rare.
In some case the `OutputStateMachineEngine` was also checking whether
parameters values were in range, but for the most part this shouldn't
have been necessary, since that is something the dispatch classes would
already have been doing themselves (in the few cases that they weren't,
I've now updated them to do so).
I've also updated the `InputStateMachineEngine` in a similar way to the
`OutputStateMachineEngine`, getting rid of a few of the parameter
extraction methods, and simplifying other parts of the implementation.
It's not as clean a replacement as the output engine, but there are
still benefits in using the new classes.
## Validation Steps Performed
For the most part I haven't had to alter existing tests other than
accounting for changes to the API. There were a couple of tests I needed
to drop because they were checking for failure cases which shouldn't
have been failing (unexpected parameters should never be an error), or
testing output engine validation that is no longer handled at that
level.
I've added a few new tests to cover operations that take sequences of
selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've
extended the cursor movement tests to make sure those operations can
handle extraneous parameters that weren't expected. I've also added a
test to verify that the state machine will correctly ignore parameters
beyond the maximum 32 parameter count limit.
I've also manual confirmed that the various test cases given in issues
#2101 are now working as expected.
Closes#2101
For whatever reason, the super linter seems to think that any file it doesn't recognize is an EDITORCONFIG file. That means all our `cpp`, `hpp`, `h`, `resw`, `xaml`, etc files are going to get linted with different rules than the clang-format ones we already use.
This PR disables the EDITORCONFIG linter, and has a minimal change to a cpp file to ensure that it's no longer linted by the action.
See also:
* #7637 added this
* #7799 is blocked by this
* #7924 is blocked by this
* Correct the behaviour of parsing `rgb:R/G/B`. It should be interpreted
as `RR/GG/BB` instead of `0R/0G/0B`
* Add support for `rgb:RRR/GGG/BBB` and `rgb:RRRR/GGGG/BBBB`. The
behaviour of 12 bit variants is to repeat the first digit at the end,
e.g. `rgb:123/456/789` becomes `rgb:1231/4564/7897`.
* Add support for `#` formats. We are following the rules of
[XParseColor] by interpreting `#RGB` as `R000G000B000`.
* Add support for XOrg app color names, which are supported by xterm, VTE
and many other terminal emulators.
* Multi-parameter OSC 4 is now supported.
* The chaining of OSC 10-12 is not yet supported. But the parameter
validation is relaxed by parsing the parameters as multi-params but
only use the first one, which means `\e]10;rgb:R/G/B;` and
`\e]10:rgb:R/G/B;invalid` will execute `OSC 10` with the first color
correctly. This fixes some of the issues mentioned in #942 but not
all of them.
[XParseColor]: https://linux.die.net/man/3/xparsecolorCloses#3715
This fixes a bug when moving backwards by word that resulted in #7742.
This also includes...
- a minor refactor that leverages `GetWordStart` in `MoveToPreviousWord`
- additional unit tests for movement by word
- a feature test comprised of the referenced bug report
`MoveToPreviousWord()` would...
- move backwards for each whitespace character
- then, move backwards for each regular character
This would actually result in moving to the beginning of the current "word" (as defined by a11y).
We actually need to do this process twice:
- the first time gets you to the beginning of the current word
- attempt to move back by one character
- the second time gets you to the beginning of the previous word
Rather than implementing 4 while loops, we leverage `GetWordStart()` to
attempt to move to the beginning of the previous word. We call it twice
(as described above). The logic is unchanged, but we instead reuse a
function that has already undergone more testing.
To make sure this works as expected, additional unit tests were
introduced covering "MoveByWord" in the TextBuffer.
## Validation Steps Performed
Added test for repro steps.
Added unit tests for movement by word.
Closes#7742
Sometimes when we were sliding the viewport to fit inside the buffer, we
would end up with left > right.
That would cause us to crash down the line when rendering.
Fixes MSFT:28387423
Fixes#7744
`EndExclusive` represents the end of the buffer. This is designed to not
point to any data on the buffer. UiaTextRange would point to this
`EndExclusive` and then attempt to move based on it. However, since it
does not point to any data, it could experience undefined behavior or
(inevitably) crash from running out of bounds.
This PR specifically checks for expansion and movement at that point,
and prevents us from moving beyond it. There are plans in the future to
define the "end" as the last character in the buffer. Until then, this
solution will suffice and provide correct behavior that doesn't crash.
## Validation Steps Performed
Performed the referenced bugs' repro steps and added test coverage.
Closes MSFT-20458595
Closes#7663Closes#7664
This PR adds support for the _blink_ graphic rendition attribute. When a
character is output with this attribute set, it "blinks" at a regular
interval, by cycling its color between the normal rendition and a dimmer
shade of that color.
The majority of the blinking mechanism is encapsulated in a new
`BlinkingState` class, which is shared between the Terminal and Conhost
implementations. This class keeps track of the position in the blinking
cycle, which determines whether characters are rendered as normal or
faint.
In Windows Terminal, the state is stored in the `Terminal` class, and in
Conhost it's stored in the `CONSOLE_INFORMATION` class. In both cases,
the `IsBlinkingFaint` method is used to determine the current blinking
rendition, and that is passed on as a parameter to the
`TextAttribute::CalculateRgbColors` method when these classes are
looking up attribute colors.
Prior to calculating the colors, the current attribute is also passed to
the `RecordBlinkingUsage` method, which keeps track of whether there are
actually any blink attributes in use. This is used to determine whether
the screen needs to be refreshed when the blinking cycle toggles between
the normal and faint renditions.
The refresh itself is handled by the `ToggleBlinkingRendition` method,
which is triggered by a timer. In Conhost this is just piggybacking on
the existing cursor blink timer, but in Windows Terminal it needs to
have its own separate timer, since the cursor timer is reset whenever a
key is pressed, which is not something we want for attribute blinking.
Although the `ToggleBlinkingRendition` is called at the same rate as the
cursor blinking, we actually only want the cells to blink at half that
frequency. We thus have a counter that cycles through four phases, and
blinking is rendered as faint for two of those four. Then every two
cycles - when the state changes - a redraw is triggered, but only if
there are actually blinking attributes in use (as previously recorded).
As mentioned earlier, the blinking frequency is based on the cursor
blink rate, so that means it'll automatically be disabled if a user has
set their cursor blink rate to none. It can also be disabled by turning
off the _Show animations in Windows_ option. In Conhost these settings
take effect immediately, but in Windows Terminal they only apply when a
new tab is opened.
This PR also adds partial support for the `SGR 6` _rapid blink_
attribute. This is not used by DEC terminals, but was defined in the
ECMA/ANSI standards. It's not widely supported, but many terminals just
it implement it as an alias for the regular `SGR 5` blink attribute, so
that's what I've done here too.
## Validation Steps Performed
I've checked the _Graphic rendition test pattern_ in Vttest, and
compared our representation of the blink attribute to that of an actual
DEC VT220 terminal as seen on [YouTube]. With the right color scheme
it's a reasonably close match.
[YouTube]: https://www.youtube.com/watch?v=03Pz5AmxbE4&t=1m55sCloses#7388
Some IME implementations do not produce composition strings, and their
users have come to rely on the cursor that conhost traditionally left on
until a composition string showed up. We shouldn't hide the cursor until
we get a string (as opposed to hiding it when composition begins) so as
to not break those IMEs.
Related to #6207.
Fixes MSFT:29219348
This fixes an issue where two CPRs could end up corrupted in the input
buffer. An application that sent two CPRs back-to-back could
end up reading the first few characters of the first prepended CPR
before handing us another CPR. We would dutifully prepend it to the
buffer, causing them to overlap.
```
^[^[2;2R[1;1R
^^ ^^^^^ First CPR
^^^^^^ Second CPR
```
The end result of this corruption is that a requesting application
would receive an unbidden `R` on stdin; for vim, this would trigger
replace mode immediately on startup.
Response prepending was implemented in !997738 without much comment.
There's very little in the way of audit trail as to why we switched.
Michael believes that we wanted to make sure that applications got DSR
responses immediately. It had the unfortunate side effect of causing
subsequence CPRs across cursor moves to come out in the wrong order.
I discussed our options with him, and he suggested that we could
implement a priority queue in InputBuffer and make sure that "response"
input was dispatched to a client application before any application- or
user-generated input. This was deemed to be too much work.
We decided that DSR responses getting top billing was likely to be a
stronger guarantee than most terminals are capable of giving, and that
we should be fine if we just switch it back to append.
Thanks to @k-takata, @tekki and @brammool for the investigation on the
vim side.
Fixes#1637.
This PR is about the behavior of DECSCUSR. This PR changes the meaning
of DECSCUSR 0 to restore the cursor style back to user default. This
differs from what VT spec says but it’s used in popular terminal
emulators like iTerm2 and VTE-based ones. See #1604.
Another change is that for parameter greater than 6, DECSCUSR should be
ignored, instead of restoring the cursor to legacy. This PR fixes it.
See #7382.
Fixes#1604.
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
## References
#204
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [X] Closes#204
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
* [ ] Schema updated.
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
Added support to:
- parse OSC8 sequences and extract URIs from them (conhost and terminal)
- add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal)
- attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal)
- process ctrl+leftclick to open a hyperlink in the clicked region if present
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
Open up a PowerShell tab and type
```PowerShell
${ESC}=[char]27
Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\"
```
Ctrl+LeftClick on the link correctly brings you to the terminal page on github
Use the Helix testing orchestration framework to run our Terminal LocalTests and Console Host UIA tests.
## References
#### Creates the following new issues:
- #7281 - re-enable local tests that were disabled to turn on Helix
- #7282 - re-enable UIA tests that were disabled to turn on Helix
- #7286 - investigate and implement appropriate compromise solution to how Skipped is handled by MUX Helix scripts
#### Consumes from:
- #7164 - The update to TAEF includes wttlog.dll. The WTT logs are what MUX's Helix scripts use to track the run state, convert to XUnit format, and notify both Helix and AzDO of what's going on.
#### Produces for:
- #671 - Making Terminal UIA tests is now possible
- #6963 - MUX's Helix scripts are already ready to capture PGO data on the Helix machines as certain tests run. Presuming we can author some reasonable scenarios, turning on the Helix environment gets us a good way toward automated PGO.
#### Related:
- #4490 - We lost the AzDO integration of our test data when I moved from the TAEF/VSTest adapter directly back to TE. Thanks to the WTTLog + Helix conversion scripts to XUnit + new upload phase, we have it back!
## PR Checklist
* [x] Closes#3838
* [x] I work here.
* [x] Literally adds tests.
* [ ] Should I update a testing doc in this repo?
* [x] Am core contributor. Hear me roar.
* [ ] Correct spell-checking the right way before merge.
## Detailed Description of the Pull Request / Additional comments
We have had two classes of tests that don't work in our usual build-machine testing environment:
1. Tests that require interactive UI automation or input injection (a.k.a. require a logged in user)
2. Tests that require the entire Windows Terminal to stand up (because our Xaml Islands dependency requires 1903 or later and the Windows Server instance for the build is based on 1809.)
The Helix testing environment solves both of these and is brought to us by our friends over in https://github.com/microsoft/microsoft-ui-xaml.
This PR takes a large portion of scripts and pipeline configuration steps from the Microsoft-UI-XAML repository and adjusts them for Terminal needs.
You can see the source of most of the files in either https://github.com/microsoft/microsoft-ui-xaml/tree/master/build/Helix or https://github.com/microsoft/microsoft-ui-xaml/tree/master/build/AzurePipelinesTemplates
Some of the modifications in the files include (but are not limited to) reasons like:
- Our test binaries are named differently than MUX's test binaries
- We don't need certain types of testing that MUX does.
- We use C++ and C# tests while MUX was using only C# tests (so the naming pattern and some of the parsing of those names is different e.g. :: separators in C++ and . separators in C#)
- Our pipeline phases work a bit differently than MUX and/or we need significantly fewer pieces to the testing matrix (like we don't test a wide variety of OS versions).
The build now runs in a few stages:
1. The usual build and run of unit tests/feature tests, packaging verification, and whatnot. This phase now also picks up and packs anything required for running tests in Helix into an artifact. (It also unifies the artifact name between the things Helix needs and the existing build outputs into the single `drop` artifact to make life a little easier.)
2. The Helix preparation build runs that picks up those artifacts, generates all the scripts required for Helix to understand the test modules/functions from our existing TAEF tests, packs it all up, and queues it on the Helix pool.
3. Helix generates a VM for our testing environment and runs all the TAEF tests that require it. The orchestrator at helix.dot.net watches over this and tracks the success/fail and progress of each module and function. The scripts from our MUX friends handle installing dependencies, making the system quiet for better reliability, detecting flaky tests and rerunning them, and coordinating all the log uploads (including for the subruns of tests that are re-run.)
4. A final build phase is run to look through the results with the Helix API and clean up the marking of tests that are flaky, link all the screenshots and console output logs into the AzDO tests panel, and other such niceities.
We are set to run Helix tests on the Feature test policy of only x64 for now.
Additionally, because the set up of the Helix VMs takes so long, we are *NOT* running these in PR trigger right now as I believe we all very much value our 15ish minute PR turnaround (and the VM takes another 15 minutes to just get going for whatever reason.) For now, they will only run as a rolling build on master after PRs are merged. We should still know when there's an issue within about an hour of something merging and multiple PRs merging fast will be done on the rolling build as a batch run (not one per).
In addition to setting up the entire Helix testing pipeline for the tests that require it, I've preserved our classic way of running unit and feature tests (that don't require an elaborate environment) directly on the build machines. But with one bonus feature... They now use some of the scripts from MUX to transform their log data and report it to AzDO so it shows up beautifully in the build report. (We used to have this before I removed the MStest/VStest wrapper for performance reasons, but now we can have reporting AND performance!) See https://dev.azure.com/ms/terminal/_build/results?buildId=101654&view=ms.vss-test-web.build-test-results-tab for an example.
I explored running all of the tests on Helix but.... the Helix setup time is long and the resources are more expensive. I felt it was better to preserve the "quick signal" by continuing to run these directly on the build machine (and skipping the more expensive/slow Helix setup if they fail.) It also works well with the split between PR builds not running Helix and the rolling build running Helix. PR builds will get a good chunk of tests for a quick turn around and the rolling build will finish the more thorough job a bit more slowly.
## Validation Steps Performed
- [x] Ran the updated pipelines with Pull Request configuration ensuring that Helix tests don't run in the usual CI
- [x] Ran with simulation of the rolling build to ensure that the tests now running in Helix will pass. All failures marked for follow on in reference issues.
This PR adds support for the ANSI _doubly underlined_ graphic rendition
attribute, which is enabled by the `SGR 21` escape sequence.
There was already an `ExtendedAttributes::DoublyUnderlined` flag in the
`TextAttribute` class, but I needed to add `SetDoublyUnderlined` and
`IsDoublyUnderlined` methods to access that flag, and update the
`SetGraphicsRendition` methods of the two dispatchers to set the
attribute on receipt of the `SGR 21` sequence. I also had to update the
existing `SGR 24` handler to reset _DoublyUnderlined_ in addition to
_Underlined_, since they share the same reset sequence.
For the rendering, I've added a new grid line type, which essentially
just draws an additional line with the same thickness as the regular
underline, but slightly below it - I found a gap of around 0.05 "em"
between the lines looked best. If there isn't enough space in the cell
for that gap, the second line will be clamped to overlap the first, so
you then just get a thicker line. If there isn't even enough space below
for a thicker line, we move the offset _above_ the first line, but just
enough to make it thicker.
The only other complication was the update of the `Xterm256Engine` in
the VT renderer. As mentioned above, the two underline attributes share
the same reset sequence, so to forward that state over conpty we require
a slightly more complicated process than with most other attributes
(similar to _Bold_ and _Faint_). We first check whether either underline
attribute needs to be turned off to send the reset sequence, and then
check individually if each of them needs to be turned back on again.
## Validation Steps Performed
For testing, I've extended the existing attribute tests in
`AdapterTest`, `VTRendererTest`, and `ScreenBufferTests`, to make sure
we're covering both the _Underlined_ and _DoublyUnderlined_ attributes.
I've also manually tested the `SGR 21` sequence in conhost and Windows
Terminal, with a variety of fonts and font sizes, to make sure the
rendering was reasonably distinguishable from a single underline.
Closes#2916
Carlos Zamora (1)
* UIA: use full buffer comparison in rects and endpoint setter (GH-6447)
Dan Thompson (2)
* Tweaks: normalize TextAttribute method names (adjective form) (GH-6951)
* Fix 'bcz exclusive' typo (GH-6938)
Dustin L. Howett (4)
* Fix VT mouse capture issues in Terminal and conhost (GH-7166)
* version: bump to 1.3 on master
* Update Cascadia Code to 2007.15 (GH-6958)
* Move to the TerminalDependencies NuGet feed (GH-6954)
James Holderness (3)
* Render the SGR "underlined" attribute in the style of the font (CC-7148)
* Add support for the "crossed-out" graphic rendition attribute (CC-7143)
* Refactor grid line renderers with support for more line types (CC-7107)
Leonard Hecker (1)
* Added til::spsc, a lock-free, single-producer/-consumer FIFO queue (CC-6751)
Michael Niksa (6)
* Update TAEF to 10.57.200731005-develop (GH-7164)
* Skip DX invalidation if we've already scrolled an entire screen worth of height (GH-6922)
* Commit attr runs less frequently by accumulating length of color run (GH-6919)
* Set memory order on slow atomics (GH-6920)
* Cache the viewport to make invalidation faster (GH-6918)
* Correct comment in this SPSC test as a quick follow up to merge.
Related work items: MSFT-28208358
Updates TAEF to 10.57.200731005-develop
## PR Checklist
* [x] Helps #6992 by bringing `wttlog.dll` along with the rest of TAEF.
* [x] I work here.
* [x] Automated tests in CI
* [x] No doc/schema update necessary (checked for docs in this repo)
* [x] Am core contributor.
This PR updates the rendering of the _underlined_ graphic rendition
attribute, using the style specified in the active font, instead of just
reusing the grid line at the bottom of the character cell.
* Support for drawing the correct underline effect in the grid line
renderer was added in #7107.
There was already an `ExtendedAttributes` flag defined for the
underlined state, but I needed to update the `SetUnderlined` and
`IsUnderlined` methods in the `TextAttribute` class to use that flag
now in place of the legacy `LVB_UNDERSCORE` attribute. This enables
underlines set via a VT sequence to be tracked separately from
`LVB_UNDERSCORE` grid lines set via the console API.
I then needed to update the `Renderer::s_GetGridlines` method to
activate the `GridLines::Underline` style when the `Underlined`
attribute was set. The `GridLines::Bottom` style is still triggered by
the `LVB_UNDERSCORE` attribute to produce the bottom grid line effect.
Validation
----------
Because this is a change from the existing behaviour, certain unit tests
that were expecting the `LVB_UNDERSCORE` to be toggled by `SGR 4` and
`SGR 24` have now had to be updated to check the `Underlined` flag
instead.
There were also some UI Automation tests that were checking for `SGR 4`
mapping to `LVB_UNDERSCORE` attribute, which I've now substituted with a
test of the `SGR 53` overline attribute mapping to
`LVB_GRID_HORIZONTAL`. These tests only work with legacy attributes, so
they can't access the extended underline state, and I thought a
replacement test that covered similar ground would be better than
dropping the tests altogether.
As far as the visual rendering is concerned, I've manually confirmed
that the VT underline sequences now draw the underline in the correct
position and style, while grid lines output via the console API are
still displayed in their original form.
Closes#2915
- Move to GSL 3.1.0 (GH-6908)
- Replace the color table init code with two const arrays (GH-6913)
- Replace basic_string_view<T> with span<const T> (GH-6921)
- Replace gsl::at with a new til::at(span) for pre-checked bounds (GH-6925)
Related work items: MSFT:27866336
[Git2Git] Git Train: Merge of building/rs_onecore_dep_uxp/200720-1445 into official/rs_onecore_dep_uxp
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_onecore_dep_uxp 5e1509a00f1acdefe1e6392468ffc9ae44dc89cb
Related work items: MSFT:25731998
## Summary of the Pull Request
Text can have various attributes, such as "bold", "italic", "underlined", etc. The TextAttribute class embodies this. It has methods to set/query these attributes.
This change tweaks a few of the method names to make them match. I.e. for an imaginary text property "Foo", we should have methods along the lines of:
```
IsFoo
SetFoo(bool isFoo)
```
And variations should match: we should have "Foo" and "OverFoo", not "Fooey" and "OverFoo".
I chose to standardize on the adjective form, since that's what we are closest to already. The attributes I attacked here are:
SetItalic**s** --> SetItalic
SetUnderline --> SetUnderline**d**
SetOverline --> SetOverline**d**
("italic" is an adjective; "italics" is a plural noun, representing letters or words in an italic typeface)
And I also added methods for "DoublyUnderlined" for good measure.
I stopped short of renaming the GraphicsOptions enum values to match, too; but I'd be willing to do that in a follow-up change if people wanted it.
## Validation Steps Performed
It builds, and tests still pass.
In `Renderer::TriggerRedraw`, the act of fetching the viewport from the
`pData` over and over is wasted time. We already have a cached variable
of the viewport that is updated on every scroll check (on
`TriggerScroll` and on `PaintFrame`.) Scrolling wouldn't be working
correctly if the clients weren't already notifying us that the viewport
has changed for scroll purposes, so we can just keep using that cached
value for the invalidation restriction to speed things up over fetching
it again.
## Validation Steps Performed
- Run `time cat big.txt`. Checked average time before/after, WPR traces
before/after.
## PR Checklist
* [x] Closes perf itch
* [x] I work here
* [x] Manual test
* [x] Documentation irrelevant.
* [x] Schema irrelevant.
* [x] Am core contributor.
The recent changes to use gsl::span everywhere added a few bounds checks
along codepaths where we were already checking bounds. Some of them may
be non-obvious to the optimizer, so we can now use til::at to help them
along.
To accomplish this, I've added a new overload of til::at that takes a
span and directly accesses its backing buffer.
We were using std::basic_string_view as a stand-in for std::span so that
we could change over all at once when C++20 dropped with full span
support. That day's not here yet, but as of 54a7fce3e we're using GSL 3,
whose span is C++20-compliant.
This commit replaces every instance of basic_string_view that was not
referring to an actual string with a span of the appropriate type.
I moved the `const` qualifier into span's `T` because while
`basic_string_view.at()` returns `const T&`, `span.at()` returns `T&`
(without the const). I wanted to maintain the invariant that members of
the span were immutable.
* Mechanical Changes
* `sv.at(x)` -> `gsl::at(sp, x)`
* `sv.c{begin,end}` -> `sp.{begin,end}` (span's iterators are const)
I had to replace a `std::basic_string<>` with a `std::vector<>` in
ConImeInfo, and I chose to replace a manual array walk in
ScreenInfoUiaProviderBase with a ranged-for. Please review those
specifically.
This will almost certainly cause a code size regression in Windows
because I'm blowing out all the PGO counts. Whoops.
Related: #3956, #975.
GSL 3, the next major version of GSL after the one we're using, replaced
their local implementation of `span` with one that more closely mimics
C++20's span. Unfortunately, that is a breaking change for all of GSL's
consumers.
This commit updates our use of span to comply with the new changes in
GSL 3.
Chief among those breaking changes is:
* `span::at` no longer exists; I replaced many instances of `span::at`
with `gsl::at(x)`
* `span::size_type` has finally given up on `ptrdiff_t` and become
`size_t` like all other containers
While I was here, I also made the following mechanical replacements:
* In some of our "early standardized" code, we used std::optional's
`has_value` and `value` back-to-back. Each `value` incurs an
additional presence test.
* Change: `x.value().member` -> `x->member` (`optional::operator->`
skips the presence test)
* Change: `x.value()` -> `*x` (as above)
* GSL 3 uses `size_t` for `size_type`.
* Change: `gsl::narrow<size_t>(x.size())` -> `x.size()`
* Change: `gsl::narrow<ptrdiff_t>(nonSpan.size())` -> `nonSpan.size()`
during span construction
I also replaced two instances of `x[x.size() - 1]` with `x.back()` and
one instance of a manual array walk (for comparison) with a direct
comparison.
NOTE: Span comparison and `make_span` are not part of the C++20 span
library.
Fixes#6251
console: switch to /Zc:wchar_t (native wchar_t)
This matches what we use in OpenConsole and makes {fmt} play nice.
I've also removed the workaround we introduced into OutputCellIterator
to work around not using /Zc:wchar_t.
Fixes MSFT:27626309.
Fixes GH-2673.
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_onecore_dep_uxp 1508f7c232ec58bebc37fedfdec3eb8f9bff5502
## Summary of the Pull Request
This PR adds support for the `SGR 2` escape sequence, which enables the
ANSI _faint_ graphic rendition attribute. When a character is output
with this attribute set, it uses a dimmer version of the active
foreground color.
## PR Checklist
* [x] Closes#6703
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. Issue number where discussion took place: #6703
## Detailed Description of the Pull Request / Additional comments
There was already an `ExtendedAttributes::Faint` flag in the
`TextAttribute` class, but I needed to add `SetFaint` and `IsFaint`
methods to access that flag, and update the `SetGraphicsRendition`
methods of the two dispatchers to set the attribute on receipt of the
`SGR 2` sequence. I also had to update the existing `SGR 22` handler to
reset _Faint_ in addition to _Bold_, since they share the same reset
sequence. For that reason, I thought it a good idea to change the name
of the `SGR 22` enum to `NotBoldOrFaint`.
For the purpose of rendering, I've updated the
`TextAttribute::CalculateRgbColors` method to return a dimmer version of
the foreground color when the _Faint_ attribute is set. This is simply
achieved by dividing each color component by two, which produces a
reasonable effect without being too complicated. Note that the _Faint_
effect is applied before _Reverse Video_, so if the output it reversed,
it's the background that will be faint.
The only other complication was the update of the `Xterm256Engine` in
the VT renderer. As mentioned above, _Bold_ and _Faint_ share the same
reset sequence, so to forward that state over conpty we have to go
through a slightly more complicated process than with other attributes.
We first check whether either attribute needs to be turned off to send
the reset sequence, and then check if the individual attributes need to
be turned on again.
## Validation
I've extended the existing SGR unit tests to cover the new attribute in
the `AdapterTest`, the `ScreenBufferTests`, and the `VtRendererTest`,
and added a test to confirm the color calculations when _Faint_ is set
in the `TextAttributeTests`.
I've also done a bunch of manual testing with all the different VT color
types and confirmed that our output is comparable to most other
terminals.
This is a refactoring of the renderer color calculations to simplify the
implementation, and to make it easier to support additional
color-altering rendition attributes in the future (e.g. _faint_ and
_conceal_).
## References
* This is a followup to PRs #3817 and #6809, which introduced additional
complexity in the color calculations, and which suggested the need for
refactoring.
## Detailed Description of the Pull Request / Additional comments
When we added support for `DECSCNM`, that required the foreground and
background color lookup methods to be able to return the opposite of
what was requested when the reversed mode was set. That made those
methods unnecessarily complicated, and I thought we could simplify them
considerably just by combining the calculations into a single method
that derived both colors at the same time.
And since both conhost and Windows Terminal needed to perform the same
calculations, it also made sense to move that functionality into the
`TextAttribute` class, where it could easily be shared.
In general this way of doing things is a bit more efficient. However, it
does result in some unnecessary work when only one of the colors is
required, as is the case for the gridline painter. So to make that less
of an issue, I've reordered the gridline code a bit so it at least
avoids looking up the colors when no gridlines are needed.
## Validation Steps Performed
Because of the API changes, quite a lot of the unit tests had to be
updated. For example instead of verifying colors with two separate calls
to `LookupForegroundColor` and `LookupBackgroundColor`, that's now
achieved with a single `LookupAttributeColors` call, comparing against a
pair of values. The specifics of the tests haven't changed though, and
they're all still working as expected.
I've also manually confirmed that the various color sequences and
rendition attributes are rendering correctly with the new refactoring.
There is going to be a very long tail of applications that will
explicitly request VT SGR 40/37 when what they really want is to
SetConsoleTextAttribute() with a black background/white foreground.
Instead of making those applications look bad (and therefore making us
look bad, because we're releasing this as an update to something that
"looks good" already), we're introducing this compatibility quirk.
Before the color reckoning in #6698 + #6506, *every* color was subject
to being spontaneously and erroneously turned into the default color.
Now, only the 16-color palette value that matches the active console
background/foreground color will be destroyed, and only when received
from specific applications.
Removal will be tracked by #6807.
Michael and I discussed what layer this quirk really belonged in. I
originally believed it would be sufficient to detect a background color
that matched the legacy default background, but @j4james provided an
example of where that wouldn't work out (powershell setting the
foreground color to white/gray). In addition, it was too heavyhanded: it
re-broke black backgrounds for every application.
Michael thought that it should live in the server, as a small VT parser
that righted the wrongs coming directly out of the application. On
further investigation, however, I realized that we'd need to push more
information up into the server (so that it could make the decision about
which VT was wrong and which was right) than should be strictly
necessary.
The host knows which colors are right and wrong, and it gets final say
in what ends up in the buffer.
Because of that, I chose to push the quirk state down through
WriteConsole to DoWriteConsole and toggle state on the
SCREEN_INFORMATION that indicates whether the colors coming out of the
application are to be distrusted. This quirk _only applies to pwsh.exe
and powershell.exe._
NOTE: This doesn't work for PowerShell the .NET Global tool, because it
is run as an assembly through dotnet.exe. I have no opinion on how to
fix this, or whether it is worth fixing.
VALIDATION
----------
I configured my terminals to have an incredibly garish color scheme to
show exactly what's going to happen as a result of this. The _default
terminal background_ is purple or red, and the foreground green. I've
printed out a heap of test colors to see how black interacts with them.
Pull request #6810 contains the images generated from this test.
The only color lines that change are the ones where black as a
background or white as a foreground is selected out of the 16-color
palette explicitly. Reverse video still works fine (because black is in
the foreground!), and it's even possible to represent "black on default"
and reverse it into "default on black", despite the black in question
having been `40`.
Fixes#6767.
## Summary of the Pull Request
This PR adds support for the `SGR 53` and `SGR 55` escapes sequences,
which enable and disable the ANSI _overline_ graphic rendition
attribute, the equivalent of the console character attribute
`COMMON_LVB_GRID_HORIZONTAL`. When a character is output with this
attribute set, a horizontal line is rendered at the top of the character
cell.
## PR Checklist
* [x] Closes#6000
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already.
## Detailed Description of the Pull Request / Additional comments
To start with, I added `SetOverline` and `IsOverlined` methods to the
`TextAttribute` class, to set and get the legacy
`COMMON_LVB_GRID_HORIZONTAL` attribute. Technically there was already an
`IsTopHorizontalDisplayed` method, but I thought it more readable to add
a separate `IsOverlined` as an alias for that.
Then it was just a matter of adding calls to set and reset the attribute
in response to the `SGR 53` and `SGR 55` sequences in the
`SetGraphicsRendition` methods of the two dispatchers. The actual
rendering was already taken care of by the `PaintBufferGridLines` method
in the rendering engines.
The only other change required was to update the `_UpdateExtendedAttrs`
method in the `Xterm256Engine` of the VT renderer, to ensure the
attribute state would be forwarded to the Windows Terminal over conpty.
## Validation Steps Performed
I've extended the existing SGR unit tests to cover the new attribute in
the `AdapterTest`, the `OutputEngineTest`, and the `VtRendererTest`.
I've also manually tested the `SGR 53` and `SGR 55` sequences to confirm
that they do actually render (or remove) an overline on the characters
being output.
This PR reimplements the VT rendering engines to do a better job of
preserving the original color types when propagating attributes over
ConPTY. For the 16-color renderers it provides better support for
default colors and improves the efficiency of the color narrowing
conversions. It also fixes problems with the ordering of character
renditions that could result in attributes being dropped.
Originally the base renderer would calculate the RGB color values and
legacy/extended attributes up front, passing that data on to the active
engine's `UpdateDrawingBrushes` method. With this new implementation,
the renderer now just passes through the original `TextAttribute` along
with an `IRenderData` interface, and leaves it to the engines to extract
the information they need.
The GDI and DirectX engines now have to lookup the RGB colors themselves
(via simple `IRenderData` calls), but have no need for the other
attributes. The VT engines extract the information that they need from
the `TextAttribute`, instead of having to reverse engineer it from
`COLORREF`s.
The process for the 256-color Xterm engine starts with a check for
default colors. If both foreground and background are default, it
outputs a SGR 0 reset, and clears the `_lastTextAttribute` completely to
make sure any reset state is reapplied. With that out the way, the
foreground and background are updated (if changed) in one of 4 ways.
They can either be a default value (SGR 39 and 49), a 16-color index
(using ANSI or AIX sequences), a 256-color index, or a 24-bit RGB value
(both using SGR 38 and 48 sequences).
Then once the colors are accounted for, there is a separate step that
handles the character rendition attributes (bold, italics, underline,
etc.) This step must come _after_ the color sequences, in case a SGR
reset is required, which would otherwise have cleared any character
rendition attributes if it came last (which is what happened in the
original implementation).
The process for the 16-color engines is a little different. The target
client in this case (Windows telnet) is incapable of setting default
colors individually, so we need to output an SGR 0 reset if _either_
color has changed to default. With that out the way, we use the
`TextColor::GetLegacyIndex` method to obtain an approximate 16-color
index for each color, and apply the bold attribute by brightening the
foreground index (setting bit 8) if the color type permits that.
However, since Windows telnet only supports the 8 basic ANSI colors, the
best we can do for bright colors is to output an SGR 1 attribute to get
a bright foreground. There is nothing we can do about a bright
background, so after that we just have to drop the high bit from the
colors. If the resulting index values have changed from what they were
before, we then output ANSI 8-color SGR sequences to update them.
As with the 256-color engine, there is also a final step to handle the
character rendition attributes. But in this case, the only supported
attributes are underline and reversed video.
Since the VT engines no longer depend on the active color table and
default color values, there was quite a lot of code that could now be
removed. This included the `IDefaultColorProvider` interface and
implementations, the `Find(Nearest)TableIndex` functions, and also the
associated HLS conversion and difference calculations.
VALIDATION
Other than simple API parameter changes, the majority of updates
required in the unit tests were to correct assumptions about the way the
colors should be rendered, which were the source of the narrowing bugs
this PR was trying to fix. Like passing white on black to the
`UpdateDrawingBrushes` API, and expecting it to output the default `SGR
0` sequence, or passing an RGB color and expecting an indexed SGR
sequence.
In addition to that, I've added some VT renderer tests to make sure the
rendition attributes (bold, underline, etc) are correctly retained when
a default color update causes an `SGR 0` sequence to be generated (the
source of bug #3076). And I've extended the VT renderer color tests
(both 256-color and 16-color) to make sure we're covering all of the
different color types (default, RGB, and both forms of indexed colors).
I've also tried to manually verify that all of the test cases in the
linked bug reports (and their associated duplicates) are now fixed when
this PR is applied.
Closes#2661Closes#3076Closes#3717Closes#5384Closes#5864
This is only a partial fix for #293, but I suspect the remaining cases
are unfixable.
Essentially what this does is map the default legacy foreground and
background attributes (typically white on black) to the `IsDefault`
color type in the `TextColor` class. As a result, we can now initialize
the buffer for "legacy" shells (like PowerShell and cmd.exe) with
default colors, instead of white on black. This fixes the startup
rendering in conpty clients, which expect an initial default background
color. It also makes these colors update appropriately when the default
palette values change.
One complication in getting this to work, is that the console permits
users to change which color indices are designated as defaults, so we
can't assume they'll always be white on black. This means that the
legacy-to-`TextAttribute` conversion will need access to those default
values.
Unfortunately the defaults are stored in the conhost `Settings` class
(the `_wFillAttribute` field), which isn't easily accessible to all the
code that needs to construct a `TextAttribute` from a legacy value. The
`OutputCellIterator` is particularly problematic, because some iterator
types need to generate a new `TextAttribute` on every iteration.
So after trying a couple of different approaches, I decided that the
least worst option would be to add a pair of static properties for the
legacy defaults in the `TextAttribute` class itself, then refresh those
values from the `Settings` class whenever the defaults changed (this
only happens on startup, or when the conhost _Properties_ dialog is
edited).
And once the `TextAttribute` class had access to those defaults, it was
fairly easy to adapt the constructor to handle the conversion of default
values to the `IsDefault` color type. I could also then simplify the
`TextAttribute::GetLegacyAttributes` method which does the reverse
mapping, and which previously required the default values to be passed
in as a parameter
VALIDATION
I had to make one small change to the `TestRoundtripExhaustive` unit
test which assumed that all legacy attributes would convert to legacy
color types, which is no longer the case, but otherwise all the existing
tests passed as is. I added a new unit test verifying that the default
legacy attributes correctly mapped to default color types, and the
default color types were mapped back to the correct legacy attributes.
I've manually confirmed that this fixed the issue raised in #5952,
namely that the conhost screen is cleared with the correct default
colors, and also that it is correctly refreshed when changing the
palette from the properties dialog. And I've combined this PR with
#6506, and confirmed that the PowerShell and the cmd shell renderings in
Windows Terminal are at least improved, if not always perfect.
This is a prerequisite for PR #6506Closes#5952
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
Many places in this codebase has an equality comparison to the boolean FALSE. This adds unneeded complexity as C and C++ has a NOT operand for use of these in if statements. This makes the code more readable in those areas.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
## References
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [X] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [X] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
One boolean being compared to FALSE was only used once, with the boolean name being "b", so it is better off not existing at all.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
Unit Testing passed, compiler refactoring
Bump Newtonsoft.Json from 10.0.3 to 12.0.3
## References
Part of #5297
## PR Checklist
* [ ] Closes (none)
* [x] CLA signed
* [ ] Tests added/passed N/A
* [ ] Requires documentation to be updated N/A
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Validation Steps Performed
CI builds successfully
This pull request moves WindowUiaProvider back into Win32 interactivity
and deletes all mention of it from Windows Terminal. Terminal does not
have a single toplevel window that requires Console-like UIA, as each
Xaml control inside it is in charge of its own destiny.
I've also merged `IUiaWindow` and `IConsoleWindow` back together, as
well as `WindowUiaProviderBase` and `WindowUiaProvider`.
Things look a lot more like they did before we tore them apart.
## PR Checklist
* [x] Closes#3564
* [x] CLA
* [x] Tests added/passed (manual)
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already
## Validation
Carlos validated conhost and terminal on this branch.
Adds support for `win32-input-mode` to conhost, conpty, and the Windows
Terminal.
* The shared `terminalInput` class supports sending these sequences when
a VT client application requests this mode.
* ConPTY supports synthesizing `INPUT_RECORD`s from the input sent to it
from a terminal
* ConPTY requests this mode immediately on startup (if started with a
new flag, `PSEUDOCONSOLE_WIN32_INPUT_MODE`)
* The Terminal now supports sending this input as well, when conpty asks
for it.
Also adds a new ConPTY flag `PSEUDOCONSOLE_WIN32_INPUT_MODE` which
requests this functionality from conpty, and the Terminal requests this
by default.
Also adds `experimental.input.forceVT` as a global setting to let a user
opt-out of this behavior, if they don't want it / this ends up breaking
horribly.
## Validation Steps Performed
* played with this mode in vtpipeterm
* played with this mode in Terminal
* checked a bunch of scenarios, as outlined in a [comment] on #4999
[comment]: https://github.com/microsoft/terminal/issues/4999#issuecomment-628718631
References #4999: The megathread
References #5887: The spec
Closes#879Closes#2865Closes#530Closes#3079Closes#1119Closes#1694Closes#3608Closes#4334Closes#4446
This PR provides a faster algorithm for converting 8-bit and 24-bit
colors into the 4-bit legacy values that are required by the Win32
console APIs. It also fixes areas of the code that were incorrectly
using a simple 16-color conversion that didn't handle 8-bit and 24-bit
values.
The faster conversion algorithm should be an improvement for issues #783
and #3950.
One of the main points of this PR was to fix the
`ReadConsoleOutputAttribute` API, which was using a simplified legacy
color conversion (the original `TextAttribute:GetLegacyAttributes`
method), which could only handle values from the 16-color table. RGB
values, and colors from the 256-color table, would be mapped to
completely nonsensical values. This API has now been updated to use the
more correct `Settings::GenerateLegacyAttributes` method.
But there were also a couple of other places in the code that were using
`GetLegacyAttributes` when they really had no reason to be working with
legacy attributes at all. This could result in colors being downgraded
to 4-bit values (often badly, as explained above), when the code was
already perfectly capable of displaying the full 24-bits.
This included the fill colors in the IME composer (in `ConsoleImeInfo`),
and the construction of the highlighting colors in the color
search/selection handler (`Selection::_HandleColorSelection`). I also
got rid of some legacy attribute code in the `Popup` class, which was
originally intended to update colors below the popup when the settings
changed, but actually caused more problems than it solved.
The other major goal of this PR was to improve the performance of the
`GenerateLegacyAttributes` method, since the existing implementation
could be quite slow when dealing with RGB values.
The simple cases are handled much the same as they were before. For an
`IsDefault` color, we get the default index from the
`Settings::_wFillAttribute` field. For an `IsIndex16` color, the index
can just be returned as is.
For an `IsRgb` color, the RGB components are compressed down to 8 bits
(3 red, 3 green, 2 blue), simply by dropping the least significant bits.
This 8-bit value is then used to lookup a representative 16-color value
from a hard-coded table. An `IsIndex256` color is also converted with a
lookup table, just using the existing 8-bit index.
The RGB mapping table was calculated by taking each compressed 8-bit
color, and picking a entry from the _Campbell_ palette that best
approximated that color. This was done by looking at a range of 24-bit
colors that mapped to the 8-bit value, finding the best _Campbell_ match
for each of them (using a [CIEDE2000] color difference calculation), and
then the most common match became the index that the 8-bit value would
map to.
The 256-color table was just a simpler version of this process. For each
entry in the table, we take the default RGB palette value, and find it's
closest match in the _Campbell_ palette.
Because these tables are hard-coded, the results won't adjust to changes
in the palette. However, they should still produce reasonable results
for palettes that follow the standard ANSI color range. And since
they're only a very loose approximation of the colors anyway, the exact
value really isn't that important.
That said, I have tried to make sure that if you take an RGB value for a
particular index in a reasonable color scheme, then the legacy color
mapped from that value should ideally match the same index. This will
never be possible for all color schemes, but I have tweaked a few of the
table entries to improve the results for some of the common schemes.
One other point worth making regarding the hard-coded tables: even if we
wanted to take the active palette into account, that wouldn't actually
be possible over a conpty connection, because we can't easily know what
color scheme the client application is using. At least this way the
results in conhost are guaranteed to be the same as in the Windows
Terminal.
[CIEDE2000]: https://en.wikipedia.org/wiki/Color_difference#CIEDE2000
## Validation Steps Performed
This code still passes the `TextAttributeTests` that check the basic
`GetLegacyAttribute` behaviour and verify the all legacy attributes
roundtrip correctly. However, some of the values in the `RgbColorTests`
had to be updated, since we're now intentionally returning different
values as a result of the changes to the RGB conversion algorithm.
I haven't added additional unit tests, but I have done a lot of manual
testing to see how well the new algorithm works with a range of colors
and a variety of different color schemes. It's not perfect in every
situation, but I think it works well enough for the purpose it serves.
I've also confirmed that the issues reported in #5940 and #6247 are now
fixed by these changes.
Closes#5940Closes#6247
In Windows, we build with /Zc:wchar_t- (which makes wchar_t an unsigned
short typedef.) This causes build breaks when we compare two wchar_t
values (or a wchar_t and an enum class that's of type wchar_t) and the
compiler decides that it might want to _promote them to TextAttribute_
before doing the comparison.
This PR improves our VT character set support, enabling the [`SCS`]
escape sequences to designate into all four G-sets with both 94- and
96-character sets, and supports invoking those G-sets into both the GL
and GR areas of the code table, with [locking shifts] and [single
shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the
ISO-2022 coding system (which is what the VT character sets require),
and adds support for a lot more characters sets, up to around the level
of a VT510.
[`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html
[locking shifts]: https://vt100.net/docs/vt510-rm/LS.html
[single shifts]: https://vt100.net/docs/vt510-rm/SS.html
[`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems
## Detailed Description of the Pull Request / Additional comments
To make it easier for us to declare a bunch of character sets, I've made
a little `constexpr` class that can build up a mapping table from a base
character set (ASCII or Latin1), along with a collection of mappings for
the characters the deviate from the base set. Many of the character sets
are simple variations of ASCII, so they're easy to define this way.
This class then casts directly to a `wstring_view` which is how the
translation tables are represented in most of the code. We have an array
of four of these tables representing the four G-sets, two instances for
the active left and right tables, and one instance for the single shift
table.
Initially we had just one `DesignateCharset` method, which could select
the active character set. We now have two designate methods (for 94- and
96- character sets), and each takes a G-set number specifying the target
of the designation, and a pair of characters identifying the character
set that will be designated (at the higher VT levels, character sets are
often identified by more than one character).
There are then two new `LockingShift` methods to invoke these G-sets
into either the GL or GR area of the code table, and a `SingleShift`
method which invokes a G-set temporarily (for just the next character
that is output).
I should mention here that I had to make some changes to the state
machine to make these single shift sequences work. The problem is that
the input state machine treats `SS3` as the start of a control sequence,
while the output state machine needs it to be dispatched immediately
(it's literally the _Single Shift 3_ escape sequence). To make that
work, I've added a `ParseControlSequenceAfterSs3` callback in the
`IStateMachineEngine` interface to decide which behavior is appropriate.
When it comes to mapping a character, it's simply an array reference
into the appropriate `wstring_view` table. If the single shift table is
set, that takes preference. Otherwise the GL table is used for
characters in the range 0x20 to 0x7F, and the GR table for characters
0xA0 to 0xFF (technically some character sets will only map up to 0x7E
and 0xFE, but that's easily controlled by the length of the
`wstring_view`).
The `DEL` character is a bit of a special case. By default it's meant to
be ignored like the `NUL` character (it's essentially a time-fill
character). However, it's possible that it could be remapped to a
printable character in a 96-character set, so we need to check for that
after the translation. This is handled in the `AdaptDispatch::Print`
method, so it doesn't interfere with the primary `PrintString` code
path.
The biggest problem with this whole process, though, is that the GR
mappings only really make sense if you have access to the raw output,
but by the time the output gets to us, it would already have been
translated to Unicode by the active code page. And in the case of UTF-8,
the characters we eventually receive may originally have been composed
from two or more code points.
The way I've dealt with this was to disable the GR translations by
default, and then added support for a pair of ISO-2022 `DOCS` sequences,
which can switch the code page between UTF-8 and ISO-8859-1. When the
code page is ISO-8859-1, we're essentially receiving the raw output
bytes, so it's safe to enable the GR translations. This is not strictly
correct ISO-2022 behavior, and there are edge cases where it's not going
to work, but it's the best solution I could come up with.
## Validation Steps Performed
As a result of the `SS3` changes in the state machine engine, I've had
to move the existing `SS3` tests from the `OutputEngineTest` to the
`InputEngineTest`, otherwise they would now fail (technically they
should never have been output tests).
I've added no additional unit tests, but I have done a lot of manual
testing, and made sure we passed all the character set tests in Vttest
(at least for the character sets we currently support). Note that this
required a slightly hacked version of the app, since by default it
doesn't expose a lot of the test to low-level terminals, and we
currently identify as a VT100.
Closes#3377Closes#3487
## Summary of the Pull Request
This is the plan that @miniksa suggested to me. Instead of trying to do lots of work in all the renderers to do backgrounds as one pass, and foregrounds as another, we can localize this change to basically just the DX renderer.
1. First, we give the DX engine a "heads up" on where the cursor is going to be drawn during the frame, in `PrepareRenderInfo`.
- This function is left unimplemented in the other render engines.
2. While printing runs of text, the DX renderer will try to paint the cursor in `CustomTextRenderer::DrawGlyphRun` INSTEAD of `DxEngine::PaintCursor`. This lets us weave the cursor background between the text background and the text.
## References
* #6151 was a spec in this general area. I should probably go back and update it, and we should probably approve that first.
* #6193 is also right up in this mess
## PR Checklist
* [x] Closes#1203
* [x] I work here
* [ ] Tests added/passed
* [n/a] Requires documentation to be updated
## Detailed Description of the Pull Request / Additional comments
* This is essentially `"cursorTextColor": "textForeground"` from #6151.
* A follow up work item is needed to add support for the current behavior, (`"cursorTextColor": null`), and hooking up that setting to the renderer.
## Summary of the Pull Request
Restores proper line drawing during IME operations in `conhost`
## PR Checklist
* [x] Closes#803
* [x] I work here.
* [x] Tested manually.
* [x] Check the performance of this and see if it's worse-enough to merit a more confusing algorithm. It was worse for the majority case so I scoped it.
* [x] No doc, it should have worked this way.
* [x] Am core contributor.
## Detailed Description of the Pull Request / Additional comments
- Changed `ConsoleImeInfo::s_ConvertToCells` to be less confusing. It's doing about the same thing, but it's way easier to read now and the compiler/linker/optimizer should just be the same.
- Edited `Renderer::_PaintBufferOutputHelper` to check each attribute for line drawing characters as the right half of a two-col character might have different line drawing characters than the left-half.
## Validation Steps Performed
- [x] Manual operation of IME in conhost with Japanese IME.
- [x] Manual operation of IME in conhost with Chinese IME.
- [x] Manual operation of IME in conhost with Chinese (Traditional) IME.
- [x] Manual operation of IME in conhost with and Korean IME. - @leonMSFT says Korean doesn't work this way. But Korean is broken worse in that it's not showing suggestions at all. Filing new bug. #6227
- [x] Validated against API-filling calls through `SetConsoleTextAttribute` per @j4james's sample code
## Summary of the Pull Request
This PR adds support for the core VT52 commands, and implements the `DECANM` private mode sequence, which switches the terminal between ANSI mode and VT52-compatible mode.
## References
PR #2017 defined the initial specification for VT52 support.
PR #4044 removed the original VT52 cursor ops that conflicted with VT100 sequences.
## PR Checklist
* [x] Closes#976
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #2017
## Detailed Description of the Pull Request / Additional comments
Most of the work involves updates to the parsing state machine, which behaves differently in VT52 mode. `CSI`, `OSC`, and `SS3` sequences are not applicable, and there is one special-case escape sequence (_Direct Cursor Address_), which requires an additional state to handle parameters that come _after_ the final character.
Once the parsing is handled though, it's mostly just a matter of dispatching the commands to existing methods in the `ITermDispatch` interface. Only one new method was required in the interface to handle the _Identify_ command.
The only real new functionality is in the `TerminalInput` class, which needs to generate different escape sequences for certain keys in VT52 mode. This does not yet support _all_ of the VT52 key sequences, because the VT100 support is itself not yet complete. But the basics are in place, and I think the rest is best left for a follow-up issue, and potentially a refactor of the `TerminalInput` class.
I should point out that the original spec called for a new _Graphic Mode_ character set, but I've since discovered that the VT terminals that _emulate_ VT52 just use the existing VT100 _Special Graphics_ set, so that is really what we should be doing too. We can always consider adding the VT52 graphic set as a option later, if there is demand for strict VT52 compatibility.
## Validation Steps Performed
I've added state machine and adapter tests to confirm that the `DECANM` mode changing sequences are correctly dispatched and forwarded to the `ConGetSet` handler. I've also added state machine tests that confirm the VT52 escape sequences are dispatched correctly when the ANSI mode is reset.
For fuzzing support, I've extended the VT command fuzzer to generate the different kinds of VT52 sequences, as well as mode change sequences to switch between the ANSI and VT52 modes.
In terms of manual testing, I've confirmed that the _Test of VT52 mode_ in Vttest now works as expected.
This PR introduces a new `ColorType` to allow us to distinguish between
`SGR` indexed colors from the 16 color table, the lower half of which
can be brightened, and the ISO/ITU indexed colors from the 256 color
table, which have a fixed brightness. Retaining the distinction between
these two types will enable us to forward the correct `SGR` sequences to
conpty when addressing issue #2661.
The other benefit of retaining the color index (which we didn't
previously do for ISO/ITU colors) is that it ensures that the colors are
updated correctly when the color scheme is changed.
## References
* This is another step towards fixing the conpty narrowing bugs in issue
#2661.
* This is technically a fix for issue #5384, but that won't be apparent
until #2661 is complete.
## PR Checklist
* [x] Closes#1223
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already.
## Detailed Description of the Pull Request / Additional comments
The first part of this PR was the introduction of a new `ColorType` in
the `TextColor` class. Instead of just the one `IsIndex` type, there is
now an `IsIndex16` and an `IsIndex256`. `IsIndex16` covers the eight
original ANSI colors set with `SGR 3x` and `SGR 4x`, as well as the
brighter aixterm variants set with `SGR 9x` and `SGR 10x`. `IsIndex256`
covers the 256 ISO/ITU indexed colors set with `SGR 38;5` and `SGR
48;5`.
There are two reasons for this distinction. The first is that the ANSI
colors have the potential to be brightened by the `SGR 1` bold
attribute, while the ISO/ITO color do not. The second reason is that
when forwarding an attributes through conpty, we want to try and
preserve the original SGR sequence that generated each color (to the
extent that that is possible). By having the two separate types, we can
map the `IsIndex16` colors back to ANSI/aixterm values, and `IsIndex256`
to the ISO/ITU sequences.
In addition to the VT colors, we also have to deal with the legacy
colors set by the Windows console APIs, but we don't really need a
separate type for those. It seemed most appropriate to me to store them
as `IsIndex256` colors, since it doesn't make sense to have them
brightened by the `SGR 1` attribute (which is what would happen if they
were stored as `IsIndex16`). If a console app wanted a bright color it
would have selected one, so we shouldn't be messing with that choice.
The second part of the PR was the unification of the two color tables.
Originally we had a 16 color table for the legacy colors, and a separate
table for the 256 ISO/ITU colors. These have now been merged into one,
so color table lookups no longer need to decide which of the two tables
they should be referencing. I've also updated all the methods that took
a color table as a parameter to use a `basic_string_view` instead of
separate pointer and length variables, which I think makes them a lot
easier and safer to work with.
With this new architecture in place, I could now update the
`AdaptDispatch` SGR implementation to store the ISO/ITU indexed colors
as `IsIndex256` values, where before they were mapped to RGB values
(which prevented them reflecting any color scheme changes). I could also
update the `TerminalDispatch` implementation to differentiate between
the two index types, so that the `SGR 1` brightening would only be
applied to the ANSI colors.
I've also done a bit of code refactoring to try and minimise any direct
access to the color tables, getting rid of a lot of places that were
copying tables with `memmove` operations. I'm hoping this will make it
easier for us to update the code in the future if we want to reorder the
table entries (which is likely a requirement for unifying the
`AdaptDispatch` and `TerminalDispatch` implementations).
## Validation Steps Performed
For testing, I've just updated the existing unit tests to account for
the API changes. The `TextColorTests` required an extra parameter
specifying the index type when setting an index. And the `AdapterTest`
and `ScreenBufferTests` required the use of the new `SetIndexedXXX`
methods in order to be explicit about the index type, instead of relying
on the `TextAttribute` constructor and the old `SetForeground` and
`SetBackground` methods which didn't have a way to differentiate index
types.
I've manually tested the various console APIs
(`SetConsoleTextAttribute`, `ReadConsoleOutputAttribute`, and
`ReadConsoleOutput`), to make sure they are still setting and reading
the attributes as well as they used to. And I've tested the
`SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs
to make sure they can read and write the color table correctly. I've
also tested the color table in the properties dialog, made sure it was
saved and restored from the registry correctly, and similarly saved and
restored from a shortcut link.
Note that there are still a bunch of issues with the color table APIs,
but no new problems have been introduced by the changes in this PR, as
far as I could tell.
I've also done a bunch of manual tests of `OSC 4` to make sure it's
updating all the colors correctly (at least in conhost), and confirmed
that the test case in issue #1223 now works as expected.
## Summary of the Pull Request
When using an _Input Method Editor_ in conhost for East Asian languages, the text cursor is temporarily hidden while the characters are being composed. When the composition is complete, the cursor visibility is meant to be restored, but that doesn't always happen if the IME composition is cancelled. This PR makes sure the cursor visibility is always restored, regardless of how the IME is closed.
## PR Checklist
* [x] Closes#810
* [x] CLA signed.
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan.
## Detailed Description of the Pull Request / Additional comments
The original implementation hid the cursor whenever `ConsoleImeInfo::WriteCompMessage` was called (which could be multiple times in the course of a composition), and then only restored the visibility when `ConsoleImeInfo::WriteResultMessage` was called. If a composition is cancelled, though, `WriteResultMessage` would never be called, so the cursor visibility wouldn't be restored.
I've now made the `SaveCursorVisibility` and `RestoreCursorVisibility` methods public, so they can instead be called from the `ImeStartComposition` and `ImeEndComposition` functions. This makes sure `RestoreCursorVisibility` is always called, regardless of how the composition ended, and `SaveCursorVisibility` is only called once at the start of the composition (which isn't essential, but seems cleaner to me).
## Validation Steps Performed
I've manually tested opening and closing the IME, both while submitting characters and while cancelling a composition, and in all cases the cursor visibility was correctly restored.
This is an attempt to simplify the SGR (Select Graphic Rendition)
implementation in conhost, to cut down on the number of methods required
in the `ConGetSet` interface, and pave the way for future improvements
and bug fixes. It already fixes one bug that prevented SGR 0 from being
correctly applied when combined with meta attributes.
* This a first step towards fixing the conpty narrowing bugs in issue
#2661
* I'm hoping the simplification of `ConGetSet` will also help with
#3849.
* Some of the `TextAttribute` refactoring in this PR overlaps with
similar work in PR #1978.
## Detailed Description of the Pull Request / Additional comments
The main point of this PR was to simplify the
`AdaptDispatch::SetGraphicsRendition` implementation. So instead of
having it call a half a dozen methods in the `ConGetSet` API, depending
on what kinds of attributes needed to be set, there is now just one call
to get current attributes, and another call to set the new value. All
adjustments to the attributes are made in the `AdaptDispatch` class, in
a simple switch statement.
To help with this refactoring, I also made some change to the
`TextAttribute` class to make it easier to work with. This included
adding a set of methods for setting (and getting) the individual
attribute flags, instead of having the calling code being exposed to the
internal attribute structures and messing with bit manipulation. I've
tried to get rid of any methods that were directly setting legacy, meta,
and extended attributes.
Other than the fix to the `SGR 0` bug, the `AdaptDispatch` refactoring
mostly follows the behaviour of the original code. In particular, it
still maps the `SGR 38/48` indexed colors to RGB instead of retaining
the index, which is what we ultimately need it to do. Fixing that will
first require the color tables to be unified (issue #1223), which I'm
hoping to address in a followup PR.
But for now, mapping the indexed colors to RGB values required adding an
an additional `ConGetSet` API to lookup the color table entries. In the
future that won't be necessary, but the API will still be useful for
other color reporting operations that we may want to support. I've made
this API, and the existing setter, standardise on index values being in
the "Xterm" order, since that'll be essential for unifying the code with
the terminal adapter one day.
I should also point out one minor change to the `SGR 38/48` behavior,
which is that out-of-range RGB colors are now ignored rather than being
clamped, since that matches the way Xterm works.
## Validation Steps Performed
This refactoring has obviously required corresponding changes to the
unit tests, but most were just minor updates to use the new
`TextAttribute` methods without any real change in behavior. However,
the adapter tests did require significant changes to accommodate the new
`ConGetSet` API. The basic structure of the tests remain the same, but
the simpler API has meant fewer values needed to be checked in each test
case. I think they are all still covering the areas there were intended
to, though, and they are all still passing.
Other than getting the unit tests to work, I've also done a bunch of
manual testing of my own. I've made sure the color tests in Vttest all
still work as well as they used to. And I've confirmed that the test
case from issue #5341 is now working correctly.
Closes#5341
[Git2Git] Merged PR 4644345: conhost: disable the DX renderer in inbox builds
We're going to be taking on some changes to the Dx renderer that are at
the very least annoying and at the very most inconsequential to the
inbox console. This commit removes support for the DX renderer from the
inbox console.
SizeBench reports that ConRenderDx contributes 55.1kb to the conhost
image (as its third largest constituent library), so this should net us
a couple pleasant WPG improvements down the line.
Related work items: #26291552 Retrieved from https://microsoft.visualstudio.com os OS official/rs_onecore_dep_uxp 6e36786d447b7975298ba31ccd77c5c649fbfbe6
Related work items: #26291552
[Git2Git] Git Train: Merge of building/rs_onecore_dep_uxp/200504-1008 into official/rs_onecore_dep_uxp Retrieved from https://microsoft.visualstudio.com os OS official/rs_onecore_dep_uxp bbdf04608ba96c3f8ee06cf100428cde01f3df79
Related work items: #26071826
The Erase All VT sequence (`^[[2J`) is supposed to erase the entire
contents of the viewport. The way it usually does this is by shifting
the entirety of the viewport contents into scrollback, and starting the
new viewport below it.
Currently, conpty doesn't propagate that state change correctly. When
conpty gets a 2J, it simply erases the content of the connected
terminal's viewport, by writing over it with spaces. Conpty didn't
really have a good way of communicating "your viewport should move", it
only knew "the buffer is now full of spaces".
This would lead to bugs like #2832, where pressing <kbd>ctrl+L</kbd> in
`bash` would delete the current contents of the viewport, instead of
moving the viewport down.
This PR makes sure that when conpty sees a 2J, it passes that through
directly to the connected terminal application as well. Fortunately, 2J
was already implemented in the Windows Terminal, so this actually fixes
the behavior of <kbd>ctrl+L</kbd>/`clear` in WSL in the Terminal.
## References
* #4252 - right now this isn't the _most_ optimal scenario, we're
literally just printing a 2J, then we'll perform "erase line" `height`
times. The erase line operations are all redundant at this point - the
entire viewport is blank, but conpty doesn't really know that.
Fortunately, #4252 was already filed for me to come through and
optimize this path.
## PR Checklist
* [x] Closes#2832
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
## Validation Steps Performed
* ran tests
* compared <kbd>ctrl+L</kbd> with its behavior in conhost
* compared `clear` with its behavior in conhost
## Summary of the Pull Request
This PR implements a pair of shims for `cmd` and `powershell`, so that their `cls` and `Clear-Host` functions will clear the entire terminal buffer (like they do in conhost), instead of just the viewport. With the conpty viewport and buffer being the same size, there's effectively no way to know if an application is calling these API's in this way with the intention of clearing the buffer or the viewport. We absolutely have to guess.
Each of these shims checks to see if the way that the API is being called exactly matches the way `cmd` or `powershell` would call these APIs. If it does, we manually write a `^[[3J` to the connected terminal, to get he Terminal to clear it's own scrollback.
~~_⚠️ If another application were trying to clear the **viewport** with an exactly similar API call, this would also cause the terminal scrollback to get cleared ⚠️_~~
* [x] Should these shims be restricted to when the process that's calling them is actually `cmd.exe` or `powershell.exe`? Can I even do this? I think we've done such a good job of isolating the client process information from the rest of the host code that I can't figure out how to do this.
- YES, this can be done, and I did it.
* [ ] **TODO**: _While I'm here_, should I have `DoSrvPrivateEraseAll` (the implementation for `^[[2J`, in `getset.cpp`) also manually trigger a EraseAll in the terminal in conpty mode?
## PR Checklist
* [x] Closes#3126
* [x] Actually closes#1305 too, which is really the same thing, but probably deserves a callout
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
## Validation Steps Performed
* ran tests
* checked `cls` in the Terminal
* checked `Clear-Host` in the Terminal
* Checked running `powershell clear-host` from `cmd.exe`
If a class has a constructor which can be called with a single argument,
then this constructor becomes conversion constructor because such a
constructor allows conversion of the single argument to the class being
constructed. To ensure these constructors are passed the argument of its
type, I labeled them explicit.
In some header files, there were constructors that took a value that
could involve implicit conversions, so I added explicit to ensure that
does not happen.
