Until there's a "Wait", there's usually only one API message inflight at
a time.
In our quest for performance, we put that single API message in charge
of its own buffer management: instead of allocating buffers on the heap
and deleting them later (storing pointers to them at the far corners of
the earth), it would instead allocate them from small internal pools (if
possible) and only heap allocate (transparently) if necessary. The
pointers flung to the corners of the earth would be pointers (1) back
into the API_MSG or (2) to a heap block owned by boost::small_vector.
It took us months to realize that those bare pointers were being held by
COOKED_READ and RAW_READ and not actually being updated when the API
message was _copied_ as it was shuffled off to the background to become
a "Wait" message.
It turns out that it's trivially possible to crash the console by
sending two API calls--one that waits and one that completes
immediately--when the waiting message or the "wait completer" has a
bunch of dangling pointers in it. It further turns out that some
accessibility software (like JAWS) attaches directly to the console
session, much like winpty and ConEmu and friends. They're trying to read
out the buffer (API call!) and sometimes there's a shell waiting for
input (API call!). Oops.
In this commit, we fix up the message's internal pointers (in lieu of
giving it a proper copy constructor; see GH-10076) and then tell the
wait completion routine (which is going to be a COOKED_READ, RAW_READ,
DirectRead or WriteData) about the new buffer location.
This is a scoped fix that should be replaced (TODO GH-10076) with a
final one after Ask mode.
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev eca0875950fd3a9735662474613405e2dc06f485
References GH-10076
Fixes MSFT-33127449
Fixes GH-9692
The ES folks removed support for implicit binary dependencies in test projects, which required our testmd to change to include its dependency on testnetv5.
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev a2a74747feda018c347ba787e166e8385bb17e47
Related work items: MSFT-31480819
When we encounter clipboard input that cannot be mapped to a keyboard
key, we try our best to map it to an event. If if is an alphanumeric
character or a wide glyph (per the old width algorithm), we will pass it
through as the UnicodeChar of a key event. If it's *not* wide and *not*
alphanumeric, we will synthesize a set of Alt+NumPad events.
Those events comprise a set containing...
1. Alt Down (modifiers = LAlt, UnicodeChar = 0)
2. Numpad 0 Down/Up (modifiers = LAlt, UnicodeChar = 0)
3. Numpad 1 Down/Up (modifiers = LAlt, UnicodeChar = 0)
4. Numpad 2 Down/Up (modifiers = LAlt, UnicodeChar = 0)
5. Alt Up (modifiers = 0, UnicodeChar = [THE CHARACTER])
Because of event group 5, application developers seem to have taken a
dependency on receiving Alt Up + Character and don't seek to recompose
the original character from its numpad representation.
In pull request GH-8035 (!5394370), we stripped the old width algorithm
out and replaced it with a lookup table (finally!). Unfortunately, that
broke clipboard input for Chinese text as it was no longer considered
"Wide" for the purposes of detecting whether we should use numpad
events.
This commit introduces a version of GetQuickCharWidth that fulfills the
exact contract CharToKeyEvents needs, and doesn't answer for a codepoint
more. We'll use it in Windows to fix MSFT-32901370.
The Terminal analogue of this bug, GH-9052, is fixed by *never emitting
numpad events again.* We think this is okay because it looks like nobody
was ever handling numpad events... and that we were only using them as a
way to communicate within conhost (which we're *also* not using) and any
public exposition of event 5 as a contract was unintended.
VALIDATION
----------
I took this new implementation (with an early return) and the old
implementation and compared whether they would yield a numpad event or a
key event over the entire supported codepoint space [0000..FFFF].
They matched.
Fixes MSFT-32901370
Fixes GH-9052
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 224751bbb061f5f59d794c2c9bdac5a9674ebde6
Add flag that will ensure we do not handoff to a registered default console. Also for a bonus, allow double-click launches or explicit command line launches of conhost.exe with a binary argument to open with the inbox one.
## PR Checklist
* [x] Closes#9791
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
* [x] Automated tests for parsing
* [x] Enable defapp, double click conhost.exe, opens as itself
* [x] Enable defapp, run conhost.exe runbox, opens as itself
* [x] Enable defapp, run conhost.exe powershell.exe runbox, opens as itself
* [x] Runbox cmd.exe/pwsh.exe trigger defapp handoff to Terminal
* [x] Shortcut to cmd.exe/pwsh.exe trigger defapp handoff to Terminal
* [x] Use CHOP tool to launch conhost with a server handle triggers handoff to Terminal
* [x] Use CHOP tool to launch conhost with a server handle and -ForceNoHandoff opens as itself
Add names to threads to make debugging a slight bit easier.
## PR Checklist
* [x] Closes personal todo item.
* [x] I work here.
* [x] Tested manually.
## Detailed Description of the Pull Request / Additional comments
Thread descriptions show up as names in both the Visual Studio debugger, WinDBG debugger, and Windows Performance Analyzer. This makes it faster and easier to identify threads of interest in our processes.
## Validation Steps Performed
* [x] Checked threads were named in OpenConsole.exe running in classic conhost window mode under VS debug
* [x] Checked threads were named in OpenConsole.exe running in conpty mode under VS debug
* [x] Checked threads were named in WindowsTerminal.exe (for a few of the threads around connections)
* [x] Checked that we could also see it in WinDBG
This PR removes the `GetConsoleCursorInfo` and `SetConsoleCursorInfo`
methods from the `ConGetSet` interface, and the `GetScrollingRegion`
method from the `SCREEN_INFORMATION` class. None of these methods are
used anymore.
PR #2764 removed the last usage of `GetScrollingRegion`.
The `Get/SetConsoleCursorInfo` methods don't seem to have ever been used
in the time that the terminal code has been open source, so whatever
purpose they originally served must have been replaced a long time ago.
The `GetScrollingRegion` method was originally called from the
`ScrollRegion` function, but that usage was removed in PR #2764 when the
margin handling was moved to a higher level.
I've checked that the code still compiles.
Closes#9771
Change TAEF nuget package to use new Microsoft.Taef name; Update to 10.58 release build version.
## PR Checklist
* [x] Closes email from Phil letting us know TAEF has a new release build and a rename.
* [x] Closes annoying duplicate TAEF import warning in `Parser.UnitTests.vcxproj`
* [x] I work here.
* [ ] Need to see the tests run off CI to confirm this is fine for those environments and Helix
## Validation Steps Performed
* [x] Build/run tests locally
* [ ] Build/run unit and feature tests in CI
* [ ] Build/run Helix-lab tests in CI
By default from ARM64 architecture projects, `WIN32` is not defined. It
is supposed to be for this proxy stub to work. So I've set it with the
preprocessor for this project.
## PR Checklist
* [x] Closes release build failure after #7489
* [x] I work here.
* [x] Built on my machine.
## Detailed Description of the Pull Request / Additional comments
`WIN32` appears to convey two meanings depending on who you are:
- To most of Windows, `WIN32` appears to mean the Win32 API surface and
sometimes the major OS version that goes with it. (Specifically in
contrast to 16-bit Windows.)
- To others, `WIN32` appears to mean a 32-bit processor or a synonym of
`x86`.
This is generally not a problem for a few reasons:
- VS defines `WIN32` in the default targets/props only for the `x86`
processor type. **BUT**
- Windows defines `WIN32` if it's not already defined in both
`minwinbase.h` and `ole2.h` which generally speaking manage to get
compiled into practically everything especially since `minwinbase.h`
tends to sneak itself in somehow through `windows.h` and that's
**THE** include to use the Windows API surface.
- Windows also defines `WIN32` for itself unconditionally and relatively
globally when building itself.
However, it's a problem here because:
- `rpcproxy.h` is the only header included in `dlldata.c`, a file
generated automatically by `midl.exe` in the SDK when making a proxy
stub.
- `rpcproxy.h` only defines its contents for a proxy when `WIN32` or
`_M_AMD64` are found.
- Therefore, it's defined pretty naturally for x86 and AMD64 targets
from VS, but not for ARM64.
- ARM64 support is pretty new and those who are attempting to build for
ARM64 and against the public SDK with Visual Studio for a classic COM
proxy... seems like a relatively unlikely combination.
I will follow up with the Visual Studio, Windows SDK, and MIDL/COM teams
to try to remove this pitfall from the public tooling. But for now, this
is the fix.
This commit introduces a new build configuration, "Fuzzing", which
enables the new address sanitizer (shipped in VS 16.9) and code
coverage over the entire solution. Only a small subset of projects
(those comprising original conhost, right now) are selected to build in
this configuration, and even then only in Fuzzing|x64.
It also adds a fuzzing-adapted build of conhost, which makes no server
connections and handles no client applications. To do this, I've
replicated a bit of the console startup routine into fuzzmain.cpp and
made up some fake data. This is the bare minimum required to boot up
Win32 interactivity (or VT interactivity!) and pretend that a process
has connected.
If we don't pretend that a process has connected, "conhost" will exit
immediately. If we don't forge the process list, conhost will exit. If
we can't provide a server handle, we can't provide a "device comm".
Minor changes were necessary to server/host such that they would accept
a preexisting "device comm". We use this new behavior to provide a
"null" one that only hangs up threads and otherwise responds to requests
successfully.
This fuzzing-adapted build links LLVM's libFuzzer, which is an excellent
coverage-based fuzzer that will produce a corpus of inputs that exercise
unique codepaths. Eventually, we can use this to generate known-"good"
inputs for anything.
I've gone ahead and added a fuzz function that yeets bytes directly into
WriteCharsLegacy, which was the original reason I went down this path.
The implementation of LLVMFuzzerTestOneInput should be replaced with
whatever you want to fuzz.
- Implements the default application behavior and handoff mechanisms
between console and terminal. The inbox portion is done already. This
adds the ability for our OpenConsole.exe to accept the incoming server
connection from the Windows OS, stand up a PTY session, start the
Windows Terminal as a listener for an incoming connection, and then
send it the incoming PTY connection for it to launch a tab.
- The tab is launched with default settings at the moment.
- You must configure the default application using the `conhost.exe`
propsheet or with the registry keys. Finishing the setting inside
Windows Terminal will be a todo after this is complete. The OS
Settings panel work to surface this setting is a dependency delivered
by another team and you will not see it here.
## Validation Steps Performed
- [x] Manual adjust of registry keys to the delegation conhost/terminal
behavior
- [x] Adjustment of the delegation options with the propsheet
- [x] Launching things from the run box manually and watching them show
in Terminal
- [x] Launching things from shortcuts and watching them show in the
Terminal
Documentation on how it works will be a TODO post completion in #9462
References #7414 - Default Terminal spec
Closes#492
The PlaySound functions were removed from OneCoreUAP_apiset.Lib in Windows 10 SDK 19041 because they did not actually belong there. Link to WinMM.Lib for PlaySoundW.
### Validation Steps Performed
* Built for x64 from repository root with: `MSBuild.exe -property:TargetPlatformVersion=10.0.19041.0`
* Installed CascadiaPackage_0.0.1.0_x64_Debug.msix and launched on 19042.867
This commit clarifies some things inside WriteCharsLegacy by adding
comments and renaming parameter and enum names. It does not change any
logic.
`WC_ECHO` was used extensively to mean only one thing: whether to print
a control character like `\x18` (Ctrl+X>) as `^X`. It's been renamed
to make that abundantly clear.
Parenthesis in incorrect spot for default app policy check.
Related work items: MSFT-32071839
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 537ab5dfd9cc6af60b570697e77bba4b45822e05
[Git2Git] Merged PR 5760120: Add propsheet chooser to Windows
Now the inbox console propsheet can choose which terminal is default
Related work items: MSFT-32007202 #492
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev d80f506858bd990c267de6cefae7ff55707b3a57
This PR adds support for the VT line rendition attributes, which allow
for double-width and double-height line renditions. These renditions are
enabled with the `DECDWL` (double-width line) and `DECDHL`
(double-height line) escape sequences. Both reset to the default
rendition with the `DECSWL` (single-width line) escape sequence. For now
this functionality is only supported by the GDI renderer in conhost.
There are a lot of changes, so this is just a general overview of the
main areas affected.
Previously it was safe to assume that the screen had a fixed width, at
least for a given point in time. But now we need to deal with the
possibility of different lines have different widths, so all the
functions that are constrained by the right border (text wrapping,
cursor movement operations, and sequences like `EL` and `ICH`) now need
to lookup the width of the active line in order to behave correctly.
Similarly it used to be safe to assume that buffer and screen
coordinates were the same thing, but that is no longer true. Lots of
places now need to translate back and forth between coordinate systems
dependent on the line rendition. This includes clipboard handling, the
conhost color selection and search, accessibility location tracking and
screen reading, IME editor positioning, "snapping" the viewport, and of
course all the rendering calculations.
For the rendering itself, I've had to introduce a new
`PrepareLineTransform` method that the render engines can use to setup
the necessary transform matrix for a given line rendition. This is also
now used to handle the horizontal viewport offset, since that could no
longer be achieved just by changing the target coordinates (on a double
width line, the viewport offset may be halfway through a character).
I've also had to change the renderer's existing `InvalidateCursor`
method to take a `SMALL_RECT` rather than a `COORD`, to allow for the
cursor being a variable width. Technically this was already a problem,
because the cursor could occupy two screen cells when over a
double-width character, but now it can be anything between one and four
screen cells (e.g. a double-width character on the double-width line).
In terms of architectural changes, there is now a new `lineRendition`
field in the `ROW` class that keeps track of the line rendition for each
row, and several new methods in the `ROW` and `TextBuffer` classes for
manipulating that state. This includes a few helper methods for handling
the various issues discussed above, e.g. position clamping and
translating between coordinate systems.
## Validation Steps Performed
I've manually confirmed all the double-width and double-height tests in
_Vttest_ are now working as expected, and the _VT100 Torture Test_ now
renders correctly (at least the line rendition aspects). I've also got
my own test scripts that check many of the line rendition boundary cases
and have confirmed that those are now passing.
I've manually tested as many areas of the conhost UI that I could think
of, that might be affected by line rendition, including things like
searching, selection, copying, and color highlighting. For
accessibility, I've confirmed that the _Magnifier_ and _Narrator_
correctly handle double-width lines. And I've also tested the Japanese
IME, which while not perfect, is at least useable.
Closes#7865
## References
* See also #8617
## PR Checklist
* [x] Supports #3075
* [x] I work here.
* [x] Manual test.
## Detailed Description of the Pull Request / Additional comments
### Window Title Generation
Every time the renderer checks the title, it's doing two bad things that
I've fixed:
1. It's assembling the prefix to the full title doing a concatenation.
No one ever gets just the prefix ever after it is set besides the
concat. So instead of storing prefix and the title, I store the
assembled prefix + title and the bare title.
2. A copy must be made because it was returning `std::wstring` instead
of `std::wstring&`. Now it returns the ref.
### Dirty Area Return
Every time the renderer checks the dirty area, which is sometimes
multiple times per pass (regular text printing, again for selection,
etc.), a vector is created off the heap to return the rectangles. The
consumers only ever iterate this data. Now we return a span over a
rectangle or rectangles that the engine must store itself.
1. For some renderers, it's always a constant 1 element. They update
that 1 element when dirty is queried and return it in the span with a
span size of 1.
2. For other renderers with more complex behavior, they're already
holding a cached vector of rectangles. Now it's effectively giving
out the ref to those in the span for iteration.
### Bitmap Runs
The `til::bitmap` used a `std::optional<std::vector<til::rectangle>>`
inside itself to cache its runs and would clear the optional when the
runs became invalidated. Unfortunately doing `.reset()` to clear the
optional will destroy the underlying vector and have it release its
memory. We know it's about to get reallocated again, so we're just going
to make it a `std::pmr::vector` and give it a memory pool.
The alternative solution here was to use a `bool` and
`std::vector<til::rectangle>` and just flag when the vector was invalid,
but that was honestly more code changes and I love excuses to try out
PMR now.
Also, instead of returning the ref to the vector... I'm just returning a
span now. Everyone just iterates it anyway, may as well not share the
implementation detail.
### UTF-8 conversions
When testing with Terminal and looking at the `conhost.exe`'s PTY
renderer, it spends a TON of allocation time on converting all the
UTF-16 stuff inside to UTF-8 before it sends it out the PTY. This was
because `ConvertToA` was allocating a string inside itself and returning
it just to have it freed after printing and looping back around again...
as a PTY does.
The change here is to use `til::u16u8` that accepts a buffer out
parameter so the caller can just hold onto it.
## Validation Steps Performed
- [x] `big.txt` in conhost.exe (GDI renderer)
- [x] `big.txt` in Terminal (DX, PTY renderer)
- [x] Ensure WDDM and BGFX build under Razzle with this change.
Contains:
- Delegation Configurator that can lookup/edit/save configuration information to registry
- Conhost can lookup the CLSID of a registered default
- Conhost has the ability to handoff a starting visible-window interactive session to the registered default
- Velocity key since this is a big deal and we want to be careful
- IDL for the interface
Related work items: MSFT-16458099
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 0ca55027d8180fbbaa145f2fe7a15005856c0f7c
Moving things out of CharRow into ROW helps us hide it as an implementation detail.
This is part one of many.
### CharRow: Hide ClearCell, use ROW::ClearColumn
### CharRow: Hide GetText, use ROW::GetText
### CharRowBaseTests: remove dead file (never used!)
### CharRow: Move DoubleBytePadded into ROW
### CharRow: Move WrapForced into ROW
### Char/AttrRow: Hide Reset, use ROW::Reset
### Remove RowCellIterator (dead code)
RCI was unused; it was replaced by TextBufferCellIterator shortly after its creation
### Move AttrRowTests to ut_textbuffer from ut_host
It had no reliance on the host.
Make a few changes to memory usage throughout the application to reduce transient allocations from the `big.txt` test from ~213,000 to ~53,000.
## PR Checklist
* [x] Supports #3075
* [x] I work here.
* [x] Tested manually and WPR'd. Test suite should still pass.
* [x] Am core contributor
## Detailed Description of the Pull Request / Additional comments
Transient allocations are those that are new'd, used, then delete'd. Going back and forth to the system allocator for things we're just going to throw away or use rapidly again is a performance detriment. Not only is it a bunch of time to go ask the system with a syscall, it also hits a whole bunch of locks on the allocators. This PR identifies a few places where we were accidentally allocating and didn't mean to or were allocating and freeing just to turn around and allocate again. I chose other strategies to avoid this.
## Validation Steps Performed
- Ran `big.txt` sample (~6MB file) before and after. Observed heap allocations with WPR.
I was looking at conhost/OpenConsole and noticed it was being pretty
inefficient with allocations due to some usages of std::deque and
std::vector that didn't need to be done quite that way.
So this uses std::vector for the TextBuffer's storage of ROW objects,
which allows one allocation to contiguously reserve space for all the
ROWs - on Desktop this is 9001 ROW objects which means it saves 9000
allocations that the std::deque would have done. Plus it has the
benefit of increasing locality of the ROW objects since deque is going
to chase pointers more often with its data structure.
Then, within each ROW there are CharRow and ATTR_ROW objects that use
std::vector today. This changes them to use Boost's small_vector, which
is a variation of vector that allows for the so-called "small string
optimization." Since we know the typical size of these vectors, we can
pre-reserve the right number of elements directly in the
CharRow/ATTR_ROW instances, avoiding any heap allocations at all for
constructing these objects.
There are a ton of variations on this "small_vector" concept out there
in the world - this one in Boost, LLVM has one called SmallVector,
Electronic Arts' STL has a small_vector, Facebook's folly library has
one...there are a silly number of these out there. But Boost seems like
it's by far the easiest to consume in terms of integration into this
repo, the CI/CD pipeline, licensing, and stuff like that, so I went with
the boost version.
In terms of numbers, I measured the startup path of OpenConsole.exe on
my dev box for Release x64 configuration. My box is an i7-6700k @ 4
Ghz, with 32 GB RAM, not that I think machine config matters much here:
| | Allocation count | Allocated bytes | CPU usage (ms) |
| ------ | ------------------- | ------------------ | -------------- |
| Before | 29,461 | 4,984,640 | 103 |
| After | 2,459 (-91%) | 4,853,931 (-2.6%) | 96 (-7%) |
Along the way, I also fixed a dynamic initializer I happened to spot in
the registry code, and updated some docs.
## Validation Steps Performed
- Ran "runut", "runft" and "runuia" locally and confirmed results are
the same as the main branch
- Profiled the before/after numbers in the Visual Studio profiler, for
the numbers shown in the table
Co-authored-by: Austin Lamb <austinl@microsoft.com>
N
* Change backslashes in include statements to forward slashes (CC-8205)
Dustin L. Howett
* Refactor DEC/ANSI modes to avoid duplication when we add SM/RM (GH-8469)
* Fix the xterm and SGR mouse encodings for CTRL, ALT, SHIFT (GH-8379)
James Holderness
* Fix rendering of DBCS characters when partially off screen (CC-8438)
* Correct horizontal coordinates in viewport overflow test (CC-8456)
* Correct paths in the the runut and runft test scripts (CC-8488)
* Retain horizontal viewport offset when moving to bottom (CC-8434)
* Fix out-of-bounds exceptions in Set...{Buffer,Screen}Size (CC-8309)
PankajBhojwani (1)
* Implement ConEmu's OSC 9;4 to set the taskbar progress indicator (CC-8055)
Chester Liu (1)
* Improve OSC 8 Hyperlink parsing logic (CC-7962)
Related work items: #31090459, #31090805, #31090808, #31090810
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_onecore_dep_uxp 91dff13f6953441afe9e28bfe3da9d5bff27bb05
Related: MSFT-26196390
Related work items: #26196390
This commit replaces DeviceComm with the interface IDeviceComm and the
concrete implementation type ConDrvDeviceComm. This work is done in
preparation for different device backends.
In addition to separating out ConDrv-specific behavior, I've introduced
a "handle exchange" interface.
HANDLE EXCHANGE
---------------
There are points where we give ConDrv opaque handle identifiers to our
input buffer, output buffer and process data. The exact content of the
opaque identifier is meaningless to ConDrv: the driver's only
interaction with these identifiers is to hold onto them and send back
whichever are pertinent for each API call.
Because of that, we used the raw register-width pointer value of the
input buffer, output buffer or process data _as_ the opaque handle
value.
This works very well for ConDrv <-> conhost using the ConDrvDeviceComm.
It works less well for something like the "logging" DeviceComm that will
log packets to a file: those packets *cannot* contain pointer values (!)
To address this, and to afford flexibility to DeviceComm implementers,
I've introduced a two-member complement of handle management functions:
* `ULONG_PTR PutHandle(void*)` registers an object with the DeviceComm
and returns an opaque identifier.
* `void* GetHandle(ULONG_PTR)` takes an opaque identifier and returns
the original object.
ConDrvDeviceComm implements PutHandle and GetHandle by casting the
object pointer to the "opaque handle value", which maintains wire format
compatibility[1] with revisions of conhost prior to this change.
Simpler DeviceComm implementations that require handle tracking but
cannot bear raw pointers can implement these functions by returning an
autoincrementing ID (or similar) and registering the raw object pointer
in a mapping.
I've audited all existing handle exchanges with the driver and updated
them to use Put/GetHandle.
(I intended to add DestroyHandle, but we are not equipped for handle
removal at the moment. ConsoleHandleData/ConsoleProcessHandle are
destroyed during wait routine completion, on client disconnect, etc.
This does mean that an id<->pointer mapping will grow without bound, but
at a cost of ~8 bytes per entry and a short-lived console session I am
not too concerned about the cost.)
[1] Wire format compatibility is not required, and later we may want to
switch ConDrvDeviceComm to `EncodePointer` and `DecodePointer` just to
insulate us against a spurious ConDrv packet allowing for an arbitrary
4/8-byte read and subsequent liftoff into space.
There are two issue with copy to clipboard when block is selected:
* We don't add new lines for lines that were wrapped
* We remove trailing whitespaces which is not intuitive in block selection.
Fixed the copy logic to always add newlines and not to remove
whitespaces when block is selected.
Even if shift is pressed!
## Detailed Description of the Pull Request / Additional comments
* Added optional parameter to `TextBuffer::GetText`
that allows to apply formatting (includeCRLF / trimming)
to lines that were wrapped
* Changed `Terminal::RetrieveSelectedTextFromBuffer`
to apply the following parameters when block is selected:
* includeCRLF = true
* trimTrailingWhitespaces = false
* apply the formatting above to all rows, including the ones
that were wrapped
## Validation Steps Performed
* Manual tests for both block and standard selection
* Copy with both right-click and command
* Added UT
Closes#6740
In conhost there is a keyboard shortcut that applies colors to the
selected range of text, and another shortcut that searches for the
selected text, and applies colors to any matching content. The former
operation doesn't work correctly when the selection is wrapped, and both
have problems when the selected text spans DBCS characters. This PR
attempts to fix those issues.
The problem with the color section was that it applied the color to a
simple rect spanning the start and end points of the selection. I've now
updated it to use the `Selection::GetSelectionRects` method, which
correctly handles a wrapped range of lines, and makes sure that double
width characters are fully covered.
The problem with the "find-and-color" operation was in the way it
obtained the search text from the selected screen cells. Since it
retrieved one cell at a time, and a DBCS character can span two cells,
that resulted in some characters being repeated in the search text. I've
now corrected that code to take the width of the characters into
account.
## Validation Steps Performed
I've manually verified that the test cases described in #8572 and #8574
are now working correctly.
Closes#8572Closes#8574
A bunch of our feature tests don't work reliably in CI. They rely on
creating a new `OpenConsole.exe` window, then running the test _in that
console_. As a part of that test setup, the test runner used to wait a
second to attach to the newly created console. Then the test goes on
it's merry way, assuming the console is ready to go. However, in CI,
that might take more than a second. If it does, then the test would fail
pretty immediately, as soon as it tries to get at the buffer of the new
console.
This PR introduces a little retry loop to the test init. After attaching
to the new console, we'll try and get at the screen buffer. If that
fails, we'll wait a second and try again. We'll try 5 times total,
before bailing entirely. Hopefully, this should mitigate most of the
random CI failures we get in the feature tests.
Closes#8495
When resizing the buffer in the `SetConsoleScreenBufferSize` and
`SetConsoleScreenBufferInfoEx` APIs, we have tests in place to make sure
that the resize doesn't result in the viewport extending past the bottom
or right of the buffer (since that can eventually result in exceptions
being thrown). Unfortunately these tests were using the wrong X
coordinate, so they failed to detect an overflow along the horizontal
axis. This PR corrects that mistake.
PR #8309 was where the overflow detection was initially added.
The original code was using the `Viewport::EndExclusive` method to
determine the extent of the viewport, mistakenly thinking that it
returned the bottom right coordinates (it actually returns the left
coordinate). So I've now added a `BottomRightExclusive` method to the
`Viewport` class, that actually does return the coordinates we need, and
have updated the overflow tests to use that method instead.
## Validation Steps Performed
I've manually confirmed that the test case is issue #8453 is no longer
throwing an exception.
Closes#8453
When the viewport is moved to the "virtual bottom" of the buffer (via
the `MoveToBottom` method), it is important that the horizontal viewport
offset be left as it is, otherwise that can result in some undesirable
side effects.
Since the VT coordinate system is relative to the top of the viewport,
many VT operations call the `MoveToBottom` method to make sure the
viewport is correctly positioned before proceeding. There is no need for
the horizontal position to be adjusted, though, since the X coordinates
are not constrained by the viewport, but are instead relative to the
underlying buffer.
Setting the viewport X coordinate to 0 in `MoveToBottom` (as we were
previously doing) could result in the cursor being pushed off screen.
And if the operation itself was moving the cursor, that would then
trigger another viewport move to bring the cursor back into view. These
conflicting movements meant the viewport was always forced as far left
as possible, and could also result in cursor "droppings" as the cursor
lost track of where it had been.
I've now fixed this by updating the `GetVirtualViewport` method to match
the horizontal offset of the active viewport, instead of having the X
coordinate hardcoded to 0.
## Validation Steps Performed
I've manually confirmed that this fixes the cursor "droppings" test case
reported in issue #8213.
I've also added a screen buffer test that makes sure the `MoveToBottom`
method is working as expected, and not changing the horizontal viewport
offset when it moves down.
Closes#8213
Many include statements use forward slashes, while others use backwards
slashes. This is inconsistent formatting. For this reason, I changed the
backward slashes to forward slashes since that is the standard.
[Git2Git] Git Train: Merge of building/rs_onecore_dep_uxp/201117-2002 into official/rs_onecore_dep_uxp Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_onecore_dep_uxp 526043ab6050397506b3cdb77e6a43b5ca9a2e9a
Related work items: MSFT-29990377
This fixes a number of exceptions that can cause conhost to crash when
the buffer is resized in such a way that the viewport or cursor position
end up out of bounds.
Technically this is a fix for issue #256, although that has been closed
as "needs-repro".
The main fix was to add checks in the `SetConsoleScreenBufferSizeImpl`
and `SetConsoleScreenBufferInfoExImpl` methods, to make sure the
viewport doesn't extend past the bottom or right of the buffer after a
resize. If it has overflowed, we move the viewport back up or left until
it's back within the buffer boundaries. We also check if the cursor
position has ended up out of bounds, and if so, clamp it back inside the
buffer.
The `SCREEN_INFORMATION::SetViewport` was also a source of viewport
overflow problems, because it was mistakenly using inclusive coordinates
in its range checks, which resulted in them being off by one. That has
now been corrected to use exclusive coordinates.
Finally, the `IsCursorDoubleWidth` method was incorrectly marked as
`noexcept`, which was preventing its exceptions from being caught.
Ideally it shouldn't be throwing exceptions at all any more, but I've
removed the `noexcept` specifier, so if it does throw an exception,
it'll at least have more chance of recovering without a crash.
## Validation Steps Performed
I put together a few test cases (based on the reports in issues #276 and
#1976) which consistently caused conhost to crash, or to generate an
exception visible in the debug output. With this PR applied, those test
cases are no longer crashing or triggering exceptions.
Closes#1976
Fixed potential errors caused by overflow or underfow in
SectionInput.cpp
## PR Checklist
* [x] CLA signed
* [x] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
In selectionInput.cpp, there is both a potential overflow and potential
underflow. To address this issue, I casted the calculation up to int,
which is then checked because of integer promotion. Underflow and
underflow is therefore impossible because now if the calculation exceeds
SHORT_MAX, it will have exceeded bufferSize.BottomInclusive() or
bufferSize.Top() anyway, and be set to them.
## Validation Steps Performed
Passed Unit Testing
Manual Validation
This pull request is the initial implementation of hyperlink auto
detection
Overall design:
- Upon startup, TerminalCore gives the TextBuffer some patterns it
should know about
- Whenever something in the viewport changes (i.e. text
output/scrolling), TerminalControl tells TerminalCore (through a
throttled function for performance) to retrieve the visible pattern
locations from the TextBuffer
- When the renderer encounters a region that is associated with a
pattern, it paints that region differently
References #5001Closes#574
It turns out that we missed part of the OSC 8 spec which indicated that
_hyperlinks with the same ID but different URIs are logically distinct._
> Character cells that have the same target URI and the same nonempty id
> are always underlined together on mouseover.
> The same id is only used for connecting character cells whose URIs is
> also the same. Character cells pointing to different URIs should never
> be underlined together when hovering over.
This pull request fixes that oversight by appending the (hashed) URI to
the generated ID.
When Terminal receives one of these links over ConPTY, it will hash the
URL a second time and therefore append a second hashed ID. This is taken
as an acceptable cost.
Fixes#7698
This PR introduces a pair of classes for managing VT parameters that
automatically handle range checking and default fallback values, so the
individual operations don't have to do that validation themselves. In
addition to simplifying the code, this fixes a few cases where we were
mishandling missing or extraneous parameters, and adds support for
parameter sequences on commands that couldn't previously handle them.
This PR also sets a limit on the number of parameters allowed, to help
thwart DoS memory consumption attacks.
## References
* The new parameter class also introduces the concept of an
omitted/default parameter which is not necessarily zero, which is a
prerequisite for addressing issue #4417.
## Detailed Description of the Pull Request / Additional comments
There are two new classes provide by this PR: a `VTParameter` class,
similar in function to a `std::optional<size_t>`, which holds an
individual parameter (which may be an omitted/default value); and a
`VTParameters` class, similar in function to `gsl:span<VTParameter>`,
which holds a sequence of those parameters.
Where `VTParameter` differs from `std::optional` is with the inclusion
of two cast operators. There is a `size_t` cast that interprets omitted
and zero values as 1 (the expected behaviour for most numeric
parameters). And there is a generic cast, for use with the enum
parameter types, which interprets omitted values as 0 (the expected
behaviour for most selective parameters).
The advantage of `VTParameters` class is that it has an `at` method that
can never fail - out of range values simply return the a default
`VTParameter` instance (this is standard behaviour in VT terminals). It
also has a `size` method that will always return a minimum count of 1,
since an empty parameter list is typically the equivalent of a single
"default" parameter, so this guarantees you'll get at least one value
when iterating over the list with `size()`.
For cases where we just need to call the same dispatch method for every
parameter, there is a helper `for_each` method, which repeatedly calls a
given predicate function with each value in the sequence. It also
collates the returned success values to determine the overall result of
the sequence. As with the `size` method, this will always make at least
one call, so it correctly handles empty sequences.
With those two classes in place, we could get rid of all the parameter
validation and default handling code in the `OutputStateMachineEngine`.
We now just use the `VTParameters::at` method to grab a parameter and
typically pass it straight to the appropriate dispatch method, letting
the cast operators automatically handle the assignment of default
values. Occasionally we might need a `value_or` call to specify a
non-standard default value, but those cases are fairly rare.
In some case the `OutputStateMachineEngine` was also checking whether
parameters values were in range, but for the most part this shouldn't
have been necessary, since that is something the dispatch classes would
already have been doing themselves (in the few cases that they weren't,
I've now updated them to do so).
I've also updated the `InputStateMachineEngine` in a similar way to the
`OutputStateMachineEngine`, getting rid of a few of the parameter
extraction methods, and simplifying other parts of the implementation.
It's not as clean a replacement as the output engine, but there are
still benefits in using the new classes.
## Validation Steps Performed
For the most part I haven't had to alter existing tests other than
accounting for changes to the API. There were a couple of tests I needed
to drop because they were checking for failure cases which shouldn't
have been failing (unexpected parameters should never be an error), or
testing output engine validation that is no longer handled at that
level.
I've added a few new tests to cover operations that take sequences of
selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've
extended the cursor movement tests to make sure those operations can
handle extraneous parameters that weren't expected. I've also added a
test to verify that the state machine will correctly ignore parameters
beyond the maximum 32 parameter count limit.
I've also manual confirmed that the various test cases given in issues
#2101 are now working as expected.
Closes#2101
For whatever reason, the super linter seems to think that any file it doesn't recognize is an EDITORCONFIG file. That means all our `cpp`, `hpp`, `h`, `resw`, `xaml`, etc files are going to get linted with different rules than the clang-format ones we already use.
This PR disables the EDITORCONFIG linter, and has a minimal change to a cpp file to ensure that it's no longer linted by the action.
See also:
* #7637 added this
* #7799 is blocked by this
* #7924 is blocked by this
* Correct the behaviour of parsing `rgb:R/G/B`. It should be interpreted
as `RR/GG/BB` instead of `0R/0G/0B`
* Add support for `rgb:RRR/GGG/BBB` and `rgb:RRRR/GGGG/BBBB`. The
behaviour of 12 bit variants is to repeat the first digit at the end,
e.g. `rgb:123/456/789` becomes `rgb:1231/4564/7897`.
* Add support for `#` formats. We are following the rules of
[XParseColor] by interpreting `#RGB` as `R000G000B000`.
* Add support for XOrg app color names, which are supported by xterm, VTE
and many other terminal emulators.
* Multi-parameter OSC 4 is now supported.
* The chaining of OSC 10-12 is not yet supported. But the parameter
validation is relaxed by parsing the parameters as multi-params but
only use the first one, which means `\e]10;rgb:R/G/B;` and
`\e]10:rgb:R/G/B;invalid` will execute `OSC 10` with the first color
correctly. This fixes some of the issues mentioned in #942 but not
all of them.
[XParseColor]: https://linux.die.net/man/3/xparsecolorCloses#3715
This fixes a bug when moving backwards by word that resulted in #7742.
This also includes...
- a minor refactor that leverages `GetWordStart` in `MoveToPreviousWord`
- additional unit tests for movement by word
- a feature test comprised of the referenced bug report
`MoveToPreviousWord()` would...
- move backwards for each whitespace character
- then, move backwards for each regular character
This would actually result in moving to the beginning of the current "word" (as defined by a11y).
We actually need to do this process twice:
- the first time gets you to the beginning of the current word
- attempt to move back by one character
- the second time gets you to the beginning of the previous word
Rather than implementing 4 while loops, we leverage `GetWordStart()` to
attempt to move to the beginning of the previous word. We call it twice
(as described above). The logic is unchanged, but we instead reuse a
function that has already undergone more testing.
To make sure this works as expected, additional unit tests were
introduced covering "MoveByWord" in the TextBuffer.
## Validation Steps Performed
Added test for repro steps.
Added unit tests for movement by word.
Closes#7742
Sometimes when we were sliding the viewport to fit inside the buffer, we
would end up with left > right.
That would cause us to crash down the line when rendering.
Fixes MSFT:28387423
Fixes#7744
`EndExclusive` represents the end of the buffer. This is designed to not
point to any data on the buffer. UiaTextRange would point to this
`EndExclusive` and then attempt to move based on it. However, since it
does not point to any data, it could experience undefined behavior or
(inevitably) crash from running out of bounds.
This PR specifically checks for expansion and movement at that point,
and prevents us from moving beyond it. There are plans in the future to
define the "end" as the last character in the buffer. Until then, this
solution will suffice and provide correct behavior that doesn't crash.
## Validation Steps Performed
Performed the referenced bugs' repro steps and added test coverage.
Closes MSFT-20458595
Closes#7663Closes#7664
This PR adds support for the _blink_ graphic rendition attribute. When a
character is output with this attribute set, it "blinks" at a regular
interval, by cycling its color between the normal rendition and a dimmer
shade of that color.
The majority of the blinking mechanism is encapsulated in a new
`BlinkingState` class, which is shared between the Terminal and Conhost
implementations. This class keeps track of the position in the blinking
cycle, which determines whether characters are rendered as normal or
faint.
In Windows Terminal, the state is stored in the `Terminal` class, and in
Conhost it's stored in the `CONSOLE_INFORMATION` class. In both cases,
the `IsBlinkingFaint` method is used to determine the current blinking
rendition, and that is passed on as a parameter to the
`TextAttribute::CalculateRgbColors` method when these classes are
looking up attribute colors.
Prior to calculating the colors, the current attribute is also passed to
the `RecordBlinkingUsage` method, which keeps track of whether there are
actually any blink attributes in use. This is used to determine whether
the screen needs to be refreshed when the blinking cycle toggles between
the normal and faint renditions.
The refresh itself is handled by the `ToggleBlinkingRendition` method,
which is triggered by a timer. In Conhost this is just piggybacking on
the existing cursor blink timer, but in Windows Terminal it needs to
have its own separate timer, since the cursor timer is reset whenever a
key is pressed, which is not something we want for attribute blinking.
Although the `ToggleBlinkingRendition` is called at the same rate as the
cursor blinking, we actually only want the cells to blink at half that
frequency. We thus have a counter that cycles through four phases, and
blinking is rendered as faint for two of those four. Then every two
cycles - when the state changes - a redraw is triggered, but only if
there are actually blinking attributes in use (as previously recorded).
As mentioned earlier, the blinking frequency is based on the cursor
blink rate, so that means it'll automatically be disabled if a user has
set their cursor blink rate to none. It can also be disabled by turning
off the _Show animations in Windows_ option. In Conhost these settings
take effect immediately, but in Windows Terminal they only apply when a
new tab is opened.
This PR also adds partial support for the `SGR 6` _rapid blink_
attribute. This is not used by DEC terminals, but was defined in the
ECMA/ANSI standards. It's not widely supported, but many terminals just
it implement it as an alias for the regular `SGR 5` blink attribute, so
that's what I've done here too.
## Validation Steps Performed
I've checked the _Graphic rendition test pattern_ in Vttest, and
compared our representation of the blink attribute to that of an actual
DEC VT220 terminal as seen on [YouTube]. With the right color scheme
it's a reasonably close match.
[YouTube]: https://www.youtube.com/watch?v=03Pz5AmxbE4&t=1m55sCloses#7388
Some IME implementations do not produce composition strings, and their
users have come to rely on the cursor that conhost traditionally left on
until a composition string showed up. We shouldn't hide the cursor until
we get a string (as opposed to hiding it when composition begins) so as
to not break those IMEs.
Related to #6207.
Fixes MSFT:29219348
