## Summary of the Pull Request
Upgrade the Windows SDK to 19041 by setting `WindowsTargetPlatformMinVersion` to 17763 and `WindowsTargetPlatformVersion` to 19041.
## PR Checklist
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
General usage of the Windows Terminal application appears fine.
Set keyword flags on all events so those sharing a provider with
telemetry do not fire unless tracing is enabled
## PR Checklist
* [x] Closes#10093
* [x] I work here
* [x] Tests passed
* [x] Documentation added in `til.h` about how keywords work and at the
only other site of keywords we define in the Host project tracing
files.
## Detailed Description of the Pull Request / Additional comments
I initially thought that we would need to split providers here to
accomplish this... but @DHowett helped me realize that might be a lot of
additional metadata and bloat binary size. So with help from a friend
from fundamentals, I realized that we could use Keywords to
differentiate here. We can no longer define 0 keywords as that
represents an any/all scenario. Every `TraceLoggingWrite` event now
needs a keyword. When our events have a keyword, they're not included in
any trace. Additionally, when we have an explicit keyword to check that
is different from the ones used for the telemetry pipeline, we can
ensure that we only do "hard work" to generate debug trace data when an
"ALL" type listener like TraceView or Windows Performance Recorder with
our profiles is listening to these providers for ALL keyworded events.
## Validation Steps Performed
- [x] - Built with full release build config to confirm performance is
worse than dev builds BECAUSE of the telemetry event collector camping
our provider and triggering full trace event generation on shared
providers.
- [x] - Built with full release build config to enable statistics
collection and validated trace event collection is excluded and trace
event short-circuits work with this change.
- [x] - Checked that TraceView still sees both telemetry and tracing
events
- [x] - Checked that WPR with our .wprp profile sees both telemetry and
tracing events
## Summary of the Pull Request
When we parse OSC 9;4, allow a trailing semicolon (i.e. allow `9;4;` or something like `9;4;3;`).
## PR Checklist
* [x] Closes#9960
* [X] Tests added/passed
## Validation Steps Performed
OSC 9;4 sequences with or without trailing semicolons work
This PR introduces a mechanism via which DCS data strings can be passed
through directly to the dispatch method that will be handling them, so
the data can be processed as it is received, rather than being buffered
in the state machine. This also simplifies the way string termination is
handled, so it now more closely matches the behaviour of the original
DEC terminals.
* Initial support for DCS sequences was introduced in PR #6328.
* Handling of DCS (and other) C1 controls was added in PR #7340.
* This is a prerequisite for Sixel (#448) and Soft Font (#9164) support.
The way this now works, a `DCS` sequence is dispatched as soon as the
final character of the `VTID` is received. Based on that ID, the
`OutputStateMachineEngine` should forward the call to the corresponding
dispatch method, and its the responsibility of that method to return an
appropriate handler function for the sequence.
From then on, the `StateMachine` will pass on all of the remaining bytes
in the data string to the handler function. When a data string is
terminated (with `CAN`, `SUB`, or `ESC`), the `StateMachine` will pass
on one final `ESC` character to let the handler know that the sequence
is finished. The handler can also end a sequence prematurely by
returning false, and then all remaining data bytes will be ignored.
Note that once a `DCS` sequence has been dispatched, it's not possible
to abort the data string. Both `CAN` and `SUB` are considered valid
forms of termination, and an `ESC` doesn't necessarily have to be
followed by a `\` for the string terminator. This is because the data
string is typically processed as it's received. For example, when
outputting a Sixel image, you wouldn't erase the parts that had already
been displayed if the data string is terminated early.
With this new way of handling the string termination, I was also able to
simplify some of the `StateMachine` processing, and get rid of a few
states that are no longer necessary. These changes don't apply to the
`OSC` sequences, though, since we're more likely to want to match the
XTerm behavior for those cases (which requires a valid `ST` control for
the sequence to be accepted).
## Validation Steps Performed
For the unit tests, I've had to make a few changes to some of the
`OutputEngineTests` to account for the updated `StateMachine`
processing. I've also added a new `StateMachineTest` to confirm that the
data strings are correctly passed through to the string handler under
all forms of termination.
To test whether the framework is actually usable, I've been working on
DRCS Soft Font support branched off of this PR, and haven't encountered
any problems. To test the throughput speed, I also hacked together a
basic Sixel parser, and that seemed to perform reasonably well.
Closes#7316
## Summary of the Pull Request
The "virtual bottom" marks the last line of the mutable viewport area, which is the part of the buffer that VT sequences can write to. This region should typically only move downwards, as new lines are added to the buffer, but there were a number of cases where it was incorrectly being moved up. This PR attempts to fix that.
## PR Checklist
* [x] Closes#9754
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #9754
## Detailed Description of the Pull Request / Additional comments
When a call is made to `UpdateBottom`, we now clamp the value so it's at least as low as the current viewport bottom (i.e. if the viewport has moved down, we want the virtual bottom to move down too), but no lower than the bottom of the buffer (we don't want it to be out of range).
There is one special case where we do actually want the virtual bottom to move up - when the scrollback has been cleared with an `ED3` escape sequence. So in that case we needed a new `ConGetSet` API (`ResetBottom`) to reset the virtual bottom to the top of the buffer (essentially one less than the viewport height, since the virtual bottom points to the last line of the viewport).
## Validation Steps Performed
I had to reset the virtual bottom manually in some parts of the `ScreenBufferTests`, since some of the tests were relying on the virtual bottom being automatically reset when the viewport was reset, which is no longer the case.
I've also added a new test to verify that the virtual bottom doesn't move upwards if an update is triggered while the visible viewport is scrolled up. This essentially reproduces the test case from issue #9754, which I've also manually confirmed is fixed.
This PR removes the `GetConsoleCursorInfo` and `SetConsoleCursorInfo`
methods from the `ConGetSet` interface, and the `GetScrollingRegion`
method from the `SCREEN_INFORMATION` class. None of these methods are
used anymore.
PR #2764 removed the last usage of `GetScrollingRegion`.
The `Get/SetConsoleCursorInfo` methods don't seem to have ever been used
in the time that the terminal code has been open source, so whatever
purpose they originally served must have been replaced a long time ago.
The `GetScrollingRegion` method was originally called from the
`ScrollRegion` function, but that usage was removed in PR #2764 when the
margin handling was moved to a higher level.
I've checked that the code still compiles.
Closes#9771
Change TAEF nuget package to use new Microsoft.Taef name; Update to 10.58 release build version.
## PR Checklist
* [x] Closes email from Phil letting us know TAEF has a new release build and a rename.
* [x] Closes annoying duplicate TAEF import warning in `Parser.UnitTests.vcxproj`
* [x] I work here.
* [ ] Need to see the tests run off CI to confirm this is fine for those environments and Helix
## Validation Steps Performed
* [x] Build/run tests locally
* [ ] Build/run unit and feature tests in CI
* [ ] Build/run Helix-lab tests in CI
This reduces by 10% the binary size of OpenConsole x64 Release.
Note | OpenConsole.exe
------ | ---------------------------
Before | 1156096
After | 1037312
Delta | -118784
%Delta | -10.27%
This PR adds support for the VT line rendition attributes, which allow
for double-width and double-height line renditions. These renditions are
enabled with the `DECDWL` (double-width line) and `DECDHL`
(double-height line) escape sequences. Both reset to the default
rendition with the `DECSWL` (single-width line) escape sequence. For now
this functionality is only supported by the GDI renderer in conhost.
There are a lot of changes, so this is just a general overview of the
main areas affected.
Previously it was safe to assume that the screen had a fixed width, at
least for a given point in time. But now we need to deal with the
possibility of different lines have different widths, so all the
functions that are constrained by the right border (text wrapping,
cursor movement operations, and sequences like `EL` and `ICH`) now need
to lookup the width of the active line in order to behave correctly.
Similarly it used to be safe to assume that buffer and screen
coordinates were the same thing, but that is no longer true. Lots of
places now need to translate back and forth between coordinate systems
dependent on the line rendition. This includes clipboard handling, the
conhost color selection and search, accessibility location tracking and
screen reading, IME editor positioning, "snapping" the viewport, and of
course all the rendering calculations.
For the rendering itself, I've had to introduce a new
`PrepareLineTransform` method that the render engines can use to setup
the necessary transform matrix for a given line rendition. This is also
now used to handle the horizontal viewport offset, since that could no
longer be achieved just by changing the target coordinates (on a double
width line, the viewport offset may be halfway through a character).
I've also had to change the renderer's existing `InvalidateCursor`
method to take a `SMALL_RECT` rather than a `COORD`, to allow for the
cursor being a variable width. Technically this was already a problem,
because the cursor could occupy two screen cells when over a
double-width character, but now it can be anything between one and four
screen cells (e.g. a double-width character on the double-width line).
In terms of architectural changes, there is now a new `lineRendition`
field in the `ROW` class that keeps track of the line rendition for each
row, and several new methods in the `ROW` and `TextBuffer` classes for
manipulating that state. This includes a few helper methods for handling
the various issues discussed above, e.g. position clamping and
translating between coordinate systems.
## Validation Steps Performed
I've manually confirmed all the double-width and double-height tests in
_Vttest_ are now working as expected, and the _VT100 Torture Test_ now
renders correctly (at least the line rendition aspects). I've also got
my own test scripts that check many of the line rendition boundary cases
and have confirmed that those are now passing.
I've manually tested as many areas of the conhost UI that I could think
of, that might be affected by line rendition, including things like
searching, selection, copying, and color highlighting. For
accessibility, I've confirmed that the _Magnifier_ and _Narrator_
correctly handle double-width lines. And I've also tested the Japanese
IME, which while not perfect, is at least useable.
Closes#7865
Implement the `XTPUSHSGR` and `XTPOPSGR` control sequences (see #1796).
This change adds a new pair of methods to `ITermDispatch`:
`PushGraphicsRendition` and `PopGraphicsRendition`, and then plumbs the
change through `AdaptDispatch`, `TerminalDispatch`, `ITerminalApi` and
`TerminalApi`.
The stack logic is encapsulated in the `SgrStack` class, to allow it to
be reused between the two APIs (`AdaptDispatch` and `TerminalDispatch`).
Like xterm, only ten levels of nesting are supported.
The stack is implemented as a "ring stack": if you push when the stack
is full, the bottom of the stack will be dropped to make room.
Partial pushes (see the description of `XTPUSHSGR` in Issue #1796) are
implemented per xterm spec.
## Validation Steps Performed
Tests added, plus manual verification of the feature.
Closes#1796
Dustin L. Howett (3)
* Move CharToKeyEvents (and friends) into InteractivityBase (GH-9106)
* Update Cascadia Code to 2102.03 (GH-9088)
* verison: bump to 1.7 on main
Josh Soref (1)
* ci: update to Spell check to 0.0.17a (CC-9014)
Leonard Hecker (3)
* Fixed GH-5205: Ctrl+Alt+2 doesn't send ^[^@ (CC-5272)
* Fix issues in tests.xml and OpenConsole.psm1 (CC-9011)
* Fix GH-8458: Handle all Ctrl-key combinations (CC-8870)
Mike Griese (1)
* Add support for running a commandline in another WT window (GH-8898)
Michael Niksa (1)
* Teach the renderer to keep thread alive if engine requests it (GH-9091)
Lachlan Picking (1)
* Fix shader time input (CC-8994)
PankajBhojwani (1)
* Separate runtime TerminalSettings from profile-TerminalSettings (CC-8602)
Chester Liu (2)
* Add support for paste filtering and bracketed paste mode (CC-9034)
* Add support for chaining OSC 10-12 (CC-8999)
Related work items: MSFT-31692939
These functions have a dependency on the "VT Redirected" versions of
VkKeyScanW, MapVirtualKeyW and GetKeyState. Those implementations depend
on the service locator and therefore the entire interactivity stack.
This meant that anybody depending on just Types had to pull in **the
entire host** worth of dependencies (!).
Since these functions are only used in places where we have or are
testing interactivity, it makes sense to consolidate them here.
(cherry picked from commit 8f73145d9d)
These functions have a dependency on the "VT Redirected" versions of
VkKeyScanW, MapVirtualKeyW and GetKeyState. Those implementations depend
on the service locator and therefore the entire interactivity stack.
This meant that anybody depending on just Types had to pull in **the
entire host** worth of dependencies (!).
Since these functions are only used in places where we have or are
testing interactivity, it makes sense to consolidate them here.
## Summary of the Pull Request
Fixes#5205, by replacing another use of `MapVirtualKeyW` with `ToUnicodeEx`.
The latter just seems to be much more consistent at translating key combinations in general.
In this particular case though it fixes the issue, because there's no differentiation in `MapVirtualKeyW` for whether it failed to return a character (`'\0'`) or succeeded in turning `^@` into `'\0'`.
`ToUnicodeEx` on the other hand returns the success state separately from the translated character.
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#5205
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #5205
## Detailed Description of the Pull Request / Additional comments
This PR changes the behavior of the `Ctrl+Alt+Key` handling slightly:
⚠️ `ToUnicodeEx` returns unshifted characters. ⚠️
For instance `Ctrl+Alt+a` is now turned into `^[^a`. Due to how ASCII works this is essentially the same though because `'A' & 0b11111` and `'a' & 0b11111` are the same.
## Validation Steps Performed
* Run `showkey -a`
* Ensured `Ctrl+Alt+Space` as well as `Ctrl+Alt+Shift+2` are turned into `^[^@`
* Ensured other, random `Ctrl+Alt+Key` combination behave identical to the current master
This adds the support for chaining OSC 10-12, allowing users to set all
of them at once.
**BREAKING CHANGE**
Before this PR, the OSC 10/11/12 command will only be dispatched iff the
first color is valid. This is no longer true. The new implementation
strictly follows xterm's behavior. Each color is treated independently.
For example, `\e]10;invalid;white\e\\` is effectively `\e]11;white\e\\`.
## Validation Steps Performed
Tests added. Manually tested.
Main OSC color tracking issue: #942
OSC 4 & Initial OSC 10-12 PR: #7578
Closes one item in #942
Pressing Ctrl+\ produces `^\` using the US keyboard layout, thanks to Ctrl-key mappings inside the keyboard layout, whereas some layouts, like the UK extended layout, don't contain those. This causes the character value to be zero and previously caused no VT sequence to be generated under these situations. This PR employs `MapVirtualKeyW` to infer the missing characters.
As a side effect this PR effectively causes _all_ major keys on the keyboard to produce Ctrl+combinations now.
## PR Checklist
* [x] Closes#8458
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests passed
## Validation Steps Performed
Compared all major keys in combination with Ctrl with the app store version of Terminal using `showkey` in WSL. All keys that previously worked still appear to continue to work.
This PR adds support for the `DECID` (Identify Device) escape sequence,
which allows for querying the terminal type in a way that is backwards
compatible with VT52 terminals.
This simply checks for the `ESC Z` sequence in the `ActionEscDispatch`
method of output state machine, and forwards the query to the existing
`DeviceAttributes` dispatch method, since the expected response is
identical to a `DA` report.
## Validation Steps Performed
I've added an output engine test that verifies that the `ESC Z` sequence
is correctly interpreted as a `DA` query when in ANSI mode, and as a
VT52 identification query when in VT52 mode.
Closes#8857
This adds the skeleton code for "bracketed paste mode" to the Windows
Terminal. No actual functionality is implemented yet, just the wiring
for handling DECSET/DECRST 2004.
References #395
Supersedes #7508
The Windows build system doesn't support PMR yet, so we had to add a workaround. :(
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 1614b27830a07484c18628ec7d9abc5f9d9b8a0b
Related work items: MSFT-31337042
Fix memory leak that occurs from not dispatching the end of sequences on all actions (since it is buffering up all characters for trace reasons.) Also don't bother storing if no one is listening.
## PR Checklist
- [x] Closes#8283
* [x] Fixes leak found while bumbling around.
* [x] I work here.
## Detailed Description of the Pull Request / Additional comments
- We trace all the things leading up to the Action phase in the VT parser for ETW tracing to make debugging the parser easier, but we made two mistakes.
- At some point, three of the actions (related to print/execute) weren't dispatching the stored up sequence to tracing and not clearing it. So printing/executing in a giant run over and over caused the vector to bloat and bloat and bloat forever.
- We're storing things even when no one is listening. That's a waste.
## Validation Steps Performed
- Watched it grow every time I did `type big.txt` under `taskman.exe`. Then watched it not do that after.
- I did technically WPR it to figure out this was the culprit.
N
* Change backslashes in include statements to forward slashes (CC-8205)
Dustin L. Howett
* Refactor DEC/ANSI modes to avoid duplication when we add SM/RM (GH-8469)
* Fix the xterm and SGR mouse encodings for CTRL, ALT, SHIFT (GH-8379)
James Holderness
* Fix rendering of DBCS characters when partially off screen (CC-8438)
* Correct horizontal coordinates in viewport overflow test (CC-8456)
* Correct paths in the the runut and runft test scripts (CC-8488)
* Retain horizontal viewport offset when moving to bottom (CC-8434)
* Fix out-of-bounds exceptions in Set...{Buffer,Screen}Size (CC-8309)
PankajBhojwani (1)
* Implement ConEmu's OSC 9;4 to set the taskbar progress indicator (CC-8055)
Chester Liu (1)
* Improve OSC 8 Hyperlink parsing logic (CC-7962)
Related work items: #31090459, #31090805, #31090808, #31090810
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_onecore_dep_uxp 91dff13f6953441afe9e28bfe3da9d5bff27bb05
Related: MSFT-26196390
Related work items: #26196390
I was about to add `SetAnsiMode`/`ResetAnsiMode` for `SM` and `RM` when I
realized that we probably don't need yet another enum of mode types, set and
reset functions, and a mode helper for ANSI standard modes when we already have
one for DEC Private modes.
This commit:
1. Changes the enum `PrivateModeParams` to just be `ModeParams`
2. Differentiates ANSI Standard modes (IRM, KAM, SRM, ...) from DEC
Private modes (DECCOLM, DECCKM, ...) using a flag bit set in the enum
value.
3. Introduces a helper class for constructing these values much like
`VTID`. That helper takes a bitmask and applies it to an input to
produce the final enum value.
4. Dispatches all mode set/reset through a common Set/Reset and
`_ModeHelper` that uses the existing enum values.
[1] These modes are in separate namespaces with some overlap. We want to
differentiate them at dispatch time to ensure that `\e[2h` and `\e[?2h` are
given different treatment, and ensure that `\e[1000h` doesn't activate xterm
mouse mode.
Fixes#8457.
We had the xterm and SGR codings for meta/ctrl backwards. Oops.
This commit also fixes an observed issue in Windows Terminal where we
were passing in a console-style modifiers enum when MouseInput is
expecting MK_ constants.
I decided to unify MouseInput around the console-style modifier
constants because they have support for META (which MK_ does not) and
can differentiate between left/right alt/ctrl.
Our tests are fundamentally flawed here: they use a copy of the
modifier key generating logic _themselves_, so we got a bit of "error
carried forward."
I did not fix the tests to use known-good control sequences, I simply
replaced the character generator with another copy of the modifier code.
I did, however, extend them to test ctrl|meta and left/right modifiers.
Fixes#8291
Many include statements use forward slashes, while others use backwards
slashes. This is inconsistent formatting. For this reason, I changed the
backward slashes to forward slashes since that is the standard.
This commit implements the OSC 9;4 sequence per the [ConEmu style].
| sequence | description |
| ------------ | ------------ |
| `ESC ] 9 ; 4 ; st ; pr ST` | Set progress state on taskbar and tab. |
| | When `st` is: |
| | |
| | `0`: remove progress. |
| | `1`: set progress value to `pr` (number, 0-100). |
| | `2`: set the taskbar to the "Error" state |
| | `3`: set the taskbar to the "Indeterminate" state |
| | `4`: set the taskbar to the "Warning" state |
We've also extended this with:
* st 3: set indeterminate state
* st 4: set paused state
We handle multiple tabs sending the sequence by using the the last focused
control's taskbar state/progress.
Upon receiving the sequence in `TerminalApi`, we send an event that gets caught
by `TerminalPage`. `TerminalPage` then fires another event that gets caught by
`AppHost` and that's where we set the taskbar progress.
Closes#3004
[ConEmu style]: https://conemu.github.io/en/AnsiEscapeCodes.html#ConEmu_specific_OSC
Some sources files changes were necessary to retain build continuity in OS.
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_onecore_dep_uxp 7ff895ff770838526b6d1d9e7d582f3c1e36d85b
In `ActionOscDispatch()` in `OutputStateMachineEngine.cpp`, we had a
section for parsing and another section for dispatching. This PR
combines those two blocks since they do not need to be distinct.
## Validation Steps Performed
TerminalApiTests still pass
The intent of this PR is to resolve the dependency errors reported in
#7931. The Types project has been added as a reference to the
FuzzWrapper project, which fixes the unresolved dependency errors
reported.
For validation steps, I:
1.) Pulled down main.
2.) Rebuilt the FuzzWrapper project and observed the unresolved
dependency errors keeping it from building successfully.
3.) Added a project reference to the Types project.
4.) Rebuilt the FuzzWrapper project and verified that the dependency
errors disappeared.
Closes#7931.
This commit adds a missing conversion utf8 to utf16 in decoding base64
for handling multibyte text in copying via OSC 52.
## Validation Steps Performed
* automatically
* Tests w/ multibyte characters
* manually
* case1
* Executed `printf "\x1b]52;;%s\x1b\\" "$(printf '👍👍🏻👍🏼👍🏽👍🏾👍🏿' | base64)"`
* Verified `👍👍🏻👍🏼👍🏽👍🏾👍🏿` in my clipboard
* case2
* Copied `👍👍🏻👍🏼👍🏽👍🏾👍🏿` by tmux 2.6 default copy function (OSC 52)
* Verified `👍👍🏻👍🏼👍🏽👍🏾👍🏿` in my clipboard
Closes#7819
This PR adds support for the `DECREQTPARM` (Request Terminal Parameters)
escape sequence, which was originally used on the VT100 terminal to
report the serial communication parameters. Modern terminal emulators
simply hardcode the reported values for backward compatibility.
The `DECREQTPARM` sequence has one parameter, which was originally used
to tell the terminal whether it was permitted to send unsolicited
reports or not. However, since we have no reason to send an unsolicited
report, we don't need to keep track of that state, but the permission
parameter does still determine the value of the first parameter in the
response.
The response parameters are as follows:
| Parameter | Value | Meaning |
| ---------------- | ------ | ------------------------ |
| response type | 2 or 3 | unsolicited or solicited |
| parity | 1 | no parity |
| data bits | 1 | 8 bits per character |
| transmit speed | 128 | 38400 baud |
| receive speed | 128 | 38400 baud |
| clock multiplier | 1 | |
| flags | 0 | |
There is some variation in the baud rate reported by modern terminal
emulators, and 9600 baud seems to be a little more common than 38400
baud, but I thought the higher speed was probably more appropriate,
especially since that's also the value reported by XTerm.
## Validation Steps Performed
I've added a couple of adapter and output engine tests to verify that
the sequence is dispatched correctly, and the expected responses are
generated. I've also manually tested in Vttest and confirmed that we now
pass the `DECREQTPARM` test in the _Test of terminal reports_.
Closes#7852
This PR introduces a pair of classes for managing VT parameters that
automatically handle range checking and default fallback values, so the
individual operations don't have to do that validation themselves. In
addition to simplifying the code, this fixes a few cases where we were
mishandling missing or extraneous parameters, and adds support for
parameter sequences on commands that couldn't previously handle them.
This PR also sets a limit on the number of parameters allowed, to help
thwart DoS memory consumption attacks.
## References
* The new parameter class also introduces the concept of an
omitted/default parameter which is not necessarily zero, which is a
prerequisite for addressing issue #4417.
## Detailed Description of the Pull Request / Additional comments
There are two new classes provide by this PR: a `VTParameter` class,
similar in function to a `std::optional<size_t>`, which holds an
individual parameter (which may be an omitted/default value); and a
`VTParameters` class, similar in function to `gsl:span<VTParameter>`,
which holds a sequence of those parameters.
Where `VTParameter` differs from `std::optional` is with the inclusion
of two cast operators. There is a `size_t` cast that interprets omitted
and zero values as 1 (the expected behaviour for most numeric
parameters). And there is a generic cast, for use with the enum
parameter types, which interprets omitted values as 0 (the expected
behaviour for most selective parameters).
The advantage of `VTParameters` class is that it has an `at` method that
can never fail - out of range values simply return the a default
`VTParameter` instance (this is standard behaviour in VT terminals). It
also has a `size` method that will always return a minimum count of 1,
since an empty parameter list is typically the equivalent of a single
"default" parameter, so this guarantees you'll get at least one value
when iterating over the list with `size()`.
For cases where we just need to call the same dispatch method for every
parameter, there is a helper `for_each` method, which repeatedly calls a
given predicate function with each value in the sequence. It also
collates the returned success values to determine the overall result of
the sequence. As with the `size` method, this will always make at least
one call, so it correctly handles empty sequences.
With those two classes in place, we could get rid of all the parameter
validation and default handling code in the `OutputStateMachineEngine`.
We now just use the `VTParameters::at` method to grab a parameter and
typically pass it straight to the appropriate dispatch method, letting
the cast operators automatically handle the assignment of default
values. Occasionally we might need a `value_or` call to specify a
non-standard default value, but those cases are fairly rare.
In some case the `OutputStateMachineEngine` was also checking whether
parameters values were in range, but for the most part this shouldn't
have been necessary, since that is something the dispatch classes would
already have been doing themselves (in the few cases that they weren't,
I've now updated them to do so).
I've also updated the `InputStateMachineEngine` in a similar way to the
`OutputStateMachineEngine`, getting rid of a few of the parameter
extraction methods, and simplifying other parts of the implementation.
It's not as clean a replacement as the output engine, but there are
still benefits in using the new classes.
## Validation Steps Performed
For the most part I haven't had to alter existing tests other than
accounting for changes to the API. There were a couple of tests I needed
to drop because they were checking for failure cases which shouldn't
have been failing (unexpected parameters should never be an error), or
testing output engine validation that is no longer handled at that
level.
I've added a few new tests to cover operations that take sequences of
selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've
extended the cursor movement tests to make sure those operations can
handle extraneous parameters that weren't expected. I've also added a
test to verify that the state machine will correctly ignore parameters
beyond the maximum 32 parameter count limit.
I've also manual confirmed that the various test cases given in issues
#2101 are now working as expected.
Closes#2101
* Correct the behaviour of parsing `rgb:R/G/B`. It should be interpreted
as `RR/GG/BB` instead of `0R/0G/0B`
* Add support for `rgb:RRR/GGG/BBB` and `rgb:RRRR/GGGG/BBBB`. The
behaviour of 12 bit variants is to repeat the first digit at the end,
e.g. `rgb:123/456/789` becomes `rgb:1231/4564/7897`.
* Add support for `#` formats. We are following the rules of
[XParseColor] by interpreting `#RGB` as `R000G000B000`.
* Add support for XOrg app color names, which are supported by xterm, VTE
and many other terminal emulators.
* Multi-parameter OSC 4 is now supported.
* The chaining of OSC 10-12 is not yet supported. But the parameter
validation is relaxed by parsing the parameters as multi-params but
only use the first one, which means `\e]10;rgb:R/G/B;` and
`\e]10:rgb:R/G/B;invalid` will execute `OSC 10` with the first color
correctly. This fixes some of the issues mentioned in #942 but not
all of them.
[XParseColor]: https://linux.die.net/man/3/xparsecolorCloses#3715
This PR adds support for the _blink_ graphic rendition attribute. When a
character is output with this attribute set, it "blinks" at a regular
interval, by cycling its color between the normal rendition and a dimmer
shade of that color.
The majority of the blinking mechanism is encapsulated in a new
`BlinkingState` class, which is shared between the Terminal and Conhost
implementations. This class keeps track of the position in the blinking
cycle, which determines whether characters are rendered as normal or
faint.
In Windows Terminal, the state is stored in the `Terminal` class, and in
Conhost it's stored in the `CONSOLE_INFORMATION` class. In both cases,
the `IsBlinkingFaint` method is used to determine the current blinking
rendition, and that is passed on as a parameter to the
`TextAttribute::CalculateRgbColors` method when these classes are
looking up attribute colors.
Prior to calculating the colors, the current attribute is also passed to
the `RecordBlinkingUsage` method, which keeps track of whether there are
actually any blink attributes in use. This is used to determine whether
the screen needs to be refreshed when the blinking cycle toggles between
the normal and faint renditions.
The refresh itself is handled by the `ToggleBlinkingRendition` method,
which is triggered by a timer. In Conhost this is just piggybacking on
the existing cursor blink timer, but in Windows Terminal it needs to
have its own separate timer, since the cursor timer is reset whenever a
key is pressed, which is not something we want for attribute blinking.
Although the `ToggleBlinkingRendition` is called at the same rate as the
cursor blinking, we actually only want the cells to blink at half that
frequency. We thus have a counter that cycles through four phases, and
blinking is rendered as faint for two of those four. Then every two
cycles - when the state changes - a redraw is triggered, but only if
there are actually blinking attributes in use (as previously recorded).
As mentioned earlier, the blinking frequency is based on the cursor
blink rate, so that means it'll automatically be disabled if a user has
set their cursor blink rate to none. It can also be disabled by turning
off the _Show animations in Windows_ option. In Conhost these settings
take effect immediately, but in Windows Terminal they only apply when a
new tab is opened.
This PR also adds partial support for the `SGR 6` _rapid blink_
attribute. This is not used by DEC terminals, but was defined in the
ECMA/ANSI standards. It's not widely supported, but many terminals just
it implement it as an alias for the regular `SGR 5` blink attribute, so
that's what I've done here too.
## Validation Steps Performed
I've checked the _Graphic rendition test pattern_ in Vttest, and
compared our representation of the blink attribute to that of an actual
DEC VT220 terminal as seen on [YouTube]. With the right color scheme
it's a reasonably close match.
[YouTube]: https://www.youtube.com/watch?v=03Pz5AmxbE4&t=1m55sCloses#7388
C1 control characters are now first converted to their 7 bit equivalent.
This allows us to unify the logic of C1 and C0 escape handling. This
also adds support for SOS/PM/APC string.
* Unify the logic for C1 and C0 escape handling by converting C1 to C0
beforehand. This adds support for various C1 characters, including
IND(8/4), NEL(8/5), HTS(8/8), RI(8/13), SS2(8/14), SS3(8/15),
OSC(9/13), etc.
* Add support for SOS/PM/APC escape sequences. Fixes#7032
* Use "Variable Length String" logic to unify the string termination
handling of OSC, DCS and SOS/PM/APC. This fixes an issue where OSC
action is successfully dispatched even when terminated with non-ST
character. Introduced by #6328, the DCS PassThrough is spared from
this issue. This PR puts them together and add test cases for them.
References:
https://vt100.net/docs/vt510-rm/chapter4.htmlhttps://vt100.net/emu/dec_ansi_parserCloses#7032Closes#7317
This fixes an issue where two CPRs could end up corrupted in the input
buffer. An application that sent two CPRs back-to-back could
end up reading the first few characters of the first prepended CPR
before handing us another CPR. We would dutifully prepend it to the
buffer, causing them to overlap.
```
^[^[2;2R[1;1R
^^ ^^^^^ First CPR
^^^^^^ Second CPR
```
The end result of this corruption is that a requesting application
would receive an unbidden `R` on stdin; for vim, this would trigger
replace mode immediately on startup.
Response prepending was implemented in !997738 without much comment.
There's very little in the way of audit trail as to why we switched.
Michael believes that we wanted to make sure that applications got DSR
responses immediately. It had the unfortunate side effect of causing
subsequence CPRs across cursor moves to come out in the wrong order.
I discussed our options with him, and he suggested that we could
implement a priority queue in InputBuffer and make sure that "response"
input was dispatched to a client application before any application- or
user-generated input. This was deemed to be too much work.
We decided that DSR responses getting top billing was likely to be a
stronger guarantee than most terminals are capable of giving, and that
we should be fine if we just switch it back to append.
Thanks to @k-takata, @tekki and @brammool for the investigation on the
vim side.
Fixes#1637.
This PR is about the behavior of DECSCUSR. This PR changes the meaning
of DECSCUSR 0 to restore the cursor style back to user default. This
differs from what VT spec says but it’s used in popular terminal
emulators like iTerm2 and VTE-based ones. See #1604.
Another change is that for parameter greater than 6, DECSCUSR should be
ignored, instead of restoring the cursor to legacy. This PR fixes it.
See #7382.
Fixes#1604.
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
## References
#204
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [X] Closes#204
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
* [ ] Schema updated.
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
Added support to:
- parse OSC8 sequences and extract URIs from them (conhost and terminal)
- add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal)
- attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal)
- process ctrl+leftclick to open a hyperlink in the clicked region if present
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
Open up a PowerShell tab and type
```PowerShell
${ESC}=[char]27
Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\"
```
Ctrl+LeftClick on the link correctly brings you to the terminal page on github
This commit removes our local copy of clang-format 8 and replaces it
with a newly-built nuget package containing clang-format 10.
This resolves the inconsistency between our version of clang-format and
the one shipped in Visual Studio.
A couple minor format changes were either required or erroneously forced
upon us--chief among them is a redistribution of `*`s around SAL
annotations in inline class members of COM classes. Don't ask why; I
couldn't figure it out.
We had some aspirational goals for our formatting, which were left in
but commented out. Enabling them changes our format a little more than
I'm comfortable with, so I uncommented them and locked them to the
format style we've been using for the past year. We may not love it, but
our aspirations may not matter here any longer. Consistent formatting is
better than perfect formatting.
This PR changes the way VT control sequences are identified and
dispatched, to be more efficient and easier to extend. Instead of
parsing the intermediate characters into a vector, and then having to
identify a sequence using both that vector and the final char, we now
use just a single `uint64_t` value as the identifier.
The way the identifier is constructed is by taking the private parameter
prefix, each of the intermediate characters, and then the final
character, and shifting them into a 64-bit integer one byte at a time,
in reverse order. For example, the `DECTLTC` control has a private
parameter prefix of `?`, one intermediate of `'`, and a final character
of `s`. The ASCII values of those characters are `0x3F`, `0x27`, and
`0x73` respectively, and reversing them gets you 0x73273F, so that would
then be the identifier for the control.
The reason for storing them in reverse order, is because sometimes we
need to look at the first intermediate to determine the operation, and
treat the rest of the sequence as a kind of sub-identifier (the
character set designation sequences are one example of this). When in
reverse order, this can easily be achieved by masking off the low byte
to get the first intermediate, and then shifting the value right by 8
bits to get a new identifier with the rest of the sequence.
With 64 bits we have enough space for a private prefix, six
intermediates, and the final char, which is way more than we should ever
need (the _DEC STD 070_ specification recommends supporting at least
three intermediates, but in practice we're unlikely to see more than
two).
With this new way of identifying controls, it should now be possible for
every action code to be unique (for the most part). So I've also used
this PR to clean up the action codes a bit, splitting the codes for the
escape sequences from the control sequences, and sorting them into
alphabetical order (which also does a reasonable job of clustering
associated controls).
## Validation Steps Performed
I think the existing unit tests should be good enough to confirm that
all sequences are still being dispatched correctly. However, I've also
manually tested a number of sequences to make sure they were still
working as expected, in particular those that used intermediates, since
they were the most affected by the dispatch code refactoring.
Since these changes also affected the input state machine, I've done
some manual testing of the conpty keyboard handling (both with and
without the new Win32 input mode enabled) to make sure the keyboard VT
sequences were processed correctly. I've also manually tested the
various VT mouse modes in Vttest to confirm that they were still working
correctly too.
Closes#7276
New warnings were added in VS 16.7 and `std::map::erase` is now `noexcept`.
Update our code to be compatible with the new enforcement.
## PR Checklist
* [x] Closes broken audit in main after Agents updated over the weekend.
* [x] I work here.
* [x] Audit mode passes now
* [x] Am core contributor.
## Validation Steps Performed
* [x] Ran audit mode locally
As the title suggests, this commit adds initial support for the VT DCS
sequences. The parameters are parsed but not yet used. The pass through
data is yet to be handled. This effectively fixes#120 by making Sixel
graphics sequences *ignored* instead of printed.
* https://vt100.net/docs/vt510-rm/chapter4.html
* https://vt100.net/emu/dec_ansi_parser
Tests added.
References #448Closes#120
This PR adds support for the ANSI _doubly underlined_ graphic rendition
attribute, which is enabled by the `SGR 21` escape sequence.
There was already an `ExtendedAttributes::DoublyUnderlined` flag in the
`TextAttribute` class, but I needed to add `SetDoublyUnderlined` and
`IsDoublyUnderlined` methods to access that flag, and update the
`SetGraphicsRendition` methods of the two dispatchers to set the
attribute on receipt of the `SGR 21` sequence. I also had to update the
existing `SGR 24` handler to reset _DoublyUnderlined_ in addition to
_Underlined_, since they share the same reset sequence.
For the rendering, I've added a new grid line type, which essentially
just draws an additional line with the same thickness as the regular
underline, but slightly below it - I found a gap of around 0.05 "em"
between the lines looked best. If there isn't enough space in the cell
for that gap, the second line will be clamped to overlap the first, so
you then just get a thicker line. If there isn't even enough space below
for a thicker line, we move the offset _above_ the first line, but just
enough to make it thicker.
The only other complication was the update of the `Xterm256Engine` in
the VT renderer. As mentioned above, the two underline attributes share
the same reset sequence, so to forward that state over conpty we require
a slightly more complicated process than with most other attributes
(similar to _Bold_ and _Faint_). We first check whether either underline
attribute needs to be turned off to send the reset sequence, and then
check individually if each of them needs to be turned back on again.
## Validation Steps Performed
For testing, I've extended the existing attribute tests in
`AdapterTest`, `VTRendererTest`, and `ScreenBufferTests`, to make sure
we're covering both the _Underlined_ and _DoublyUnderlined_ attributes.
I've also manually tested the `SGR 21` sequence in conhost and Windows
Terminal, with a variety of fonts and font sizes, to make sure the
rendering was reasonably distinguishable from a single underline.
Closes#2916
MouseInput was directly asking user32 about the state of the mouse buttons,
which was somewhat of a layering violation. This commit makes all callers
have to pass the mouse state in themselves.
Closes#4869
Carlos Zamora (1)
* UIA: use full buffer comparison in rects and endpoint setter (GH-6447)
Dan Thompson (2)
* Tweaks: normalize TextAttribute method names (adjective form) (GH-6951)
* Fix 'bcz exclusive' typo (GH-6938)
Dustin L. Howett (4)
* Fix VT mouse capture issues in Terminal and conhost (GH-7166)
* version: bump to 1.3 on master
* Update Cascadia Code to 2007.15 (GH-6958)
* Move to the TerminalDependencies NuGet feed (GH-6954)
James Holderness (3)
* Render the SGR "underlined" attribute in the style of the font (CC-7148)
* Add support for the "crossed-out" graphic rendition attribute (CC-7143)
* Refactor grid line renderers with support for more line types (CC-7107)
Leonard Hecker (1)
* Added til::spsc, a lock-free, single-producer/-consumer FIFO queue (CC-6751)
Michael Niksa (6)
* Update TAEF to 10.57.200731005-develop (GH-7164)
* Skip DX invalidation if we've already scrolled an entire screen worth of height (GH-6922)
* Commit attr runs less frequently by accumulating length of color run (GH-6919)
* Set memory order on slow atomics (GH-6920)
* Cache the viewport to make invalidation faster (GH-6918)
* Correct comment in this SPSC test as a quick follow up to merge.
Related work items: MSFT-28208358
