6140fd9 causes a binary size regression in conhost.
This PR fixes most if not all of the regression, by replacing `FMT_STRING`
with `FMT_COMPILE` allowing us to drop most of the formatters built
into fmt during linking (for instance floating point formatters).
Additionally `std::wstring` was replaced with `fmt::basic_memory_buffer`
in the same vein as was done for VtEngine. Stack is
cheap and this prevents any unnecessary allocations.
## PR Checklist
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
* vttest 11.2.5.3.6.7 and .8 (DECSTBM and SGR) complete successfully ✅
## Summary of the Pull Request
In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day.
## References
This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728.
## PR Checklist
* [x] Closes#11461
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461
## Detailed Description of the Pull Request / Additional comments
Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code.
And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values.
There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values.
These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame.
The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty.
The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before.
One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive.
The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone.
## Validation Steps Performed
A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate.
In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation.
This includes:
- `WriteConsoleOutput`
- `ReadConsoleOutput`
- `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter`
- `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter`
- `ScrollConsoleScreenBuffer`
- `GetConsoleScreenBufferInfo`
- `GetConsoleScreenBufferInfoEx`
- `SetConsoleScreenBufferInfoEx`
I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell.
In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation.
I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
Instead of having a separate method for setting each mouse and keyboard
mode, this PR consolidates them all into a single method which takes a
mode parameter, and stores the modes in a `til::enumset` rather than
having a separate `bool` for each mode.
This enables us to get rid of a lot of boilerplate code, and makes the
code easier to extend when we want to introduce additional modes in the
future. It'll also makes it easier to read back the state of the various
modes when implementing the `DECRQM` query.
Most of the complication is in the `TerminalInput` class, which had to
be adjusted to work with an `enumset` in place of all the `bool` fields.
For the rest, it was largely a matter of replacing calls to all the old
mode setting methods with the new `SetInputMode` method, and deleting a
bunch of unused code.
One thing worth mentioning is that the `AdaptDispatch` implementation
used to have a `_ShouldPassThroughInputModeChange` method that was
called after every mode change. This code has now been moved up into the
`SetInputMode` implementation in `ConhostInternalGetSet` so it's just
handled in one place. Keeping this out of the dispatch class will also
be beneficial for sharing the implementation with `TerminalDispatch`.
## Validation
The updated interface necessitated some adjustments to the tests in
`AdapterTest` and `MouseInputTest`, but the essential structure of the
tests remains unchanged, and everything still passes.
I've also tested the keyboard and mouse modes in Vttest and confirmed
they still work at least as well as they did before (both conhost and
Windows Terminal), and I tested the alternate scroll mode manually
(conhost only).
Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan
to de-duplicate the `AdaptDispatch` and `TerminalDispatch`
implementation (#3849).
This PR adds support for the `DECRQSS` (Request Selection or Setting)
escape sequence, which is a standard VT query for reporting the state of
various control functions. This initial implementation only supports
queries for the `DECSTBM` margins, and the `SGR` graphic rendition
attributes.
This can be useful for certain forms of capability detection (#1040). As
one example in particular, it can serve as an alternative to the
`COLORTERM` environment variable for detecting truecolor support
(#11057).
Of the settings that can be queried by `DECRQSS`, the only other one
that we could be supporting at the moment is `DECSCUSR` (Cursor Style).
However, that would require passing the query through to the conpty
client, which is a lot more complicated, so I thought it best to leave
for a future PR.
For now this gets the basic framework in place, so we are at least
responding to queries, and even just supporting the `SGR` attributes
query is useful in itself.
Validation
----------
I've added a unit test verifying the reports for the `DECSTBM` and `SGR`
settings with a range of different parameters. I've also tested the
`DECSTBM` and `SGR` reports manually in _Vttest_, under menu 11.2.5.3.6
(Status-String Reports).
This PR adds conhost support for downloadable soft fonts - also known as
dynamically redefinable character sets (DRCS) - using the `DECDLD`
escape sequence.
These fonts are typically designed to work on a specific terminal model,
and each model tends to have a different character cell size. So in
order to support as many models as possible, the code attempts to detect
the original target size of the font, and then scale the glyphs to fit
our current cell size.
Once a font has been downloaded to the terminal, it can be designated in
the same way you would a standard character set, using an `SCS` escape
sequence. The identification string for the set is defined by the
`DECDLD` sequence. Internally we map the characters in this set to code
points `U+EF20` to `U+EF7F` in the Unicode private use are (PUA).
Then in the renderer, any characters in that range are split off into
separate runs, which get painted with a special font. The font itself is
dynamically generated as an in-memory resource, constructed from the
downloaded character bitmaps which have been scaled to the appropriate
size.
If no soft fonts are in use, then no mapping of the PUA code points will
take place, so this shouldn't interfere with anyone using those code
points for something else, as along as they aren't also trying to use
soft fonts. I also tried to pick a PUA range that hadn't already been
snatched up by Nerd Fonts, but if we do receive reports of a conflict,
it's easy enough to change.
## Validation Steps Performed
I added an adapter test that runs through a bunch of parameter
variations for the `DECDLD` sequence, to make sure we're correctly
detecting the font sizes for most of the known DEC terminal models.
I've also tested manually on a wide range of existing fonts, of varying
dimensions, and from multiple sources, and made sure they all worked
reasonably well.
Closes#9164
## Summary of the Pull Request
RIS resets mouse mode and encoding
## PR Checklist
* [x] Closes#8613
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
* [ ] Schema updated.
* [x] I work here
## Validation Steps Performed
## Summary of the Pull Request
The "virtual bottom" marks the last line of the mutable viewport area, which is the part of the buffer that VT sequences can write to. This region should typically only move downwards, as new lines are added to the buffer, but there were a number of cases where it was incorrectly being moved up. This PR attempts to fix that.
## PR Checklist
* [x] Closes#9754
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #9754
## Detailed Description of the Pull Request / Additional comments
When a call is made to `UpdateBottom`, we now clamp the value so it's at least as low as the current viewport bottom (i.e. if the viewport has moved down, we want the virtual bottom to move down too), but no lower than the bottom of the buffer (we don't want it to be out of range).
There is one special case where we do actually want the virtual bottom to move up - when the scrollback has been cleared with an `ED3` escape sequence. So in that case we needed a new `ConGetSet` API (`ResetBottom`) to reset the virtual bottom to the top of the buffer (essentially one less than the viewport height, since the virtual bottom points to the last line of the viewport).
## Validation Steps Performed
I had to reset the virtual bottom manually in some parts of the `ScreenBufferTests`, since some of the tests were relying on the virtual bottom being automatically reset when the viewport was reset, which is no longer the case.
I've also added a new test to verify that the virtual bottom doesn't move upwards if an update is triggered while the visible viewport is scrolled up. This essentially reproduces the test case from issue #9754, which I've also manually confirmed is fixed.
This PR adds support for the VT line rendition attributes, which allow
for double-width and double-height line renditions. These renditions are
enabled with the `DECDWL` (double-width line) and `DECDHL`
(double-height line) escape sequences. Both reset to the default
rendition with the `DECSWL` (single-width line) escape sequence. For now
this functionality is only supported by the GDI renderer in conhost.
There are a lot of changes, so this is just a general overview of the
main areas affected.
Previously it was safe to assume that the screen had a fixed width, at
least for a given point in time. But now we need to deal with the
possibility of different lines have different widths, so all the
functions that are constrained by the right border (text wrapping,
cursor movement operations, and sequences like `EL` and `ICH`) now need
to lookup the width of the active line in order to behave correctly.
Similarly it used to be safe to assume that buffer and screen
coordinates were the same thing, but that is no longer true. Lots of
places now need to translate back and forth between coordinate systems
dependent on the line rendition. This includes clipboard handling, the
conhost color selection and search, accessibility location tracking and
screen reading, IME editor positioning, "snapping" the viewport, and of
course all the rendering calculations.
For the rendering itself, I've had to introduce a new
`PrepareLineTransform` method that the render engines can use to setup
the necessary transform matrix for a given line rendition. This is also
now used to handle the horizontal viewport offset, since that could no
longer be achieved just by changing the target coordinates (on a double
width line, the viewport offset may be halfway through a character).
I've also had to change the renderer's existing `InvalidateCursor`
method to take a `SMALL_RECT` rather than a `COORD`, to allow for the
cursor being a variable width. Technically this was already a problem,
because the cursor could occupy two screen cells when over a
double-width character, but now it can be anything between one and four
screen cells (e.g. a double-width character on the double-width line).
In terms of architectural changes, there is now a new `lineRendition`
field in the `ROW` class that keeps track of the line rendition for each
row, and several new methods in the `ROW` and `TextBuffer` classes for
manipulating that state. This includes a few helper methods for handling
the various issues discussed above, e.g. position clamping and
translating between coordinate systems.
## Validation Steps Performed
I've manually confirmed all the double-width and double-height tests in
_Vttest_ are now working as expected, and the _VT100 Torture Test_ now
renders correctly (at least the line rendition aspects). I've also got
my own test scripts that check many of the line rendition boundary cases
and have confirmed that those are now passing.
I've manually tested as many areas of the conhost UI that I could think
of, that might be affected by line rendition, including things like
searching, selection, copying, and color highlighting. For
accessibility, I've confirmed that the _Magnifier_ and _Narrator_
correctly handle double-width lines. And I've also tested the Japanese
IME, which while not perfect, is at least useable.
Closes#7865
This adds the skeleton code for "bracketed paste mode" to the Windows
Terminal. No actual functionality is implemented yet, just the wiring
for handling DECSET/DECRST 2004.
References #395
Supersedes #7508
I was about to add `SetAnsiMode`/`ResetAnsiMode` for `SM` and `RM` when I
realized that we probably don't need yet another enum of mode types, set and
reset functions, and a mode helper for ANSI standard modes when we already have
one for DEC Private modes.
This commit:
1. Changes the enum `PrivateModeParams` to just be `ModeParams`
2. Differentiates ANSI Standard modes (IRM, KAM, SRM, ...) from DEC
Private modes (DECCOLM, DECCKM, ...) using a flag bit set in the enum
value.
3. Introduces a helper class for constructing these values much like
`VTID`. That helper takes a bitmask and applies it to an input to
produce the final enum value.
4. Dispatches all mode set/reset through a common Set/Reset and
`_ModeHelper` that uses the existing enum values.
[1] These modes are in separate namespaces with some overlap. We want to
differentiate them at dispatch time to ensure that `\e[2h` and `\e[?2h` are
given different treatment, and ensure that `\e[1000h` doesn't activate xterm
mouse mode.
Fixes#8457.
This commit implements the OSC 9;4 sequence per the [ConEmu style].
| sequence | description |
| ------------ | ------------ |
| `ESC ] 9 ; 4 ; st ; pr ST` | Set progress state on taskbar and tab. |
| | When `st` is: |
| | |
| | `0`: remove progress. |
| | `1`: set progress value to `pr` (number, 0-100). |
| | `2`: set the taskbar to the "Error" state |
| | `3`: set the taskbar to the "Indeterminate" state |
| | `4`: set the taskbar to the "Warning" state |
We've also extended this with:
* st 3: set indeterminate state
* st 4: set paused state
We handle multiple tabs sending the sequence by using the the last focused
control's taskbar state/progress.
Upon receiving the sequence in `TerminalApi`, we send an event that gets caught
by `TerminalPage`. `TerminalPage` then fires another event that gets caught by
`AppHost` and that's where we set the taskbar progress.
Closes#3004
[ConEmu style]: https://conemu.github.io/en/AnsiEscapeCodes.html#ConEmu_specific_OSC
This PR adds support for the `DECREQTPARM` (Request Terminal Parameters)
escape sequence, which was originally used on the VT100 terminal to
report the serial communication parameters. Modern terminal emulators
simply hardcode the reported values for backward compatibility.
The `DECREQTPARM` sequence has one parameter, which was originally used
to tell the terminal whether it was permitted to send unsolicited
reports or not. However, since we have no reason to send an unsolicited
report, we don't need to keep track of that state, but the permission
parameter does still determine the value of the first parameter in the
response.
The response parameters are as follows:
| Parameter | Value | Meaning |
| ---------------- | ------ | ------------------------ |
| response type | 2 or 3 | unsolicited or solicited |
| parity | 1 | no parity |
| data bits | 1 | 8 bits per character |
| transmit speed | 128 | 38400 baud |
| receive speed | 128 | 38400 baud |
| clock multiplier | 1 | |
| flags | 0 | |
There is some variation in the baud rate reported by modern terminal
emulators, and 9600 baud seems to be a little more common than 38400
baud, but I thought the higher speed was probably more appropriate,
especially since that's also the value reported by XTerm.
## Validation Steps Performed
I've added a couple of adapter and output engine tests to verify that
the sequence is dispatched correctly, and the expected responses are
generated. I've also manually tested in Vttest and confirmed that we now
pass the `DECREQTPARM` test in the _Test of terminal reports_.
Closes#7852
This PR introduces a pair of classes for managing VT parameters that
automatically handle range checking and default fallback values, so the
individual operations don't have to do that validation themselves. In
addition to simplifying the code, this fixes a few cases where we were
mishandling missing or extraneous parameters, and adds support for
parameter sequences on commands that couldn't previously handle them.
This PR also sets a limit on the number of parameters allowed, to help
thwart DoS memory consumption attacks.
## References
* The new parameter class also introduces the concept of an
omitted/default parameter which is not necessarily zero, which is a
prerequisite for addressing issue #4417.
## Detailed Description of the Pull Request / Additional comments
There are two new classes provide by this PR: a `VTParameter` class,
similar in function to a `std::optional<size_t>`, which holds an
individual parameter (which may be an omitted/default value); and a
`VTParameters` class, similar in function to `gsl:span<VTParameter>`,
which holds a sequence of those parameters.
Where `VTParameter` differs from `std::optional` is with the inclusion
of two cast operators. There is a `size_t` cast that interprets omitted
and zero values as 1 (the expected behaviour for most numeric
parameters). And there is a generic cast, for use with the enum
parameter types, which interprets omitted values as 0 (the expected
behaviour for most selective parameters).
The advantage of `VTParameters` class is that it has an `at` method that
can never fail - out of range values simply return the a default
`VTParameter` instance (this is standard behaviour in VT terminals). It
also has a `size` method that will always return a minimum count of 1,
since an empty parameter list is typically the equivalent of a single
"default" parameter, so this guarantees you'll get at least one value
when iterating over the list with `size()`.
For cases where we just need to call the same dispatch method for every
parameter, there is a helper `for_each` method, which repeatedly calls a
given predicate function with each value in the sequence. It also
collates the returned success values to determine the overall result of
the sequence. As with the `size` method, this will always make at least
one call, so it correctly handles empty sequences.
With those two classes in place, we could get rid of all the parameter
validation and default handling code in the `OutputStateMachineEngine`.
We now just use the `VTParameters::at` method to grab a parameter and
typically pass it straight to the appropriate dispatch method, letting
the cast operators automatically handle the assignment of default
values. Occasionally we might need a `value_or` call to specify a
non-standard default value, but those cases are fairly rare.
In some case the `OutputStateMachineEngine` was also checking whether
parameters values were in range, but for the most part this shouldn't
have been necessary, since that is something the dispatch classes would
already have been doing themselves (in the few cases that they weren't,
I've now updated them to do so).
I've also updated the `InputStateMachineEngine` in a similar way to the
`OutputStateMachineEngine`, getting rid of a few of the parameter
extraction methods, and simplifying other parts of the implementation.
It's not as clean a replacement as the output engine, but there are
still benefits in using the new classes.
## Validation Steps Performed
For the most part I haven't had to alter existing tests other than
accounting for changes to the API. There were a couple of tests I needed
to drop because they were checking for failure cases which shouldn't
have been failing (unexpected parameters should never be an error), or
testing output engine validation that is no longer handled at that
level.
I've added a few new tests to cover operations that take sequences of
selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've
extended the cursor movement tests to make sure those operations can
handle extraneous parameters that weren't expected. I've also added a
test to verify that the state machine will correctly ignore parameters
beyond the maximum 32 parameter count limit.
I've also manual confirmed that the various test cases given in issues
#2101 are now working as expected.
Closes#2101
This fixes an issue where two CPRs could end up corrupted in the input
buffer. An application that sent two CPRs back-to-back could
end up reading the first few characters of the first prepended CPR
before handing us another CPR. We would dutifully prepend it to the
buffer, causing them to overlap.
```
^[^[2;2R[1;1R
^^ ^^^^^ First CPR
^^^^^^ Second CPR
```
The end result of this corruption is that a requesting application
would receive an unbidden `R` on stdin; for vim, this would trigger
replace mode immediately on startup.
Response prepending was implemented in !997738 without much comment.
There's very little in the way of audit trail as to why we switched.
Michael believes that we wanted to make sure that applications got DSR
responses immediately. It had the unfortunate side effect of causing
subsequence CPRs across cursor moves to come out in the wrong order.
I discussed our options with him, and he suggested that we could
implement a priority queue in InputBuffer and make sure that "response"
input was dispatched to a client application before any application- or
user-generated input. This was deemed to be too much work.
We decided that DSR responses getting top billing was likely to be a
stronger guarantee than most terminals are capable of giving, and that
we should be fine if we just switch it back to append.
Thanks to @k-takata, @tekki and @brammool for the investigation on the
vim side.
Fixes#1637.
This PR is about the behavior of DECSCUSR. This PR changes the meaning
of DECSCUSR 0 to restore the cursor style back to user default. This
differs from what VT spec says but it’s used in popular terminal
emulators like iTerm2 and VTE-based ones. See #1604.
Another change is that for parameter greater than 6, DECSCUSR should be
ignored, instead of restoring the cursor to legacy. This PR fixes it.
See #7382.
Fixes#1604.
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
## References
#204
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [X] Closes#204
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
* [ ] Schema updated.
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
Added support to:
- parse OSC8 sequences and extract URIs from them (conhost and terminal)
- add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal)
- attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal)
- process ctrl+leftclick to open a hyperlink in the clicked region if present
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
Open up a PowerShell tab and type
```PowerShell
${ESC}=[char]27
Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\"
```
Ctrl+LeftClick on the link correctly brings you to the terminal page on github
This PR changes the way VT control sequences are identified and
dispatched, to be more efficient and easier to extend. Instead of
parsing the intermediate characters into a vector, and then having to
identify a sequence using both that vector and the final char, we now
use just a single `uint64_t` value as the identifier.
The way the identifier is constructed is by taking the private parameter
prefix, each of the intermediate characters, and then the final
character, and shifting them into a 64-bit integer one byte at a time,
in reverse order. For example, the `DECTLTC` control has a private
parameter prefix of `?`, one intermediate of `'`, and a final character
of `s`. The ASCII values of those characters are `0x3F`, `0x27`, and
`0x73` respectively, and reversing them gets you 0x73273F, so that would
then be the identifier for the control.
The reason for storing them in reverse order, is because sometimes we
need to look at the first intermediate to determine the operation, and
treat the rest of the sequence as a kind of sub-identifier (the
character set designation sequences are one example of this). When in
reverse order, this can easily be achieved by masking off the low byte
to get the first intermediate, and then shifting the value right by 8
bits to get a new identifier with the rest of the sequence.
With 64 bits we have enough space for a private prefix, six
intermediates, and the final char, which is way more than we should ever
need (the _DEC STD 070_ specification recommends supporting at least
three intermediates, but in practice we're unlikely to see more than
two).
With this new way of identifying controls, it should now be possible for
every action code to be unique (for the most part). So I've also used
this PR to clean up the action codes a bit, splitting the codes for the
escape sequences from the control sequences, and sorting them into
alphabetical order (which also does a reasonable job of clustering
associated controls).
## Validation Steps Performed
I think the existing unit tests should be good enough to confirm that
all sequences are still being dispatched correctly. However, I've also
manually tested a number of sequences to make sure they were still
working as expected, in particular those that used intermediates, since
they were the most affected by the dispatch code refactoring.
Since these changes also affected the input state machine, I've done
some manual testing of the conpty keyboard handling (both with and
without the new Win32 input mode enabled) to make sure the keyboard VT
sequences were processed correctly. I've also manually tested the
various VT mouse modes in Vttest to confirm that they were still working
correctly too.
Closes#7276
New warnings were added in VS 16.7 and `std::map::erase` is now `noexcept`.
Update our code to be compatible with the new enforcement.
## PR Checklist
* [x] Closes broken audit in main after Agents updated over the weekend.
* [x] I work here.
* [x] Audit mode passes now
* [x] Am core contributor.
## Validation Steps Performed
* [x] Ran audit mode locally
We were using std::basic_string_view as a stand-in for std::span so that
we could change over all at once when C++20 dropped with full span
support. That day's not here yet, but as of 54a7fce3e we're using GSL 3,
whose span is C++20-compliant.
This commit replaces every instance of basic_string_view that was not
referring to an actual string with a span of the appropriate type.
I moved the `const` qualifier into span's `T` because while
`basic_string_view.at()` returns `const T&`, `span.at()` returns `T&`
(without the const). I wanted to maintain the invariant that members of
the span were immutable.
* Mechanical Changes
* `sv.at(x)` -> `gsl::at(sp, x)`
* `sv.c{begin,end}` -> `sp.{begin,end}` (span's iterators are const)
I had to replace a `std::basic_string<>` with a `std::vector<>` in
ConImeInfo, and I chose to replace a manual array walk in
ScreenInfoUiaProviderBase with a ranged-for. Please review those
specifically.
This will almost certainly cause a code size regression in Windows
because I'm blowing out all the PGO counts. Whoops.
Related: #3956, #975.
This PR adds support for the `DA2` (Secondary Device Attributes) and
`DA3` (Tertiary Device Attributes) escape sequences, which are standard
VT queries reporting basic information about the terminal.
The _Secondary Device Attributes_ response is made up of a number of
parameters:
1. An identification code, for which I've used 0 to indicate that we
have the capabilities of a VT100 (using code 0 for this is an XTerm
convention, since technically DA2 would not have been supported by a
VT100).
2. A firmware revision level, which some terminal emulators use to
report their actual version number, but I thought it best we just
hardcode a value of 10 (the DEC convention for 1.0).
3. Additional hardware options, which tend to be device specific, but
I've followed the convention of the later DEC terminals using 1 to
indicate the presence of a PC keyboard.
The _Tertiary Device Attributes_ response was originally used to provide
a unique terminal identification code, and which some terminal emulators
use as a way to identify themselves. However, I think that's information
we'd probably prefer not to reveal, so I've followed the more common
practice of returning all zeros for the ID.
In terms of implementation, the only complication was the need to add an
additional code path in the `OutputStateMachine` to handle the `>` and
`=` intermediates (technically private parameter prefixes) that these
sequences require. I've done this as a single method - rather than one
for each prefix - since I think that makes the code easier to follow.
VALIDATION
----------
I've added output engine tests to make sure the sequences are dispatched
correctly, and adapter tests to confirm that they are returning the
responses we expect. I've also manually confirmed that they pass the
_Test of terminal reports_ in Vttest.
Closes#5836
## Summary of the Pull Request
This PR adds full support for the `DECSCNM` reverse screen mode in the Windows Terminal to align with the implementation in conhost.
## References
* The conhost implementation of `DECSCNM` was in PR #3817.
* WT originally inherited that functionality via the colors being passed through, but that behaviour was lost in PR #6506.
## PR Checklist
* [x] Closes#6622
* [x] CLA signed.
* [ ] Tests added/passed
* [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #6622
## Detailed Description of the Pull Request / Additional comments
The `AdaptDispatch::SetScreenMode` now checks if it's in conpty mode and simply returns false to force a pass-through of the mode change. And the `TerminalDispatch` now has its own `SetScreenMode` implementation that tracks any changes to the reversed state, and triggers a redraw in the renderer.
To make the renderer work, we just needed to update the `GetForegroundColor` and `GetBackgroundColor` methods of the terminal's `IRenderData` implementation to check the reversed state, and switch the colors being calculated, the same way the `LookupForegroundColor` and `LookupBackgroundColor` methods work in the conhost `Settings` class.
## Validation Steps Performed
I've manually tested the `DECSCNM` functionality for Windows Terminal in Vttest, and also with some of my own test scripts.
The VT reset operations `RIS` and `DECSTR` are implemented as a series
of steps, each of which could potentially fail. Currently these
operations abort as soon as an error is detected, which is particularly
problematic in conpty mode, where some steps deliberately "fail" to
indicate that they need to be "passed through" to the conpty client. As
a result, the reset won't be fully executed. This PR changes that
behaviour, so the error state is recorded for any failures, but the
subsequent steps are still run.
Originally the structure of these operations was of the form:
bool success = DoSomething();
if (success)
{
success = DoSomethingElse();
}
But I've now changed the code so it looks more like this:
bool success = DoSomething();
success = DoSomethingElse() && success;
This means that every one of the steps should execute, regardless of
whether previous steps were successful, but the final _success_ state
will only be true if none of the steps has failed.
While this is only really an issue in the conhost code, I've updated
both the `AdaptDispatch` and `TerminalDispatch` classes, since I thought
it would be best to have them in sync, and in general this seems like a
better way to handle multi-step operations anyway.
VALIDATION
I've manually tested the `RIS` escape sequence (`\ec`) in the Windows
Terminal, and confirmed that it now correctly resets the cursor
position, which it wasn't doing before.
Closes#6545
With this commit, terminal will be able to copy text to the system
clipboard by using OSC 52 MANIPULATE SELECTION DAATA.
We chose not to implement the clipboard querying functionality offered
by OSC 52, as sending the clipboard text to an application without the
user's knowledge or consent is an immense security hole.
We do not currently support the clipboard specifier Pc to specify which
clipboard buffer should be filled
# Base64 encoded `foo`
$ echo -en "\e]52;;Zm9v\a"
# Multiple lines
# Base64 encoded `foo\r\nbar`
$ echo -en "\e]52;;Zm9vDQpiYXI=\a"
Closes#2946.
This commit reverts the removal of the "SSH hack" in #5383. It was
originally added as a solution to #4911, when we realized that SSH would
request the SS3 cursor key encoding but we weren't equipped to handle
it.
A number of folks have filed issues that, in summary, say "when I use
SSH, I can't select/copy/paste text". It turns out that SSH will _also_
pass through requests for mouse input. Terminal dutifully responds to
those requests, of course, by disabling mouse selection/copy/paste. SSH
is **NOT** actually in VT_INPUT_MODE, so it will never receive the mouse
messages.
It's important to note that even with #376 fixed, we are still required
to keep this check. With the closure of #376, we'll be able to convert
VT mouse input back into Win32 mouse input for Win32 applications . . .
but SSH also doesn't know how to handle Win32 mouse input.
Fixes#6476.
Fixes#6196.
Fixes#5704.
Fixes#5608.
Adds support for `win32-input-mode` to conhost, conpty, and the Windows
Terminal.
* The shared `terminalInput` class supports sending these sequences when
a VT client application requests this mode.
* ConPTY supports synthesizing `INPUT_RECORD`s from the input sent to it
from a terminal
* ConPTY requests this mode immediately on startup (if started with a
new flag, `PSEUDOCONSOLE_WIN32_INPUT_MODE`)
* The Terminal now supports sending this input as well, when conpty asks
for it.
Also adds a new ConPTY flag `PSEUDOCONSOLE_WIN32_INPUT_MODE` which
requests this functionality from conpty, and the Terminal requests this
by default.
Also adds `experimental.input.forceVT` as a global setting to let a user
opt-out of this behavior, if they don't want it / this ends up breaking
horribly.
## Validation Steps Performed
* played with this mode in vtpipeterm
* played with this mode in Terminal
* checked a bunch of scenarios, as outlined in a [comment] on #4999
[comment]: https://github.com/microsoft/terminal/issues/4999#issuecomment-628718631
References #4999: The megathread
References #5887: The spec
Closes#879Closes#2865Closes#530Closes#3079Closes#1119Closes#1694Closes#3608Closes#4334Closes#4446
This PR improves our VT character set support, enabling the [`SCS`]
escape sequences to designate into all four G-sets with both 94- and
96-character sets, and supports invoking those G-sets into both the GL
and GR areas of the code table, with [locking shifts] and [single
shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the
ISO-2022 coding system (which is what the VT character sets require),
and adds support for a lot more characters sets, up to around the level
of a VT510.
[`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html
[locking shifts]: https://vt100.net/docs/vt510-rm/LS.html
[single shifts]: https://vt100.net/docs/vt510-rm/SS.html
[`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems
## Detailed Description of the Pull Request / Additional comments
To make it easier for us to declare a bunch of character sets, I've made
a little `constexpr` class that can build up a mapping table from a base
character set (ASCII or Latin1), along with a collection of mappings for
the characters the deviate from the base set. Many of the character sets
are simple variations of ASCII, so they're easy to define this way.
This class then casts directly to a `wstring_view` which is how the
translation tables are represented in most of the code. We have an array
of four of these tables representing the four G-sets, two instances for
the active left and right tables, and one instance for the single shift
table.
Initially we had just one `DesignateCharset` method, which could select
the active character set. We now have two designate methods (for 94- and
96- character sets), and each takes a G-set number specifying the target
of the designation, and a pair of characters identifying the character
set that will be designated (at the higher VT levels, character sets are
often identified by more than one character).
There are then two new `LockingShift` methods to invoke these G-sets
into either the GL or GR area of the code table, and a `SingleShift`
method which invokes a G-set temporarily (for just the next character
that is output).
I should mention here that I had to make some changes to the state
machine to make these single shift sequences work. The problem is that
the input state machine treats `SS3` as the start of a control sequence,
while the output state machine needs it to be dispatched immediately
(it's literally the _Single Shift 3_ escape sequence). To make that
work, I've added a `ParseControlSequenceAfterSs3` callback in the
`IStateMachineEngine` interface to decide which behavior is appropriate.
When it comes to mapping a character, it's simply an array reference
into the appropriate `wstring_view` table. If the single shift table is
set, that takes preference. Otherwise the GL table is used for
characters in the range 0x20 to 0x7F, and the GR table for characters
0xA0 to 0xFF (technically some character sets will only map up to 0x7E
and 0xFE, but that's easily controlled by the length of the
`wstring_view`).
The `DEL` character is a bit of a special case. By default it's meant to
be ignored like the `NUL` character (it's essentially a time-fill
character). However, it's possible that it could be remapped to a
printable character in a 96-character set, so we need to check for that
after the translation. This is handled in the `AdaptDispatch::Print`
method, so it doesn't interfere with the primary `PrintString` code
path.
The biggest problem with this whole process, though, is that the GR
mappings only really make sense if you have access to the raw output,
but by the time the output gets to us, it would already have been
translated to Unicode by the active code page. And in the case of UTF-8,
the characters we eventually receive may originally have been composed
from two or more code points.
The way I've dealt with this was to disable the GR translations by
default, and then added support for a pair of ISO-2022 `DOCS` sequences,
which can switch the code page between UTF-8 and ISO-8859-1. When the
code page is ISO-8859-1, we're essentially receiving the raw output
bytes, so it's safe to enable the GR translations. This is not strictly
correct ISO-2022 behavior, and there are edge cases where it's not going
to work, but it's the best solution I could come up with.
## Validation Steps Performed
As a result of the `SS3` changes in the state machine engine, I've had
to move the existing `SS3` tests from the `OutputEngineTest` to the
`InputEngineTest`, otherwise they would now fail (technically they
should never have been output tests).
I've added no additional unit tests, but I have done a lot of manual
testing, and made sure we passed all the character set tests in Vttest
(at least for the character sets we currently support). Note that this
required a slightly hacked version of the app, since by default it
doesn't expose a lot of the test to low-level terminals, and we
currently identify as a VT100.
Closes#3377Closes#3487
## Summary of the Pull Request
This PR adds support for the core VT52 commands, and implements the `DECANM` private mode sequence, which switches the terminal between ANSI mode and VT52-compatible mode.
## References
PR #2017 defined the initial specification for VT52 support.
PR #4044 removed the original VT52 cursor ops that conflicted with VT100 sequences.
## PR Checklist
* [x] Closes#976
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #2017
## Detailed Description of the Pull Request / Additional comments
Most of the work involves updates to the parsing state machine, which behaves differently in VT52 mode. `CSI`, `OSC`, and `SS3` sequences are not applicable, and there is one special-case escape sequence (_Direct Cursor Address_), which requires an additional state to handle parameters that come _after_ the final character.
Once the parsing is handled though, it's mostly just a matter of dispatching the commands to existing methods in the `ITermDispatch` interface. Only one new method was required in the interface to handle the _Identify_ command.
The only real new functionality is in the `TerminalInput` class, which needs to generate different escape sequences for certain keys in VT52 mode. This does not yet support _all_ of the VT52 key sequences, because the VT100 support is itself not yet complete. But the basics are in place, and I think the rest is best left for a follow-up issue, and potentially a refactor of the `TerminalInput` class.
I should point out that the original spec called for a new _Graphic Mode_ character set, but I've since discovered that the VT terminals that _emulate_ VT52 just use the existing VT100 _Special Graphics_ set, so that is really what we should be doing too. We can always consider adding the VT52 graphic set as a option later, if there is demand for strict VT52 compatibility.
## Validation Steps Performed
I've added state machine and adapter tests to confirm that the `DECANM` mode changing sequences are correctly dispatched and forwarded to the `ConGetSet` handler. I've also added state machine tests that confirm the VT52 escape sequences are dispatched correctly when the ANSI mode is reset.
For fuzzing support, I've extended the VT command fuzzer to generate the different kinds of VT52 sequences, as well as mode change sequences to switch between the ANSI and VT52 modes.
In terms of manual testing, I've confirmed that the _Test of VT52 mode_ in Vttest now works as expected.
This is an attempt to simplify the SGR (Select Graphic Rendition)
implementation in conhost, to cut down on the number of methods required
in the `ConGetSet` interface, and pave the way for future improvements
and bug fixes. It already fixes one bug that prevented SGR 0 from being
correctly applied when combined with meta attributes.
* This a first step towards fixing the conpty narrowing bugs in issue
#2661
* I'm hoping the simplification of `ConGetSet` will also help with
#3849.
* Some of the `TextAttribute` refactoring in this PR overlaps with
similar work in PR #1978.
## Detailed Description of the Pull Request / Additional comments
The main point of this PR was to simplify the
`AdaptDispatch::SetGraphicsRendition` implementation. So instead of
having it call a half a dozen methods in the `ConGetSet` API, depending
on what kinds of attributes needed to be set, there is now just one call
to get current attributes, and another call to set the new value. All
adjustments to the attributes are made in the `AdaptDispatch` class, in
a simple switch statement.
To help with this refactoring, I also made some change to the
`TextAttribute` class to make it easier to work with. This included
adding a set of methods for setting (and getting) the individual
attribute flags, instead of having the calling code being exposed to the
internal attribute structures and messing with bit manipulation. I've
tried to get rid of any methods that were directly setting legacy, meta,
and extended attributes.
Other than the fix to the `SGR 0` bug, the `AdaptDispatch` refactoring
mostly follows the behaviour of the original code. In particular, it
still maps the `SGR 38/48` indexed colors to RGB instead of retaining
the index, which is what we ultimately need it to do. Fixing that will
first require the color tables to be unified (issue #1223), which I'm
hoping to address in a followup PR.
But for now, mapping the indexed colors to RGB values required adding an
an additional `ConGetSet` API to lookup the color table entries. In the
future that won't be necessary, but the API will still be useful for
other color reporting operations that we may want to support. I've made
this API, and the existing setter, standardise on index values being in
the "Xterm" order, since that'll be essential for unifying the code with
the terminal adapter one day.
I should also point out one minor change to the `SGR 38/48` behavior,
which is that out-of-range RGB colors are now ignored rather than being
clamped, since that matches the way Xterm works.
## Validation Steps Performed
This refactoring has obviously required corresponding changes to the
unit tests, but most were just minor updates to use the new
`TextAttribute` methods without any real change in behavior. However,
the adapter tests did require significant changes to accommodate the new
`ConGetSet` API. The basic structure of the tests remain the same, but
the simpler API has meant fewer values needed to be checked in each test
case. I think they are all still covering the areas there were intended
to, though, and they are all still passing.
Other than getting the unit tests to work, I've also done a bunch of
manual testing of my own. I've made sure the color tests in Vttest all
still work as well as they used to. And I've confirmed that the test
case from issue #5341 is now working correctly.
Closes#5341
The Erase All VT sequence (`^[[2J`) is supposed to erase the entire
contents of the viewport. The way it usually does this is by shifting
the entirety of the viewport contents into scrollback, and starting the
new viewport below it.
Currently, conpty doesn't propagate that state change correctly. When
conpty gets a 2J, it simply erases the content of the connected
terminal's viewport, by writing over it with spaces. Conpty didn't
really have a good way of communicating "your viewport should move", it
only knew "the buffer is now full of spaces".
This would lead to bugs like #2832, where pressing <kbd>ctrl+L</kbd> in
`bash` would delete the current contents of the viewport, instead of
moving the viewport down.
This PR makes sure that when conpty sees a 2J, it passes that through
directly to the connected terminal application as well. Fortunately, 2J
was already implemented in the Windows Terminal, so this actually fixes
the behavior of <kbd>ctrl+L</kbd>/`clear` in WSL in the Terminal.
## References
* #4252 - right now this isn't the _most_ optimal scenario, we're
literally just printing a 2J, then we'll perform "erase line" `height`
times. The erase line operations are all redundant at this point - the
entire viewport is blank, but conpty doesn't really know that.
Fortunately, #4252 was already filed for me to come through and
optimize this path.
## PR Checklist
* [x] Closes#2832
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
## Validation Steps Performed
* ran tests
* compared <kbd>ctrl+L</kbd> with its behavior in conhost
* compared `clear` with its behavior in conhost
## Summary of the Pull Request
Adds SS3 cursor encoding for cursor keys and home/end button. Reverts a portion of #4913 that checks for VT Input Mode.
## PR Checklist
* [X] Closes#4873
## Validation Steps Performed
1. Open pwsh
2. run `wsl`
3. execute `printf "\e[?1h"`
4. verify keys work
5. exit back to pwsh
6. verify keys work still (didn't previously)
Also verified that those keys work in vim when connected to my Raspberry Pi over SSH.
This adds support for the VT escape sequence that requests the
terminal's operating status. There is no attempt to actually verify the
status of the app, though. We always return a response indicating a good
operating condition (the same as most terminal emulators).
## PR Checklist
* [x] CLA signed.
* [x] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
This required an update to the `OutputStateMachineEngine` to accept the
`DSR-OS` type, since it only dispatches types that it recognises (I
think that's unnecessary, but that's an issue for another day).
The actual processing of the request is handled in the `AdaptDispatch`
class, where it simply responds with a hard coded sequence (`CSI 0 n`),
indicating a good operating condition.
## Validation Steps Performed
I've added unit tests to confirm that the request is dispatched
correctly, and the appropriate response is returned. I've also manually
confirmed that the test of the _Device Status Report_ in _Vttest_ is now
succeeding.
Closes#5052
## Summary of the Pull Request
If we receive a _Reset to Initial State_ (`RIS`) sequence while in the alternate screen buffer, we should be switching back to the main buffer. This PR fixes that behavior.
## PR Checklist
* [x] Closes#3685
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Detailed Description of the Pull Request / Additional comments
I've added a condition at the start of the `AdaptDispatch::HardReset` method to check whether we're using the alt buffer, and if so, call the `ConGetSet::PrivateUseMainScreenBuffer` API to switch back to the main buffer.
Calling `AdaptDispatch::UseMainScreenBuffer` would probably be neater for this, but it would also attempt to restore the cursor state, which seems pointless when we're in the process of resetting everything anyway.
## Validation Steps Performed
I've added a screen buffer test to confirm that the `RIS` sequence does actually switch back to the main buffer. I've also manually confirmed that the test case in issue #3685 does now behave as expected.
## Summary of the Pull Request
This is essentially a rewrite of the VT tab stop functionality, implemented entirely within the `AdaptDispatch` class. This significantly simplifies the `ConGetSet` interface, and should hopefully make it easier to share the functionality with the Windows Terminal VT implementation in the future.
By removing the dependence on the `SCREEN_INFORMATION` class, it fixes the problem of the the tab state not being preserved when switching between the main and alternate buffers. And the new architecture also fixes problems with the tabs not being correctly initialized when the screen is resized.
## References
This fixes one aspect of issue #3545.
It also supersedes the fix for #411 (PR #2816).
I'm hoping the simplification of `ConGetSet` will help with #3849.
## PR Checklist
* [x] Closes#4669
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Detailed Description of the Pull Request / Additional comments
In the new tab architecture, there is now a `vector<bool>` (__tabStopColumns_), which tracks whether any particular column is a tab stop or not. There is also a __initDefaultTabStops_ flag indicating whether the default tab stop positions need to be initialised when the screen is resized.
The way this works, the vector is initially empty, and only initialized (to the current width of the screen) when it needs to be used. When the vector grows in size, the __initDefaultTabStops_ flag determines whether the new columns are set to false, or if every 8th column is set to true.
By default we want the latter behaviour - newly revealed columns should have default tab stops assigned to them - so __initDefaultTabStops_ is set to true. However, after a `TBC 3` operation (i.e. we've cleared all tab stops), there should be no tab stops in any newly revealed columns, so __initDefaultTabStops_ is set to false.
Note that the __tabStopColumns_ vector is never made smaller when the window is shrunk, and that way it can preserve the state of tab stops that are off screen, but which may come into range if the window is made bigger again.
However, we can can still reset the vector completely after an `RIS` or `TBC 3` operation, since the state can then be reconstructed automatically based on just the __initDefaultTabStops_ flag.
## Validation Steps Performed
The original screen buffer tests had to be rewritten to set and query the tab stop state using escape sequences rather than interacting with the `SCREEN_INFORMATION` class directly, but otherwise the structure of most tests remained largely the same.
However, the alt buffer test was significantly rewritten, since the original behaviour was incorrect, and the initialization test was dropped completely, since it was no longer applicable. The adapter tests were also dropped, since they were testing the `ConGetSet` interface which has now been removed.
I also had to make an addition to the method setup of the screen buffer tests (making sure the viewport was appropriately initialized), since there were some tests (unrelated to tab stops) that were previously dependent on the state being set in the tab initialization test which has now been removed.
I've manually tested the issue described in #4669 and confirmed that the tabs now produce the correct spacing after a resize.
This commit introduces a github action to check our spelling and fixes
the following misspelled words so that we come up green.
It also renames TfEditSes to TfEditSession, because Ses is not a word.
currently, excerpt, fallthrough, identified, occurred, propagate,
provided, rendered, resetting, separate, succeeded, successfully,
terminal, transferred, adheres, breaks, combining, preceded,
architecture, populated, previous, setter, visible, window, within,
appxmanifest, hyphen, control, offset, powerpoint, suppress, parsing,
prioritized, aforementioned, check in, build, filling, indices, layout,
mapping, trying, scroll, terabyte, vetoes, viewport, whose
## Summary of the Pull Request
This _actually_ implements `\033c`
([RIS](https://vt100.net/docs/vt220-rm/chapter4.html)) for the Windows Terminal.
I thought I had done this in #4433, but that PR actually only passthrough'd
`\x1b[3J`. I didn't realize at the time that #2715 was mostly about hard reset,
not erase scrollback.
Not only should conpty pass through RIS, but the Terminal should also be
prepared to actually handle that sequence. So this PR adds that support as well.
## References
* #4433: original PR I thought fixed this.
## PR Checklist
* [x] Closes#2715 for real this time
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
## Validation Steps Performed
Actually tested `printf \033c` in the Terminal this time
When ConPTY encounters a string we don't understand, immediately flush the frame.
## References
This PR supersedes #2665. This solution is much simpler than what was proposed in that PR.
As mentioned in #2665: "This might have some long-term consequences for #1173."
## PR Checklist
* [x] Closes#2011
* [x] Closes#4106
* [x] I work here
* [x] Tests added/passed
This commit enables passthrough mode for VT Input Mode in ConPty. This
will be used to pass VT Input from Mouse Mode directly to the app on the
other side.
## References
#545 - VT Mouse Mode (Terminal)
#376 - VT Mouse Mode (ConPty)
## Detailed Description of the Pull Request / Additional comments
### ConHost
- Set the callback for the InputEngine.
- Retrieve `IsInVirtualTerminalInputMode` from the InputBuffer
### Adapter (Dispatch)
Retrieve `VTInputMode` setting from ConHost
### Parser
- Add a callback to passthrough unknown input sequences directly to the
input queue.
- If we're in VTInputMode, use the callback
## Validation Steps Performed
Tests should still pass.
Generated by https://github.com/jsoref/spelling `f`; to maintain your repo, please consider `fchurn`
I generally try to ignore upstream bits. I've accidentally included some items from the `deps/` directory. I expect someone will give me a list of items to drop, I'm happy to drop whole files/directories, or to split the PR into multiple items (E.g. comments/locals/public).
Closes#4294
## Summary of the Pull Request
Conpty doesn't need `CSI 3 J`, it doesn't have a scrollback. The terminal that's connected should use that. This makes conpty pass it through, like other sequences that conpty has no need for.
## References
## PR Checklist
* [x] Closes#2715
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
## Summary of the Pull Request
This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests.
## PR Checklist
* [x] Closes#3826
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826
## Detailed Description of the Pull Request / Additional comments
The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked,
To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set.
Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line.
This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode.
Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received.
One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied.
## Validation Steps Performed
I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states.
I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
## Summary of the Pull Request
This adds support for the [`DECSCNM`](https://vt100.net/docs/vt510-rm/DECSCNM.html) private mode escape sequence, which toggles the display between normal and reverse screen modes. When reversed, the background and foreground colors are switched. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests.
## References
This also fixes issue #72 for the most part, although if you toggle the mode too fast, there is no discernible flash.
## PR Checklist
* [x] Closes#3773
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Detailed Description of the Pull Request / Additional comments
I've implemented this as a new flag in the `Settings` class, along with updates to the `LookupForegroundColor` and `LookupBackgroundColor` methods, to switch the returned foreground and background colors when that flag is set.
It also required a new private API in the `ConGetSet` interface to toggle the setting. And that API is then called from the `AdaptDispatch` class when the screen mode escape sequence is received.
The last thing needed was to add a step to the `HardReset` method, to reset the mode back to normal, which is one of the `RIS` requirements.
Note that this does currently work in the Windows Terminal, but once #2661 is implemented that may no longer be the case. It might become necessary to let the mode change sequences pass through conpty, and handle the color reversing on the client side.
## Validation Steps Performed
I've added a state machine test to make sure the escape sequence is dispatched correctly, and a screen buffer test to confirm that the mode change does alter the interpretation of colors as expected.
I've also confirmed that the various "light background" tests in Vttest now display correctly, and that the `tput flash` command (in a bash shell) does actually cause the screen to flash.
## Summary of the Pull Request
This PR adds support for the `HPR` and `VPR` escape sequences from the VT510 terminal. `HPR` moves the cursor position forward by a given number of columns, and `VPR` moves the cursor position downward by a given number of rows. They're similar in function to the `CUF` and `CUD` escape sequences, except that they're not constrained by the scrolling margins.
## References
#3628 provided the new `_CursorMovePosition` method that made these operations possible
## PR Checklist
* [x] Closes#3428
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Detailed Description of the Pull Request / Additional comments
Most of the implementation is in the new `_CursorMovePosition` method that was created in PR #3628, so all we're really doing here is hooking up the escape sequences to call that method with the appropriate parameters.
## Validation Steps Performed
I've extended the existing state machine tests for CSI cursor movement to confirm that the `HPR` and `VPR` sequences are dispatched correctly, and also added screen buffer tests to make sure the movement is clamped by the screen boundaries and not the scrolling margins (we don't yet support horizontal margins, but the test is at least in place for when we do eventually add that support).
I've also checked the `HPR` and `VPR` tests in Vttest (under _Test non-VT100 / ISO-6429 cursor-movement_) and confirmed that they are now working as expected.
This commit moves the handling of the `BEL`, `BS`, `TAB`, and `CR`
controls characters into the state machine (when in VT mode), instead of
forwarding them on to the default string writer, which would otherwise
have to parse them out all over again.
This doesn't cover all the control characters, but `ESC`, `SUB`, and
`CAN` are already an integral part of the `StateMachine` itself; `NUL`
is filtered out by the `OutputStateMachineEngine`; and `LF`, `FF`, and
`VT` are due to be implemented as part of PR #3271.
Once all of these controls are handled at the state machine level, we
can strip out all the VT-specific code from the `WriteCharsLegacy`
function, which should simplify it considerably. This would also let us
simplify the `Terminal::_WriteBuffer` implementation, and the planned
replacement stream writer for issue #780.
On the conhost side, the implementation is handled as follows:
* The `BS` control is dispatched to the existing `CursorBackward`
method, with a distance of 1.
* The `TAB` control is dispatched to the existing `ForwardTab` method,
with a tab count of 1.
* The `CR` control required a new dispatch method, but the
implementation was a simple call to the new `_CursorMovePosition` method
from PR #3628.
* The `BEL` control also required a new dispatch method, as well as an
additional private API in the `ConGetSet` interface. But that's mostly
boilerplate code - ultimately it just calls the `SendNotifyBeep` method.
On the Windows Terminal side, not all dispatch methods are implemented.
* There is an existing `CursorBackward` implementation, so `BS` works
OK.
* There isn't a `ForwardTab` implementation, but `TAB` isn't currently
required by the conpty protocol.
* I had to implement the `CarriageReturn` dispatch method, but that was
a simple call to `Terminal::SetCursorPosition`.
* The `WarningBell` method I've left unimplemented, because that
functionality wasn't previously supported anyway, and there's an
existing issue for that (#4046).
## Validation Steps Performed
I've added a state machine test to confirm that the updated control
characters are now forwarded to the appropriate dispatch handlers. But
since the actual implementation is mostly relying on existing
functionality, I'm assuming that code is already adequately tested
elsewhere. That said, I have also run various manual tests of my own,
and confirmed that everything still worked as well as before.
References #3271
References #780
References #3628
References #4046
## Summary of the Pull Request
Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins.
## References
If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428.
## PR Checklist
* [x] Closes#2926
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Detailed Description of the Pull Request / Additional comments
The new [`AdaptDispatch::_CursorMovePosition`](d6c4f35cf6/src/terminal/adapter/adaptDispatch.cpp (L169)) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins.
To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](d6c4f35cf6/src/terminal/adapter/adaptDispatch.hpp (L116-L125)). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand.
While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time.
1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem.
2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges.
Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](9cf7a9b577) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly.
## Validation Steps Performed
Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](49887a3589) as the allowed horizontal extent.
Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](6e80d0de19) altogether.
For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](d6c4f35cf6) to check that those operations now work correctly, both with and without margins.
## Summary of the Pull Request
This adds support for the `FF` (form feed) and `VT` (vertical tab) [control characters](https://vt100.net/docs/vt510-rm/chapter4.html#T4-1), as well as the [`NEL` (Next Line)](https://vt100.net/docs/vt510-rm/NEL.html) and [`IND` (Index)](https://vt100.net/docs/vt510-rm/IND.html) escape sequences.
## References
#976 discusses the conflict between VT100 Index sequence and the VT52 cursor back sequence.
## PR Checklist
* [x] Closes#3189
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3189
## Detailed Description of the Pull Request / Additional comments
I've added a `LineFeed` method to the `ITermDispatch` interface, with an enum parameter specifying the required line feed type (i.e. with carriage return, without carriage return, or dependent on the [`LNM` mode](https://vt100.net/docs/vt510-rm/LNM.html)). The output state machine can then call that method to handle the various line feed control characters (parsed in the `ActionExecute` method), as well the `NEL` and `IND` escape sequences (parsed in the `ActionEscDispatch` method).
The `AdaptDispatch` implementation of `LineFeed` then forwards the call to a new `PrivateLineFeed` method in the `ConGetSet` interface, which simply takes a bool parameter specifying whether a carriage return is required or not. In the case of mode-dependent line feeds, the `AdaptDispatch` implementation determines whether the return is necessary or not, based on the existing _AutoReturnOnNewLine_ setting (which I'm obtaining via another new `PrivateGetLineFeedMode` method).
Ultimately we'll want to support changing the mode via the [`LNM` escape sequence](https://vt100.net/docs/vt510-rm/LNM.html), but there's no urgent need for that now. And using the existing _AutoReturnOnNewLine_ setting as a substitute for the mode gives us backwards compatible behaviour, since that will be true for the Windows shells (which expect a linefeed to also generate a carriage return), and false in a WSL bash shell (which won't want the carriage return by default).
As for the actual `PrivateLineFeed` implementation, that is just a simplified version of how the line feed would previously have been executed in the `WriteCharsLegacy` function. This includes setting the cursor to "On" (with `Cursor::SetIsOn`), potentially clearing the wrap property of the line being left (with `CharRow::SetWrapForced` false), and then setting the new position using `AdjustCursorPosition` with the _fKeepCursorVisible_ parameter set to false.
I'm unsure whether the `SetIsOn` call is really necessary, and I think the way the forced wrap is handled needs a rethink in general, but for now this should at least be compatible with the existing behaviour.
Finally, in order to make this all work in the _Windows Terminal_ app, I also had to add a basic implementation of the `ITermDispatch::LineFeed` method in the `TerminalDispatch` class. There is currently no need to support mode-specific line feeds here, so this simply forwards a `\n` or `\r\n` to the `Execute` method, which is ultimately handled by the `Terminal::_WriteBuffer` implementation.
## Validation Steps Performed
I've added output engine tests which confirm that the various control characters and escape sequences trigger the dispatch method correctly. Then I've added adapter tests which confirm the various dispatch options trigger the `PrivateLineFeed` API correctly. And finally I added some screen buffer tests that check the actual results of the `NEL` and `IND` sequences, which covers both forms of the `PrivateLineFeed` API (i.e. with and without a carriage return).
I've also run the _Test of cursor movements_ in the [Vttest](https://invisible-island.net/vttest/) utility, and confirmed that screens 1, 2, and 5 are now working correctly. The first two depend on `NEL` and `IND` being supported, and screen 5 requires the `VT` control character.
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
- Enables auditing of Virtual Terminal libraries (input, adapter, parser)
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Rolls audit out to more things
* [x] I work here
* [x] Tests should still pass
* [x] Am core contributor
* [x] Closes#3957
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
This is turning on the auditing of these projects (as enabled by the heavier lifting in the other refactor) and then cleaning up the remaining warnings.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
- [x] Built it
- [x] Ran the tests
