This commit approximately doubles the performance of til::enumset
and reduces it's binary footprint by approximately 1kB.
Most of the binary size can be attributed to exception handling.
Unfortunately this commit removes assertions that the given values are less than
the number of bits in the `underlying_type`. However I believe this to be a good
trade-off as the tests previously only happened at runtime, while tests at
compile time would be highly preferable. Such tests are technically possible,
however MSVC fails to compile (valid) `static_assert`s containing
`static_cast`s over a parameter pack at the time of writing.
With future MSVC versions such checks can be added to this class.
This change was initially discussed in #10492, but was forgotten to
be considered before it was merged. Since the work was already done,
this commit re-introduces the optimization. It's free!
## PR Checklist
* [x] I work here.
* [x] Tests added/passed
## Validation Steps Performed
* Run `printf '\e]8;;http://example.com\e\\This is a link\e]8;;\e\\\n'` in WSL
* A wild dotted line appears ✔️
This PR merges the default colors and cursor color into the main color
table, enabling us to simplify the `ConGetSet` and `ITerminalApi`
interfaces, with just two methods required for getting and setting any
form of color palette entry.
The is a follow-up to the color table standardization in #11602, and a
another small step towards de-duplicating `AdaptDispatch` and
`TerminalDispatch` for issue #3849. It should also make it easier to
support color queries (#3718) and a configurable bold color (#5682) in
the future.
On the conhost side, default colors could originally be either indexed
positions in the 16-color table, or separate standalone RGB values. With
the new system, the default colors will always be in the color table, so
we just need to track their index positions.
To make this work, those positions need to be calculated at startup
based on the loaded registry/shortcut settings, and updated when
settings are changed (this is handled in
`CalculateDefaultColorIndices`). But the plus side is that it's now much
easier to lookup the default color values for rendering.
For now the default colors in Windows Terminal use hardcoded positions,
because it doesn't need indexed default colors like conhost. But in the
future I'd like to extend the index handling to both terminals, so we
can eventually support the VT525 indexed color operations.
As for the cursor color, that was previously stored in the `Cursor`
class, which meant that it needed to be copied around in various places
where cursors were being instantiated. Now that it's managed separately
in the color table, a lot of that code is no longer required.
## Validation
Some of the unit test initialization code needed to be updated to setup
the color table and default index values as required for the new system.
There were also some adjustments needed to account for API changes, in
particular for methods that now take index values for the default colors
in place of COLORREFs. But for the most part, the essential behavior of
the tests remains unchanged.
I've also run a variety of manual tests looking at the legacy console
APIs as well as the various VT color sequences, and checking that
everything works as expected when color schemes are changed, both in
Windows Terminal and conhost, and in the latter case with both indexed
colors and RGB values.
Closes#11768
There are some code pages with "unmapped" code points in the C1 range,
which results in them being translated into Unicode C1 control codes,
even though that is not their intended use. To avoid having these
characters triggering unintentional escape sequences, this PR now
disables C1 controls by default.
Switching to ISO-2022 encoding will re-enable them, though, since that
is the most likely scenario in which they would be required. They can
also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape
sequence.
What I've done is add a new mode to the `StateMachine` class that
controls whether C1 code points are interpreted as control characters or
not. When disabled, these code points are simply dropped from the
output, similar to the way a `NUL` is interpreted.
This isn't exactly the way they were handled in the v1 console (which I
think replaces them with the font _notdef_ glyph), but it matches the
XTerm behavior, which seems more appropriate considering this is in VT
mode. And it's worth noting that Windows Explorer seems to work the same
way.
As mentioned above, the mode can be enabled by designating the ISO-2022
coding system with a `DOCS` sequence, and it will be disabled again when
UTF-8 is designated. You can also enable it explicitly with a `DECAC1`
sequence (originally this was actually a DEC printer sequence, but it
doesn't seem unreasonable to use it in a terminal).
I've also extended the operations that save and restore "cursor state"
(e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode,
since it's closely tied to the code page and character sets which are
also saved there. Similarly, when a `DECSTR` sequence resets the code
page and character sets, I've now made it reset the C1 mode as well.
I should note that the new `StateMachine` mode is controlled via a
generic `SetParserMode` method (with a matching API in the `ConGetSet`
interface) to allow for easier addition of other modes in the future.
And I've reimplemented the existing ANSI/VT52 mode in terms of these
generic methods instead of it having to have its own separate APIs.
## Validation Steps Performed
Some of the unit tests for OSC sequences were using a C1 `0x9C` for the
string terminator, which doesn't work by default anymore. Since that's
not a good practice anyway, I thought it best to change those to a
standard 7-bit terminator. However, in tests that were explicitly
validating the C1 controls, I've just enabled the C1 parser mode at the
start of the tests in order to get them working again.
There were also some ANSI mode adapter tests that had to be updated to
account for the fact that it has now been reimplemented in terms of the
`SetParserMode` API.
I've added a new state machine test to validate the changes in behavior
when the C1 parser mode is enabled or disabled. And I've added an
adapter test to verify that the `DesignateCodingSystems` and
`AcceptC1Controls` methods toggle the C1 parser mode as expected.
I've manually verified the test cases in #10069 and #10310 to confirm
that they're no longer triggering control sequences by default.
Although, as I explained above, the C1 code points are completely
dropped from the output rather than displayed as _notdef_ glyphs. I
think this is a reasonable compromise though.
Closes#10069Closes#10310
6140fd9 causes a binary size regression in conhost.
This PR fixes most if not all of the regression, by replacing `FMT_STRING`
with `FMT_COMPILE` allowing us to drop most of the formatters built
into fmt during linking (for instance floating point formatters).
Additionally `std::wstring` was replaced with `fmt::basic_memory_buffer`
in the same vein as was done for VtEngine. Stack is
cheap and this prevents any unnecessary allocations.
## PR Checklist
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
* vttest 11.2.5.3.6.7 and .8 (DECSTBM and SGR) complete successfully ✅
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 063b86ac10af16cade5c0754adcbf27e7e9ae266
Related work items: MSFT-34534216, MSFT-36986009, MSFT-36986203
## Summary of the Pull Request
In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day.
## References
This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728.
## PR Checklist
* [x] Closes#11461
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461
## Detailed Description of the Pull Request / Additional comments
Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code.
And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values.
There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values.
These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame.
The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty.
The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before.
One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive.
The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone.
## Validation Steps Performed
A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate.
In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation.
This includes:
- `WriteConsoleOutput`
- `ReadConsoleOutput`
- `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter`
- `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter`
- `ScrollConsoleScreenBuffer`
- `GetConsoleScreenBufferInfo`
- `GetConsoleScreenBufferInfoEx`
- `SetConsoleScreenBufferInfoEx`
I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell.
In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation.
I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
## Summary of the Pull Request
There is a non-zero subset of applications that randomly output _Locking Shift_ escape sequences which will invoke a character set from G2 or G3 into the left half of the code table. If those G-sets are mapped to Latin1, that can result in the terminal producing output that appears to be broken. This PR now defaults all G-sets to ASCII, to prevent an unintentional _Locking Shift_ from having any effect.
## PR Checklist
* [x] Closes#10408
* [x] CLA signed.
* [ ] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. Issue number where discussion took place: #10408
## Detailed Description of the Pull Request / Additional comments
Most other modern terminals also default to ASCII in all G-sets, so this shouldn't break any modern applications. Legacy 8-bit applications may still expect the G2 and G3 sets mapped to Latin1, but they would also need to have the ISO-2022 encoding enabled, so we can keep them happy by setting G2 and G3 correctly when the ISO-2022 encoding is requested.
## Validation Steps Performed
I've manually confirmed that `echo -e "\en"` and `echo -e "\eo"` no longer have any visible effect on the output (at least without first invoking another character set into G2 or G3). I've also confirmed that they do still work as expected (i.e. selecting Latin1) after enabling the ISO-2022 encoding.
This commit renames `Base64::s_Decode` into `Base64::Decode` and improves its
average performance on short strings of less than 200 characters by 4.5x.
This is achieved by implementing a classic base64 decoder that reads 4
characters at a time and produces 3 output bytes. Furthermore a small
128 byte lookup table is used to quickly map characters to values.
## PR Checklist
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
* Run WSL in Windows Terminal
* Run `printf "\033]52;c;aHR0cHM6Ly9naXRodWIuY29tL21pY3Jvc29mdC90ZXJtaW5hbC9wdWxsLzExNDY3\a"`
* Clipboard contains `https://github.com/microsoft/terminal/pull/11467` ✔️
Instead of having a separate method for setting each mouse and keyboard
mode, this PR consolidates them all into a single method which takes a
mode parameter, and stores the modes in a `til::enumset` rather than
having a separate `bool` for each mode.
This enables us to get rid of a lot of boilerplate code, and makes the
code easier to extend when we want to introduce additional modes in the
future. It'll also makes it easier to read back the state of the various
modes when implementing the `DECRQM` query.
Most of the complication is in the `TerminalInput` class, which had to
be adjusted to work with an `enumset` in place of all the `bool` fields.
For the rest, it was largely a matter of replacing calls to all the old
mode setting methods with the new `SetInputMode` method, and deleting a
bunch of unused code.
One thing worth mentioning is that the `AdaptDispatch` implementation
used to have a `_ShouldPassThroughInputModeChange` method that was
called after every mode change. This code has now been moved up into the
`SetInputMode` implementation in `ConhostInternalGetSet` so it's just
handled in one place. Keeping this out of the dispatch class will also
be beneficial for sharing the implementation with `TerminalDispatch`.
## Validation
The updated interface necessitated some adjustments to the tests in
`AdapterTest` and `MouseInputTest`, but the essential structure of the
tests remains unchanged, and everything still passes.
I've also tested the keyboard and mouse modes in Vttest and confirmed
they still work at least as well as they did before (both conhost and
Windows Terminal), and I tested the alternate scroll mode manually
(conhost only).
Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan
to de-duplicate the `AdaptDispatch` and `TerminalDispatch`
implementation (#3849).
This PR adds support for the `DECRQSS` (Request Selection or Setting)
escape sequence, which is a standard VT query for reporting the state of
various control functions. This initial implementation only supports
queries for the `DECSTBM` margins, and the `SGR` graphic rendition
attributes.
This can be useful for certain forms of capability detection (#1040). As
one example in particular, it can serve as an alternative to the
`COLORTERM` environment variable for detecting truecolor support
(#11057).
Of the settings that can be queried by `DECRQSS`, the only other one
that we could be supporting at the moment is `DECSCUSR` (Cursor Style).
However, that would require passing the query through to the conpty
client, which is a lot more complicated, so I thought it best to leave
for a future PR.
For now this gets the basic framework in place, so we are at least
responding to queries, and even just supporting the `SGR` attributes
query is useful in itself.
Validation
----------
I've added a unit test verifying the reports for the `DECSTBM` and `SGR`
settings with a range of different parameters. I've also tested the
`DECSTBM` and `SGR` reports manually in _Vttest_, under menu 11.2.5.3.6
(Status-String Reports).
## Summary of the Pull Request
This adds a new action, `clearBuffer`. It accepts 3 values for the `clear` type:
* `"clear": "screen"`: Clear the terminal viewport content. Leaves the scrollback untouched. Moves the cursor row to the top of the viewport (unmodified).
* `"clear": "scrollback"`: Clear the scrollback. Leaves the viewport untouched.
* `"clear": "all"`: (**default**) Clear the scrollback and the visible viewport. Moves the cursor row to the top of the viewport (unmodified).
"Clear Buffer" has also been added to `defaults.json`.
## References
* From microsoft/vscode#75141 originally
## PR Checklist
* [x] Closes#1193
* [x] Closes#1882
* [x] I work here
* [x] Tests added/passed
* [ ] Requires documentation to be updated
## Detailed Description of the Pull Request / Additional comments
This is a bit tricky, because we need to plumb it all the way through conpty to clear the buffer. If we don't, then conpty will immediately just redraw the screen. So this sends a signal to the attached conpty, and then waits for conpty to draw the updated, cleared, screen back to us.
## Validation Steps Performed
* works for each of the three clear types as expected
* tests pass.
* works even with `ping -t 8.8.8.8` as you'd hope.
This PR adds conhost support for downloadable soft fonts - also known as
dynamically redefinable character sets (DRCS) - using the `DECDLD`
escape sequence.
These fonts are typically designed to work on a specific terminal model,
and each model tends to have a different character cell size. So in
order to support as many models as possible, the code attempts to detect
the original target size of the font, and then scale the glyphs to fit
our current cell size.
Once a font has been downloaded to the terminal, it can be designated in
the same way you would a standard character set, using an `SCS` escape
sequence. The identification string for the set is defined by the
`DECDLD` sequence. Internally we map the characters in this set to code
points `U+EF20` to `U+EF7F` in the Unicode private use are (PUA).
Then in the renderer, any characters in that range are split off into
separate runs, which get painted with a special font. The font itself is
dynamically generated as an in-memory resource, constructed from the
downloaded character bitmaps which have been scaled to the appropriate
size.
If no soft fonts are in use, then no mapping of the PUA code points will
take place, so this shouldn't interfere with anyone using those code
points for something else, as along as they aren't also trying to use
soft fonts. I also tried to pick a PUA range that hadn't already been
snatched up by Nerd Fonts, but if we do receive reports of a conflict,
it's easy enough to change.
## Validation Steps Performed
I added an adapter test that runs through a bunch of parameter
variations for the `DECDLD` sequence, to make sure we're correctly
detecting the font sizes for most of the known DEC terminal models.
I've also tested manually on a wide range of existing fonts, of varying
dimensions, and from multiple sources, and made sure they all worked
reasonably well.
Closes#9164
This commit is a preparation for upcoming changes to KeyChordSerialization for #7539 and #10203.
In order to support variadic macros, /Zc:preprocessor was enabled, which required changing unrelated parts of the project.
## PR Checklist
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
* Project still compiles ✔️
## Summary of the Pull Request
RIS resets mouse mode and encoding
## PR Checklist
* [x] Closes#8613
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
* [ ] Schema updated.
* [x] I work here
## Validation Steps Performed
Passing structures larger than the register size is very expensive
due to Microsoft's x64 calling convention. We could reduce the
overhead by passing the string-view by reference, but this forces us
to allocate the parameters as static string-views on the data
segment of our binary. I've found that passing them as classic
C-strings is more ergonomic instead and fits the need for
high performance in this particular code.
This improves performance for VT-heavy output by 15-20%.
## PR Checklist
* [x] I work here
* [x] Tests added/passed
Each mouse-down event's time and position is now stored, and if we
process a left-mouse-down event at the same position as the previous one
and within the double click time we set the double click flag.
Also adds a case statement to `_UpdateSGRMouseButtonState` so that we
send hover events instead of ignoring them.
Note: The 'right-click menu in far manager shows up at the wrong
location' bug still exists with this, as it seems to use the cursor
position as told by user32.
Related to #376
## Validation Steps Performed
Double click in far manager works, hover in far manager works (hovering
over items in the right-click menu correctly highlights them)
## Summary of the Pull Request
Upgrade the Windows SDK to 19041 by setting `WindowsTargetPlatformMinVersion` to 17763 and `WindowsTargetPlatformVersion` to 19041.
## PR Checklist
* [x] I work here
* [x] Tests added/passed
## Validation Steps Performed
General usage of the Windows Terminal application appears fine.
Set keyword flags on all events so those sharing a provider with
telemetry do not fire unless tracing is enabled
## PR Checklist
* [x] Closes#10093
* [x] I work here
* [x] Tests passed
* [x] Documentation added in `til.h` about how keywords work and at the
only other site of keywords we define in the Host project tracing
files.
## Detailed Description of the Pull Request / Additional comments
I initially thought that we would need to split providers here to
accomplish this... but @DHowett helped me realize that might be a lot of
additional metadata and bloat binary size. So with help from a friend
from fundamentals, I realized that we could use Keywords to
differentiate here. We can no longer define 0 keywords as that
represents an any/all scenario. Every `TraceLoggingWrite` event now
needs a keyword. When our events have a keyword, they're not included in
any trace. Additionally, when we have an explicit keyword to check that
is different from the ones used for the telemetry pipeline, we can
ensure that we only do "hard work" to generate debug trace data when an
"ALL" type listener like TraceView or Windows Performance Recorder with
our profiles is listening to these providers for ALL keyworded events.
## Validation Steps Performed
- [x] - Built with full release build config to confirm performance is
worse than dev builds BECAUSE of the telemetry event collector camping
our provider and triggering full trace event generation on shared
providers.
- [x] - Built with full release build config to enable statistics
collection and validated trace event collection is excluded and trace
event short-circuits work with this change.
- [x] - Checked that TraceView still sees both telemetry and tracing
events
- [x] - Checked that WPR with our .wprp profile sees both telemetry and
tracing events
## Summary of the Pull Request
When we parse OSC 9;4, allow a trailing semicolon (i.e. allow `9;4;` or something like `9;4;3;`).
## PR Checklist
* [x] Closes#9960
* [X] Tests added/passed
## Validation Steps Performed
OSC 9;4 sequences with or without trailing semicolons work
This PR introduces a mechanism via which DCS data strings can be passed
through directly to the dispatch method that will be handling them, so
the data can be processed as it is received, rather than being buffered
in the state machine. This also simplifies the way string termination is
handled, so it now more closely matches the behaviour of the original
DEC terminals.
* Initial support for DCS sequences was introduced in PR #6328.
* Handling of DCS (and other) C1 controls was added in PR #7340.
* This is a prerequisite for Sixel (#448) and Soft Font (#9164) support.
The way this now works, a `DCS` sequence is dispatched as soon as the
final character of the `VTID` is received. Based on that ID, the
`OutputStateMachineEngine` should forward the call to the corresponding
dispatch method, and its the responsibility of that method to return an
appropriate handler function for the sequence.
From then on, the `StateMachine` will pass on all of the remaining bytes
in the data string to the handler function. When a data string is
terminated (with `CAN`, `SUB`, or `ESC`), the `StateMachine` will pass
on one final `ESC` character to let the handler know that the sequence
is finished. The handler can also end a sequence prematurely by
returning false, and then all remaining data bytes will be ignored.
Note that once a `DCS` sequence has been dispatched, it's not possible
to abort the data string. Both `CAN` and `SUB` are considered valid
forms of termination, and an `ESC` doesn't necessarily have to be
followed by a `\` for the string terminator. This is because the data
string is typically processed as it's received. For example, when
outputting a Sixel image, you wouldn't erase the parts that had already
been displayed if the data string is terminated early.
With this new way of handling the string termination, I was also able to
simplify some of the `StateMachine` processing, and get rid of a few
states that are no longer necessary. These changes don't apply to the
`OSC` sequences, though, since we're more likely to want to match the
XTerm behavior for those cases (which requires a valid `ST` control for
the sequence to be accepted).
## Validation Steps Performed
For the unit tests, I've had to make a few changes to some of the
`OutputEngineTests` to account for the updated `StateMachine`
processing. I've also added a new `StateMachineTest` to confirm that the
data strings are correctly passed through to the string handler under
all forms of termination.
To test whether the framework is actually usable, I've been working on
DRCS Soft Font support branched off of this PR, and haven't encountered
any problems. To test the throughput speed, I also hacked together a
basic Sixel parser, and that seemed to perform reasonably well.
Closes#7316
## Summary of the Pull Request
The "virtual bottom" marks the last line of the mutable viewport area, which is the part of the buffer that VT sequences can write to. This region should typically only move downwards, as new lines are added to the buffer, but there were a number of cases where it was incorrectly being moved up. This PR attempts to fix that.
## PR Checklist
* [x] Closes#9754
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Documentation updated.
* [ ] Schema updated.
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #9754
## Detailed Description of the Pull Request / Additional comments
When a call is made to `UpdateBottom`, we now clamp the value so it's at least as low as the current viewport bottom (i.e. if the viewport has moved down, we want the virtual bottom to move down too), but no lower than the bottom of the buffer (we don't want it to be out of range).
There is one special case where we do actually want the virtual bottom to move up - when the scrollback has been cleared with an `ED3` escape sequence. So in that case we needed a new `ConGetSet` API (`ResetBottom`) to reset the virtual bottom to the top of the buffer (essentially one less than the viewport height, since the virtual bottom points to the last line of the viewport).
## Validation Steps Performed
I had to reset the virtual bottom manually in some parts of the `ScreenBufferTests`, since some of the tests were relying on the virtual bottom being automatically reset when the viewport was reset, which is no longer the case.
I've also added a new test to verify that the virtual bottom doesn't move upwards if an update is triggered while the visible viewport is scrolled up. This essentially reproduces the test case from issue #9754, which I've also manually confirmed is fixed.
This PR removes the `GetConsoleCursorInfo` and `SetConsoleCursorInfo`
methods from the `ConGetSet` interface, and the `GetScrollingRegion`
method from the `SCREEN_INFORMATION` class. None of these methods are
used anymore.
PR #2764 removed the last usage of `GetScrollingRegion`.
The `Get/SetConsoleCursorInfo` methods don't seem to have ever been used
in the time that the terminal code has been open source, so whatever
purpose they originally served must have been replaced a long time ago.
The `GetScrollingRegion` method was originally called from the
`ScrollRegion` function, but that usage was removed in PR #2764 when the
margin handling was moved to a higher level.
I've checked that the code still compiles.
Closes#9771
Change TAEF nuget package to use new Microsoft.Taef name; Update to 10.58 release build version.
## PR Checklist
* [x] Closes email from Phil letting us know TAEF has a new release build and a rename.
* [x] Closes annoying duplicate TAEF import warning in `Parser.UnitTests.vcxproj`
* [x] I work here.
* [ ] Need to see the tests run off CI to confirm this is fine for those environments and Helix
## Validation Steps Performed
* [x] Build/run tests locally
* [ ] Build/run unit and feature tests in CI
* [ ] Build/run Helix-lab tests in CI
This reduces by 10% the binary size of OpenConsole x64 Release.
Note | OpenConsole.exe
------ | ---------------------------
Before | 1156096
After | 1037312
Delta | -118784
%Delta | -10.27%
This PR adds support for the VT line rendition attributes, which allow
for double-width and double-height line renditions. These renditions are
enabled with the `DECDWL` (double-width line) and `DECDHL`
(double-height line) escape sequences. Both reset to the default
rendition with the `DECSWL` (single-width line) escape sequence. For now
this functionality is only supported by the GDI renderer in conhost.
There are a lot of changes, so this is just a general overview of the
main areas affected.
Previously it was safe to assume that the screen had a fixed width, at
least for a given point in time. But now we need to deal with the
possibility of different lines have different widths, so all the
functions that are constrained by the right border (text wrapping,
cursor movement operations, and sequences like `EL` and `ICH`) now need
to lookup the width of the active line in order to behave correctly.
Similarly it used to be safe to assume that buffer and screen
coordinates were the same thing, but that is no longer true. Lots of
places now need to translate back and forth between coordinate systems
dependent on the line rendition. This includes clipboard handling, the
conhost color selection and search, accessibility location tracking and
screen reading, IME editor positioning, "snapping" the viewport, and of
course all the rendering calculations.
For the rendering itself, I've had to introduce a new
`PrepareLineTransform` method that the render engines can use to setup
the necessary transform matrix for a given line rendition. This is also
now used to handle the horizontal viewport offset, since that could no
longer be achieved just by changing the target coordinates (on a double
width line, the viewport offset may be halfway through a character).
I've also had to change the renderer's existing `InvalidateCursor`
method to take a `SMALL_RECT` rather than a `COORD`, to allow for the
cursor being a variable width. Technically this was already a problem,
because the cursor could occupy two screen cells when over a
double-width character, but now it can be anything between one and four
screen cells (e.g. a double-width character on the double-width line).
In terms of architectural changes, there is now a new `lineRendition`
field in the `ROW` class that keeps track of the line rendition for each
row, and several new methods in the `ROW` and `TextBuffer` classes for
manipulating that state. This includes a few helper methods for handling
the various issues discussed above, e.g. position clamping and
translating between coordinate systems.
## Validation Steps Performed
I've manually confirmed all the double-width and double-height tests in
_Vttest_ are now working as expected, and the _VT100 Torture Test_ now
renders correctly (at least the line rendition aspects). I've also got
my own test scripts that check many of the line rendition boundary cases
and have confirmed that those are now passing.
I've manually tested as many areas of the conhost UI that I could think
of, that might be affected by line rendition, including things like
searching, selection, copying, and color highlighting. For
accessibility, I've confirmed that the _Magnifier_ and _Narrator_
correctly handle double-width lines. And I've also tested the Japanese
IME, which while not perfect, is at least useable.
Closes#7865
Implement the `XTPUSHSGR` and `XTPOPSGR` control sequences (see #1796).
This change adds a new pair of methods to `ITermDispatch`:
`PushGraphicsRendition` and `PopGraphicsRendition`, and then plumbs the
change through `AdaptDispatch`, `TerminalDispatch`, `ITerminalApi` and
`TerminalApi`.
The stack logic is encapsulated in the `SgrStack` class, to allow it to
be reused between the two APIs (`AdaptDispatch` and `TerminalDispatch`).
Like xterm, only ten levels of nesting are supported.
The stack is implemented as a "ring stack": if you push when the stack
is full, the bottom of the stack will be dropped to make room.
Partial pushes (see the description of `XTPUSHSGR` in Issue #1796) are
implemented per xterm spec.
## Validation Steps Performed
Tests added, plus manual verification of the feature.
Closes#1796
Dustin L. Howett (3)
* Move CharToKeyEvents (and friends) into InteractivityBase (GH-9106)
* Update Cascadia Code to 2102.03 (GH-9088)
* verison: bump to 1.7 on main
Josh Soref (1)
* ci: update to Spell check to 0.0.17a (CC-9014)
Leonard Hecker (3)
* Fixed GH-5205: Ctrl+Alt+2 doesn't send ^[^@ (CC-5272)
* Fix issues in tests.xml and OpenConsole.psm1 (CC-9011)
* Fix GH-8458: Handle all Ctrl-key combinations (CC-8870)
Mike Griese (1)
* Add support for running a commandline in another WT window (GH-8898)
Michael Niksa (1)
* Teach the renderer to keep thread alive if engine requests it (GH-9091)
Lachlan Picking (1)
* Fix shader time input (CC-8994)
PankajBhojwani (1)
* Separate runtime TerminalSettings from profile-TerminalSettings (CC-8602)
Chester Liu (2)
* Add support for paste filtering and bracketed paste mode (CC-9034)
* Add support for chaining OSC 10-12 (CC-8999)
Related work items: MSFT-31692939
These functions have a dependency on the "VT Redirected" versions of
VkKeyScanW, MapVirtualKeyW and GetKeyState. Those implementations depend
on the service locator and therefore the entire interactivity stack.
This meant that anybody depending on just Types had to pull in **the
entire host** worth of dependencies (!).
Since these functions are only used in places where we have or are
testing interactivity, it makes sense to consolidate them here.
(cherry picked from commit 8f73145d9d)
These functions have a dependency on the "VT Redirected" versions of
VkKeyScanW, MapVirtualKeyW and GetKeyState. Those implementations depend
on the service locator and therefore the entire interactivity stack.
This meant that anybody depending on just Types had to pull in **the
entire host** worth of dependencies (!).
Since these functions are only used in places where we have or are
testing interactivity, it makes sense to consolidate them here.
## Summary of the Pull Request
Fixes#5205, by replacing another use of `MapVirtualKeyW` with `ToUnicodeEx`.
The latter just seems to be much more consistent at translating key combinations in general.
In this particular case though it fixes the issue, because there's no differentiation in `MapVirtualKeyW` for whether it failed to return a character (`'\0'`) or succeeded in turning `^@` into `'\0'`.
`ToUnicodeEx` on the other hand returns the success state separately from the translated character.
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#5205
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #5205
## Detailed Description of the Pull Request / Additional comments
This PR changes the behavior of the `Ctrl+Alt+Key` handling slightly:
⚠️ `ToUnicodeEx` returns unshifted characters. ⚠️
For instance `Ctrl+Alt+a` is now turned into `^[^a`. Due to how ASCII works this is essentially the same though because `'A' & 0b11111` and `'a' & 0b11111` are the same.
## Validation Steps Performed
* Run `showkey -a`
* Ensured `Ctrl+Alt+Space` as well as `Ctrl+Alt+Shift+2` are turned into `^[^@`
* Ensured other, random `Ctrl+Alt+Key` combination behave identical to the current master
This adds the support for chaining OSC 10-12, allowing users to set all
of them at once.
**BREAKING CHANGE**
Before this PR, the OSC 10/11/12 command will only be dispatched iff the
first color is valid. This is no longer true. The new implementation
strictly follows xterm's behavior. Each color is treated independently.
For example, `\e]10;invalid;white\e\\` is effectively `\e]11;white\e\\`.
## Validation Steps Performed
Tests added. Manually tested.
Main OSC color tracking issue: #942
OSC 4 & Initial OSC 10-12 PR: #7578
Closes one item in #942
Pressing Ctrl+\ produces `^\` using the US keyboard layout, thanks to Ctrl-key mappings inside the keyboard layout, whereas some layouts, like the UK extended layout, don't contain those. This causes the character value to be zero and previously caused no VT sequence to be generated under these situations. This PR employs `MapVirtualKeyW` to infer the missing characters.
As a side effect this PR effectively causes _all_ major keys on the keyboard to produce Ctrl+combinations now.
## PR Checklist
* [x] Closes#8458
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests passed
## Validation Steps Performed
Compared all major keys in combination with Ctrl with the app store version of Terminal using `showkey` in WSL. All keys that previously worked still appear to continue to work.
This PR adds support for the `DECID` (Identify Device) escape sequence,
which allows for querying the terminal type in a way that is backwards
compatible with VT52 terminals.
This simply checks for the `ESC Z` sequence in the `ActionEscDispatch`
method of output state machine, and forwards the query to the existing
`DeviceAttributes` dispatch method, since the expected response is
identical to a `DA` report.
## Validation Steps Performed
I've added an output engine test that verifies that the `ESC Z` sequence
is correctly interpreted as a `DA` query when in ANSI mode, and as a
VT52 identification query when in VT52 mode.
Closes#8857
This adds the skeleton code for "bracketed paste mode" to the Windows
Terminal. No actual functionality is implemented yet, just the wiring
for handling DECSET/DECRST 2004.
References #395
Supersedes #7508
The Windows build system doesn't support PMR yet, so we had to add a workaround. :(
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_wdx_dxp_windev 1614b27830a07484c18628ec7d9abc5f9d9b8a0b
Related work items: MSFT-31337042
Fix memory leak that occurs from not dispatching the end of sequences on all actions (since it is buffering up all characters for trace reasons.) Also don't bother storing if no one is listening.
## PR Checklist
- [x] Closes#8283
* [x] Fixes leak found while bumbling around.
* [x] I work here.
## Detailed Description of the Pull Request / Additional comments
- We trace all the things leading up to the Action phase in the VT parser for ETW tracing to make debugging the parser easier, but we made two mistakes.
- At some point, three of the actions (related to print/execute) weren't dispatching the stored up sequence to tracing and not clearing it. So printing/executing in a giant run over and over caused the vector to bloat and bloat and bloat forever.
- We're storing things even when no one is listening. That's a waste.
## Validation Steps Performed
- Watched it grow every time I did `type big.txt` under `taskman.exe`. Then watched it not do that after.
- I did technically WPR it to figure out this was the culprit.
N
* Change backslashes in include statements to forward slashes (CC-8205)
Dustin L. Howett
* Refactor DEC/ANSI modes to avoid duplication when we add SM/RM (GH-8469)
* Fix the xterm and SGR mouse encodings for CTRL, ALT, SHIFT (GH-8379)
James Holderness
* Fix rendering of DBCS characters when partially off screen (CC-8438)
* Correct horizontal coordinates in viewport overflow test (CC-8456)
* Correct paths in the the runut and runft test scripts (CC-8488)
* Retain horizontal viewport offset when moving to bottom (CC-8434)
* Fix out-of-bounds exceptions in Set...{Buffer,Screen}Size (CC-8309)
PankajBhojwani (1)
* Implement ConEmu's OSC 9;4 to set the taskbar progress indicator (CC-8055)
Chester Liu (1)
* Improve OSC 8 Hyperlink parsing logic (CC-7962)
Related work items: #31090459, #31090805, #31090808, #31090810
Retrieved from https://microsoft.visualstudio.com os.2020 OS official/rs_onecore_dep_uxp 91dff13f6953441afe9e28bfe3da9d5bff27bb05
Related: MSFT-26196390
Related work items: #26196390
I was about to add `SetAnsiMode`/`ResetAnsiMode` for `SM` and `RM` when I
realized that we probably don't need yet another enum of mode types, set and
reset functions, and a mode helper for ANSI standard modes when we already have
one for DEC Private modes.
This commit:
1. Changes the enum `PrivateModeParams` to just be `ModeParams`
2. Differentiates ANSI Standard modes (IRM, KAM, SRM, ...) from DEC
Private modes (DECCOLM, DECCKM, ...) using a flag bit set in the enum
value.
3. Introduces a helper class for constructing these values much like
`VTID`. That helper takes a bitmask and applies it to an input to
produce the final enum value.
4. Dispatches all mode set/reset through a common Set/Reset and
`_ModeHelper` that uses the existing enum values.
[1] These modes are in separate namespaces with some overlap. We want to
differentiate them at dispatch time to ensure that `\e[2h` and `\e[?2h` are
given different treatment, and ensure that `\e[1000h` doesn't activate xterm
mouse mode.
Fixes#8457.
We had the xterm and SGR codings for meta/ctrl backwards. Oops.
This commit also fixes an observed issue in Windows Terminal where we
were passing in a console-style modifiers enum when MouseInput is
expecting MK_ constants.
I decided to unify MouseInput around the console-style modifier
constants because they have support for META (which MK_ does not) and
can differentiate between left/right alt/ctrl.
Our tests are fundamentally flawed here: they use a copy of the
modifier key generating logic _themselves_, so we got a bit of "error
carried forward."
I did not fix the tests to use known-good control sequences, I simply
replaced the character generator with another copy of the modifier code.
I did, however, extend them to test ctrl|meta and left/right modifiers.
Fixes#8291
Many include statements use forward slashes, while others use backwards
slashes. This is inconsistent formatting. For this reason, I changed the
backward slashes to forward slashes since that is the standard.
This commit implements the OSC 9;4 sequence per the [ConEmu style].
| sequence | description |
| ------------ | ------------ |
| `ESC ] 9 ; 4 ; st ; pr ST` | Set progress state on taskbar and tab. |
| | When `st` is: |
| | |
| | `0`: remove progress. |
| | `1`: set progress value to `pr` (number, 0-100). |
| | `2`: set the taskbar to the "Error" state |
| | `3`: set the taskbar to the "Indeterminate" state |
| | `4`: set the taskbar to the "Warning" state |
We've also extended this with:
* st 3: set indeterminate state
* st 4: set paused state
We handle multiple tabs sending the sequence by using the the last focused
control's taskbar state/progress.
Upon receiving the sequence in `TerminalApi`, we send an event that gets caught
by `TerminalPage`. `TerminalPage` then fires another event that gets caught by
`AppHost` and that's where we set the taskbar progress.
Closes#3004
[ConEmu style]: https://conemu.github.io/en/AnsiEscapeCodes.html#ConEmu_specific_OSC
