terminal/src/terminal/adapter/adaptDispatch.cpp

2632 lines
98 KiB
C++
Raw Normal View History

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "adaptDispatch.hpp"
#include "conGetSet.hpp"
#include "../../types/inc/Viewport.hpp"
#include "../../types/inc/utils.hpp"
#include "../../inc/unicode.hpp"
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
#include "../parser/ascii.hpp"
using namespace Microsoft::Console::Types;
using namespace Microsoft::Console::VirtualTerminal;
// Routine Description:
// - No Operation helper. It's just here to make sure they're always all the same.
// Arguments:
// - <none>
// Return Value:
// - Always false to signify we didn't handle it.
bool NoOp() noexcept
{
return false;
}
// Note: AdaptDispatch will take ownership of pConApi and pDefaults
AdaptDispatch::AdaptDispatch(std::unique_ptr<ConGetSet> pConApi,
std::unique_ptr<AdaptDefaults> pDefaults) :
_pConApi{ std::move(pConApi) },
_pDefaults{ std::move(pDefaults) },
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
_usingAltBuffer(false),
_isOriginModeRelative(false), // by default, the DECOM origin mode is absolute.
_isDECCOLMAllowed(false), // by default, DECCOLM is not allowed.
_termOutput()
{
THROW_HR_IF_NULL(E_INVALIDARG, _pConApi.get());
THROW_HR_IF_NULL(E_INVALIDARG, _pDefaults.get());
_scrollMargins = { 0 }; // initially, there are no scroll margins.
}
// Routine Description:
// - Translates and displays a single character
// Arguments:
// - wchPrintable - Printable character
// Return Value:
// - <none>
void AdaptDispatch::Print(const wchar_t wchPrintable)
{
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
const auto wchTranslated = _termOutput.TranslateKey(wchPrintable);
// By default the DEL character is meant to be ignored in the same way as a
// NUL character. However, it's possible that it could be translated to a
// printable character in a 96-character set. This condition makes sure that
// a character is only output if the DEL is translated to something else.
if (wchTranslated != AsciiChars::DEL)
{
_pDefaults->Print(wchTranslated);
}
}
// Routine Description
// - Forward an entire string through. May translate, if necessary, to key input sequences
// based on the locale
// Arguments:
// - string - Text to display
// Return Value:
// - <none>
void AdaptDispatch::PrintString(const std::wstring_view string)
{
try
{
if (_termOutput.NeedToTranslate())
{
std::wstring buffer;
buffer.reserve(string.size());
for (auto& wch : string)
{
buffer.push_back(_termOutput.TranslateKey(wch));
}
_pDefaults->PrintString(buffer);
}
else
{
_pDefaults->PrintString(string);
}
}
CATCH_LOG();
}
// Routine Description:
// - CUU - Handles cursor upward movement by given distance.
// CUU and CUD are handled separately from other CUP sequences, because they are
// constrained by the margins.
// See: https://vt100.net/docs/vt510-rm/CUU.html
// "The cursor stops at the top margin. If the cursor is already above the top
// margin, then the cursor stops at the top line."
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CursorUp(const size_t distance)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
return _CursorMovePosition(Offset::Backward(distance), Offset::Unchanged(), true);
}
// Routine Description:
// - CUD - Handles cursor downward movement by given distance
// CUU and CUD are handled separately from other CUP sequences, because they are
// constrained by the margins.
// See: https://vt100.net/docs/vt510-rm/CUD.html
// "The cursor stops at the bottom margin. If the cursor is already above the
// bottom margin, then the cursor stops at the bottom line."
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CursorDown(const size_t distance)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
return _CursorMovePosition(Offset::Forward(distance), Offset::Unchanged(), true);
}
// Routine Description:
// - CUF - Handles cursor forward movement by given distance
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CursorForward(const size_t distance)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
return _CursorMovePosition(Offset::Unchanged(), Offset::Forward(distance), true);
}
// Routine Description:
// - CUB - Handles cursor backward movement by given distance
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CursorBackward(const size_t distance)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
return _CursorMovePosition(Offset::Unchanged(), Offset::Backward(distance), true);
}
// Routine Description:
// - CNL - Handles cursor movement to the following line (or N lines down)
// - Moves to the beginning X/Column position of the line.
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CursorNextLine(const size_t distance)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
return _CursorMovePosition(Offset::Forward(distance), Offset::Absolute(1), true);
}
// Routine Description:
// - CPL - Handles cursor movement to the previous line (or N lines up)
// - Moves to the beginning X/Column position of the line.
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CursorPrevLine(const size_t distance)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
return _CursorMovePosition(Offset::Backward(distance), Offset::Absolute(1), true);
}
// Routine Description:
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
// - Generalizes cursor movement to a specific position, which can be absolute or relative.
// Arguments:
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
// - rowOffset - The row to move to
// - colOffset - The column to move to
// - clampInMargins - Should the position be clamped within the scrolling margins
// Return Value:
// - True if handled successfully. False otherwise.
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
bool AdaptDispatch::_CursorMovePosition(const Offset rowOffset, const Offset colOffset, const bool clampInMargins) const
{
bool success = true;
// First retrieve some information about the buffer
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
// Make sure to reset the viewport (with MoveToBottom )to where it was
// before the user scrolled the console output
success = (_pConApi->MoveToBottom() && _pConApi->GetConsoleScreenBufferInfoEx(csbiex));
if (success)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
// Calculate the viewport boundaries as inclusive values.
// srWindow is exclusive so we need to subtract 1 from the bottom.
const int viewportTop = csbiex.srWindow.Top;
const int viewportBottom = csbiex.srWindow.Bottom - 1;
// Calculate the absolute margins of the scrolling area.
const int topMargin = viewportTop + _scrollMargins.Top;
const int bottomMargin = viewportTop + _scrollMargins.Bottom;
const bool marginsSet = topMargin < bottomMargin;
// For relative movement, the given offsets will be relative to
// the current cursor position.
int row = csbiex.dwCursorPosition.Y;
int col = csbiex.dwCursorPosition.X;
// But if the row is absolute, it will be relative to the top of the
// viewport, or the top margin, depending on the origin mode.
if (rowOffset.IsAbsolute)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
row = _isOriginModeRelative ? topMargin : viewportTop;
}
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
// And if the column is absolute, it'll be relative to column 0.
// Horizontal positions are not affected by the viewport.
if (colOffset.IsAbsolute)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
col = 0;
}
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
// Adjust the base position by the given offsets and clamp the results.
// The row is constrained within the viewport's vertical boundaries,
// while the column is constrained by the buffer width.
row = std::clamp(row + rowOffset.Value, viewportTop, viewportBottom);
col = std::clamp(col + colOffset.Value, 0, csbiex.dwSize.X - 1);
// If the operation needs to be clamped inside the margins, or the origin
// mode is relative (which always requires margin clamping), then the row
// may need to be adjusted further.
if (marginsSet && (clampInMargins || _isOriginModeRelative))
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
// See microsoft/terminal#2929 - If the cursor is _below_ the top
// margin, it should stay below the top margin. If it's _above_ the
// bottom, it should stay above the bottom. Cursor movements that stay
// outside the margins shouldn't necessarily be affected. For example,
// moving up while below the bottom margin shouldn't just jump straight
// to the bottom margin. See
// ScreenBufferTests::CursorUpDownOutsideMargins for a test of that
// behavior.
if (csbiex.dwCursorPosition.Y >= topMargin)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
row = std::max(row, topMargin);
}
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
if (csbiex.dwCursorPosition.Y <= bottomMargin)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
row = std::min(row, bottomMargin);
}
}
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
// Finally, attempt to set the adjusted cursor position back into the console.
const COORD newPos = { gsl::narrow_cast<SHORT>(col), gsl::narrow_cast<SHORT>(row) };
success = _pConApi->SetConsoleCursorPosition(newPos);
}
return success;
}
// Routine Description:
// - CHA - Moves the cursor to an exact X/Column position on the current line.
// Arguments:
// - column - Specific X/Column position to move to
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CursorHorizontalPositionAbsolute(const size_t column)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
return _CursorMovePosition(Offset::Unchanged(), Offset::Absolute(column), false);
}
// Routine Description:
// - VPA - Moves the cursor to an exact Y/row position on the current column.
// Arguments:
// - line - Specific Y/Row position to move to
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::VerticalLinePositionAbsolute(const size_t line)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
return _CursorMovePosition(Offset::Absolute(line), Offset::Unchanged(), false);
}
Add support for the HPR and VPR escape sequences (#4297) ## Summary of the Pull Request This PR adds support for the `HPR` and `VPR` escape sequences from the VT510 terminal. `HPR` moves the cursor position forward by a given number of columns, and `VPR` moves the cursor position downward by a given number of rows. They're similar in function to the `CUF` and `CUD` escape sequences, except that they're not constrained by the scrolling margins. ## References #3628 provided the new `_CursorMovePosition` method that made these operations possible ## PR Checklist * [x] Closes #3428 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments Most of the implementation is in the new `_CursorMovePosition` method that was created in PR #3628, so all we're really doing here is hooking up the escape sequences to call that method with the appropriate parameters. ## Validation Steps Performed I've extended the existing state machine tests for CSI cursor movement to confirm that the `HPR` and `VPR` sequences are dispatched correctly, and also added screen buffer tests to make sure the movement is clamped by the screen boundaries and not the scrolling margins (we don't yet support horizontal margins, but the test is at least in place for when we do eventually add that support). I've also checked the `HPR` and `VPR` tests in Vttest (under _Test non-VT100 / ISO-6429 cursor-movement_) and confirmed that they are now working as expected.
2020-01-21 23:39:15 +01:00
// Routine Description:
// - HPR - Handles cursor forward movement by given distance
// - Unlike CUF, this is not constrained by margin settings.
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::HorizontalPositionRelative(const size_t distance)
{
return _CursorMovePosition(Offset::Unchanged(), Offset::Forward(distance), false);
}
// Routine Description:
// - VPR - Handles cursor downward movement by given distance
// - Unlike CUD, this is not constrained by margin settings.
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::VerticalPositionRelative(const size_t distance)
{
return _CursorMovePosition(Offset::Forward(distance), Offset::Unchanged(), false);
}
// Routine Description:
// - CUP - Moves the cursor to an exact X/Column and Y/Row/Line coordinate position.
// Arguments:
// - line - Specific Y/Row/Line position to move to
// - column - Specific X/Column position to move to
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CursorPosition(const size_t line, const size_t column)
{
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
return _CursorMovePosition(Offset::Absolute(line), Offset::Absolute(column), false);
}
// Routine Description:
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
// - DECSC - Saves the current "cursor state" into a memory buffer. This
// includes the cursor position, origin mode, graphic rendition, and
// active character set.
// Arguments:
// - <none>
// Return Value:
// - True if handled successfully. False otherwise.
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
bool AdaptDispatch::CursorSaveState()
{
// First retrieve some information about the buffer
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
// Make sure to reset the viewport (with MoveToBottom )to where it was
// before the user scrolled the console output
bool success = (_pConApi->MoveToBottom() && _pConApi->GetConsoleScreenBufferInfoEx(csbiex));
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
TextAttribute attributes;
success = success && (_pConApi->PrivateGetTextAttributes(attributes));
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
if (success)
{
// The cursor is given to us by the API as relative to the whole buffer.
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
// But in VT speak, the cursor row should be relative to the current viewport top.
COORD coordCursor = csbiex.dwCursorPosition;
coordCursor.Y -= csbiex.srWindow.Top;
// VT is also 1 based, not 0 based, so correct by 1.
auto& savedCursorState = _savedCursorState.at(_usingAltBuffer);
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
savedCursorState.Column = coordCursor.X + 1;
savedCursorState.Row = coordCursor.Y + 1;
savedCursorState.IsOriginModeRelative = _isOriginModeRelative;
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
savedCursorState.Attributes = attributes;
savedCursorState.TermOutput = _termOutput;
Disable the acceptance of C1 control codes by default (#11690) There are some code pages with "unmapped" code points in the C1 range, which results in them being translated into Unicode C1 control codes, even though that is not their intended use. To avoid having these characters triggering unintentional escape sequences, this PR now disables C1 controls by default. Switching to ISO-2022 encoding will re-enable them, though, since that is the most likely scenario in which they would be required. They can also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape sequence. What I've done is add a new mode to the `StateMachine` class that controls whether C1 code points are interpreted as control characters or not. When disabled, these code points are simply dropped from the output, similar to the way a `NUL` is interpreted. This isn't exactly the way they were handled in the v1 console (which I think replaces them with the font _notdef_ glyph), but it matches the XTerm behavior, which seems more appropriate considering this is in VT mode. And it's worth noting that Windows Explorer seems to work the same way. As mentioned above, the mode can be enabled by designating the ISO-2022 coding system with a `DOCS` sequence, and it will be disabled again when UTF-8 is designated. You can also enable it explicitly with a `DECAC1` sequence (originally this was actually a DEC printer sequence, but it doesn't seem unreasonable to use it in a terminal). I've also extended the operations that save and restore "cursor state" (e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode, since it's closely tied to the code page and character sets which are also saved there. Similarly, when a `DECSTR` sequence resets the code page and character sets, I've now made it reset the C1 mode as well. I should note that the new `StateMachine` mode is controlled via a generic `SetParserMode` method (with a matching API in the `ConGetSet` interface) to allow for easier addition of other modes in the future. And I've reimplemented the existing ANSI/VT52 mode in terms of these generic methods instead of it having to have its own separate APIs. ## Validation Steps Performed Some of the unit tests for OSC sequences were using a C1 `0x9C` for the string terminator, which doesn't work by default anymore. Since that's not a good practice anyway, I thought it best to change those to a standard 7-bit terminator. However, in tests that were explicitly validating the C1 controls, I've just enabled the C1 parser mode at the start of the tests in order to get them working again. There were also some ANSI mode adapter tests that had to be updated to account for the fact that it has now been reimplemented in terms of the `SetParserMode` API. I've added a new state machine test to validate the changes in behavior when the C1 parser mode is enabled or disabled. And I've added an adapter test to verify that the `DesignateCodingSystems` and `AcceptC1Controls` methods toggle the C1 parser mode as expected. I've manually verified the test cases in #10069 and #10310 to confirm that they're no longer triggering control sequences by default. Although, as I explained above, the C1 code points are completely dropped from the output rather than displayed as _notdef_ glyphs. I think this is a reasonable compromise though. Closes #10069 Closes #10310
2021-11-18 00:40:31 +01:00
savedCursorState.C1ControlsAccepted = _pConApi->GetParserMode(StateMachine::Mode::AcceptC1);
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
_pConApi->GetConsoleOutputCP(savedCursorState.CodePage);
}
return success;
}
// Routine Description:
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
// - DECRC - Restores a saved "cursor state" from the DECSC command back into
// the console state. This includes the cursor position, origin mode, graphic
// rendition, and active character set.
// Arguments:
// - <none>
// Return Value:
// - True if handled successfully. False otherwise.
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
bool AdaptDispatch::CursorRestoreState()
{
auto& savedCursorState = _savedCursorState.at(_usingAltBuffer);
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
auto row = savedCursorState.Row;
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
const auto col = savedCursorState.Column;
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
// If the origin mode is relative, and the scrolling region is set (the bottom is non-zero),
// we need to make sure the restored position is clamped within the margins.
if (savedCursorState.IsOriginModeRelative && _scrollMargins.Bottom != 0)
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
{
// VT origin is at 1,1 so we need to add 1 to these margins.
row = std::clamp(row, _scrollMargins.Top + 1u, _scrollMargins.Bottom + 1u);
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
}
// The saved coordinates are always absolute, so we need reset the origin mode temporarily.
_isOriginModeRelative = false;
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
bool success = CursorPosition(row, col);
// Once the cursor position is restored, we can then restore the actual origin mode.
_isOriginModeRelative = savedCursorState.IsOriginModeRelative;
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
// Restore text attributes.
success = (_pConApi->PrivateSetTextAttributes(savedCursorState.Attributes)) && success;
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
// Restore designated character set.
_termOutput = savedCursorState.TermOutput;
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
Disable the acceptance of C1 control codes by default (#11690) There are some code pages with "unmapped" code points in the C1 range, which results in them being translated into Unicode C1 control codes, even though that is not their intended use. To avoid having these characters triggering unintentional escape sequences, this PR now disables C1 controls by default. Switching to ISO-2022 encoding will re-enable them, though, since that is the most likely scenario in which they would be required. They can also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape sequence. What I've done is add a new mode to the `StateMachine` class that controls whether C1 code points are interpreted as control characters or not. When disabled, these code points are simply dropped from the output, similar to the way a `NUL` is interpreted. This isn't exactly the way they were handled in the v1 console (which I think replaces them with the font _notdef_ glyph), but it matches the XTerm behavior, which seems more appropriate considering this is in VT mode. And it's worth noting that Windows Explorer seems to work the same way. As mentioned above, the mode can be enabled by designating the ISO-2022 coding system with a `DOCS` sequence, and it will be disabled again when UTF-8 is designated. You can also enable it explicitly with a `DECAC1` sequence (originally this was actually a DEC printer sequence, but it doesn't seem unreasonable to use it in a terminal). I've also extended the operations that save and restore "cursor state" (e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode, since it's closely tied to the code page and character sets which are also saved there. Similarly, when a `DECSTR` sequence resets the code page and character sets, I've now made it reset the C1 mode as well. I should note that the new `StateMachine` mode is controlled via a generic `SetParserMode` method (with a matching API in the `ConGetSet` interface) to allow for easier addition of other modes in the future. And I've reimplemented the existing ANSI/VT52 mode in terms of these generic methods instead of it having to have its own separate APIs. ## Validation Steps Performed Some of the unit tests for OSC sequences were using a C1 `0x9C` for the string terminator, which doesn't work by default anymore. Since that's not a good practice anyway, I thought it best to change those to a standard 7-bit terminator. However, in tests that were explicitly validating the C1 controls, I've just enabled the C1 parser mode at the start of the tests in order to get them working again. There were also some ANSI mode adapter tests that had to be updated to account for the fact that it has now been reimplemented in terms of the `SetParserMode` API. I've added a new state machine test to validate the changes in behavior when the C1 parser mode is enabled or disabled. And I've added an adapter test to verify that the `DesignateCodingSystems` and `AcceptC1Controls` methods toggle the C1 parser mode as expected. I've manually verified the test cases in #10069 and #10310 to confirm that they're no longer triggering control sequences by default. Although, as I explained above, the C1 code points are completely dropped from the output rather than displayed as _notdef_ glyphs. I think this is a reasonable compromise though. Closes #10069 Closes #10310
2021-11-18 00:40:31 +01:00
// Restore the parsing state of C1 control codes.
AcceptC1Controls(savedCursorState.C1ControlsAccepted);
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
// Restore the code page if it was previously saved.
if (savedCursorState.CodePage != 0)
{
success = _pConApi->SetConsoleOutputCP(savedCursorState.CodePage);
}
return success;
}
// Routine Description:
// - DECTCEM - Sets the show/hide visibility status of the cursor.
// Arguments:
// - fIsVisible - Turns the cursor rendering on (TRUE) or off (FALSE).
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CursorVisibility(const bool fIsVisible)
{
// This uses a private API instead of the public one, because the public API
// will set the cursor shape back to legacy.
return _pConApi->PrivateShowCursor(fIsVisible);
}
// Routine Description:
// - This helper will do the work of performing an insert or delete character operation
// - Both operations are similar in that they cut text and move it left or right in the buffer, padding the leftover area with spaces.
// Arguments:
// - count - The number of characters to insert
// - isInsert - TRUE if insert mode (cut and paste to the right, away from the cursor). FALSE if delete mode (cut and paste to the left, toward the cursor)
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_InsertDeleteHelper(const size_t count, const bool isInsert) const
{
// We'll be doing short math on the distance since all console APIs use shorts. So check that we can successfully convert the uint into a short first.
SHORT distance;
RETURN_BOOL_IF_FALSE(SUCCEEDED(SizeTToShort(count, &distance)));
Correct the boundaries of the scrolling commands (#2505) There are a number of VT escape sequences that rely on the `ScrollRegion` function to scroll the viewport (RI, DL, IL, SU, SD, ICH, and DCH) , and all of them have got the clipping rect or scroll boundaries wrong in some way, resulting in content being scrolled off the screen that should have been clipped, revealed areas not being correctly filled, or parts of the screen not being moved that should have been. This PR attempts to fix all of those issues. The `ScrollRegion` function is what ultimately handles the scrolling, but it's typically called via the `ApiRoutines::ScrollConsoleScreenBufferWImpl` method, and it's the callers of that method that have needed correcting. One "mistake" that many of these operations made, was in setting a clipping rect that was different from the scrolling rect. This should never have been necessary, since the area being scrolled is also the boundary into which the content needs to be clipped, so the easiest thing to do is just use the same rect for both parameters. Another common mistake was in clipping the horizontal boundaries to the width of the viewport. But it's really the buffer width that represents the active width of the screen - the viewport width and offset are merely a window on that active area. As such, the viewport should only be used to clip vertically - the horizontal extent should typically be the full buffer width. On that note, there is really no need to actually calculate the buffer width when we want to set any of the scrolling parameters to that width. The `ScrollRegion` function already takes care of clipping everything within the buffer boundary, so we can simply set the `Left` of the rect to `0` and the `Right` to `SHORT_MAX`. More details on individual commands: * RI (the `DoSrvPrivateReverseLineFeed` function) This now uses a single rect for both the scroll region and clipping boundary, and the width is set to `SHORT_MAX` to cover the full buffer width. Also the bottom of the scrolling region is now the bottom of the viewport (rather than bottom-1), otherwise it would be off by one. * DL and IL (the `DoSrvPrivateModifyLinesImpl` function) Again this uses a single rect for both the scroll region and clipping boundary, and the width is set to `SHORT_MAX` to cover the full width. The most significant change, though, is that the bottom boundary is now the viewport bottom rather than the buffer bottom. Using the buffer bottom prevented it clipping the content that scrolled off screen when inserting, and failed to fill the revealed area when deleting. * SU and SD (the `AdaptDispatch::_ScrollMovement` method) This was already using a single rect for both the scroll region and clipping boundary, but it was previously constrained to the width of the viewport rather than the buffer width, so some areas of the screen weren't correctly scrolled. Also, the bottom boundary was off by 1, because it was using an exclusive rect while the `ScrollRegion` function expects inclusive rects. * ICH and DCH (the `AdaptDispatch::_InsertDeleteHelper` method) This method has been considerably simplified, because it was reimplementing a lot of functionality that was already provided by the `ScrollRegion` function. And like many of the other cases, it has been updated to use a single rect for both the scroll region and clipping boundary, and clip to the full buffer width rather than the viewport width. I should add that if we were following the specs exactly, then the SU and SD commands should technically be panning the viewport over the buffer instead of moving the buffer contents within the viewport boundary. So SU would be the equivalent of a newline at the bottom of the viewport (assuming no margins). And SD would assumedly do the opposite, scrolling the back buffer back into view (an RI at the top of the viewport should do the same). This doesn't seem to be something that is consistently implemented, though. Some terminals do implement SU as a viewport pan, but I haven't seen anyone implement SD or RI as a pan. If we do want to do something about this, I think it's best addressed as a separate issue. ## Validation Steps Performed There were already existing tests for the SU, SD, ICH, and DCH commands, but they were implemented as adapter tests, which weren't effectively testing anything - the `ScrollConsoleScreenBufferW` method used in those tests was just a mock (an incomplete reimplementation of the `ScrollRegion` function), so confirming that the mock produced the correct result told you nothing about the validity of the real code. To address that, I've now reimplemented those adapter tests as screen buffer tests. For the most part I've tried to duplicate the functionality of the original tests, but there are significant differences to account for the fact that scrolling region now covers the full width of the buffer rather than just the viewport width. I've also extended those tests with additional coverage for the RI, DL, and IL commands, which are really just a variation of the SU and SD functionality. Closes #2174
2019-09-11 03:20:46 +02:00
// get current cursor, attributes
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
// Make sure to reset the viewport (with MoveToBottom )to where it was
// before the user scrolled the console output
RETURN_BOOL_IF_FALSE(_pConApi->MoveToBottom());
RETURN_BOOL_IF_FALSE(_pConApi->GetConsoleScreenBufferInfoEx(csbiex));
const auto cursor = csbiex.dwCursorPosition;
Correct the boundaries of the scrolling commands (#2505) There are a number of VT escape sequences that rely on the `ScrollRegion` function to scroll the viewport (RI, DL, IL, SU, SD, ICH, and DCH) , and all of them have got the clipping rect or scroll boundaries wrong in some way, resulting in content being scrolled off the screen that should have been clipped, revealed areas not being correctly filled, or parts of the screen not being moved that should have been. This PR attempts to fix all of those issues. The `ScrollRegion` function is what ultimately handles the scrolling, but it's typically called via the `ApiRoutines::ScrollConsoleScreenBufferWImpl` method, and it's the callers of that method that have needed correcting. One "mistake" that many of these operations made, was in setting a clipping rect that was different from the scrolling rect. This should never have been necessary, since the area being scrolled is also the boundary into which the content needs to be clipped, so the easiest thing to do is just use the same rect for both parameters. Another common mistake was in clipping the horizontal boundaries to the width of the viewport. But it's really the buffer width that represents the active width of the screen - the viewport width and offset are merely a window on that active area. As such, the viewport should only be used to clip vertically - the horizontal extent should typically be the full buffer width. On that note, there is really no need to actually calculate the buffer width when we want to set any of the scrolling parameters to that width. The `ScrollRegion` function already takes care of clipping everything within the buffer boundary, so we can simply set the `Left` of the rect to `0` and the `Right` to `SHORT_MAX`. More details on individual commands: * RI (the `DoSrvPrivateReverseLineFeed` function) This now uses a single rect for both the scroll region and clipping boundary, and the width is set to `SHORT_MAX` to cover the full buffer width. Also the bottom of the scrolling region is now the bottom of the viewport (rather than bottom-1), otherwise it would be off by one. * DL and IL (the `DoSrvPrivateModifyLinesImpl` function) Again this uses a single rect for both the scroll region and clipping boundary, and the width is set to `SHORT_MAX` to cover the full width. The most significant change, though, is that the bottom boundary is now the viewport bottom rather than the buffer bottom. Using the buffer bottom prevented it clipping the content that scrolled off screen when inserting, and failed to fill the revealed area when deleting. * SU and SD (the `AdaptDispatch::_ScrollMovement` method) This was already using a single rect for both the scroll region and clipping boundary, but it was previously constrained to the width of the viewport rather than the buffer width, so some areas of the screen weren't correctly scrolled. Also, the bottom boundary was off by 1, because it was using an exclusive rect while the `ScrollRegion` function expects inclusive rects. * ICH and DCH (the `AdaptDispatch::_InsertDeleteHelper` method) This method has been considerably simplified, because it was reimplementing a lot of functionality that was already provided by the `ScrollRegion` function. And like many of the other cases, it has been updated to use a single rect for both the scroll region and clipping boundary, and clip to the full buffer width rather than the viewport width. I should add that if we were following the specs exactly, then the SU and SD commands should technically be panning the viewport over the buffer instead of moving the buffer contents within the viewport boundary. So SU would be the equivalent of a newline at the bottom of the viewport (assuming no margins). And SD would assumedly do the opposite, scrolling the back buffer back into view (an RI at the top of the viewport should do the same). This doesn't seem to be something that is consistently implemented, though. Some terminals do implement SU as a viewport pan, but I haven't seen anyone implement SD or RI as a pan. If we do want to do something about this, I think it's best addressed as a separate issue. ## Validation Steps Performed There were already existing tests for the SU, SD, ICH, and DCH commands, but they were implemented as adapter tests, which weren't effectively testing anything - the `ScrollConsoleScreenBufferW` method used in those tests was just a mock (an incomplete reimplementation of the `ScrollRegion` function), so confirming that the mock produced the correct result told you nothing about the validity of the real code. To address that, I've now reimplemented those adapter tests as screen buffer tests. For the most part I've tried to duplicate the functionality of the original tests, but there are significant differences to account for the fact that scrolling region now covers the full width of the buffer rather than just the viewport width. I've also extended those tests with additional coverage for the RI, DL, and IL commands, which are really just a variation of the SU and SD functionality. Closes #2174
2019-09-11 03:20:46 +02:00
// Rectangle to cut out of the existing buffer. This is inclusive.
SMALL_RECT srScroll;
srScroll.Left = cursor.X;
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
srScroll.Right = _pConApi->PrivateGetLineWidth(cursor.Y) - 1;
srScroll.Top = cursor.Y;
srScroll.Bottom = srScroll.Top;
// Paste coordinate for cut text above
COORD coordDestination;
coordDestination.Y = cursor.Y;
coordDestination.X = cursor.X;
bool success = false;
if (isInsert)
{
// Insert makes space by moving characters out to the right. So move the destination of the cut/paste region.
success = SUCCEEDED(ShortAdd(coordDestination.X, distance, &coordDestination.X));
}
else
{
Correct the boundaries of the scrolling commands (#2505) There are a number of VT escape sequences that rely on the `ScrollRegion` function to scroll the viewport (RI, DL, IL, SU, SD, ICH, and DCH) , and all of them have got the clipping rect or scroll boundaries wrong in some way, resulting in content being scrolled off the screen that should have been clipped, revealed areas not being correctly filled, or parts of the screen not being moved that should have been. This PR attempts to fix all of those issues. The `ScrollRegion` function is what ultimately handles the scrolling, but it's typically called via the `ApiRoutines::ScrollConsoleScreenBufferWImpl` method, and it's the callers of that method that have needed correcting. One "mistake" that many of these operations made, was in setting a clipping rect that was different from the scrolling rect. This should never have been necessary, since the area being scrolled is also the boundary into which the content needs to be clipped, so the easiest thing to do is just use the same rect for both parameters. Another common mistake was in clipping the horizontal boundaries to the width of the viewport. But it's really the buffer width that represents the active width of the screen - the viewport width and offset are merely a window on that active area. As such, the viewport should only be used to clip vertically - the horizontal extent should typically be the full buffer width. On that note, there is really no need to actually calculate the buffer width when we want to set any of the scrolling parameters to that width. The `ScrollRegion` function already takes care of clipping everything within the buffer boundary, so we can simply set the `Left` of the rect to `0` and the `Right` to `SHORT_MAX`. More details on individual commands: * RI (the `DoSrvPrivateReverseLineFeed` function) This now uses a single rect for both the scroll region and clipping boundary, and the width is set to `SHORT_MAX` to cover the full buffer width. Also the bottom of the scrolling region is now the bottom of the viewport (rather than bottom-1), otherwise it would be off by one. * DL and IL (the `DoSrvPrivateModifyLinesImpl` function) Again this uses a single rect for both the scroll region and clipping boundary, and the width is set to `SHORT_MAX` to cover the full width. The most significant change, though, is that the bottom boundary is now the viewport bottom rather than the buffer bottom. Using the buffer bottom prevented it clipping the content that scrolled off screen when inserting, and failed to fill the revealed area when deleting. * SU and SD (the `AdaptDispatch::_ScrollMovement` method) This was already using a single rect for both the scroll region and clipping boundary, but it was previously constrained to the width of the viewport rather than the buffer width, so some areas of the screen weren't correctly scrolled. Also, the bottom boundary was off by 1, because it was using an exclusive rect while the `ScrollRegion` function expects inclusive rects. * ICH and DCH (the `AdaptDispatch::_InsertDeleteHelper` method) This method has been considerably simplified, because it was reimplementing a lot of functionality that was already provided by the `ScrollRegion` function. And like many of the other cases, it has been updated to use a single rect for both the scroll region and clipping boundary, and clip to the full buffer width rather than the viewport width. I should add that if we were following the specs exactly, then the SU and SD commands should technically be panning the viewport over the buffer instead of moving the buffer contents within the viewport boundary. So SU would be the equivalent of a newline at the bottom of the viewport (assuming no margins). And SD would assumedly do the opposite, scrolling the back buffer back into view (an RI at the top of the viewport should do the same). This doesn't seem to be something that is consistently implemented, though. Some terminals do implement SU as a viewport pan, but I haven't seen anyone implement SD or RI as a pan. If we do want to do something about this, I think it's best addressed as a separate issue. ## Validation Steps Performed There were already existing tests for the SU, SD, ICH, and DCH commands, but they were implemented as adapter tests, which weren't effectively testing anything - the `ScrollConsoleScreenBufferW` method used in those tests was just a mock (an incomplete reimplementation of the `ScrollRegion` function), so confirming that the mock produced the correct result told you nothing about the validity of the real code. To address that, I've now reimplemented those adapter tests as screen buffer tests. For the most part I've tried to duplicate the functionality of the original tests, but there are significant differences to account for the fact that scrolling region now covers the full width of the buffer rather than just the viewport width. I've also extended those tests with additional coverage for the RI, DL, and IL commands, which are really just a variation of the SU and SD functionality. Closes #2174
2019-09-11 03:20:46 +02:00
// Delete scrolls the affected region to the left, relying on the clipping rect to actually delete the characters.
success = SUCCEEDED(ShortSub(coordDestination.X, distance, &coordDestination.X));
}
if (success)
{
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// Note the revealed characters are filled with the standard erase attributes.
success = _pConApi->PrivateScrollRegion(srScroll, srScroll, coordDestination, true);
}
return success;
}
// Routine Description:
// ICH - Insert Character - Blank/default attribute characters will be inserted at the current cursor position.
// - Each inserted character will push all text in the row to the right.
// Arguments:
// - count - The number of characters to insert
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::InsertCharacter(const size_t count)
{
return _InsertDeleteHelper(count, true);
}
// Routine Description:
// DCH - Delete Character - The character at the cursor position will be deleted. Blank/attribute characters will
// be inserted from the right edge of the current line.
// Arguments:
// - count - The number of characters to delete
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::DeleteCharacter(const size_t count)
{
return _InsertDeleteHelper(count, false);
}
// Routine Description:
// - Internal helper to erase one particular line of the buffer. Either from beginning to the cursor, from the cursor to the end, or the entire line.
// - Used by both erase line (used just once) and by erase screen (used in a loop) to erase a portion of the buffer.
// Arguments:
// - csbiex - Pointer to the console screen buffer that we will be erasing (and getting cursor data from within)
// - eraseType - Enumeration mode of which kind of erase to perform: beginning to cursor, cursor to end, or entire line.
// - lineId - The line number (array index value, starts at 0) of the line to operate on within the buffer.
// - This is not aware of circular buffer. Line 0 is always the top visible line if you scrolled the whole way up the window.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_EraseSingleLineHelper(const CONSOLE_SCREEN_BUFFER_INFOEX& csbiex,
const DispatchTypes::EraseType eraseType,
const size_t lineId) const
{
COORD coordStartPosition = { 0 };
if (FAILED(SizeTToShort(lineId, &coordStartPosition.Y)))
{
return false;
}
// determine start position from the erase type
// remember that erases are inclusive of the current cursor position.
switch (eraseType)
{
case DispatchTypes::EraseType::FromBeginning:
case DispatchTypes::EraseType::All:
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
coordStartPosition.X = 0; // from beginning and the whole line start from the left most edge of the buffer.
break;
case DispatchTypes::EraseType::ToEnd:
coordStartPosition.X = csbiex.dwCursorPosition.X; // from the current cursor position (including it)
break;
}
DWORD nLength = 0;
// determine length of erase from erase type
switch (eraseType)
{
case DispatchTypes::EraseType::FromBeginning:
// +1 because if cursor were at the left edge, the length would be 0 and we want to paint at least the 1 character the cursor is on.
nLength = csbiex.dwCursorPosition.X + 1;
break;
case DispatchTypes::EraseType::ToEnd:
case DispatchTypes::EraseType::All:
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// Remember the .X value is 1 farther than the right most column in the buffer. Therefore no +1.
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
nLength = _pConApi->PrivateGetLineWidth(lineId) - coordStartPosition.X;
break;
}
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// Note that the region is filled with the standard erase attributes.
return _pConApi->PrivateFillRegion(coordStartPosition, nLength, L' ', true);
}
// Routine Description:
// - ECH - Erase Characters from the current cursor position, by replacing
// them with a space. This will only erase characters in the current line,
// and won't wrap to the next. The attributes of any erased positions
// receive the currently selected attributes.
// Arguments:
// - numChars - The number of characters to erase.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::EraseCharacters(const size_t numChars)
{
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
bool success = _pConApi->GetConsoleScreenBufferInfoEx(csbiex);
if (success)
{
const COORD startPosition = csbiex.dwCursorPosition;
const SHORT remainingSpaces = csbiex.dwSize.X - startPosition.X;
const size_t actualRemaining = gsl::narrow_cast<size_t>((remainingSpaces < 0) ? 0 : remainingSpaces);
// erase at max the number of characters remaining in the line from the current position.
const auto eraseLength = (numChars <= actualRemaining) ? numChars : actualRemaining;
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// Note that the region is filled with the standard erase attributes.
success = _pConApi->PrivateFillRegion(startPosition, eraseLength, L' ', true);
}
return success;
}
// Routine Description:
// - ED - Erases a portion of the current viewable area (viewport) of the console.
// Arguments:
// - eraseType - Determines whether to erase:
// From beginning (top-left corner) to the cursor
// From cursor to end (bottom-right corner)
// The entire viewport area
// The scrollback (outside the viewport area)
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::EraseInDisplay(const DispatchTypes::EraseType eraseType)
{
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
RETURN_BOOL_IF_FALSE(eraseType <= DispatchTypes::EraseType::Scrollback);
// First things first. If this is a "Scrollback" clear, then just do that.
// Scrollback clears erase everything in the "scrollback" of a *nix terminal
// Everything that's scrolled off the screen so far.
// Or if it's an Erase All, then we also need to handle that specially
// by moving the current contents of the viewport into the scrollback.
if (eraseType == DispatchTypes::EraseType::Scrollback)
{
const bool eraseScrollbackResult = _EraseScrollback();
// GH#2715 - If this succeeded, but we're in a conpty, return `false` to
// make the state machine propagate this ED sequence to the connected
// terminal application. While we're in conpty mode, we don't really
// have a scrollback, but the attached terminal might.
const bool isPty = _pConApi->IsConsolePty();
return eraseScrollbackResult && (!isPty);
}
else if (eraseType == DispatchTypes::EraseType::All)
{
Make sure that EraseAll moves the Terminal viewport (#5683) The Erase All VT sequence (`^[[2J`) is supposed to erase the entire contents of the viewport. The way it usually does this is by shifting the entirety of the viewport contents into scrollback, and starting the new viewport below it. Currently, conpty doesn't propagate that state change correctly. When conpty gets a 2J, it simply erases the content of the connected terminal's viewport, by writing over it with spaces. Conpty didn't really have a good way of communicating "your viewport should move", it only knew "the buffer is now full of spaces". This would lead to bugs like #2832, where pressing <kbd>ctrl+L</kbd> in `bash` would delete the current contents of the viewport, instead of moving the viewport down. This PR makes sure that when conpty sees a 2J, it passes that through directly to the connected terminal application as well. Fortunately, 2J was already implemented in the Windows Terminal, so this actually fixes the behavior of <kbd>ctrl+L</kbd>/`clear` in WSL in the Terminal. ## References * #4252 - right now this isn't the _most_ optimal scenario, we're literally just printing a 2J, then we'll perform "erase line" `height` times. The erase line operations are all redundant at this point - the entire viewport is blank, but conpty doesn't really know that. Fortunately, #4252 was already filed for me to come through and optimize this path. ## PR Checklist * [x] Closes #2832 * [x] I work here * [x] Tests added/passed * [n/a] Requires documentation to be updated ## Validation Steps Performed * ran tests * compared <kbd>ctrl+L</kbd> with its behavior in conhost * compared `clear` with its behavior in conhost
2020-05-05 03:36:30 +02:00
// GH#5683 - If this succeeded, but we're in a conpty, return `false` to
// make the state machine propagate this ED sequence to the connected
// terminal application. While we're in conpty mode, when the client
// requests a Erase All operation, we need to manually tell the
// connected terminal to do the same thing, so that the terminal will
// move it's own buffer contents into the scrollback.
const bool eraseAllResult = _EraseAll();
const bool isPty = _pConApi->IsConsolePty();
return eraseAllResult && (!isPty);
}
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
// Make sure to reset the viewport (with MoveToBottom )to where it was
// before the user scrolled the console output
bool success = (_pConApi->MoveToBottom() && _pConApi->GetConsoleScreenBufferInfoEx(csbiex));
if (success)
{
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
// When erasing the display, every line that is erased in full should be
// reset to single width. When erasing to the end, this could include
// the current line, if the cursor is in the first column. When erasing
// from the beginning, though, the current line would never be included,
// because the cursor could never be in the rightmost column (assuming
// the line is double width).
if (eraseType == DispatchTypes::EraseType::FromBeginning)
{
const auto endRow = csbiex.dwCursorPosition.Y;
_pConApi->PrivateResetLineRenditionRange(csbiex.srWindow.Top, endRow);
}
if (eraseType == DispatchTypes::EraseType::ToEnd)
{
const auto startRow = csbiex.dwCursorPosition.Y + (csbiex.dwCursorPosition.X > 0 ? 1 : 0);
_pConApi->PrivateResetLineRenditionRange(startRow, csbiex.srWindow.Bottom);
}
// What we need to erase is grouped into 3 types:
// 1. Lines before cursor
// 2. Cursor Line
// 3. Lines after cursor
// We erase one or more of these based on the erase type:
// A. FromBeginning - Erase 1 and Some of 2.
// B. ToEnd - Erase some of 2 and 3.
// C. All - Erase 1, 2, and 3.
// 1. Lines before cursor line
if (eraseType == DispatchTypes::EraseType::FromBeginning)
{
// For beginning and all, erase all complete lines before (above vertically) from the cursor position.
for (SHORT startLine = csbiex.srWindow.Top; startLine < csbiex.dwCursorPosition.Y; startLine++)
{
success = _EraseSingleLineHelper(csbiex, DispatchTypes::EraseType::All, startLine);
if (!success)
{
break;
}
}
}
if (success)
{
// 2. Cursor Line
success = _EraseSingleLineHelper(csbiex, eraseType, csbiex.dwCursorPosition.Y);
}
if (success)
{
// 3. Lines after cursor line
if (eraseType == DispatchTypes::EraseType::ToEnd)
{
// For beginning and all, erase all complete lines after (below vertically) the cursor position.
// Remember that the viewport bottom value is 1 beyond the viewable area of the viewport.
for (SHORT startLine = csbiex.dwCursorPosition.Y + 1; startLine < csbiex.srWindow.Bottom; startLine++)
{
success = _EraseSingleLineHelper(csbiex, DispatchTypes::EraseType::All, startLine);
if (!success)
{
break;
}
}
}
}
}
return success;
}
// Routine Description:
// - EL - Erases the line that the cursor is currently on.
// Arguments:
// - eraseType - Determines whether to erase: From beginning (left edge) to the cursor, from cursor to end (right edge), or the entire line.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::EraseInLine(const DispatchTypes::EraseType eraseType)
{
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
RETURN_BOOL_IF_FALSE(eraseType <= DispatchTypes::EraseType::All);
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
bool success = _pConApi->GetConsoleScreenBufferInfoEx(csbiex);
if (success)
{
success = _EraseSingleLineHelper(csbiex, eraseType, csbiex.dwCursorPosition.Y);
}
return success;
}
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
// Routine Description:
// - DECSWL/DECDWL/DECDHL - Sets the line rendition attribute for the current line.
// Arguments:
// - rendition - Determines whether the line will be rendered as single width, double
// width, or as one half of a double height line.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetLineRendition(const LineRendition rendition)
{
return _pConApi->PrivateSetCurrentLineRendition(rendition);
}
// Routine Description:
// - DSR - Reports status of a console property back to the STDIN based on the type of status requested.
// - This particular routine responds to ANSI status patterns only (CSI # n), not the DEC format (CSI ? # n)
// Arguments:
// - statusType - ANSI status type indicating what property we should report back
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::DeviceStatusReport(const DispatchTypes::AnsiStatusType statusType)
{
bool success = false;
switch (statusType)
{
case DispatchTypes::AnsiStatusType::OS_OperatingStatus:
success = _OperatingStatus();
break;
case DispatchTypes::AnsiStatusType::CPR_CursorPositionReport:
success = _CursorPositionReport();
break;
}
return success;
}
// Routine Description:
// - DA - Reports the identity of this Virtual Terminal machine to the caller.
// - In our case, we'll report back to acknowledge we understand, but reveal no "hardware" upgrades like physical terminals of old.
// Arguments:
// - <none>
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::DeviceAttributes()
{
// See: http://vt100.net/docs/vt100-ug/chapter3.html#DA
return _WriteResponse(L"\x1b[?1;0c");
}
Add support for DA2 and DA3 device attributes reports (#6850) This PR adds support for the `DA2` (Secondary Device Attributes) and `DA3` (Tertiary Device Attributes) escape sequences, which are standard VT queries reporting basic information about the terminal. The _Secondary Device Attributes_ response is made up of a number of parameters: 1. An identification code, for which I've used 0 to indicate that we have the capabilities of a VT100 (using code 0 for this is an XTerm convention, since technically DA2 would not have been supported by a VT100). 2. A firmware revision level, which some terminal emulators use to report their actual version number, but I thought it best we just hardcode a value of 10 (the DEC convention for 1.0). 3. Additional hardware options, which tend to be device specific, but I've followed the convention of the later DEC terminals using 1 to indicate the presence of a PC keyboard. The _Tertiary Device Attributes_ response was originally used to provide a unique terminal identification code, and which some terminal emulators use as a way to identify themselves. However, I think that's information we'd probably prefer not to reveal, so I've followed the more common practice of returning all zeros for the ID. In terms of implementation, the only complication was the need to add an additional code path in the `OutputStateMachine` to handle the `>` and `=` intermediates (technically private parameter prefixes) that these sequences require. I've done this as a single method - rather than one for each prefix - since I think that makes the code easier to follow. VALIDATION ---------- I've added output engine tests to make sure the sequences are dispatched correctly, and adapter tests to confirm that they are returning the responses we expect. I've also manually confirmed that they pass the _Test of terminal reports_ in Vttest. Closes #5836
2020-07-11 00:27:47 +02:00
// Routine Description:
// - DA2 - Reports the terminal type, firmware version, and hardware options.
// For now we're following the XTerm practice of using 0 to represent a VT100
// terminal, the version is hardcoded as 10 (1.0), and the hardware option
// is set to 1 (indicating a PC Keyboard).
// Arguments:
// - <none>
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SecondaryDeviceAttributes()
{
return _WriteResponse(L"\x1b[>0;10;1c");
}
// Routine Description:
// - DA3 - Reports the terminal unit identification code. Terminal emulators
// typically return a hardcoded value, the most common being all zeros.
// Arguments:
// - <none>
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::TertiaryDeviceAttributes()
{
return _WriteResponse(L"\x1bP!|00000000\x1b\\");
}
Add support for VT52 emulation (#4789) ## Summary of the Pull Request This PR adds support for the core VT52 commands, and implements the `DECANM` private mode sequence, which switches the terminal between ANSI mode and VT52-compatible mode. ## References PR #2017 defined the initial specification for VT52 support. PR #4044 removed the original VT52 cursor ops that conflicted with VT100 sequences. ## PR Checklist * [x] Closes #976 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #2017 ## Detailed Description of the Pull Request / Additional comments Most of the work involves updates to the parsing state machine, which behaves differently in VT52 mode. `CSI`, `OSC`, and `SS3` sequences are not applicable, and there is one special-case escape sequence (_Direct Cursor Address_), which requires an additional state to handle parameters that come _after_ the final character. Once the parsing is handled though, it's mostly just a matter of dispatching the commands to existing methods in the `ITermDispatch` interface. Only one new method was required in the interface to handle the _Identify_ command. The only real new functionality is in the `TerminalInput` class, which needs to generate different escape sequences for certain keys in VT52 mode. This does not yet support _all_ of the VT52 key sequences, because the VT100 support is itself not yet complete. But the basics are in place, and I think the rest is best left for a follow-up issue, and potentially a refactor of the `TerminalInput` class. I should point out that the original spec called for a new _Graphic Mode_ character set, but I've since discovered that the VT terminals that _emulate_ VT52 just use the existing VT100 _Special Graphics_ set, so that is really what we should be doing too. We can always consider adding the VT52 graphic set as a option later, if there is demand for strict VT52 compatibility. ## Validation Steps Performed I've added state machine and adapter tests to confirm that the `DECANM` mode changing sequences are correctly dispatched and forwarded to the `ConGetSet` handler. I've also added state machine tests that confirm the VT52 escape sequences are dispatched correctly when the ANSI mode is reset. For fuzzing support, I've extended the VT command fuzzer to generate the different kinds of VT52 sequences, as well as mode change sequences to switch between the ANSI and VT52 modes. In terms of manual testing, I've confirmed that the _Test of VT52 mode_ in Vttest now works as expected.
2020-06-01 23:20:40 +02:00
// Routine Description:
// - VT52 Identify - Reports the identity of the terminal in VT52 emulation mode.
// An actual VT52 terminal would typically identify itself with ESC / K.
// But for a terminal that is emulating a VT52, the sequence should be ESC / Z.
// Arguments:
// - <none>
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::Vt52DeviceAttributes()
{
return _WriteResponse(L"\x1b/Z");
}
Add support for the DECREQTPARM report (#7939) This PR adds support for the `DECREQTPARM` (Request Terminal Parameters) escape sequence, which was originally used on the VT100 terminal to report the serial communication parameters. Modern terminal emulators simply hardcode the reported values for backward compatibility. The `DECREQTPARM` sequence has one parameter, which was originally used to tell the terminal whether it was permitted to send unsolicited reports or not. However, since we have no reason to send an unsolicited report, we don't need to keep track of that state, but the permission parameter does still determine the value of the first parameter in the response. The response parameters are as follows: | Parameter | Value | Meaning | | ---------------- | ------ | ------------------------ | | response type | 2 or 3 | unsolicited or solicited | | parity | 1 | no parity | | data bits | 1 | 8 bits per character | | transmit speed | 128 | 38400 baud | | receive speed | 128 | 38400 baud | | clock multiplier | 1 | | | flags | 0 | | There is some variation in the baud rate reported by modern terminal emulators, and 9600 baud seems to be a little more common than 38400 baud, but I thought the higher speed was probably more appropriate, especially since that's also the value reported by XTerm. ## Validation Steps Performed I've added a couple of adapter and output engine tests to verify that the sequence is dispatched correctly, and the expected responses are generated. I've also manually tested in Vttest and confirmed that we now pass the `DECREQTPARM` test in the _Test of terminal reports_. Closes #7852
2020-10-16 00:50:02 +02:00
// Routine Description:
// - DECREQTPARM - This sequence was originally used on the VT100 terminal to
// report the serial communication parameters (baud rate, data bits, parity,
// etc.). On modern terminal emulators, the response is simply hardcoded.
// Arguments:
// - permission - This would originally have determined whether the terminal
// was allowed to send unsolicited reports or not.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::RequestTerminalParameters(const DispatchTypes::ReportingPermission permission)
{
// We don't care whether unsolicited reports are allowed or not, but the
// requested permission does determine the value of the first response
// parameter. The remaining parameters are just hardcoded to indicate a
// 38400 baud connection, which matches the XTerm response. The full
// parameter sequence is as follows:
// - response type: 2 or 3 (unsolicited or solicited)
// - parity: 1 (no parity)
// - data bits: 1 (8 bits per character)
// - transmit speed: 128 (38400 baud)
// - receive speed: 128 (38400 baud)
// - clock multiplier: 1
// - flags: 0
switch (permission)
{
case DispatchTypes::ReportingPermission::Unsolicited:
return _WriteResponse(L"\x1b[2;1;1;128;128;1;0x");
case DispatchTypes::ReportingPermission::Solicited:
return _WriteResponse(L"\x1b[3;1;1;128;128;1;0x");
default:
return false;
}
}
// Routine Description:
// - DSR-OS - Reports the operating status back to the input channel
// Arguments:
// - <none>
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_OperatingStatus() const
{
// We always report a good operating condition.
return _WriteResponse(L"\x1b[0n");
}
// Routine Description:
// - DSR-CPR - Reports the current cursor position within the viewport back to the input channel
// Arguments:
// - <none>
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_CursorPositionReport() const
{
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
// Make sure to reset the viewport (with MoveToBottom )to where it was
// before the user scrolled the console output
bool success = (_pConApi->MoveToBottom() && _pConApi->GetConsoleScreenBufferInfoEx(csbiex));
if (success)
{
// First pull the cursor position relative to the entire buffer out of the console.
COORD coordCursorPos = csbiex.dwCursorPosition;
Improve the VT cursor movement implementation (#3628) ## Summary of the Pull Request Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins. ## References If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428. ## PR Checklist * [x] Closes #2926 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments The new [`AdaptDispatch::_CursorMovePosition`](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.cpp#L169) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins. To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](https://github.com/microsoft/terminal/blob/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884/src/terminal/adapter/adaptDispatch.hpp#L116-L125). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand. While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time. 1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem. 2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges. Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](https://github.com/microsoft/terminal/pull/3628/commits/9cf7a9b577ed7831908bb9353d4f8e0a6e6fcc5e) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly. ## Validation Steps Performed Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](https://github.com/microsoft/terminal/pull/3628/commits/49887a3589169b2724f4046c1773836384543c10) as the allowed horizontal extent. Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](https://github.com/microsoft/terminal/pull/3628/commits/6e80d0de19cb3313cfdd4fea555f6be41cc9fcd8) altogether. For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](https://github.com/microsoft/terminal/pull/3628/commits/d6c4f35cf60a239cb1b8b7b7cc5796b06f78a884) to check that those operations now work correctly, both with and without margins.
2020-01-16 23:33:35 +01:00
// Now adjust it for its position in respect to the current viewport top.
coordCursorPos.Y -= csbiex.srWindow.Top;
// NOTE: 1,1 is the top-left corner of the viewport in VT-speak, so add 1.
coordCursorPos.X++;
coordCursorPos.Y++;
// If the origin mode is relative, line numbers start at top margin of the scrolling region.
if (_isOriginModeRelative)
{
coordCursorPos.Y -= _scrollMargins.Top;
}
// Now send it back into the input channel of the console.
// First format the response string.
const auto response = wil::str_printf<std::wstring>(L"\x1b[%d;%dR", coordCursorPos.Y, coordCursorPos.X);
success = _WriteResponse(response);
}
return success;
}
// Routine Description:
// - Helper to send a string reply to the input stream of the console.
// - Used by various commands where the program attached would like a reply to one of the commands issued.
// - This will generate two "key presses" (one down, one up) for every character in the string and place them into the head of the console's input stream.
// Arguments:
// - reply - The reply string to transmit back to the input stream
// Return Value:
// - True if the string was converted to input events and placed into the console input buffer successfully. False otherwise.
bool AdaptDispatch::_WriteResponse(const std::wstring_view reply) const
{
bool success = false;
std::deque<std::unique_ptr<IInputEvent>> inEvents;
try
{
// generate a paired key down and key up event for every
// character to be sent into the console's input buffer
for (const auto& wch : reply)
{
// This wasn't from a real keyboard, so we're leaving key/scan codes blank.
KeyEvent keyEvent{ TRUE, 1, 0, 0, wch, 0 };
inEvents.push_back(std::make_unique<KeyEvent>(keyEvent));
keyEvent.SetKeyDown(false);
inEvents.push_back(std::make_unique<KeyEvent>(keyEvent));
}
}
catch (...)
{
LOG_HR(wil::ResultFromCaughtException());
return false;
}
size_t eventsWritten;
Switch all DSR responses to appending instead of prepending (#7583) This fixes an issue where two CPRs could end up corrupted in the input buffer. An application that sent two CPRs back-to-back could end up reading the first few characters of the first prepended CPR before handing us another CPR. We would dutifully prepend it to the buffer, causing them to overlap. ``` ^[^[2;2R[1;1R ^^ ^^^^^ First CPR ^^^^^^ Second CPR ``` The end result of this corruption is that a requesting application would receive an unbidden `R` on stdin; for vim, this would trigger replace mode immediately on startup. Response prepending was implemented in !997738 without much comment. There's very little in the way of audit trail as to why we switched. Michael believes that we wanted to make sure that applications got DSR responses immediately. It had the unfortunate side effect of causing subsequence CPRs across cursor moves to come out in the wrong order. I discussed our options with him, and he suggested that we could implement a priority queue in InputBuffer and make sure that "response" input was dispatched to a client application before any application- or user-generated input. This was deemed to be too much work. We decided that DSR responses getting top billing was likely to be a stronger guarantee than most terminals are capable of giving, and that we should be fine if we just switch it back to append. Thanks to @k-takata, @tekki and @brammool for the investigation on the vim side. Fixes #1637.
2020-09-10 01:55:22 +02:00
// TODO GH#4954 During the input refactor we may want to add a "priority" input list
// to make sure that "response" input is spooled directly into the application.
// We switched this to an append (vs. a prepend) to fix GH#1637, a bug where two CPR
// could collide with eachother.
success = _pConApi->PrivateWriteConsoleInputW(inEvents, eventsWritten);
return success;
}
// Routine Description:
// - Generalizes scrolling movement for up/down
// Arguments:
// - scrollDirection - Specific direction to move
// - distance - Magnitude of the move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_ScrollMovement(const ScrollDirection scrollDirection, const size_t distance) const
{
// We'll be doing short math on the distance since all console APIs use shorts. So check that we can successfully convert the size_t into a short first.
SHORT dist;
bool success = SUCCEEDED(SizeTToShort(distance, &dist));
if (success)
{
// get current cursor
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
// Make sure to reset the viewport (with MoveToBottom )to where it was
// before the user scrolled the console output
success = (_pConApi->MoveToBottom() && _pConApi->GetConsoleScreenBufferInfoEx(csbiex));
if (success)
{
Correct the boundaries of the scrolling commands (#2505) There are a number of VT escape sequences that rely on the `ScrollRegion` function to scroll the viewport (RI, DL, IL, SU, SD, ICH, and DCH) , and all of them have got the clipping rect or scroll boundaries wrong in some way, resulting in content being scrolled off the screen that should have been clipped, revealed areas not being correctly filled, or parts of the screen not being moved that should have been. This PR attempts to fix all of those issues. The `ScrollRegion` function is what ultimately handles the scrolling, but it's typically called via the `ApiRoutines::ScrollConsoleScreenBufferWImpl` method, and it's the callers of that method that have needed correcting. One "mistake" that many of these operations made, was in setting a clipping rect that was different from the scrolling rect. This should never have been necessary, since the area being scrolled is also the boundary into which the content needs to be clipped, so the easiest thing to do is just use the same rect for both parameters. Another common mistake was in clipping the horizontal boundaries to the width of the viewport. But it's really the buffer width that represents the active width of the screen - the viewport width and offset are merely a window on that active area. As such, the viewport should only be used to clip vertically - the horizontal extent should typically be the full buffer width. On that note, there is really no need to actually calculate the buffer width when we want to set any of the scrolling parameters to that width. The `ScrollRegion` function already takes care of clipping everything within the buffer boundary, so we can simply set the `Left` of the rect to `0` and the `Right` to `SHORT_MAX`. More details on individual commands: * RI (the `DoSrvPrivateReverseLineFeed` function) This now uses a single rect for both the scroll region and clipping boundary, and the width is set to `SHORT_MAX` to cover the full buffer width. Also the bottom of the scrolling region is now the bottom of the viewport (rather than bottom-1), otherwise it would be off by one. * DL and IL (the `DoSrvPrivateModifyLinesImpl` function) Again this uses a single rect for both the scroll region and clipping boundary, and the width is set to `SHORT_MAX` to cover the full width. The most significant change, though, is that the bottom boundary is now the viewport bottom rather than the buffer bottom. Using the buffer bottom prevented it clipping the content that scrolled off screen when inserting, and failed to fill the revealed area when deleting. * SU and SD (the `AdaptDispatch::_ScrollMovement` method) This was already using a single rect for both the scroll region and clipping boundary, but it was previously constrained to the width of the viewport rather than the buffer width, so some areas of the screen weren't correctly scrolled. Also, the bottom boundary was off by 1, because it was using an exclusive rect while the `ScrollRegion` function expects inclusive rects. * ICH and DCH (the `AdaptDispatch::_InsertDeleteHelper` method) This method has been considerably simplified, because it was reimplementing a lot of functionality that was already provided by the `ScrollRegion` function. And like many of the other cases, it has been updated to use a single rect for both the scroll region and clipping boundary, and clip to the full buffer width rather than the viewport width. I should add that if we were following the specs exactly, then the SU and SD commands should technically be panning the viewport over the buffer instead of moving the buffer contents within the viewport boundary. So SU would be the equivalent of a newline at the bottom of the viewport (assuming no margins). And SD would assumedly do the opposite, scrolling the back buffer back into view (an RI at the top of the viewport should do the same). This doesn't seem to be something that is consistently implemented, though. Some terminals do implement SU as a viewport pan, but I haven't seen anyone implement SD or RI as a pan. If we do want to do something about this, I think it's best addressed as a separate issue. ## Validation Steps Performed There were already existing tests for the SU, SD, ICH, and DCH commands, but they were implemented as adapter tests, which weren't effectively testing anything - the `ScrollConsoleScreenBufferW` method used in those tests was just a mock (an incomplete reimplementation of the `ScrollRegion` function), so confirming that the mock produced the correct result told you nothing about the validity of the real code. To address that, I've now reimplemented those adapter tests as screen buffer tests. For the most part I've tried to duplicate the functionality of the original tests, but there are significant differences to account for the fact that scrolling region now covers the full width of the buffer rather than just the viewport width. I've also extended those tests with additional coverage for the RI, DL, and IL commands, which are really just a variation of the SU and SD functionality. Closes #2174
2019-09-11 03:20:46 +02:00
// Rectangle to cut out of the existing buffer. This is inclusive.
// It will be clipped to the buffer boundaries so SHORT_MAX gives us the full buffer width.
SMALL_RECT srScreen;
srScreen.Left = 0;
srScreen.Right = SHORT_MAX;
srScreen.Top = csbiex.srWindow.Top;
srScreen.Bottom = csbiex.srWindow.Bottom - 1; // srWindow is exclusive, hence the - 1
Remove unwanted DECSTBM clipping (#2764) The `DECSTBM` margins are meant to define the range of lines within which certain vertical scrolling operations take place. However, we were applying these margin restrictions in the `ScrollRegion` function, which is also used in a number of places that shouldn't be affected by `DECSTBM`. This includes the `ICH` and `DCH` escape sequences (which are only affected by the horizontal margins, which we don't yet support), the `ScrollConsoleScreenBuffer` API (which is public Console API, not meant to be affected by the VT terminal emulation), and the `CSI 3 J` erase scrollback extension (which isn't really scrolling as such, but uses the `ScrollRegion` function to manipulate the scrollback buffer). This commit moves the margin clipping out of the `ScrollRegion` function, so it can be applied exclusively in the places that need it. With the margin clipping removed from the `ScrollRegion` function, it now had to be applied manually in the places it was actually required. This included: * The `DoSrvPrivateReverseLineFeed` function (for the `RI` control): This was * just a matter of updating the bottom of the scroll rect to the bottom margin * (at least when the margins were actually set), since the top of the scroll * rect would always be the top of the viewport. The * `DoSrvPrivateModifyLinesImpl` function (for the `IL` and `DL` commands): * Again this was just a matter of updating the bottom of the scroll rect, since * the cursor position would always determine the top of the scroll rect. The * `AdaptDispatch::_ScrollMovement` method (for the `SU` and `SD` commands): * This required updating both the top and bottom coordinates of the scroll * rect, and also a simpler destination Y coordinate (the way the `ScrollRegion` * function worked before, the caller was expected to take the margins into * account when determining the destination). On the plus side, there was now no longer a need to override the margins when calling `ScrollRegion` in the `AdjustCursorPosition` function. In the first case, the margins had needed to be cleared (_stream.cpp 143-145), but that is now the default behaviour. In the second case, there had been a more complicated adjustment of the margins (_stream.cpp 196-209), but that code was never actually used so could be removed completely (to get to that point either _fScrollUp_ was true, so _scrollDownAtTop_ couldn't also be true, or _fScrollDown_ was true, but in that case there is a check to make sure _scrollDownAtTop_ is false). While testing, I also noticed that one of the `ScrollRegion` calls in the `AdjustCursorPosition` function was not setting the horizontal range correctly - the scrolling should always affect the full buffer width rather than just the viewport width - so I've fixed that now as well. ## Validation Steps Performed For commands like `RI`, `IL`, `DL`, etc. where we've changed the implementation but not the behaviour, there were already unit tests that could confirm that the new implementation was still producing the correct results. Where there has been a change in behaviour - namely for the `ICH` and `DCH` commands, and the `ScrollConsoleScreenBuffer` API - I've extended the existing unit tests to check that they still function correctly even when the `DECSTBM` margins are set (which would previously have caused them to fail). I've also tested manually with the test cases in issues #2543 and #2659, and confirmed that they now work as expected. Closes #2543 Closes #2659
2019-09-24 01:16:54 +02:00
// Clip to the DECSTBM margin boundaries
if (_scrollMargins.Top < _scrollMargins.Bottom)
Remove unwanted DECSTBM clipping (#2764) The `DECSTBM` margins are meant to define the range of lines within which certain vertical scrolling operations take place. However, we were applying these margin restrictions in the `ScrollRegion` function, which is also used in a number of places that shouldn't be affected by `DECSTBM`. This includes the `ICH` and `DCH` escape sequences (which are only affected by the horizontal margins, which we don't yet support), the `ScrollConsoleScreenBuffer` API (which is public Console API, not meant to be affected by the VT terminal emulation), and the `CSI 3 J` erase scrollback extension (which isn't really scrolling as such, but uses the `ScrollRegion` function to manipulate the scrollback buffer). This commit moves the margin clipping out of the `ScrollRegion` function, so it can be applied exclusively in the places that need it. With the margin clipping removed from the `ScrollRegion` function, it now had to be applied manually in the places it was actually required. This included: * The `DoSrvPrivateReverseLineFeed` function (for the `RI` control): This was * just a matter of updating the bottom of the scroll rect to the bottom margin * (at least when the margins were actually set), since the top of the scroll * rect would always be the top of the viewport. The * `DoSrvPrivateModifyLinesImpl` function (for the `IL` and `DL` commands): * Again this was just a matter of updating the bottom of the scroll rect, since * the cursor position would always determine the top of the scroll rect. The * `AdaptDispatch::_ScrollMovement` method (for the `SU` and `SD` commands): * This required updating both the top and bottom coordinates of the scroll * rect, and also a simpler destination Y coordinate (the way the `ScrollRegion` * function worked before, the caller was expected to take the margins into * account when determining the destination). On the plus side, there was now no longer a need to override the margins when calling `ScrollRegion` in the `AdjustCursorPosition` function. In the first case, the margins had needed to be cleared (_stream.cpp 143-145), but that is now the default behaviour. In the second case, there had been a more complicated adjustment of the margins (_stream.cpp 196-209), but that code was never actually used so could be removed completely (to get to that point either _fScrollUp_ was true, so _scrollDownAtTop_ couldn't also be true, or _fScrollDown_ was true, but in that case there is a check to make sure _scrollDownAtTop_ is false). While testing, I also noticed that one of the `ScrollRegion` calls in the `AdjustCursorPosition` function was not setting the horizontal range correctly - the scrolling should always affect the full buffer width rather than just the viewport width - so I've fixed that now as well. ## Validation Steps Performed For commands like `RI`, `IL`, `DL`, etc. where we've changed the implementation but not the behaviour, there were already unit tests that could confirm that the new implementation was still producing the correct results. Where there has been a change in behaviour - namely for the `ICH` and `DCH` commands, and the `ScrollConsoleScreenBuffer` API - I've extended the existing unit tests to check that they still function correctly even when the `DECSTBM` margins are set (which would previously have caused them to fail). I've also tested manually with the test cases in issues #2543 and #2659, and confirmed that they now work as expected. Closes #2543 Closes #2659
2019-09-24 01:16:54 +02:00
{
srScreen.Top = csbiex.srWindow.Top + _scrollMargins.Top;
srScreen.Bottom = csbiex.srWindow.Top + _scrollMargins.Bottom;
Remove unwanted DECSTBM clipping (#2764) The `DECSTBM` margins are meant to define the range of lines within which certain vertical scrolling operations take place. However, we were applying these margin restrictions in the `ScrollRegion` function, which is also used in a number of places that shouldn't be affected by `DECSTBM`. This includes the `ICH` and `DCH` escape sequences (which are only affected by the horizontal margins, which we don't yet support), the `ScrollConsoleScreenBuffer` API (which is public Console API, not meant to be affected by the VT terminal emulation), and the `CSI 3 J` erase scrollback extension (which isn't really scrolling as such, but uses the `ScrollRegion` function to manipulate the scrollback buffer). This commit moves the margin clipping out of the `ScrollRegion` function, so it can be applied exclusively in the places that need it. With the margin clipping removed from the `ScrollRegion` function, it now had to be applied manually in the places it was actually required. This included: * The `DoSrvPrivateReverseLineFeed` function (for the `RI` control): This was * just a matter of updating the bottom of the scroll rect to the bottom margin * (at least when the margins were actually set), since the top of the scroll * rect would always be the top of the viewport. The * `DoSrvPrivateModifyLinesImpl` function (for the `IL` and `DL` commands): * Again this was just a matter of updating the bottom of the scroll rect, since * the cursor position would always determine the top of the scroll rect. The * `AdaptDispatch::_ScrollMovement` method (for the `SU` and `SD` commands): * This required updating both the top and bottom coordinates of the scroll * rect, and also a simpler destination Y coordinate (the way the `ScrollRegion` * function worked before, the caller was expected to take the margins into * account when determining the destination). On the plus side, there was now no longer a need to override the margins when calling `ScrollRegion` in the `AdjustCursorPosition` function. In the first case, the margins had needed to be cleared (_stream.cpp 143-145), but that is now the default behaviour. In the second case, there had been a more complicated adjustment of the margins (_stream.cpp 196-209), but that code was never actually used so could be removed completely (to get to that point either _fScrollUp_ was true, so _scrollDownAtTop_ couldn't also be true, or _fScrollDown_ was true, but in that case there is a check to make sure _scrollDownAtTop_ is false). While testing, I also noticed that one of the `ScrollRegion` calls in the `AdjustCursorPosition` function was not setting the horizontal range correctly - the scrolling should always affect the full buffer width rather than just the viewport width - so I've fixed that now as well. ## Validation Steps Performed For commands like `RI`, `IL`, `DL`, etc. where we've changed the implementation but not the behaviour, there were already unit tests that could confirm that the new implementation was still producing the correct results. Where there has been a change in behaviour - namely for the `ICH` and `DCH` commands, and the `ScrollConsoleScreenBuffer` API - I've extended the existing unit tests to check that they still function correctly even when the `DECSTBM` margins are set (which would previously have caused them to fail). I've also tested manually with the test cases in issues #2543 and #2659, and confirmed that they now work as expected. Closes #2543 Closes #2659
2019-09-24 01:16:54 +02:00
}
// Paste coordinate for cut text above
COORD coordDestination;
coordDestination.X = srScreen.Left;
coordDestination.Y = srScreen.Top + dist * (scrollDirection == ScrollDirection::Up ? -1 : 1);
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// Note the revealed lines are filled with the standard erase attributes.
success = _pConApi->PrivateScrollRegion(srScreen, srScreen, coordDestination, true);
}
}
return success;
}
// Routine Description:
// - SU - Pans the window DOWN by given distance (distance new lines appear at the bottom of the screen)
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::ScrollUp(const size_t uiDistance)
{
return _ScrollMovement(ScrollDirection::Up, uiDistance);
}
// Routine Description:
// - SD - Pans the window UP by given distance (distance new lines appear at the top of the screen)
// Arguments:
// - distance - Distance to move
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::ScrollDown(const size_t uiDistance)
{
return _ScrollMovement(ScrollDirection::Down, uiDistance);
}
// Routine Description:
// - DECSCPP / DECCOLM Sets the number of columns "per page" AKA sets the console width.
// DECCOLM also clear the screen (like a CSI 2 J sequence), while DECSCPP just sets the width.
// (DECCOLM will do this separately of this function)
// Arguments:
// - columns - Number of columns
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetColumns(const size_t columns)
{
SHORT col;
bool success = SUCCEEDED(SizeTToShort(columns, &col));
if (success)
{
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
success = _pConApi->GetConsoleScreenBufferInfoEx(csbiex);
if (success)
{
csbiex.dwSize.X = col;
success = _pConApi->SetConsoleScreenBufferInfoEx(csbiex);
}
}
return success;
}
// Routine Description:
// - DECCOLM not only sets the number of columns, but also clears the screen buffer,
// resets the page margins and origin mode, and places the cursor at 1,1
// Arguments:
// - columns - Number of columns
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_DoDECCOLMHelper(const size_t columns)
{
if (!_isDECCOLMAllowed)
{
// Only proceed if DECCOLM is allowed. Return true, as this is technically a successful handling.
return true;
}
bool success = SetColumns(columns);
if (success)
{
success = SetOriginMode(false);
if (success)
{
success = CursorPosition(1, 1);
if (success)
{
success = EraseInDisplay(DispatchTypes::EraseType::All);
if (success)
{
success = _DoSetTopBottomScrollingMargins(0, 0);
}
}
}
}
return success;
}
// Routine Description:
// - Support routine for routing private mode parameters to be set/reset as flags
// Arguments:
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
// - param - mode parameter to set/reset
// - enable - True for set, false for unset.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_ModeParamsHelper(const DispatchTypes::ModeParams param, const bool enable)
{
bool success = false;
switch (param)
{
case DispatchTypes::ModeParams::DECCKM_CursorKeysMode:
// set - Enable Application Mode, reset - Normal mode
success = SetCursorKeysMode(enable);
break;
case DispatchTypes::ModeParams::DECANM_AnsiMode:
Add support for VT52 emulation (#4789) ## Summary of the Pull Request This PR adds support for the core VT52 commands, and implements the `DECANM` private mode sequence, which switches the terminal between ANSI mode and VT52-compatible mode. ## References PR #2017 defined the initial specification for VT52 support. PR #4044 removed the original VT52 cursor ops that conflicted with VT100 sequences. ## PR Checklist * [x] Closes #976 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #2017 ## Detailed Description of the Pull Request / Additional comments Most of the work involves updates to the parsing state machine, which behaves differently in VT52 mode. `CSI`, `OSC`, and `SS3` sequences are not applicable, and there is one special-case escape sequence (_Direct Cursor Address_), which requires an additional state to handle parameters that come _after_ the final character. Once the parsing is handled though, it's mostly just a matter of dispatching the commands to existing methods in the `ITermDispatch` interface. Only one new method was required in the interface to handle the _Identify_ command. The only real new functionality is in the `TerminalInput` class, which needs to generate different escape sequences for certain keys in VT52 mode. This does not yet support _all_ of the VT52 key sequences, because the VT100 support is itself not yet complete. But the basics are in place, and I think the rest is best left for a follow-up issue, and potentially a refactor of the `TerminalInput` class. I should point out that the original spec called for a new _Graphic Mode_ character set, but I've since discovered that the VT terminals that _emulate_ VT52 just use the existing VT100 _Special Graphics_ set, so that is really what we should be doing too. We can always consider adding the VT52 graphic set as a option later, if there is demand for strict VT52 compatibility. ## Validation Steps Performed I've added state machine and adapter tests to confirm that the `DECANM` mode changing sequences are correctly dispatched and forwarded to the `ConGetSet` handler. I've also added state machine tests that confirm the VT52 escape sequences are dispatched correctly when the ANSI mode is reset. For fuzzing support, I've extended the VT command fuzzer to generate the different kinds of VT52 sequences, as well as mode change sequences to switch between the ANSI and VT52 modes. In terms of manual testing, I've confirmed that the _Test of VT52 mode_ in Vttest now works as expected.
2020-06-01 23:20:40 +02:00
success = SetAnsiMode(enable);
break;
case DispatchTypes::ModeParams::DECCOLM_SetNumberOfColumns:
success = _DoDECCOLMHelper(enable ? DispatchTypes::s_sDECCOLMSetColumns : DispatchTypes::s_sDECCOLMResetColumns);
break;
case DispatchTypes::ModeParams::DECSCNM_ScreenMode:
Add support for the DECSCNM screen mode (#3817) ## Summary of the Pull Request This adds support for the [`DECSCNM`](https://vt100.net/docs/vt510-rm/DECSCNM.html) private mode escape sequence, which toggles the display between normal and reverse screen modes. When reversed, the background and foreground colors are switched. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## References This also fixes issue #72 for the most part, although if you toggle the mode too fast, there is no discernible flash. ## PR Checklist * [x] Closes #3773 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments I've implemented this as a new flag in the `Settings` class, along with updates to the `LookupForegroundColor` and `LookupBackgroundColor` methods, to switch the returned foreground and background colors when that flag is set. It also required a new private API in the `ConGetSet` interface to toggle the setting. And that API is then called from the `AdaptDispatch` class when the screen mode escape sequence is received. The last thing needed was to add a step to the `HardReset` method, to reset the mode back to normal, which is one of the `RIS` requirements. Note that this does currently work in the Windows Terminal, but once #2661 is implemented that may no longer be the case. It might become necessary to let the mode change sequences pass through conpty, and handle the color reversing on the client side. ## Validation Steps Performed I've added a state machine test to make sure the escape sequence is dispatched correctly, and a screen buffer test to confirm that the mode change does alter the interpretation of colors as expected. I've also confirmed that the various "light background" tests in Vttest now display correctly, and that the `tput flash` command (in a bash shell) does actually cause the screen to flash.
2020-01-22 23:29:50 +01:00
success = SetScreenMode(enable);
break;
case DispatchTypes::ModeParams::DECOM_OriginMode:
// The cursor is also moved to the new home position when the origin mode is set or reset.
success = SetOriginMode(enable) && CursorPosition(1, 1);
break;
case DispatchTypes::ModeParams::DECAWM_AutoWrapMode:
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
success = SetAutoWrapMode(enable);
break;
case DispatchTypes::ModeParams::ATT610_StartCursorBlink:
success = EnableCursorBlinking(enable);
break;
case DispatchTypes::ModeParams::DECTCEM_TextCursorEnableMode:
success = CursorVisibility(enable);
break;
case DispatchTypes::ModeParams::XTERM_EnableDECCOLMSupport:
success = EnableDECCOLMSupport(enable);
break;
case DispatchTypes::ModeParams::VT200_MOUSE_MODE:
success = EnableVT200MouseMode(enable);
break;
case DispatchTypes::ModeParams::BUTTON_EVENT_MOUSE_MODE:
success = EnableButtonEventMouseMode(enable);
break;
case DispatchTypes::ModeParams::ANY_EVENT_MOUSE_MODE:
success = EnableAnyEventMouseMode(enable);
break;
case DispatchTypes::ModeParams::UTF8_EXTENDED_MODE:
success = EnableUTF8ExtendedMouseMode(enable);
break;
case DispatchTypes::ModeParams::SGR_EXTENDED_MODE:
success = EnableSGRExtendedMouseMode(enable);
break;
case DispatchTypes::ModeParams::ALTERNATE_SCROLL:
success = EnableAlternateScroll(enable);
break;
case DispatchTypes::ModeParams::ASB_AlternateScreenBuffer:
success = enable ? UseAlternateScreenBuffer() : UseMainScreenBuffer();
break;
case DispatchTypes::ModeParams::W32IM_Win32InputMode:
success = EnableWin32InputMode(enable);
break;
default:
// If no functions to call, overall dispatch was a failure.
success = false;
break;
}
return success;
}
// Routine Description:
// - DECSET - Enables the given DEC private mode params.
// Arguments:
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
// - param - mode parameter to set
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetMode(const DispatchTypes::ModeParams param)
{
return _ModeParamsHelper(param, true);
}
// Routine Description:
// - DECRST - Disables the given DEC private mode params.
// Arguments:
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
// - param - mode parameter to reset
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::ResetMode(const DispatchTypes::ModeParams param)
{
return _ModeParamsHelper(param, false);
}
// - DECKPAM, DECKPNM - Sets the keypad input mode to either Application mode or Numeric mode (true, false respectively)
// Arguments:
// - applicationMode - set to true to enable Application Mode Input, false for Numeric Mode Input.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetKeypadMode(const bool fApplicationMode)
{
Consolidate the interfaces for setting VT input modes (#11384) Instead of having a separate method for setting each mouse and keyboard mode, this PR consolidates them all into a single method which takes a mode parameter, and stores the modes in a `til::enumset` rather than having a separate `bool` for each mode. This enables us to get rid of a lot of boilerplate code, and makes the code easier to extend when we want to introduce additional modes in the future. It'll also makes it easier to read back the state of the various modes when implementing the `DECRQM` query. Most of the complication is in the `TerminalInput` class, which had to be adjusted to work with an `enumset` in place of all the `bool` fields. For the rest, it was largely a matter of replacing calls to all the old mode setting methods with the new `SetInputMode` method, and deleting a bunch of unused code. One thing worth mentioning is that the `AdaptDispatch` implementation used to have a `_ShouldPassThroughInputModeChange` method that was called after every mode change. This code has now been moved up into the `SetInputMode` implementation in `ConhostInternalGetSet` so it's just handled in one place. Keeping this out of the dispatch class will also be beneficial for sharing the implementation with `TerminalDispatch`. ## Validation The updated interface necessitated some adjustments to the tests in `AdapterTest` and `MouseInputTest`, but the essential structure of the tests remains unchanged, and everything still passes. I've also tested the keyboard and mouse modes in Vttest and confirmed they still work at least as well as they did before (both conhost and Windows Terminal), and I tested the alternate scroll mode manually (conhost only). Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan to de-duplicate the `AdaptDispatch` and `TerminalDispatch` implementation (#3849).
2021-10-26 23:12:22 +02:00
return _pConApi->SetInputMode(TerminalInput::Mode::Keypad, fApplicationMode);
}
// Method Description:
// - win32-input-mode: Enable sending full input records encoded as a string of
// characters to the client application.
// Arguments:
// - win32InputMode - set to true to enable win32-input-mode, false to disable.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::EnableWin32InputMode(const bool win32InputMode)
{
Consolidate the interfaces for setting VT input modes (#11384) Instead of having a separate method for setting each mouse and keyboard mode, this PR consolidates them all into a single method which takes a mode parameter, and stores the modes in a `til::enumset` rather than having a separate `bool` for each mode. This enables us to get rid of a lot of boilerplate code, and makes the code easier to extend when we want to introduce additional modes in the future. It'll also makes it easier to read back the state of the various modes when implementing the `DECRQM` query. Most of the complication is in the `TerminalInput` class, which had to be adjusted to work with an `enumset` in place of all the `bool` fields. For the rest, it was largely a matter of replacing calls to all the old mode setting methods with the new `SetInputMode` method, and deleting a bunch of unused code. One thing worth mentioning is that the `AdaptDispatch` implementation used to have a `_ShouldPassThroughInputModeChange` method that was called after every mode change. This code has now been moved up into the `SetInputMode` implementation in `ConhostInternalGetSet` so it's just handled in one place. Keeping this out of the dispatch class will also be beneficial for sharing the implementation with `TerminalDispatch`. ## Validation The updated interface necessitated some adjustments to the tests in `AdapterTest` and `MouseInputTest`, but the essential structure of the tests remains unchanged, and everything still passes. I've also tested the keyboard and mouse modes in Vttest and confirmed they still work at least as well as they did before (both conhost and Windows Terminal), and I tested the alternate scroll mode manually (conhost only). Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan to de-duplicate the `AdaptDispatch` and `TerminalDispatch` implementation (#3849).
2021-10-26 23:12:22 +02:00
return _pConApi->SetInputMode(TerminalInput::Mode::Win32, win32InputMode);
}
// - DECCKM - Sets the cursor keys input mode to either Application mode or Normal mode (true, false respectively)
// Arguments:
// - applicationMode - set to true to enable Application Mode Input, false for Normal Mode Input.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetCursorKeysMode(const bool applicationMode)
{
Consolidate the interfaces for setting VT input modes (#11384) Instead of having a separate method for setting each mouse and keyboard mode, this PR consolidates them all into a single method which takes a mode parameter, and stores the modes in a `til::enumset` rather than having a separate `bool` for each mode. This enables us to get rid of a lot of boilerplate code, and makes the code easier to extend when we want to introduce additional modes in the future. It'll also makes it easier to read back the state of the various modes when implementing the `DECRQM` query. Most of the complication is in the `TerminalInput` class, which had to be adjusted to work with an `enumset` in place of all the `bool` fields. For the rest, it was largely a matter of replacing calls to all the old mode setting methods with the new `SetInputMode` method, and deleting a bunch of unused code. One thing worth mentioning is that the `AdaptDispatch` implementation used to have a `_ShouldPassThroughInputModeChange` method that was called after every mode change. This code has now been moved up into the `SetInputMode` implementation in `ConhostInternalGetSet` so it's just handled in one place. Keeping this out of the dispatch class will also be beneficial for sharing the implementation with `TerminalDispatch`. ## Validation The updated interface necessitated some adjustments to the tests in `AdapterTest` and `MouseInputTest`, but the essential structure of the tests remains unchanged, and everything still passes. I've also tested the keyboard and mouse modes in Vttest and confirmed they still work at least as well as they did before (both conhost and Windows Terminal), and I tested the alternate scroll mode manually (conhost only). Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan to de-duplicate the `AdaptDispatch` and `TerminalDispatch` implementation (#3849).
2021-10-26 23:12:22 +02:00
return _pConApi->SetInputMode(TerminalInput::Mode::CursorKey, applicationMode);
}
// - att610 - Enables or disables the cursor blinking.
// Arguments:
// - enable - set to true to enable blinking, false to disable
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::EnableCursorBlinking(const bool enable)
{
return _pConApi->PrivateAllowCursorBlinking(enable);
}
// Routine Description:
// - IL - This control function inserts one or more blank lines, starting at the cursor.
// As lines are inserted, lines below the cursor and in the scrolling region move down.
// Lines scrolled off the page are lost. IL has no effect outside the page margins.
// Arguments:
// - distance - number of lines to insert
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::InsertLine(const size_t distance)
{
return _pConApi->InsertLines(distance);
}
// Routine Description:
// - DL - This control function deletes one or more lines in the scrolling
// region, starting with the line that has the cursor.
// As lines are deleted, lines below the cursor and in the scrolling region
// move up. The terminal adds blank lines with no visual character
// attributes at the bottom of the scrolling region. If distance is greater than
// the number of lines remaining on the page, DL deletes only the remaining
// lines. DL has no effect outside the scrolling margins.
// Arguments:
// - distance - number of lines to delete
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::DeleteLine(const size_t distance)
{
return _pConApi->DeleteLines(distance);
}
Add support for VT52 emulation (#4789) ## Summary of the Pull Request This PR adds support for the core VT52 commands, and implements the `DECANM` private mode sequence, which switches the terminal between ANSI mode and VT52-compatible mode. ## References PR #2017 defined the initial specification for VT52 support. PR #4044 removed the original VT52 cursor ops that conflicted with VT100 sequences. ## PR Checklist * [x] Closes #976 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #2017 ## Detailed Description of the Pull Request / Additional comments Most of the work involves updates to the parsing state machine, which behaves differently in VT52 mode. `CSI`, `OSC`, and `SS3` sequences are not applicable, and there is one special-case escape sequence (_Direct Cursor Address_), which requires an additional state to handle parameters that come _after_ the final character. Once the parsing is handled though, it's mostly just a matter of dispatching the commands to existing methods in the `ITermDispatch` interface. Only one new method was required in the interface to handle the _Identify_ command. The only real new functionality is in the `TerminalInput` class, which needs to generate different escape sequences for certain keys in VT52 mode. This does not yet support _all_ of the VT52 key sequences, because the VT100 support is itself not yet complete. But the basics are in place, and I think the rest is best left for a follow-up issue, and potentially a refactor of the `TerminalInput` class. I should point out that the original spec called for a new _Graphic Mode_ character set, but I've since discovered that the VT terminals that _emulate_ VT52 just use the existing VT100 _Special Graphics_ set, so that is really what we should be doing too. We can always consider adding the VT52 graphic set as a option later, if there is demand for strict VT52 compatibility. ## Validation Steps Performed I've added state machine and adapter tests to confirm that the `DECANM` mode changing sequences are correctly dispatched and forwarded to the `ConGetSet` handler. I've also added state machine tests that confirm the VT52 escape sequences are dispatched correctly when the ANSI mode is reset. For fuzzing support, I've extended the VT command fuzzer to generate the different kinds of VT52 sequences, as well as mode change sequences to switch between the ANSI and VT52 modes. In terms of manual testing, I've confirmed that the _Test of VT52 mode_ in Vttest now works as expected.
2020-06-01 23:20:40 +02:00
// - DECANM - Sets the terminal emulation mode to either ANSI-compatible or VT52.
// Arguments:
// - ansiMode - set to true to enable the ANSI mode, false for VT52 mode.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetAnsiMode(const bool ansiMode)
{
// When an attempt is made to update the mode, the designated character sets
// need to be reset to defaults, even if the mode doesn't actually change.
_termOutput = {};
Disable the acceptance of C1 control codes by default (#11690) There are some code pages with "unmapped" code points in the C1 range, which results in them being translated into Unicode C1 control codes, even though that is not their intended use. To avoid having these characters triggering unintentional escape sequences, this PR now disables C1 controls by default. Switching to ISO-2022 encoding will re-enable them, though, since that is the most likely scenario in which they would be required. They can also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape sequence. What I've done is add a new mode to the `StateMachine` class that controls whether C1 code points are interpreted as control characters or not. When disabled, these code points are simply dropped from the output, similar to the way a `NUL` is interpreted. This isn't exactly the way they were handled in the v1 console (which I think replaces them with the font _notdef_ glyph), but it matches the XTerm behavior, which seems more appropriate considering this is in VT mode. And it's worth noting that Windows Explorer seems to work the same way. As mentioned above, the mode can be enabled by designating the ISO-2022 coding system with a `DOCS` sequence, and it will be disabled again when UTF-8 is designated. You can also enable it explicitly with a `DECAC1` sequence (originally this was actually a DEC printer sequence, but it doesn't seem unreasonable to use it in a terminal). I've also extended the operations that save and restore "cursor state" (e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode, since it's closely tied to the code page and character sets which are also saved there. Similarly, when a `DECSTR` sequence resets the code page and character sets, I've now made it reset the C1 mode as well. I should note that the new `StateMachine` mode is controlled via a generic `SetParserMode` method (with a matching API in the `ConGetSet` interface) to allow for easier addition of other modes in the future. And I've reimplemented the existing ANSI/VT52 mode in terms of these generic methods instead of it having to have its own separate APIs. ## Validation Steps Performed Some of the unit tests for OSC sequences were using a C1 `0x9C` for the string terminator, which doesn't work by default anymore. Since that's not a good practice anyway, I thought it best to change those to a standard 7-bit terminator. However, in tests that were explicitly validating the C1 controls, I've just enabled the C1 parser mode at the start of the tests in order to get them working again. There were also some ANSI mode adapter tests that had to be updated to account for the fact that it has now been reimplemented in terms of the `SetParserMode` API. I've added a new state machine test to validate the changes in behavior when the C1 parser mode is enabled or disabled. And I've added an adapter test to verify that the `DesignateCodingSystems` and `AcceptC1Controls` methods toggle the C1 parser mode as expected. I've manually verified the test cases in #10069 and #10310 to confirm that they're no longer triggering control sequences by default. Although, as I explained above, the C1 code points are completely dropped from the output rather than displayed as _notdef_ glyphs. I think this is a reasonable compromise though. Closes #10069 Closes #10310
2021-11-18 00:40:31 +01:00
_pConApi->SetParserMode(StateMachine::Mode::Ansi, ansiMode);
_pConApi->SetInputMode(TerminalInput::Mode::Ansi, ansiMode);
// We don't check the SetInputMode return value, because we'll never want
// to forward a DECANM mode change over conpty.
return true;
Add support for VT52 emulation (#4789) ## Summary of the Pull Request This PR adds support for the core VT52 commands, and implements the `DECANM` private mode sequence, which switches the terminal between ANSI mode and VT52-compatible mode. ## References PR #2017 defined the initial specification for VT52 support. PR #4044 removed the original VT52 cursor ops that conflicted with VT100 sequences. ## PR Checklist * [x] Closes #976 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #2017 ## Detailed Description of the Pull Request / Additional comments Most of the work involves updates to the parsing state machine, which behaves differently in VT52 mode. `CSI`, `OSC`, and `SS3` sequences are not applicable, and there is one special-case escape sequence (_Direct Cursor Address_), which requires an additional state to handle parameters that come _after_ the final character. Once the parsing is handled though, it's mostly just a matter of dispatching the commands to existing methods in the `ITermDispatch` interface. Only one new method was required in the interface to handle the _Identify_ command. The only real new functionality is in the `TerminalInput` class, which needs to generate different escape sequences for certain keys in VT52 mode. This does not yet support _all_ of the VT52 key sequences, because the VT100 support is itself not yet complete. But the basics are in place, and I think the rest is best left for a follow-up issue, and potentially a refactor of the `TerminalInput` class. I should point out that the original spec called for a new _Graphic Mode_ character set, but I've since discovered that the VT terminals that _emulate_ VT52 just use the existing VT100 _Special Graphics_ set, so that is really what we should be doing too. We can always consider adding the VT52 graphic set as a option later, if there is demand for strict VT52 compatibility. ## Validation Steps Performed I've added state machine and adapter tests to confirm that the `DECANM` mode changing sequences are correctly dispatched and forwarded to the `ConGetSet` handler. I've also added state machine tests that confirm the VT52 escape sequences are dispatched correctly when the ANSI mode is reset. For fuzzing support, I've extended the VT command fuzzer to generate the different kinds of VT52 sequences, as well as mode change sequences to switch between the ANSI and VT52 modes. In terms of manual testing, I've confirmed that the _Test of VT52 mode_ in Vttest now works as expected.
2020-06-01 23:20:40 +02:00
}
Add support for the DECSCNM screen mode (#3817) ## Summary of the Pull Request This adds support for the [`DECSCNM`](https://vt100.net/docs/vt510-rm/DECSCNM.html) private mode escape sequence, which toggles the display between normal and reverse screen modes. When reversed, the background and foreground colors are switched. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## References This also fixes issue #72 for the most part, although if you toggle the mode too fast, there is no discernible flash. ## PR Checklist * [x] Closes #3773 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments I've implemented this as a new flag in the `Settings` class, along with updates to the `LookupForegroundColor` and `LookupBackgroundColor` methods, to switch the returned foreground and background colors when that flag is set. It also required a new private API in the `ConGetSet` interface to toggle the setting. And that API is then called from the `AdaptDispatch` class when the screen mode escape sequence is received. The last thing needed was to add a step to the `HardReset` method, to reset the mode back to normal, which is one of the `RIS` requirements. Note that this does currently work in the Windows Terminal, but once #2661 is implemented that may no longer be the case. It might become necessary to let the mode change sequences pass through conpty, and handle the color reversing on the client side. ## Validation Steps Performed I've added a state machine test to make sure the escape sequence is dispatched correctly, and a screen buffer test to confirm that the mode change does alter the interpretation of colors as expected. I've also confirmed that the various "light background" tests in Vttest now display correctly, and that the `tput flash` command (in a bash shell) does actually cause the screen to flash.
2020-01-22 23:29:50 +01:00
// Routine Description:
// - DECSCNM - Sets the screen mode to either normal or reverse.
// When in reverse screen mode, the background and foreground colors are switched.
// Arguments:
// - reverseMode - set to true to enable reverse screen mode, false for normal mode.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetScreenMode(const bool reverseMode)
{
Add support for DECSCNM in Windows Terminal (#6809) ## Summary of the Pull Request This PR adds full support for the `DECSCNM` reverse screen mode in the Windows Terminal to align with the implementation in conhost. ## References * The conhost implementation of `DECSCNM` was in PR #3817. * WT originally inherited that functionality via the colors being passed through, but that behaviour was lost in PR #6506. ## PR Checklist * [x] Closes #6622 * [x] CLA signed. * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #6622 ## Detailed Description of the Pull Request / Additional comments The `AdaptDispatch::SetScreenMode` now checks if it's in conpty mode and simply returns false to force a pass-through of the mode change. And the `TerminalDispatch` now has its own `SetScreenMode` implementation that tracks any changes to the reversed state, and triggers a redraw in the renderer. To make the renderer work, we just needed to update the `GetForegroundColor` and `GetBackgroundColor` methods of the terminal's `IRenderData` implementation to check the reversed state, and switch the colors being calculated, the same way the `LookupForegroundColor` and `LookupBackgroundColor` methods work in the conhost `Settings` class. ## Validation Steps Performed I've manually tested the `DECSCNM` functionality for Windows Terminal in Vttest, and also with some of my own test scripts.
2020-07-09 13:25:30 +02:00
// If we're a conpty, always return false
if (_pConApi->IsConsolePty())
{
return false;
}
Add support for the DECSCNM screen mode (#3817) ## Summary of the Pull Request This adds support for the [`DECSCNM`](https://vt100.net/docs/vt510-rm/DECSCNM.html) private mode escape sequence, which toggles the display between normal and reverse screen modes. When reversed, the background and foreground colors are switched. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## References This also fixes issue #72 for the most part, although if you toggle the mode too fast, there is no discernible flash. ## PR Checklist * [x] Closes #3773 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments I've implemented this as a new flag in the `Settings` class, along with updates to the `LookupForegroundColor` and `LookupBackgroundColor` methods, to switch the returned foreground and background colors when that flag is set. It also required a new private API in the `ConGetSet` interface to toggle the setting. And that API is then called from the `AdaptDispatch` class when the screen mode escape sequence is received. The last thing needed was to add a step to the `HardReset` method, to reset the mode back to normal, which is one of the `RIS` requirements. Note that this does currently work in the Windows Terminal, but once #2661 is implemented that may no longer be the case. It might become necessary to let the mode change sequences pass through conpty, and handle the color reversing on the client side. ## Validation Steps Performed I've added a state machine test to make sure the escape sequence is dispatched correctly, and a screen buffer test to confirm that the mode change does alter the interpretation of colors as expected. I've also confirmed that the various "light background" tests in Vttest now display correctly, and that the `tput flash` command (in a bash shell) does actually cause the screen to flash.
2020-01-22 23:29:50 +01:00
return _pConApi->PrivateSetScreenMode(reverseMode);
}
// Routine Description:
// - DECOM - Sets the cursor addressing origin mode to relative or absolute.
// When relative, line numbers start at top margin of the user-defined scrolling region.
// When absolute, line numbers are independent of the scrolling region.
// Arguments:
// - relativeMode - set to true to use relative addressing, false for absolute addressing.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetOriginMode(const bool relativeMode) noexcept
{
_isOriginModeRelative = relativeMode;
return true;
}
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
// Routine Description:
// - DECAWM - Sets the Auto Wrap Mode.
// This controls whether the cursor moves to the beginning of the next row
// when it reaches the end of the current row.
// Arguments:
// - wrapAtEOL - set to true to wrap, false to overwrite the last character.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetAutoWrapMode(const bool wrapAtEOL)
{
return _pConApi->PrivateSetAutoWrapMode(wrapAtEOL);
}
// Routine Description:
// - DECSTBM - Set Scrolling Region
// This control function sets the top and bottom margins for the current page.
// You cannot perform scrolling outside the margins.
// Default: Margins are at the page limits.
// Arguments:
// - topMargin - the line number for the top margin.
// - bottomMargin - the line number for the bottom margin.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_DoSetTopBottomScrollingMargins(const size_t topMargin,
const size_t bottomMargin)
{
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
// Make sure to reset the viewport (with MoveToBottom )to where it was
// before the user scrolled the console output
bool success = (_pConApi->MoveToBottom() && _pConApi->GetConsoleScreenBufferInfoEx(csbiex));
// so notes time: (input -> state machine out -> adapter out -> conhost internal)
// having only a top param is legal ([3;r -> 3,0 -> 3,h -> 3,h,true)
// having only a bottom param is legal ([;3r -> 0,3 -> 1,3 -> 1,3,true)
// having neither uses the defaults ([;r [r -> 0,0 -> 0,0 -> 0,0,false)
// an illegal combo (eg, 3;2r) is ignored
if (success)
{
SHORT actualTop = 0;
SHORT actualBottom = 0;
success = SUCCEEDED(SizeTToShort(topMargin, &actualTop)) && SUCCEEDED(SizeTToShort(bottomMargin, &actualBottom));
if (success)
{
const SHORT screenHeight = csbiex.srWindow.Bottom - csbiex.srWindow.Top;
// The default top margin is line 1
if (actualTop == 0)
{
actualTop = 1;
}
// The default bottom margin is the screen height
if (actualBottom == 0)
{
actualBottom = screenHeight;
}
// The top margin must be less than the bottom margin, and the
// bottom margin must be less than or equal to the screen height
success = (actualTop < actualBottom && actualBottom <= screenHeight);
if (success)
{
if (actualTop == 1 && actualBottom == screenHeight)
{
// Client requests setting margins to the entire screen
// - clear them instead of setting them.
// This is for apps like `apt` (NOT `apt-get` which set scroll
// margins, but don't use the alt buffer.)
actualTop = 0;
actualBottom = 0;
}
else
{
// In VT, the origin is 1,1. For our array, it's 0,0. So subtract 1.
actualTop -= 1;
actualBottom -= 1;
}
_scrollMargins.Top = actualTop;
_scrollMargins.Bottom = actualBottom;
success = _pConApi->PrivateSetScrollingRegion(_scrollMargins);
}
}
}
return success;
}
// Routine Description:
// - DECSTBM - Set Scrolling Region
// This control function sets the top and bottom margins for the current page.
// You cannot perform scrolling outside the margins.
// Default: Margins are at the page limits.
// Arguments:
// - topMargin - the line number for the top margin.
// - bottomMargin - the line number for the bottom margin.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetTopBottomScrollingMargins(const size_t topMargin,
const size_t bottomMargin)
{
// When this is called, the cursor should also be moved to home.
// Other functions that only need to set/reset the margins should call _DoSetTopBottomScrollingMargins
return _DoSetTopBottomScrollingMargins(topMargin, bottomMargin) && CursorPosition(1, 1);
}
Dispatch more C0 control characters from the VT state machine (#4171) This commit moves the handling of the `BEL`, `BS`, `TAB`, and `CR` controls characters into the state machine (when in VT mode), instead of forwarding them on to the default string writer, which would otherwise have to parse them out all over again. This doesn't cover all the control characters, but `ESC`, `SUB`, and `CAN` are already an integral part of the `StateMachine` itself; `NUL` is filtered out by the `OutputStateMachineEngine`; and `LF`, `FF`, and `VT` are due to be implemented as part of PR #3271. Once all of these controls are handled at the state machine level, we can strip out all the VT-specific code from the `WriteCharsLegacy` function, which should simplify it considerably. This would also let us simplify the `Terminal::_WriteBuffer` implementation, and the planned replacement stream writer for issue #780. On the conhost side, the implementation is handled as follows: * The `BS` control is dispatched to the existing `CursorBackward` method, with a distance of 1. * The `TAB` control is dispatched to the existing `ForwardTab` method, with a tab count of 1. * The `CR` control required a new dispatch method, but the implementation was a simple call to the new `_CursorMovePosition` method from PR #3628. * The `BEL` control also required a new dispatch method, as well as an additional private API in the `ConGetSet` interface. But that's mostly boilerplate code - ultimately it just calls the `SendNotifyBeep` method. On the Windows Terminal side, not all dispatch methods are implemented. * There is an existing `CursorBackward` implementation, so `BS` works OK. * There isn't a `ForwardTab` implementation, but `TAB` isn't currently required by the conpty protocol. * I had to implement the `CarriageReturn` dispatch method, but that was a simple call to `Terminal::SetCursorPosition`. * The `WarningBell` method I've left unimplemented, because that functionality wasn't previously supported anyway, and there's an existing issue for that (#4046). ## Validation Steps Performed I've added a state machine test to confirm that the updated control characters are now forwarded to the appropriate dispatch handlers. But since the actual implementation is mostly relying on existing functionality, I'm assuming that code is already adequately tested elsewhere. That said, I have also run various manual tests of my own, and confirmed that everything still worked as well as before. References #3271 References #780 References #3628 References #4046
2020-01-17 02:43:21 +01:00
// Routine Description:
// - BEL - Rings the warning bell.
// Causes the terminal to emit an audible tone of brief duration.
// Arguments:
// - None
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::WarningBell()
{
return _pConApi->PrivateWarningBell();
}
// Routine Description:
// - CR - Performs a carriage return.
// Moves the cursor to the leftmost column.
// Arguments:
// - None
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::CarriageReturn()
{
return _CursorMovePosition(Offset::Unchanged(), Offset::Absolute(1), true);
}
Add support for all the line feed control sequences (#3271) ## Summary of the Pull Request This adds support for the `FF` (form feed) and `VT` (vertical tab) [control characters](https://vt100.net/docs/vt510-rm/chapter4.html#T4-1), as well as the [`NEL` (Next Line)](https://vt100.net/docs/vt510-rm/NEL.html) and [`IND` (Index)](https://vt100.net/docs/vt510-rm/IND.html) escape sequences. ## References #976 discusses the conflict between VT100 Index sequence and the VT52 cursor back sequence. ## PR Checklist * [x] Closes #3189 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3189 ## Detailed Description of the Pull Request / Additional comments I've added a `LineFeed` method to the `ITermDispatch` interface, with an enum parameter specifying the required line feed type (i.e. with carriage return, without carriage return, or dependent on the [`LNM` mode](https://vt100.net/docs/vt510-rm/LNM.html)). The output state machine can then call that method to handle the various line feed control characters (parsed in the `ActionExecute` method), as well the `NEL` and `IND` escape sequences (parsed in the `ActionEscDispatch` method). The `AdaptDispatch` implementation of `LineFeed` then forwards the call to a new `PrivateLineFeed` method in the `ConGetSet` interface, which simply takes a bool parameter specifying whether a carriage return is required or not. In the case of mode-dependent line feeds, the `AdaptDispatch` implementation determines whether the return is necessary or not, based on the existing _AutoReturnOnNewLine_ setting (which I'm obtaining via another new `PrivateGetLineFeedMode` method). Ultimately we'll want to support changing the mode via the [`LNM` escape sequence](https://vt100.net/docs/vt510-rm/LNM.html), but there's no urgent need for that now. And using the existing _AutoReturnOnNewLine_ setting as a substitute for the mode gives us backwards compatible behaviour, since that will be true for the Windows shells (which expect a linefeed to also generate a carriage return), and false in a WSL bash shell (which won't want the carriage return by default). As for the actual `PrivateLineFeed` implementation, that is just a simplified version of how the line feed would previously have been executed in the `WriteCharsLegacy` function. This includes setting the cursor to "On" (with `Cursor::SetIsOn`), potentially clearing the wrap property of the line being left (with `CharRow::SetWrapForced` false), and then setting the new position using `AdjustCursorPosition` with the _fKeepCursorVisible_ parameter set to false. I'm unsure whether the `SetIsOn` call is really necessary, and I think the way the forced wrap is handled needs a rethink in general, but for now this should at least be compatible with the existing behaviour. Finally, in order to make this all work in the _Windows Terminal_ app, I also had to add a basic implementation of the `ITermDispatch::LineFeed` method in the `TerminalDispatch` class. There is currently no need to support mode-specific line feeds here, so this simply forwards a `\n` or `\r\n` to the `Execute` method, which is ultimately handled by the `Terminal::_WriteBuffer` implementation. ## Validation Steps Performed I've added output engine tests which confirm that the various control characters and escape sequences trigger the dispatch method correctly. Then I've added adapter tests which confirm the various dispatch options trigger the `PrivateLineFeed` API correctly. And finally I added some screen buffer tests that check the actual results of the `NEL` and `IND` sequences, which covers both forms of the `PrivateLineFeed` API (i.e. with and without a carriage return). I've also run the _Test of cursor movements_ in the [Vttest](https://invisible-island.net/vttest/) utility, and confirmed that screens 1, 2, and 5 are now working correctly. The first two depend on `NEL` and `IND` being supported, and screen 5 requires the `VT` control character.
2020-01-15 14:41:55 +01:00
// Routine Description:
// - IND/NEL - Performs a line feed, possibly preceded by carriage return.
// Moves the cursor down one line, and possibly also to the leftmost column.
// Arguments:
// - lineFeedType - Specify whether a carriage return should be performed as well.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::LineFeed(const DispatchTypes::LineFeedType lineFeedType)
{
switch (lineFeedType)
{
case DispatchTypes::LineFeedType::DependsOnMode:
return _pConApi->PrivateLineFeed(_pConApi->PrivateGetLineFeedMode());
case DispatchTypes::LineFeedType::WithoutReturn:
return _pConApi->PrivateLineFeed(false);
case DispatchTypes::LineFeedType::WithReturn:
return _pConApi->PrivateLineFeed(true);
default:
return false;
}
}
// Routine Description:
// - RI - Performs a "Reverse line feed", essentially, the opposite of '\n'.
// Moves the cursor up one line, and tries to keep its position in the line
// Arguments:
// - None
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::ReverseLineFeed()
{
return _pConApi->PrivateReverseLineFeed();
}
// Routine Description:
// - OSC Set Window Title - Sets the title of the window
// Arguments:
// - title - The string to set the title to. Must be null terminated.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetWindowTitle(std::wstring_view title)
{
return _pConApi->SetConsoleTitleW(title);
}
// - ASBSET - Creates and swaps to the alternate screen buffer. In virtual terminals, there exists both a "main"
// screen buffer and an alternate. ASBSET creates a new alternate, and switches to it. If there is an already
// existing alternate, it is discarded.
// Arguments:
// - None
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::UseAlternateScreenBuffer()
{
bool success = CursorSaveState();
if (success)
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
{
success = _pConApi->PrivateUseAlternateScreenBuffer();
if (success)
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
{
_usingAltBuffer = true;
}
}
return success;
}
// Routine Description:
// - ASBRST - From the alternate buffer, returns to the main screen buffer.
// From the main screen buffer, does nothing. The alternate is discarded.
// Arguments:
// - None
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::UseMainScreenBuffer()
{
bool success = _pConApi->PrivateUseMainScreenBuffer();
if (success)
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
{
_usingAltBuffer = false;
if (success)
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
{
success = CursorRestoreState();
Improve the DECSC/DECRC implementation (#3160) The current DECSC implementation only saves the cursor position and origin mode. This PR improves that functionality with additional support for saving the SGR attributes, as well as the active character set. It also fixes the way the saved state interacts with the alt buffer (private mode 1049), triggering a save when switching to the alt buffer, and a restore when switching back, and tracking the alt buffer state independently from the main state. In order to properly save and restore the SGR attributes, we first needed to add a pair of APIs in the `ConGetSet` interface which could round-trip the attributes with full 32-bit colors (the existing methods only work with legacy attributes). I then added a struct in the `AdaptDispatch` implementation to make it easier to manage all of the parameters that needed to be saved. This includes the cursor position and origin mode that we were already tracking, and now also the SGR text attributes and the active character set (via the `TermOutput` class). Two instances of this structure are required, since changes made to the saved state in the alt buffer need to be tracked separately from changes in the main buffer. I've added a boolean property that specifies whether we're in the alt buffer or not, and use that to decide which of the two instances we're working with. I also needed to explicitly trigger a save when switching to the alt buffer, and a restore when switching back, since we weren't already doing that (the existing implementation gave the impression that the state was saved, because each buffer has its own cursor position, but that doesn't properly match the XTerm behaviour). For the state tracking itself, we've now got two additional properties - the SGR attributes, which we obtain via the new private API, and the active character set, which we get from a local `AdaptDispatch` field. I'm saving the whole `TermOutput` class for the character set, since I'm hoping that will make it automatically supports future enhancements. When restoring the cursor position, there is also now a fix to handle the relative origin mode correctly. If the margins are changed between the position being saved and restored, it's possible for the cursor to end up outside of the new margins, which would be illegal. So there is now an additional step that clamps the Y coordinate within the margin boundaries if the origin mode is relative. # Validation I've added a couple of screen buffer tests which check that the various parameters are saved and restored as expected, as well as checking that the Y coordinate is clamped appropriately when the relative origin mode is set. I've also tested manually with vttest and confirmed that the _SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen features_)) is now working a lot better than it used to. Closes #148.
2019-11-05 22:35:50 +01:00
}
}
return success;
}
//Routine Description:
// HTS - sets a VT tab stop in the cursor's current column.
//Arguments:
// - None
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::HorizontalTabSet()
{
Reimplement the VT tab stop functionality (#5173) ## Summary of the Pull Request This is essentially a rewrite of the VT tab stop functionality, implemented entirely within the `AdaptDispatch` class. This significantly simplifies the `ConGetSet` interface, and should hopefully make it easier to share the functionality with the Windows Terminal VT implementation in the future. By removing the dependence on the `SCREEN_INFORMATION` class, it fixes the problem of the the tab state not being preserved when switching between the main and alternate buffers. And the new architecture also fixes problems with the tabs not being correctly initialized when the screen is resized. ## References This fixes one aspect of issue #3545. It also supersedes the fix for #411 (PR #2816). I'm hoping the simplification of `ConGetSet` will help with #3849. ## PR Checklist * [x] Closes #4669 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments In the new tab architecture, there is now a `vector<bool>` (__tabStopColumns_), which tracks whether any particular column is a tab stop or not. There is also a __initDefaultTabStops_ flag indicating whether the default tab stop positions need to be initialised when the screen is resized. The way this works, the vector is initially empty, and only initialized (to the current width of the screen) when it needs to be used. When the vector grows in size, the __initDefaultTabStops_ flag determines whether the new columns are set to false, or if every 8th column is set to true. By default we want the latter behaviour - newly revealed columns should have default tab stops assigned to them - so __initDefaultTabStops_ is set to true. However, after a `TBC 3` operation (i.e. we've cleared all tab stops), there should be no tab stops in any newly revealed columns, so __initDefaultTabStops_ is set to false. Note that the __tabStopColumns_ vector is never made smaller when the window is shrunk, and that way it can preserve the state of tab stops that are off screen, but which may come into range if the window is made bigger again. However, we can can still reset the vector completely after an `RIS` or `TBC 3` operation, since the state can then be reconstructed automatically based on just the __initDefaultTabStops_ flag. ## Validation Steps Performed The original screen buffer tests had to be rewritten to set and query the tab stop state using escape sequences rather than interacting with the `SCREEN_INFORMATION` class directly, but otherwise the structure of most tests remained largely the same. However, the alt buffer test was significantly rewritten, since the original behaviour was incorrect, and the initialization test was dropped completely, since it was no longer applicable. The adapter tests were also dropped, since they were testing the `ConGetSet` interface which has now been removed. I also had to make an addition to the method setup of the screen buffer tests (making sure the viewport was appropriately initialized), since there were some tests (unrelated to tab stops) that were previously dependent on the state being set in the tab initialization test which has now been removed. I've manually tested the issue described in #4669 and confirmed that the tabs now produce the correct spacing after a resize.
2020-04-01 14:49:27 +02:00
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
const bool success = _pConApi->GetConsoleScreenBufferInfoEx(csbiex);
if (success)
{
const auto width = csbiex.dwSize.X;
const auto column = csbiex.dwCursorPosition.X;
_InitTabStopsForWidth(width);
_tabStopColumns.at(column) = true;
}
return success;
}
//Routine Description:
// CHT - performing a forwards tab. This will take the
// cursor to the tab stop following its current location. If there are no
// more tabs in this row, it will take it to the right side of the window.
// If it's already in the last column of the row, it will move it to the next line.
//Arguments:
// - numTabs - the number of tabs to perform
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::ForwardTab(const size_t numTabs)
{
Reimplement the VT tab stop functionality (#5173) ## Summary of the Pull Request This is essentially a rewrite of the VT tab stop functionality, implemented entirely within the `AdaptDispatch` class. This significantly simplifies the `ConGetSet` interface, and should hopefully make it easier to share the functionality with the Windows Terminal VT implementation in the future. By removing the dependence on the `SCREEN_INFORMATION` class, it fixes the problem of the the tab state not being preserved when switching between the main and alternate buffers. And the new architecture also fixes problems with the tabs not being correctly initialized when the screen is resized. ## References This fixes one aspect of issue #3545. It also supersedes the fix for #411 (PR #2816). I'm hoping the simplification of `ConGetSet` will help with #3849. ## PR Checklist * [x] Closes #4669 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments In the new tab architecture, there is now a `vector<bool>` (__tabStopColumns_), which tracks whether any particular column is a tab stop or not. There is also a __initDefaultTabStops_ flag indicating whether the default tab stop positions need to be initialised when the screen is resized. The way this works, the vector is initially empty, and only initialized (to the current width of the screen) when it needs to be used. When the vector grows in size, the __initDefaultTabStops_ flag determines whether the new columns are set to false, or if every 8th column is set to true. By default we want the latter behaviour - newly revealed columns should have default tab stops assigned to them - so __initDefaultTabStops_ is set to true. However, after a `TBC 3` operation (i.e. we've cleared all tab stops), there should be no tab stops in any newly revealed columns, so __initDefaultTabStops_ is set to false. Note that the __tabStopColumns_ vector is never made smaller when the window is shrunk, and that way it can preserve the state of tab stops that are off screen, but which may come into range if the window is made bigger again. However, we can can still reset the vector completely after an `RIS` or `TBC 3` operation, since the state can then be reconstructed automatically based on just the __initDefaultTabStops_ flag. ## Validation Steps Performed The original screen buffer tests had to be rewritten to set and query the tab stop state using escape sequences rather than interacting with the `SCREEN_INFORMATION` class directly, but otherwise the structure of most tests remained largely the same. However, the alt buffer test was significantly rewritten, since the original behaviour was incorrect, and the initialization test was dropped completely, since it was no longer applicable. The adapter tests were also dropped, since they were testing the `ConGetSet` interface which has now been removed. I also had to make an addition to the method setup of the screen buffer tests (making sure the viewport was appropriately initialized), since there were some tests (unrelated to tab stops) that were previously dependent on the state being set in the tab initialization test which has now been removed. I've manually tested the issue described in #4669 and confirmed that the tabs now produce the correct spacing after a resize.
2020-04-01 14:49:27 +02:00
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
bool success = _pConApi->GetConsoleScreenBufferInfoEx(csbiex);
if (success)
{
const auto width = csbiex.dwSize.X;
const auto row = csbiex.dwCursorPosition.Y;
auto column = csbiex.dwCursorPosition.X;
auto tabsPerformed = 0u;
_InitTabStopsForWidth(width);
while (column + 1 < width && tabsPerformed < numTabs)
{
column++;
if (til::at(_tabStopColumns, column))
{
tabsPerformed++;
}
}
success = _pConApi->SetConsoleCursorPosition({ column, row });
}
return success;
}
//Routine Description:
// CBT - performing a backwards tab. This will take the cursor to the tab stop
// previous to its current location. It will not reverse line feed.
//Arguments:
// - numTabs - the number of tabs to perform
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::BackwardsTab(const size_t numTabs)
{
Reimplement the VT tab stop functionality (#5173) ## Summary of the Pull Request This is essentially a rewrite of the VT tab stop functionality, implemented entirely within the `AdaptDispatch` class. This significantly simplifies the `ConGetSet` interface, and should hopefully make it easier to share the functionality with the Windows Terminal VT implementation in the future. By removing the dependence on the `SCREEN_INFORMATION` class, it fixes the problem of the the tab state not being preserved when switching between the main and alternate buffers. And the new architecture also fixes problems with the tabs not being correctly initialized when the screen is resized. ## References This fixes one aspect of issue #3545. It also supersedes the fix for #411 (PR #2816). I'm hoping the simplification of `ConGetSet` will help with #3849. ## PR Checklist * [x] Closes #4669 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments In the new tab architecture, there is now a `vector<bool>` (__tabStopColumns_), which tracks whether any particular column is a tab stop or not. There is also a __initDefaultTabStops_ flag indicating whether the default tab stop positions need to be initialised when the screen is resized. The way this works, the vector is initially empty, and only initialized (to the current width of the screen) when it needs to be used. When the vector grows in size, the __initDefaultTabStops_ flag determines whether the new columns are set to false, or if every 8th column is set to true. By default we want the latter behaviour - newly revealed columns should have default tab stops assigned to them - so __initDefaultTabStops_ is set to true. However, after a `TBC 3` operation (i.e. we've cleared all tab stops), there should be no tab stops in any newly revealed columns, so __initDefaultTabStops_ is set to false. Note that the __tabStopColumns_ vector is never made smaller when the window is shrunk, and that way it can preserve the state of tab stops that are off screen, but which may come into range if the window is made bigger again. However, we can can still reset the vector completely after an `RIS` or `TBC 3` operation, since the state can then be reconstructed automatically based on just the __initDefaultTabStops_ flag. ## Validation Steps Performed The original screen buffer tests had to be rewritten to set and query the tab stop state using escape sequences rather than interacting with the `SCREEN_INFORMATION` class directly, but otherwise the structure of most tests remained largely the same. However, the alt buffer test was significantly rewritten, since the original behaviour was incorrect, and the initialization test was dropped completely, since it was no longer applicable. The adapter tests were also dropped, since they were testing the `ConGetSet` interface which has now been removed. I also had to make an addition to the method setup of the screen buffer tests (making sure the viewport was appropriately initialized), since there were some tests (unrelated to tab stops) that were previously dependent on the state being set in the tab initialization test which has now been removed. I've manually tested the issue described in #4669 and confirmed that the tabs now produce the correct spacing after a resize.
2020-04-01 14:49:27 +02:00
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
bool success = _pConApi->GetConsoleScreenBufferInfoEx(csbiex);
if (success)
{
const auto width = csbiex.dwSize.X;
const auto row = csbiex.dwCursorPosition.Y;
auto column = csbiex.dwCursorPosition.X;
auto tabsPerformed = 0u;
_InitTabStopsForWidth(width);
while (column > 0 && tabsPerformed < numTabs)
{
column--;
if (til::at(_tabStopColumns, column))
{
tabsPerformed++;
}
}
success = _pConApi->SetConsoleCursorPosition({ column, row });
}
return success;
}
//Routine Description:
// TBC - Used to clear set tab stops. ClearType ClearCurrentColumn (0) results
// in clearing only the tab stop in the cursor's current column, if there
// is one. ClearAllColumns (3) results in resetting all set tab stops.
//Arguments:
// - clearType - Whether to clear the current column, or all columns, defined in DispatchTypes::TabClearType
// Return value:
// True if handled successfully. False otherwise.
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
bool AdaptDispatch::TabClear(const DispatchTypes::TabClearType clearType)
{
bool success = false;
switch (clearType)
{
case DispatchTypes::TabClearType::ClearCurrentColumn:
Reimplement the VT tab stop functionality (#5173) ## Summary of the Pull Request This is essentially a rewrite of the VT tab stop functionality, implemented entirely within the `AdaptDispatch` class. This significantly simplifies the `ConGetSet` interface, and should hopefully make it easier to share the functionality with the Windows Terminal VT implementation in the future. By removing the dependence on the `SCREEN_INFORMATION` class, it fixes the problem of the the tab state not being preserved when switching between the main and alternate buffers. And the new architecture also fixes problems with the tabs not being correctly initialized when the screen is resized. ## References This fixes one aspect of issue #3545. It also supersedes the fix for #411 (PR #2816). I'm hoping the simplification of `ConGetSet` will help with #3849. ## PR Checklist * [x] Closes #4669 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments In the new tab architecture, there is now a `vector<bool>` (__tabStopColumns_), which tracks whether any particular column is a tab stop or not. There is also a __initDefaultTabStops_ flag indicating whether the default tab stop positions need to be initialised when the screen is resized. The way this works, the vector is initially empty, and only initialized (to the current width of the screen) when it needs to be used. When the vector grows in size, the __initDefaultTabStops_ flag determines whether the new columns are set to false, or if every 8th column is set to true. By default we want the latter behaviour - newly revealed columns should have default tab stops assigned to them - so __initDefaultTabStops_ is set to true. However, after a `TBC 3` operation (i.e. we've cleared all tab stops), there should be no tab stops in any newly revealed columns, so __initDefaultTabStops_ is set to false. Note that the __tabStopColumns_ vector is never made smaller when the window is shrunk, and that way it can preserve the state of tab stops that are off screen, but which may come into range if the window is made bigger again. However, we can can still reset the vector completely after an `RIS` or `TBC 3` operation, since the state can then be reconstructed automatically based on just the __initDefaultTabStops_ flag. ## Validation Steps Performed The original screen buffer tests had to be rewritten to set and query the tab stop state using escape sequences rather than interacting with the `SCREEN_INFORMATION` class directly, but otherwise the structure of most tests remained largely the same. However, the alt buffer test was significantly rewritten, since the original behaviour was incorrect, and the initialization test was dropped completely, since it was no longer applicable. The adapter tests were also dropped, since they were testing the `ConGetSet` interface which has now been removed. I also had to make an addition to the method setup of the screen buffer tests (making sure the viewport was appropriately initialized), since there were some tests (unrelated to tab stops) that were previously dependent on the state being set in the tab initialization test which has now been removed. I've manually tested the issue described in #4669 and confirmed that the tabs now produce the correct spacing after a resize.
2020-04-01 14:49:27 +02:00
success = _ClearSingleTabStop();
break;
case DispatchTypes::TabClearType::ClearAllColumns:
Reimplement the VT tab stop functionality (#5173) ## Summary of the Pull Request This is essentially a rewrite of the VT tab stop functionality, implemented entirely within the `AdaptDispatch` class. This significantly simplifies the `ConGetSet` interface, and should hopefully make it easier to share the functionality with the Windows Terminal VT implementation in the future. By removing the dependence on the `SCREEN_INFORMATION` class, it fixes the problem of the the tab state not being preserved when switching between the main and alternate buffers. And the new architecture also fixes problems with the tabs not being correctly initialized when the screen is resized. ## References This fixes one aspect of issue #3545. It also supersedes the fix for #411 (PR #2816). I'm hoping the simplification of `ConGetSet` will help with #3849. ## PR Checklist * [x] Closes #4669 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments In the new tab architecture, there is now a `vector<bool>` (__tabStopColumns_), which tracks whether any particular column is a tab stop or not. There is also a __initDefaultTabStops_ flag indicating whether the default tab stop positions need to be initialised when the screen is resized. The way this works, the vector is initially empty, and only initialized (to the current width of the screen) when it needs to be used. When the vector grows in size, the __initDefaultTabStops_ flag determines whether the new columns are set to false, or if every 8th column is set to true. By default we want the latter behaviour - newly revealed columns should have default tab stops assigned to them - so __initDefaultTabStops_ is set to true. However, after a `TBC 3` operation (i.e. we've cleared all tab stops), there should be no tab stops in any newly revealed columns, so __initDefaultTabStops_ is set to false. Note that the __tabStopColumns_ vector is never made smaller when the window is shrunk, and that way it can preserve the state of tab stops that are off screen, but which may come into range if the window is made bigger again. However, we can can still reset the vector completely after an `RIS` or `TBC 3` operation, since the state can then be reconstructed automatically based on just the __initDefaultTabStops_ flag. ## Validation Steps Performed The original screen buffer tests had to be rewritten to set and query the tab stop state using escape sequences rather than interacting with the `SCREEN_INFORMATION` class directly, but otherwise the structure of most tests remained largely the same. However, the alt buffer test was significantly rewritten, since the original behaviour was incorrect, and the initialization test was dropped completely, since it was no longer applicable. The adapter tests were also dropped, since they were testing the `ConGetSet` interface which has now been removed. I also had to make an addition to the method setup of the screen buffer tests (making sure the viewport was appropriately initialized), since there were some tests (unrelated to tab stops) that were previously dependent on the state being set in the tab initialization test which has now been removed. I've manually tested the issue described in #4669 and confirmed that the tabs now produce the correct spacing after a resize.
2020-04-01 14:49:27 +02:00
success = _ClearAllTabStops();
break;
default:
success = false;
break;
}
return success;
}
Reimplement the VT tab stop functionality (#5173) ## Summary of the Pull Request This is essentially a rewrite of the VT tab stop functionality, implemented entirely within the `AdaptDispatch` class. This significantly simplifies the `ConGetSet` interface, and should hopefully make it easier to share the functionality with the Windows Terminal VT implementation in the future. By removing the dependence on the `SCREEN_INFORMATION` class, it fixes the problem of the the tab state not being preserved when switching between the main and alternate buffers. And the new architecture also fixes problems with the tabs not being correctly initialized when the screen is resized. ## References This fixes one aspect of issue #3545. It also supersedes the fix for #411 (PR #2816). I'm hoping the simplification of `ConGetSet` will help with #3849. ## PR Checklist * [x] Closes #4669 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments In the new tab architecture, there is now a `vector<bool>` (__tabStopColumns_), which tracks whether any particular column is a tab stop or not. There is also a __initDefaultTabStops_ flag indicating whether the default tab stop positions need to be initialised when the screen is resized. The way this works, the vector is initially empty, and only initialized (to the current width of the screen) when it needs to be used. When the vector grows in size, the __initDefaultTabStops_ flag determines whether the new columns are set to false, or if every 8th column is set to true. By default we want the latter behaviour - newly revealed columns should have default tab stops assigned to them - so __initDefaultTabStops_ is set to true. However, after a `TBC 3` operation (i.e. we've cleared all tab stops), there should be no tab stops in any newly revealed columns, so __initDefaultTabStops_ is set to false. Note that the __tabStopColumns_ vector is never made smaller when the window is shrunk, and that way it can preserve the state of tab stops that are off screen, but which may come into range if the window is made bigger again. However, we can can still reset the vector completely after an `RIS` or `TBC 3` operation, since the state can then be reconstructed automatically based on just the __initDefaultTabStops_ flag. ## Validation Steps Performed The original screen buffer tests had to be rewritten to set and query the tab stop state using escape sequences rather than interacting with the `SCREEN_INFORMATION` class directly, but otherwise the structure of most tests remained largely the same. However, the alt buffer test was significantly rewritten, since the original behaviour was incorrect, and the initialization test was dropped completely, since it was no longer applicable. The adapter tests were also dropped, since they were testing the `ConGetSet` interface which has now been removed. I also had to make an addition to the method setup of the screen buffer tests (making sure the viewport was appropriately initialized), since there were some tests (unrelated to tab stops) that were previously dependent on the state being set in the tab initialization test which has now been removed. I've manually tested the issue described in #4669 and confirmed that the tabs now produce the correct spacing after a resize.
2020-04-01 14:49:27 +02:00
// Routine Description:
// - Clears the tab stop in the cursor's current column, if there is one.
// Arguments:
// - <none>
// Return value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_ClearSingleTabStop()
{
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
const bool success = _pConApi->GetConsoleScreenBufferInfoEx(csbiex);
if (success)
{
const auto width = csbiex.dwSize.X;
const auto column = csbiex.dwCursorPosition.X;
_InitTabStopsForWidth(width);
_tabStopColumns.at(column) = false;
}
return success;
}
// Routine Description:
// - Clears all tab stops and resets the _initDefaultTabStops flag to indicate
// that they shouldn't be reinitialized at the default positions.
// Arguments:
// - <none>
// Return value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_ClearAllTabStops() noexcept
{
_tabStopColumns.clear();
_initDefaultTabStops = false;
return true;
}
// Routine Description:
// - Clears all tab stops and sets the _initDefaultTabStops flag to indicate
// that the default positions should be reinitialized when needed.
// Arguments:
// - <none>
// Return value:
// - <none>
void AdaptDispatch::_ResetTabStops() noexcept
{
_tabStopColumns.clear();
_initDefaultTabStops = true;
}
// Routine Description:
// - Resizes the _tabStopColumns table so it's large enough to support the
// current screen width, initializing tab stops every 8 columns in the
// newly allocated space, iff the _initDefaultTabStops flag is set.
// Arguments:
// - width - the width of the screen buffer that we need to accomodate
// Return value:
// - <none>
void AdaptDispatch::_InitTabStopsForWidth(const size_t width)
{
const auto initialWidth = _tabStopColumns.size();
if (width > initialWidth)
{
_tabStopColumns.resize(width);
if (_initDefaultTabStops)
{
for (auto column = 8u; column < _tabStopColumns.size(); column += 8)
{
if (column >= initialWidth)
{
til::at(_tabStopColumns, column) = true;
}
}
}
}
}
//Routine Description:
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
// DOCS - Selects the coding system through which character sets are activated.
Disable the acceptance of C1 control codes by default (#11690) There are some code pages with "unmapped" code points in the C1 range, which results in them being translated into Unicode C1 control codes, even though that is not their intended use. To avoid having these characters triggering unintentional escape sequences, this PR now disables C1 controls by default. Switching to ISO-2022 encoding will re-enable them, though, since that is the most likely scenario in which they would be required. They can also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape sequence. What I've done is add a new mode to the `StateMachine` class that controls whether C1 code points are interpreted as control characters or not. When disabled, these code points are simply dropped from the output, similar to the way a `NUL` is interpreted. This isn't exactly the way they were handled in the v1 console (which I think replaces them with the font _notdef_ glyph), but it matches the XTerm behavior, which seems more appropriate considering this is in VT mode. And it's worth noting that Windows Explorer seems to work the same way. As mentioned above, the mode can be enabled by designating the ISO-2022 coding system with a `DOCS` sequence, and it will be disabled again when UTF-8 is designated. You can also enable it explicitly with a `DECAC1` sequence (originally this was actually a DEC printer sequence, but it doesn't seem unreasonable to use it in a terminal). I've also extended the operations that save and restore "cursor state" (e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode, since it's closely tied to the code page and character sets which are also saved there. Similarly, when a `DECSTR` sequence resets the code page and character sets, I've now made it reset the C1 mode as well. I should note that the new `StateMachine` mode is controlled via a generic `SetParserMode` method (with a matching API in the `ConGetSet` interface) to allow for easier addition of other modes in the future. And I've reimplemented the existing ANSI/VT52 mode in terms of these generic methods instead of it having to have its own separate APIs. ## Validation Steps Performed Some of the unit tests for OSC sequences were using a C1 `0x9C` for the string terminator, which doesn't work by default anymore. Since that's not a good practice anyway, I thought it best to change those to a standard 7-bit terminator. However, in tests that were explicitly validating the C1 controls, I've just enabled the C1 parser mode at the start of the tests in order to get them working again. There were also some ANSI mode adapter tests that had to be updated to account for the fact that it has now been reimplemented in terms of the `SetParserMode` API. I've added a new state machine test to validate the changes in behavior when the C1 parser mode is enabled or disabled. And I've added an adapter test to verify that the `DesignateCodingSystems` and `AcceptC1Controls` methods toggle the C1 parser mode as expected. I've manually verified the test cases in #10069 and #10310 to confirm that they're no longer triggering control sequences by default. Although, as I explained above, the C1 code points are completely dropped from the output rather than displayed as _notdef_ glyphs. I think this is a reasonable compromise though. Closes #10069 Closes #10310
2021-11-18 00:40:31 +01:00
// When ISO2022 is selected, the code page is set to ISO-8859-1, C1 control
// codes are accepted, and both GL and GR areas of the code table can be
// remapped. When UTF8 is selected, the code page is set to UTF-8, the C1
// control codes are disabled, and only the GL area can be remapped.
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
//Arguments:
// - codingSystem - The coding system that will be selected.
// Return value:
// True if handled successfully. False otherwise.
Refactor VT control sequence identification (#7304) This PR changes the way VT control sequences are identified and dispatched, to be more efficient and easier to extend. Instead of parsing the intermediate characters into a vector, and then having to identify a sequence using both that vector and the final char, we now use just a single `uint64_t` value as the identifier. The way the identifier is constructed is by taking the private parameter prefix, each of the intermediate characters, and then the final character, and shifting them into a 64-bit integer one byte at a time, in reverse order. For example, the `DECTLTC` control has a private parameter prefix of `?`, one intermediate of `'`, and a final character of `s`. The ASCII values of those characters are `0x3F`, `0x27`, and `0x73` respectively, and reversing them gets you 0x73273F, so that would then be the identifier for the control. The reason for storing them in reverse order, is because sometimes we need to look at the first intermediate to determine the operation, and treat the rest of the sequence as a kind of sub-identifier (the character set designation sequences are one example of this). When in reverse order, this can easily be achieved by masking off the low byte to get the first intermediate, and then shifting the value right by 8 bits to get a new identifier with the rest of the sequence. With 64 bits we have enough space for a private prefix, six intermediates, and the final char, which is way more than we should ever need (the _DEC STD 070_ specification recommends supporting at least three intermediates, but in practice we're unlikely to see more than two). With this new way of identifying controls, it should now be possible for every action code to be unique (for the most part). So I've also used this PR to clean up the action codes a bit, splitting the codes for the escape sequences from the control sequences, and sorting them into alphabetical order (which also does a reasonable job of clustering associated controls). ## Validation Steps Performed I think the existing unit tests should be good enough to confirm that all sequences are still being dispatched correctly. However, I've also manually tested a number of sequences to make sure they were still working as expected, in particular those that used intermediates, since they were the most affected by the dispatch code refactoring. Since these changes also affected the input state machine, I've done some manual testing of the conpty keyboard handling (both with and without the new Win32 input mode enabled) to make sure the keyboard VT sequences were processed correctly. I've also manually tested the various VT mouse modes in Vttest to confirm that they were still working correctly too. Closes #7276
2020-08-18 20:57:52 +02:00
bool AdaptDispatch::DesignateCodingSystem(const VTID codingSystem)
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
{
// If we haven't previously saved the initial code page, do so now.
// This will be used to restore the code page in response to a reset.
if (!_initialCodePage.has_value())
{
unsigned int currentCodePage;
_pConApi->GetConsoleOutputCP(currentCodePage);
_initialCodePage = currentCodePage;
}
bool success = false;
switch (codingSystem)
{
case DispatchTypes::CodingSystem::ISO2022:
success = _pConApi->SetConsoleOutputCP(28591);
if (success)
{
Disable the acceptance of C1 control codes by default (#11690) There are some code pages with "unmapped" code points in the C1 range, which results in them being translated into Unicode C1 control codes, even though that is not their intended use. To avoid having these characters triggering unintentional escape sequences, this PR now disables C1 controls by default. Switching to ISO-2022 encoding will re-enable them, though, since that is the most likely scenario in which they would be required. They can also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape sequence. What I've done is add a new mode to the `StateMachine` class that controls whether C1 code points are interpreted as control characters or not. When disabled, these code points are simply dropped from the output, similar to the way a `NUL` is interpreted. This isn't exactly the way they were handled in the v1 console (which I think replaces them with the font _notdef_ glyph), but it matches the XTerm behavior, which seems more appropriate considering this is in VT mode. And it's worth noting that Windows Explorer seems to work the same way. As mentioned above, the mode can be enabled by designating the ISO-2022 coding system with a `DOCS` sequence, and it will be disabled again when UTF-8 is designated. You can also enable it explicitly with a `DECAC1` sequence (originally this was actually a DEC printer sequence, but it doesn't seem unreasonable to use it in a terminal). I've also extended the operations that save and restore "cursor state" (e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode, since it's closely tied to the code page and character sets which are also saved there. Similarly, when a `DECSTR` sequence resets the code page and character sets, I've now made it reset the C1 mode as well. I should note that the new `StateMachine` mode is controlled via a generic `SetParserMode` method (with a matching API in the `ConGetSet` interface) to allow for easier addition of other modes in the future. And I've reimplemented the existing ANSI/VT52 mode in terms of these generic methods instead of it having to have its own separate APIs. ## Validation Steps Performed Some of the unit tests for OSC sequences were using a C1 `0x9C` for the string terminator, which doesn't work by default anymore. Since that's not a good practice anyway, I thought it best to change those to a standard 7-bit terminator. However, in tests that were explicitly validating the C1 controls, I've just enabled the C1 parser mode at the start of the tests in order to get them working again. There were also some ANSI mode adapter tests that had to be updated to account for the fact that it has now been reimplemented in terms of the `SetParserMode` API. I've added a new state machine test to validate the changes in behavior when the C1 parser mode is enabled or disabled. And I've added an adapter test to verify that the `DesignateCodingSystems` and `AcceptC1Controls` methods toggle the C1 parser mode as expected. I've manually verified the test cases in #10069 and #10310 to confirm that they're no longer triggering control sequences by default. Although, as I explained above, the C1 code points are completely dropped from the output rather than displayed as _notdef_ glyphs. I think this is a reasonable compromise though. Closes #10069 Closes #10310
2021-11-18 00:40:31 +01:00
AcceptC1Controls(true);
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
_termOutput.EnableGrTranslation(true);
}
break;
case DispatchTypes::CodingSystem::UTF8:
success = _pConApi->SetConsoleOutputCP(CP_UTF8);
if (success)
{
Disable the acceptance of C1 control codes by default (#11690) There are some code pages with "unmapped" code points in the C1 range, which results in them being translated into Unicode C1 control codes, even though that is not their intended use. To avoid having these characters triggering unintentional escape sequences, this PR now disables C1 controls by default. Switching to ISO-2022 encoding will re-enable them, though, since that is the most likely scenario in which they would be required. They can also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape sequence. What I've done is add a new mode to the `StateMachine` class that controls whether C1 code points are interpreted as control characters or not. When disabled, these code points are simply dropped from the output, similar to the way a `NUL` is interpreted. This isn't exactly the way they were handled in the v1 console (which I think replaces them with the font _notdef_ glyph), but it matches the XTerm behavior, which seems more appropriate considering this is in VT mode. And it's worth noting that Windows Explorer seems to work the same way. As mentioned above, the mode can be enabled by designating the ISO-2022 coding system with a `DOCS` sequence, and it will be disabled again when UTF-8 is designated. You can also enable it explicitly with a `DECAC1` sequence (originally this was actually a DEC printer sequence, but it doesn't seem unreasonable to use it in a terminal). I've also extended the operations that save and restore "cursor state" (e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode, since it's closely tied to the code page and character sets which are also saved there. Similarly, when a `DECSTR` sequence resets the code page and character sets, I've now made it reset the C1 mode as well. I should note that the new `StateMachine` mode is controlled via a generic `SetParserMode` method (with a matching API in the `ConGetSet` interface) to allow for easier addition of other modes in the future. And I've reimplemented the existing ANSI/VT52 mode in terms of these generic methods instead of it having to have its own separate APIs. ## Validation Steps Performed Some of the unit tests for OSC sequences were using a C1 `0x9C` for the string terminator, which doesn't work by default anymore. Since that's not a good practice anyway, I thought it best to change those to a standard 7-bit terminator. However, in tests that were explicitly validating the C1 controls, I've just enabled the C1 parser mode at the start of the tests in order to get them working again. There were also some ANSI mode adapter tests that had to be updated to account for the fact that it has now been reimplemented in terms of the `SetParserMode` API. I've added a new state machine test to validate the changes in behavior when the C1 parser mode is enabled or disabled. And I've added an adapter test to verify that the `DesignateCodingSystems` and `AcceptC1Controls` methods toggle the C1 parser mode as expected. I've manually verified the test cases in #10069 and #10310 to confirm that they're no longer triggering control sequences by default. Although, as I explained above, the C1 code points are completely dropped from the output rather than displayed as _notdef_ glyphs. I think this is a reasonable compromise though. Closes #10069 Closes #10310
2021-11-18 00:40:31 +01:00
AcceptC1Controls(false);
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
_termOutput.EnableGrTranslation(false);
}
break;
default:
success = false;
break;
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
}
return success;
}
//Routine Description:
// Designate Charset - Selects a specific 94-character set into one of the four G-sets.
// See http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h3-Controls-beginning-with-ESC
// for a list of all charsets and their codes.
// If the specified charset is unsupported, we do nothing (remain on the current one)
//Arguments:
// - gsetNumber - The G-set into which the charset will be selected.
Refactor VT control sequence identification (#7304) This PR changes the way VT control sequences are identified and dispatched, to be more efficient and easier to extend. Instead of parsing the intermediate characters into a vector, and then having to identify a sequence using both that vector and the final char, we now use just a single `uint64_t` value as the identifier. The way the identifier is constructed is by taking the private parameter prefix, each of the intermediate characters, and then the final character, and shifting them into a 64-bit integer one byte at a time, in reverse order. For example, the `DECTLTC` control has a private parameter prefix of `?`, one intermediate of `'`, and a final character of `s`. The ASCII values of those characters are `0x3F`, `0x27`, and `0x73` respectively, and reversing them gets you 0x73273F, so that would then be the identifier for the control. The reason for storing them in reverse order, is because sometimes we need to look at the first intermediate to determine the operation, and treat the rest of the sequence as a kind of sub-identifier (the character set designation sequences are one example of this). When in reverse order, this can easily be achieved by masking off the low byte to get the first intermediate, and then shifting the value right by 8 bits to get a new identifier with the rest of the sequence. With 64 bits we have enough space for a private prefix, six intermediates, and the final char, which is way more than we should ever need (the _DEC STD 070_ specification recommends supporting at least three intermediates, but in practice we're unlikely to see more than two). With this new way of identifying controls, it should now be possible for every action code to be unique (for the most part). So I've also used this PR to clean up the action codes a bit, splitting the codes for the escape sequences from the control sequences, and sorting them into alphabetical order (which also does a reasonable job of clustering associated controls). ## Validation Steps Performed I think the existing unit tests should be good enough to confirm that all sequences are still being dispatched correctly. However, I've also manually tested a number of sequences to make sure they were still working as expected, in particular those that used intermediates, since they were the most affected by the dispatch code refactoring. Since these changes also affected the input state machine, I've done some manual testing of the conpty keyboard handling (both with and without the new Win32 input mode enabled) to make sure the keyboard VT sequences were processed correctly. I've also manually tested the various VT mouse modes in Vttest to confirm that they were still working correctly too. Closes #7276
2020-08-18 20:57:52 +02:00
// - charset - The identifier indicating the charset that will be used.
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
// Return value:
// True if handled successfully. False otherwise.
Refactor VT control sequence identification (#7304) This PR changes the way VT control sequences are identified and dispatched, to be more efficient and easier to extend. Instead of parsing the intermediate characters into a vector, and then having to identify a sequence using both that vector and the final char, we now use just a single `uint64_t` value as the identifier. The way the identifier is constructed is by taking the private parameter prefix, each of the intermediate characters, and then the final character, and shifting them into a 64-bit integer one byte at a time, in reverse order. For example, the `DECTLTC` control has a private parameter prefix of `?`, one intermediate of `'`, and a final character of `s`. The ASCII values of those characters are `0x3F`, `0x27`, and `0x73` respectively, and reversing them gets you 0x73273F, so that would then be the identifier for the control. The reason for storing them in reverse order, is because sometimes we need to look at the first intermediate to determine the operation, and treat the rest of the sequence as a kind of sub-identifier (the character set designation sequences are one example of this). When in reverse order, this can easily be achieved by masking off the low byte to get the first intermediate, and then shifting the value right by 8 bits to get a new identifier with the rest of the sequence. With 64 bits we have enough space for a private prefix, six intermediates, and the final char, which is way more than we should ever need (the _DEC STD 070_ specification recommends supporting at least three intermediates, but in practice we're unlikely to see more than two). With this new way of identifying controls, it should now be possible for every action code to be unique (for the most part). So I've also used this PR to clean up the action codes a bit, splitting the codes for the escape sequences from the control sequences, and sorting them into alphabetical order (which also does a reasonable job of clustering associated controls). ## Validation Steps Performed I think the existing unit tests should be good enough to confirm that all sequences are still being dispatched correctly. However, I've also manually tested a number of sequences to make sure they were still working as expected, in particular those that used intermediates, since they were the most affected by the dispatch code refactoring. Since these changes also affected the input state machine, I've done some manual testing of the conpty keyboard handling (both with and without the new Win32 input mode enabled) to make sure the keyboard VT sequences were processed correctly. I've also manually tested the various VT mouse modes in Vttest to confirm that they were still working correctly too. Closes #7276
2020-08-18 20:57:52 +02:00
bool AdaptDispatch::Designate94Charset(const size_t gsetNumber, const VTID charset)
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
{
return _termOutput.Designate94Charset(gsetNumber, charset);
}
//Routine Description:
// Designate Charset - Selects a specific 96-character set into one of the four G-sets.
// See http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h3-Controls-beginning-with-ESC
// for a list of all charsets and their codes.
// If the specified charset is unsupported, we do nothing (remain on the current one)
//Arguments:
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
// - gsetNumber - The G-set into which the charset will be selected.
Refactor VT control sequence identification (#7304) This PR changes the way VT control sequences are identified and dispatched, to be more efficient and easier to extend. Instead of parsing the intermediate characters into a vector, and then having to identify a sequence using both that vector and the final char, we now use just a single `uint64_t` value as the identifier. The way the identifier is constructed is by taking the private parameter prefix, each of the intermediate characters, and then the final character, and shifting them into a 64-bit integer one byte at a time, in reverse order. For example, the `DECTLTC` control has a private parameter prefix of `?`, one intermediate of `'`, and a final character of `s`. The ASCII values of those characters are `0x3F`, `0x27`, and `0x73` respectively, and reversing them gets you 0x73273F, so that would then be the identifier for the control. The reason for storing them in reverse order, is because sometimes we need to look at the first intermediate to determine the operation, and treat the rest of the sequence as a kind of sub-identifier (the character set designation sequences are one example of this). When in reverse order, this can easily be achieved by masking off the low byte to get the first intermediate, and then shifting the value right by 8 bits to get a new identifier with the rest of the sequence. With 64 bits we have enough space for a private prefix, six intermediates, and the final char, which is way more than we should ever need (the _DEC STD 070_ specification recommends supporting at least three intermediates, but in practice we're unlikely to see more than two). With this new way of identifying controls, it should now be possible for every action code to be unique (for the most part). So I've also used this PR to clean up the action codes a bit, splitting the codes for the escape sequences from the control sequences, and sorting them into alphabetical order (which also does a reasonable job of clustering associated controls). ## Validation Steps Performed I think the existing unit tests should be good enough to confirm that all sequences are still being dispatched correctly. However, I've also manually tested a number of sequences to make sure they were still working as expected, in particular those that used intermediates, since they were the most affected by the dispatch code refactoring. Since these changes also affected the input state machine, I've done some manual testing of the conpty keyboard handling (both with and without the new Win32 input mode enabled) to make sure the keyboard VT sequences were processed correctly. I've also manually tested the various VT mouse modes in Vttest to confirm that they were still working correctly too. Closes #7276
2020-08-18 20:57:52 +02:00
// - charset - The identifier indicating the charset that will be used.
// Return value:
// True if handled successfully. False otherwise.
Refactor VT control sequence identification (#7304) This PR changes the way VT control sequences are identified and dispatched, to be more efficient and easier to extend. Instead of parsing the intermediate characters into a vector, and then having to identify a sequence using both that vector and the final char, we now use just a single `uint64_t` value as the identifier. The way the identifier is constructed is by taking the private parameter prefix, each of the intermediate characters, and then the final character, and shifting them into a 64-bit integer one byte at a time, in reverse order. For example, the `DECTLTC` control has a private parameter prefix of `?`, one intermediate of `'`, and a final character of `s`. The ASCII values of those characters are `0x3F`, `0x27`, and `0x73` respectively, and reversing them gets you 0x73273F, so that would then be the identifier for the control. The reason for storing them in reverse order, is because sometimes we need to look at the first intermediate to determine the operation, and treat the rest of the sequence as a kind of sub-identifier (the character set designation sequences are one example of this). When in reverse order, this can easily be achieved by masking off the low byte to get the first intermediate, and then shifting the value right by 8 bits to get a new identifier with the rest of the sequence. With 64 bits we have enough space for a private prefix, six intermediates, and the final char, which is way more than we should ever need (the _DEC STD 070_ specification recommends supporting at least three intermediates, but in practice we're unlikely to see more than two). With this new way of identifying controls, it should now be possible for every action code to be unique (for the most part). So I've also used this PR to clean up the action codes a bit, splitting the codes for the escape sequences from the control sequences, and sorting them into alphabetical order (which also does a reasonable job of clustering associated controls). ## Validation Steps Performed I think the existing unit tests should be good enough to confirm that all sequences are still being dispatched correctly. However, I've also manually tested a number of sequences to make sure they were still working as expected, in particular those that used intermediates, since they were the most affected by the dispatch code refactoring. Since these changes also affected the input state machine, I've done some manual testing of the conpty keyboard handling (both with and without the new Win32 input mode enabled) to make sure the keyboard VT sequences were processed correctly. I've also manually tested the various VT mouse modes in Vttest to confirm that they were still working correctly too. Closes #7276
2020-08-18 20:57:52 +02:00
bool AdaptDispatch::Designate96Charset(const size_t gsetNumber, const VTID charset)
{
Improve support for VT character sets (#4496) This PR improves our VT character set support, enabling the [`SCS`] escape sequences to designate into all four G-sets with both 94- and 96-character sets, and supports invoking those G-sets into both the GL and GR areas of the code table, with [locking shifts] and [single shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the ISO-2022 coding system (which is what the VT character sets require), and adds support for a lot more characters sets, up to around the level of a VT510. [`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html [locking shifts]: https://vt100.net/docs/vt510-rm/LS.html [single shifts]: https://vt100.net/docs/vt510-rm/SS.html [`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems ## Detailed Description of the Pull Request / Additional comments To make it easier for us to declare a bunch of character sets, I've made a little `constexpr` class that can build up a mapping table from a base character set (ASCII or Latin1), along with a collection of mappings for the characters the deviate from the base set. Many of the character sets are simple variations of ASCII, so they're easy to define this way. This class then casts directly to a `wstring_view` which is how the translation tables are represented in most of the code. We have an array of four of these tables representing the four G-sets, two instances for the active left and right tables, and one instance for the single shift table. Initially we had just one `DesignateCharset` method, which could select the active character set. We now have two designate methods (for 94- and 96- character sets), and each takes a G-set number specifying the target of the designation, and a pair of characters identifying the character set that will be designated (at the higher VT levels, character sets are often identified by more than one character). There are then two new `LockingShift` methods to invoke these G-sets into either the GL or GR area of the code table, and a `SingleShift` method which invokes a G-set temporarily (for just the next character that is output). I should mention here that I had to make some changes to the state machine to make these single shift sequences work. The problem is that the input state machine treats `SS3` as the start of a control sequence, while the output state machine needs it to be dispatched immediately (it's literally the _Single Shift 3_ escape sequence). To make that work, I've added a `ParseControlSequenceAfterSs3` callback in the `IStateMachineEngine` interface to decide which behavior is appropriate. When it comes to mapping a character, it's simply an array reference into the appropriate `wstring_view` table. If the single shift table is set, that takes preference. Otherwise the GL table is used for characters in the range 0x20 to 0x7F, and the GR table for characters 0xA0 to 0xFF (technically some character sets will only map up to 0x7E and 0xFE, but that's easily controlled by the length of the `wstring_view`). The `DEL` character is a bit of a special case. By default it's meant to be ignored like the `NUL` character (it's essentially a time-fill character). However, it's possible that it could be remapped to a printable character in a 96-character set, so we need to check for that after the translation. This is handled in the `AdaptDispatch::Print` method, so it doesn't interfere with the primary `PrintString` code path. The biggest problem with this whole process, though, is that the GR mappings only really make sense if you have access to the raw output, but by the time the output gets to us, it would already have been translated to Unicode by the active code page. And in the case of UTF-8, the characters we eventually receive may originally have been composed from two or more code points. The way I've dealt with this was to disable the GR translations by default, and then added support for a pair of ISO-2022 `DOCS` sequences, which can switch the code page between UTF-8 and ISO-8859-1. When the code page is ISO-8859-1, we're essentially receiving the raw output bytes, so it's safe to enable the GR translations. This is not strictly correct ISO-2022 behavior, and there are edge cases where it's not going to work, but it's the best solution I could come up with. ## Validation Steps Performed As a result of the `SS3` changes in the state machine engine, I've had to move the existing `SS3` tests from the `OutputEngineTest` to the `InputEngineTest`, otherwise they would now fail (technically they should never have been output tests). I've added no additional unit tests, but I have done a lot of manual testing, and made sure we passed all the character set tests in Vttest (at least for the character sets we currently support). Note that this required a slightly hacked version of the app, since by default it doesn't expose a lot of the test to low-level terminals, and we currently identify as a VT100. Closes #3377 Closes #3487
2020-06-04 21:40:15 +02:00
return _termOutput.Designate96Charset(gsetNumber, charset);
}
//Routine Description:
// Locking Shift - Invoke one of the G-sets into the left half of the code table.
//Arguments:
// - gsetNumber - The G-set that will be invoked.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::LockingShift(const size_t gsetNumber)
{
return _termOutput.LockingShift(gsetNumber);
}
//Routine Description:
// Locking Shift Right - Invoke one of the G-sets into the right half of the code table.
//Arguments:
// - gsetNumber - The G-set that will be invoked.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::LockingShiftRight(const size_t gsetNumber)
{
return _termOutput.LockingShiftRight(gsetNumber);
}
//Routine Description:
// Single Shift - Temporarily invoke one of the G-sets into the code table.
//Arguments:
// - gsetNumber - The G-set that will be invoked.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::SingleShift(const size_t gsetNumber)
{
return _termOutput.SingleShift(gsetNumber);
}
Disable the acceptance of C1 control codes by default (#11690) There are some code pages with "unmapped" code points in the C1 range, which results in them being translated into Unicode C1 control codes, even though that is not their intended use. To avoid having these characters triggering unintentional escape sequences, this PR now disables C1 controls by default. Switching to ISO-2022 encoding will re-enable them, though, since that is the most likely scenario in which they would be required. They can also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape sequence. What I've done is add a new mode to the `StateMachine` class that controls whether C1 code points are interpreted as control characters or not. When disabled, these code points are simply dropped from the output, similar to the way a `NUL` is interpreted. This isn't exactly the way they were handled in the v1 console (which I think replaces them with the font _notdef_ glyph), but it matches the XTerm behavior, which seems more appropriate considering this is in VT mode. And it's worth noting that Windows Explorer seems to work the same way. As mentioned above, the mode can be enabled by designating the ISO-2022 coding system with a `DOCS` sequence, and it will be disabled again when UTF-8 is designated. You can also enable it explicitly with a `DECAC1` sequence (originally this was actually a DEC printer sequence, but it doesn't seem unreasonable to use it in a terminal). I've also extended the operations that save and restore "cursor state" (e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode, since it's closely tied to the code page and character sets which are also saved there. Similarly, when a `DECSTR` sequence resets the code page and character sets, I've now made it reset the C1 mode as well. I should note that the new `StateMachine` mode is controlled via a generic `SetParserMode` method (with a matching API in the `ConGetSet` interface) to allow for easier addition of other modes in the future. And I've reimplemented the existing ANSI/VT52 mode in terms of these generic methods instead of it having to have its own separate APIs. ## Validation Steps Performed Some of the unit tests for OSC sequences were using a C1 `0x9C` for the string terminator, which doesn't work by default anymore. Since that's not a good practice anyway, I thought it best to change those to a standard 7-bit terminator. However, in tests that were explicitly validating the C1 controls, I've just enabled the C1 parser mode at the start of the tests in order to get them working again. There were also some ANSI mode adapter tests that had to be updated to account for the fact that it has now been reimplemented in terms of the `SetParserMode` API. I've added a new state machine test to validate the changes in behavior when the C1 parser mode is enabled or disabled. And I've added an adapter test to verify that the `DesignateCodingSystems` and `AcceptC1Controls` methods toggle the C1 parser mode as expected. I've manually verified the test cases in #10069 and #10310 to confirm that they're no longer triggering control sequences by default. Although, as I explained above, the C1 code points are completely dropped from the output rather than displayed as _notdef_ glyphs. I think this is a reasonable compromise though. Closes #10069 Closes #10310
2021-11-18 00:40:31 +01:00
//Routine Description:
// DECAC1 - Enable or disable the reception of C1 control codes in the parser.
//Arguments:
// - enabled - true to allow C1 controls to be used, false to disallow.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::AcceptC1Controls(const bool enabled)
{
return _pConApi->SetParserMode(StateMachine::Mode::AcceptC1, enabled);
}
//Routine Description:
// Soft Reset - Perform a soft reset. See http://www.vt100.net/docs/vt510-rm/DECSTR.html
// The following table lists everything that should be done, 'X's indicate the ones that
// we actually perform. As the appropriate functionality is added to our ANSI support,
// we should update this.
// X Text cursor enable DECTCEM Cursor enabled.
// Insert/replace IRM Replace mode.
// X Origin DECOM Absolute (cursor origin at upper-left of screen.)
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
// X Autowrap DECAWM Autowrap enabled (matches XTerm behavior).
// National replacement DECNRCM Multinational set.
// character set
// Keyboard action KAM Unlocked.
// X Numeric keypad DECNKM Numeric characters.
// X Cursor keys DECCKM Normal (arrow keys).
// X Set top and bottom margins DECSTBM Top margin = 1; bottom margin = page length.
// X All character sets G0, G1, G2, Default settings.
// G3, GL, GR
// X Select graphic rendition SGR Normal rendition.
// Select character attribute DECSCA Normal (erasable by DECSEL and DECSED).
// X Save cursor state DECSC Home position.
// Assign user preference DECAUPSS Set selected in Set-Up.
// supplemental set
// Select active DECSASD Main display.
// status display
// Keyboard position mode DECKPM Character codes.
// Cursor direction DECRLM Reset (Left-to-right), regardless of NVR setting.
// PC Term mode DECPCTERM Always reset.
//Arguments:
// <none>
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::SoftReset()
{
bool success = CursorVisibility(true); // Cursor enabled.
Don't abort early in VT reset operations if one of the steps fails (#6763) The VT reset operations `RIS` and `DECSTR` are implemented as a series of steps, each of which could potentially fail. Currently these operations abort as soon as an error is detected, which is particularly problematic in conpty mode, where some steps deliberately "fail" to indicate that they need to be "passed through" to the conpty client. As a result, the reset won't be fully executed. This PR changes that behaviour, so the error state is recorded for any failures, but the subsequent steps are still run. Originally the structure of these operations was of the form: bool success = DoSomething(); if (success) { success = DoSomethingElse(); } But I've now changed the code so it looks more like this: bool success = DoSomething(); success = DoSomethingElse() && success; This means that every one of the steps should execute, regardless of whether previous steps were successful, but the final _success_ state will only be true if none of the steps has failed. While this is only really an issue in the conhost code, I've updated both the `AdaptDispatch` and `TerminalDispatch` classes, since I thought it would be best to have them in sync, and in general this seems like a better way to handle multi-step operations anyway. VALIDATION I've manually tested the `RIS` escape sequence (`\ec`) in the Windows Terminal, and confirmed that it now correctly resets the cursor position, which it wasn't doing before. Closes #6545
2020-07-06 16:09:03 +02:00
success = SetOriginMode(false) && success; // Absolute cursor addressing.
success = SetAutoWrapMode(true) && success; // Wrap at end of line.
success = SetCursorKeysMode(false) && success; // Normal characters.
success = SetKeypadMode(false) && success; // Numeric characters.
// Top margin = 1; bottom margin = page length.
success = _DoSetTopBottomScrollingMargins(0, 0) && success;
_termOutput = {}; // Reset all character set designations.
if (_initialCodePage.has_value())
{
Don't abort early in VT reset operations if one of the steps fails (#6763) The VT reset operations `RIS` and `DECSTR` are implemented as a series of steps, each of which could potentially fail. Currently these operations abort as soon as an error is detected, which is particularly problematic in conpty mode, where some steps deliberately "fail" to indicate that they need to be "passed through" to the conpty client. As a result, the reset won't be fully executed. This PR changes that behaviour, so the error state is recorded for any failures, but the subsequent steps are still run. Originally the structure of these operations was of the form: bool success = DoSomething(); if (success) { success = DoSomethingElse(); } But I've now changed the code so it looks more like this: bool success = DoSomething(); success = DoSomethingElse() && success; This means that every one of the steps should execute, regardless of whether previous steps were successful, but the final _success_ state will only be true if none of the steps has failed. While this is only really an issue in the conhost code, I've updated both the `AdaptDispatch` and `TerminalDispatch` classes, since I thought it would be best to have them in sync, and in general this seems like a better way to handle multi-step operations anyway. VALIDATION I've manually tested the `RIS` escape sequence (`\ec`) in the Windows Terminal, and confirmed that it now correctly resets the cursor position, which it wasn't doing before. Closes #6545
2020-07-06 16:09:03 +02:00
// Restore initial code page if previously changed by a DOCS sequence.
success = _pConApi->SetConsoleOutputCP(_initialCodePage.value()) && success;
}
Disable the acceptance of C1 control codes by default (#11690) There are some code pages with "unmapped" code points in the C1 range, which results in them being translated into Unicode C1 control codes, even though that is not their intended use. To avoid having these characters triggering unintentional escape sequences, this PR now disables C1 controls by default. Switching to ISO-2022 encoding will re-enable them, though, since that is the most likely scenario in which they would be required. They can also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape sequence. What I've done is add a new mode to the `StateMachine` class that controls whether C1 code points are interpreted as control characters or not. When disabled, these code points are simply dropped from the output, similar to the way a `NUL` is interpreted. This isn't exactly the way they were handled in the v1 console (which I think replaces them with the font _notdef_ glyph), but it matches the XTerm behavior, which seems more appropriate considering this is in VT mode. And it's worth noting that Windows Explorer seems to work the same way. As mentioned above, the mode can be enabled by designating the ISO-2022 coding system with a `DOCS` sequence, and it will be disabled again when UTF-8 is designated. You can also enable it explicitly with a `DECAC1` sequence (originally this was actually a DEC printer sequence, but it doesn't seem unreasonable to use it in a terminal). I've also extended the operations that save and restore "cursor state" (e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode, since it's closely tied to the code page and character sets which are also saved there. Similarly, when a `DECSTR` sequence resets the code page and character sets, I've now made it reset the C1 mode as well. I should note that the new `StateMachine` mode is controlled via a generic `SetParserMode` method (with a matching API in the `ConGetSet` interface) to allow for easier addition of other modes in the future. And I've reimplemented the existing ANSI/VT52 mode in terms of these generic methods instead of it having to have its own separate APIs. ## Validation Steps Performed Some of the unit tests for OSC sequences were using a C1 `0x9C` for the string terminator, which doesn't work by default anymore. Since that's not a good practice anyway, I thought it best to change those to a standard 7-bit terminator. However, in tests that were explicitly validating the C1 controls, I've just enabled the C1 parser mode at the start of the tests in order to get them working again. There were also some ANSI mode adapter tests that had to be updated to account for the fact that it has now been reimplemented in terms of the `SetParserMode` API. I've added a new state machine test to validate the changes in behavior when the C1 parser mode is enabled or disabled. And I've added an adapter test to verify that the `DesignateCodingSystems` and `AcceptC1Controls` methods toggle the C1 parser mode as expected. I've manually verified the test cases in #10069 and #10310 to confirm that they're no longer triggering control sequences by default. Although, as I explained above, the C1 code points are completely dropped from the output rather than displayed as _notdef_ glyphs. I think this is a reasonable compromise though. Closes #10069 Closes #10310
2021-11-18 00:40:31 +01:00
// Disable parsing of C1 control codes.
success = AcceptC1Controls(false) && success;
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
success = SetGraphicsRendition({}) && success; // Normal rendition.
Don't abort early in VT reset operations if one of the steps fails (#6763) The VT reset operations `RIS` and `DECSTR` are implemented as a series of steps, each of which could potentially fail. Currently these operations abort as soon as an error is detected, which is particularly problematic in conpty mode, where some steps deliberately "fail" to indicate that they need to be "passed through" to the conpty client. As a result, the reset won't be fully executed. This PR changes that behaviour, so the error state is recorded for any failures, but the subsequent steps are still run. Originally the structure of these operations was of the form: bool success = DoSomething(); if (success) { success = DoSomethingElse(); } But I've now changed the code so it looks more like this: bool success = DoSomething(); success = DoSomethingElse() && success; This means that every one of the steps should execute, regardless of whether previous steps were successful, but the final _success_ state will only be true if none of the steps has failed. While this is only really an issue in the conhost code, I've updated both the `AdaptDispatch` and `TerminalDispatch` classes, since I thought it would be best to have them in sync, and in general this seems like a better way to handle multi-step operations anyway. VALIDATION I've manually tested the `RIS` escape sequence (`\ec`) in the Windows Terminal, and confirmed that it now correctly resets the cursor position, which it wasn't doing before. Closes #6545
2020-07-06 16:09:03 +02:00
// Reset the saved cursor state.
// Note that XTerm only resets the main buffer state, but that
// seems likely to be a bug. Most other terminals reset both.
_savedCursorState.at(0) = {}; // Main buffer
_savedCursorState.at(1) = {}; // Alt buffer
return success;
}
//Routine Description:
// Full Reset - Perform a hard reset of the terminal. http://vt100.net/docs/vt220-rm/chapter4.html
// RIS performs the following actions: (Items with sub-bullets are supported)
// - Switches to the main screen buffer if in the alt buffer.
// * This matches the XTerm behaviour, which is the de facto standard for the alt buffer.
// - Performs a communications line disconnect.
// - Clears UDKs.
// - Clears a down-line-loaded character set.
Add support for downloadable soft fonts (#10011) This PR adds conhost support for downloadable soft fonts - also known as dynamically redefinable character sets (DRCS) - using the `DECDLD` escape sequence. These fonts are typically designed to work on a specific terminal model, and each model tends to have a different character cell size. So in order to support as many models as possible, the code attempts to detect the original target size of the font, and then scale the glyphs to fit our current cell size. Once a font has been downloaded to the terminal, it can be designated in the same way you would a standard character set, using an `SCS` escape sequence. The identification string for the set is defined by the `DECDLD` sequence. Internally we map the characters in this set to code points `U+EF20` to `U+EF7F` in the Unicode private use are (PUA). Then in the renderer, any characters in that range are split off into separate runs, which get painted with a special font. The font itself is dynamically generated as an in-memory resource, constructed from the downloaded character bitmaps which have been scaled to the appropriate size. If no soft fonts are in use, then no mapping of the PUA code points will take place, so this shouldn't interfere with anyone using those code points for something else, as along as they aren't also trying to use soft fonts. I also tried to pick a PUA range that hadn't already been snatched up by Nerd Fonts, but if we do receive reports of a conflict, it's easy enough to change. ## Validation Steps Performed I added an adapter test that runs through a bunch of parameter variations for the `DECDLD` sequence, to make sure we're correctly detecting the font sizes for most of the known DEC terminal models. I've also tested manually on a wide range of existing fonts, of varying dimensions, and from multiple sources, and made sure they all worked reasonably well. Closes #9164
2021-08-06 22:41:02 +02:00
// * The soft font is reset in the renderer and the font buffer is deleted.
// - Clears the screen.
// * This is like Erase in Display (3), also clearing scrollback, as well as ED(2)
// - Returns the cursor to the upper-left corner of the screen.
// * CUP(1;1)
// - Sets the SGR state to normal.
// * SGR(Off)
// - Sets the selective erase attribute write state to "not erasable".
// - Sets all character sets to the default.
// * G0(USASCII)
//Arguments:
// <none>
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::HardReset()
{
bool success = true;
// If in the alt buffer, switch back to main before doing anything else.
if (_usingAltBuffer)
{
success = _pConApi->PrivateUseMainScreenBuffer();
_usingAltBuffer = !success;
}
Make the RIS command clear the display and scrollback correctly (#2367) When the scrollback buffer is empty, the RIS escape sequence (Reset to Initial State) will fail to clear the screen, or reset any of the state. And when there is something in the scrollback, it doesn't get cleared completely, and the screen may get filled with the wrong background color (it should use the default color, but it actually uses the previously active background color). This commit attempts to fix those issues. The initial failure is caused by the `SCREEN_INFORMATION::WriteRect` method throwing an exception when passed an empty viewport. And the reason it's passed an empty viewport is because that's what the `Viewport::Subtract` method returns when the result of the subtraction is nothing. The PR fixes the problem by making the `Viewport::Subtract` method actually return nothing in that situation. This is a change in the defined behavior that also required the associated viewport tests to be updated. However, it does seem a sensible change, since the `Subtract` method never returns empty viewports under any other circumstances. And the only place the method seems to be used is in the `ScrollRegion` implementation, where the previous behavior is guaranteed to throw an exception. The other issues are fixed simply by changing the order in which things are reset in the `AdaptDispatch::HardReset` method. The call to `SoftReset` needed to be made first, so that the SGR attributes would be reset before the screen was cleared, thus making sure that the default background color would be used. And the screen needed to be cleared before the scrollback was erased, otherwise the last view of the screen would be retained in the scrollback buffer. These changes also required existing adapter tests to be updated, but not because of a change in the expected behaviour. It's just that certain tests relied on the `SoftReset` happening later in the order, so weren't expecting it to be called if say the scrollback erase had failed. It doesn't seem like the tests were deliberately trying to verify that the SoftReset _hadn't_ been called. In addition to the updates to existing tests, this PR also add a new screen buffer test which verifies the display and scrollback are correctly cleared under the conditions that were previously failing. Fixes #2307.
2019-08-28 03:45:38 +02:00
// Sets the SGR state to normal - this must be done before EraseInDisplay
// to ensure that it clears with the default background color.
Don't abort early in VT reset operations if one of the steps fails (#6763) The VT reset operations `RIS` and `DECSTR` are implemented as a series of steps, each of which could potentially fail. Currently these operations abort as soon as an error is detected, which is particularly problematic in conpty mode, where some steps deliberately "fail" to indicate that they need to be "passed through" to the conpty client. As a result, the reset won't be fully executed. This PR changes that behaviour, so the error state is recorded for any failures, but the subsequent steps are still run. Originally the structure of these operations was of the form: bool success = DoSomething(); if (success) { success = DoSomethingElse(); } But I've now changed the code so it looks more like this: bool success = DoSomething(); success = DoSomethingElse() && success; This means that every one of the steps should execute, regardless of whether previous steps were successful, but the final _success_ state will only be true if none of the steps has failed. While this is only really an issue in the conhost code, I've updated both the `AdaptDispatch` and `TerminalDispatch` classes, since I thought it would be best to have them in sync, and in general this seems like a better way to handle multi-step operations anyway. VALIDATION I've manually tested the `RIS` escape sequence (`\ec`) in the Windows Terminal, and confirmed that it now correctly resets the cursor position, which it wasn't doing before. Closes #6545
2020-07-06 16:09:03 +02:00
success = SoftReset() && success;
Make the RIS command clear the display and scrollback correctly (#2367) When the scrollback buffer is empty, the RIS escape sequence (Reset to Initial State) will fail to clear the screen, or reset any of the state. And when there is something in the scrollback, it doesn't get cleared completely, and the screen may get filled with the wrong background color (it should use the default color, but it actually uses the previously active background color). This commit attempts to fix those issues. The initial failure is caused by the `SCREEN_INFORMATION::WriteRect` method throwing an exception when passed an empty viewport. And the reason it's passed an empty viewport is because that's what the `Viewport::Subtract` method returns when the result of the subtraction is nothing. The PR fixes the problem by making the `Viewport::Subtract` method actually return nothing in that situation. This is a change in the defined behavior that also required the associated viewport tests to be updated. However, it does seem a sensible change, since the `Subtract` method never returns empty viewports under any other circumstances. And the only place the method seems to be used is in the `ScrollRegion` implementation, where the previous behavior is guaranteed to throw an exception. The other issues are fixed simply by changing the order in which things are reset in the `AdaptDispatch::HardReset` method. The call to `SoftReset` needed to be made first, so that the SGR attributes would be reset before the screen was cleared, thus making sure that the default background color would be used. And the screen needed to be cleared before the scrollback was erased, otherwise the last view of the screen would be retained in the scrollback buffer. These changes also required existing adapter tests to be updated, but not because of a change in the expected behaviour. It's just that certain tests relied on the `SoftReset` happening later in the order, so weren't expecting it to be called if say the scrollback erase had failed. It doesn't seem like the tests were deliberately trying to verify that the SoftReset _hadn't_ been called. In addition to the updates to existing tests, this PR also add a new screen buffer test which verifies the display and scrollback are correctly cleared under the conditions that were previously failing. Fixes #2307.
2019-08-28 03:45:38 +02:00
// Clears the screen - Needs to be done in two operations.
Don't abort early in VT reset operations if one of the steps fails (#6763) The VT reset operations `RIS` and `DECSTR` are implemented as a series of steps, each of which could potentially fail. Currently these operations abort as soon as an error is detected, which is particularly problematic in conpty mode, where some steps deliberately "fail" to indicate that they need to be "passed through" to the conpty client. As a result, the reset won't be fully executed. This PR changes that behaviour, so the error state is recorded for any failures, but the subsequent steps are still run. Originally the structure of these operations was of the form: bool success = DoSomething(); if (success) { success = DoSomethingElse(); } But I've now changed the code so it looks more like this: bool success = DoSomething(); success = DoSomethingElse() && success; This means that every one of the steps should execute, regardless of whether previous steps were successful, but the final _success_ state will only be true if none of the steps has failed. While this is only really an issue in the conhost code, I've updated both the `AdaptDispatch` and `TerminalDispatch` classes, since I thought it would be best to have them in sync, and in general this seems like a better way to handle multi-step operations anyway. VALIDATION I've manually tested the `RIS` escape sequence (`\ec`) in the Windows Terminal, and confirmed that it now correctly resets the cursor position, which it wasn't doing before. Closes #6545
2020-07-06 16:09:03 +02:00
success = EraseInDisplay(DispatchTypes::EraseType::All) && success;
success = EraseInDisplay(DispatchTypes::EraseType::Scrollback) && success;
Add support for the DECSCNM screen mode (#3817) ## Summary of the Pull Request This adds support for the [`DECSCNM`](https://vt100.net/docs/vt510-rm/DECSCNM.html) private mode escape sequence, which toggles the display between normal and reverse screen modes. When reversed, the background and foreground colors are switched. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## References This also fixes issue #72 for the most part, although if you toggle the mode too fast, there is no discernible flash. ## PR Checklist * [x] Closes #3773 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments I've implemented this as a new flag in the `Settings` class, along with updates to the `LookupForegroundColor` and `LookupBackgroundColor` methods, to switch the returned foreground and background colors when that flag is set. It also required a new private API in the `ConGetSet` interface to toggle the setting. And that API is then called from the `AdaptDispatch` class when the screen mode escape sequence is received. The last thing needed was to add a step to the `HardReset` method, to reset the mode back to normal, which is one of the `RIS` requirements. Note that this does currently work in the Windows Terminal, but once #2661 is implemented that may no longer be the case. It might become necessary to let the mode change sequences pass through conpty, and handle the color reversing on the client side. ## Validation Steps Performed I've added a state machine test to make sure the escape sequence is dispatched correctly, and a screen buffer test to confirm that the mode change does alter the interpretation of colors as expected. I've also confirmed that the various "light background" tests in Vttest now display correctly, and that the `tput flash` command (in a bash shell) does actually cause the screen to flash.
2020-01-22 23:29:50 +01:00
// Set the DECSCNM screen mode back to normal.
Don't abort early in VT reset operations if one of the steps fails (#6763) The VT reset operations `RIS` and `DECSTR` are implemented as a series of steps, each of which could potentially fail. Currently these operations abort as soon as an error is detected, which is particularly problematic in conpty mode, where some steps deliberately "fail" to indicate that they need to be "passed through" to the conpty client. As a result, the reset won't be fully executed. This PR changes that behaviour, so the error state is recorded for any failures, but the subsequent steps are still run. Originally the structure of these operations was of the form: bool success = DoSomething(); if (success) { success = DoSomethingElse(); } But I've now changed the code so it looks more like this: bool success = DoSomething(); success = DoSomethingElse() && success; This means that every one of the steps should execute, regardless of whether previous steps were successful, but the final _success_ state will only be true if none of the steps has failed. While this is only really an issue in the conhost code, I've updated both the `AdaptDispatch` and `TerminalDispatch` classes, since I thought it would be best to have them in sync, and in general this seems like a better way to handle multi-step operations anyway. VALIDATION I've manually tested the `RIS` escape sequence (`\ec`) in the Windows Terminal, and confirmed that it now correctly resets the cursor position, which it wasn't doing before. Closes #6545
2020-07-06 16:09:03 +02:00
success = SetScreenMode(false) && success;
Add support for the DECSCNM screen mode (#3817) ## Summary of the Pull Request This adds support for the [`DECSCNM`](https://vt100.net/docs/vt510-rm/DECSCNM.html) private mode escape sequence, which toggles the display between normal and reverse screen modes. When reversed, the background and foreground colors are switched. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## References This also fixes issue #72 for the most part, although if you toggle the mode too fast, there is no discernible flash. ## PR Checklist * [x] Closes #3773 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments I've implemented this as a new flag in the `Settings` class, along with updates to the `LookupForegroundColor` and `LookupBackgroundColor` methods, to switch the returned foreground and background colors when that flag is set. It also required a new private API in the `ConGetSet` interface to toggle the setting. And that API is then called from the `AdaptDispatch` class when the screen mode escape sequence is received. The last thing needed was to add a step to the `HardReset` method, to reset the mode back to normal, which is one of the `RIS` requirements. Note that this does currently work in the Windows Terminal, but once #2661 is implemented that may no longer be the case. It might become necessary to let the mode change sequences pass through conpty, and handle the color reversing on the client side. ## Validation Steps Performed I've added a state machine test to make sure the escape sequence is dispatched correctly, and a screen buffer test to confirm that the mode change does alter the interpretation of colors as expected. I've also confirmed that the various "light background" tests in Vttest now display correctly, and that the `tput flash` command (in a bash shell) does actually cause the screen to flash.
2020-01-22 23:29:50 +01:00
// Cursor to 1,1 - the Soft Reset guarantees this is absolute
Don't abort early in VT reset operations if one of the steps fails (#6763) The VT reset operations `RIS` and `DECSTR` are implemented as a series of steps, each of which could potentially fail. Currently these operations abort as soon as an error is detected, which is particularly problematic in conpty mode, where some steps deliberately "fail" to indicate that they need to be "passed through" to the conpty client. As a result, the reset won't be fully executed. This PR changes that behaviour, so the error state is recorded for any failures, but the subsequent steps are still run. Originally the structure of these operations was of the form: bool success = DoSomething(); if (success) { success = DoSomethingElse(); } But I've now changed the code so it looks more like this: bool success = DoSomething(); success = DoSomethingElse() && success; This means that every one of the steps should execute, regardless of whether previous steps were successful, but the final _success_ state will only be true if none of the steps has failed. While this is only really an issue in the conhost code, I've updated both the `AdaptDispatch` and `TerminalDispatch` classes, since I thought it would be best to have them in sync, and in general this seems like a better way to handle multi-step operations anyway. VALIDATION I've manually tested the `RIS` escape sequence (`\ec`) in the Windows Terminal, and confirmed that it now correctly resets the cursor position, which it wasn't doing before. Closes #6545
2020-07-06 16:09:03 +02:00
success = CursorPosition(1, 1) && success;
// Reset the mouse mode
success = EnableSGRExtendedMouseMode(false) && success;
success = EnableAnyEventMouseMode(false) && success;
Don't abort early in VT reset operations if one of the steps fails (#6763) The VT reset operations `RIS` and `DECSTR` are implemented as a series of steps, each of which could potentially fail. Currently these operations abort as soon as an error is detected, which is particularly problematic in conpty mode, where some steps deliberately "fail" to indicate that they need to be "passed through" to the conpty client. As a result, the reset won't be fully executed. This PR changes that behaviour, so the error state is recorded for any failures, but the subsequent steps are still run. Originally the structure of these operations was of the form: bool success = DoSomething(); if (success) { success = DoSomethingElse(); } But I've now changed the code so it looks more like this: bool success = DoSomething(); success = DoSomethingElse() && success; This means that every one of the steps should execute, regardless of whether previous steps were successful, but the final _success_ state will only be true if none of the steps has failed. While this is only really an issue in the conhost code, I've updated both the `AdaptDispatch` and `TerminalDispatch` classes, since I thought it would be best to have them in sync, and in general this seems like a better way to handle multi-step operations anyway. VALIDATION I've manually tested the `RIS` escape sequence (`\ec`) in the Windows Terminal, and confirmed that it now correctly resets the cursor position, which it wasn't doing before. Closes #6545
2020-07-06 16:09:03 +02:00
// Delete all current tab stops and reapply
Reimplement the VT tab stop functionality (#5173) ## Summary of the Pull Request This is essentially a rewrite of the VT tab stop functionality, implemented entirely within the `AdaptDispatch` class. This significantly simplifies the `ConGetSet` interface, and should hopefully make it easier to share the functionality with the Windows Terminal VT implementation in the future. By removing the dependence on the `SCREEN_INFORMATION` class, it fixes the problem of the the tab state not being preserved when switching between the main and alternate buffers. And the new architecture also fixes problems with the tabs not being correctly initialized when the screen is resized. ## References This fixes one aspect of issue #3545. It also supersedes the fix for #411 (PR #2816). I'm hoping the simplification of `ConGetSet` will help with #3849. ## PR Checklist * [x] Closes #4669 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments In the new tab architecture, there is now a `vector<bool>` (__tabStopColumns_), which tracks whether any particular column is a tab stop or not. There is also a __initDefaultTabStops_ flag indicating whether the default tab stop positions need to be initialised when the screen is resized. The way this works, the vector is initially empty, and only initialized (to the current width of the screen) when it needs to be used. When the vector grows in size, the __initDefaultTabStops_ flag determines whether the new columns are set to false, or if every 8th column is set to true. By default we want the latter behaviour - newly revealed columns should have default tab stops assigned to them - so __initDefaultTabStops_ is set to true. However, after a `TBC 3` operation (i.e. we've cleared all tab stops), there should be no tab stops in any newly revealed columns, so __initDefaultTabStops_ is set to false. Note that the __tabStopColumns_ vector is never made smaller when the window is shrunk, and that way it can preserve the state of tab stops that are off screen, but which may come into range if the window is made bigger again. However, we can can still reset the vector completely after an `RIS` or `TBC 3` operation, since the state can then be reconstructed automatically based on just the __initDefaultTabStops_ flag. ## Validation Steps Performed The original screen buffer tests had to be rewritten to set and query the tab stop state using escape sequences rather than interacting with the `SCREEN_INFORMATION` class directly, but otherwise the structure of most tests remained largely the same. However, the alt buffer test was significantly rewritten, since the original behaviour was incorrect, and the initialization test was dropped completely, since it was no longer applicable. The adapter tests were also dropped, since they were testing the `ConGetSet` interface which has now been removed. I also had to make an addition to the method setup of the screen buffer tests (making sure the viewport was appropriately initialized), since there were some tests (unrelated to tab stops) that were previously dependent on the state being set in the tab initialization test which has now been removed. I've manually tested the issue described in #4669 and confirmed that the tabs now produce the correct spacing after a resize.
2020-04-01 14:49:27 +02:00
_ResetTabStops();
Add support for downloadable soft fonts (#10011) This PR adds conhost support for downloadable soft fonts - also known as dynamically redefinable character sets (DRCS) - using the `DECDLD` escape sequence. These fonts are typically designed to work on a specific terminal model, and each model tends to have a different character cell size. So in order to support as many models as possible, the code attempts to detect the original target size of the font, and then scale the glyphs to fit our current cell size. Once a font has been downloaded to the terminal, it can be designated in the same way you would a standard character set, using an `SCS` escape sequence. The identification string for the set is defined by the `DECDLD` sequence. Internally we map the characters in this set to code points `U+EF20` to `U+EF7F` in the Unicode private use are (PUA). Then in the renderer, any characters in that range are split off into separate runs, which get painted with a special font. The font itself is dynamically generated as an in-memory resource, constructed from the downloaded character bitmaps which have been scaled to the appropriate size. If no soft fonts are in use, then no mapping of the PUA code points will take place, so this shouldn't interfere with anyone using those code points for something else, as along as they aren't also trying to use soft fonts. I also tried to pick a PUA range that hadn't already been snatched up by Nerd Fonts, but if we do receive reports of a conflict, it's easy enough to change. ## Validation Steps Performed I added an adapter test that runs through a bunch of parameter variations for the `DECDLD` sequence, to make sure we're correctly detecting the font sizes for most of the known DEC terminal models. I've also tested manually on a wide range of existing fonts, of varying dimensions, and from multiple sources, and made sure they all worked reasonably well. Closes #9164
2021-08-06 22:41:02 +02:00
// Clear the soft font in the renderer and delete the font buffer.
success = _pConApi->PrivateUpdateSoftFont({}, {}, false) && success;
_fontBuffer = nullptr;
// GH#2715 - If all this succeeded, but we're in a conpty, return `false` to
// make the state machine propagate this RIS sequence to the connected
// terminal application. We've reset our state, but the connected terminal
// might need to do more.
if (_pConApi->IsConsolePty())
{
return false;
}
return success;
}
Add support for the DECALN escape sequence (#3968) ## Summary of the Pull Request This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite. ## PR Checklist * [x] Closes #3671 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface. The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset: * The meta/extended attributes are reset (although the active colors must be left unchanged). * The origin mode is set to absolute positioning. * The scrolling margins are cleared. * The cursor position is moved to home. ## Validation Steps Performed I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately. I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
2019-12-17 19:11:52 +01:00
// Routine Description:
// - DECALN - Fills the entire screen with a test pattern of uppercase Es,
// resets the margins and rendition attributes, and moves the cursor to
// the home position.
// Arguments:
// - None
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::ScreenAlignmentPattern()
{
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
// Make sure to reset the viewport (with MoveToBottom )to where it was
// before the user scrolled the console output
bool success = _pConApi->MoveToBottom() && _pConApi->GetConsoleScreenBufferInfoEx(csbiex);
Add support for the DECALN escape sequence (#3968) ## Summary of the Pull Request This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite. ## PR Checklist * [x] Closes #3671 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface. The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset: * The meta/extended attributes are reset (although the active colors must be left unchanged). * The origin mode is set to absolute positioning. * The scrolling margins are cleared. * The cursor position is moved to home. ## Validation Steps Performed I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately. I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
2019-12-17 19:11:52 +01:00
if (success)
Add support for the DECALN escape sequence (#3968) ## Summary of the Pull Request This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite. ## PR Checklist * [x] Closes #3671 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface. The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset: * The meta/extended attributes are reset (although the active colors must be left unchanged). * The origin mode is set to absolute positioning. * The scrolling margins are cleared. * The cursor position is moved to home. ## Validation Steps Performed I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately. I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
2019-12-17 19:11:52 +01:00
{
// Fill the screen with the letter E using the default attributes.
auto fillPosition = COORD{ 0, csbiex.srWindow.Top };
const auto fillLength = (csbiex.srWindow.Bottom - csbiex.srWindow.Top) * csbiex.dwSize.X;
success = _pConApi->PrivateFillRegion(fillPosition, fillLength, L'E', false);
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
// Reset the line rendition for all of these rows.
success = success && _pConApi->PrivateResetLineRenditionRange(csbiex.srWindow.Top, csbiex.srWindow.Bottom);
Add support for the DECALN escape sequence (#3968) ## Summary of the Pull Request This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite. ## PR Checklist * [x] Closes #3671 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface. The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset: * The meta/extended attributes are reset (although the active colors must be left unchanged). * The origin mode is set to absolute positioning. * The scrolling margins are cleared. * The cursor position is moved to home. ## Validation Steps Performed I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately. I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
2019-12-17 19:11:52 +01:00
// Reset the meta/extended attributes (but leave the colors unchanged).
Refactor the SGR implementation in AdaptDispatch (#5758) This is an attempt to simplify the SGR (Select Graphic Rendition) implementation in conhost, to cut down on the number of methods required in the `ConGetSet` interface, and pave the way for future improvements and bug fixes. It already fixes one bug that prevented SGR 0 from being correctly applied when combined with meta attributes. * This a first step towards fixing the conpty narrowing bugs in issue #2661 * I'm hoping the simplification of `ConGetSet` will also help with #3849. * Some of the `TextAttribute` refactoring in this PR overlaps with similar work in PR #1978. ## Detailed Description of the Pull Request / Additional comments The main point of this PR was to simplify the `AdaptDispatch::SetGraphicsRendition` implementation. So instead of having it call a half a dozen methods in the `ConGetSet` API, depending on what kinds of attributes needed to be set, there is now just one call to get current attributes, and another call to set the new value. All adjustments to the attributes are made in the `AdaptDispatch` class, in a simple switch statement. To help with this refactoring, I also made some change to the `TextAttribute` class to make it easier to work with. This included adding a set of methods for setting (and getting) the individual attribute flags, instead of having the calling code being exposed to the internal attribute structures and messing with bit manipulation. I've tried to get rid of any methods that were directly setting legacy, meta, and extended attributes. Other than the fix to the `SGR 0` bug, the `AdaptDispatch` refactoring mostly follows the behaviour of the original code. In particular, it still maps the `SGR 38/48` indexed colors to RGB instead of retaining the index, which is what we ultimately need it to do. Fixing that will first require the color tables to be unified (issue #1223), which I'm hoping to address in a followup PR. But for now, mapping the indexed colors to RGB values required adding an an additional `ConGetSet` API to lookup the color table entries. In the future that won't be necessary, but the API will still be useful for other color reporting operations that we may want to support. I've made this API, and the existing setter, standardise on index values being in the "Xterm" order, since that'll be essential for unifying the code with the terminal adapter one day. I should also point out one minor change to the `SGR 38/48` behavior, which is that out-of-range RGB colors are now ignored rather than being clamped, since that matches the way Xterm works. ## Validation Steps Performed This refactoring has obviously required corresponding changes to the unit tests, but most were just minor updates to use the new `TextAttribute` methods without any real change in behavior. However, the adapter tests did require significant changes to accommodate the new `ConGetSet` API. The basic structure of the tests remain the same, but the simpler API has meant fewer values needed to be checked in each test case. I think they are all still covering the areas there were intended to, though, and they are all still passing. Other than getting the unit tests to work, I've also done a bunch of manual testing of my own. I've made sure the color tests in Vttest all still work as well as they used to. And I've confirmed that the test case from issue #5341 is now working correctly. Closes #5341
2020-05-09 01:04:16 +02:00
TextAttribute attr;
if (_pConApi->PrivateGetTextAttributes(attr))
{
attr.SetStandardErase();
success = success && _pConApi->PrivateSetTextAttributes(attr);
}
Add support for the DECALN escape sequence (#3968) ## Summary of the Pull Request This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite. ## PR Checklist * [x] Closes #3671 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface. The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset: * The meta/extended attributes are reset (although the active colors must be left unchanged). * The origin mode is set to absolute positioning. * The scrolling margins are cleared. * The cursor position is moved to home. ## Validation Steps Performed I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately. I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
2019-12-17 19:11:52 +01:00
// Reset the origin mode to absolute.
success = success && SetOriginMode(false);
Add support for the DECALN escape sequence (#3968) ## Summary of the Pull Request This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite. ## PR Checklist * [x] Closes #3671 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface. The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset: * The meta/extended attributes are reset (although the active colors must be left unchanged). * The origin mode is set to absolute positioning. * The scrolling margins are cleared. * The cursor position is moved to home. ## Validation Steps Performed I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately. I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
2019-12-17 19:11:52 +01:00
// Clear the scrolling margins.
success = success && _DoSetTopBottomScrollingMargins(0, 0);
Add support for the DECALN escape sequence (#3968) ## Summary of the Pull Request This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite. ## PR Checklist * [x] Closes #3671 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface. The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset: * The meta/extended attributes are reset (although the active colors must be left unchanged). * The origin mode is set to absolute positioning. * The scrolling margins are cleared. * The cursor position is moved to home. ## Validation Steps Performed I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately. I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
2019-12-17 19:11:52 +01:00
// Set the cursor position to home.
success = success && CursorPosition(1, 1);
Add support for the DECALN escape sequence (#3968) ## Summary of the Pull Request This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite. ## PR Checklist * [x] Closes #3671 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface. The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset: * The meta/extended attributes are reset (although the active colors must be left unchanged). * The origin mode is set to absolute positioning. * The scrolling margins are cleared. * The cursor position is moved to home. ## Validation Steps Performed I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately. I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
2019-12-17 19:11:52 +01:00
}
return success;
Add support for the DECALN escape sequence (#3968) ## Summary of the Pull Request This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite. ## PR Checklist * [x] Closes #3671 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface. The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset: * The meta/extended attributes are reset (although the active colors must be left unchanged). * The origin mode is set to absolute positioning. * The scrolling margins are cleared. * The cursor position is moved to home. ## Validation Steps Performed I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately. I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
2019-12-17 19:11:52 +01:00
}
//Routine Description:
// - Erase Scrollback (^[[3J - ED extension by xterm)
// Because conhost doesn't exactly have a scrollback, We have to be tricky here.
// We need to move the entire viewport to 0,0, and clear everything outside
// (0, 0, viewportWidth, viewportHeight) To give the appearance that
// everything above the viewport was cleared.
// We don't want to save the text BELOW the viewport, because in *nix, there isn't anything there
// (There isn't a scroll-forward, only a scrollback)
// Arguments:
// - <none>
// Return value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::_EraseScrollback()
{
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(csbiex);
// Make sure to reset the viewport (with MoveToBottom )to where it was
// before the user scrolled the console output
bool success = (_pConApi->GetConsoleScreenBufferInfoEx(csbiex) && _pConApi->MoveToBottom());
if (success)
{
const SMALL_RECT screen = csbiex.srWindow;
const SHORT height = screen.Bottom - screen.Top;
FAIL_FAST_IF(!(height > 0));
const COORD cursor = csbiex.dwCursorPosition;
// Rectangle to cut out of the existing buffer
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// It will be clipped to the buffer boundaries so SHORT_MAX gives us the full buffer width.
SMALL_RECT scroll = screen;
scroll.Left = 0;
scroll.Right = SHORT_MAX;
// Paste coordinate for cut text above
COORD destination;
destination.X = 0;
destination.Y = 0;
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// Typically a scroll operation should fill with standard erase attributes, but in
// this case we need to use the default attributes, hence standardFillAttrs is false.
success = _pConApi->PrivateScrollRegion(scroll, std::nullopt, destination, false);
if (success)
{
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// Clear everything after the viewport.
const DWORD totalAreaBelow = csbiex.dwSize.X * (csbiex.dwSize.Y - height);
const COORD coordBelowStartPosition = { 0, height };
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// Again we need to use the default attributes, hence standardFillAttrs is false.
success = _pConApi->PrivateFillRegion(coordBelowStartPosition, totalAreaBelow, L' ', false);
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
// Also reset the line rendition for all of the cleared rows.
success = success && _pConApi->PrivateResetLineRenditionRange(height, csbiex.dwSize.Y);
if (success)
{
// Move the viewport (CAN'T be done in one call with SetConsolescreenBufferInfoEx, because legacy)
SMALL_RECT newViewport;
newViewport.Left = screen.Left;
newViewport.Top = 0;
// SetConsoleWindowInfo uses an inclusive rect, while GetConsolescreenBufferInfo is exclusive
newViewport.Right = screen.Right - 1;
newViewport.Bottom = height - 1;
success = _pConApi->SetConsoleWindowInfo(true, newViewport);
if (success)
{
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// Move the cursor to the same relative location.
const COORD newcursor = { cursor.X, cursor.Y - screen.Top };
success = _pConApi->SetConsoleCursorPosition(newcursor);
}
}
}
}
return success;
}
//Routine Description:
// Erase All (^[[2J - ED)
// Erase the current contents of the viewport. In most terminals, because they
// only have a scrollback (and not a buffer per-se), they implement this
// by scrolling the current contents of the buffer off of the screen.
// We can't properly replicate this behavior with only the public API, because
// we need to know where the last character in the buffer is. (it may be below the viewport)
//Arguments:
// <none>
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::_EraseAll()
{
return _pConApi->PrivateEraseAll();
}
// Routine Description:
// - Enables or disables support for the DECCOLM escape sequence.
// Arguments:
// - enabled - set to true to allow DECCOLM to be used, false to disallow.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::EnableDECCOLMSupport(const bool enabled) noexcept
{
_isDECCOLMAllowed = enabled;
return true;
}
//Routine Description:
// Enable VT200 Mouse Mode - Enables/disables the mouse input handler in default tracking mode.
//Arguments:
// - enabled - true to enable, false to disable.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::EnableVT200MouseMode(const bool enabled)
{
Consolidate the interfaces for setting VT input modes (#11384) Instead of having a separate method for setting each mouse and keyboard mode, this PR consolidates them all into a single method which takes a mode parameter, and stores the modes in a `til::enumset` rather than having a separate `bool` for each mode. This enables us to get rid of a lot of boilerplate code, and makes the code easier to extend when we want to introduce additional modes in the future. It'll also makes it easier to read back the state of the various modes when implementing the `DECRQM` query. Most of the complication is in the `TerminalInput` class, which had to be adjusted to work with an `enumset` in place of all the `bool` fields. For the rest, it was largely a matter of replacing calls to all the old mode setting methods with the new `SetInputMode` method, and deleting a bunch of unused code. One thing worth mentioning is that the `AdaptDispatch` implementation used to have a `_ShouldPassThroughInputModeChange` method that was called after every mode change. This code has now been moved up into the `SetInputMode` implementation in `ConhostInternalGetSet` so it's just handled in one place. Keeping this out of the dispatch class will also be beneficial for sharing the implementation with `TerminalDispatch`. ## Validation The updated interface necessitated some adjustments to the tests in `AdapterTest` and `MouseInputTest`, but the essential structure of the tests remains unchanged, and everything still passes. I've also tested the keyboard and mouse modes in Vttest and confirmed they still work at least as well as they did before (both conhost and Windows Terminal), and I tested the alternate scroll mode manually (conhost only). Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan to de-duplicate the `AdaptDispatch` and `TerminalDispatch` implementation (#3849).
2021-10-26 23:12:22 +02:00
return _pConApi->SetInputMode(TerminalInput::Mode::DefaultMouseTracking, enabled);
}
//Routine Description:
// Enable UTF-8 Extended Encoding - this changes the encoding scheme for sequences
// emitted by the mouse input handler. Does not enable/disable mouse mode on its own.
//Arguments:
// - enabled - true to enable, false to disable.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::EnableUTF8ExtendedMouseMode(const bool enabled)
{
Consolidate the interfaces for setting VT input modes (#11384) Instead of having a separate method for setting each mouse and keyboard mode, this PR consolidates them all into a single method which takes a mode parameter, and stores the modes in a `til::enumset` rather than having a separate `bool` for each mode. This enables us to get rid of a lot of boilerplate code, and makes the code easier to extend when we want to introduce additional modes in the future. It'll also makes it easier to read back the state of the various modes when implementing the `DECRQM` query. Most of the complication is in the `TerminalInput` class, which had to be adjusted to work with an `enumset` in place of all the `bool` fields. For the rest, it was largely a matter of replacing calls to all the old mode setting methods with the new `SetInputMode` method, and deleting a bunch of unused code. One thing worth mentioning is that the `AdaptDispatch` implementation used to have a `_ShouldPassThroughInputModeChange` method that was called after every mode change. This code has now been moved up into the `SetInputMode` implementation in `ConhostInternalGetSet` so it's just handled in one place. Keeping this out of the dispatch class will also be beneficial for sharing the implementation with `TerminalDispatch`. ## Validation The updated interface necessitated some adjustments to the tests in `AdapterTest` and `MouseInputTest`, but the essential structure of the tests remains unchanged, and everything still passes. I've also tested the keyboard and mouse modes in Vttest and confirmed they still work at least as well as they did before (both conhost and Windows Terminal), and I tested the alternate scroll mode manually (conhost only). Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan to de-duplicate the `AdaptDispatch` and `TerminalDispatch` implementation (#3849).
2021-10-26 23:12:22 +02:00
return _pConApi->SetInputMode(TerminalInput::Mode::Utf8MouseEncoding, enabled);
}
//Routine Description:
// Enable SGR Extended Encoding - this changes the encoding scheme for sequences
// emitted by the mouse input handler. Does not enable/disable mouse mode on its own.
//Arguments:
// - enabled - true to enable, false to disable.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::EnableSGRExtendedMouseMode(const bool enabled)
{
Consolidate the interfaces for setting VT input modes (#11384) Instead of having a separate method for setting each mouse and keyboard mode, this PR consolidates them all into a single method which takes a mode parameter, and stores the modes in a `til::enumset` rather than having a separate `bool` for each mode. This enables us to get rid of a lot of boilerplate code, and makes the code easier to extend when we want to introduce additional modes in the future. It'll also makes it easier to read back the state of the various modes when implementing the `DECRQM` query. Most of the complication is in the `TerminalInput` class, which had to be adjusted to work with an `enumset` in place of all the `bool` fields. For the rest, it was largely a matter of replacing calls to all the old mode setting methods with the new `SetInputMode` method, and deleting a bunch of unused code. One thing worth mentioning is that the `AdaptDispatch` implementation used to have a `_ShouldPassThroughInputModeChange` method that was called after every mode change. This code has now been moved up into the `SetInputMode` implementation in `ConhostInternalGetSet` so it's just handled in one place. Keeping this out of the dispatch class will also be beneficial for sharing the implementation with `TerminalDispatch`. ## Validation The updated interface necessitated some adjustments to the tests in `AdapterTest` and `MouseInputTest`, but the essential structure of the tests remains unchanged, and everything still passes. I've also tested the keyboard and mouse modes in Vttest and confirmed they still work at least as well as they did before (both conhost and Windows Terminal), and I tested the alternate scroll mode manually (conhost only). Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan to de-duplicate the `AdaptDispatch` and `TerminalDispatch` implementation (#3849).
2021-10-26 23:12:22 +02:00
return _pConApi->SetInputMode(TerminalInput::Mode::SgrMouseEncoding, enabled);
}
//Routine Description:
// Enable Button Event mode - send mouse move events WITH A BUTTON PRESSED to the input.
//Arguments:
// - enabled - true to enable, false to disable.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::EnableButtonEventMouseMode(const bool enabled)
{
Consolidate the interfaces for setting VT input modes (#11384) Instead of having a separate method for setting each mouse and keyboard mode, this PR consolidates them all into a single method which takes a mode parameter, and stores the modes in a `til::enumset` rather than having a separate `bool` for each mode. This enables us to get rid of a lot of boilerplate code, and makes the code easier to extend when we want to introduce additional modes in the future. It'll also makes it easier to read back the state of the various modes when implementing the `DECRQM` query. Most of the complication is in the `TerminalInput` class, which had to be adjusted to work with an `enumset` in place of all the `bool` fields. For the rest, it was largely a matter of replacing calls to all the old mode setting methods with the new `SetInputMode` method, and deleting a bunch of unused code. One thing worth mentioning is that the `AdaptDispatch` implementation used to have a `_ShouldPassThroughInputModeChange` method that was called after every mode change. This code has now been moved up into the `SetInputMode` implementation in `ConhostInternalGetSet` so it's just handled in one place. Keeping this out of the dispatch class will also be beneficial for sharing the implementation with `TerminalDispatch`. ## Validation The updated interface necessitated some adjustments to the tests in `AdapterTest` and `MouseInputTest`, but the essential structure of the tests remains unchanged, and everything still passes. I've also tested the keyboard and mouse modes in Vttest and confirmed they still work at least as well as they did before (both conhost and Windows Terminal), and I tested the alternate scroll mode manually (conhost only). Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan to de-duplicate the `AdaptDispatch` and `TerminalDispatch` implementation (#3849).
2021-10-26 23:12:22 +02:00
return _pConApi->SetInputMode(TerminalInput::Mode::ButtonEventMouseTracking, enabled);
}
//Routine Description:
// Enable Any Event mode - send all mouse events to the input.
//Arguments:
// - enabled - true to enable, false to disable.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::EnableAnyEventMouseMode(const bool enabled)
{
Consolidate the interfaces for setting VT input modes (#11384) Instead of having a separate method for setting each mouse and keyboard mode, this PR consolidates them all into a single method which takes a mode parameter, and stores the modes in a `til::enumset` rather than having a separate `bool` for each mode. This enables us to get rid of a lot of boilerplate code, and makes the code easier to extend when we want to introduce additional modes in the future. It'll also makes it easier to read back the state of the various modes when implementing the `DECRQM` query. Most of the complication is in the `TerminalInput` class, which had to be adjusted to work with an `enumset` in place of all the `bool` fields. For the rest, it was largely a matter of replacing calls to all the old mode setting methods with the new `SetInputMode` method, and deleting a bunch of unused code. One thing worth mentioning is that the `AdaptDispatch` implementation used to have a `_ShouldPassThroughInputModeChange` method that was called after every mode change. This code has now been moved up into the `SetInputMode` implementation in `ConhostInternalGetSet` so it's just handled in one place. Keeping this out of the dispatch class will also be beneficial for sharing the implementation with `TerminalDispatch`. ## Validation The updated interface necessitated some adjustments to the tests in `AdapterTest` and `MouseInputTest`, but the essential structure of the tests remains unchanged, and everything still passes. I've also tested the keyboard and mouse modes in Vttest and confirmed they still work at least as well as they did before (both conhost and Windows Terminal), and I tested the alternate scroll mode manually (conhost only). Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan to de-duplicate the `AdaptDispatch` and `TerminalDispatch` implementation (#3849).
2021-10-26 23:12:22 +02:00
return _pConApi->SetInputMode(TerminalInput::Mode::AnyEventMouseTracking, enabled);
}
//Routine Description:
// Enable Alternate Scroll Mode - When in the Alt Buffer, send CUP and CUD on
// scroll up/down events instead of the usual sequences
//Arguments:
// - enabled - true to enable, false to disable.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::EnableAlternateScroll(const bool enabled)
{
Consolidate the interfaces for setting VT input modes (#11384) Instead of having a separate method for setting each mouse and keyboard mode, this PR consolidates them all into a single method which takes a mode parameter, and stores the modes in a `til::enumset` rather than having a separate `bool` for each mode. This enables us to get rid of a lot of boilerplate code, and makes the code easier to extend when we want to introduce additional modes in the future. It'll also makes it easier to read back the state of the various modes when implementing the `DECRQM` query. Most of the complication is in the `TerminalInput` class, which had to be adjusted to work with an `enumset` in place of all the `bool` fields. For the rest, it was largely a matter of replacing calls to all the old mode setting methods with the new `SetInputMode` method, and deleting a bunch of unused code. One thing worth mentioning is that the `AdaptDispatch` implementation used to have a `_ShouldPassThroughInputModeChange` method that was called after every mode change. This code has now been moved up into the `SetInputMode` implementation in `ConhostInternalGetSet` so it's just handled in one place. Keeping this out of the dispatch class will also be beneficial for sharing the implementation with `TerminalDispatch`. ## Validation The updated interface necessitated some adjustments to the tests in `AdapterTest` and `MouseInputTest`, but the essential structure of the tests remains unchanged, and everything still passes. I've also tested the keyboard and mouse modes in Vttest and confirmed they still work at least as well as they did before (both conhost and Windows Terminal), and I tested the alternate scroll mode manually (conhost only). Simplifying the `ConGetSet` and `ITerminalApi` is also part of the plan to de-duplicate the `AdaptDispatch` and `TerminalDispatch` implementation (#3849).
2021-10-26 23:12:22 +02:00
return _pConApi->SetInputMode(TerminalInput::Mode::AlternateScroll, enabled);
}
//Routine Description:
// Enable "bracketed paste mode".
//Arguments:
// - enabled - true to enable, false to disable.
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::EnableXtermBracketedPasteMode(const bool /*enabled*/) noexcept
{
return NoOp();
}
//Routine Description:
// Set Cursor Style - Changes the cursor's style to match the given Dispatch
// cursor style. Unix styles are a combination of the shape and the blinking state.
//Arguments:
// - cursorStyle - The unix-like cursor style to apply to the cursor
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::SetCursorStyle(const DispatchTypes::CursorStyle cursorStyle)
{
CursorType actualType = CursorType::Legacy;
bool fEnableBlinking = false;
switch (cursorStyle)
{
case DispatchTypes::CursorStyle::UserDefault:
_pConApi->GetUserDefaultCursorStyle(actualType);
fEnableBlinking = true;
break;
case DispatchTypes::CursorStyle::BlinkingBlock:
fEnableBlinking = true;
actualType = CursorType::FullBox;
break;
case DispatchTypes::CursorStyle::SteadyBlock:
fEnableBlinking = false;
actualType = CursorType::FullBox;
break;
case DispatchTypes::CursorStyle::BlinkingUnderline:
fEnableBlinking = true;
actualType = CursorType::Underscore;
break;
case DispatchTypes::CursorStyle::SteadyUnderline:
fEnableBlinking = false;
actualType = CursorType::Underscore;
break;
case DispatchTypes::CursorStyle::BlinkingBar:
fEnableBlinking = true;
actualType = CursorType::VerticalBar;
break;
case DispatchTypes::CursorStyle::SteadyBar:
fEnableBlinking = false;
actualType = CursorType::VerticalBar;
break;
default:
// Invalid argument should be handled by the connected terminal.
return false;
}
bool success = _pConApi->SetCursorStyle(actualType);
if (success)
{
success = _pConApi->PrivateAllowCursorBlinking(fEnableBlinking);
}
// If we're a conpty, always return false, so that this cursor state will be
// sent to the connected terminal
if (_pConApi->IsConsolePty())
{
return false;
}
return success;
}
// Method Description:
// - Sets a single entry of the colortable to a new value
// Arguments:
// - tableIndex: The VT color table index
// - dwColor: The new RGB color value to use.
// Return Value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::SetCursorColor(const COLORREF cursorColor)
{
if (_pConApi->IsConsolePty())
{
return false;
}
Consolidate the color palette APIs (#11784) This PR merges the default colors and cursor color into the main color table, enabling us to simplify the `ConGetSet` and `ITerminalApi` interfaces, with just two methods required for getting and setting any form of color palette entry. The is a follow-up to the color table standardization in #11602, and a another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849. It should also make it easier to support color queries (#3718) and a configurable bold color (#5682) in the future. On the conhost side, default colors could originally be either indexed positions in the 16-color table, or separate standalone RGB values. With the new system, the default colors will always be in the color table, so we just need to track their index positions. To make this work, those positions need to be calculated at startup based on the loaded registry/shortcut settings, and updated when settings are changed (this is handled in `CalculateDefaultColorIndices`). But the plus side is that it's now much easier to lookup the default color values for rendering. For now the default colors in Windows Terminal use hardcoded positions, because it doesn't need indexed default colors like conhost. But in the future I'd like to extend the index handling to both terminals, so we can eventually support the VT525 indexed color operations. As for the cursor color, that was previously stored in the `Cursor` class, which meant that it needed to be copied around in various places where cursors were being instantiated. Now that it's managed separately in the color table, a lot of that code is no longer required. ## Validation Some of the unit test initialization code needed to be updated to setup the color table and default index values as required for the new system. There were also some adjustments needed to account for API changes, in particular for methods that now take index values for the default colors in place of COLORREFs. But for the most part, the essential behavior of the tests remains unchanged. I've also run a variety of manual tests looking at the legacy console APIs as well as the various VT color sequences, and checking that everything works as expected when color schemes are changed, both in Windows Terminal and conhost, and in the latter case with both indexed colors and RGB values. Closes #11768
2021-11-23 19:28:55 +01:00
return _pConApi->SetColorTableEntry(TextColor::CURSOR_COLOR, cursorColor);
}
// Routine Description:
// - OSC Copy to Clipboard
// Arguments:
// - content - The content to copy to clipboard. Must be null terminated.
// Return Value:
// - True if handled successfully. False otherwise.
bool AdaptDispatch::SetClipboard(const std::wstring_view /*content*/) noexcept
{
return false;
}
// Method Description:
// - Sets a single entry of the colortable to a new value
// Arguments:
// - tableIndex: The VT color table index
// - dwColor: The new RGB color value to use.
// Return Value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::SetColorTableEntry(const size_t tableIndex, const DWORD dwColor)
{
Consolidate the color palette APIs (#11784) This PR merges the default colors and cursor color into the main color table, enabling us to simplify the `ConGetSet` and `ITerminalApi` interfaces, with just two methods required for getting and setting any form of color palette entry. The is a follow-up to the color table standardization in #11602, and a another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849. It should also make it easier to support color queries (#3718) and a configurable bold color (#5682) in the future. On the conhost side, default colors could originally be either indexed positions in the 16-color table, or separate standalone RGB values. With the new system, the default colors will always be in the color table, so we just need to track their index positions. To make this work, those positions need to be calculated at startup based on the loaded registry/shortcut settings, and updated when settings are changed (this is handled in `CalculateDefaultColorIndices`). But the plus side is that it's now much easier to lookup the default color values for rendering. For now the default colors in Windows Terminal use hardcoded positions, because it doesn't need indexed default colors like conhost. But in the future I'd like to extend the index handling to both terminals, so we can eventually support the VT525 indexed color operations. As for the cursor color, that was previously stored in the `Cursor` class, which meant that it needed to be copied around in various places where cursors were being instantiated. Now that it's managed separately in the color table, a lot of that code is no longer required. ## Validation Some of the unit test initialization code needed to be updated to setup the color table and default index values as required for the new system. There were also some adjustments needed to account for API changes, in particular for methods that now take index values for the default colors in place of COLORREFs. But for the most part, the essential behavior of the tests remains unchanged. I've also run a variety of manual tests looking at the legacy console APIs as well as the various VT color sequences, and checking that everything works as expected when color schemes are changed, both in Windows Terminal and conhost, and in the latter case with both indexed colors and RGB values. Closes #11768
2021-11-23 19:28:55 +01:00
const bool success = _pConApi->SetColorTableEntry(tableIndex, dwColor);
// If we're a conpty, always return false, so that we send the updated color
// value to the terminal. Still handle the sequence so apps that use
// the API or VT to query the values of the color table still read the
// correct color.
if (_pConApi->IsConsolePty())
{
return false;
}
return success;
}
// Method Description:
// - Sets the default foreground color to a new value
// Arguments:
// - dwColor: The new RGB color value to use, as a COLORREF, format 0x00BBGGRR.
// Return Value:
// True if handled successfully. False otherwise.
Consolidate the color palette APIs (#11784) This PR merges the default colors and cursor color into the main color table, enabling us to simplify the `ConGetSet` and `ITerminalApi` interfaces, with just two methods required for getting and setting any form of color palette entry. The is a follow-up to the color table standardization in #11602, and a another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849. It should also make it easier to support color queries (#3718) and a configurable bold color (#5682) in the future. On the conhost side, default colors could originally be either indexed positions in the 16-color table, or separate standalone RGB values. With the new system, the default colors will always be in the color table, so we just need to track their index positions. To make this work, those positions need to be calculated at startup based on the loaded registry/shortcut settings, and updated when settings are changed (this is handled in `CalculateDefaultColorIndices`). But the plus side is that it's now much easier to lookup the default color values for rendering. For now the default colors in Windows Terminal use hardcoded positions, because it doesn't need indexed default colors like conhost. But in the future I'd like to extend the index handling to both terminals, so we can eventually support the VT525 indexed color operations. As for the cursor color, that was previously stored in the `Cursor` class, which meant that it needed to be copied around in various places where cursors were being instantiated. Now that it's managed separately in the color table, a lot of that code is no longer required. ## Validation Some of the unit test initialization code needed to be updated to setup the color table and default index values as required for the new system. There were also some adjustments needed to account for API changes, in particular for methods that now take index values for the default colors in place of COLORREFs. But for the most part, the essential behavior of the tests remains unchanged. I've also run a variety of manual tests looking at the legacy console APIs as well as the various VT color sequences, and checking that everything works as expected when color schemes are changed, both in Windows Terminal and conhost, and in the latter case with both indexed colors and RGB values. Closes #11768
2021-11-23 19:28:55 +01:00
bool AdaptDispatch::SetDefaultForeground(const DWORD dwColor)
{
bool success = true;
Consolidate the color palette APIs (#11784) This PR merges the default colors and cursor color into the main color table, enabling us to simplify the `ConGetSet` and `ITerminalApi` interfaces, with just two methods required for getting and setting any form of color palette entry. The is a follow-up to the color table standardization in #11602, and a another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849. It should also make it easier to support color queries (#3718) and a configurable bold color (#5682) in the future. On the conhost side, default colors could originally be either indexed positions in the 16-color table, or separate standalone RGB values. With the new system, the default colors will always be in the color table, so we just need to track their index positions. To make this work, those positions need to be calculated at startup based on the loaded registry/shortcut settings, and updated when settings are changed (this is handled in `CalculateDefaultColorIndices`). But the plus side is that it's now much easier to lookup the default color values for rendering. For now the default colors in Windows Terminal use hardcoded positions, because it doesn't need indexed default colors like conhost. But in the future I'd like to extend the index handling to both terminals, so we can eventually support the VT525 indexed color operations. As for the cursor color, that was previously stored in the `Cursor` class, which meant that it needed to be copied around in various places where cursors were being instantiated. Now that it's managed separately in the color table, a lot of that code is no longer required. ## Validation Some of the unit test initialization code needed to be updated to setup the color table and default index values as required for the new system. There were also some adjustments needed to account for API changes, in particular for methods that now take index values for the default colors in place of COLORREFs. But for the most part, the essential behavior of the tests remains unchanged. I've also run a variety of manual tests looking at the legacy console APIs as well as the various VT color sequences, and checking that everything works as expected when color schemes are changed, both in Windows Terminal and conhost, and in the latter case with both indexed colors and RGB values. Closes #11768
2021-11-23 19:28:55 +01:00
success = _pConApi->SetColorTableEntry(TextColor::DEFAULT_FOREGROUND, dwColor);
// If we're a conpty, always return false, so that we send the updated color
// value to the terminal. Still handle the sequence so apps that use
// the API or VT to query the values of the color table still read the
// correct color.
if (_pConApi->IsConsolePty())
{
return false;
}
return success;
}
// Method Description:
// - Sets the default background color to a new value
// Arguments:
// - dwColor: The new RGB color value to use, as a COLORREF, format 0x00BBGGRR.
// Return Value:
// True if handled successfully. False otherwise.
Consolidate the color palette APIs (#11784) This PR merges the default colors and cursor color into the main color table, enabling us to simplify the `ConGetSet` and `ITerminalApi` interfaces, with just two methods required for getting and setting any form of color palette entry. The is a follow-up to the color table standardization in #11602, and a another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849. It should also make it easier to support color queries (#3718) and a configurable bold color (#5682) in the future. On the conhost side, default colors could originally be either indexed positions in the 16-color table, or separate standalone RGB values. With the new system, the default colors will always be in the color table, so we just need to track their index positions. To make this work, those positions need to be calculated at startup based on the loaded registry/shortcut settings, and updated when settings are changed (this is handled in `CalculateDefaultColorIndices`). But the plus side is that it's now much easier to lookup the default color values for rendering. For now the default colors in Windows Terminal use hardcoded positions, because it doesn't need indexed default colors like conhost. But in the future I'd like to extend the index handling to both terminals, so we can eventually support the VT525 indexed color operations. As for the cursor color, that was previously stored in the `Cursor` class, which meant that it needed to be copied around in various places where cursors were being instantiated. Now that it's managed separately in the color table, a lot of that code is no longer required. ## Validation Some of the unit test initialization code needed to be updated to setup the color table and default index values as required for the new system. There were also some adjustments needed to account for API changes, in particular for methods that now take index values for the default colors in place of COLORREFs. But for the most part, the essential behavior of the tests remains unchanged. I've also run a variety of manual tests looking at the legacy console APIs as well as the various VT color sequences, and checking that everything works as expected when color schemes are changed, both in Windows Terminal and conhost, and in the latter case with both indexed colors and RGB values. Closes #11768
2021-11-23 19:28:55 +01:00
bool AdaptDispatch::SetDefaultBackground(const DWORD dwColor)
{
bool success = true;
Consolidate the color palette APIs (#11784) This PR merges the default colors and cursor color into the main color table, enabling us to simplify the `ConGetSet` and `ITerminalApi` interfaces, with just two methods required for getting and setting any form of color palette entry. The is a follow-up to the color table standardization in #11602, and a another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849. It should also make it easier to support color queries (#3718) and a configurable bold color (#5682) in the future. On the conhost side, default colors could originally be either indexed positions in the 16-color table, or separate standalone RGB values. With the new system, the default colors will always be in the color table, so we just need to track their index positions. To make this work, those positions need to be calculated at startup based on the loaded registry/shortcut settings, and updated when settings are changed (this is handled in `CalculateDefaultColorIndices`). But the plus side is that it's now much easier to lookup the default color values for rendering. For now the default colors in Windows Terminal use hardcoded positions, because it doesn't need indexed default colors like conhost. But in the future I'd like to extend the index handling to both terminals, so we can eventually support the VT525 indexed color operations. As for the cursor color, that was previously stored in the `Cursor` class, which meant that it needed to be copied around in various places where cursors were being instantiated. Now that it's managed separately in the color table, a lot of that code is no longer required. ## Validation Some of the unit test initialization code needed to be updated to setup the color table and default index values as required for the new system. There were also some adjustments needed to account for API changes, in particular for methods that now take index values for the default colors in place of COLORREFs. But for the most part, the essential behavior of the tests remains unchanged. I've also run a variety of manual tests looking at the legacy console APIs as well as the various VT color sequences, and checking that everything works as expected when color schemes are changed, both in Windows Terminal and conhost, and in the latter case with both indexed colors and RGB values. Closes #11768
2021-11-23 19:28:55 +01:00
success = _pConApi->SetColorTableEntry(TextColor::DEFAULT_BACKGROUND, dwColor);
// If we're a conpty, always return false, so that we send the updated color
// value to the terminal. Still handle the sequence so apps that use
// the API or VT to query the values of the color table still read the
// correct color.
if (_pConApi->IsConsolePty())
{
return false;
}
return success;
}
//Routine Description:
// Window Manipulation - Performs a variety of actions relating to the window,
// such as moving the window position, resizing the window, querying
// window state, forcing the window to repaint, etc.
// This is kept separate from the input version, as there may be
// codes that are supported in one direction but not the other.
//Arguments:
// - function - An identifier of the WindowManipulation function to perform
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
// - parameter1 - The first optional parameter for the function
// - parameter2 - The second optional parameter for the function
// Return value:
// True if handled successfully. False otherwise.
bool AdaptDispatch::WindowManipulation(const DispatchTypes::WindowManipulationType function,
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
const VTParameter parameter1,
const VTParameter parameter2)
{
bool success = false;
// Other Window Manipulation functions:
// MSFT:13271098 - QueryViewport
// MSFT:13271146 - QueryScreenSize
switch (function)
{
case DispatchTypes::WindowManipulationType::RefreshWindow:
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
success = DispatchCommon::s_RefreshWindow(*_pConApi);
break;
case DispatchTypes::WindowManipulationType::ResizeWindowInCharacters:
Refactor VT parameter handling (#7799) This PR introduces a pair of classes for managing VT parameters that automatically handle range checking and default fallback values, so the individual operations don't have to do that validation themselves. In addition to simplifying the code, this fixes a few cases where we were mishandling missing or extraneous parameters, and adds support for parameter sequences on commands that couldn't previously handle them. This PR also sets a limit on the number of parameters allowed, to help thwart DoS memory consumption attacks. ## References * The new parameter class also introduces the concept of an omitted/default parameter which is not necessarily zero, which is a prerequisite for addressing issue #4417. ## Detailed Description of the Pull Request / Additional comments There are two new classes provide by this PR: a `VTParameter` class, similar in function to a `std::optional<size_t>`, which holds an individual parameter (which may be an omitted/default value); and a `VTParameters` class, similar in function to `gsl:span<VTParameter>`, which holds a sequence of those parameters. Where `VTParameter` differs from `std::optional` is with the inclusion of two cast operators. There is a `size_t` cast that interprets omitted and zero values as 1 (the expected behaviour for most numeric parameters). And there is a generic cast, for use with the enum parameter types, which interprets omitted values as 0 (the expected behaviour for most selective parameters). The advantage of `VTParameters` class is that it has an `at` method that can never fail - out of range values simply return the a default `VTParameter` instance (this is standard behaviour in VT terminals). It also has a `size` method that will always return a minimum count of 1, since an empty parameter list is typically the equivalent of a single "default" parameter, so this guarantees you'll get at least one value when iterating over the list with `size()`. For cases where we just need to call the same dispatch method for every parameter, there is a helper `for_each` method, which repeatedly calls a given predicate function with each value in the sequence. It also collates the returned success values to determine the overall result of the sequence. As with the `size` method, this will always make at least one call, so it correctly handles empty sequences. With those two classes in place, we could get rid of all the parameter validation and default handling code in the `OutputStateMachineEngine`. We now just use the `VTParameters::at` method to grab a parameter and typically pass it straight to the appropriate dispatch method, letting the cast operators automatically handle the assignment of default values. Occasionally we might need a `value_or` call to specify a non-standard default value, but those cases are fairly rare. In some case the `OutputStateMachineEngine` was also checking whether parameters values were in range, but for the most part this shouldn't have been necessary, since that is something the dispatch classes would already have been doing themselves (in the few cases that they weren't, I've now updated them to do so). I've also updated the `InputStateMachineEngine` in a similar way to the `OutputStateMachineEngine`, getting rid of a few of the parameter extraction methods, and simplifying other parts of the implementation. It's not as clean a replacement as the output engine, but there are still benefits in using the new classes. ## Validation Steps Performed For the most part I haven't had to alter existing tests other than accounting for changes to the API. There were a couple of tests I needed to drop because they were checking for failure cases which shouldn't have been failing (unexpected parameters should never be an error), or testing output engine validation that is no longer handled at that level. I've added a few new tests to cover operations that take sequences of selective parameters (`ED`, `EL`, `TBC`, `SM`, and `RM`). And I've extended the cursor movement tests to make sure those operations can handle extraneous parameters that weren't expected. I've also added a test to verify that the state machine will correctly ignore parameters beyond the maximum 32 parameter count limit. I've also manual confirmed that the various test cases given in issues #2101 are now working as expected. Closes #2101
2020-10-15 18:12:52 +02:00
success = DispatchCommon::s_ResizeWindow(*_pConApi, parameter2.value_or(0), parameter1.value_or(0));
break;
default:
success = false;
}
return success;
}
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
// Method Description:
// - Starts a hyperlink
// Arguments:
// - The hyperlink URI, optional additional parameters
// Return Value:
// - true
bool AdaptDispatch::AddHyperlink(const std::wstring_view uri, const std::wstring_view params)
{
return _pConApi->PrivateAddHyperlink(uri, params);
}
// Method Description:
// - Ends a hyperlink
// Return Value:
// - true
bool AdaptDispatch::EndHyperlink()
{
return _pConApi->PrivateEndHyperlink();
}
// Method Description:
// - Ascribes to the ITermDispatch interface
// - Not actually used in conhost
// Return Value:
// - false (so that the command gets flushed to terminal)
bool AdaptDispatch::DoConEmuAction(const std::wstring_view /*string*/) noexcept
{
return false;
}
Add support for downloadable soft fonts (#10011) This PR adds conhost support for downloadable soft fonts - also known as dynamically redefinable character sets (DRCS) - using the `DECDLD` escape sequence. These fonts are typically designed to work on a specific terminal model, and each model tends to have a different character cell size. So in order to support as many models as possible, the code attempts to detect the original target size of the font, and then scale the glyphs to fit our current cell size. Once a font has been downloaded to the terminal, it can be designated in the same way you would a standard character set, using an `SCS` escape sequence. The identification string for the set is defined by the `DECDLD` sequence. Internally we map the characters in this set to code points `U+EF20` to `U+EF7F` in the Unicode private use are (PUA). Then in the renderer, any characters in that range are split off into separate runs, which get painted with a special font. The font itself is dynamically generated as an in-memory resource, constructed from the downloaded character bitmaps which have been scaled to the appropriate size. If no soft fonts are in use, then no mapping of the PUA code points will take place, so this shouldn't interfere with anyone using those code points for something else, as along as they aren't also trying to use soft fonts. I also tried to pick a PUA range that hadn't already been snatched up by Nerd Fonts, but if we do receive reports of a conflict, it's easy enough to change. ## Validation Steps Performed I added an adapter test that runs through a bunch of parameter variations for the `DECDLD` sequence, to make sure we're correctly detecting the font sizes for most of the known DEC terminal models. I've also tested manually on a wide range of existing fonts, of varying dimensions, and from multiple sources, and made sure they all worked reasonably well. Closes #9164
2021-08-06 22:41:02 +02:00
// Method Description:
// - DECDLD - Downloads one or more characters of a dynamically redefinable
// character set (DRCS) with a specified pixel pattern. The pixel array is
// transmitted in sixel format via the returned StringHandler function.
// Arguments:
// - fontNumber - The buffer number into which the font will be loaded.
// - startChar - The first character in the set that will be replaced.
// - eraseControl - Which characters to erase before loading the new data.
// - cellMatrix - The character cell width (sometimes also height in legacy formats).
// - fontSet - The screen size for which the font is designed.
// - fontUsage - Whether it is a text font or a full-cell font.
// - cellHeight - The character cell height (if not defined by cellMatrix).
// - charsetSize - Whether the character set is 94 or 96 characters.
// Return Value:
// - a function to receive the pixel data or nullptr if parameters are invalid
ITermDispatch::StringHandler AdaptDispatch::DownloadDRCS(const size_t fontNumber,
const VTParameter startChar,
const DispatchTypes::DrcsEraseControl eraseControl,
const DispatchTypes::DrcsCellMatrix cellMatrix,
const DispatchTypes::DrcsFontSet fontSet,
const DispatchTypes::DrcsFontUsage fontUsage,
const VTParameter cellHeight,
const DispatchTypes::DrcsCharsetSize charsetSize)
{
// If we're a conpty, we're just going to ignore the operation for now.
// There's no point in trying to pass it through without also being able
// to pass through the character set designations.
if (_pConApi->IsConsolePty())
{
return nullptr;
}
// The font buffer is created on demand.
if (!_fontBuffer)
{
_fontBuffer = std::make_unique<FontBuffer>();
}
// Only one font buffer is supported, so only 0 (default) and 1 are valid.
auto success = fontNumber <= 1;
success = success && _fontBuffer->SetEraseControl(eraseControl);
success = success && _fontBuffer->SetAttributes(cellMatrix, cellHeight, fontSet, fontUsage);
success = success && _fontBuffer->SetStartChar(startChar, charsetSize);
// If any of the parameters are invalid, we return a null handler to let
// the state machine know we want to ignore the subsequent data string.
if (!success)
{
return nullptr;
}
return [=](const auto ch) {
// We pass the data string straight through to the font buffer class
// until we receive an ESC, indicating the end of the string. At that
// point we can finalize the buffer, and if valid, update the renderer
// with the constructed bit pattern.
if (ch != AsciiChars::ESC)
{
_fontBuffer->AddSixelData(ch);
}
else if (_fontBuffer->FinalizeSixelData())
{
// We also need to inform the character set mapper of the ID that
// will map to this font (we only support one font buffer so there
// will only ever be one active dynamic character set).
if (charsetSize == DispatchTypes::DrcsCharsetSize::Size96)
{
_termOutput.SetDrcs96Designation(_fontBuffer->GetDesignation());
}
else
{
_termOutput.SetDrcs94Designation(_fontBuffer->GetDesignation());
}
const auto bitPattern = _fontBuffer->GetBitPattern();
const auto cellSize = _fontBuffer->GetCellSize();
const auto centeringHint = _fontBuffer->GetTextCenteringHint();
_pConApi->PrivateUpdateSoftFont(bitPattern, cellSize, centeringHint);
}
return true;
};
}
// Method Description:
// - DECRQSS - Requests the state of a VT setting. The value being queried is
// identified by the intermediate and final characters of its control
// sequence, which are passed to the string handler.
// Arguments:
// - None
// Return Value:
// - a function to receive the VTID of the setting being queried
ITermDispatch::StringHandler AdaptDispatch::RequestSetting()
{
// We use a VTIDBuilder to parse the characters in the control string into
// an ID which represents the setting being queried. If the given ID isn't
// supported, we respond with an error sequence: DCS 0 $ r ST. Note that
// this is the opposite of what is documented in most DEC manuals, which
// say that 0 is for a valid response, and 1 is for an error. The correct
// interpretation is documented in the DEC STD 070 reference.
const auto idBuilder = std::make_shared<VTIDBuilder>();
return [=](const auto ch) {
if (ch >= '\x40' && ch <= '\x7e')
{
const auto id = idBuilder->Finalize(ch);
switch (id)
{
case VTID('m'):
_ReportSGRSetting();
break;
case VTID('r'):
_ReportDECSTBMSetting();
break;
default:
_WriteResponse(L"\033P0$r\033\\");
break;
}
return false;
}
else
{
if (ch >= '\x20' && ch <= '\x2f')
{
idBuilder->AddIntermediate(ch);
}
return true;
}
};
}
// Method Description:
// - Reports the current SGR attributes in response to a DECRQSS query.
// Arguments:
// - None
// Return Value:
// - None
void AdaptDispatch::_ReportSGRSetting() const
{
using namespace std::string_view_literals;
// A valid response always starts with DCS 1 $ r.
// Then the '0' parameter is to reset the SGR attributes to the defaults.
fmt::basic_memory_buffer<wchar_t, 64> response;
response.append(L"\033P1$r0"sv);
TextAttribute attr;
if (_pConApi->PrivateGetTextAttributes(attr))
{
// For each boolean attribute that is set, we add the appropriate
// parameter value to the response string.
const auto addAttribute = [&](const auto& parameter, const auto enabled) {
if (enabled)
{
response.append(parameter);
}
};
addAttribute(L";1"sv, attr.IsBold());
addAttribute(L";2"sv, attr.IsFaint());
addAttribute(L";3"sv, attr.IsItalic());
addAttribute(L";4"sv, attr.IsUnderlined());
addAttribute(L";5"sv, attr.IsBlinking());
addAttribute(L";7"sv, attr.IsReverseVideo());
addAttribute(L";8"sv, attr.IsInvisible());
addAttribute(L";9"sv, attr.IsCrossedOut());
addAttribute(L";21"sv, attr.IsDoublyUnderlined());
addAttribute(L";53"sv, attr.IsOverlined());
// We also need to add the appropriate color encoding parameters for
// both the foreground and background colors.
const auto addColor = [&](const auto base, const auto color) {
if (color.IsIndex16())
{
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const auto index = color.GetIndex();
const auto colorParameter = base + (index >= 8 ? 60 : 0) + (index % 8);
fmt::format_to(std::back_inserter(response), FMT_COMPILE(L";{}"), colorParameter);
}
else if (color.IsIndex256())
{
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const auto index = color.GetIndex();
fmt::format_to(std::back_inserter(response), FMT_COMPILE(L";{};5;{}"), base + 8, index);
}
else if (color.IsRgb())
{
const auto r = GetRValue(color.GetRGB());
const auto g = GetGValue(color.GetRGB());
const auto b = GetBValue(color.GetRGB());
fmt::format_to(std::back_inserter(response), FMT_COMPILE(L";{};2;{};{};{}"), base + 8, r, g, b);
}
};
addColor(30, attr.GetForeground());
addColor(40, attr.GetBackground());
}
// The 'm' indicates this is an SGR response, and ST ends the sequence.
response.append(L"m\033\\"sv);
_WriteResponse({ response.data(), response.size() });
}
// Method Description:
// - Reports the DECSTBM margin range in response to a DECRQSS query.
// Arguments:
// - None
// Return Value:
// - None
void AdaptDispatch::_ReportDECSTBMSetting() const
{
using namespace std::string_view_literals;
// A valid response always starts with DCS 1 $ r.
fmt::basic_memory_buffer<wchar_t, 64> response;
response.append(L"\033P1$r"sv);
CONSOLE_SCREEN_BUFFER_INFOEX csbiex = { 0 };
csbiex.cbSize = sizeof(CONSOLE_SCREEN_BUFFER_INFOEX);
if (_pConApi->GetConsoleScreenBufferInfoEx(csbiex))
{
auto marginTop = _scrollMargins.Top + 1;
auto marginBottom = _scrollMargins.Bottom + 1;
// If the margin top is greater than or equal to the bottom, then the
// margins aren't actually set, so we need to return the full height
// of the window for the margin range.
if (marginTop >= marginBottom)
{
marginTop = 1;
marginBottom = csbiex.srWindow.Bottom - csbiex.srWindow.Top;
}
fmt::format_to(std::back_inserter(response), FMT_COMPILE(L"{};{}"), marginTop, marginBottom);
}
// The 'r' indicates this is an DECSTBM response, and ST ends the sequence.
response.append(L"r\033\\"sv);
_WriteResponse({ response.data(), response.size() });
}