terminal/src/host/_stream.cpp

1325 lines
58 KiB
C++
Raw Normal View History

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "ApiRoutines.h"
#include "_stream.h"
#include "stream.h"
#include "writeData.hpp"
#include "_output.h"
#include "output.h"
#include "dbcs.h"
#include "handle.h"
#include "misc.h"
#include "../types/inc/convert.hpp"
#include "../types/inc/GlyphWidth.hpp"
#include "../types/inc/Viewport.hpp"
#include "../interactivity/inc/ServiceLocator.hpp"
#pragma hdrstop
using namespace Microsoft::Console::Types;
using Microsoft::Console::Interactivity::ServiceLocator;
using Microsoft::Console::VirtualTerminal::StateMachine;
// Used by WriteCharsLegacy.
Reverse the behavior of the IS_GLYPH_CHAR macro so its function now matches its name. (#4209) ## Summary of the Pull Request This PR reverses the behaviour of the `IS_GLYPH_CHAR` macro, so it now actually returns true if the given char is a glyph, and false if it isn't. Previously it returned the opposite of that, which meant it had to be called as `!IS_GLYPH_CHAR` to get the correct result. ## PR Checklist * [x] Closes #4185 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #4185 ## Detailed Description of the Pull Request / Additional comments The original implementation returned true if the given character was a C0 control, or a DEL: #define IS_GLYPH_CHAR(wch) (((wch) < L' ') || ((wch) == 0x007F)) It's now the exact opposite, so returns true for characters that are _not_ C0 controls, and are not the DEL character either: #define IS_GLYPH_CHAR(wch) (((wch) >= L' ') && ((wch) != 0x007F)) The macro was only used in one place, where is was being called as `!IS_GLYPH_CHAR` when the intent was actually to test whether the char _was_ a glyph. That code could now be updated to remove the `!`, so it makes more sense. ## Validation Steps Performed I've just tested manually and confirmed that basic output of text and control chars still worked as expected in a conhost shell.
2020-01-14 17:43:38 +01:00
#define IS_GLYPH_CHAR(wch) (((wch) >= L' ') && ((wch) != 0x007F))
// Routine Description:
// - This routine updates the cursor position. Its input is the non-special
// cased new location of the cursor. For example, if the cursor were being
// moved one space backwards from the left edge of the screen, the X
// coordinate would be -1. This routine would set the X coordinate to
// the right edge of the screen and decrement the Y coordinate by one.
// Arguments:
// - screenInfo - reference to screen buffer information structure.
// - coordCursor - New location of cursor.
// - fKeepCursorVisible - TRUE if changing window origin desirable when hit right edge
// Return Value:
[[nodiscard]] NTSTATUS AdjustCursorPosition(SCREEN_INFORMATION& screenInfo,
_In_ COORD coordCursor,
const BOOL fKeepCursorVisible,
_Inout_opt_ PSHORT psScrollY)
{
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
const bool inVtMode = WI_IsFlagSet(screenInfo.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
const COORD bufferSize = screenInfo.GetBufferSize().Dimensions();
if (coordCursor.X < 0)
{
if (coordCursor.Y > 0)
{
coordCursor.X = (SHORT)(bufferSize.X + coordCursor.X);
coordCursor.Y = (SHORT)(coordCursor.Y - 1);
}
else
{
coordCursor.X = 0;
}
}
else if (coordCursor.X >= bufferSize.X)
{
// at end of line. if wrap mode, wrap cursor. otherwise leave it where it is.
if (screenInfo.OutputMode & ENABLE_WRAP_AT_EOL_OUTPUT)
{
coordCursor.Y += coordCursor.X / bufferSize.X;
coordCursor.X = coordCursor.X % bufferSize.X;
}
else
{
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
if (inVtMode)
{
// In VT mode, the cursor must be left in the last column.
coordCursor.X = bufferSize.X - 1;
}
else
{
// For legacy apps, it is left where it was at the start of the write.
coordCursor.X = screenInfo.GetTextBuffer().GetCursor().GetPosition().X;
}
}
}
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
// The VT standard requires the lines revealed when scrolling are filled
// with the current background color, but with no meta attributes set.
auto fillAttributes = screenInfo.GetAttributes();
fillAttributes.SetStandardErase();
const auto relativeMargins = screenInfo.GetRelativeScrollMargins();
auto viewport = screenInfo.GetViewport();
SMALL_RECT srMargins = screenInfo.GetAbsoluteScrollMargins().ToInclusive();
const bool fMarginsSet = srMargins.Bottom > srMargins.Top;
COORD currentCursor = screenInfo.GetTextBuffer().GetCursor().GetPosition();
const int iCurrentCursorY = currentCursor.Y;
const bool fCursorInMargins = iCurrentCursorY <= srMargins.Bottom && iCurrentCursorY >= srMargins.Top;
const bool cursorAboveViewport = coordCursor.Y < 0 && inVtMode;
const bool fScrollDown = fMarginsSet && fCursorInMargins && (coordCursor.Y > srMargins.Bottom);
bool fScrollUp = fMarginsSet && fCursorInMargins && (coordCursor.Y < srMargins.Top);
const bool fScrollUpWithoutMargins = (!fMarginsSet) && cursorAboveViewport;
// if we're in VT mode, AND MARGINS AREN'T SET and a Reverse Line Feed took the cursor up past the top of the viewport,
// VT style scroll the contents of the screen.
// This can happen in applications like `less`, that don't set margins, because they're going to
// scroll the entire screen anyways, so no need for them to ever set the margins.
if (fScrollUpWithoutMargins)
{
fScrollUp = true;
srMargins.Top = 0;
srMargins.Bottom = screenInfo.GetViewport().BottomInclusive();
}
const bool scrollDownAtTop = fScrollDown && relativeMargins.Top() == 0;
if (scrollDownAtTop)
{
// We're trying to scroll down, and the top margin is at the top of the viewport.
// In this case, we want the lines that are "scrolled off" to appear in
// the scrollback instead of being discarded.
// To do this, we're going to scroll everything starting at the bottom
// margin down, then move the viewport down.
const SHORT delta = coordCursor.Y - srMargins.Bottom;
SMALL_RECT scrollRect{ 0 };
scrollRect.Left = 0;
scrollRect.Top = srMargins.Bottom + 1; // One below margins
scrollRect.Bottom = bufferSize.Y - 1; // -1, otherwise this would be an exclusive rect.
scrollRect.Right = bufferSize.X - 1; // -1, otherwise this would be an exclusive rect.
// This is the Y position we're moving the contents below the bottom margin to.
SHORT moveToYPosition = scrollRect.Top + delta;
// This is where the viewport will need to be to give the effect of
// scrolling the contents in the margins.
SHORT newViewTop = viewport.Top() + delta;
// This is how many new lines need to be added to the buffer to support this operation.
const SHORT newRows = (viewport.BottomExclusive() + delta) - bufferSize.Y;
// If we're near the bottom of the buffer, we might need to insert some
// new rows at the bottom.
// If we do this, then the viewport is now one line higher than it used
// to be, so it needs to move down by one less line.
for (auto i = 0; i < newRows; i++)
{
screenInfo.GetTextBuffer().IncrementCircularBuffer();
moveToYPosition--;
newViewTop--;
scrollRect.Top--;
}
const COORD newPostMarginsOrigin = { 0, moveToYPosition };
const COORD newViewOrigin = { 0, newViewTop };
try
{
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
ScrollRegion(screenInfo, scrollRect, std::nullopt, newPostMarginsOrigin, UNICODE_SPACE, fillAttributes);
}
CATCH_LOG();
// Move the viewport down
auto hr = screenInfo.SetViewportOrigin(true, newViewOrigin, true);
if (FAILED(hr))
{
return NTSTATUS_FROM_HRESULT(hr);
}
// If we didn't actually move the viewport, it's because we're at the
// bottom of the buffer, and the top lines of the viewport have
// changed. Manually invalidate here, to make sure the screen
// displays the correct text.
if (newViewOrigin == viewport.Origin())
{
Correct scrolling invalidation region for tmux in pty w/ bitmap (#5122) Correct scrolling invalidation region for tmux in pty w/ bitmap Add tracing for circling and scrolling operations. Fix improper invalidation within AdjustCursorPosition routine in the subsection about scrolling down at the bottom with a set of margins enabled. ## References - Introduced with #5024 ## Detailed Description of the Pull Request / Additional comments - This occurs when there is a scroll region restriction applied and a newline operation is performed to attempt to spin the contents of just the scroll region. This is a frequent behavior of tmux. - Right now, the Terminal doesn't support any sort of "scroll content" operation, so what happens here generally speaking is that the PTY in the ConHost will repaint everything when this happens. - The PTY when doing `AdjustCursorPosition` with a scroll region restriction would do the following things: 1. Slide literally everything in the direction it needed to go to take advantage of rotating the circular buffer. (This would force a repaint in PTY as the PTY always forces repaint when the buffer circles.) 2. Copy the lines that weren't supposed to move back to where they were supposed to go. 3. Backfill the "revealed" region that encompasses what was supposed to be the newline. - The invalidations for the three operations above were: 1. Invalidate the number of rows of the delta at the top of the buffer (this part was wrong) 2. Invalidate the lines that got copied back into position (probably unnecessary, but OK) 3. Invalidate the revealed/filled-with-spaces line (this is good). - When we were using a simple single rectangle for invalidation, the union of the top row of the buffer from 1 and the bottom row of the buffer from 2 (and 3 was irrelevant as it was already unioned it) resulted in repainting the entire buffer and all was good. - When we switched to a bitmap, it dutifully only repainted the top line and the bottom two lines as the middle ones weren't a consequence of intersect. - The logic was wrong. We shouldn't be invalidating rows-from-the-top for the amount of the delta. The 1 part should be invalidating everything BUT the lines that were invalidated in parts 2 and 3. (Arguably part 2 shouldn't be happening at all, but I'm not optimizing for that right now.) - So this solves it by restoring an entire screen repaint for this sort of slide data operation by giving the correct number of invalidated lines to the bitmap. ## Validation Steps Performed - Manual validation with the steps described in #5104 - Automatic test `ConptyRoundtripTests::ScrollWithMargins`. Closes #5104
2020-03-27 23:37:23 +01:00
// Inside this block, we're shifting down at the bottom.
// This means that we had something like this:
// AAAA
// BBBB
// CCCC
// DDDD
// EEEE
//
// Our margins were set for lines A-D, but not on line E.
// So we circled the whole buffer up by one:
// BBBB
// CCCC
// DDDD
// EEEE
// <blank, was AAAA>
//
// Then we scrolled the contents of everything OUTSIDE the margin frame down.
// BBBB
// CCCC
// DDDD
// <blank, filled during scroll down of EEEE>
// EEEE
//
// And now we need to report that only the bottom line didn't "move" as we put the EEEE
// back where it started, but everything else moved.
// In this case, delta was 1. So the amount that moved is the entire viewport height minus the delta.
Viewport invalid = Viewport::FromDimensions(viewport.Origin(), { viewport.Width(), viewport.Height() - delta });
screenInfo.GetRenderTarget().TriggerRedraw(invalid);
}
// reset where our local viewport is, and recalculate the cursor and
// margin positions.
viewport = screenInfo.GetViewport();
if (newRows > 0)
{
currentCursor.Y -= newRows;
coordCursor.Y -= newRows;
}
srMargins = screenInfo.GetAbsoluteScrollMargins().ToInclusive();
}
// If we did the above scrollDownAtTop case, then we've already scrolled
// the margins content, and we can skip this.
if (fScrollUp || (fScrollDown && !scrollDownAtTop))
{
SHORT diff = coordCursor.Y - (fScrollUp ? srMargins.Top : srMargins.Bottom);
SMALL_RECT scrollRect = { 0 };
scrollRect.Top = srMargins.Top;
scrollRect.Bottom = srMargins.Bottom;
Remove unwanted DECSTBM clipping (#2764) The `DECSTBM` margins are meant to define the range of lines within which certain vertical scrolling operations take place. However, we were applying these margin restrictions in the `ScrollRegion` function, which is also used in a number of places that shouldn't be affected by `DECSTBM`. This includes the `ICH` and `DCH` escape sequences (which are only affected by the horizontal margins, which we don't yet support), the `ScrollConsoleScreenBuffer` API (which is public Console API, not meant to be affected by the VT terminal emulation), and the `CSI 3 J` erase scrollback extension (which isn't really scrolling as such, but uses the `ScrollRegion` function to manipulate the scrollback buffer). This commit moves the margin clipping out of the `ScrollRegion` function, so it can be applied exclusively in the places that need it. With the margin clipping removed from the `ScrollRegion` function, it now had to be applied manually in the places it was actually required. This included: * The `DoSrvPrivateReverseLineFeed` function (for the `RI` control): This was * just a matter of updating the bottom of the scroll rect to the bottom margin * (at least when the margins were actually set), since the top of the scroll * rect would always be the top of the viewport. The * `DoSrvPrivateModifyLinesImpl` function (for the `IL` and `DL` commands): * Again this was just a matter of updating the bottom of the scroll rect, since * the cursor position would always determine the top of the scroll rect. The * `AdaptDispatch::_ScrollMovement` method (for the `SU` and `SD` commands): * This required updating both the top and bottom coordinates of the scroll * rect, and also a simpler destination Y coordinate (the way the `ScrollRegion` * function worked before, the caller was expected to take the margins into * account when determining the destination). On the plus side, there was now no longer a need to override the margins when calling `ScrollRegion` in the `AdjustCursorPosition` function. In the first case, the margins had needed to be cleared (_stream.cpp 143-145), but that is now the default behaviour. In the second case, there had been a more complicated adjustment of the margins (_stream.cpp 196-209), but that code was never actually used so could be removed completely (to get to that point either _fScrollUp_ was true, so _scrollDownAtTop_ couldn't also be true, or _fScrollDown_ was true, but in that case there is a check to make sure _scrollDownAtTop_ is false). While testing, I also noticed that one of the `ScrollRegion` calls in the `AdjustCursorPosition` function was not setting the horizontal range correctly - the scrolling should always affect the full buffer width rather than just the viewport width - so I've fixed that now as well. ## Validation Steps Performed For commands like `RI`, `IL`, `DL`, etc. where we've changed the implementation but not the behaviour, there were already unit tests that could confirm that the new implementation was still producing the correct results. Where there has been a change in behaviour - namely for the `ICH` and `DCH` commands, and the `ScrollConsoleScreenBuffer` API - I've extended the existing unit tests to check that they still function correctly even when the `DECSTBM` margins are set (which would previously have caused them to fail). I've also tested manually with the test cases in issues #2543 and #2659, and confirmed that they now work as expected. Closes #2543 Closes #2659
2019-09-24 01:16:54 +02:00
scrollRect.Left = 0; // NOTE: Left/Right Scroll margins don't do anything currently.
scrollRect.Right = bufferSize.X - 1; // -1, otherwise this would be an exclusive rect.
COORD dest;
dest.X = scrollRect.Left;
dest.Y = scrollRect.Top - diff;
try
{
Correct fill attributes when scrolling and erasing (#3100) ## Summary of the Pull Request Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing. ## PR Checklist * [x] Closes #2553 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. ## Detailed Description of the Pull Request / Additional comments My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all. To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset). With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`. The functions affected by scrolling included: * `DoSrvPrivateReverseLineFeed` (the RI command) * `DoSrvPrivateModifyLinesImpl` (the IL and DL commands) * `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands) * `AdaptDispatch::_ScrollMovement` (the SU and SD commands) The functions affected by erasing included: * `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants) * `AdaptDispatch::EraseCharacters` (the ECH command) While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time. In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes: * `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer. * `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised. * `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes. The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above). Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations. Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed. Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). ## Validation Steps Performed For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region. In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer. The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests. Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-11 00:14:40 +01:00
ScrollRegion(screenInfo, scrollRect, scrollRect, dest, UNICODE_SPACE, fillAttributes);
}
CATCH_LOG();
coordCursor.Y -= diff;
}
// If the margins are set, then it shouldn't be possible for the cursor to
// move below the bottom of the viewport. Either it should be constrained
// inside the margins by one of the scrollDown cases handled above, or
// we'll need to clamp it inside the viewport here.
if (fMarginsSet && coordCursor.Y > viewport.BottomInclusive())
{
coordCursor.Y = viewport.BottomInclusive();
}
NTSTATUS Status = STATUS_SUCCESS;
if (coordCursor.Y >= bufferSize.Y)
{
// At the end of the buffer. Scroll contents of screen buffer so new position is visible.
FAIL_FAST_IF(!(coordCursor.Y == bufferSize.Y));
if (!StreamScrollRegion(screenInfo))
{
Status = STATUS_NO_MEMORY;
}
if (nullptr != psScrollY)
{
*psScrollY += (SHORT)(bufferSize.Y - coordCursor.Y - 1);
}
coordCursor.Y += (SHORT)(bufferSize.Y - coordCursor.Y - 1);
}
const bool cursorMovedPastViewport = coordCursor.Y > screenInfo.GetViewport().BottomInclusive();
const bool cursorMovedPastVirtualViewport = coordCursor.Y > screenInfo.GetVirtualViewport().BottomInclusive();
if (NT_SUCCESS(Status))
{
// if at right or bottom edge of window, scroll right or down one char.
if (cursorMovedPastViewport)
{
COORD WindowOrigin;
WindowOrigin.X = 0;
WindowOrigin.Y = coordCursor.Y - screenInfo.GetViewport().BottomInclusive();
Status = screenInfo.SetViewportOrigin(false, WindowOrigin, true);
}
}
if (NT_SUCCESS(Status))
{
if (fKeepCursorVisible)
{
screenInfo.MakeCursorVisible(coordCursor);
}
Status = screenInfo.SetCursorPosition(coordCursor, !!fKeepCursorVisible);
// MSFT:19989333 - Only re-initialize the cursor row if the cursor moved
// below the terminal section of the buffer (the virtual viewport),
// and the visible part of the buffer (the actual viewport).
// If this is only cursorMovedPastViewport, and you scroll up, then type
// a character, we'll re-initialize the line the cursor is on.
// If this is only cursorMovedPastVirtualViewport and you scroll down,
// (with terminal scrolling disabled) then all lines newly exposed
// will get their attributes constantly cleared out.
// Both cursorMovedPastViewport and cursorMovedPastVirtualViewport works
if (inVtMode && cursorMovedPastViewport && cursorMovedPastVirtualViewport)
{
screenInfo.InitializeCursorRowAttributes();
}
}
return Status;
}
// Routine Description:
// - This routine writes a string to the screen, processing any embedded
// unicode characters. The string is also copied to the input buffer, if
// the output mode is line mode.
// Arguments:
// - screenInfo - reference to screen buffer information structure.
// - pwchBufferBackupLimit - Pointer to beginning of buffer.
// - pwchBuffer - Pointer to buffer to copy string to. assumed to be at least as long as pwchRealUnicode.
// This pointer is updated to point to the next position in the buffer.
// - pwchRealUnicode - Pointer to string to write.
// - pcb - On input, number of bytes to write. On output, number of bytes written.
// - pcSpaces - On output, the number of spaces consumed by the written characters.
// - dwFlags -
// WC_DESTRUCTIVE_BACKSPACE backspace overwrites characters.
// WC_KEEP_CURSOR_VISIBLE change window origin desirable when hit rt. edge
// WC_PRINTABLE_CONTROL_CHARS if control characters should be expanded (as in, to "^X")
// Return Value:
// Note:
// - This routine does not process tabs and backspace properly. That code will be implemented as part of the line editing services.
[[nodiscard]] NTSTATUS WriteCharsLegacy(SCREEN_INFORMATION& screenInfo,
_In_range_(<=, pwchBuffer) const wchar_t* const pwchBufferBackupLimit,
_In_ const wchar_t* pwchBuffer,
_In_reads_bytes_(*pcb) const wchar_t* pwchRealUnicode,
_Inout_ size_t* const pcb,
_Out_opt_ size_t* const pcSpaces,
const SHORT sOriginalXPosition,
const DWORD dwFlags,
_Inout_opt_ PSHORT const psScrollY)
{
const CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
TextBuffer& textBuffer = screenInfo.GetTextBuffer();
Cursor& cursor = textBuffer.GetCursor();
COORD CursorPosition = cursor.GetPosition();
NTSTATUS Status = STATUS_SUCCESS;
SHORT XPosition;
size_t TempNumSpaces = 0;
const bool fUnprocessed = WI_IsFlagClear(screenInfo.OutputMode, ENABLE_PROCESSED_OUTPUT);
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
const bool fWrapAtEOL = WI_IsFlagSet(screenInfo.OutputMode, ENABLE_WRAP_AT_EOL_OUTPUT);
// Must not adjust cursor here. It has to stay on for many write scenarios. Consumers should call for the
// cursor to be turned off if they want that.
const TextAttribute Attributes = screenInfo.GetAttributes();
const size_t BufferSize = *pcb;
*pcb = 0;
const wchar_t* lpString = pwchRealUnicode;
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
COORD coordScreenBufferSize = screenInfo.GetBufferSize().Dimensions();
// In VT mode, the width at which we wrap is determined by the line rendition attribute.
if (WI_IsFlagSet(screenInfo.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING))
{
coordScreenBufferSize.X = textBuffer.GetLineWidth(CursorPosition.Y);
}
static constexpr unsigned int LOCAL_BUFFER_SIZE = 1024;
WCHAR LocalBuffer[LOCAL_BUFFER_SIZE];
while (*pcb < BufferSize)
{
// correct for delayed EOL
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
if (cursor.IsDelayedEOLWrap() && fWrapAtEOL)
{
const COORD coordDelayedAt = cursor.GetDelayedAtPosition();
cursor.ResetDelayEOLWrap();
// Only act on a delayed EOL if we didn't move the cursor to a different position from where the EOL was marked.
if (coordDelayedAt.X == CursorPosition.X && coordDelayedAt.Y == CursorPosition.Y)
{
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
CursorPosition.X = 0;
CursorPosition.Y++;
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
Status = AdjustCursorPosition(screenInfo, CursorPosition, WI_IsFlagSet(dwFlags, WC_KEEP_CURSOR_VISIBLE), psScrollY);
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
CursorPosition = cursor.GetPosition();
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
// In VT mode, we need to recalculate the width when moving to a new line.
if (WI_IsFlagSet(screenInfo.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING))
{
coordScreenBufferSize.X = textBuffer.GetLineWidth(CursorPosition.Y);
}
}
}
// As an optimization, collect characters in buffer and print out all at once.
XPosition = cursor.GetPosition().X;
size_t i = 0;
wchar_t* LocalBufPtr = LocalBuffer;
while (*pcb < BufferSize && i < LOCAL_BUFFER_SIZE && XPosition < coordScreenBufferSize.X)
{
#pragma prefast(suppress : 26019, "Buffer is taken in multiples of 2. Validation is ok.")
const wchar_t Char = *lpString;
// WCL-NOTE: We believe RealUnicodeChar to be identical to Char, because we believe pwchRealUnicode
// WCL-NOTE: to be identical to lpString. They are incremented in lockstep, never separately, and lpString
// WCL-NOTE: is initialized from pwchRealUnicode.
const wchar_t RealUnicodeChar = *pwchRealUnicode;
Reverse the behavior of the IS_GLYPH_CHAR macro so its function now matches its name. (#4209) ## Summary of the Pull Request This PR reverses the behaviour of the `IS_GLYPH_CHAR` macro, so it now actually returns true if the given char is a glyph, and false if it isn't. Previously it returned the opposite of that, which meant it had to be called as `!IS_GLYPH_CHAR` to get the correct result. ## PR Checklist * [x] Closes #4185 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #4185 ## Detailed Description of the Pull Request / Additional comments The original implementation returned true if the given character was a C0 control, or a DEL: #define IS_GLYPH_CHAR(wch) (((wch) < L' ') || ((wch) == 0x007F)) It's now the exact opposite, so returns true for characters that are _not_ C0 controls, and are not the DEL character either: #define IS_GLYPH_CHAR(wch) (((wch) >= L' ') && ((wch) != 0x007F)) The macro was only used in one place, where is was being called as `!IS_GLYPH_CHAR` when the intent was actually to test whether the char _was_ a glyph. That code could now be updated to remove the `!`, so it makes more sense. ## Validation Steps Performed I've just tested manually and confirmed that basic output of text and control chars still worked as expected in a conhost shell.
2020-01-14 17:43:38 +01:00
if (IS_GLYPH_CHAR(RealUnicodeChar) || fUnprocessed)
{
// WCL-NOTE: This operates on a single code unit instead of a whole codepoint. It will mis-measure surrogate pairs.
if (IsGlyphFullWidth(Char))
{
if (i < (LOCAL_BUFFER_SIZE - 1) && XPosition < (coordScreenBufferSize.X - 1))
{
*LocalBufPtr++ = Char;
// cursor adjusted by 2 because the char is double width
XPosition += 2;
i += 1;
pwchBuffer++;
}
else
{
goto EndWhile;
}
}
else
{
*LocalBufPtr = Char;
LocalBufPtr++;
XPosition++;
i++;
pwchBuffer++;
}
}
else
{
FAIL_FAST_IF(!(WI_IsFlagSet(screenInfo.OutputMode, ENABLE_PROCESSED_OUTPUT)));
switch (RealUnicodeChar)
{
case UNICODE_BELL:
if (dwFlags & WC_PRINTABLE_CONTROL_CHARS)
{
goto CtrlChar;
}
else
{
screenInfo.SendNotifyBeep();
}
break;
case UNICODE_BACKSPACE:
// automatically go to EndWhile. this is because
// backspace is not destructive, so "aBkSp" prints
// a with the cursor on the "a". we could achieve
// this behavior staying in this loop and figuring out
// the string that needs to be printed, but it would
// be expensive and it's the exceptional case.
goto EndWhile;
break;
case UNICODE_TAB:
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
{
const ULONG TabSize = NUMBER_OF_SPACES_IN_TAB(XPosition);
XPosition = (SHORT)(XPosition + TabSize);
if (XPosition >= coordScreenBufferSize.X)
{
goto EndWhile;
}
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
for (ULONG j = 0; j < TabSize && i < LOCAL_BUFFER_SIZE; j++, i++)
{
*LocalBufPtr = UNICODE_SPACE;
LocalBufPtr++;
}
pwchBuffer++;
break;
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
}
case UNICODE_LINEFEED:
case UNICODE_CARRIAGERETURN:
goto EndWhile;
default:
// if char is ctrl char, write ^char.
if ((dwFlags & WC_PRINTABLE_CONTROL_CHARS) && (IS_CONTROL_CHAR(RealUnicodeChar)))
{
CtrlChar:
if (i < (LOCAL_BUFFER_SIZE - 1))
{
// WCL-NOTE: We do not properly measure that there is space for two characters
// WCL-NOTE: left on the screen.
*LocalBufPtr = (WCHAR)'^';
LocalBufPtr++;
XPosition++;
i++;
*LocalBufPtr = (WCHAR)(RealUnicodeChar + (WCHAR)'@');
LocalBufPtr++;
XPosition++;
i++;
pwchBuffer++;
}
else
{
goto EndWhile;
}
}
else
{
Fix display of the "low ASCII" glyphs in PC code pages (#1964) In the legacy console, it used to be possible to write out characters from the C0 range of a PC code page (e.g. CP437), and get the actual glyphs defined for those code points (at least those that weren't processed as control codes). In the v2 console this stopped working so you'd get an FFFD replacement glyph (�) for those characters instead. This PR fixes the issue so the correct glyphs are displayed again. There was already code in place to achieve this in the `WriteCharsLegacy` method. It used the `GetStringTypeW` method to determine the character type of the value being output, and if it was a `C1_CNTRL` character it performed the appropriate mapping. The problem was that the test of the character type flag was done as a direct comparision, when it should have been a bit test, so the condition was never met. With this condition fixed, the code also needed to be reordered slightly to handle the null character. That had a special-case mapping to space, which was previously performed after the control test, but since a null character now successfully matches `C1_CNTRL`, it no longer falls through to that special case. To address that, I've had to move the null check above the control test. I've tested this manually, by trying to output all the characters in the affected range (ASCII values 0 to 31, and 127, excluding the actual control codes 8,9,10 and 13). In all cases they now match the output that the legacy console produced. Note that this only applies to PC code pages that have glyphs defined for the C0 range, so it won't work with the UTF-8 code page, but that was to be expected - the legacy console behaved the same way. Also, note that this only works when the `ENABLE_PROCESSED_OUTPUT` console mode is set. That seems wrong to me (I'd expect the glyphs to work in both cases), but that's the way the legacy console behaved as well, so if that's a bug it's a separate issue. I haven't added any unit tests, because I expect the behaviour of some of these characters to change over time (as support is added for more control codes), which could then cause the tests to fail. But if that's not a concern, I could probably add something to the ScreenBufferTests (perhaps with a comment warning that the tests might be expected to fail in the future). Closes #166.
2019-09-20 22:42:36 +02:00
if (Char == UNICODE_NULL)
{
*LocalBufPtr = UNICODE_SPACE;
}
else
{
Fix display of the "low ASCII" glyphs in PC code pages (#1964) In the legacy console, it used to be possible to write out characters from the C0 range of a PC code page (e.g. CP437), and get the actual glyphs defined for those code points (at least those that weren't processed as control codes). In the v2 console this stopped working so you'd get an FFFD replacement glyph (�) for those characters instead. This PR fixes the issue so the correct glyphs are displayed again. There was already code in place to achieve this in the `WriteCharsLegacy` method. It used the `GetStringTypeW` method to determine the character type of the value being output, and if it was a `C1_CNTRL` character it performed the appropriate mapping. The problem was that the test of the character type flag was done as a direct comparision, when it should have been a bit test, so the condition was never met. With this condition fixed, the code also needed to be reordered slightly to handle the null character. That had a special-case mapping to space, which was previously performed after the control test, but since a null character now successfully matches `C1_CNTRL`, it no longer falls through to that special case. To address that, I've had to move the null check above the control test. I've tested this manually, by trying to output all the characters in the affected range (ASCII values 0 to 31, and 127, excluding the actual control codes 8,9,10 and 13). In all cases they now match the output that the legacy console produced. Note that this only applies to PC code pages that have glyphs defined for the C0 range, so it won't work with the UTF-8 code page, but that was to be expected - the legacy console behaved the same way. Also, note that this only works when the `ENABLE_PROCESSED_OUTPUT` console mode is set. That seems wrong to me (I'd expect the glyphs to work in both cases), but that's the way the legacy console behaved as well, so if that's a bug it's a separate issue. I haven't added any unit tests, because I expect the behaviour of some of these characters to change over time (as support is added for more control codes), which could then cause the tests to fail. But if that's not a concern, I could probably add something to the ScreenBufferTests (perhaps with a comment warning that the tests might be expected to fail in the future). Closes #166.
2019-09-20 22:42:36 +02:00
// As a special favor to incompetent apps that attempt to display control chars,
// convert to corresponding OEM Glyph Chars
WORD CharType;
GetStringTypeW(CT_CTYPE1, &RealUnicodeChar, 1, &CharType);
if (WI_IsFlagSet(CharType, C1_CNTRL))
{
ConvertOutputToUnicode(gci.OutputCP,
(LPSTR)&RealUnicodeChar,
1,
LocalBufPtr,
1);
}
else
{
// WCL-NOTE: We should never hit this.
// WCL-NOTE: 1. Normal characters are handled via the early check for IS_GLYPH_CHAR
// WCL-NOTE: 2. Control characters are handled via the CtrlChar label (if WC_PRINTABLE_CONTROL_CHARS is on)
// WCL-NOTE: And if they are control characters they will trigger the C1_CNTRL check above.
Fix display of the "low ASCII" glyphs in PC code pages (#1964) In the legacy console, it used to be possible to write out characters from the C0 range of a PC code page (e.g. CP437), and get the actual glyphs defined for those code points (at least those that weren't processed as control codes). In the v2 console this stopped working so you'd get an FFFD replacement glyph (�) for those characters instead. This PR fixes the issue so the correct glyphs are displayed again. There was already code in place to achieve this in the `WriteCharsLegacy` method. It used the `GetStringTypeW` method to determine the character type of the value being output, and if it was a `C1_CNTRL` character it performed the appropriate mapping. The problem was that the test of the character type flag was done as a direct comparision, when it should have been a bit test, so the condition was never met. With this condition fixed, the code also needed to be reordered slightly to handle the null character. That had a special-case mapping to space, which was previously performed after the control test, but since a null character now successfully matches `C1_CNTRL`, it no longer falls through to that special case. To address that, I've had to move the null check above the control test. I've tested this manually, by trying to output all the characters in the affected range (ASCII values 0 to 31, and 127, excluding the actual control codes 8,9,10 and 13). In all cases they now match the output that the legacy console produced. Note that this only applies to PC code pages that have glyphs defined for the C0 range, so it won't work with the UTF-8 code page, but that was to be expected - the legacy console behaved the same way. Also, note that this only works when the `ENABLE_PROCESSED_OUTPUT` console mode is set. That seems wrong to me (I'd expect the glyphs to work in both cases), but that's the way the legacy console behaved as well, so if that's a bug it's a separate issue. I haven't added any unit tests, because I expect the behaviour of some of these characters to change over time (as support is added for more control codes), which could then cause the tests to fail. But if that's not a concern, I could probably add something to the ScreenBufferTests (perhaps with a comment warning that the tests might be expected to fail in the future). Closes #166.
2019-09-20 22:42:36 +02:00
*LocalBufPtr = Char;
}
}
LocalBufPtr++;
XPosition++;
i++;
pwchBuffer++;
}
}
}
lpString++;
pwchRealUnicode++;
*pcb += sizeof(WCHAR);
}
EndWhile:
if (i != 0)
{
CursorPosition = cursor.GetPosition();
// Make sure we don't write past the end of the buffer.
// WCL-NOTE: This check uses a code unit count instead of a column count. That is incorrect.
if (i > gsl::narrow_cast<size_t>(coordScreenBufferSize.X) - CursorPosition.X)
{
i = gsl::narrow_cast<size_t>(coordScreenBufferSize.X) - CursorPosition.X;
}
// line was wrapped if we're writing up to the end of the current row
OutputCellIterator it(std::wstring_view(LocalBuffer, i), Attributes);
const auto itEnd = screenInfo.Write(it);
// Notify accessibility
if (screenInfo.HasAccessibilityEventing())
{
screenInfo.NotifyAccessibilityEventing(CursorPosition.X, CursorPosition.Y, CursorPosition.X + gsl::narrow<SHORT>(i - 1), CursorPosition.Y);
}
// The number of "spaces" or "cells" we have consumed needs to be reported and stored for later
// when/if we need to erase the command line.
TempNumSpaces += itEnd.GetCellDistance(it);
// WCL-NOTE: We are using the "estimated" X position delta instead of the actual delta from
// WCL-NOTE: the iterator. It is not clear why. If they differ, the cursor ends up in the
// WCL-NOTE: wrong place (typically inside another character).
CursorPosition.X = XPosition;
// enforce a delayed newline if we're about to pass the end and the WC_DELAY_EOL_WRAP flag is set.
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
if (WI_IsFlagSet(dwFlags, WC_DELAY_EOL_WRAP) && CursorPosition.X >= coordScreenBufferSize.X && fWrapAtEOL)
{
// Our cursor position as of this time is going to remain on the last position in this column.
CursorPosition.X = coordScreenBufferSize.X - 1;
// Update in the structures that we're still pointing to the last character in the row
cursor.SetPosition(CursorPosition);
// Record for the delay comparison that we're delaying on the last character in the row
cursor.DelayEOLWrap(CursorPosition);
}
else
{
Status = AdjustCursorPosition(screenInfo, CursorPosition, WI_IsFlagSet(dwFlags, WC_KEEP_CURSOR_VISIBLE), psScrollY);
}
// WCL-NOTE: If we have processed the entire input string during our "fast one-line print" handler,
// WCL-NOTE: we are done as there is nothing more to do. Neat!
if (*pcb == BufferSize)
{
if (nullptr != pcSpaces)
{
*pcSpaces = TempNumSpaces;
}
return STATUS_SUCCESS;
}
continue;
}
else if (*pcb >= BufferSize)
{
// WCL-NOTE: This case looks like it is never encountered, but it *is* if WC_PRINTABLE_CONTROL_CHARS is off.
// WCL-NOTE: If the string is entirely nonprinting control characters, there will be
// WCL-NOTE: no output in the buffer (LocalBuffer; i == 0) but we will have processed
// WCL-NOTE: "every" character. We can just bail out and report the number of spaces consumed.
FAIL_FAST_IF(!(WI_IsFlagSet(screenInfo.OutputMode, ENABLE_PROCESSED_OUTPUT)));
// this catches the case where the number of backspaces == the number of characters.
if (nullptr != pcSpaces)
{
*pcSpaces = TempNumSpaces;
}
return STATUS_SUCCESS;
}
FAIL_FAST_IF(!(WI_IsFlagSet(screenInfo.OutputMode, ENABLE_PROCESSED_OUTPUT)));
switch (*lpString)
{
case UNICODE_BACKSPACE:
{
// move cursor backwards one space. overwrite current char with blank.
// we get here because we have to backspace from the beginning of the line
TempNumSpaces -= 1;
if (pwchBuffer == pwchBufferBackupLimit)
{
CursorPosition.X -= 1;
}
else
{
const wchar_t* Tmp;
wchar_t* Tmp2 = nullptr;
WCHAR LastChar;
const size_t bufferSize = pwchBuffer - pwchBufferBackupLimit;
std::unique_ptr<wchar_t[]> buffer;
try
{
buffer = std::make_unique<wchar_t[]>(bufferSize);
std::fill_n(buffer.get(), bufferSize, UNICODE_NULL);
}
catch (...)
{
return NTSTATUS_FROM_HRESULT(wil::ResultFromCaughtException());
}
for (i = 0, Tmp2 = buffer.get(), Tmp = pwchBufferBackupLimit;
i < bufferSize;
i++, Tmp++)
{
// see 18120085, these two need to be separate if statements
if (*Tmp == UNICODE_BACKSPACE)
{
//it is important we do nothing in the else case for
// this one instead of falling through to the below else.
if (Tmp2 > buffer.get())
{
Tmp2--;
}
}
else
{
FAIL_FAST_IF(!(Tmp2 >= buffer.get()));
*Tmp2++ = *Tmp;
}
}
if (Tmp2 == buffer.get())
{
LastChar = UNICODE_SPACE;
}
else
{
#pragma prefast(suppress : 26001, "This is fine. Tmp2 has to have advanced or it would equal pBuffer.")
LastChar = *(Tmp2 - 1);
}
if (LastChar == UNICODE_TAB)
{
CursorPosition.X -= (SHORT)(RetrieveNumberOfSpaces(sOriginalXPosition,
pwchBufferBackupLimit,
(ULONG)(pwchBuffer - pwchBufferBackupLimit - 1)));
if (CursorPosition.X < 0)
{
CursorPosition.X = (coordScreenBufferSize.X - 1) / TAB_SIZE;
CursorPosition.X *= TAB_SIZE;
CursorPosition.X += 1;
CursorPosition.Y -= 1;
// since you just backspaced yourself back up into the previous row, unset the wrap
// flag on the prev row if it was set
textBuffer.GetRowByOffset(CursorPosition.Y).SetWrapForced(false);
}
}
else if (IS_CONTROL_CHAR(LastChar))
{
CursorPosition.X -= 1;
TempNumSpaces -= 1;
// overwrite second character of ^x sequence.
if (dwFlags & WC_DESTRUCTIVE_BACKSPACE)
{
try
{
screenInfo.Write(OutputCellIterator(UNICODE_SPACE, Attributes, 1), CursorPosition);
Status = STATUS_SUCCESS;
}
CATCH_LOG();
}
CursorPosition.X -= 1;
}
else if (IsGlyphFullWidth(LastChar))
{
CursorPosition.X -= 1;
TempNumSpaces -= 1;
Status = AdjustCursorPosition(screenInfo, CursorPosition, dwFlags & WC_KEEP_CURSOR_VISIBLE, psScrollY);
if (dwFlags & WC_DESTRUCTIVE_BACKSPACE)
{
try
{
screenInfo.Write(OutputCellIterator(UNICODE_SPACE, Attributes, 1), CursorPosition);
Status = STATUS_SUCCESS;
}
CATCH_LOG();
}
CursorPosition.X -= 1;
}
else
{
CursorPosition.X--;
}
}
if ((dwFlags & WC_LIMIT_BACKSPACE) && (CursorPosition.X < 0))
{
CursorPosition.X = 0;
OutputDebugStringA(("CONSRV: Ignoring backspace to previous line\n"));
}
Status = AdjustCursorPosition(screenInfo, CursorPosition, (dwFlags & WC_KEEP_CURSOR_VISIBLE) != 0, psScrollY);
if (dwFlags & WC_DESTRUCTIVE_BACKSPACE)
{
try
{
screenInfo.Write(OutputCellIterator(UNICODE_SPACE, Attributes, 1), cursor.GetPosition());
}
CATCH_LOG();
}
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
if (cursor.GetPosition().X == 0 && fWrapAtEOL && pwchBuffer > pwchBufferBackupLimit)
{
if (CheckBisectProcessW(screenInfo,
pwchBufferBackupLimit,
pwchBuffer + 1 - pwchBufferBackupLimit,
gsl::narrow_cast<size_t>(coordScreenBufferSize.X) - sOriginalXPosition,
sOriginalXPosition,
dwFlags & WC_PRINTABLE_CONTROL_CHARS))
{
CursorPosition.X = coordScreenBufferSize.X - 1;
CursorPosition.Y = (SHORT)(cursor.GetPosition().Y - 1);
// since you just backspaced yourself back up into the previous row, unset the wrap flag
// on the prev row if it was set
textBuffer.GetRowByOffset(CursorPosition.Y).SetWrapForced(false);
Status = AdjustCursorPosition(screenInfo, CursorPosition, dwFlags & WC_KEEP_CURSOR_VISIBLE, psScrollY);
}
}
break;
}
case UNICODE_TAB:
{
const size_t TabSize = gsl::narrow_cast<size_t>(NUMBER_OF_SPACES_IN_TAB(cursor.GetPosition().X));
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
CursorPosition.X = (SHORT)(cursor.GetPosition().X + TabSize);
// move cursor forward to next tab stop. fill space with blanks.
// we get here when the tab extends beyond the right edge of the
// window. if the tab goes wraps the line, set the cursor to the first
// position in the next line.
pwchBuffer++;
TempNumSpaces += TabSize;
size_t NumChars = 0;
if (CursorPosition.X >= coordScreenBufferSize.X)
{
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
NumChars = gsl::narrow<size_t>(coordScreenBufferSize.X - cursor.GetPosition().X);
CursorPosition.X = 0;
CursorPosition.Y = cursor.GetPosition().Y + 1;
// since you just tabbed yourself past the end of the row, set the wrap
textBuffer.GetRowByOffset(cursor.GetPosition().Y).SetWrapForced(true);
}
else
{
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
NumChars = gsl::narrow<size_t>(CursorPosition.X - cursor.GetPosition().X);
CursorPosition.Y = cursor.GetPosition().Y;
}
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
try
{
const OutputCellIterator it(UNICODE_SPACE, Attributes, NumChars);
const auto done = screenInfo.Write(it, cursor.GetPosition());
NumChars = done.GetCellDistance(it);
}
Remove unneeded VT-specific control character handling (#4289) ## Summary of the Pull Request This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason. ## References This is a followup to PR #4171 ## PR Checklist * [x] Closes #3971 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: https://github.com/microsoft/terminal/issues/780#issuecomment-570287435 ## Detailed Description of the Pull Request / Additional comments There are four changes to the `WriteCharsLegacy` implementation: 1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971. 2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version. 3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line. Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR. Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private. ## Validation Steps Performed I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.
2020-01-29 20:18:46 +01:00
CATCH_LOG();
Status = AdjustCursorPosition(screenInfo, CursorPosition, (dwFlags & WC_KEEP_CURSOR_VISIBLE) != 0, psScrollY);
break;
}
case UNICODE_CARRIAGERETURN:
{
// Carriage return moves the cursor to the beginning of the line.
// We don't need to worry about handling cr or lf for
// backspace because input is sent to the user on cr or lf.
pwchBuffer++;
CursorPosition.X = 0;
CursorPosition.Y = cursor.GetPosition().Y;
Status = AdjustCursorPosition(screenInfo, CursorPosition, (dwFlags & WC_KEEP_CURSOR_VISIBLE) != 0, psScrollY);
break;
}
case UNICODE_LINEFEED:
{
// move cursor to the next line.
pwchBuffer++;
if (gci.IsReturnOnNewlineAutomatic())
{
// Traditionally, we reset the X position to 0 with a newline automatically.
// Some things might not want this automatic "ONLCR line discipline" (for example, things that are expecting a *NIX behavior.)
// They will turn it off with an output mode flag.
CursorPosition.X = 0;
}
CursorPosition.Y = (SHORT)(cursor.GetPosition().Y + 1);
{
// since we explicitly just moved down a row, clear the wrap status on the row we just came from
textBuffer.GetRowByOffset(cursor.GetPosition().Y).SetWrapForced(false);
}
Status = AdjustCursorPosition(screenInfo, CursorPosition, (dwFlags & WC_KEEP_CURSOR_VISIBLE) != 0, psScrollY);
break;
}
default:
{
const wchar_t Char = *lpString;
if (Char >= UNICODE_SPACE &&
IsGlyphFullWidth(Char) &&
XPosition >= (coordScreenBufferSize.X - 1) &&
Add support for VT100 Auto Wrap Mode (DECAWM) (#3943) ## Summary of the Pull Request This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests. ## PR Checklist * [x] Closes #3826 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826 ## Detailed Description of the Pull Request / Additional comments The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked, To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT` is also set. Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line. This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode. Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received. One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied. ## Validation Steps Performed I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states. I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.
2020-02-04 01:20:21 +01:00
fWrapAtEOL)
{
const COORD TargetPoint = cursor.GetPosition();
ROW& Row = textBuffer.GetRowByOffset(TargetPoint.Y);
const CharRow& charRow = Row.GetCharRow();
try
{
// If we're on top of a trailing cell, clear it and the previous cell.
if (charRow.DbcsAttrAt(TargetPoint.X).IsTrailing())
{
// Space to clear for 2 cells.
OutputCellIterator it(UNICODE_SPACE, 2);
// Back target point up one.
auto writeTarget = TargetPoint;
writeTarget.X--;
// Write 2 clear cells.
screenInfo.Write(it, writeTarget);
}
}
catch (...)
{
return NTSTATUS_FROM_HRESULT(wil::ResultFromCaughtException());
}
CursorPosition.X = 0;
CursorPosition.Y = (SHORT)(TargetPoint.Y + 1);
// since you just moved yourself down onto the next row with 1 character, that sounds like a
// forced wrap so set the flag
Row.SetWrapForced(true);
// Additionally, this padding is only called for IsConsoleFullWidth (a.k.a. when a character
// is too wide to fit on the current line).
Row.SetDoubleBytePadded(true);
Status = AdjustCursorPosition(screenInfo, CursorPosition, dwFlags & WC_KEEP_CURSOR_VISIBLE, psScrollY);
continue;
}
break;
}
}
if (!NT_SUCCESS(Status))
{
return Status;
}
*pcb += sizeof(WCHAR);
lpString++;
pwchRealUnicode++;
}
if (nullptr != pcSpaces)
{
*pcSpaces = TempNumSpaces;
}
return STATUS_SUCCESS;
}
// Routine Description:
// - This routine writes a string to the screen, processing any embedded
// unicode characters. The string is also copied to the input buffer, if
// the output mode is line mode.
// Arguments:
// - screenInfo - reference to screen buffer information structure.
// - pwchBufferBackupLimit - Pointer to beginning of buffer.
// - pwchBuffer - Pointer to buffer to copy string to. assumed to be at least as long as pwchRealUnicode.
// This pointer is updated to point to the next position in the buffer.
// - pwchRealUnicode - Pointer to string to write.
// - pcb - On input, number of bytes to write. On output, number of bytes written.
// - pcSpaces - On output, the number of spaces consumed by the written characters.
// - dwFlags -
// WC_DESTRUCTIVE_BACKSPACE backspace overwrites characters.
// WC_KEEP_CURSOR_VISIBLE change window origin (viewport) desirable when hit rt. edge
// WC_PRINTABLE_CONTROL_CHARS if control characters should be expanded (as in, to "^X")
// Return Value:
// Note:
// - This routine does not process tabs and backspace properly. That code will be implemented as part of the line editing services.
[[nodiscard]] NTSTATUS WriteChars(SCREEN_INFORMATION& screenInfo,
_In_range_(<=, pwchBuffer) const wchar_t* const pwchBufferBackupLimit,
_In_ const wchar_t* pwchBuffer,
_In_reads_bytes_(*pcb) const wchar_t* pwchRealUnicode,
_Inout_ size_t* const pcb,
_Out_opt_ size_t* const pcSpaces,
const SHORT sOriginalXPosition,
const DWORD dwFlags,
_Inout_opt_ PSHORT const psScrollY)
{
if (!WI_IsFlagSet(screenInfo.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING) ||
!WI_IsFlagSet(screenInfo.OutputMode, ENABLE_PROCESSED_OUTPUT))
{
return WriteCharsLegacy(screenInfo,
pwchBufferBackupLimit,
pwchBuffer,
pwchRealUnicode,
pcb,
pcSpaces,
sOriginalXPosition,
dwFlags,
psScrollY);
}
NTSTATUS Status = STATUS_SUCCESS;
size_t const BufferSize = *pcb;
*pcb = 0;
{
size_t TempNumSpaces = 0;
{
if (NT_SUCCESS(Status))
{
FAIL_FAST_IF(!(WI_IsFlagSet(screenInfo.OutputMode, ENABLE_PROCESSED_OUTPUT)));
FAIL_FAST_IF(!(WI_IsFlagSet(screenInfo.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING)));
// defined down in the WriteBuffer default case hiding on the other end of the state machine. See outputStream.cpp
// This is the only mode used by DoWriteConsole.
FAIL_FAST_IF(!(WI_IsFlagSet(dwFlags, WC_LIMIT_BACKSPACE)));
StateMachine& machine = screenInfo.GetStateMachine();
size_t const cch = BufferSize / sizeof(WCHAR);
machine.ProcessString({ pwchRealUnicode, cch });
*pcb += BufferSize;
}
}
if (nullptr != pcSpaces)
{
*pcSpaces = TempNumSpaces;
}
}
return Status;
}
// Routine Description:
// - Takes the given text and inserts it into the given screen buffer.
// Note:
// - Console lock must be held when calling this routine
// - String has been translated to unicode at this point.
// Arguments:
// - pwchBuffer - wide character text to be inserted into buffer
// - pcbBuffer - byte count of pwchBuffer on the way in, number of bytes consumed on the way out.
// - screenInfo - Screen Information class to write the text into at the current cursor position
// - ppWaiter - If writing to the console is blocked for whatever reason, this will be filled with a pointer to context
// that can be used by the server to resume the call at a later time.
// Return Value:
// - STATUS_SUCCESS if OK.
// - CONSOLE_STATUS_WAIT if we couldn't finish now and need to be called back later (see ppWaiter).
// - Or a suitable NTSTATUS format error code for memory/string/math failures.
[[nodiscard]] NTSTATUS DoWriteConsole(_In_reads_bytes_(*pcbBuffer) PWCHAR pwchBuffer,
_Inout_ size_t* const pcbBuffer,
SCREEN_INFORMATION& screenInfo,
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
bool requiresVtQuirk,
std::unique_ptr<WriteData>& waiter)
{
const CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
if (WI_IsAnyFlagSet(gci.Flags, (CONSOLE_SUSPENDED | CONSOLE_SELECTING | CONSOLE_SCROLLBAR_TRACKING)))
{
try
{
waiter = std::make_unique<WriteData>(screenInfo,
pwchBuffer,
*pcbBuffer,
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
gci.OutputCP,
requiresVtQuirk);
}
catch (...)
{
return NTSTATUS_FROM_HRESULT(wil::ResultFromCaughtException());
}
return CONSOLE_STATUS_WAIT;
}
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
auto restoreVtQuirk{
wil::scope_exit([&]() { screenInfo.ResetIgnoreLegacyEquivalentVTAttributes(); })
};
if (requiresVtQuirk)
{
screenInfo.SetIgnoreLegacyEquivalentVTAttributes();
}
else
{
restoreVtQuirk.release();
}
const auto& textBuffer = screenInfo.GetTextBuffer();
return WriteChars(screenInfo,
pwchBuffer,
pwchBuffer,
pwchBuffer,
pcbBuffer,
nullptr,
textBuffer.GetCursor().GetPosition().X,
WC_LIMIT_BACKSPACE,
nullptr);
}
// Routine Description:
// - This method performs the actual work of attempting to write to the console, converting data types as necessary
// to adapt from the server types to the legacy internal host types.
// - It operates on Unicode data only. It's assumed the text is translated by this point.
// Arguments:
// - OutContext - the console output object to write the new text into
// - pwsTextBuffer - wide character text buffer provided by client application to insert
// - cchTextBufferLength - text buffer counted in characters
// - pcchTextBufferRead - character count of the number of characters we were able to insert before returning
// - ppWaiter - If we are blocked from writing now and need to wait, this is filled with contextual data for the server to restore the call later
// Return Value:
// - S_OK if successful.
// - S_OK if we need to wait (check if ppWaiter is not nullptr).
// - Or a suitable HRESULT code for math/string/memory failures.
[[nodiscard]] HRESULT WriteConsoleWImplHelper(IConsoleOutputObject& context,
const std::wstring_view buffer,
size_t& read,
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
bool requiresVtQuirk,
std::unique_ptr<WriteData>& waiter) noexcept
{
try
{
// Set out variables in case we exit early.
read = 0;
waiter.reset();
// Convert characters to bytes to give to DoWriteConsole.
size_t cbTextBufferLength;
RETURN_IF_FAILED(SizeTMult(buffer.size(), sizeof(wchar_t), &cbTextBufferLength));
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
NTSTATUS Status = DoWriteConsole(const_cast<wchar_t*>(buffer.data()), &cbTextBufferLength, context, requiresVtQuirk, waiter);
// Convert back from bytes to characters for the resulting string length written.
read = cbTextBufferLength / sizeof(wchar_t);
if (Status == CONSOLE_STATUS_WAIT)
{
FAIL_FAST_IF_NULL(waiter.get());
Status = STATUS_SUCCESS;
}
RETURN_NTSTATUS(Status);
}
CATCH_RETURN();
}
// Routine Description:
// - Writes non-Unicode formatted data into the given console output object.
// - This method will convert from the given input into wide characters before chain calling the wide character version of the function.
// It uses the current Output Codepage for conversions (set via SetConsoleOutputCP).
// - NOTE: This may be blocked for various console states and will return a wait context pointer if necessary.
// Arguments:
// - context - the console output object to write the new text into
// - buffer - char/byte text buffer provided by client application to insert
// - read - character count of the number of characters (also bytes because A version) we were able to insert before returning
// - waiter - If we are blocked from writing now and need to wait, this is filled with contextual data for the server to restore the call later
// Return Value:
// - S_OK if successful.
// - S_OK if we need to wait (check if ppWaiter is not nullptr).
// - Or a suitable HRESULT code for math/string/memory failures.
[[nodiscard]] HRESULT ApiRoutines::WriteConsoleAImpl(IConsoleOutputObject& context,
const std::string_view buffer,
size_t& read,
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
bool requiresVtQuirk,
std::unique_ptr<IWaitRoutine>& waiter) noexcept
{
try
{
// Ensure output variables are initialized.
read = 0;
waiter.reset();
if (buffer.empty())
{
return S_OK;
}
LockConsole();
auto unlock{ wil::scope_exit([&] { UnlockConsole(); }) };
auto& screenInfo{ context.GetActiveBuffer() };
const auto& consoleInfo{ ServiceLocator::LocateGlobals().getConsoleInformation() };
const auto codepage{ consoleInfo.OutputCP };
auto leadByteCaptured{ false };
auto leadByteConsumed{ false };
std::wstring wstr{};
static til::u8state u8State{};
// Convert our input parameters to Unicode
if (codepage == CP_UTF8)
{
RETURN_IF_FAILED(til::u8u16(buffer, wstr, u8State));
read = buffer.size();
}
else
{
// In case the codepage changes from UTF-8 to another,
// we discard partials that might still be cached.
u8State.reset();
int mbPtrLength{};
RETURN_IF_FAILED(SizeTToInt(buffer.size(), &mbPtrLength));
// (buffer.size() + 2) I think because we might be shoving another unicode char
// from screenInfo->WriteConsoleDbcsLeadByte in front
// because we previously checked that buffer.size() fits into an int, +2 won't cause an overflow of size_t
wstr.resize(buffer.size() + 2);
wchar_t* wcPtr{ wstr.data() };
auto mbPtr{ buffer.data() };
size_t dbcsLength{};
if (screenInfo.WriteConsoleDbcsLeadByte[0] != 0 && gsl::narrow_cast<byte>(*mbPtr) >= byte{ ' ' })
{
// there was a portion of a dbcs character stored from a previous
// call so we take the 2nd half from mbPtr[0], put them together
// and write the wide char to wcPtr[0]
screenInfo.WriteConsoleDbcsLeadByte[1] = gsl::narrow_cast<byte>(*mbPtr);
try
{
const auto wFromComplemented{
ConvertToW(codepage, { reinterpret_cast<const char*>(screenInfo.WriteConsoleDbcsLeadByte), ARRAYSIZE(screenInfo.WriteConsoleDbcsLeadByte) })
};
FAIL_FAST_IF(wFromComplemented.size() != 1);
dbcsLength = sizeof(wchar_t);
wcPtr[0] = wFromComplemented.at(0);
mbPtr++;
}
catch (...)
{
dbcsLength = 0;
}
// this looks weird to be always incrementing even if the conversion failed, but this is the
// original behavior so it's left unchanged.
wcPtr++;
mbPtrLength--;
// Note that we used a stored lead byte from a previous call in order to complete this write
// Use this to offset the "number of bytes consumed" calculation at the end by -1 to account
// for using a byte we had internally, not off the stream.
leadByteConsumed = true;
}
screenInfo.WriteConsoleDbcsLeadByte[0] = 0;
// if the last byte in mbPtr is a lead byte for the current code page,
// save it for the next time this function is called and we can piece it
// back together then
if (mbPtrLength != 0 && CheckBisectStringA(const_cast<char*>(mbPtr), mbPtrLength, &consoleInfo.OutputCPInfo))
{
screenInfo.WriteConsoleDbcsLeadByte[0] = gsl::narrow_cast<byte>(mbPtr[mbPtrLength - 1]);
mbPtrLength--;
// Note that we captured a lead byte during this call, but won't actually draw it until later.
// Use this to offset the "number of bytes consumed" calculation at the end by +1 to account
// for taking a byte off the stream.
leadByteCaptured = true;
}
if (mbPtrLength != 0)
{
// convert the remaining bytes in mbPtr to wide chars
mbPtrLength = sizeof(wchar_t) * MultiByteToWideChar(codepage, 0, mbPtr, mbPtrLength, wcPtr, mbPtrLength);
}
wstr.resize((dbcsLength + mbPtrLength) / sizeof(wchar_t));
}
// Hold the specific version of the waiter locally so we can tinker with it if we must to store additional context.
std::unique_ptr<WriteData> writeDataWaiter{};
// Make the W version of the call
size_t wcBufferWritten{};
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
const auto hr{ WriteConsoleWImplHelper(screenInfo, wstr, wcBufferWritten, requiresVtQuirk, writeDataWaiter) };
// If there is no waiter, process the byte count now.
if (nullptr == writeDataWaiter.get())
{
// Calculate how many bytes of the original A buffer were consumed in the W version of the call to satisfy mbBufferRead.
// For UTF-8 conversions, we've already returned this information above.
if (CP_UTF8 != codepage)
{
size_t mbBufferRead{};
// Start by counting the number of A bytes we used in printing our W string to the screen.
try
{
mbBufferRead = GetALengthFromW(codepage, { wstr.data(), wcBufferWritten });
}
CATCH_LOG();
// If we captured a byte off the string this time around up above, it means we didn't feed
// it into the WriteConsoleW above, and therefore its consumption isn't accounted for
// in the count we just made. Add +1 to compensate.
if (leadByteCaptured)
{
mbBufferRead++;
}
// If we consumed an internally-stored lead byte this time around up above, it means that we
// fed a byte into WriteConsoleW that wasn't a part of this particular call's request.
// We need to -1 to compensate and tell the caller the right number of bytes consumed this request.
if (leadByteConsumed)
{
mbBufferRead--;
}
read = mbBufferRead;
}
}
else
{
// If there is a waiter, then we need to stow some additional information in the wait structure so
// we can synthesize the correct byte count later when the wait routine is triggered.
if (CP_UTF8 != codepage)
{
// For non-UTF8 codepages, save the lead byte captured/consumed data so we can +1 or -1 the final decoded count
// in the WaitData::Notify method later.
writeDataWaiter->SetLeadByteAdjustmentStatus(leadByteCaptured, leadByteConsumed);
}
else
{
// For UTF8 codepages, just remember the consumption count from the UTF-8 parser.
writeDataWaiter->SetUtf8ConsumedCharacters(read);
}
}
// Give back the waiter now that we're done with tinkering with it.
waiter.reset(writeDataWaiter.release());
return hr;
}
CATCH_RETURN();
}
// Routine Description:
// - Writes Unicode formatted data into the given console output object.
// - NOTE: This may be blocked for various console states and will return a wait context pointer if necessary.
// Arguments:
// - OutContext - the console output object to write the new text into
// - pwsTextBuffer - wide character text buffer provided by client application to insert
// - cchTextBufferLength - text buffer counted in characters
// - pcchTextBufferRead - character count of the number of characters we were able to insert before returning
// - ppWaiter - If we are blocked from writing now and need to wait, this is filled with contextual data for the server to restore the call later
// Return Value:
// - S_OK if successful.
// - S_OK if we need to wait (check if ppWaiter is not nullptr).
// - Or a suitable HRESULT code for math/string/memory failures.
[[nodiscard]] HRESULT ApiRoutines::WriteConsoleWImpl(IConsoleOutputObject& context,
const std::wstring_view buffer,
size_t& read,
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
bool requiresVtQuirk,
std::unique_ptr<IWaitRoutine>& waiter) noexcept
{
try
{
LockConsole();
auto unlock = wil::scope_exit([&] { UnlockConsole(); });
std::unique_ptr<WriteData> writeDataWaiter;
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
RETURN_IF_FAILED(WriteConsoleWImplHelper(context.GetActiveBuffer(), buffer, read, requiresVtQuirk, writeDataWaiter));
// Transfer specific waiter pointer into the generic interface wrapper.
waiter.reset(writeDataWaiter.release());
return S_OK;
}
CATCH_RETURN();
}