terminal/src/host/output.cpp
James Holderness 381b11521a Correct fill attributes when scrolling and erasing (#3100)
## Summary of the Pull Request

Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing.

## PR Checklist
* [x] Closes #2553
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan. 

## Detailed Description of the Pull Request / Additional comments

My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all.

To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset).

With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`.

The functions affected by scrolling included:
* `DoSrvPrivateReverseLineFeed` (the RI command)
* `DoSrvPrivateModifyLinesImpl` (the IL and DL commands)
* `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands)
* `AdaptDispatch::_ScrollMovement` (the SU and SD commands)

The functions affected by erasing included:
* `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants)
* `AdaptDispatch::EraseCharacters` (the ECH command)

While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time.

In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes:

* `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer.
* `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised.
* `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes.

The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above).

Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations.

Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed.

Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs). 

## Validation Steps Performed

For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region.

In the screen buffer tests, I made updates of that sort to  the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer.

The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests.

Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
2019-12-10 23:14:40 +00:00

504 lines
20 KiB
C++

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "_output.h"
#include "output.h"
#include "handle.h"
#include "getset.h"
#include "misc.h"
#include "../interactivity/inc/ServiceLocator.hpp"
#include "../types/inc/Viewport.hpp"
#include "../types/inc/convert.hpp"
#pragma hdrstop
using namespace Microsoft::Console::Types;
using namespace Microsoft::Console::Interactivity;
// This routine figures out what parameters to pass to CreateScreenBuffer based on the data from STARTUPINFO and the
// registry defaults, and then calls CreateScreenBuffer.
[[nodiscard]] NTSTATUS DoCreateScreenBuffer()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
FontInfo fiFont(gci.GetFaceName(),
gsl::narrow_cast<unsigned char>(gci.GetFontFamily()),
gci.GetFontWeight(),
gci.GetFontSize(),
gci.GetCodePage());
// For East Asian version, we want to get the code page from the registry or shell32, so we can specify console
// codepage by console.cpl or shell32. The default codepage is OEMCP.
gci.CP = gci.GetCodePage();
gci.OutputCP = gci.GetCodePage();
gci.Flags |= CONSOLE_USE_PRIVATE_FLAGS;
NTSTATUS Status = SCREEN_INFORMATION::CreateInstance(gci.GetWindowSize(),
fiFont,
gci.GetScreenBufferSize(),
gci.GetDefaultAttributes(),
TextAttribute{ gci.GetPopupFillAttribute() },
gci.GetCursorSize(),
&gci.ScreenBuffers);
// TODO: MSFT 9355013: This needs to be resolved. We increment it once with no handle to ensure it's never cleaned up
// and one always exists for the renderer (and potentially other functions.)
// It's currently a load-bearing piece of code. http://osgvsowi/9355013
if (NT_SUCCESS(Status))
{
gci.ScreenBuffers[0].IncrementOriginalScreenBuffer();
}
return Status;
}
// Routine Description:
// - This routine copies a rectangular region from the screen buffer to the screen buffer.
// Arguments:
// - screenInfo - reference to screen info
// - source - rectangle in source buffer to copy
// - targetOrigin - upper left coordinates of new location rectangle
static void _CopyRectangle(SCREEN_INFORMATION& screenInfo,
const Viewport& source,
const COORD targetOrigin)
{
const auto sourceOrigin = source.Origin();
// 0. If the source and the target are the same... we have nothing to do. Leave.
if (sourceOrigin == targetOrigin)
{
return;
}
// 1. If we're copying entire rows of the buffer and moving them directly up or down,
// then we can send a rotate command to the underlying buffer to just adjust the
// row locations instead of copying or moving anything.
{
const auto bufferSize = screenInfo.GetBufferSize().Dimensions();
const auto sourceFullRows = source.Width() == bufferSize.X;
const auto verticalCopyOnly = source.Left() == 0 && targetOrigin.X == 0;
if (sourceFullRows && verticalCopyOnly)
{
const auto delta = targetOrigin.Y - source.Top();
screenInfo.GetTextBuffer().ScrollRows(source.Top(), source.Height(), gsl::narrow<SHORT>(delta));
return;
}
}
// 2. We can move any other scenario in-place without copying. We just have to carefully
// choose which direction we walk through filling up the target so it doesn't accidentally
// erase the source material before it can be copied/moved to the new location.
{
const auto target = Viewport::FromDimensions(targetOrigin, source.Dimensions());
const auto walkDirection = Viewport::DetermineWalkDirection(source, target);
auto sourcePos = source.GetWalkOrigin(walkDirection);
auto targetPos = target.GetWalkOrigin(walkDirection);
do
{
const auto data = OutputCell(*screenInfo.GetCellDataAt(sourcePos));
screenInfo.Write(OutputCellIterator({ &data, 1 }), targetPos);
source.WalkInBounds(sourcePos, walkDirection);
} while (target.WalkInBounds(targetPos, walkDirection));
}
}
// Routine Description:
// - This routine reads a sequence of attributes from the screen buffer.
// Arguments:
// - screenInfo - reference to screen buffer information.
// - coordRead - Screen buffer coordinate to begin reading from.
// - amountToRead - the number of elements to read
// Return Value:
// - vector of attribute data
std::vector<WORD> ReadOutputAttributes(const SCREEN_INFORMATION& screenInfo,
const COORD coordRead,
const size_t amountToRead)
{
// Short circuit. If nothing to read, leave early.
if (amountToRead == 0)
{
return {};
}
// Short circuit, if reading out of bounds, leave early.
if (!screenInfo.GetBufferSize().IsInBounds(coordRead))
{
return {};
}
// Get iterator to the position we should start reading at.
auto it = screenInfo.GetCellDataAt(coordRead);
// Count up the number of cells we've attempted to read.
ULONG amountRead = 0;
// Prepare the return value string.
std::vector<WORD> retVal;
// Reserve the number of cells. If we have >U+FFFF, it will auto-grow later and that's OK.
retVal.reserve(amountToRead);
// While we haven't read enough cells yet and the iterator is still valid (hasn't reached end of buffer)
while (amountRead < amountToRead && it)
{
// If the first thing we read is trailing, pad with a space.
// OR If the last thing we read is leading, pad with a space.
if ((amountRead == 0 && it->DbcsAttr().IsTrailing()) ||
(amountRead == (amountToRead - 1) && it->DbcsAttr().IsLeading()))
{
retVal.push_back(it->TextAttr().GetLegacyAttributes());
}
else
{
retVal.push_back(it->TextAttr().GetLegacyAttributes() | it->DbcsAttr().GeneratePublicApiAttributeFormat());
}
amountRead++;
it++;
}
return retVal;
}
// Routine Description:
// - This routine reads a sequence of unicode characters from the screen buffer
// Arguments:
// - screenInfo - reference to screen buffer information.
// - coordRead - Screen buffer coordinate to begin reading from.
// - amountToRead - the number of elements to read
// Return Value:
// - wstring
std::wstring ReadOutputStringW(const SCREEN_INFORMATION& screenInfo,
const COORD coordRead,
const size_t amountToRead)
{
// Short circuit. If nothing to read, leave early.
if (amountToRead == 0)
{
return {};
}
// Short circuit, if reading out of bounds, leave early.
if (!screenInfo.GetBufferSize().IsInBounds(coordRead))
{
return {};
}
// Get iterator to the position we should start reading at.
auto it = screenInfo.GetCellDataAt(coordRead);
// Count up the number of cells we've attempted to read.
ULONG amountRead = 0;
// Prepare the return value string.
std::wstring retVal;
retVal.reserve(amountToRead); // Reserve the number of cells. If we have >U+FFFF, it will auto-grow later and that's OK.
// While we haven't read enough cells yet and the iterator is still valid (hasn't reached end of buffer)
while (amountRead < amountToRead && it)
{
// If the first thing we read is trailing, pad with a space.
// OR If the last thing we read is leading, pad with a space.
if ((amountRead == 0 && it->DbcsAttr().IsTrailing()) ||
(amountRead == (amountToRead - 1) && it->DbcsAttr().IsLeading()))
{
retVal += UNICODE_SPACE;
}
else
{
// Otherwise, add anything that isn't a trailing cell. (Trailings are duplicate copies of the leading.)
if (!it->DbcsAttr().IsTrailing())
{
retVal += it->Chars();
}
}
amountRead++;
it++;
}
return retVal;
}
// Routine Description:
// - This routine reads a sequence of ascii characters from the screen buffer
// Arguments:
// - screenInfo - reference to screen buffer information.
// - coordRead - Screen buffer coordinate to begin reading from.
// - amountToRead - the number of elements to read
// Return Value:
// - string of char data
std::string ReadOutputStringA(const SCREEN_INFORMATION& screenInfo,
const COORD coordRead,
const size_t amountToRead)
{
const auto wstr = ReadOutputStringW(screenInfo, coordRead, amountToRead);
const auto& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
return ConvertToA(gci.OutputCP, wstr);
}
void ScreenBufferSizeChange(const COORD coordNewSize)
{
const CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
try
{
gci.pInputBuffer->Write(std::make_unique<WindowBufferSizeEvent>(coordNewSize));
}
catch (...)
{
LOG_HR(wil::ResultFromCaughtException());
}
}
// Routine Description:
// - This is simply a notifier method to let accessibility and renderers know that a region of the buffer
// has been copied/moved to another location in a block fashion.
// Arguments:
// - screenInfo - The relevant screen buffer where data was moved
// - source - The viewport describing the region where data was copied from
// - fill - The viewport describing the area that was filled in with the fill character (uncovered area)
// - target - The viewport describing the region where data was copied to
static void _ScrollScreen(SCREEN_INFORMATION& screenInfo, const Viewport& source, const Viewport& fill, const Viewport& target)
{
if (screenInfo.IsActiveScreenBuffer())
{
IAccessibilityNotifier* pNotifier = ServiceLocator::LocateAccessibilityNotifier();
if (pNotifier != nullptr)
{
pNotifier->NotifyConsoleUpdateScrollEvent(target.Origin().X - source.Left(), target.Origin().Y - source.RightInclusive());
}
}
// Get the render target and send it commands.
// It will figure out whether or not we're active and where the messages need to go.
auto& render = screenInfo.GetRenderTarget();
// Redraw anything in the target area
render.TriggerRedraw(target);
// Also redraw anything that was filled.
render.TriggerRedraw(fill);
}
// Routine Description:
// - This routine is a special-purpose scroll for use by AdjustCursorPosition.
// Arguments:
// - screenInfo - reference to screen buffer info.
// Return Value:
// - true if we succeeded in scrolling the buffer, otherwise false (if we're out of memory)
bool StreamScrollRegion(SCREEN_INFORMATION& screenInfo)
{
// Rotate the circular buffer around and wipe out the previous final line.
const bool inVtMode = WI_IsFlagSet(screenInfo.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
bool fSuccess = screenInfo.GetTextBuffer().IncrementCircularBuffer(inVtMode);
if (fSuccess)
{
// Trigger a graphical update if we're active.
if (screenInfo.IsActiveScreenBuffer())
{
COORD coordDelta = { 0 };
coordDelta.Y = -1;
IAccessibilityNotifier* pNotifier = ServiceLocator::LocateAccessibilityNotifier();
if (pNotifier)
{
// Notify accessibility that a scroll has occurred.
pNotifier->NotifyConsoleUpdateScrollEvent(coordDelta.X, coordDelta.Y);
}
if (ServiceLocator::LocateGlobals().pRender != nullptr)
{
ServiceLocator::LocateGlobals().pRender->TriggerScroll(&coordDelta);
}
}
}
return fSuccess;
}
// Routine Description:
// - This routine copies ScrollRectangle to DestinationOrigin then fills in ScrollRectangle with Fill.
// - The scroll region is copied to a third buffer, the scroll region is filled, then the original contents of the scroll region are copied to the destination.
// Arguments:
// - screenInfo - reference to screen buffer info.
// - scrollRectGiven - Region to copy/move (source and size)
// - clipRectGiven - Optional clip region to contain buffer change effects
// - destinationOriginGiven - Upper left corner of target region.
// - fillCharGiven - Character to fill source region with.
// - fillAttrsGiven - Attribute to fill source region with.
// NOTE: Throws exceptions
void ScrollRegion(SCREEN_INFORMATION& screenInfo,
const SMALL_RECT scrollRectGiven,
const std::optional<SMALL_RECT> clipRectGiven,
const COORD destinationOriginGiven,
const wchar_t fillCharGiven,
const TextAttribute fillAttrsGiven)
{
// ------ 1. PREP SOURCE ------
// Set up the source viewport.
auto source = Viewport::FromInclusive(scrollRectGiven);
const auto originalSourceOrigin = source.Origin();
// Alright, let's make sure that our source fits inside the buffer.
const auto buffer = screenInfo.GetBufferSize();
source = Viewport::Intersect(source, buffer);
// If the source is no longer valid, then there's nowhere we can copy from
// and also nowhere we can fill. We're done. Return early.
if (!source.IsValid())
{
return;
}
// ------ 2. PREP CLIP ------
// Now figure out our clipping area. If we have clipping specified, it will limit
// the area that can be affected (targeted or filling) throughout this operation.
// If there was no clip rect, we'll clip to the entire buffer size.
auto clip = Viewport::FromInclusive(clipRectGiven.value_or(buffer.ToInclusive()));
// OK, make sure that the clip rectangle also fits inside the buffer
clip = Viewport::Intersect(buffer, clip);
// ------ 3. PREP FILL ------
// Then think about fill. We will fill in any area of the source that we copied from
// with the fill character as long as it falls inside the clip region (the area
// that is allowed to be affected).
auto fill = Viewport::Intersect(clip, source);
// If fill is no longer valid, then there is no area that we're allowed to write to
// within the buffer. So we can just exit early.
if (!fill.IsValid())
{
return;
}
// Determine the cell we will use to fill in any revealed/uncovered space.
// We generally use exactly what was given to us.
OutputCellIterator fillData(fillCharGiven, fillAttrsGiven);
// However, if the character is null and we were given a null attribute (represented as legacy 0),
// then we'll just fill with spaces and whatever the buffer's default colors are.
if (fillCharGiven == UNICODE_NULL && fillAttrsGiven.IsLegacy() && fillAttrsGiven.GetLegacyAttributes() == 0)
{
fillData = OutputCellIterator(UNICODE_SPACE, screenInfo.GetAttributes());
}
// ------ 4. PREP TARGET ------
// Now it's time to think about the target. We're only given the origin of the target
// because it is assumed that it will have the same relative dimensions as the original source.
auto targetOrigin = destinationOriginGiven;
// However, if we got to this point, we may have clipped the source because some part of it
// fell outside of the buffer.
// Apply any delta between the original source rectangle's origin and its current position to
// the target origin.
{
auto currentSourceOrigin = source.Origin();
targetOrigin.X += currentSourceOrigin.X - originalSourceOrigin.X;
targetOrigin.Y += currentSourceOrigin.Y - originalSourceOrigin.Y;
}
// And now the target viewport is the same size as the source viewport but at the different position.
auto target = Viewport::FromDimensions(targetOrigin, source.Dimensions());
// However, this might mean that the target is falling outside of the region we're allowed to edit
// (the clip area). So we need to reduce the target to only inside the clip.
// But backup the original target origin first, because we need to know how it has changed.
const auto originalTargetOrigin = target.Origin();
target = Viewport::Intersect(clip, target);
// OK, if the target became smaller than before, we need to also adjust the source accordingly
// so we don't waste time loading up/copying things that have no place to go within the target.
{
const auto currentTargetOrigin = target.Origin();
auto sourceOrigin = source.Origin();
sourceOrigin.X += currentTargetOrigin.X - originalTargetOrigin.X;
sourceOrigin.Y += currentTargetOrigin.Y - originalTargetOrigin.Y;
source = Viewport::FromDimensions(sourceOrigin, target.Dimensions());
}
// ------ 5. COPY ------
// If the target region is valid, let's do this.
if (target.IsValid())
{
// Perform the copy from the source to the target.
_CopyRectangle(screenInfo, source, target.Origin());
// Notify the renderer and accessibility as to what moved and where.
_ScrollScreen(screenInfo, source, fill, target);
}
// ------ 6. FILL ------
// Now fill in anything that wasn't already touched by the copy above.
// Fill as a single viewport represents the entire region we were allowed to
// write into. But since we already copied, filling the whole thing might
// overwrite what we just placed at the target.
// So use the special subtraction function to get the viewports that fall
// within the fill area but outside of the target area.
const auto remaining = Viewport::Subtract(fill, target);
// Apply the fill data to each of the viewports we're given here.
for (size_t i = 0; i < remaining.size(); i++)
{
const auto& view = remaining.at(i);
screenInfo.WriteRect(fillData, view);
}
}
void SetActiveScreenBuffer(SCREEN_INFORMATION& screenInfo)
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
gci.pCurrentScreenBuffer = &screenInfo;
// initialize cursor
screenInfo.GetTextBuffer().GetCursor().SetIsOn(false);
// set font
screenInfo.RefreshFontWithRenderer();
// Empty input buffer.
gci.pInputBuffer->FlushAllButKeys();
// Set window size.
screenInfo.PostUpdateWindowSize();
gci.ConsoleIme.RefreshAreaAttributes();
// Write data to screen.
WriteToScreen(screenInfo, screenInfo.GetViewport());
}
// TODO: MSFT 9450717 This should join the ProcessList class when CtrlEvents become moved into the server. https://osgvsowi/9450717
void CloseConsoleProcessState()
{
const CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
// If there are no connected processes, sending control events is pointless as there's no one do send them to. In
// this case we'll just exit conhost.
// N.B. We can get into this state when a process has a reference to the console but hasn't connected. For example,
// when it's created suspended and never resumed.
if (gci.ProcessHandleList.IsEmpty())
{
ServiceLocator::RundownAndExit(STATUS_SUCCESS);
}
HandleCtrlEvent(CTRL_CLOSE_EVENT);
// Jiggle the handle: (see MSFT:19419231)
// When we call this function, we'll only actually close the console once
// we're totally unlocked. If our caller has the console locked, great,
// we'll displatch the ctrl event once they unlock. However, if they're
// not running under lock (eg PtySignalInputThread::_GetData), then the
// ctrl event will never actually get dispatched.
// So, lock and unlock here, to make sure the ctrl event gets handled.
LockConsole();
auto Unlock = wil::scope_exit([&] { UnlockConsole(); });
}