terminal/src/host/ut_host/TextBufferTests.cpp

2656 lines
96 KiB
C++
Raw Normal View History

Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "WexTestClass.h"
#include "../inc/consoletaeftemplates.hpp"
#include "CommonState.hpp"
#include "globals.h"
#include "../buffer/out/textBuffer.hpp"
#include "../buffer/out/CharRow.hpp"
#include "input.h"
#include "_stream.h"
#include "../interactivity/inc/ServiceLocator.hpp"
#include "../renderer/inc/DummyRenderTarget.hpp"
using namespace Microsoft::Console::Types;
using namespace Microsoft::Console::Interactivity;
using namespace Microsoft::Console::VirtualTerminal;
using namespace WEX::Common;
using namespace WEX::Logging;
using namespace WEX::TestExecution;
class TextBufferTests
{
DummyRenderTarget _renderTarget;
CommonState* m_state;
TEST_CLASS(TextBufferTests);
TEST_CLASS_SETUP(ClassSetup)
{
m_state = new CommonState();
m_state->PrepareGlobalFont();
m_state->PrepareGlobalScreenBuffer();
return true;
}
TEST_CLASS_CLEANUP(ClassCleanup)
{
m_state->CleanupGlobalScreenBuffer();
m_state->CleanupGlobalFont();
delete m_state;
return true;
}
TEST_METHOD_SETUP(MethodSetup)
{
m_state->PrepareNewTextBufferInfo();
return true;
}
TEST_METHOD_CLEANUP(MethodCleanup)
{
m_state->CleanupNewTextBufferInfo();
return true;
}
TEST_METHOD(TestBufferCreate);
TextBuffer& GetTbi();
SHORT GetBufferWidth();
SHORT GetBufferHeight();
TEST_METHOD(TestBufferRowByOffset);
TEST_METHOD(TestWrapFlag);
TEST_METHOD(TestWrapThroughWriteLine);
TEST_METHOD(TestDoubleBytePadFlag);
void DoBoundaryTest(PWCHAR const pwszInputString,
short const cLength,
short const cMax,
short const cLeft,
short const cRight);
TEST_METHOD(TestBoundaryMeasuresRegularString);
TEST_METHOD(TestBoundaryMeasuresFloatingString);
TEST_METHOD(TestCopyProperties);
TEST_METHOD(TestInsertCharacter);
TEST_METHOD(TestIncrementCursor);
TEST_METHOD(TestNewlineCursor);
void TestLastNonSpace(short const cursorPosY);
TEST_METHOD(TestGetLastNonSpaceCharacter);
TEST_METHOD(TestSetWrapOnCurrentRow);
TEST_METHOD(TestIncrementCircularBuffer);
TEST_METHOD(TestMixedRgbAndLegacyForeground);
TEST_METHOD(TestMixedRgbAndLegacyBackground);
TEST_METHOD(TestMixedRgbAndLegacyUnderline);
TEST_METHOD(TestMixedRgbAndLegacyBrightness);
TEST_METHOD(TestRgbEraseLine);
TEST_METHOD(TestUnBold);
TEST_METHOD(TestUnBoldRgb);
TEST_METHOD(TestComplexUnBold);
TEST_METHOD(CopyAttrs);
TEST_METHOD(EmptySgrTest);
TEST_METHOD(TestReverseReset);
TEST_METHOD(CopyLastAttr);
TEST_METHOD(TestRgbThenBold);
TEST_METHOD(TestResetClearsBoldness);
TEST_METHOD(TestBackspaceRightSideVt);
TEST_METHOD(TestBackspaceStrings);
TEST_METHOD(TestBackspaceStringsAPI);
TEST_METHOD(TestRepeatCharacter);
TEST_METHOD(ResizeTraditional);
TEST_METHOD(ResizeTraditionalRotationPreservesHighUnicode);
TEST_METHOD(ScrollBufferRotationPreservesHighUnicode);
TEST_METHOD(ResizeTraditionalHighUnicodeRowRemoval);
TEST_METHOD(ResizeTraditionalHighUnicodeColumnRemoval);
TEST_METHOD(TestBurrito);
Refactor UiaTextRange For Improved Navigation and Reliability (#4018) ## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
2020-01-31 21:59:39 +01:00
void WriteLinesToBuffer(const std::vector<std::wstring>& text, TextBuffer& buffer);
TEST_METHOD(GetWordBoundaries);
TEST_METHOD(MoveByWord);
TEST_METHOD(GetGlyphBoundaries);
Move rect expansion to textbuffer; refactor selection code (#4560) - When performing chunk selection, the expansion now occurs at the time of the selection, not the rendering of the selection - `GetSelectionRects()` was moved to the `TextBuffer` and is now shared between ConHost and Windows Terminal - Some of the selection variables were renamed for clarity - Selection COORDs are now in the Text Buffer coordinate space - Fixes an issue with Shift+Click after performing a Multi-Click Selection ## References This also contributes to... - #4509: UIA Box Selection - #2447: UIA Signaling for Selection - #1354: UIA support for Wide Glyphs Now that the expansion occurs at before render-time, the selection anchors are an accurate representation of what is selected. We just need to move `GetText` to the `TextBuffer`. Then we can have those three issues just rely on code from the text buffer. This also means ConHost gets some of this stuff for free 😀 ### TextBuffer - `GetTextRects` is the abstracted form of `GetSelectionRects` - `_ExpandTextRow` is still needed to handle wide glyphs properly ### Terminal - Rename... - `_boxSelection` --> `_blockSelection` for consistency with ConHost - `_selectionAnchor` --> `_selectionStart` for consistency with UIA - `_endSelectionPosition` --> `_selectionEnd` for consistency with UIA - Selection anchors are in Text Buffer coordinates now - Really rely on `SetSelectionEnd` to accomplish appropriate chunk selection and shift+click actions ## Validation Steps Performed - Shift+Click - Multi-Click --> Shift+Click - Chunk Selection at... - top of buffer - bottom of buffer - random region in scrollback Closes #4465 Closes #4547
2020-02-28 01:42:26 +01:00
TEST_METHOD(GetTextRects);
TEST_METHOD(GetText);
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
TEST_METHOD(HyperlinkTrim);
TEST_METHOD(NoHyperlinkTrim);
};
void TextBufferTests::TestBufferCreate()
{
Create tests that roundtrip output through a conpty to a Terminal (#4213) ## Summary of the Pull Request This PR adds two tests: * First, I started by writing a test where I could write output to the console host and inspect what output came out of conpty. This is the `ConptyOutputTests` in the host unit tests. * Then I got crazy and thought _"what if I could take that output and dump it straight into the `Terminal`"_? Hence, the `ConptyRoundtripTests` were born, into the TerminalCore unit tests. ## References Done in pursuit of #4200, but I felt this warranted it's own atomic PR ## PR Checklist * [x] Doesn't close anything on it's own. * [x] I work here * [x] you better believe this adds tests * [n/a] Requires documentation to be updated ## Detailed Description of the Pull Request / Additional comments From the comment in `ConptyRoundtripTests`: > This test class creates an in-proc conpty host as well as a Terminal, to > validate that strings written to the conpty create the same resopnse on the > terminal end. Tests can be written that validate both the contents of the > host buffer as well as the terminal buffer. Everytime that > `renderer.PaintFrame()` is called, the tests will validate the expected > output, and then flush the output of the VtEngine straight to th Also, some other bits had to be updated: * The renderer needed to be able to survive without a thread, so I hadded a simple check that it actually had a thread before calling `pThread->NotifyPaint` * Bits in `CommonState` used `NTSTATUS_FROM_HRESULT` which did _not_ work outside the host project. Since the `NTSTATUS` didn't seem that important, I replaced that with a `HRESULT` * `CommonState` likes to initialize the console to some _weird_ defaults. I added an optional param to let us just use the defaults.
2020-01-17 17:40:12 +01:00
VERIFY_SUCCEEDED(m_state->GetTextBufferInfoInitResult());
}
TextBuffer& TextBufferTests::GetTbi()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
return gci.GetActiveOutputBuffer().GetTextBuffer();
}
SHORT TextBufferTests::GetBufferWidth()
{
return GetTbi().GetSize().Width();
}
SHORT TextBufferTests::GetBufferHeight()
{
return GetTbi().GetSize().Height();
}
void TextBufferTests::TestBufferRowByOffset()
{
TextBuffer& textBuffer = GetTbi();
SHORT csBufferHeight = GetBufferHeight();
VERIFY_IS_TRUE(csBufferHeight > 20);
short sId = csBufferHeight / 2 - 5;
const ROW& row = textBuffer.GetRowByOffset(sId);
VERIFY_ARE_EQUAL(row.GetId(), sId);
}
void TextBufferTests::TestWrapFlag()
{
TextBuffer& textBuffer = GetTbi();
ROW& Row = textBuffer._GetFirstRow();
// no wrap by default
VERIFY_IS_FALSE(Row.WasWrapForced());
// try set wrap and check
Row.SetWrapForced(true);
VERIFY_IS_TRUE(Row.WasWrapForced());
// try unset wrap and check
Row.SetWrapForced(false);
VERIFY_IS_FALSE(Row.WasWrapForced());
}
void TextBufferTests::TestWrapThroughWriteLine()
{
TextBuffer& textBuffer = GetTbi();
auto VerifyWrap = [&](bool expected) {
ROW& Row = textBuffer._GetFirstRow();
if (expected)
{
VERIFY_IS_TRUE(Row.WasWrapForced());
}
else
{
VERIFY_IS_FALSE(Row.WasWrapForced());
}
};
// Construct string for testing
const auto width = textBuffer.GetSize().Width();
std::wstring chars = L"";
for (auto i = 0; i < width; i++)
{
chars.append(L"a");
}
const auto lineOfText = std::move(chars);
Log::Comment(L"Case 1 : Implicit wrap (false)");
{
TextAttribute expectedAttr(FOREGROUND_RED);
OutputCellIterator it(lineOfText, expectedAttr);
textBuffer.WriteLine(it, { 0, 0 });
VerifyWrap(false);
}
Log::Comment(L"Case 2 : wrap = true");
{
TextAttribute expectedAttr(FOREGROUND_RED);
OutputCellIterator it(lineOfText, expectedAttr);
textBuffer.WriteLine(it, { 0, 0 }, true);
VerifyWrap(true);
}
Log::Comment(L"Case 3: wrap = nullopt (remain as TRUE)");
{
TextAttribute expectedAttr(FOREGROUND_RED);
OutputCellIterator it(lineOfText, expectedAttr);
textBuffer.WriteLine(it, { 0, 0 }, std::nullopt);
VerifyWrap(true);
}
Log::Comment(L"Case 4: wrap = false");
{
TextAttribute expectedAttr(FOREGROUND_RED);
OutputCellIterator it(lineOfText, expectedAttr);
textBuffer.WriteLine(it, { 0, 0 }, false);
VerifyWrap(false);
}
Log::Comment(L"Case 5: wrap = nullopt (remain as false)");
{
TextAttribute expectedAttr(FOREGROUND_RED);
OutputCellIterator it(lineOfText, expectedAttr);
textBuffer.WriteLine(it, { 0, 0 }, std::nullopt);
VerifyWrap(false);
}
}
void TextBufferTests::TestDoubleBytePadFlag()
{
TextBuffer& textBuffer = GetTbi();
ROW& Row = textBuffer._GetFirstRow();
// no padding by default
VERIFY_IS_FALSE(Row.WasDoubleBytePadded());
// try set and check
Row.SetDoubleBytePadded(true);
VERIFY_IS_TRUE(Row.WasDoubleBytePadded());
// try unset and check
Row.SetDoubleBytePadded(false);
VERIFY_IS_FALSE(Row.WasDoubleBytePadded());
}
void TextBufferTests::DoBoundaryTest(PWCHAR const pwszInputString,
short const cLength,
short const cMax,
short const cLeft,
short const cRight)
{
TextBuffer& textBuffer = GetTbi();
CharRow& charRow = textBuffer._GetFirstRow().GetCharRow();
// copy string into buffer
for (size_t i = 0; i < static_cast<size_t>(cLength); ++i)
{
charRow.GlyphAt(i) = { &pwszInputString[i], 1 };
}
// space pad the rest of the string
if (cLength < cMax)
{
for (short cStart = cLength; cStart < cMax; cStart++)
{
charRow.ClearGlyph(cStart);
}
}
// left edge should be 0 since there are no leading spaces
VERIFY_ARE_EQUAL(charRow.MeasureLeft(), static_cast<size_t>(cLeft));
// right edge should be one past the index of the last character or the string length
VERIFY_ARE_EQUAL(charRow.MeasureRight(), static_cast<size_t>(cRight));
}
void TextBufferTests::TestBoundaryMeasuresRegularString()
{
SHORT csBufferWidth = GetBufferWidth();
// length 44, left 0, right 44
const PWCHAR pwszLazyDog = L"The quick brown fox jumps over the lazy dog.";
DoBoundaryTest(pwszLazyDog, 44, csBufferWidth, 0, 44);
}
void TextBufferTests::TestBoundaryMeasuresFloatingString()
{
SHORT csBufferWidth = GetBufferWidth();
// length 5 spaces + 4 chars + 5 spaces = 14, left 5, right 9
const PWCHAR pwszOffsets = L" C:\\> ";
DoBoundaryTest(pwszOffsets, 14, csBufferWidth, 5, 9);
}
void TextBufferTests::TestCopyProperties()
{
TextBuffer& otherTbi = GetTbi();
std::unique_ptr<TextBuffer> testTextBuffer = std::make_unique<TextBuffer>(otherTbi.GetSize().Dimensions(),
otherTbi._currentAttributes,
12,
otherTbi._renderTarget);
VERIFY_IS_NOT_NULL(testTextBuffer.get());
// set initial mapping values
testTextBuffer->GetCursor().SetHasMoved(false);
otherTbi.GetCursor().SetHasMoved(true);
testTextBuffer->GetCursor().SetIsVisible(false);
otherTbi.GetCursor().SetIsVisible(true);
testTextBuffer->GetCursor().SetIsOn(false);
otherTbi.GetCursor().SetIsOn(true);
testTextBuffer->GetCursor().SetIsDouble(false);
otherTbi.GetCursor().SetIsDouble(true);
testTextBuffer->GetCursor().SetDelay(false);
otherTbi.GetCursor().SetDelay(true);
// run copy
testTextBuffer->CopyProperties(otherTbi);
// test that new now contains values from other
VERIFY_IS_TRUE(testTextBuffer->GetCursor().HasMoved());
VERIFY_IS_TRUE(testTextBuffer->GetCursor().IsVisible());
VERIFY_IS_TRUE(testTextBuffer->GetCursor().IsOn());
VERIFY_IS_TRUE(testTextBuffer->GetCursor().IsDouble());
VERIFY_IS_TRUE(testTextBuffer->GetCursor().GetDelay());
}
void TextBufferTests::TestInsertCharacter()
{
TextBuffer& textBuffer = GetTbi();
// get starting cursor position
COORD const coordCursorBefore = textBuffer.GetCursor().GetPosition();
// Get current row from the buffer
ROW& Row = textBuffer.GetRowByOffset(coordCursorBefore.Y);
// create some sample test data
const auto wch = L'Z';
const std::wstring_view wchTest(&wch, 1);
DbcsAttribute dbcsAttribute;
dbcsAttribute.SetTrailing();
WORD const wAttrTest = BACKGROUND_INTENSITY | FOREGROUND_INTENSITY | FOREGROUND_RED | FOREGROUND_BLUE;
TextAttribute TestAttributes = TextAttribute(wAttrTest);
CharRow& charRow = Row.GetCharRow();
charRow.DbcsAttrAt(coordCursorBefore.X).SetLeading();
// ensure that the buffer didn't start with these fields
VERIFY_ARE_NOT_EQUAL(charRow.GlyphAt(coordCursorBefore.X), wchTest);
VERIFY_ARE_NOT_EQUAL(charRow.DbcsAttrAt(coordCursorBefore.X), dbcsAttribute);
auto attr = Row.GetAttrRow().GetAttrByColumn(coordCursorBefore.X);
VERIFY_ARE_NOT_EQUAL(attr, TestAttributes);
// now apply the new data to the buffer
textBuffer.InsertCharacter(wchTest, dbcsAttribute, TestAttributes);
// ensure that the buffer position where the cursor WAS contains the test items
VERIFY_ARE_EQUAL(charRow.GlyphAt(coordCursorBefore.X), wchTest);
VERIFY_ARE_EQUAL(charRow.DbcsAttrAt(coordCursorBefore.X), dbcsAttribute);
attr = Row.GetAttrRow().GetAttrByColumn(coordCursorBefore.X);
VERIFY_ARE_EQUAL(attr, TestAttributes);
// ensure that the cursor moved to a new position (X or Y or both have changed)
VERIFY_IS_TRUE((coordCursorBefore.X != textBuffer.GetCursor().GetPosition().X) ||
(coordCursorBefore.Y != textBuffer.GetCursor().GetPosition().Y));
// the proper advancement of the cursor (e.g. which position it goes to) is validated in other tests
}
void TextBufferTests::TestIncrementCursor()
{
TextBuffer& textBuffer = GetTbi();
// only checking X increments here
// Y increments are covered in the NewlineCursor test
short const sBufferWidth = textBuffer.GetSize().Width();
short const sBufferHeight = textBuffer.GetSize().Height();
VERIFY_IS_TRUE(sBufferWidth > 1 && sBufferHeight > 1);
Log::Comment(L"Test normal case of moving once to the right within a single line");
textBuffer.GetCursor().SetXPosition(0);
textBuffer.GetCursor().SetYPosition(0);
COORD coordCursorBefore = textBuffer.GetCursor().GetPosition();
textBuffer.IncrementCursor();
VERIFY_ARE_EQUAL(textBuffer.GetCursor().GetPosition().X, 1); // X should advance by 1
VERIFY_ARE_EQUAL(textBuffer.GetCursor().GetPosition().Y, coordCursorBefore.Y); // Y shouldn't have moved
Log::Comment(L"Test line wrap case where cursor is on the right edge of the line");
textBuffer.GetCursor().SetXPosition(sBufferWidth - 1);
textBuffer.GetCursor().SetYPosition(0);
coordCursorBefore = textBuffer.GetCursor().GetPosition();
textBuffer.IncrementCursor();
VERIFY_ARE_EQUAL(textBuffer.GetCursor().GetPosition().X, 0); // position should be reset to the left edge when passing right edge
VERIFY_ARE_EQUAL(textBuffer.GetCursor().GetPosition().Y - 1, coordCursorBefore.Y); // the cursor should be moved one row down from where it used to be
}
void TextBufferTests::TestNewlineCursor()
{
TextBuffer& textBuffer = GetTbi();
const short sBufferHeight = textBuffer.GetSize().Height();
const short sBufferWidth = textBuffer.GetSize().Width();
// width and height are sufficiently large for upcoming math
VERIFY_IS_TRUE(sBufferWidth > 4 && sBufferHeight > 4);
Log::Comment(L"Verify standard row increment from somewhere in the buffer");
// set cursor X position to non zero, any position in buffer
textBuffer.GetCursor().SetXPosition(3);
// set cursor Y position to not-the-final row in the buffer
textBuffer.GetCursor().SetYPosition(3);
COORD coordCursorBefore = textBuffer.GetCursor().GetPosition();
// perform operation
textBuffer.NewlineCursor();
// verify
VERIFY_ARE_EQUAL(textBuffer.GetCursor().GetPosition().X, 0); // move to left edge of buffer
VERIFY_ARE_EQUAL(textBuffer.GetCursor().GetPosition().Y, coordCursorBefore.Y + 1); // move down one row
Log::Comment(L"Verify increment when already on last row of buffer");
// X position still doesn't matter
textBuffer.GetCursor().SetXPosition(3);
// Y position needs to be on the last row of the buffer
textBuffer.GetCursor().SetYPosition(sBufferHeight - 1);
coordCursorBefore = textBuffer.GetCursor().GetPosition();
// perform operation
textBuffer.NewlineCursor();
// verify
VERIFY_ARE_EQUAL(textBuffer.GetCursor().GetPosition().X, 0); // move to left edge
VERIFY_ARE_EQUAL(textBuffer.GetCursor().GetPosition().Y, coordCursorBefore.Y); // cursor Y position should not have moved. stays on same logical final line of buffer
// This is okay because the backing circular buffer changes, not the logical screen position (final visible line of the buffer)
}
void TextBufferTests::TestLastNonSpace(short const cursorPosY)
{
TextBuffer& textBuffer = GetTbi();
textBuffer.GetCursor().SetYPosition(cursorPosY);
COORD coordLastNonSpace = textBuffer.GetLastNonSpaceCharacter();
// We expect the last non space character to be the last printable character in the row.
// The .Right property on a row is 1 past the last printable character in the row.
// If there is one character in the row, the last character would be 0.
// If there are no characters in the row, the last character would be -1 and we need to seek backwards to find the previous row with a character.
// start expected position from cursor
COORD coordExpected = textBuffer.GetCursor().GetPosition();
// Try to get the X position from the current cursor position.
coordExpected.X = static_cast<short>(textBuffer.GetRowByOffset(coordExpected.Y).GetCharRow().MeasureRight()) - 1;
// If we went negative, this row was empty and we need to continue seeking upward...
// - As long as X is negative (empty rows)
// - As long as we have space before the top of the buffer (Y isn't the 0th/top row).
while (coordExpected.X < 0 && coordExpected.Y > 0)
{
coordExpected.Y--;
coordExpected.X = static_cast<short>(textBuffer.GetRowByOffset(coordExpected.Y).GetCharRow().MeasureRight()) - 1;
}
VERIFY_ARE_EQUAL(coordLastNonSpace.X, coordExpected.X);
VERIFY_ARE_EQUAL(coordLastNonSpace.Y, coordExpected.Y);
}
void TextBufferTests::TestGetLastNonSpaceCharacter()
{
m_state->FillTextBuffer(); // fill buffer with some text, it should be 4 rows. See CommonState for details
Log::Comment(L"Test with cursor inside last row of text");
TestLastNonSpace(3);
Log::Comment(L"Test with cursor one beyond last row of text");
TestLastNonSpace(4);
Log::Comment(L"Test with cursor way beyond last row of text");
TestLastNonSpace(14);
}
void TextBufferTests::TestSetWrapOnCurrentRow()
{
TextBuffer& textBuffer = GetTbi();
short sCurrentRow = textBuffer.GetCursor().GetPosition().Y;
ROW& Row = textBuffer.GetRowByOffset(sCurrentRow);
Log::Comment(L"Testing off to on");
// turn wrap status off first
Row.SetWrapForced(false);
// trigger wrap
textBuffer._SetWrapOnCurrentRow();
// ensure this row was flipped
VERIFY_IS_TRUE(Row.WasWrapForced());
Log::Comment(L"Testing on stays on");
// make sure wrap status is on
Row.SetWrapForced(true);
// trigger wrap
textBuffer._SetWrapOnCurrentRow();
// ensure row is still on
VERIFY_IS_TRUE(Row.WasWrapForced());
}
void TextBufferTests::TestIncrementCircularBuffer()
{
TextBuffer& textBuffer = GetTbi();
short const sBufferHeight = textBuffer.GetSize().Height();
VERIFY_IS_TRUE(sBufferHeight > 4); // buffer should be sufficiently large
Log::Comment(L"Test 1 = FirstRow of circular buffer is not the final row of the buffer");
Log::Comment(L"Test 2 = FirstRow of circular buffer IS THE FINAL ROW of the buffer (and therefore circles)");
short rgRowsToTest[] = { 2, sBufferHeight - 1 };
for (UINT iTestIndex = 0; iTestIndex < ARRAYSIZE(rgRowsToTest); iTestIndex++)
{
const short iRowToTestIndex = rgRowsToTest[iTestIndex];
short iNextRowIndex = iRowToTestIndex + 1;
// if we're at or crossing the height, loop back to 0 (circular buffer)
if (iNextRowIndex >= sBufferHeight)
{
iNextRowIndex = 0;
}
textBuffer._firstRow = iRowToTestIndex;
// fill first row with some stuff
ROW& FirstRow = textBuffer._GetFirstRow();
CharRow& charRow = FirstRow.GetCharRow();
const auto stuff = L'A';
charRow.GlyphAt(0) = { &stuff, 1 };
// ensure it does say that it contains text
VERIFY_IS_TRUE(FirstRow.GetCharRow().ContainsText());
// try increment
textBuffer.IncrementCircularBuffer();
// validate that first row has moved
VERIFY_ARE_EQUAL(textBuffer._firstRow, iNextRowIndex); // first row has incremented
VERIFY_ARE_NOT_EQUAL(textBuffer._GetFirstRow(), FirstRow); // the old first row is no longer the first
// ensure old first row has been emptied
VERIFY_IS_FALSE(FirstRow.GetCharRow().ContainsText());
}
}
void TextBufferTests::TestMixedRgbAndLegacyForeground()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
const TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
const Cursor& cursor = tbi.GetCursor();
// Case 1 -
// Write '\E[m\E[38;2;64;128;255mX\E[49mX\E[m'
// Make sure that the second X has RGB attributes (FG and BG)
// FG = rgb(64;128;255), BG = rgb(default)
Log::Comment(L"Case 1 \"\\E[m\\E[38;2;64;128;255mX\\E[49mX\\E[m\"");
wchar_t* sequence = L"\x1b[m\x1b[38;2;64;128;255mX\x1b[49mX\x1b[m";
stateMachine.ProcessString(sequence);
const short x = cursor.GetPosition().X;
const short y = cursor.GetPosition().Y;
const ROW& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 2];
const auto attrB = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
VERIFY_ARE_EQUAL(attrA.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrB.IsLegacy(), false);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
const auto fgColor = RGB(64, 128, 255);
const auto bgColor = gci.LookupAttributeColors(si.GetAttributes()).second;
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA), std::make_pair(fgColor, bgColor));
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB), std::make_pair(fgColor, bgColor));
wchar_t* reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestMixedRgbAndLegacyBackground()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
const TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
const Cursor& cursor = tbi.GetCursor();
// Case 2 -
// \E[m\E[48;2;64;128;255mX\E[39mX\E[m
// Make sure that the second X has RGB attributes (FG and BG)
// FG = rgb(default), BG = rgb(64;128;255)
Log::Comment(L"Case 2 \"\\E[m\\E[48;2;64;128;255mX\\E[39mX\\E[m\"");
wchar_t* sequence = L"\x1b[m\x1b[48;2;64;128;255mX\x1b[39mX\x1b[m";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 2];
const auto attrB = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
VERIFY_ARE_EQUAL(attrA.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrB.IsLegacy(), false);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
const auto bgColor = RGB(64, 128, 255);
const auto fgColor = gci.LookupAttributeColors(si.GetAttributes()).first;
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA), std::make_pair(fgColor, bgColor));
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB), std::make_pair(fgColor, bgColor));
wchar_t* reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestMixedRgbAndLegacyUnderline()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
const TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
const Cursor& cursor = tbi.GetCursor();
// Case 3 -
// '\E[m\E[48;2;64;128;255mX\E[4mX\E[m'
// Make sure that the second X has RGB attributes AND underline
Log::Comment(L"Case 3 \"\\E[m\\E[48;2;64;128;255mX\\E[4mX\\E[m\"");
wchar_t* sequence = L"\x1b[m\x1b[48;2;64;128;255mX\x1b[4mX\x1b[m";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 2];
const auto attrB = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
VERIFY_ARE_EQUAL(attrA.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrB.IsLegacy(), false);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
const auto bgColor = RGB(64, 128, 255);
const auto fgColor = gci.LookupAttributeColors(si.GetAttributes()).first;
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA), std::make_pair(fgColor, bgColor));
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB), std::make_pair(fgColor, bgColor));
Render the SGR "underlined" attribute in the style of the font (#7148) This PR updates the rendering of the _underlined_ graphic rendition attribute, using the style specified in the active font, instead of just reusing the grid line at the bottom of the character cell. * Support for drawing the correct underline effect in the grid line renderer was added in #7107. There was already an `ExtendedAttributes` flag defined for the underlined state, but I needed to update the `SetUnderlined` and `IsUnderlined` methods in the `TextAttribute` class to use that flag now in place of the legacy `LVB_UNDERSCORE` attribute. This enables underlines set via a VT sequence to be tracked separately from `LVB_UNDERSCORE` grid lines set via the console API. I then needed to update the `Renderer::s_GetGridlines` method to activate the `GridLines::Underline` style when the `Underlined` attribute was set. The `GridLines::Bottom` style is still triggered by the `LVB_UNDERSCORE` attribute to produce the bottom grid line effect. Validation ---------- Because this is a change from the existing behaviour, certain unit tests that were expecting the `LVB_UNDERSCORE` to be toggled by `SGR 4` and `SGR 24` have now had to be updated to check the `Underlined` flag instead. There were also some UI Automation tests that were checking for `SGR 4` mapping to `LVB_UNDERSCORE` attribute, which I've now substituted with a test of the `SGR 53` overline attribute mapping to `LVB_GRID_HORIZONTAL`. These tests only work with legacy attributes, so they can't access the extended underline state, and I thought a replacement test that covered similar ground would be better than dropping the tests altogether. As far as the visual rendering is concerned, I've manually confirmed that the VT underline sequences now draw the underline in the correct position and style, while grid lines output via the console API are still displayed in their original form. Closes #2915
2020-08-03 14:49:25 +02:00
VERIFY_ARE_EQUAL(attrA.IsUnderlined(), false);
VERIFY_ARE_EQUAL(attrB.IsUnderlined(), true);
wchar_t* reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestMixedRgbAndLegacyBrightness()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
const TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
const Cursor& cursor = tbi.GetCursor();
// Case 4 -
// '\E[m\E[32mX\E[1mX'
// Make sure that the second X is a BRIGHT green, not white.
Log::Comment(L"Case 4 ;\"\\E[m\\E[32mX\\E[1mX\"");
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const auto dark_green = gci.GetColorTableEntry(TextColor::DARK_GREEN);
const auto bright_green = gci.GetColorTableEntry(TextColor::BRIGHT_GREEN);
VERIFY_ARE_NOT_EQUAL(dark_green, bright_green);
wchar_t* sequence = L"\x1b[m\x1b[32mX\x1b[1mX";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 2];
const auto attrB = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
VERIFY_ARE_EQUAL(attrA.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrB.IsLegacy(), false);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA).first, dark_green);
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB).first, bright_green);
wchar_t* reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestRgbEraseLine()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
Cursor& cursor = tbi.GetCursor();
WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
cursor.SetXPosition(0);
// Case 1 -
// Write '\E[m\E[48;2;64;128;255X\E[48;2;128;128;255\E[KX'
// Make sure that all the characters after the first have the rgb attrs
// BG = rgb(128;128;255)
{
std::wstring sequence = L"\x1b[m\x1b[48;2;64;128;255m";
stateMachine.ProcessString(sequence);
sequence = L"X";
stateMachine.ProcessString(sequence);
sequence = L"\x1b[48;2;128;128;255m";
stateMachine.ProcessString(sequence);
sequence = L"\x1b[K";
stateMachine.ProcessString(sequence);
sequence = L"X";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
VERIFY_ARE_EQUAL(x, 2);
VERIFY_ARE_EQUAL(y, 0);
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const auto len = tbi.GetSize().Width();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attr0 = attrs[0];
VERIFY_ARE_EQUAL(attr0.IsLegacy(), false);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attr0).second, RGB(64, 128, 255));
for (auto i = 1; i < len; i++)
{
const auto attr = attrs[i];
LOG_ATTR(attr);
VERIFY_ARE_EQUAL(attr.IsLegacy(), false);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attr).second, RGB(128, 128, 255));
}
std::wstring reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
}
void TextBufferTests::TestUnBold()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
Cursor& cursor = tbi.GetCursor();
WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
cursor.SetXPosition(0);
// Case 1 -
// Write '\E[1;32mX\E[22mX'
// The first X should be bright green.
// The second x should be dark green.
std::wstring sequence = L"\x1b[1;32mX\x1b[22mX";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const auto dark_green = gci.GetColorTableEntry(TextColor::DARK_GREEN);
const auto bright_green = gci.GetColorTableEntry(TextColor::BRIGHT_GREEN);
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
VERIFY_ARE_EQUAL(x, 2);
VERIFY_ARE_EQUAL(y, 0);
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 2];
const auto attrB = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA).first, bright_green);
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB).first, dark_green);
std::wstring reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestUnBoldRgb()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
Cursor& cursor = tbi.GetCursor();
WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
cursor.SetXPosition(0);
// Case 2 -
// Write '\E[1;32m\E[48;2;1;2;3mX\E[22mX'
// The first X should be bright green, and not legacy.
// The second X should be dark green, and not legacy.
// BG = rgb(1;2;3)
std::wstring sequence = L"\x1b[1;32m\x1b[48;2;1;2;3mX\x1b[22mX";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const auto dark_green = gci.GetColorTableEntry(TextColor::DARK_GREEN);
const auto bright_green = gci.GetColorTableEntry(TextColor::BRIGHT_GREEN);
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
VERIFY_ARE_EQUAL(x, 2);
VERIFY_ARE_EQUAL(y, 0);
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 2];
const auto attrB = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
VERIFY_ARE_EQUAL(attrA.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrB.IsLegacy(), false);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA).first, bright_green);
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB).first, dark_green);
std::wstring reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestComplexUnBold()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
Cursor& cursor = tbi.GetCursor();
WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
cursor.SetXPosition(0);
// Case 3 -
// Write '\E[1;32m\E[48;2;1;2;3mA\E[22mB\E[38;2;32;32;32mC\E[1mD\E[38;2;64;64;64mE\E[22mF'
// The A should be bright green, and not legacy.
// The B should be dark green, and not legacy.
// The C should be rgb(32, 32, 32), and not legacy.
// The D should be unchanged from the third.
// The E should be rgb(64, 64, 64), and not legacy.
// The F should be rgb(64, 64, 64), and not legacy.
// BG = rgb(1;2;3)
std::wstring sequence = L"\x1b[1;32m\x1b[48;2;1;2;3mA\x1b[22mB\x1b[38;2;32;32;32mC\x1b[1mD\x1b[38;2;64;64;64mE\x1b[22mF";
Log::Comment(NoThrowString().Format(sequence.c_str()));
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const auto dark_green = gci.GetColorTableEntry(TextColor::DARK_GREEN);
const auto bright_green = gci.GetColorTableEntry(TextColor::BRIGHT_GREEN);
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
VERIFY_ARE_EQUAL(x, 6);
VERIFY_ARE_EQUAL(y, 0);
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 6];
const auto attrB = attrs[x - 5];
const auto attrC = attrs[x - 4];
const auto attrD = attrs[x - 3];
const auto attrE = attrs[x - 2];
const auto attrF = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
Log::Comment(NoThrowString().Format(
L"attrA=%s", VerifyOutputTraits<TextAttribute>::ToString(attrA).GetBuffer()));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
LOG_ATTR(attrC);
LOG_ATTR(attrD);
LOG_ATTR(attrE);
LOG_ATTR(attrF);
VERIFY_ARE_EQUAL(attrA.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrB.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrC.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrD.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrE.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrF.IsLegacy(), false);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA), std::make_pair(bright_green, RGB(1, 2, 3)));
VERIFY_IS_TRUE(attrA.IsBold());
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB), std::make_pair(dark_green, RGB(1, 2, 3)));
VERIFY_IS_FALSE(attrB.IsBold());
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrC), std::make_pair(RGB(32, 32, 32), RGB(1, 2, 3)));
VERIFY_IS_FALSE(attrC.IsBold());
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrD), gci.LookupAttributeColors(attrC));
VERIFY_IS_TRUE(attrD.IsBold());
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrE), std::make_pair(RGB(64, 64, 64), RGB(1, 2, 3)));
VERIFY_IS_TRUE(attrE.IsBold());
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrF), std::make_pair(RGB(64, 64, 64), RGB(1, 2, 3)));
VERIFY_IS_FALSE(attrF.IsBold());
std::wstring reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
void TextBufferTests::CopyAttrs()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
Cursor& cursor = tbi.GetCursor();
WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
cursor.SetXPosition(0);
cursor.SetYPosition(0);
// Write '\E[32mX\E[33mX\n\E[34mX\E[35mX\E[H\E[M'
// The first two X's should get deleted.
// The third X should be blue
// The fourth X should be magenta
std::wstring sequence = L"\x1b[32mX\x1b[33mX\n\x1b[34mX\x1b[35mX\x1b[H\x1b[M";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const auto dark_blue = gci.GetColorTableEntry(TextColor::DARK_BLUE);
const auto dark_magenta = gci.GetColorTableEntry(TextColor::DARK_MAGENTA);
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
VERIFY_ARE_EQUAL(x, 0);
VERIFY_ARE_EQUAL(y, 0);
const auto& row = tbi.GetRowByOffset(0);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[0];
const auto attrB = attrs[1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA).first, dark_blue);
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB).first, dark_magenta);
}
void TextBufferTests::EmptySgrTest()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
Cursor& cursor = tbi.GetCursor();
WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
cursor.SetXPosition(0);
cursor.SetYPosition(0);
std::wstring reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
const auto [defaultFg, defaultBg] = gci.LookupAttributeColors(si.GetAttributes());
// Case 1 -
// Write '\x1b[0mX\x1b[31mX\x1b[31;m'
// The first X should be default colors.
// The second X should be (darkRed,default).
// The third X should be default colors.
std::wstring sequence = L"\x1b[0mX\x1b[31mX\x1b[31;mX";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const COLORREF darkRed = gci.GetColorTableEntry(TextColor::DARK_RED);
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
VERIFY_IS_TRUE(x >= 3);
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 3];
const auto attrB = attrs[x - 2];
const auto attrC = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
LOG_ATTR(attrC);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA), std::make_pair(defaultFg, defaultBg));
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB), std::make_pair(darkRed, defaultBg));
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrC), std::make_pair(defaultFg, defaultBg));
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestReverseReset()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
Cursor& cursor = tbi.GetCursor();
WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
cursor.SetXPosition(0);
cursor.SetYPosition(0);
std::wstring reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
const auto [defaultFg, defaultBg] = gci.LookupAttributeColors(si.GetAttributes());
// Case 1 -
// Write '\E[42m\E[38;2;128;5;255mX\E[7mX\E[27mX'
// The first X should be (fg,bg) = (rgb(128;5;255), dark_green)
// The second X should be (fg,bg) = (dark_green, rgb(128;5;255))
// The third X should be (fg,bg) = (rgb(128;5;255), dark_green)
std::wstring sequence = L"\x1b[42m\x1b[38;2;128;5;255mX\x1b[7mX\x1b[27mX";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const auto dark_green = gci.GetColorTableEntry(TextColor::DARK_GREEN);
const COLORREF rgbColor = RGB(128, 5, 255);
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
VERIFY_IS_TRUE(x >= 3);
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 3];
const auto attrB = attrs[x - 2];
const auto attrC = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
LOG_ATTR(attrC);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA), std::make_pair(rgbColor, dark_green));
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB), std::make_pair(dark_green, rgbColor));
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrC), std::make_pair(rgbColor, dark_green));
stateMachine.ProcessString(reset);
}
void TextBufferTests::CopyLastAttr()
{
DisableVerifyExceptions disable;
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
Cursor& cursor = tbi.GetCursor();
WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
cursor.SetXPosition(0);
cursor.SetYPosition(0);
std::wstring reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
const auto [defaultFg, defaultBg] = gci.LookupAttributeColors(si.GetAttributes());
const COLORREF solFg = RGB(101, 123, 131);
const COLORREF solBg = RGB(0, 43, 54);
const COLORREF solCyan = RGB(42, 161, 152);
std::wstring solFgSeq = L"\x1b[38;2;101;123;131m";
std::wstring solBgSeq = L"\x1b[48;2;0;43;54m";
std::wstring solCyanSeq = L"\x1b[38;2;42;161;152m";
// Make sure that the color table has certain values we expect
const COLORREF defaultBrightBlack = RGB(118, 118, 118);
const COLORREF defaultBrightYellow = RGB(249, 241, 165);
const COLORREF defaultBrightCyan = RGB(97, 214, 214);
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
gci.SetColorTableEntry(TextColor::BRIGHT_BLACK, defaultBrightBlack);
gci.SetColorTableEntry(TextColor::BRIGHT_YELLOW, defaultBrightYellow);
gci.SetColorTableEntry(TextColor::BRIGHT_CYAN, defaultBrightCyan);
// Write (solFg, solBG) X \n
// (solFg, solBG) X (solCyan, solBG) X \n
// (solFg, solBG) X (solCyan, solBG) X (solFg, solBG) X
// then go home, and insert a line.
// Row 1
stateMachine.ProcessString(solFgSeq);
stateMachine.ProcessString(solBgSeq);
stateMachine.ProcessString(L"X");
stateMachine.ProcessString(L"\n");
// Row 2
// Remember that the colors from before persist here too, so we don't need
// to emit both the FG and BG if they haven't changed.
stateMachine.ProcessString(L"X");
stateMachine.ProcessString(solCyanSeq);
stateMachine.ProcessString(L"X");
stateMachine.ProcessString(L"\n");
// Row 3
stateMachine.ProcessString(solFgSeq);
stateMachine.ProcessString(solBgSeq);
stateMachine.ProcessString(L"X");
stateMachine.ProcessString(solCyanSeq);
stateMachine.ProcessString(L"X");
stateMachine.ProcessString(solFgSeq);
stateMachine.ProcessString(L"X");
std::wstring insertLineAtHome = L"\x1b[H\x1b[L";
stateMachine.ProcessString(insertLineAtHome);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
const ROW& row1 = tbi.GetRowByOffset(y + 1);
const ROW& row2 = tbi.GetRowByOffset(y + 2);
const ROW& row3 = tbi.GetRowByOffset(y + 3);
const std::vector<TextAttribute> attrs1{ row1.GetAttrRow().begin(), row1.GetAttrRow().end() };
const std::vector<TextAttribute> attrs2{ row2.GetAttrRow().begin(), row2.GetAttrRow().end() };
const std::vector<TextAttribute> attrs3{ row3.GetAttrRow().begin(), row3.GetAttrRow().end() };
const auto attr1A = attrs1[0];
const auto attr2A = attrs2[0];
const auto attr2B = attrs2[1];
const auto attr3A = attrs3[0];
const auto attr3B = attrs3[1];
const auto attr3C = attrs3[2];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
LOG_ATTR(attr1A);
LOG_ATTR(attr2A);
LOG_ATTR(attr2A);
LOG_ATTR(attr3A);
LOG_ATTR(attr3B);
LOG_ATTR(attr3C);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attr1A), std::make_pair(solFg, solBg));
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attr2A), std::make_pair(solFg, solBg));
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attr2B), std::make_pair(solCyan, solBg));
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attr3A), std::make_pair(solFg, solBg));
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attr3B), std::make_pair(solCyan, solBg));
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attr3C), std::make_pair(solFg, solBg));
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestRgbThenBold()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
const TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
const Cursor& cursor = tbi.GetCursor();
// See MSFT:16398982
Log::Comment(NoThrowString().Format(
L"Test that a bold following a RGB color doesn't remove the RGB color"));
Log::Comment(L"\"\\x1b[38;2;40;40;40m\\x1b[48;2;168;153;132mX\\x1b[1mX\\x1b[m\"");
const auto foreground = RGB(40, 40, 40);
const auto background = RGB(168, 153, 132);
const wchar_t* const sequence = L"\x1b[38;2;40;40;40m\x1b[48;2;168;153;132mX\x1b[1mX\x1b[m";
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x - 2];
const auto attrB = attrs[x - 1];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
Log::Comment(NoThrowString().Format(
L"attrA should be RGB, and attrB should be the same as attrA, NOT bolded"));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
VERIFY_ARE_EQUAL(attrA.IsLegacy(), false);
VERIFY_ARE_EQUAL(attrB.IsLegacy(), false);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA), std::make_pair(foreground, background));
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB), std::make_pair(foreground, background));
wchar_t* reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestResetClearsBoldness()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
const TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
const Cursor& cursor = tbi.GetCursor();
Log::Comment(NoThrowString().Format(
L"Test that resetting bold attributes clears the boldness."));
const auto x0 = cursor.GetPosition().X;
// Test assumes that the background/foreground were default attribute when it starts up,
// so set that here.
TextAttribute defaultAttribute;
si.SetAttributes(defaultAttribute);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
const auto [defaultFg, defaultBg] = gci.LookupAttributeColors(si.GetAttributes());
Standardize the color table order (#11602) ## Summary of the Pull Request In the original implementation, we used two different orderings for the color tables. The WT color table used ANSI order, while the conhost color table used a Windows-specific order. This PR standardizes on the ANSI color order everywhere, so the usage of indexed colors is consistent across both parts of the code base, which will hopefully allow more of the code to be shared one day. ## References This is another small step towards de-duplicating `AdaptDispatch` and `TerminalDispatch` for issue #3849, and is essentially a followup to the SGR dispatch refactoring in PR #6728. ## PR Checklist * [x] Closes #11461 * [x] CLA signed. * [x] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #11461 ## Detailed Description of the Pull Request / Additional comments Conhost still needs to deal with legacy attributes using Windows color order, so those values now need to be transposed to ANSI colors order when creating a `TextAttribute` object. This is done with a simple mapping table, which also handles the translation of the default color entries, so it's actually slightly faster than the original code. And when converting `TextAttribute` values back to legacy console attributes, we were already using a mapping table to handle the narrowing of 256-color values down to 16 colors, so we just needed to adjust that table to account for the translation from ANSI to Windows, and then could make use of the same table for both 256-color and 16-color values. There are also a few places in conhost that read from or write to the color tables, and those now need to transpose the index values. I've addressed this by creating separate `SetLegacyColorTableEntry` and `GetLegacyColorTableEntry` methods in the `Settings` class which take care of the mapping, so it's now clearer in which cases the code is dealing with legacy values, and which are ANSI values. These methods are used in the `SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs, as well as a few place where color preferences are handled (the registry, shortcut links, and the properties dialog), none of which are particularly sensitive to performance. However, we also use the legacy table when looking up the default colors for rendering (which happens a lot), so I've refactored that code so the default color calculations now only occur once per frame. The plus side of all of this is that the VT code doesn't need to do the index translation anymore, so we can finally get rid of all the calls to `XTermToWindowsIndex`, and we no longer need a separate color table initialization method for conhost, so I was able to merge a number of color initialization methods into one. We also no longer need to translate from legacy values to ANSI when generating VT sequences for conpty. The one exception to that is the 16-color VT renderer, which uses the `TextColor::GetLegacyIndex` method to approximate 16-color equivalents for RGB and 256-color values. Since that method returns a legacy index, it still needs to be translated to ANSI before it can be used in a VT sequence. But this should be no worse than it was before. One more special case is conhost's secret _Color Selection_ feature. That uses `Ctrl`+Number and `Alt`+Number key sequences to highlight parts of the buffer, and the mapping from number to color is based on the Windows color order. So that mapping now needs to be transposed, but that's also not performance sensitive. The only thing that I haven't bothered to update is the trace logging code in the `Telemetry` class, which logs the first 16 entries in the color table. Those entries are now going to be in a different order, but I didn't think that would be of great concern to anyone. ## Validation Steps Performed A lot of unit tests needed to be updated to use ANSI color constants when setting indexed colors, where before they might have been expecting values in Windows order. But this replaced a wild mix of different constants, sometimes having to use bit shifting, as well as values mapped with `XTermToWindowsIndex`, so I think the tests are a whole lot clearer now. Only a few cases have been left with literal numbers where that seemed more appropriate. In addition to getting the unit tests working, I've also manually tested the behaviour of all the console APIs which I thought could be affected by these changes, and confirmed that they produced the same results in the new code as they did in the original implementation. This includes: - `WriteConsoleOutput` - `ReadConsoleOutput` - `SetConsoleTextAttribute` with `WriteConsoleOutputCharacter` - `FillConsoleOutputAttribute` and `FillConsoleOutputCharacter` - `ScrollConsoleScreenBuffer` - `GetConsoleScreenBufferInfo` - `GetConsoleScreenBufferInfoEx` - `SetConsoleScreenBufferInfoEx` I've also manually tested changing colors via the console properties menu, the registry, and shortcut links, including setting default colors and popup colors. And I've tested that the "Quirks Mode" is still working as expected in PowerShell. In terms of performance, I wrote a little test app that filled a 80x9999 buffer with random color combinations using `WriteConsoleOutput`, which I figured was likely to be the most performance sensitive call, and I think it now actually performs slightly better than the original implementation. I've also tested similar code - just filling the visible window - with SGR VT sequences of various types, and the performance seems about the same as it was before.
2021-11-04 23:13:22 +01:00
const auto dark_green = gci.GetColorTableEntry(TextColor::DARK_GREEN);
const auto bright_green = gci.GetColorTableEntry(TextColor::BRIGHT_GREEN);
wchar_t* sequence = L"\x1b[32mA\x1b[1mB\x1b[0mC\x1b[32mD";
Log::Comment(NoThrowString().Format(sequence));
stateMachine.ProcessString(sequence);
const auto x = cursor.GetPosition().X;
const auto y = cursor.GetPosition().Y;
const auto& row = tbi.GetRowByOffset(y);
const auto attrRow = &row.GetAttrRow();
const std::vector<TextAttribute> attrs{ attrRow->begin(), attrRow->end() };
const auto attrA = attrs[x0];
const auto attrB = attrs[x0 + 1];
const auto attrC = attrs[x0 + 2];
const auto attrD = attrs[x0 + 3];
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x,
y));
Log::Comment(NoThrowString().Format(
L"attrA should be RGB, and attrB should be the same as attrA, NOT bolded"));
LOG_ATTR(attrA);
LOG_ATTR(attrB);
LOG_ATTR(attrC);
LOG_ATTR(attrD);
Refactor the renderer color calculations (#6853) This is a refactoring of the renderer color calculations to simplify the implementation, and to make it easier to support additional color-altering rendition attributes in the future (e.g. _faint_ and _conceal_). ## References * This is a followup to PRs #3817 and #6809, which introduced additional complexity in the color calculations, and which suggested the need for refactoring. ## Detailed Description of the Pull Request / Additional comments When we added support for `DECSCNM`, that required the foreground and background color lookup methods to be able to return the opposite of what was requested when the reversed mode was set. That made those methods unnecessarily complicated, and I thought we could simplify them considerably just by combining the calculations into a single method that derived both colors at the same time. And since both conhost and Windows Terminal needed to perform the same calculations, it also made sense to move that functionality into the `TextAttribute` class, where it could easily be shared. In general this way of doing things is a bit more efficient. However, it does result in some unnecessary work when only one of the colors is required, as is the case for the gridline painter. So to make that less of an issue, I've reordered the gridline code a bit so it at least avoids looking up the colors when no gridlines are needed. ## Validation Steps Performed Because of the API changes, quite a lot of the unit tests had to be updated. For example instead of verifying colors with two separate calls to `LookupForegroundColor` and `LookupBackgroundColor`, that's now achieved with a single `LookupAttributeColors` call, comparing against a pair of values. The specifics of the tests haven't changed though, and they're all still working as expected. I've also manually confirmed that the various color sequences and rendition attributes are rendering correctly with the new refactoring.
2020-07-11 00:26:34 +02:00
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrA).first, dark_green);
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrB).first, bright_green);
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrC).first, defaultFg);
VERIFY_ARE_EQUAL(gci.LookupAttributeColors(attrD).first, dark_green);
VERIFY_IS_FALSE(attrA.IsBold());
VERIFY_IS_TRUE(attrB.IsBold());
VERIFY_IS_FALSE(attrC.IsBold());
VERIFY_IS_FALSE(attrD.IsBold());
wchar_t* reset = L"\x1b[0m";
stateMachine.ProcessString(reset);
}
void TextBufferTests::TestBackspaceRightSideVt()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
const TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
const Cursor& cursor = tbi.GetCursor();
Log::Comment(L"verify that backspace has the same behavior as a vt CUB sequence once "
L"we've traversed to the right side of the current row");
const wchar_t* const sequence = L"\033[1000Cx\by\n";
Log::Comment(NoThrowString().Format(sequence));
const auto preCursorPosition = cursor.GetPosition();
stateMachine.ProcessString(sequence);
const auto postCursorPosition = cursor.GetPosition();
// make sure newline was handled correctly
VERIFY_ARE_EQUAL(0, postCursorPosition.X);
VERIFY_ARE_EQUAL(preCursorPosition.Y, postCursorPosition.Y - 1);
// make sure "yx" was written to the end of the line the cursor started on
const auto& row = tbi.GetRowByOffset(preCursorPosition.Y);
const auto rowText = row.GetText();
auto it = rowText.crbegin();
VERIFY_ARE_EQUAL(*it, L'x');
++it;
VERIFY_ARE_EQUAL(*it, L'y');
}
void TextBufferTests::TestBackspaceStrings()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
const TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
const Cursor& cursor = tbi.GetCursor();
const auto x0 = cursor.GetPosition().X;
const auto y0 = cursor.GetPosition().Y;
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x0,
y0));
std::wstring seq = L"a\b \b";
stateMachine.ProcessString(seq);
const auto x1 = cursor.GetPosition().X;
const auto y1 = cursor.GetPosition().Y;
VERIFY_ARE_EQUAL(x1, x0);
VERIFY_ARE_EQUAL(y1, y0);
seq = L"a";
stateMachine.ProcessString(seq);
seq = L"\b";
stateMachine.ProcessString(seq);
seq = L" ";
stateMachine.ProcessString(seq);
seq = L"\b";
stateMachine.ProcessString(seq);
const auto x2 = cursor.GetPosition().X;
const auto y2 = cursor.GetPosition().Y;
VERIFY_ARE_EQUAL(x2, x0);
VERIFY_ARE_EQUAL(y2, y0);
}
void TextBufferTests::TestBackspaceStringsAPI()
{
// Pretty much the same as the above test, but explicitly DOESN'T use the
// state machine.
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
const TextBuffer& tbi = si.GetTextBuffer();
const Cursor& cursor = tbi.GetCursor();
gci.SetVirtTermLevel(0);
WI_ClearFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
const auto x0 = cursor.GetPosition().X;
const auto y0 = cursor.GetPosition().Y;
Log::Comment(NoThrowString().Format(
L"cursor={X:%d,Y:%d}",
x0,
y0));
// We're going to write an "a" to the buffer in various ways, then try
// backspacing it with "\b \b".
// Regardless of how we write those sequences of characters, the end result
// should be the same.
std::unique_ptr<WriteData> waiter;
size_t aCb = 2;
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
VERIFY_SUCCEEDED(DoWriteConsole(L"a", &aCb, si, false, waiter));
size_t seqCb = 6;
Log::Comment(NoThrowString().Format(
L"Using WriteCharsLegacy, write \\b \\b as a single string."));
{
wchar_t* str = L"\b \b";
VERIFY_SUCCESS_NTSTATUS(WriteCharsLegacy(si, str, str, str, &seqCb, nullptr, cursor.GetPosition().X, 0, nullptr));
VERIFY_ARE_EQUAL(cursor.GetPosition().X, x0);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, y0);
Log::Comment(NoThrowString().Format(
L"Using DoWriteConsole, write \\b \\b as a single string."));
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
VERIFY_SUCCEEDED(DoWriteConsole(L"a", &aCb, si, false, waiter));
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
VERIFY_SUCCEEDED(DoWriteConsole(str, &seqCb, si, false, waiter));
VERIFY_ARE_EQUAL(cursor.GetPosition().X, x0);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, y0);
}
seqCb = 2;
Log::Comment(NoThrowString().Format(
L"Using DoWriteConsole, write \\b \\b as separate strings."));
Reintroduce a color compatibility hack, but only for PowerShells (#6810) There is going to be a very long tail of applications that will explicitly request VT SGR 40/37 when what they really want is to SetConsoleTextAttribute() with a black background/white foreground. Instead of making those applications look bad (and therefore making us look bad, because we're releasing this as an update to something that "looks good" already), we're introducing this compatibility quirk. Before the color reckoning in #6698 + #6506, *every* color was subject to being spontaneously and erroneously turned into the default color. Now, only the 16-color palette value that matches the active console background/foreground color will be destroyed, and only when received from specific applications. Removal will be tracked by #6807. Michael and I discussed what layer this quirk really belonged in. I originally believed it would be sufficient to detect a background color that matched the legacy default background, but @j4james provided an example of where that wouldn't work out (powershell setting the foreground color to white/gray). In addition, it was too heavyhanded: it re-broke black backgrounds for every application. Michael thought that it should live in the server, as a small VT parser that righted the wrongs coming directly out of the application. On further investigation, however, I realized that we'd need to push more information up into the server (so that it could make the decision about which VT was wrong and which was right) than should be strictly necessary. The host knows which colors are right and wrong, and it gets final say in what ends up in the buffer. Because of that, I chose to push the quirk state down through WriteConsole to DoWriteConsole and toggle state on the SCREEN_INFORMATION that indicates whether the colors coming out of the application are to be distrusted. This quirk _only applies to pwsh.exe and powershell.exe._ NOTE: This doesn't work for PowerShell the .NET Global tool, because it is run as an assembly through dotnet.exe. I have no opinion on how to fix this, or whether it is worth fixing. VALIDATION ---------- I configured my terminals to have an incredibly garish color scheme to show exactly what's going to happen as a result of this. The _default terminal background_ is purple or red, and the foreground green. I've printed out a heap of test colors to see how black interacts with them. Pull request #6810 contains the images generated from this test. The only color lines that change are the ones where black as a background or white as a foreground is selected out of the 16-color palette explicitly. Reverse video still works fine (because black is in the foreground!), and it's even possible to represent "black on default" and reverse it into "default on black", despite the black in question having been `40`. Fixes #6767.
2020-07-11 00:25:39 +02:00
VERIFY_SUCCEEDED(DoWriteConsole(L"a", &seqCb, si, false, waiter));
VERIFY_SUCCEEDED(DoWriteConsole(L"\b", &seqCb, si, false, waiter));
VERIFY_SUCCEEDED(DoWriteConsole(L" ", &seqCb, si, false, waiter));
VERIFY_SUCCEEDED(DoWriteConsole(L"\b", &seqCb, si, false, waiter));
VERIFY_ARE_EQUAL(cursor.GetPosition().X, x0);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, y0);
Log::Comment(NoThrowString().Format(
L"Using WriteCharsLegacy, write \\b \\b as separate strings."));
{
wchar_t* str = L"a";
VERIFY_SUCCESS_NTSTATUS(WriteCharsLegacy(si, str, str, str, &seqCb, nullptr, cursor.GetPosition().X, 0, nullptr));
}
{
wchar_t* str = L"\b";
VERIFY_SUCCESS_NTSTATUS(WriteCharsLegacy(si, str, str, str, &seqCb, nullptr, cursor.GetPosition().X, 0, nullptr));
}
{
wchar_t* str = L" ";
VERIFY_SUCCESS_NTSTATUS(WriteCharsLegacy(si, str, str, str, &seqCb, nullptr, cursor.GetPosition().X, 0, nullptr));
}
{
wchar_t* str = L"\b";
VERIFY_SUCCESS_NTSTATUS(WriteCharsLegacy(si, str, str, str, &seqCb, nullptr, cursor.GetPosition().X, 0, nullptr));
}
VERIFY_ARE_EQUAL(cursor.GetPosition().X, x0);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, y0);
}
void TextBufferTests::TestRepeatCharacter()
{
CONSOLE_INFORMATION& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
SCREEN_INFORMATION& si = gci.GetActiveOutputBuffer().GetActiveBuffer();
TextBuffer& tbi = si.GetTextBuffer();
StateMachine& stateMachine = si.GetStateMachine();
Cursor& cursor = tbi.GetCursor();
WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
cursor.SetXPosition(0);
cursor.SetYPosition(0);
Log::Comment(
L"Test 0: Simply repeat a single character.");
std::wstring sequence = L"X";
stateMachine.ProcessString(sequence);
sequence = L"\x1b[b";
stateMachine.ProcessString(sequence);
VERIFY_ARE_EQUAL(cursor.GetPosition().X, 2);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, 0);
{
const auto& row0 = tbi.GetRowByOffset(0);
const auto row0Text = row0.GetText();
VERIFY_ARE_EQUAL(L'X', row0Text[0]);
VERIFY_ARE_EQUAL(L'X', row0Text[1]);
VERIFY_ARE_EQUAL(L' ', row0Text[2]);
}
Log::Comment(
L"Test 1: Try repeating characters after another VT action. It should do nothing.");
stateMachine.ProcessString(L"\n");
stateMachine.ProcessString(L"A");
stateMachine.ProcessString(L"B");
stateMachine.ProcessString(L"\x1b[A");
stateMachine.ProcessString(L"\x1b[b");
VERIFY_ARE_EQUAL(cursor.GetPosition().X, 2);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, 0);
{
const auto& row0 = tbi.GetRowByOffset(0);
const auto& row1 = tbi.GetRowByOffset(1);
const auto row0Text = row0.GetText();
const auto row1Text = row1.GetText();
VERIFY_ARE_EQUAL(L'X', row0Text[0]);
VERIFY_ARE_EQUAL(L'X', row0Text[1]);
VERIFY_ARE_EQUAL(L' ', row0Text[2]);
VERIFY_ARE_EQUAL(L'A', row1Text[0]);
VERIFY_ARE_EQUAL(L'B', row1Text[1]);
VERIFY_ARE_EQUAL(L' ', row1Text[2]);
}
Log::Comment(
L"Test 2: Repeat a character lots of times");
stateMachine.ProcessString(L"\x1b[3;H");
stateMachine.ProcessString(L"C");
stateMachine.ProcessString(L"\x1b[5b");
VERIFY_ARE_EQUAL(cursor.GetPosition().X, 6);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, 2);
{
const auto& row2 = tbi.GetRowByOffset(2);
const auto row2Text = row2.GetText();
VERIFY_ARE_EQUAL(L'C', row2Text[0]);
VERIFY_ARE_EQUAL(L'C', row2Text[1]);
VERIFY_ARE_EQUAL(L'C', row2Text[2]);
VERIFY_ARE_EQUAL(L'C', row2Text[3]);
VERIFY_ARE_EQUAL(L'C', row2Text[4]);
VERIFY_ARE_EQUAL(L'C', row2Text[5]);
VERIFY_ARE_EQUAL(L' ', row2Text[6]);
}
Log::Comment(
L"Test 3: try repeating a non-graphical character. It should do nothing.");
stateMachine.ProcessString(L"\r\n");
VERIFY_ARE_EQUAL(cursor.GetPosition().X, 0);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, 3);
stateMachine.ProcessString(L"D\n");
stateMachine.ProcessString(L"\x1b[b");
VERIFY_ARE_EQUAL(cursor.GetPosition().X, 0);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, 4);
Log::Comment(
L"Test 4: try repeating multiple times. It should do nothing.");
stateMachine.ProcessString(L"\r\n");
VERIFY_ARE_EQUAL(cursor.GetPosition().X, 0);
VERIFY_ARE_EQUAL(cursor.GetPosition().Y, 5);
stateMachine.ProcessString(L"E");
VERIFY_ARE_EQUAL(cursor.GetPosition().X, 1);
stateMachine.ProcessString(L"\x1b[b");
VERIFY_ARE_EQUAL(cursor.GetPosition().X, 2);
stateMachine.ProcessString(L"\x1b[b");
VERIFY_ARE_EQUAL(cursor.GetPosition().X, 2);
{
const auto& row5 = tbi.GetRowByOffset(5);
const auto row5Text = row5.GetText();
VERIFY_ARE_EQUAL(L'E', row5Text[0]);
VERIFY_ARE_EQUAL(L'E', row5Text[1]);
VERIFY_ARE_EQUAL(L' ', row5Text[2]);
}
}
void TextBufferTests::ResizeTraditional()
{
BEGIN_TEST_METHOD_PROPERTIES()
TEST_METHOD_PROPERTY(L"Data:shrinkX", L"{false, true}")
TEST_METHOD_PROPERTY(L"Data:shrinkY", L"{false, true}")
END_TEST_METHOD_PROPERTIES();
bool shrinkX;
VERIFY_SUCCEEDED(TestData::TryGetValue(L"shrinkX", shrinkX), L"Shrink X = true, Grow X = false");
bool shrinkY;
VERIFY_SUCCEEDED(TestData::TryGetValue(L"shrinkY", shrinkY), L"Shrink Y = true, Grow Y = false");
const COORD smallSize = { 5, 5 };
Refactor the SGR implementation in AdaptDispatch (#5758) This is an attempt to simplify the SGR (Select Graphic Rendition) implementation in conhost, to cut down on the number of methods required in the `ConGetSet` interface, and pave the way for future improvements and bug fixes. It already fixes one bug that prevented SGR 0 from being correctly applied when combined with meta attributes. * This a first step towards fixing the conpty narrowing bugs in issue #2661 * I'm hoping the simplification of `ConGetSet` will also help with #3849. * Some of the `TextAttribute` refactoring in this PR overlaps with similar work in PR #1978. ## Detailed Description of the Pull Request / Additional comments The main point of this PR was to simplify the `AdaptDispatch::SetGraphicsRendition` implementation. So instead of having it call a half a dozen methods in the `ConGetSet` API, depending on what kinds of attributes needed to be set, there is now just one call to get current attributes, and another call to set the new value. All adjustments to the attributes are made in the `AdaptDispatch` class, in a simple switch statement. To help with this refactoring, I also made some change to the `TextAttribute` class to make it easier to work with. This included adding a set of methods for setting (and getting) the individual attribute flags, instead of having the calling code being exposed to the internal attribute structures and messing with bit manipulation. I've tried to get rid of any methods that were directly setting legacy, meta, and extended attributes. Other than the fix to the `SGR 0` bug, the `AdaptDispatch` refactoring mostly follows the behaviour of the original code. In particular, it still maps the `SGR 38/48` indexed colors to RGB instead of retaining the index, which is what we ultimately need it to do. Fixing that will first require the color tables to be unified (issue #1223), which I'm hoping to address in a followup PR. But for now, mapping the indexed colors to RGB values required adding an an additional `ConGetSet` API to lookup the color table entries. In the future that won't be necessary, but the API will still be useful for other color reporting operations that we may want to support. I've made this API, and the existing setter, standardise on index values being in the "Xterm" order, since that'll be essential for unifying the code with the terminal adapter one day. I should also point out one minor change to the `SGR 38/48` behavior, which is that out-of-range RGB colors are now ignored rather than being clamped, since that matches the way Xterm works. ## Validation Steps Performed This refactoring has obviously required corresponding changes to the unit tests, but most were just minor updates to use the new `TextAttribute` methods without any real change in behavior. However, the adapter tests did require significant changes to accommodate the new `ConGetSet` API. The basic structure of the tests remain the same, but the simpler API has meant fewer values needed to be checked in each test case. I think they are all still covering the areas there were intended to, though, and they are all still passing. Other than getting the unit tests to work, I've also done a bunch of manual testing of my own. I've made sure the color tests in Vttest all still work as well as they used to. And I've confirmed that the test case from issue #5341 is now working correctly. Closes #5341
2020-05-09 01:04:16 +02:00
const TextAttribute defaultAttr(0);
TextBuffer buffer(smallSize, defaultAttr, 12, _renderTarget);
Log::Comment(L"Fill buffer with some data and do assorted resize operations.");
wchar_t expectedChar = L'A';
const std::wstring_view expectedView(&expectedChar, 1);
TextAttribute expectedAttr(FOREGROUND_RED);
OutputCellIterator it(expectedChar, expectedAttr);
const auto finalIt = buffer.Write(it);
VERIFY_ARE_EQUAL(smallSize.X * smallSize.Y, finalIt.GetCellDistance(it), L"Verify we said we filled every cell.");
const Viewport writtenView = Viewport::FromDimensions({ 0, 0 }, smallSize);
Log::Comment(L"Ensure every cell has our test pattern value.");
{
TextBufferCellIterator viewIt(buffer, { 0, 0 });
while (viewIt)
{
VERIFY_ARE_EQUAL(expectedView, viewIt->Chars());
VERIFY_ARE_EQUAL(expectedAttr, viewIt->TextAttr());
viewIt++;
}
}
Log::Comment(L"Resize to X and Y.");
COORD newSize = smallSize;
if (shrinkX)
{
newSize.X -= 2;
}
else
{
newSize.X += 2;
}
if (shrinkY)
{
newSize.Y -= 2;
}
else
{
newSize.Y += 2;
}
// When we grow, we extend the last color. Therefore, this region covers the area colored the same as the letters but filled with a blank.
const auto widthAdjustedView = Viewport::FromDimensions(writtenView.Origin(), { newSize.X, smallSize.Y });
// When we resize, we expect the attributes to be unchanged, but the new cells
// to be filled with spaces
wchar_t expectedSpace = UNICODE_SPACE;
std::wstring_view expectedSpaceView(&expectedSpace, 1);
VERIFY_SUCCEEDED(buffer.ResizeTraditional(newSize));
Log::Comment(L"Verify every cell in the X dimension is still the same as when filled and the new Y row is just empty default cells.");
{
TextBufferCellIterator viewIt(buffer, { 0, 0 });
while (viewIt)
{
Log::Comment(NoThrowString().Format(L"Checking cell (Y=%d, X=%d)", viewIt._pos.Y, viewIt._pos.X));
if (writtenView.IsInBounds(viewIt._pos))
{
Log::Comment(L"This position is inside our original write area. It should have the original character and color.");
// If the position is in bounds with what we originally wrote, it should have that character and color.
VERIFY_ARE_EQUAL(expectedView, viewIt->Chars());
VERIFY_ARE_EQUAL(expectedAttr, viewIt->TextAttr());
}
else if (widthAdjustedView.IsInBounds(viewIt._pos))
{
Log::Comment(L"This position is right of our original write area. It should have extended the color rightward and filled with a space.");
// If we missed the original fill, but we're still in region defined by the adjusted width, then
// the color was extended outward but without the character value.
VERIFY_ARE_EQUAL(expectedSpaceView, viewIt->Chars());
VERIFY_ARE_EQUAL(expectedAttr, viewIt->TextAttr());
}
else
{
Log::Comment(L"This position is below our original write area. It should have filled blank lines (space lines) with the default fill color.");
// Otherwise, we use the default.
VERIFY_ARE_EQUAL(expectedSpaceView, viewIt->Chars());
VERIFY_ARE_EQUAL(defaultAttr, viewIt->TextAttr());
}
viewIt++;
}
}
}
// This tests that when buffer storage rows are rotated around during a resize traditional operation,
// that the Unicode Storage-held high unicode items like emoji rotate properly with it.
void TextBufferTests::ResizeTraditionalRotationPreservesHighUnicode()
{
// Set up a text buffer for us
const COORD bufferSize{ 80, 10 };
const UINT cursorSize = 12;
const TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// Get a position inside the buffer
const COORD pos{ 2, 1 };
auto position = _buffer->_storage[pos.Y].GetCharRow().GlyphAt(pos.X);
// Fill it up with a sequence that will have to hit the high unicode storage.
// This is the negative squared latin capital letter B emoji: 🅱
// It's encoded in UTF-16, as needed by the buffer.
const auto bButton = L"\xD83C\xDD71";
position = bButton;
// Read back the text at that position and ensure that it matches what we wrote.
const auto readBack = _buffer->GetTextDataAt(pos);
const auto readBackText = *readBack;
VERIFY_ARE_EQUAL(String(bButton), String(readBackText.data(), gsl::narrow<int>(readBackText.size())));
// Make it the first row in the buffer so it will rotate around when we resize and cause renumbering
const SHORT delta = _buffer->GetFirstRowIndex() - pos.Y;
const COORD newPos{ pos.X, pos.Y + delta };
_buffer->_SetFirstRowIndex(pos.Y);
// Perform resize to rotate the rows around
VERIFY_NT_SUCCESS(_buffer->ResizeTraditional(bufferSize));
// Retrieve the text at the old and new positions.
const auto shouldBeEmptyText = *_buffer->GetTextDataAt(pos);
const auto shouldBeEmojiText = *_buffer->GetTextDataAt(newPos);
VERIFY_ARE_EQUAL(String(L" "), String(shouldBeEmptyText.data(), gsl::narrow<int>(shouldBeEmptyText.size())));
VERIFY_ARE_EQUAL(String(bButton), String(shouldBeEmojiText.data(), gsl::narrow<int>(shouldBeEmojiText.size())));
}
// This tests that when buffer storage rows are rotated around during a scroll buffer operation,
// that the Unicode Storage-held high unicode items like emoji rotate properly with it.
void TextBufferTests::ScrollBufferRotationPreservesHighUnicode()
{
// Set up a text buffer for us
const COORD bufferSize{ 80, 10 };
const UINT cursorSize = 12;
const TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// Get a position inside the buffer
const COORD pos{ 2, 1 };
auto position = _buffer->_storage[pos.Y].GetCharRow().GlyphAt(pos.X);
// Fill it up with a sequence that will have to hit the high unicode storage.
// This is the fire emoji: 🔥
// It's encoded in UTF-16, as needed by the buffer.
const auto fire = L"\xD83D\xDD25";
position = fire;
// Read back the text at that position and ensure that it matches what we wrote.
const auto readBack = _buffer->GetTextDataAt(pos);
const auto readBackText = *readBack;
VERIFY_ARE_EQUAL(String(fire), String(readBackText.data(), gsl::narrow<int>(readBackText.size())));
// Prepare a delta and the new position we expect the symbol to be moved into.
const SHORT delta = 5;
const COORD newPos{ pos.X, pos.Y + delta };
// Scroll the row with our data by delta.
_buffer->ScrollRows(pos.Y, 1, delta);
// Retrieve the text at the old and new positions.
const auto shouldBeEmptyText = *_buffer->GetTextDataAt(pos);
const auto shouldBeFireText = *_buffer->GetTextDataAt(newPos);
VERIFY_ARE_EQUAL(String(L" "), String(shouldBeEmptyText.data(), gsl::narrow<int>(shouldBeEmptyText.size())));
VERIFY_ARE_EQUAL(String(fire), String(shouldBeFireText.data(), gsl::narrow<int>(shouldBeFireText.size())));
}
// This tests that rows removed from the buffer while resizing traditionally will also drop the high unicode
// characters from the Unicode Storage buffer
void TextBufferTests::ResizeTraditionalHighUnicodeRowRemoval()
{
// Set up a text buffer for us
const COORD bufferSize{ 80, 10 };
const UINT cursorSize = 12;
const TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// Get a position inside the buffer in the bottom row
const COORD pos{ 0, bufferSize.Y - 1 };
auto position = _buffer->_storage[pos.Y].GetCharRow().GlyphAt(pos.X);
// Fill it up with a sequence that will have to hit the high unicode storage.
// This is the eggplant emoji: 🍆
// It's encoded in UTF-16, as needed by the buffer.
const auto emoji = L"\xD83C\xDF46";
position = emoji;
// Read back the text at that position and ensure that it matches what we wrote.
const auto readBack = _buffer->GetTextDataAt(pos);
const auto readBackText = *readBack;
VERIFY_ARE_EQUAL(String(emoji), String(readBackText.data(), gsl::narrow<int>(readBackText.size())));
VERIFY_ARE_EQUAL(1u, _buffer->GetUnicodeStorage()._map.size(), L"There should be one item in the map.");
// Perform resize to trim off the row of the buffer that included the emoji
COORD trimmedBufferSize{ bufferSize.X, bufferSize.Y - 1 };
VERIFY_NT_SUCCESS(_buffer->ResizeTraditional(trimmedBufferSize));
VERIFY_IS_TRUE(_buffer->GetUnicodeStorage()._map.empty(), L"The map should now be empty.");
}
// This tests that columns removed from the buffer while resizing traditionally will also drop the high unicode
// characters from the Unicode Storage buffer
void TextBufferTests::ResizeTraditionalHighUnicodeColumnRemoval()
{
// Set up a text buffer for us
const COORD bufferSize{ 80, 10 };
const UINT cursorSize = 12;
const TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// Get a position inside the buffer in the last column
const COORD pos{ bufferSize.X - 1, 0 };
auto position = _buffer->_storage[pos.Y].GetCharRow().GlyphAt(pos.X);
// Fill it up with a sequence that will have to hit the high unicode storage.
// This is the peach emoji: 🍑
// It's encoded in UTF-16, as needed by the buffer.
const auto emoji = L"\xD83C\xDF51";
position = emoji;
// Read back the text at that position and ensure that it matches what we wrote.
const auto readBack = _buffer->GetTextDataAt(pos);
const auto readBackText = *readBack;
VERIFY_ARE_EQUAL(String(emoji), String(readBackText.data(), gsl::narrow<int>(readBackText.size())));
VERIFY_ARE_EQUAL(1u, _buffer->GetUnicodeStorage()._map.size(), L"There should be one item in the map.");
// Perform resize to trim off the column of the buffer that included the emoji
COORD trimmedBufferSize{ bufferSize.X - 1, bufferSize.Y };
VERIFY_NT_SUCCESS(_buffer->ResizeTraditional(trimmedBufferSize));
VERIFY_IS_TRUE(_buffer->GetUnicodeStorage()._map.empty(), L"The map should now be empty.");
}
void TextBufferTests::TestBurrito()
{
COORD bufferSize{ 80, 9001 };
UINT cursorSize = 12;
TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// This is the burrito emoji: 🌯
// It's encoded in UTF-16, as needed by the buffer.
const auto burrito = L"\xD83C\xDF2F";
OutputCellIterator burriter{ burrito };
auto afterFIter = _buffer->Write({ L"F" });
_buffer->IncrementCursor();
auto afterBurritoIter = _buffer->Write(burriter);
_buffer->IncrementCursor();
_buffer->IncrementCursor();
VERIFY_IS_FALSE(afterBurritoIter);
}
Refactor UiaTextRange For Improved Navigation and Reliability (#4018) ## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
2020-01-31 21:59:39 +01:00
void TextBufferTests::WriteLinesToBuffer(const std::vector<std::wstring>& text, TextBuffer& buffer)
{
const auto bufferSize = buffer.GetSize();
Refactor UiaTextRange For Improved Navigation and Reliability (#4018) ## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
2020-01-31 21:59:39 +01:00
for (size_t row = 0; row < text.size(); ++row)
{
auto line = text[row];
if (!line.empty())
{
// TODO GH#780: writing up to (but not past) the end of the line
// should NOT set the wrap flag
std::optional<bool> wrap = true;
if (line.size() == static_cast<size_t>(bufferSize.RightExclusive()))
{
wrap = std::nullopt;
}
OutputCellIterator iter{ line };
buffer.Write(iter, { 0, gsl::narrow<SHORT>(row) }, wrap);
}
Refactor UiaTextRange For Improved Navigation and Reliability (#4018) ## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
2020-01-31 21:59:39 +01:00
}
}
void TextBufferTests::GetWordBoundaries()
{
COORD bufferSize{ 80, 9001 };
UINT cursorSize = 12;
TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// Setup: Write lines of text to the buffer
const std::vector<std::wstring> text = { L"word other",
L" more words" };
WriteLinesToBuffer(text, *_buffer);
// Test Data:
// - COORD - starting position
// - COORD - expected result (accessibilityMode = false)
// - COORD - expected result (accessibilityMode = true)
struct ExpectedResult
{
COORD accessibilityModeDisabled;
COORD accessibilityModeEnabled;
};
struct Test
{
COORD startPos;
ExpectedResult expected;
};
// Set testData for GetWordStart tests
// clang-format off
std::vector<Test> testData = {
// tests for first line of text
{ { 0, 0 }, {{ 0, 0 }, { 0, 0 }} },
{ { 1, 0 }, {{ 0, 0 }, { 0, 0 }} },
{ { 3, 0 }, {{ 0, 0 }, { 0, 0 }} },
{ { 4, 0 }, {{ 4, 0 }, { 0, 0 }} },
{ { 5, 0 }, {{ 5, 0 }, { 5, 0 }} },
{ { 6, 0 }, {{ 5, 0 }, { 5, 0 }} },
{ { 20, 0 }, {{ 10, 0 }, { 5, 0 }} },
{ { 79, 0 }, {{ 10, 0 }, { 5, 0 }} },
// tests for second line of text
{ { 0, 1 }, {{ 0, 1 }, { 5, 0 }} },
Refactor UiaTextRange For Improved Navigation and Reliability (#4018) ## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
2020-01-31 21:59:39 +01:00
{ { 1, 1 }, {{ 0, 1 }, { 5, 0 }} },
{ { 2, 1 }, {{ 2, 1 }, { 2, 1 }} },
{ { 3, 1 }, {{ 2, 1 }, { 2, 1 }} },
{ { 5, 1 }, {{ 2, 1 }, { 2, 1 }} },
{ { 6, 1 }, {{ 6, 1 }, { 2, 1 }} },
{ { 7, 1 }, {{ 6, 1 }, { 2, 1 }} },
{ { 9, 1 }, {{ 9, 1 }, { 9, 1 }} },
{ { 10, 1 }, {{ 9, 1 }, { 9, 1 }} },
{ { 20, 1 }, {{14, 1 }, { 9, 1 }} },
{ { 79, 1 }, {{14, 1 }, { 9, 1 }} },
};
// clang-format on
BEGIN_TEST_METHOD_PROPERTIES()
TEST_METHOD_PROPERTY(L"Data:accessibilityMode", L"{false, true}")
END_TEST_METHOD_PROPERTIES();
bool accessibilityMode;
VERIFY_SUCCEEDED(TestData::TryGetValue(L"accessibilityMode", accessibilityMode), L"Get accessibility mode variant");
const std::wstring_view delimiters = L" ";
for (const auto& test : testData)
{
Log::Comment(NoThrowString().Format(L"COORD (%hd, %hd)", test.startPos.X, test.startPos.Y));
const auto result = _buffer->GetWordStart(test.startPos, delimiters, accessibilityMode);
const auto expected = accessibilityMode ? test.expected.accessibilityModeEnabled : test.expected.accessibilityModeDisabled;
VERIFY_ARE_EQUAL(expected, result);
}
// Update testData for GetWordEnd tests
// clang-format off
testData = {
// tests for first line of text
{ { 0, 0 }, { { 3, 0 }, { 5, 0 } } },
{ { 1, 0 }, { { 3, 0 }, { 5, 0 } } },
{ { 3, 0 }, { { 3, 0 }, { 5, 0 } } },
{ { 4, 0 }, { { 4, 0 }, { 5, 0 } } },
{ { 5, 0 }, { { 9, 0 }, { 2, 1 } } },
{ { 6, 0 }, { { 9, 0 }, { 2, 1 } } },
{ { 20, 0 }, { { 79, 0 }, { 2, 1 } } },
{ { 79, 0 }, { { 79, 0 }, { 2, 1 } } },
// tests for second line of text
{ { 0, 1 }, { { 1, 1 }, { 2, 1 } } },
{ { 1, 1 }, { { 1, 1 }, { 2, 1 } } },
{ { 2, 1 }, { { 5, 1 }, { 9, 1 } } },
{ { 3, 1 }, { { 5, 1 }, { 9, 1 } } },
{ { 5, 1 }, { { 5, 1 }, { 9, 1 } } },
{ { 6, 1 }, { { 8, 1 }, { 9, 1 } } },
{ { 7, 1 }, { { 8, 1 }, { 9, 1 } } },
{ { 9, 1 }, { { 13, 1 }, { 0, 9001 } } },
{ { 10, 1 }, { { 13, 1 }, { 0, 9001 } } },
{ { 20, 1 }, { { 79, 1 }, { 0, 9001 } } },
{ { 79, 1 }, { { 79, 1 }, { 0, 9001 } } },
};
// clang-format on
for (const auto& test : testData)
{
Log::Comment(NoThrowString().Format(L"COORD (%hd, %hd)", test.startPos.X, test.startPos.Y));
COORD result = _buffer->GetWordEnd(test.startPos, delimiters, accessibilityMode);
const auto expected = accessibilityMode ? test.expected.accessibilityModeEnabled : test.expected.accessibilityModeDisabled;
VERIFY_ARE_EQUAL(expected, result);
}
}
Move rect expansion to textbuffer; refactor selection code (#4560) - When performing chunk selection, the expansion now occurs at the time of the selection, not the rendering of the selection - `GetSelectionRects()` was moved to the `TextBuffer` and is now shared between ConHost and Windows Terminal - Some of the selection variables were renamed for clarity - Selection COORDs are now in the Text Buffer coordinate space - Fixes an issue with Shift+Click after performing a Multi-Click Selection ## References This also contributes to... - #4509: UIA Box Selection - #2447: UIA Signaling for Selection - #1354: UIA support for Wide Glyphs Now that the expansion occurs at before render-time, the selection anchors are an accurate representation of what is selected. We just need to move `GetText` to the `TextBuffer`. Then we can have those three issues just rely on code from the text buffer. This also means ConHost gets some of this stuff for free 😀 ### TextBuffer - `GetTextRects` is the abstracted form of `GetSelectionRects` - `_ExpandTextRow` is still needed to handle wide glyphs properly ### Terminal - Rename... - `_boxSelection` --> `_blockSelection` for consistency with ConHost - `_selectionAnchor` --> `_selectionStart` for consistency with UIA - `_endSelectionPosition` --> `_selectionEnd` for consistency with UIA - Selection anchors are in Text Buffer coordinates now - Really rely on `SetSelectionEnd` to accomplish appropriate chunk selection and shift+click actions ## Validation Steps Performed - Shift+Click - Multi-Click --> Shift+Click - Chunk Selection at... - top of buffer - bottom of buffer - random region in scrollback Closes #4465 Closes #4547
2020-02-28 01:42:26 +01:00
void TextBufferTests::MoveByWord()
{
COORD bufferSize{ 80, 9001 };
UINT cursorSize = 12;
TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// Setup: Write lines of text to the buffer
const std::vector<std::wstring> text = { L"word other",
L" more words" };
WriteLinesToBuffer(text, *_buffer);
// Test Data:
// - COORD - starting position
// - COORD - expected result (moving forwards)
// - COORD - expected result (moving backwards)
struct ExpectedResult
{
COORD moveForwards;
COORD moveBackwards;
};
struct Test
{
COORD startPos;
ExpectedResult expected;
};
// Set testData for GetWordStart tests
// clang-format off
std::vector<Test> testData = {
// tests for first line of text
{ { 0, 0 }, {{ 5, 0 }, { 0, 0 }} },
{ { 1, 0 }, {{ 5, 0 }, { 1, 0 }} },
{ { 3, 0 }, {{ 5, 0 }, { 3, 0 }} },
{ { 4, 0 }, {{ 5, 0 }, { 4, 0 }} },
{ { 5, 0 }, {{ 2, 1 }, { 0, 0 }} },
{ { 6, 0 }, {{ 2, 1 }, { 0, 0 }} },
{ { 20, 0 }, {{ 2, 1 }, { 0, 0 }} },
{ { 79, 0 }, {{ 2, 1 }, { 0, 0 }} },
// tests for second line of text
{ { 0, 1 }, {{ 2, 1 }, { 0, 0 }} },
{ { 1, 1 }, {{ 2, 1 }, { 0, 0 }} },
{ { 2, 1 }, {{ 9, 1 }, { 5, 0 }} },
{ { 3, 1 }, {{ 9, 1 }, { 5, 0 }} },
{ { 5, 1 }, {{ 9, 1 }, { 5, 0 }} },
{ { 6, 1 }, {{ 9, 1 }, { 5, 0 }} },
{ { 7, 1 }, {{ 9, 1 }, { 5, 0 }} },
{ { 9, 1 }, {{ 9, 1 }, { 2, 1 }} },
{ { 10, 1 }, {{10, 1 }, { 2, 1 }} },
{ { 20, 1 }, {{20, 1 }, { 2, 1 }} },
{ { 79, 1 }, {{79, 1 }, { 2, 1 }} },
};
// clang-format on
BEGIN_TEST_METHOD_PROPERTIES()
TEST_METHOD_PROPERTY(L"Data:movingForwards", L"{false, true}")
END_TEST_METHOD_PROPERTIES();
bool movingForwards;
VERIFY_SUCCEEDED(TestData::TryGetValue(L"movingForwards", movingForwards), L"Get movingForwards variant");
const std::wstring_view delimiters = L" ";
const COORD lastCharPos = _buffer->GetLastNonSpaceCharacter();
for (const auto& test : testData)
{
Log::Comment(NoThrowString().Format(L"COORD (%hd, %hd)", test.startPos.X, test.startPos.Y));
auto pos{ test.startPos };
const auto result = movingForwards ?
_buffer->MoveToNextWord(pos, delimiters, lastCharPos) :
_buffer->MoveToPreviousWord(pos, delimiters);
const auto expected = movingForwards ? test.expected.moveForwards : test.expected.moveBackwards;
VERIFY_ARE_EQUAL(expected, pos);
// if we moved, result is true and pos != startPos.
// otherwise, result is false and pos == startPos.
VERIFY_ARE_EQUAL(result, pos != test.startPos);
}
}
void TextBufferTests::GetGlyphBoundaries()
{
struct ExpectedResult
{
std::wstring name;
til::point start;
til::point wideGlyphEnd;
til::point normalEnd;
};
// clang-format off
const std::vector<ExpectedResult> expected = {
{ L"Buffer Start", { 0, 0 }, { 2, 0 }, { 1, 0 } },
{ L"Line Start", { 0, 1 }, { 2, 1 }, { 1, 1 } },
{ L"General Case", { 1, 1 }, { 3, 1 }, { 2, 1 } },
{ L"Line End", { 9, 1 }, { 0, 2 }, { 0, 2 } },
{ L"Buffer End", { 9, 9 }, { 0, 10 }, { 0, 10 } },
};
// clang-format on
BEGIN_TEST_METHOD_PROPERTIES()
TEST_METHOD_PROPERTY(L"Data:wideGlyph", L"{false, true}")
END_TEST_METHOD_PROPERTIES();
bool wideGlyph;
VERIFY_SUCCEEDED(TestData::TryGetValue(L"wideGlyph", wideGlyph), L"Get wide glyph variant");
COORD bufferSize{ 10, 10 };
UINT cursorSize = 12;
TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// This is the burrito emoji: 🌯
// It's encoded in UTF-16, as needed by the buffer.
const auto burrito = L"\xD83C\xDF2F";
const wchar_t* const output = wideGlyph ? burrito : L"X";
const OutputCellIterator iter{ output };
for (const auto& test : expected)
{
Log::Comment(test.name.c_str());
auto target = test.start;
_buffer->Write(iter, target);
auto start = _buffer->GetGlyphStart(target);
auto end = _buffer->GetGlyphEnd(target, true);
VERIFY_ARE_EQUAL(test.start, start);
VERIFY_ARE_EQUAL(wideGlyph ? test.wideGlyphEnd : test.normalEnd, end);
}
}
Move rect expansion to textbuffer; refactor selection code (#4560) - When performing chunk selection, the expansion now occurs at the time of the selection, not the rendering of the selection - `GetSelectionRects()` was moved to the `TextBuffer` and is now shared between ConHost and Windows Terminal - Some of the selection variables were renamed for clarity - Selection COORDs are now in the Text Buffer coordinate space - Fixes an issue with Shift+Click after performing a Multi-Click Selection ## References This also contributes to... - #4509: UIA Box Selection - #2447: UIA Signaling for Selection - #1354: UIA support for Wide Glyphs Now that the expansion occurs at before render-time, the selection anchors are an accurate representation of what is selected. We just need to move `GetText` to the `TextBuffer`. Then we can have those three issues just rely on code from the text buffer. This also means ConHost gets some of this stuff for free 😀 ### TextBuffer - `GetTextRects` is the abstracted form of `GetSelectionRects` - `_ExpandTextRow` is still needed to handle wide glyphs properly ### Terminal - Rename... - `_boxSelection` --> `_blockSelection` for consistency with ConHost - `_selectionAnchor` --> `_selectionStart` for consistency with UIA - `_endSelectionPosition` --> `_selectionEnd` for consistency with UIA - Selection anchors are in Text Buffer coordinates now - Really rely on `SetSelectionEnd` to accomplish appropriate chunk selection and shift+click actions ## Validation Steps Performed - Shift+Click - Multi-Click --> Shift+Click - Chunk Selection at... - top of buffer - bottom of buffer - random region in scrollback Closes #4465 Closes #4547
2020-02-28 01:42:26 +01:00
void TextBufferTests::GetTextRects()
{
// GetTextRects() is used to...
// - Represent selection rects
// - Represent UiaTextRanges for accessibility
// This is the burrito emoji: 🌯
// It's encoded in UTF-16, as needed by the buffer.
const auto burrito = std::wstring(L"\xD83C\xDF2F");
COORD bufferSize{ 20, 50 };
UINT cursorSize = 12;
TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// Setup: Write lines of text to the buffer
const std::vector<std::wstring> text = { L"0123456789",
L" " + burrito + L"3456" + burrito,
L" " + burrito + L"45" + burrito,
burrito + L"234567" + burrito,
L"0123456789" };
WriteLinesToBuffer(text, *_buffer);
// - - - Text Buffer Contents - - -
// |0123456789
// | 🌯3456🌯
// | 🌯45🌯
// |🌯234567🌯
// |0123456789
// - - - - - - - - - - - - - - - -
BEGIN_TEST_METHOD_PROPERTIES()
TEST_METHOD_PROPERTY(L"Data:blockSelection", L"{false, true}")
END_TEST_METHOD_PROPERTIES();
bool blockSelection;
VERIFY_SUCCEEDED(TestData::TryGetValue(L"blockSelection", blockSelection), L"Get 'blockSelection' variant");
std::vector<SMALL_RECT> expected{};
if (blockSelection)
{
expected.push_back({ 1, 0, 7, 0 });
expected.push_back({ 1, 1, 8, 1 }); // expand right
expected.push_back({ 1, 2, 7, 2 });
expected.push_back({ 0, 3, 7, 3 }); // expand left
expected.push_back({ 1, 4, 7, 4 });
}
else
{
expected.push_back({ 1, 0, 19, 0 });
expected.push_back({ 0, 1, 19, 1 });
expected.push_back({ 0, 2, 19, 2 });
expected.push_back({ 0, 3, 19, 3 });
expected.push_back({ 0, 4, 7, 4 });
}
COORD start{ 1, 0 };
COORD end{ 7, 4 };
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
const auto result = _buffer->GetTextRects(start, end, blockSelection, false);
Move rect expansion to textbuffer; refactor selection code (#4560) - When performing chunk selection, the expansion now occurs at the time of the selection, not the rendering of the selection - `GetSelectionRects()` was moved to the `TextBuffer` and is now shared between ConHost and Windows Terminal - Some of the selection variables were renamed for clarity - Selection COORDs are now in the Text Buffer coordinate space - Fixes an issue with Shift+Click after performing a Multi-Click Selection ## References This also contributes to... - #4509: UIA Box Selection - #2447: UIA Signaling for Selection - #1354: UIA support for Wide Glyphs Now that the expansion occurs at before render-time, the selection anchors are an accurate representation of what is selected. We just need to move `GetText` to the `TextBuffer`. Then we can have those three issues just rely on code from the text buffer. This also means ConHost gets some of this stuff for free 😀 ### TextBuffer - `GetTextRects` is the abstracted form of `GetSelectionRects` - `_ExpandTextRow` is still needed to handle wide glyphs properly ### Terminal - Rename... - `_boxSelection` --> `_blockSelection` for consistency with ConHost - `_selectionAnchor` --> `_selectionStart` for consistency with UIA - `_endSelectionPosition` --> `_selectionEnd` for consistency with UIA - Selection anchors are in Text Buffer coordinates now - Really rely on `SetSelectionEnd` to accomplish appropriate chunk selection and shift+click actions ## Validation Steps Performed - Shift+Click - Multi-Click --> Shift+Click - Chunk Selection at... - top of buffer - bottom of buffer - random region in scrollback Closes #4465 Closes #4547
2020-02-28 01:42:26 +01:00
VERIFY_ARE_EQUAL(expected.size(), result.size());
for (size_t i = 0; i < expected.size(); ++i)
{
VERIFY_ARE_EQUAL(expected.at(i), result.at(i));
}
}
void TextBufferTests::GetText()
{
// GetText() is used by...
// - Copying text to the clipboard regularly
// - Copying text to the clipboard, with shift held (collapse to one line)
// - Extracting text from a UiaTextRange
BEGIN_TEST_METHOD_PROPERTIES()
TEST_METHOD_PROPERTY(L"Data:wrappedText", L"{false, true}")
TEST_METHOD_PROPERTY(L"Data:blockSelection", L"{false, true}")
TEST_METHOD_PROPERTY(L"Data:includeCRLF", L"{false, true}")
TEST_METHOD_PROPERTY(L"Data:trimTrailingWhitespace", L"{false, true}")
END_TEST_METHOD_PROPERTIES();
bool wrappedText;
bool blockSelection;
bool includeCRLF;
bool trimTrailingWhitespace;
VERIFY_SUCCEEDED(TestData::TryGetValue(L"wrappedText", wrappedText), L"Get 'wrappedText' variant");
VERIFY_SUCCEEDED(TestData::TryGetValue(L"blockSelection", blockSelection), L"Get 'blockSelection' variant");
VERIFY_SUCCEEDED(TestData::TryGetValue(L"includeCRLF", includeCRLF), L"Get 'includeCRLF' variant");
VERIFY_SUCCEEDED(TestData::TryGetValue(L"trimTrailingWhitespace", trimTrailingWhitespace), L"Get 'trimTrailingWhitespace' variant");
if (!wrappedText)
{
COORD bufferSize{ 10, 20 };
UINT cursorSize = 12;
TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// Setup: Write lines of text to the buffer
const std::vector<std::wstring> bufferText = { L"12345",
L" 345",
L"123 ",
L" 3 " };
WriteLinesToBuffer(bufferText, *_buffer);
// simulate a selection from origin to {4,4}
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
const auto textRects = _buffer->GetTextRects({ 0, 0 }, { 4, 4 }, blockSelection, false);
std::wstring result = L"";
const auto textData = _buffer->GetText(includeCRLF, trimTrailingWhitespace, textRects).text;
for (auto& text : textData)
{
result += text;
}
std::wstring expectedText = L"";
if (includeCRLF)
{
if (trimTrailingWhitespace)
{
Log::Comment(L"Standard Copy to Clipboard");
expectedText += L"12345\r\n";
expectedText += L" 345\r\n";
expectedText += L"123\r\n";
expectedText += L" 3\r\n";
}
else
{
Log::Comment(L"UI Automation");
if (blockSelection)
{
expectedText += L"12345\r\n";
expectedText += L" 345\r\n";
expectedText += L"123 \r\n";
expectedText += L" 3 \r\n";
expectedText += L" ";
}
else
{
expectedText += L"12345 \r\n";
expectedText += L" 345 \r\n";
expectedText += L"123 \r\n";
expectedText += L" 3 \r\n";
expectedText += L" ";
}
}
}
else
{
if (trimTrailingWhitespace)
{
Log::Comment(L"UNDEFINED");
expectedText += L"12345";
expectedText += L" 345";
expectedText += L"123";
expectedText += L" 3";
}
else
{
Log::Comment(L"Shift+Copy to Clipboard");
if (blockSelection)
{
expectedText += L"12345";
expectedText += L" 345";
expectedText += L"123 ";
expectedText += L" 3 ";
expectedText += L" ";
}
else
{
expectedText += L"12345 ";
expectedText += L" 345 ";
expectedText += L"123 ";
expectedText += L" 3 ";
expectedText += L" ";
}
}
}
// Verify expected output and actual output are the same
VERIFY_ARE_EQUAL(expectedText, result);
}
else
{
// Case 2: Wrapped Text
COORD bufferSize{ 5, 20 };
UINT cursorSize = 12;
TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
// Setup: Write lines of text to the buffer
const std::vector<std::wstring> bufferText = { L"1234567",
L"",
L" 345",
L"123 ",
L"" };
WriteLinesToBuffer(bufferText, *_buffer);
// buffer should look like this:
// ______
// |12345| <-- wrapped
// |67 |
// | 345|
// |123 | <-- wrapped
// | |
// |_____|
// simulate a selection from origin to {4,5}
Add support for double-width/double-height lines in conhost (#8664) This PR adds support for the VT line rendition attributes, which allow for double-width and double-height line renditions. These renditions are enabled with the `DECDWL` (double-width line) and `DECDHL` (double-height line) escape sequences. Both reset to the default rendition with the `DECSWL` (single-width line) escape sequence. For now this functionality is only supported by the GDI renderer in conhost. There are a lot of changes, so this is just a general overview of the main areas affected. Previously it was safe to assume that the screen had a fixed width, at least for a given point in time. But now we need to deal with the possibility of different lines have different widths, so all the functions that are constrained by the right border (text wrapping, cursor movement operations, and sequences like `EL` and `ICH`) now need to lookup the width of the active line in order to behave correctly. Similarly it used to be safe to assume that buffer and screen coordinates were the same thing, but that is no longer true. Lots of places now need to translate back and forth between coordinate systems dependent on the line rendition. This includes clipboard handling, the conhost color selection and search, accessibility location tracking and screen reading, IME editor positioning, "snapping" the viewport, and of course all the rendering calculations. For the rendering itself, I've had to introduce a new `PrepareLineTransform` method that the render engines can use to setup the necessary transform matrix for a given line rendition. This is also now used to handle the horizontal viewport offset, since that could no longer be achieved just by changing the target coordinates (on a double width line, the viewport offset may be halfway through a character). I've also had to change the renderer's existing `InvalidateCursor` method to take a `SMALL_RECT` rather than a `COORD`, to allow for the cursor being a variable width. Technically this was already a problem, because the cursor could occupy two screen cells when over a double-width character, but now it can be anything between one and four screen cells (e.g. a double-width character on the double-width line). In terms of architectural changes, there is now a new `lineRendition` field in the `ROW` class that keeps track of the line rendition for each row, and several new methods in the `ROW` and `TextBuffer` classes for manipulating that state. This includes a few helper methods for handling the various issues discussed above, e.g. position clamping and translating between coordinate systems. ## Validation Steps Performed I've manually confirmed all the double-width and double-height tests in _Vttest_ are now working as expected, and the _VT100 Torture Test_ now renders correctly (at least the line rendition aspects). I've also got my own test scripts that check many of the line rendition boundary cases and have confirmed that those are now passing. I've manually tested as many areas of the conhost UI that I could think of, that might be affected by line rendition, including things like searching, selection, copying, and color highlighting. For accessibility, I've confirmed that the _Magnifier_ and _Narrator_ correctly handle double-width lines. And I've also tested the Japanese IME, which while not perfect, is at least useable. Closes #7865
2021-02-18 06:44:50 +01:00
const auto textRects = _buffer->GetTextRects({ 0, 0 }, { 4, 5 }, blockSelection, false);
std::wstring result = L"";
const auto formatWrappedRows = blockSelection;
const auto textData = _buffer->GetText(includeCRLF, trimTrailingWhitespace, textRects, nullptr, formatWrappedRows).text;
for (auto& text : textData)
{
result += text;
}
std::wstring expectedText = L"";
if (formatWrappedRows)
{
if (includeCRLF)
{
if (trimTrailingWhitespace)
{
Log::Comment(L"UNDEFINED");
expectedText += L"12345\r\n";
expectedText += L"67\r\n";
expectedText += L" 345\r\n";
expectedText += L"123\r\n";
expectedText += L"\r\n";
}
else
{
Log::Comment(L"Copy block selection to Clipboard");
expectedText += L"12345\r\n";
expectedText += L"67 \r\n";
expectedText += L" 345\r\n";
expectedText += L"123 \r\n";
expectedText += L" \r\n";
expectedText += L" ";
}
}
else
{
if (trimTrailingWhitespace)
{
Log::Comment(L"UNDEFINED");
expectedText += L"12345";
expectedText += L"67";
expectedText += L" 345";
expectedText += L"123";
}
else
{
Log::Comment(L"UNDEFINED");
expectedText += L"12345";
expectedText += L"67 ";
expectedText += L" 345";
expectedText += L"123 ";
expectedText += L" ";
expectedText += L" ";
}
}
}
else
{
if (includeCRLF)
{
if (trimTrailingWhitespace)
{
Log::Comment(L"Standard Copy to Clipboard");
expectedText += L"12345";
expectedText += L"67\r\n";
expectedText += L" 345\r\n";
expectedText += L"123 \r\n";
}
else
{
Log::Comment(L"UI Automation");
expectedText += L"12345";
expectedText += L"67 \r\n";
expectedText += L" 345\r\n";
expectedText += L"123 ";
expectedText += L" \r\n";
expectedText += L" ";
}
}
else
{
if (trimTrailingWhitespace)
{
Log::Comment(L"UNDEFINED");
expectedText += L"12345";
expectedText += L"67";
expectedText += L" 345";
expectedText += L"123 ";
}
else
{
Log::Comment(L"Shift+Copy to Clipboard");
expectedText += L"12345";
expectedText += L"67 ";
expectedText += L" 345";
expectedText += L"123 ";
expectedText += L" ";
expectedText += L" ";
}
}
}
// Verify expected output and actual output are the same
VERIFY_ARE_EQUAL(expectedText, result);
}
}
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
// This tests that when we increment the circular buffer, obsolete hyperlink references
// are removed from the hyperlink map
void TextBufferTests::HyperlinkTrim()
{
// Set up a text buffer for us
const COORD bufferSize{ 80, 10 };
const UINT cursorSize = 12;
const TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
const auto url = L"test.url";
const auto otherUrl = L"other.url";
const auto customId = L"CustomId";
const auto otherCustomId = L"OtherCustomId";
// Set a hyperlink id in the first row and add a hyperlink to our map
const COORD pos{ 70, 0 };
const auto id = _buffer->GetHyperlinkId(url, customId);
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
TextAttribute newAttr{ 0x7f };
newAttr.SetHyperlinkId(id);
_buffer->GetRowByOffset(pos.Y).GetAttrRow().SetAttrToEnd(pos.X, newAttr);
_buffer->AddHyperlinkToMap(url, id);
// Set a different hyperlink id somewhere else in the buffer
const COORD otherPos{ 70, 5 };
const auto otherId = _buffer->GetHyperlinkId(otherUrl, otherCustomId);
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
newAttr.SetHyperlinkId(otherId);
_buffer->GetRowByOffset(otherPos.Y).GetAttrRow().SetAttrToEnd(otherPos.X, newAttr);
_buffer->AddHyperlinkToMap(otherUrl, otherId);
// Increment the circular buffer
_buffer->IncrementCircularBuffer();
const auto finalCustomId = fmt::format(L"{}%{}", customId, std::hash<std::wstring_view>{}(url));
const auto finalOtherCustomId = fmt::format(L"{}%{}", otherCustomId, std::hash<std::wstring_view>{}(otherUrl));
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
// The hyperlink reference that was only in the first row should be deleted from the map
VERIFY_ARE_EQUAL(_buffer->_hyperlinkMap.find(id), _buffer->_hyperlinkMap.end());
// Since there was a custom id, that should be deleted as well
VERIFY_ARE_EQUAL(_buffer->_hyperlinkCustomIdMap.find(finalCustomId), _buffer->_hyperlinkCustomIdMap.end());
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
// The other hyperlink reference should not be deleted
VERIFY_ARE_EQUAL(_buffer->_hyperlinkMap[otherId], otherUrl);
VERIFY_ARE_EQUAL(_buffer->_hyperlinkCustomIdMap[finalOtherCustomId], otherId);
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
}
// This tests that when we increment the circular buffer, non-obsolete hyperlink references
// do not get removed from the hyperlink map
void TextBufferTests::NoHyperlinkTrim()
{
// Set up a text buffer for us
const COORD bufferSize{ 80, 10 };
const UINT cursorSize = 12;
const TextAttribute attr{ 0x7f };
auto _buffer = std::make_unique<TextBuffer>(bufferSize, attr, cursorSize, _renderTarget);
const auto url = L"test.url";
const auto customId = L"CustomId";
// Set a hyperlink id in the first row and add a hyperlink to our map
const COORD pos{ 70, 0 };
const auto id = _buffer->GetHyperlinkId(url, customId);
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
TextAttribute newAttr{ 0x7f };
newAttr.SetHyperlinkId(id);
_buffer->GetRowByOffset(pos.Y).GetAttrRow().SetAttrToEnd(pos.X, newAttr);
_buffer->AddHyperlinkToMap(url, id);
// Set the same hyperlink id somewhere else in the buffer
const COORD otherPos{ 70, 5 };
_buffer->GetRowByOffset(otherPos.Y).GetAttrRow().SetAttrToEnd(otherPos.X, newAttr);
// Increment the circular buffer
_buffer->IncrementCircularBuffer();
const auto finalCustomId = fmt::format(L"{}%{}", customId, std::hash<std::wstring_view>{}(url));
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
// The hyperlink reference should not be deleted from the map since it is still present in the buffer
VERIFY_ARE_EQUAL(_buffer->GetHyperlinkUriFromId(id), url);
VERIFY_ARE_EQUAL(_buffer->_hyperlinkCustomIdMap[finalCustomId], id);
OSC 8 support for conhost and terminal (#7251) <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request Conhost can now support OSC8 sequences (as specified [here](https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda)). Terminal also supports those sequences and additionally hyperlinks can be opened by Ctrl+LeftClicking on them. <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References #204 <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #204 * [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments Added support to: - parse OSC8 sequences and extract URIs from them (conhost and terminal) - add hyperlink uri data to textbuffer/screeninformation, associated with a hyperlink id (conhost and terminal) - attach hyperlink ids to text to allow for uri extraction from the textbuffer/screeninformation (conhost and terminal) - process ctrl+leftclick to open a hyperlink in the clicked region if present <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed Open up a PowerShell tab and type ```PowerShell ${ESC}=[char]27 Write-Host "${ESC}]8;;https://github.com/microsoft/terminal${ESC}\This is a link!${ESC}]8;;${ESC}\" ``` Ctrl+LeftClick on the link correctly brings you to the terminal page on github ![hyperlink](https://user-images.githubusercontent.com/26824113/89953536-45a6f580-dbfd-11ea-8e0d-8a3cd25c634a.gif)
2020-09-03 19:52:39 +02:00
}