terminal/src/cascadia/UnitTests_TerminalCore/TerminalApiTest.cpp
James Holderness 6742965bb8
Disable the acceptance of C1 control codes by default (#11690)
There are some code pages with "unmapped" code points in the C1 range,
which results in them being translated into Unicode C1 control codes,
even though that is not their intended use. To avoid having these
characters triggering unintentional escape sequences, this PR now
disables C1 controls by default.

Switching to ISO-2022 encoding will re-enable them, though, since that
is the most likely scenario in which they would be required. They can
also be explicitly enabled, even in UTF-8 mode, with the `DECAC1` escape
sequence.

What I've done is add a new mode to the `StateMachine` class that
controls whether C1 code points are interpreted as control characters or
not. When disabled, these code points are simply dropped from the
output, similar to the way a `NUL` is interpreted.

This isn't exactly the way they were handled in the v1 console (which I
think replaces them with the font _notdef_ glyph), but it matches the
XTerm behavior, which seems more appropriate considering this is in VT
mode. And it's worth noting that Windows Explorer seems to work the same
way.

As mentioned above, the mode can be enabled by designating the ISO-2022
coding system with a `DOCS` sequence, and it will be disabled again when
UTF-8 is designated. You can also enable it explicitly with a `DECAC1`
sequence (originally this was actually a DEC printer sequence, but it
doesn't seem unreasonable to use it in a terminal).

I've also extended the operations that save and restore "cursor state"
(e.g. `DECSC` and `DECRC`) to include the state of the C1 parser mode,
since it's closely tied to the code page and character sets which are
also saved there. Similarly, when a `DECSTR` sequence resets the code
page and character sets, I've now made it reset the C1 mode as well.

I should note that the new `StateMachine` mode is controlled via a
generic `SetParserMode` method (with a matching API in the `ConGetSet`
interface) to allow for easier addition of other modes in the future.
And I've reimplemented the existing ANSI/VT52 mode in terms of these
generic methods instead of it having to have its own separate APIs.

## Validation Steps Performed

Some of the unit tests for OSC sequences were using a C1 `0x9C` for the
string terminator, which doesn't work by default anymore. Since that's
not a good practice anyway, I thought it best to change those to a
standard 7-bit terminator. However, in tests that were explicitly
validating the C1 controls, I've just enabled the C1 parser mode at the
start of the tests in order to get them working again.

There were also some ANSI mode adapter tests that had to be updated to
account for the fact that it has now been reimplemented in terms of the
`SetParserMode` API.

I've added a new state machine test to validate the changes in behavior
when the C1 parser mode is enabled or disabled. And I've added an
adapter test to verify that the `DesignateCodingSystems` and
`AcceptC1Controls` methods toggle the C1 parser mode as expected.

I've manually verified the test cases in #10069 and #10310 to confirm
that they're no longer triggering control sequences by default.
Although, as I explained above, the C1 code points are completely
dropped from the output rather than displayed as _notdef_ glyphs. I
think this is a reasonable compromise though.

Closes #10069
Closes #10310
2021-11-17 23:40:31 +00:00

472 lines
18 KiB
C++

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "pch.h"
#include <WexTestClass.h>
#include "../cascadia/TerminalCore/Terminal.hpp"
#include "MockTermSettings.h"
#include "../renderer/inc/DummyRenderTarget.hpp"
#include "consoletaeftemplates.hpp"
using namespace winrt::Microsoft::Terminal::Core;
using namespace Microsoft::Terminal::Core;
using namespace WEX::Logging;
using namespace WEX::TestExecution;
namespace TerminalCoreUnitTests
{
#define WCS(x) WCSHELPER(x)
#define WCSHELPER(x) L## #x
class TerminalApiTest
{
TEST_CLASS(TerminalApiTest);
TEST_METHOD(SetColorTableEntry);
TEST_METHOD(CursorVisibility);
TEST_METHOD(CursorVisibilityViaStateMachine);
// Terminal::_WriteBuffer used to enter infinite loops under certain conditions.
// This test ensures that Terminal::_WriteBuffer doesn't get stuck when
// PrintString() is called with more code units than the buffer width.
TEST_METHOD(PrintStringOfSurrogatePairs);
TEST_METHOD(CheckDoubleWidthCursor);
TEST_METHOD(AddHyperlink);
TEST_METHOD(AddHyperlinkCustomId);
TEST_METHOD(AddHyperlinkCustomIdDifferentUri);
TEST_METHOD(SetTaskbarProgress);
TEST_METHOD(SetWorkingDirectory);
};
};
using namespace TerminalCoreUnitTests;
void TerminalApiTest::SetColorTableEntry()
{
Terminal term;
DummyRenderTarget emptyRT;
term.Create({ 100, 100 }, 0, emptyRT);
auto settings = winrt::make<MockTermSettings>(100, 100, 100);
term.UpdateSettings(settings);
VERIFY_IS_TRUE(term.SetColorTableEntry(0, 100));
VERIFY_IS_TRUE(term.SetColorTableEntry(128, 100));
VERIFY_IS_TRUE(term.SetColorTableEntry(255, 100));
VERIFY_IS_FALSE(term.SetColorTableEntry(256, 100));
VERIFY_IS_FALSE(term.SetColorTableEntry(512, 100));
}
// Terminal::_WriteBuffer used to enter infinite loops under certain conditions.
// This test ensures that Terminal::_WriteBuffer doesn't get stuck when
// PrintString() is called with more code units than the buffer width.
void TerminalApiTest::PrintStringOfSurrogatePairs()
{
DummyRenderTarget renderTarget;
Terminal term;
term.Create({ 100, 100 }, 3, renderTarget);
std::wstring text;
text.reserve(600);
for (size_t i = 0; i < 100; ++i)
{
text.append(L"𐐌𐐜𐐬");
}
struct Baton
{
HANDLE done;
std::wstring text;
Terminal* pTerm;
} baton{
CreateEventW(nullptr, TRUE, FALSE, L"done signal"),
text,
&term,
};
Log::Comment(L"Launching thread to write data.");
const auto thread = CreateThread(
nullptr,
0,
[](LPVOID data) -> DWORD {
const Baton& baton = *reinterpret_cast<Baton*>(data);
Log::Comment(L"Writing data.");
baton.pTerm->PrintString(baton.text);
Log::Comment(L"Setting event.");
SetEvent(baton.done);
return 0;
},
(LPVOID)&baton,
0,
nullptr);
Log::Comment(L"Waiting for the write.");
switch (WaitForSingleObject(baton.done, 2000))
{
case WAIT_OBJECT_0:
Log::Comment(L"Didn't get stuck. Success.");
break;
case WAIT_TIMEOUT:
Log::Comment(L"Wait timed out. It got stuck.");
Log::Result(WEX::Logging::TestResults::Failed);
break;
case WAIT_FAILED:
Log::Comment(L"Wait failed for some reason. We didn't expect this.");
Log::Result(WEX::Logging::TestResults::Failed);
break;
default:
Log::Comment(L"Wait return code that no one expected. Fail.");
Log::Result(WEX::Logging::TestResults::Failed);
break;
}
TerminateThread(thread, 0);
CloseHandle(baton.done);
return;
}
void TerminalApiTest::CursorVisibility()
{
// GH#3093 - Cursor Visibility and On states shouldn't affect each other
Terminal term;
DummyRenderTarget emptyRT;
term.Create({ 100, 100 }, 0, emptyRT);
VERIFY_IS_TRUE(term._buffer->GetCursor().IsVisible());
VERIFY_IS_TRUE(term._buffer->GetCursor().IsOn());
VERIFY_IS_TRUE(term._buffer->GetCursor().IsBlinkingAllowed());
term.SetCursorOn(false);
VERIFY_IS_TRUE(term._buffer->GetCursor().IsVisible());
VERIFY_IS_FALSE(term._buffer->GetCursor().IsOn());
VERIFY_IS_TRUE(term._buffer->GetCursor().IsBlinkingAllowed());
term.SetCursorOn(true);
VERIFY_IS_TRUE(term._buffer->GetCursor().IsVisible());
VERIFY_IS_TRUE(term._buffer->GetCursor().IsOn());
VERIFY_IS_TRUE(term._buffer->GetCursor().IsBlinkingAllowed());
term.SetCursorVisibility(false);
VERIFY_IS_FALSE(term._buffer->GetCursor().IsVisible());
VERIFY_IS_TRUE(term._buffer->GetCursor().IsOn());
VERIFY_IS_TRUE(term._buffer->GetCursor().IsBlinkingAllowed());
term.SetCursorOn(false);
VERIFY_IS_FALSE(term._buffer->GetCursor().IsVisible());
VERIFY_IS_FALSE(term._buffer->GetCursor().IsOn());
VERIFY_IS_TRUE(term._buffer->GetCursor().IsBlinkingAllowed());
}
void TerminalApiTest::CursorVisibilityViaStateMachine()
{
// This is a nearly literal copy-paste of ScreenBufferTests::TestCursorIsOn, adapted for the Terminal
Terminal term;
DummyRenderTarget emptyRT;
term.Create({ 100, 100 }, 0, emptyRT);
auto& tbi = *(term._buffer);
auto& stateMachine = *(term._stateMachine);
auto& cursor = tbi.GetCursor();
stateMachine.ProcessString(L"Hello World");
VERIFY_IS_TRUE(cursor.IsOn());
VERIFY_IS_TRUE(cursor.IsBlinkingAllowed());
VERIFY_IS_TRUE(cursor.IsVisible());
stateMachine.ProcessString(L"\x1b[?12l");
VERIFY_IS_TRUE(cursor.IsOn());
VERIFY_IS_FALSE(cursor.IsBlinkingAllowed());
VERIFY_IS_TRUE(cursor.IsVisible());
stateMachine.ProcessString(L"\x1b[?12h");
VERIFY_IS_TRUE(cursor.IsOn());
VERIFY_IS_TRUE(cursor.IsBlinkingAllowed());
VERIFY_IS_TRUE(cursor.IsVisible());
cursor.SetIsOn(false);
stateMachine.ProcessString(L"\x1b[?12l");
VERIFY_IS_TRUE(cursor.IsOn());
VERIFY_IS_FALSE(cursor.IsBlinkingAllowed());
VERIFY_IS_TRUE(cursor.IsVisible());
stateMachine.ProcessString(L"\x1b[?12h");
VERIFY_IS_TRUE(cursor.IsOn());
VERIFY_IS_TRUE(cursor.IsBlinkingAllowed());
VERIFY_IS_TRUE(cursor.IsVisible());
stateMachine.ProcessString(L"\x1b[?25l");
VERIFY_IS_TRUE(cursor.IsOn());
VERIFY_IS_TRUE(cursor.IsBlinkingAllowed());
VERIFY_IS_FALSE(cursor.IsVisible());
stateMachine.ProcessString(L"\x1b[?25h");
VERIFY_IS_TRUE(cursor.IsOn());
VERIFY_IS_TRUE(cursor.IsBlinkingAllowed());
VERIFY_IS_TRUE(cursor.IsVisible());
stateMachine.ProcessString(L"\x1b[?12;25l");
VERIFY_IS_TRUE(cursor.IsOn());
VERIFY_IS_FALSE(cursor.IsBlinkingAllowed());
VERIFY_IS_FALSE(cursor.IsVisible());
}
void TerminalApiTest::CheckDoubleWidthCursor()
{
DummyRenderTarget renderTarget;
Terminal term;
term.Create({ 100, 100 }, 0, renderTarget);
auto& tbi = *(term._buffer);
auto& stateMachine = *(term._stateMachine);
auto& cursor = tbi.GetCursor();
// Lets stuff the buffer with single width characters,
// but leave the last two columns empty for double width.
std::wstring singleWidthText;
singleWidthText.reserve(98);
for (size_t i = 0; i < 98; ++i)
{
singleWidthText.append(L"A");
}
stateMachine.ProcessString(singleWidthText);
VERIFY_IS_TRUE(cursor.GetPosition().X == 98);
// Stuff two double width characters.
std::wstring doubleWidthText{ L"我愛" };
stateMachine.ProcessString(doubleWidthText);
// The last 'A'
term.SetCursorPosition(97, 0);
VERIFY_IS_FALSE(term.IsCursorDoubleWidth());
// This and the next CursorPos are taken up by '我‘
term.SetCursorPosition(98, 0);
VERIFY_IS_TRUE(term.IsCursorDoubleWidth());
term.SetCursorPosition(99, 0);
VERIFY_IS_TRUE(term.IsCursorDoubleWidth());
// This and the next CursorPos are taken up by ’愛‘
term.SetCursorPosition(0, 1);
VERIFY_IS_TRUE(term.IsCursorDoubleWidth());
term.SetCursorPosition(1, 1);
VERIFY_IS_TRUE(term.IsCursorDoubleWidth());
}
void TerminalCoreUnitTests::TerminalApiTest::AddHyperlink()
{
// This is a nearly literal copy-paste of ScreenBufferTests::TestAddHyperlink, adapted for the Terminal
Terminal term;
DummyRenderTarget emptyRT;
term.Create({ 100, 100 }, 0, emptyRT);
auto& tbi = *(term._buffer);
auto& stateMachine = *(term._stateMachine);
// Process the opening osc 8 sequence
stateMachine.ProcessString(L"\x1b]8;;test.url\x1b\\");
VERIFY_IS_TRUE(tbi.GetCurrentAttributes().IsHyperlink());
VERIFY_ARE_EQUAL(tbi.GetHyperlinkUriFromId(tbi.GetCurrentAttributes().GetHyperlinkId()), L"test.url");
// Send any other text
stateMachine.ProcessString(L"Hello World");
VERIFY_IS_TRUE(tbi.GetCurrentAttributes().IsHyperlink());
VERIFY_ARE_EQUAL(tbi.GetHyperlinkUriFromId(tbi.GetCurrentAttributes().GetHyperlinkId()), L"test.url");
// Process the closing osc 8 sequences
stateMachine.ProcessString(L"\x1b]8;;\x1b\\");
VERIFY_IS_FALSE(tbi.GetCurrentAttributes().IsHyperlink());
}
void TerminalCoreUnitTests::TerminalApiTest::AddHyperlinkCustomId()
{
// This is a nearly literal copy-paste of ScreenBufferTests::TestAddHyperlinkCustomId, adapted for the Terminal
Terminal term;
DummyRenderTarget emptyRT;
term.Create({ 100, 100 }, 0, emptyRT);
auto& tbi = *(term._buffer);
auto& stateMachine = *(term._stateMachine);
// Process the opening osc 8 sequence
stateMachine.ProcessString(L"\x1b]8;id=myId;test.url\x1b\\");
VERIFY_IS_TRUE(tbi.GetCurrentAttributes().IsHyperlink());
VERIFY_ARE_EQUAL(tbi.GetHyperlinkUriFromId(tbi.GetCurrentAttributes().GetHyperlinkId()), L"test.url");
VERIFY_ARE_EQUAL(tbi.GetHyperlinkId(L"test.url", L"myId"), tbi.GetCurrentAttributes().GetHyperlinkId());
// Send any other text
stateMachine.ProcessString(L"Hello World");
VERIFY_IS_TRUE(tbi.GetCurrentAttributes().IsHyperlink());
VERIFY_ARE_EQUAL(tbi.GetHyperlinkUriFromId(tbi.GetCurrentAttributes().GetHyperlinkId()), L"test.url");
VERIFY_ARE_EQUAL(tbi.GetHyperlinkId(L"test.url", L"myId"), tbi.GetCurrentAttributes().GetHyperlinkId());
// Process the closing osc 8 sequences
stateMachine.ProcessString(L"\x1b]8;;\x1b\\");
VERIFY_IS_FALSE(tbi.GetCurrentAttributes().IsHyperlink());
}
void TerminalCoreUnitTests::TerminalApiTest::AddHyperlinkCustomIdDifferentUri()
{
// This is a nearly literal copy-paste of ScreenBufferTests::TestAddHyperlinkCustomId, adapted for the Terminal
Terminal term;
DummyRenderTarget emptyRT;
term.Create({ 100, 100 }, 0, emptyRT);
auto& tbi = *(term._buffer);
auto& stateMachine = *(term._stateMachine);
// Process the opening osc 8 sequence
stateMachine.ProcessString(L"\x1b]8;id=myId;test.url\x1b\\");
VERIFY_IS_TRUE(tbi.GetCurrentAttributes().IsHyperlink());
VERIFY_ARE_EQUAL(tbi.GetHyperlinkUriFromId(tbi.GetCurrentAttributes().GetHyperlinkId()), L"test.url");
VERIFY_ARE_EQUAL(tbi.GetHyperlinkId(L"test.url", L"myId"), tbi.GetCurrentAttributes().GetHyperlinkId());
const auto oldAttributes{ tbi.GetCurrentAttributes() };
// Send any other text
stateMachine.ProcessString(L"\x1b]8;id=myId;other.url\x1b\\");
VERIFY_IS_TRUE(tbi.GetCurrentAttributes().IsHyperlink());
VERIFY_ARE_EQUAL(tbi.GetHyperlinkUriFromId(tbi.GetCurrentAttributes().GetHyperlinkId()), L"other.url");
VERIFY_ARE_EQUAL(tbi.GetHyperlinkId(L"other.url", L"myId"), tbi.GetCurrentAttributes().GetHyperlinkId());
// This second URL should not change the URL of the original ID!
VERIFY_ARE_EQUAL(tbi.GetHyperlinkUriFromId(oldAttributes.GetHyperlinkId()), L"test.url");
VERIFY_ARE_NOT_EQUAL(oldAttributes.GetHyperlinkId(), tbi.GetCurrentAttributes().GetHyperlinkId());
}
void TerminalCoreUnitTests::TerminalApiTest::SetTaskbarProgress()
{
Terminal term;
DummyRenderTarget emptyRT;
term.Create({ 100, 100 }, 0, emptyRT);
auto& stateMachine = *(term._stateMachine);
// Initial values for taskbar state and progress should be 0
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(0));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(0));
// Set some values for taskbar state and progress through state machine
stateMachine.ProcessString(L"\x1b]9;4;1;50\x1b\\");
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(1));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(50));
// Reset to 0
stateMachine.ProcessString(L"\x1b]9;4;0;0\x1b\\");
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(0));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(0));
// Set an out of bounds value for state
stateMachine.ProcessString(L"\x1b]9;4;5;50\x1b\\");
// Nothing should have changed (dispatch should have returned false)
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(0));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(0));
// Set an out of bounds value for progress
stateMachine.ProcessString(L"\x1b]9;4;1;999\x1b\\");
// Progress should have been clamped to 100
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(1));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(100));
// Don't specify any params
stateMachine.ProcessString(L"\x1b]9;4\x1b\\");
// State and progress should both be reset to 0
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(0));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(0));
// Specify additional params
stateMachine.ProcessString(L"\x1b]9;4;1;80;123\x1b\\");
// Additional params should be ignored, state and progress still set normally
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(1));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(80));
// Edge cases + trailing semicolon testing
stateMachine.ProcessString(L"\x1b]9;4;2;\x1b\\");
// String should be processed correctly despite the trailing semicolon,
// taskbar progress should remain unchanged from previous value
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(2));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(80));
stateMachine.ProcessString(L"\x1b]9;4;3;75\x1b\\");
// Given progress value should be ignored because this is the indeterminate state,
// so the progress value should remain unchanged
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(3));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(80));
stateMachine.ProcessString(L"\x1b]9;4;0;50\x1b\\");
// Taskbar progress should be 0 (the given value should be ignored)
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(0));
VERIFY_ARE_EQUAL(term.GetTaskbarProgress(), gsl::narrow<size_t>(0));
stateMachine.ProcessString(L"\x1b]9;4;2;\x1b\\");
// String should be processed correctly despite the trailing semicolon,
// taskbar progress should be set to a 'minimum', non-zero value
VERIFY_ARE_EQUAL(term.GetTaskbarState(), gsl::narrow<size_t>(2));
VERIFY_IS_GREATER_THAN(term.GetTaskbarProgress(), gsl::narrow<size_t>(0));
}
void TerminalCoreUnitTests::TerminalApiTest::SetWorkingDirectory()
{
Terminal term;
DummyRenderTarget emptyRT;
term.Create({ 100, 100 }, 0, emptyRT);
auto& stateMachine = *(term._stateMachine);
// Test setting working directory using OSC 9;9
// Initial CWD should be empty
VERIFY_IS_TRUE(term.GetWorkingDirectory().empty());
// Invalid sequences should not change CWD
stateMachine.ProcessString(L"\x1b]9;9\x1b\\");
VERIFY_IS_TRUE(term.GetWorkingDirectory().empty());
stateMachine.ProcessString(L"\x1b]9;9\"\x1b\\");
VERIFY_IS_TRUE(term.GetWorkingDirectory().empty());
stateMachine.ProcessString(L"\x1b]9;9\"C:\\\"\x1b\\");
VERIFY_IS_TRUE(term.GetWorkingDirectory().empty());
// Valid sequences should change CWD
stateMachine.ProcessString(L"\x1b]9;9;\"C:\\\"\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"C:\\");
stateMachine.ProcessString(L"\x1b]9;9;\"C:\\Program Files\"\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"C:\\Program Files");
stateMachine.ProcessString(L"\x1b]9;9;\"D:\\中文\"\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"D:\\中文");
// Test OSC 9;9 sequences without quotation marks
stateMachine.ProcessString(L"\x1b]9;9;C:\\\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"C:\\");
stateMachine.ProcessString(L"\x1b]9;9;C:\\Program Files\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"C:\\Program Files");
stateMachine.ProcessString(L"\x1b]9;9;D:\\中文\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"D:\\中文");
// These OSC 9;9 sequences will result in invalid CWD. We shouldn't crash on these.
stateMachine.ProcessString(L"\x1b]9;9;\"\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"\"");
stateMachine.ProcessString(L"\x1b]9;9;\"\"\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"\"\"");
stateMachine.ProcessString(L"\x1b]9;9;\"\"\"\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"\"");
stateMachine.ProcessString(L"\x1b]9;9;\"\"\"\"\x1b\\");
VERIFY_ARE_EQUAL(term.GetWorkingDirectory(), L"\"\"");
}