terminal/src/server/ConsoleShimPolicy.cpp
Dustin L. Howett 1bf4c082b4
Reintroduce a color compatibility hack, but only for PowerShells (#6810)
There is going to be a very long tail of applications that will
explicitly request VT SGR 40/37 when what they really want is to
SetConsoleTextAttribute() with a black background/white foreground.
Instead of making those applications look bad (and therefore making us
look bad, because we're releasing this as an update to something that
"looks good" already), we're introducing this compatibility quirk.
Before the color reckoning in #6698 + #6506, *every* color was subject
to being spontaneously and erroneously turned into the default color.
Now, only the 16-color palette value that matches the active console
background/foreground color will be destroyed, and only when received
from specific applications.

Removal will be tracked by #6807.

Michael and I discussed what layer this quirk really belonged in. I
originally believed it would be sufficient to detect a background color
that matched the legacy default background, but @j4james provided an
example of where that wouldn't work out (powershell setting the
foreground color to white/gray). In addition, it was too heavyhanded: it
re-broke black backgrounds for every application.

Michael thought that it should live in the server, as a small VT parser
that righted the wrongs coming directly out of the application. On
further investigation, however, I realized that we'd need to push more
information up into the server (so that it could make the decision about
which VT was wrong and which was right) than should be strictly
necessary.

The host knows which colors are right and wrong, and it gets final say
in what ends up in the buffer.

Because of that, I chose to push the quirk state down through
WriteConsole to DoWriteConsole and toggle state on the
SCREEN_INFORMATION that indicates whether the colors coming out of the
application are to be distrusted. This quirk _only applies to pwsh.exe
and powershell.exe._

NOTE: This doesn't work for PowerShell the .NET Global tool, because it
is run as an assembly through dotnet.exe. I have no opinion on how to
fix this, or whether it is worth fixing.

VALIDATION
----------
I configured my terminals to have an incredibly garish color scheme to
show exactly what's going to happen as a result of this. The _default
terminal background_ is purple or red, and the foreground green. I've
printed out a heap of test colors to see how black interacts with them.

Pull request #6810 contains the images generated from this test.

The only color lines that change are the ones where black as a
background or white as a foreground is selected out of the 16-color
palette explicitly. Reverse video still works fine (because black is in
the foreground!), and it's even possible to represent "black on default"
and reverse it into "default on black", despite the black in question
having been `40`.

Fixes #6767.
2020-07-10 15:25:39 -07:00

96 lines
3.4 KiB
C++

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "ConsoleShimPolicy.h"
// Routine Description:
// - Constructs a new instance of the shim policy class.
// Arguments:
// - All arguments specify a true/false status to a policy that could be applied to a console client app.
ConsoleShimPolicy::ConsoleShimPolicy(const bool isCmd,
const bool isPowershell) :
_isCmd{ isCmd },
_isPowershell{ isPowershell }
{
}
// Routine Description:
// - Opens the process token for the given handle and resolves the process name.
// We'll initialize the new ConsoleShimPolicy based on whether the client
// process is "cmd.exe" or "powershell.exe".
// - For more info, see GH#3126
// Arguments:
// - hProcess - Handle to a connected process
// Return Value:
// - ConsoleShimPolicy object containing resolved shim policy data.
ConsoleShimPolicy ConsoleShimPolicy::s_CreateInstance(const HANDLE hProcess)
{
// If we cannot determine the exe name, then we're probably not cmd or powershell.
bool isCmd = false;
bool isPowershell = false;
try
{
const std::filesystem::path processName = wil::GetModuleFileNameExW<std::wstring>(hProcess, nullptr);
auto clientName = processName.filename().wstring();
// For whatever reason, wil::GetModuleFileNameExW leaves trailing nulls, so get rid of them.
clientName.erase(std::find(clientName.begin(), clientName.end(), '\0'), clientName.end());
// Convert to lower case, just in case
std::transform(clientName.begin(), clientName.end(), clientName.begin(), std::towlower);
isCmd = clientName.compare(L"cmd.exe") == 0;
// For powershell, we need both Windows PowersShell (powershell.exe) and
// PowerShell Core (pwsh.exe). If PowerShell Core is ever updated to use
// ^[[3J for Clear-Host, then it won't ever hit the shim code path, but
// we're keeping this for the long tail of pwsh versions that still
// _don't_ use that sequence.
isPowershell = (clientName.compare(L"powershell.exe") == 0) ||
(clientName.compare(L"pwsh.exe") == 0);
}
CATCH_LOG();
return ConsoleShimPolicy(isCmd, isPowershell);
}
// Method Description:
// - Returns true if the connected client application is literally "cmd.exe". If
// it is, we'll need to enable certain compatibility shims.
// Arguments:
// - <none>
// Return Value:
// - rue iff the process is "cmd.exe"
bool ConsoleShimPolicy::IsCmdExe() const noexcept
{
return _isCmd;
}
// Method Description:
// - Returns true if the connected client application is literally
// "powershell.exe" or "pwsh.exe". If it is, we'll need to enable certain
// compatibility shims.
// Arguments:
// - <none>
// Return Value:
// - rue iff the process is "powershell.exe" or "pwsh.exe"
bool ConsoleShimPolicy::IsPowershellExe() const noexcept
{
return _isPowershell;
}
// Method Description:
// - Returns true if the connected client application is known to
// attempt VT color promotion of legacy colors. See GH#6807 for more.
// Arguments:
// - <none>
// Return Value:
// - True as laid out above.
bool ConsoleShimPolicy::IsVtColorQuirkRequired() const noexcept
{
// Right now, the only client we're shimming is powershell.
return IsPowershellExe();
}