Make a few changes to memory usage throughout the application to reduce transient allocations from the `big.txt` test from ~213,000 to ~53,000.
## PR Checklist
* [x] Supports #3075
* [x] I work here.
* [x] Tested manually and WPR'd. Test suite should still pass.
* [x] Am core contributor
## Detailed Description of the Pull Request / Additional comments
Transient allocations are those that are new'd, used, then delete'd. Going back and forth to the system allocator for things we're just going to throw away or use rapidly again is a performance detriment. Not only is it a bunch of time to go ask the system with a syscall, it also hits a whole bunch of locks on the allocators. This PR identifies a few places where we were accidentally allocating and didn't mean to or were allocating and freeing just to turn around and allocate again. I chose other strategies to avoid this.
## Validation Steps Performed
- Ran `big.txt` sample (~6MB file) before and after. Observed heap allocations with WPR.
This commit replaces DeviceComm with the interface IDeviceComm and the
concrete implementation type ConDrvDeviceComm. This work is done in
preparation for different device backends.
In addition to separating out ConDrv-specific behavior, I've introduced
a "handle exchange" interface.
HANDLE EXCHANGE
---------------
There are points where we give ConDrv opaque handle identifiers to our
input buffer, output buffer and process data. The exact content of the
opaque identifier is meaningless to ConDrv: the driver's only
interaction with these identifiers is to hold onto them and send back
whichever are pertinent for each API call.
Because of that, we used the raw register-width pointer value of the
input buffer, output buffer or process data _as_ the opaque handle
value.
This works very well for ConDrv <-> conhost using the ConDrvDeviceComm.
It works less well for something like the "logging" DeviceComm that will
log packets to a file: those packets *cannot* contain pointer values (!)
To address this, and to afford flexibility to DeviceComm implementers,
I've introduced a two-member complement of handle management functions:
* `ULONG_PTR PutHandle(void*)` registers an object with the DeviceComm
and returns an opaque identifier.
* `void* GetHandle(ULONG_PTR)` takes an opaque identifier and returns
the original object.
ConDrvDeviceComm implements PutHandle and GetHandle by casting the
object pointer to the "opaque handle value", which maintains wire format
compatibility[1] with revisions of conhost prior to this change.
Simpler DeviceComm implementations that require handle tracking but
cannot bear raw pointers can implement these functions by returning an
autoincrementing ID (or similar) and registering the raw object pointer
in a mapping.
I've audited all existing handle exchanges with the driver and updated
them to use Put/GetHandle.
(I intended to add DestroyHandle, but we are not equipped for handle
removal at the moment. ConsoleHandleData/ConsoleProcessHandle are
destroyed during wait routine completion, on client disconnect, etc.
This does mean that an id<->pointer mapping will grow without bound, but
at a cost of ~8 bytes per entry and a short-lived console session I am
not too concerned about the cost.)
[1] Wire format compatibility is not required, and later we may want to
switch ConDrvDeviceComm to `EncodePointer` and `DecodePointer` just to
insulate us against a spurious ConDrv packet allowing for an arbitrary
4/8-byte read and subsequent liftoff into space.
Many include statements use forward slashes, while others use backwards
slashes. This is inconsistent formatting. For this reason, I changed the
backward slashes to forward slashes since that is the standard.
We were using std::basic_string_view as a stand-in for std::span so that
we could change over all at once when C++20 dropped with full span
support. That day's not here yet, but as of 54a7fce3e we're using GSL 3,
whose span is C++20-compliant.
This commit replaces every instance of basic_string_view that was not
referring to an actual string with a span of the appropriate type.
I moved the `const` qualifier into span's `T` because while
`basic_string_view.at()` returns `const T&`, `span.at()` returns `T&`
(without the const). I wanted to maintain the invariant that members of
the span were immutable.
* Mechanical Changes
* `sv.at(x)` -> `gsl::at(sp, x)`
* `sv.c{begin,end}` -> `sp.{begin,end}` (span's iterators are const)
I had to replace a `std::basic_string<>` with a `std::vector<>` in
ConImeInfo, and I chose to replace a manual array walk in
ScreenInfoUiaProviderBase with a ranged-for. Please review those
specifically.
This will almost certainly cause a code size regression in Windows
because I'm blowing out all the PGO counts. Whoops.
Related: #3956, #975.
There is going to be a very long tail of applications that will
explicitly request VT SGR 40/37 when what they really want is to
SetConsoleTextAttribute() with a black background/white foreground.
Instead of making those applications look bad (and therefore making us
look bad, because we're releasing this as an update to something that
"looks good" already), we're introducing this compatibility quirk.
Before the color reckoning in #6698 + #6506, *every* color was subject
to being spontaneously and erroneously turned into the default color.
Now, only the 16-color palette value that matches the active console
background/foreground color will be destroyed, and only when received
from specific applications.
Removal will be tracked by #6807.
Michael and I discussed what layer this quirk really belonged in. I
originally believed it would be sufficient to detect a background color
that matched the legacy default background, but @j4james provided an
example of where that wouldn't work out (powershell setting the
foreground color to white/gray). In addition, it was too heavyhanded: it
re-broke black backgrounds for every application.
Michael thought that it should live in the server, as a small VT parser
that righted the wrongs coming directly out of the application. On
further investigation, however, I realized that we'd need to push more
information up into the server (so that it could make the decision about
which VT was wrong and which was right) than should be strictly
necessary.
The host knows which colors are right and wrong, and it gets final say
in what ends up in the buffer.
Because of that, I chose to push the quirk state down through
WriteConsole to DoWriteConsole and toggle state on the
SCREEN_INFORMATION that indicates whether the colors coming out of the
application are to be distrusted. This quirk _only applies to pwsh.exe
and powershell.exe._
NOTE: This doesn't work for PowerShell the .NET Global tool, because it
is run as an assembly through dotnet.exe. I have no opinion on how to
fix this, or whether it is worth fixing.
VALIDATION
----------
I configured my terminals to have an incredibly garish color scheme to
show exactly what's going to happen as a result of this. The _default
terminal background_ is purple or red, and the foreground green. I've
printed out a heap of test colors to see how black interacts with them.
Pull request #6810 contains the images generated from this test.
The only color lines that change are the ones where black as a
background or white as a foreground is selected out of the 16-color
palette explicitly. Reverse video still works fine (because black is in
the foreground!), and it's even possible to represent "black on default"
and reverse it into "default on black", despite the black in question
having been `40`.
Fixes#6767.
## Summary of the Pull Request
This PR implements a pair of shims for `cmd` and `powershell`, so that their `cls` and `Clear-Host` functions will clear the entire terminal buffer (like they do in conhost), instead of just the viewport. With the conpty viewport and buffer being the same size, there's effectively no way to know if an application is calling these API's in this way with the intention of clearing the buffer or the viewport. We absolutely have to guess.
Each of these shims checks to see if the way that the API is being called exactly matches the way `cmd` or `powershell` would call these APIs. If it does, we manually write a `^[[3J` to the connected terminal, to get he Terminal to clear it's own scrollback.
~~_⚠️ If another application were trying to clear the **viewport** with an exactly similar API call, this would also cause the terminal scrollback to get cleared ⚠️_~~
* [x] Should these shims be restricted to when the process that's calling them is actually `cmd.exe` or `powershell.exe`? Can I even do this? I think we've done such a good job of isolating the client process information from the rest of the host code that I can't figure out how to do this.
- YES, this can be done, and I did it.
* [ ] **TODO**: _While I'm here_, should I have `DoSrvPrivateEraseAll` (the implementation for `^[[2J`, in `getset.cpp`) also manually trigger a EraseAll in the terminal in conpty mode?
## PR Checklist
* [x] Closes#3126
* [x] Actually closes#1305 too, which is really the same thing, but probably deserves a callout
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
## Validation Steps Performed
* ran tests
* checked `cls` in the Terminal
* checked `Clear-Host` in the Terminal
* Checked running `powershell clear-host` from `cmd.exe`
This commit introduces a github action to check our spelling and fixes
the following misspelled words so that we come up green.
It also renames TfEditSes to TfEditSession, because Ses is not a word.
currently, excerpt, fallthrough, identified, occurred, propagate,
provided, rendered, resetting, separate, succeeded, successfully,
terminal, transferred, adheres, breaks, combining, preceded,
architecture, populated, previous, setter, visible, window, within,
appxmanifest, hyphen, control, offset, powerpoint, suppress, parsing,
prioritized, aforementioned, check in, build, filling, indices, layout,
mapping, trying, scroll, terabyte, vetoes, viewport, whose
Generated by https://github.com/jsoref/spelling `f`; to maintain your repo, please consider `fchurn`
I generally try to ignore upstream bits. I've accidentally included some items from the `deps/` directory. I expect someone will give me a list of items to drop, I'm happy to drop whole files/directories, or to split the PR into multiple items (E.g. comments/locals/public).
Closes#4294
## Summary of the Pull Request
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#4013
* [x] I work here.
* [x] Existing tests should be OK. Real changes, just adding a lib to use.
* [x] Couldn't find any existing docs about intsafe.
* [x] Am core contributor.
## Detailed Description of the Pull Request / Additional comments
* [x] Can we remove min/max completely or rename it in the two projects where it had to be reintroduced? This is now moved into #4152
* [x] How many usages of the old safe math are there? **79**
* [x] If not a ton, can we migrate them here or in a follow on PR? This is now moved into #4153
Files with old safe math:
- TerminalControl: TSFInputControl.cpp
- TerminalCore: TerminalDispatch.cpp
- TerminalCore: TerminalSelection.cpp
- Host: directio.cpp
- RendererGdi: invalidate.cpp
- RendererGdi: math.cpp
- RendererGdi: paint.cpp
- RendererVt: paint.cpp
- TerminalAdapter: adaptDispatch.cpp
- Types: viewport.cpp
- Types: WindowUiaProviderBase.cpp
## Validation Steps Performed
## Summary of the Pull Request
- Enables auditing of some Terminal libraries (Connection, Core, Settings)
- Also audit WinConPTY.LIB since Connection depends on it
## PR Checklist
* [x] Rolls audit out to more things
* [x] I work here
* [x] Tests should still pass
* [x] Am core contributor
## Detailed Description of the Pull Request / Additional comments
This is turning on the auditing of these projects (as enabled by the heavier lifting in the other refactor) and then cleaning up the remaining warnings.
## Validation Steps Performed
- [x] Built it
- [x] Ran the tests
This commit cleans up and deduplicates all of the common build
preamble/postamble across exe, dll, lib and c++/winrt projects.
The following specific changes have been made:
* All projects now define their ConfigurationType
* All projects now set all their properties *before* including a common
build file (or any other build files)
* cppwinrt.pre and cppwinrt.post now delegate most of their
configuration to common.pre and common.post
* (becuase of the above,) all build options are conserved between
console and c++/winrt components, including specific warnings and
preprocessor definitions.
* More properties that are configurable per-project are now
conditioned so the common props don't override them.
* The exe, dll, exe.or.dll, and lib postincludes have been merged into
pre or post and switched based on condition as required
* Shared items (-shared, -common) are now explicitly vcxitems instead of
vcxproj files.
* The link line is now manipulated after Microsoft.Cpp sets it, so the
libraries we specify "win". All console things link first against
onecore_apiset.lib.
* Fix all compilation errors caused by build unification
* Move CascadiaPackage's resources into a separate item file
Fixes#922.
* Ensure rights check and increments pass before assigning an object to a handle (since deallocation of handles will automatically attempt to cleanup).
Adjusts the startup and shutdown behavior of most threads in the console host to alleviate race conditions that are either exacerbated or introduced by the VT PTY threads.
### User Stories:
1. A user wants to be able to use the executable path as their starting title
- Does anyone want this?
2. A user wants to be able to set a custom starting title, but have that title be overridable
3. A user wants to be able to set an overridable starting title, different from the profile name
- Presumably someone will want this
4. A user totally wants to ignore the VT title and use something else
- This will make more sense in the post [#1320] "Support runtime variables in the custom user title" settings
### Solutions:
1. `name`, `startingTitle`, `tabTitle`
* a. `name` is only ever used as the profile name.
* b. If `startingTitle` isn't set, then the executable path is used
* c. If `startingTitle` is set, it's used as the initial title
* d. If `tabTitle` is set, it overrides the title from the terminal
* e. Current users of `tabTitle` need to manually update to the new behavior.
2. `name` as starting title, `tabTitle` as a different starting title
* a. `name` is used as the starting title and the profile name in the dropdown
* b. If `tabTitle` is set, we'll use that as the overridable starting title instead.
* c. In the future, `dynamicTabTitle` or `tabTitleOverride` could be added to support [#1320]
* d. Current users of `tabTitle` automatically get the new (different!) behavior.
* e. User Story 1 is impossible
- Does anyone want the behavior _ever_? Perhaps making that scenario impossible is good?
3. `name` unchanged, `tabTitle` as the starting title
* a. `name` is only ever used as the profile name.
* b. If `tabTitle` is set, we'll use that as the overridable starting title.
* c. In the future, `dynamicTabTitle` or `tabTitleOverride` could be added to support [#1320]
* d. Current users of `tabTitle` automatically get the new (different!) behavior.
4. `name` as starting title, `tabTitle` as different starting title, `suppressApplicationTitle` Boolean to force it to override
* a. `name`, `tabTitle` work as in Solution 2.
* b. When someone wants to be able to statically totally override that title (story 4), they can use `suppressApplicationTitle`
* c. `suppressApplicationTitle` name is WIP
* d. We'll add `suppressApplicationTitle` when someone complains
* e. If you really want story 1, use `tabTitle: c:\path\to\foo.exe` and `suppressApplicationTitle`.
[#1320]: https://github.com/microsoft/terminal/issues/1320
We've decided to pursue path 4.
* Removed using namespace directive from header files and put these in cpp files where they are used
* Fixed tabbing issues by replacing them with spaces.
Also regrouped the using directives.
* Update src/host/exemain.cpp
Co-Authored-By: Mike Griese <migrie@microsoft.com>
* Update src/interactivity/win32/find.cpp
Co-Authored-By: Mike Griese <migrie@microsoft.com>
* Apply [[nodiscard]] to functions returning error codes
- applied [[nodiscard]] for all HRESULT, LRESULT, and NTSTATUS functions
- fixed IntelliSense declaration complaints leading to function not
implemented warnings
- deleted declared but never implemented functions
- fixed unused parameter warnings
How verified:
- bcz dbg
- opencon
- testcon
- VS2019 debug build
* - use LOG_IF_FAILED where applicable
- remove use of goto
- make MakeAltRasterFont return void
* - add missing [[nodiscard]]
- remove vestigal function declarations
- fix inconsistent function declaration
* static analysis fixes
* using C++ style casts
* explicit delete changed to reset(nullptr)
* fix for null apiMsg.OtherId during tracing in Compare()
* changed INVALID_ID macro to constexpr
* properly handle null ReplyMsg in ConsoleIoThread()
* Fixed wrong static_cast for State.InputBuffer
* compensate for null reply message to fix deref problem of ReplyMsg in srvinit.cpp by changing signature in DeviceComm.h
