This commit moves the handling of the `BEL`, `BS`, `TAB`, and `CR`
controls characters into the state machine (when in VT mode), instead of
forwarding them on to the default string writer, which would otherwise
have to parse them out all over again.
This doesn't cover all the control characters, but `ESC`, `SUB`, and
`CAN` are already an integral part of the `StateMachine` itself; `NUL`
is filtered out by the `OutputStateMachineEngine`; and `LF`, `FF`, and
`VT` are due to be implemented as part of PR #3271.
Once all of these controls are handled at the state machine level, we
can strip out all the VT-specific code from the `WriteCharsLegacy`
function, which should simplify it considerably. This would also let us
simplify the `Terminal::_WriteBuffer` implementation, and the planned
replacement stream writer for issue #780.
On the conhost side, the implementation is handled as follows:
* The `BS` control is dispatched to the existing `CursorBackward`
method, with a distance of 1.
* The `TAB` control is dispatched to the existing `ForwardTab` method,
with a tab count of 1.
* The `CR` control required a new dispatch method, but the
implementation was a simple call to the new `_CursorMovePosition` method
from PR #3628.
* The `BEL` control also required a new dispatch method, as well as an
additional private API in the `ConGetSet` interface. But that's mostly
boilerplate code - ultimately it just calls the `SendNotifyBeep` method.
On the Windows Terminal side, not all dispatch methods are implemented.
* There is an existing `CursorBackward` implementation, so `BS` works
OK.
* There isn't a `ForwardTab` implementation, but `TAB` isn't currently
required by the conpty protocol.
* I had to implement the `CarriageReturn` dispatch method, but that was
a simple call to `Terminal::SetCursorPosition`.
* The `WarningBell` method I've left unimplemented, because that
functionality wasn't previously supported anyway, and there's an
existing issue for that (#4046).
## Validation Steps Performed
I've added a state machine test to confirm that the updated control
characters are now forwarded to the appropriate dispatch handlers. But
since the actual implementation is mostly relying on existing
functionality, I'm assuming that code is already adequately tested
elsewhere. That said, I have also run various manual tests of my own,
and confirmed that everything still worked as well as before.
References #3271
References #780
References #3628
References #4046
## Summary of the Pull Request
Originally there were 3 different methods for implementing VT cursor movement, and between them they still couldn't handle some of the operations correctly. This PR unifies those operations into a single method that can handle every type of cursor movement, and which fixes some of the issues with the existing implementations. In particular it fixes the `CNL` and `CPL` operations, so they're now correctly constrained by the `DECSTBM` margins.
## References
If this PR is accepted, the method added here should make it trivial to implement the `VPR` and `HPR` commands in issue #3428.
## PR Checklist
* [x] Closes#2926
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Detailed Description of the Pull Request / Additional comments
The new [`AdaptDispatch::_CursorMovePosition`](d6c4f35cf6/src/terminal/adapter/adaptDispatch.cpp (L169)) method is based on the proposal I made in issue #3428 for the `VPR` and `HPR` comands. It takes three arguments: a row offset (which can be absolute or relative), a column offset (ditto), and a flag specifying whether the position should be constrained by the `DECSTBM` margins.
To make the code more readable, I've implemented the offsets using [a `struct` with some `constexpr` helper functions for the construction](d6c4f35cf6/src/terminal/adapter/adaptDispatch.hpp (L116-L125)). This lets you specify the parameters with expressions like `Offset::Absolute(col)` or `Offset::Forward(distance)` which I think makes the calling code a little easier to understand.
While implementing this new method, I noticed a couple of issues in the existing movement implementations which I thought would be good to fix at the same time.
1. When cursor movement is constrained horizontally, it should be constrained by the buffer width, and not the horizontal viewport boundaries. This is an issue I've previously corrected in other parts of the codebase, and I think the cursor movement was one of the last areas where it was still a problem.
2. A number of the commands had range and overflow checks for their parameters that were either unnecessary (testing for a condition that could never occur) or incorrect (if an operation overflows, the correct behavior is to clamp it, and not just fail). The new implementation handles legitimate overflows correctly, but doesn't check for impossible ranges.
Because of the change of behavior in point 1, I also had to update the implementations of [the `DECSC` and `CPR` commands](9cf7a9b577) to account for the column offset now being relative to the buffer and not the viewport, otherwise those operations would no longer work correctly.
## Validation Steps Performed
Because of the two changes in behavior mentioned above, there were a number of adapter tests that stopped working and needed to be updated. First off there were those that expected the column offset to be relative to the left viewport position and constrained by the viewport width. These now had to be updated to [use the full buffer width](49887a3589) as the allowed horizontal extent.
Then there were all the overflow and out-of-range tests that were testing conditions that could never occur in practice, or where the expected behavior that was tested was actually incorrect. I did spend some time trying to see if there was value in updating these tests somehow, but in the end I decided it was best to just [drop them](6e80d0de19) altogether.
For the `CNL` and `CPL` operations, there didn't appear to be any existing tests, so I added some [new screen buffer tests](d6c4f35cf6) to check that those operations now work correctly, both with and without margins.
## Summary of the Pull Request
This adds support for the `FF` (form feed) and `VT` (vertical tab) [control characters](https://vt100.net/docs/vt510-rm/chapter4.html#T4-1), as well as the [`NEL` (Next Line)](https://vt100.net/docs/vt510-rm/NEL.html) and [`IND` (Index)](https://vt100.net/docs/vt510-rm/IND.html) escape sequences.
## References
#976 discusses the conflict between VT100 Index sequence and the VT52 cursor back sequence.
## PR Checklist
* [x] Closes#3189
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3189
## Detailed Description of the Pull Request / Additional comments
I've added a `LineFeed` method to the `ITermDispatch` interface, with an enum parameter specifying the required line feed type (i.e. with carriage return, without carriage return, or dependent on the [`LNM` mode](https://vt100.net/docs/vt510-rm/LNM.html)). The output state machine can then call that method to handle the various line feed control characters (parsed in the `ActionExecute` method), as well the `NEL` and `IND` escape sequences (parsed in the `ActionEscDispatch` method).
The `AdaptDispatch` implementation of `LineFeed` then forwards the call to a new `PrivateLineFeed` method in the `ConGetSet` interface, which simply takes a bool parameter specifying whether a carriage return is required or not. In the case of mode-dependent line feeds, the `AdaptDispatch` implementation determines whether the return is necessary or not, based on the existing _AutoReturnOnNewLine_ setting (which I'm obtaining via another new `PrivateGetLineFeedMode` method).
Ultimately we'll want to support changing the mode via the [`LNM` escape sequence](https://vt100.net/docs/vt510-rm/LNM.html), but there's no urgent need for that now. And using the existing _AutoReturnOnNewLine_ setting as a substitute for the mode gives us backwards compatible behaviour, since that will be true for the Windows shells (which expect a linefeed to also generate a carriage return), and false in a WSL bash shell (which won't want the carriage return by default).
As for the actual `PrivateLineFeed` implementation, that is just a simplified version of how the line feed would previously have been executed in the `WriteCharsLegacy` function. This includes setting the cursor to "On" (with `Cursor::SetIsOn`), potentially clearing the wrap property of the line being left (with `CharRow::SetWrapForced` false), and then setting the new position using `AdjustCursorPosition` with the _fKeepCursorVisible_ parameter set to false.
I'm unsure whether the `SetIsOn` call is really necessary, and I think the way the forced wrap is handled needs a rethink in general, but for now this should at least be compatible with the existing behaviour.
Finally, in order to make this all work in the _Windows Terminal_ app, I also had to add a basic implementation of the `ITermDispatch::LineFeed` method in the `TerminalDispatch` class. There is currently no need to support mode-specific line feeds here, so this simply forwards a `\n` or `\r\n` to the `Execute` method, which is ultimately handled by the `Terminal::_WriteBuffer` implementation.
## Validation Steps Performed
I've added output engine tests which confirm that the various control characters and escape sequences trigger the dispatch method correctly. Then I've added adapter tests which confirm the various dispatch options trigger the `PrivateLineFeed` API correctly. And finally I added some screen buffer tests that check the actual results of the `NEL` and `IND` sequences, which covers both forms of the `PrivateLineFeed` API (i.e. with and without a carriage return).
I've also run the _Test of cursor movements_ in the [Vttest](https://invisible-island.net/vttest/) utility, and confirmed that screens 1, 2, and 5 are now working correctly. The first two depend on `NEL` and `IND` being supported, and screen 5 requires the `VT` control character.
## Summary of the Pull Request
In pursuit of reflowing the terminal buffer on resize, move the reflow algorithm to the TextBuffer. This does _not_ yet add support for reflowing in the Windows Terminal.
## References
## PR Checklist
* [ ] There's not really an issue for this yet, I'm just breaking this work up into as many PRs as possible to help the inevitable bisect.
* [x] I work here
* [x] Ideally, all the existing tests will pass
* [n/a] Requires documentation to be updated
## Detailed Description of the Pull Request / Additional comments
In `SCREEN_INFORMATION::ResizeScreenBuffer`, the screenbuffer needs to create a new buffer, and copy the contents of the old buffer into the new one. I'm moving that "copy contents from the old buffer to the new one" step to it's own helper, as a static function on `TextBuffer`. That way, when the time comes to implement this for the Terminal, the hard part of the code will already be there.
## Validation Steps Performed
Ideally, all the tests will still pass.
## Summary of the Pull Request
This PR reverses the behaviour of the `IS_GLYPH_CHAR` macro, so it now actually returns true if the given char is a glyph, and false if it isn't. Previously it returned the opposite of that, which meant it had to be called as `!IS_GLYPH_CHAR` to get the correct result.
## PR Checklist
* [x] Closes#4185
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #4185
## Detailed Description of the Pull Request / Additional comments
The original implementation returned true if the given character was a C0 control, or a DEL:
#define IS_GLYPH_CHAR(wch) (((wch) < L' ') || ((wch) == 0x007F))
It's now the exact opposite, so returns true for characters that are _not_ C0 controls, and are not the DEL character either:
#define IS_GLYPH_CHAR(wch) (((wch) >= L' ') && ((wch) != 0x007F))
The macro was only used in one place, where is was being called as `!IS_GLYPH_CHAR` when the intent was actually to test whether the char _was_ a glyph. That code could now be updated to remove the `!`, so it makes more sense.
## Validation Steps Performed
I've just tested manually and confirmed that basic output of text and control chars still worked as expected in a conhost shell.
## Summary of the Pull Request
New year, new unittests.
This PR introduces a new project, `TestHostApp`. This project is largely taken from the TAEF samples, and allows us to easily construct a helper executable and `resources.pri` for running TerminalApp unittests.
## References
## PR Checklist
* [x] Closes#3986
* [x] I work here
* [x] is Tests
* [n/a] Requires documentation to be updated
* [x] **Waiting for an updated version of TAEF to be available**
## Detailed Description of the Pull Request / Additional comments
Unittesting for the TerminalApp project has been a horrifying process to try getting everything pieced together just right. Dependencies need to get added to manifests, binplaced correctly, and XAML resources need to get compiled together as well. In addition, using a MUX `Application` (as opposed to the Windows.UI.Xaml `Application`) has led to additional problems.
This was always a horrifying house of cards for us. Turns out, the reason this was so horrible is that the test infrastructure for doing what we're doing _literally didn't exist_ when I started doing all that work last year.
So, with help from the TAEF team, I was able to get rid of our entire house of cards, and use a much simpler project to build and run the tests.
Unfortunately, the latest TAEF release has a minor bug in it's build rules, and only publishes the x86 version of a dll we need from them. But, the rest of this PR works for x86, and I'll bump this when that updated version is available. We should be able to review this even in the state it's in.
## Validation Steps Performed
ran the tests yo
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
It's 2020 now. It's *about* time that we move on from 1990's macros.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
## References
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [X] Closes#4152
* [X] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
Remove global namespaced min/max and replace it with STL min/max.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
Run it.
## Summary of the Pull Request
Refactors parsing/adapting libraries and consumers to use safer and/or more consistent mechanisms for passing information.
## PR Checklist
* [x] I work here
* [x] Tests still pass
* [x] Am a core contributor.
## Detailed Description of the Pull Request / Additional comments
This is in support of hopefully turning audit mode on to more projects. If I turned it on, it would immediately complain about certain classes of issues like pointer and size, pointer math, etc. The changes in this refactoring will eliminate those off the top.
Additionally, this has caught a bunch of comments all over the VT classes that weren't updated to match the parameters lists.
Additionally, this has caught a handful of member variables on classes that were completely unused (and now gone).
Additionally, I'm killing almost all hungarian and shortening variable names. I'm only really leaving 'p' for pointers.
Additionally, this is vaguely in support of a future where we can have "infinite scrollback" in that I'm moving things to size_t across the board. I know it's a bit of a memory cost, but all the casting and moving between types is error prone and unfun to save a couple bytes.
## Validation Steps Performed
- [x] build it
- [x] run all the tests
- [x] everyone looked real hard at it
This pull request also includes build break fixes for things that do not build in the OSS repo
and disables the retro terminal effect in conhost.
Retrieved from https://microsoft.visualstudio.com os OS official/rs_onecore_dep_uxp f10445678e59197c1ae2ee29d8f009c9607c4e5d
Related work items: #24387718
## Summary of the Pull Request
This adds support for the [`DECALN`](https://vt100.net/docs/vt510-rm/DECALN.html) escape sequence, which produces a kind of test pattern, originally used on VT terminals to adjust the screen alignment. It's needed to pass several of the tests in the [Vttest](https://invisible-island.net/vttest/) suite.
## PR Checklist
* [x] Closes#3671
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Detailed Description of the Pull Request / Additional comments
To start with, the `ActionEscDispatch` method in the `OutputStateMachineEngine` needed to be extended to check for a new intermediate type (`#`). Then when that intermediate is followed by an `8`, it dispatches to a new `ScreenAlignmentPattern` method in the `ITermDispatch` interface.
The implementation of the `ScreenAlignmentPattern` itself is fairly simple. It uses the recently added `PrivateFillRegion` API to fill the screen with the character `E` using default attributes. Then in addition to that, a bunch of VT properties are reset:
* The meta/extended attributes are reset (although the active colors must be left unchanged).
* The origin mode is set to absolute positioning.
* The scrolling margins are cleared.
* The cursor position is moved to home.
## Validation Steps Performed
I've added a screen buffer test that makes sure the `DECALN` sequence fills the screen with the correct character and attributes, and that the above mentioned properties are all updated appropriately.
I've also tested in Vttest, and confirmed that the first two pages of the _Test of cursor movements_ are now showing the frame of E's that are expected there.
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
This is the PR for feature Search: #605
This PR includes the newly introduced SearchBoxControl in TermControl dir, which is the search bar for the search experience. And the codes that enable Search in Windows Terminal.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
The PR that migrates the Conhost search module: https://github.com/microsoft/terminal/pull/3279
Spec (still actively updating): https://github.com/microsoft/terminal/pull/3299
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#605
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
These functionalities are included in the search experience.
1. Search in Terminal text buffer.
2. Automatic wrap-around.
3. Search up or down switch by clicking different buttons.
4. Search case sensitively/insensitively by clicking a button. S. Move the search box to the top/bottom by clicking a button.
6. Close by clicking 'X'.
7. Open search by ctrl + F.
When the searchbox is open, the user could still interact with the terminal by clicking the terminal input area.
While I already have the search functionalities, currently there are still some known to-do works and I will keep updating my PR:
1. Optimize the search box UI, this includes:
1) Theme adaptation. The search box background and font color
should change according to the theme,
2) Add background. Currently the elements in search box are all
transparent. However, we need a background.
3) Move button should be highlighted once clicked.
2. Accessibility: search process should be able to performed without mouse. Once the search box is focused, the user should be able to navigate between all interactive elements on the searchbox using keyboard.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
To test:
1. checkout this branch.
2. Build the project.
3. Start Windows Terminal and press Ctrl+F
4. The search box should appear on the top right corner.
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
Cool retro terminal effects
- glow
- scan lines
- cool
- will make terminal competitive with iterm2
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
## References
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [ ] Closes #xxx
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
## Summary of the Pull Request
When the `DECSTBM` margins are set, scrolling should only allowed within
those margins. So if the cursor is below the bottom margin, it should
just be clamped when it tries to move down from the bottom line of the
viewport, instead of scrolling the screen up. This PR implements that
restriction, i.e. it prevents scrolling taking place outside the
`DECSTBM` margins, which was previously allowed.
## PR Checklist
* [x] Closes#2657
* [x] CLA signed.
* [x] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
This simply adds a condition in the `AdjustCursorPosition` function to
check if the cursor is moving below the bottom of the viewport when the
margins are set. If so, it clamps the y coordinate to the bottom line of
the viewport.
The only time it's acceptable for scrolling to happen when margins are
set, is if the scrolling is taking place within those margins. But those
cases would already have been handled earlier in the function (in the
`fScrollDown` or `scrollDownAtTop` conditions), and thus the y
coordinate would have already been prevented from moving out of the
viewport.
## Validation Steps Performed
I've added some screen buffer tests to confirm this new behaviour, and
I've also checked the test case from the initial bug report (#2657) and
made sure it now matches the results in XTerm.
## Summary of the Pull Request
Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing.
## PR Checklist
* [x] Closes#2553
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan.
## Detailed Description of the Pull Request / Additional comments
My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all.
To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset).
With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`.
The functions affected by scrolling included:
* `DoSrvPrivateReverseLineFeed` (the RI command)
* `DoSrvPrivateModifyLinesImpl` (the IL and DL commands)
* `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands)
* `AdaptDispatch::_ScrollMovement` (the SU and SD commands)
The functions affected by erasing included:
* `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants)
* `AdaptDispatch::EraseCharacters` (the ECH command)
While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time.
In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes:
* `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer.
* `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised.
* `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes.
The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above).
Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations.
Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed.
Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs).
## Validation Steps Performed
For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region.
In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer.
The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests.
Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
# Summary of the Pull Request
When a horizontal tab ('\t') is output on the last column of the screen, the current implementation moves the cursor position to the start of the next line. However, the DEC STD 070 manual specifies that a horizontal tab shouldn't move past the last column of the active line (or the right margin, if we supported horizontal margins). This PR updates the forward tab implementation, to prevent it wrapping onto a new line when it reaches the end of a line.
# Detailed Description of the Pull Request / Additional comments
Originally the SCREEN_INFORMATION::GetForwardTab method had a condition which handled a tab at the end of the line as a special case, moving the cursor to the start of the next line. I've simply removed that condition, so an end-of-line tab is handled the same way as any other position (in this case it will just leaves the cursor where it is).
While testing, though, I found that there were circumstances where you could have tab stops greater than the width of the screen, and when that happens, a tab can still end up wrapping onto the next line. To fix that I had to add an additional check to make sure the tab position was always clamped to the width of the buffer.
With these fixes in place, a tab control should now never move off the active line, so I realised that the DoPrivateTabHelper function could be optimized to calculate all of the tab movements in advance, and then only make a single call to AdjustCursorPosition with the final coordinates. This change is not strictly necessary, though, so it can easily be reverted if there are any objections.
Regarding backwards compatibility, note that the GetForwardTab method is only used in two places:
when handling a tab character in the WriteCharsLegacy function, but this only applies in VT mode (see here).
when handling the CHT escape sequence in the DoPrivateTabHelper function, and obviously an escape sequence would also only be applicable in VT mode.
So this change should have no effect on legacy console applications, which wouldn't have VT mode activated.
# Validation Steps Performed
I've added another step to the TestGetForwardTab test which makes sure that a horizontal tab won't wrap at the end of a line.
I've also confirmed that this fixes the last remaining issue in the Test of autowrap in Vttest (pages 3 and 4 of the Test of cursor movements). Although I should note that this only works in conhost.
* Make search a shared component for conhost and terminal
* Remove inclusion of deprecated interface file
* Code review changes, remove text buffer modification in Terminal
* remove unreferenced objects to fix build errors
* Fix test failure, guarantee uiaData object is correctly initialized in Search
* minor comment typo fix and format fix
* minor PR comments change
* ColorSeclection directly throw and return
* remove coordAnchor initialization
* minor method signature change
* Change to use App CRT in preparation for universal.
* Try to make project build again by setting winconpty to static lib so it'll use the CRT inside TerminalConnection (or its other consumers) instead of linking its own.
* Remove test for conpty dll, it's a lib now. Add additional commentary on how CRT linking works for future reference. I'm sure this will come up again.
* use the _apiset variant until proven otherwise to match the existing one.
* Clarification in the comments for linking.
This commit cleans up and deduplicates all of the common build
preamble/postamble across exe, dll, lib and c++/winrt projects.
The following specific changes have been made:
* All projects now define their ConfigurationType
* All projects now set all their properties *before* including a common
build file (or any other build files)
* cppwinrt.pre and cppwinrt.post now delegate most of their
configuration to common.pre and common.post
* (becuase of the above,) all build options are conserved between
console and c++/winrt components, including specific warnings and
preprocessor definitions.
* More properties that are configurable per-project are now
conditioned so the common props don't override them.
* The exe, dll, exe.or.dll, and lib postincludes have been merged into
pre or post and switched based on condition as required
* Shared items (-shared, -common) are now explicitly vcxitems instead of
vcxproj files.
* The link line is now manipulated after Microsoft.Cpp sets it, so the
libraries we specify "win". All console things link first against
onecore_apiset.lib.
* Fix all compilation errors caused by build unification
* Move CascadiaPackage's resources into a separate item file
Fixes#922.
* Ensure rights check and increments pass before assigning an object to a handle (since deallocation of handles will automatically attempt to cleanup).
The current DECSC implementation only saves the cursor position and
origin mode. This PR improves that functionality with additional support
for saving the SGR attributes, as well as the active character set.
It also fixes the way the saved state interacts with the alt buffer
(private mode 1049), triggering a save when switching to the alt buffer,
and a restore when switching back, and tracking the alt buffer state
independently from the main state.
In order to properly save and restore the SGR attributes, we first
needed to add a pair of APIs in the `ConGetSet` interface which could
round-trip the attributes with full 32-bit colors (the existing methods
only work with legacy attributes).
I then added a struct in the `AdaptDispatch` implementation to make it
easier to manage all of the parameters that needed to be saved. This
includes the cursor position and origin mode that we were already
tracking, and now also the SGR text attributes and the active character
set (via the `TermOutput` class).
Two instances of this structure are required, since changes made to the
saved state in the alt buffer need to be tracked separately from changes
in the main buffer. I've added a boolean property that specifies whether
we're in the alt buffer or not, and use that to decide which of the two
instances we're working with.
I also needed to explicitly trigger a save when switching to the alt
buffer, and a restore when switching back, since we weren't already
doing that (the existing implementation gave the impression that the
state was saved, because each buffer has its own cursor position, but
that doesn't properly match the XTerm behaviour).
For the state tracking itself, we've now got two additional properties -
the SGR attributes, which we obtain via the new private API, and the
active character set, which we get from a local `AdaptDispatch` field.
I'm saving the whole `TermOutput` class for the character set, since I'm
hoping that will make it automatically supports future enhancements.
When restoring the cursor position, there is also now a fix to handle
the relative origin mode correctly. If the margins are changed between
the position being saved and restored, it's possible for the cursor to
end up outside of the new margins, which would be illegal. So there is
now an additional step that clamps the Y coordinate within the margin
boundaries if the origin mode is relative.
# Validation
I've added a couple of screen buffer tests which check that the various
parameters are saved and restored as expected, as well as checking that
the Y coordinate is clamped appropriately when the relative origin mode
is set.
I've also tested manually with vttest and confirmed that the
_SAVE/RESTORE CURSOR_ test (the last page of the _Test of screen
features_)) is now working a lot better than it used to.
Closes#148.
[Git2Git] Git Train: Merge of building/rs_onecore_dep_uxp/191011-1234 into official/rs_onecore_dep_uxp Retrieved from https://microsoft.visualstudio.com os OS official/rs_onecore_dep_uxp b80345479891d1e7a9f7e38b6b5f40083c6a564a
sources changes from 21H1
Merged PR 3896217: [Git2Git] Changes from vb_release_dep_dev1
server init changes from 20H1 (onecore headless mode)
We now truncate the font name as it goes out to GDI APIs, in console API
servicing, and in the propsheet.
I attempted to defer truncating the font to as far up the stack as
possible, so as to make FontInfo usable for the broadest set of cases.
There were a couple questions that came up: I know that `Settings` gets
memset (memsat?) by the registry deserializer, and perhaps that's
another place for us to tackle. Right now, this pull request enables
fonts whose names are >= 32 characters _in Windows Terminal only_, but
the underpinnings are there for conhost as well. We'd need to explicitly
break at the API, or perhaps return a failure or log something to
telemetry.
* Should we log truncation at the API boundary to telemetry?
-> Later; followup filed (#3123)
* Should we fix Settings here, or later?
-> Later; followup filed (#3123)
* `TrueTypeFontList` is built out of things in winconp, the private
console header. Concern about interop structures.
-> Not used for interop, followup filed to clean it up (#3123)
* Is `unsigned int` right for codepage? For width?
-> Yes: codepage became UINT (from WORD) when we moved from Win16 to
Win32
This commit also includes a workaround for #3170. Growing
CONSOLE_INFORMATION made us lose the struct layout lottery during
release builds, and this was an expedient fix.
Closes#602.
Related to #3123.
## Summary of the Pull Request
Adds support for Italics, Blinking, Invisible, CrossedOut text, THROUGH CONPTY. This does **NOT** add support for those styles to conhost or the terminal.
We will store these "Extended Text Attributes" in a `TextAttribute`. When we go to render a line, we'll see if the state has changed from our previous state, and if so, we'll appropriately toggle that state with VT. Boldness has been moved from a `bool` to a single bit in these flags.
Technically, now that these are stored in the buffer, we only need to make changes to the renderers to be able to support them. That's not being done as a part of this PR however.
## References
See also #2915 and #2916, which are some follow-up tasks from this fix. I thought them too risky for 20H1.
## PR Checklist
* [x] Closes#2554
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
<hr>
* store text with extended attributes too
* Plumb attributes through all the renderers
* parse extended attrs, though we're not renderering them right
* Render these states correctly
* Add a very extensive test
* Cleanup for PR
* a block of PR feedback
* add 512 test cases
* Fix the build
* Fix @carlos-zamora's suggestions
* @miniksa's PR feedback
* Potentially fixes#1825
I haven't had a chance to test this fix on my machine with a CentOS VM quite yet, but this _should_ work
Also adds a test
* add a comment
* woah hey this test was wrong
* Revert bx.ps1
If we're moving the cursor up, its vertical movement should be stopped
at the top margin. It should not magically jump up to the bottom margin.
Similarly, this applies to moving down and the bottom margin.
Furthermore, this constraint should always apply, not just when the
start position is within BOTH margins
Fixes#2929.
EraseInLine calls `FillConsoleOutputCharacterW()`. In filling the row with
chars, we were setting the wrap flag. We need to specifically not do this on
ANY _FILL_ operation. Now a fill operation UNSETS the wrap flag if we fill to
the end of the line.
Originally, we had a boolean `setWrap` that would mean...
- **true**: if writing to the end of the row, SET the wrap value to true
- **false**: if writing to the end of the row, DON'T CHANGE the wrap value
Now we're making this bool a std::optional to allow for a ternary state. This
allows for us to handle the following cases completely. Refer to the table
below:
,- current wrap value
| ,- are we filling the last cell in the row?
| | ,- new wrap value
| | | ,- comments
|-- |-- |-- |
| 0 | 0 | 0 |
| 0 | 1 | 0 |
| 0 | 1 | 1 | THIS CASE WAS HANDLED CORRECTLY
| 1 | 0 | 0 | THIS CASE WAS UNHANDLED
| 1 | 0 | 1 |
| 1 | 1 | 1 |
To handle that special case (1-0-0), we need to UNSET the wrap. So now, we have
~setWrap~ `wrap` mean the following:
- **true**: if writing to the end of the row, SET the wrap value to TRUE
- **false**: if writing to the end of the row, SET the wrap value to FALSE
- **nullopt**: leave the wrap value as it is
Closes#1126
* Revert "Add source linking information during the build (#2857)"
This reverts commit 6b728cd6d0.
* Need reference to renderer base inside UnitTests_TerminalCore
* add dependency for TerminalControl to Types project.
* Set build to single threaded as parallel build is broken by 16.3 build toolchain.
* Disable new rule C26814 as it's breaking builds
Wrote a follow up task #2941 to roll it out later.
* Add noexcept to dx header.
fixes#1222
PSReadline calls SetConsoleCursorPosition on each character they emit (go
figure). When that function is called, and we set the cursor position, we'll
try and "snap" the viewport to the location of the cursor, so that the cursor
remains visible.
However, we'd only ever do this with the visible viewport, the viewport
defined by `SCREEN_INFORMATION::_viewport`. When there's a virtual viewport in
Terminal Scrolling mode, we actually need to snap the virtual viewport, so
that this behavior looks more regular.
Copies source linking scripts and processes from Microsoft/Microsoft-UI-XAML. This embeds source information inside the PDBs in two formats: One for WinDBG using a PowerShell script that runs during the build, and one for Visual Studio using the Microsoft.SourceLink.GitHub NuGet pacakge. Sources are automatically pulled from raw.githubusercontent.com when debugging a release build inside either of these utilities as of this change.
* conhost: if we start with invalid terminal colors, reset them to sanity
We've seen a number of cases where the user's settings can get corrupted
and their default foreground/background and cursor color get set to all
black (black on black). This results in a fairly unhappy user and
probably a great number of support incidents.
Let's declare that an invalid state.
* Add some comments to the comments
The `DECSTBM` margins are meant to define the range of lines within which
certain vertical scrolling operations take place. However, we were applying
these margin restrictions in the `ScrollRegion` function, which is also used in
a number of places that shouldn't be affected by `DECSTBM`.
This includes the `ICH` and `DCH` escape sequences (which are only affected by
the horizontal margins, which we don't yet support), the
`ScrollConsoleScreenBuffer` API (which is public Console API, not meant to be
affected by the VT terminal emulation), and the `CSI 3 J` erase scrollback
extension (which isn't really scrolling as such, but uses the `ScrollRegion`
function to manipulate the scrollback buffer).
This commit moves the margin clipping out of the `ScrollRegion` function, so it
can be applied exclusively in the places that need it.
With the margin clipping removed from the `ScrollRegion` function, it now had
to be applied manually in the places it was actually required. This included:
* The `DoSrvPrivateReverseLineFeed` function (for the `RI` control): This was
* just a matter of updating the bottom of the scroll rect to the bottom margin
* (at least when the margins were actually set), since the top of the scroll
* rect would always be the top of the viewport. The
* `DoSrvPrivateModifyLinesImpl` function (for the `IL` and `DL` commands):
* Again this was just a matter of updating the bottom of the scroll rect, since
* the cursor position would always determine the top of the scroll rect. The
* `AdaptDispatch::_ScrollMovement` method (for the `SU` and `SD` commands):
* This required updating both the top and bottom coordinates of the scroll
* rect, and also a simpler destination Y coordinate (the way the `ScrollRegion`
* function worked before, the caller was expected to take the margins into
* account when determining the destination).
On the plus side, there was now no longer a need to override the margins when
calling `ScrollRegion` in the `AdjustCursorPosition` function. In the first
case, the margins had needed to be cleared (_stream.cpp 143-145), but that is
now the default behaviour. In the second case, there had been a more
complicated adjustment of the margins (_stream.cpp 196-209), but that code was
never actually used so could be removed completely (to get to that point either
_fScrollUp_ was true, so _scrollDownAtTop_ couldn't also be true, or
_fScrollDown_ was true, but in that case there is a check to make sure
_scrollDownAtTop_ is false).
While testing, I also noticed that one of the `ScrollRegion` calls in the
`AdjustCursorPosition` function was not setting the horizontal range correctly
- the scrolling should always affect the full buffer width rather than just the
viewport width - so I've fixed that now as well.
## Validation Steps Performed
For commands like `RI`, `IL`, `DL`, etc. where we've changed the implementation
but not the behaviour, there were already unit tests that could confirm that
the new implementation was still producing the correct results.
Where there has been a change in behaviour - namely for the `ICH` and `DCH`
commands, and the `ScrollConsoleScreenBuffer` API - I've extended the existing
unit tests to check that they still function correctly even when the `DECSTBM`
margins are set (which would previously have caused them to fail).
I've also tested manually with the test cases in issues #2543 and #2659, and
confirmed that they now work as expected.
Closes#2543Closes#2659
This change enables VT processing by default for _all_ conpty clients. See #1965 for a discussion on why we believe this is a righteous change.
Also mentioned in the issue was the idea of only checking the `VirtualTerminalLevel` reg key in the conpty startup. I don't think this would be a more difficult change, looks like all we'd need is a simple `reg.LoadGlobalsFromRegistry();` call instead of this change.
# Validation Steps Performed
Manually launched a scratch app in both the terminal and the console. The console launch's output mode was 0x3, and the terminal's was 0x7. 0x4 is the ` ENABLE_VIRTUAL_TERMINAL_PROCESSING` flag, which the client now had by default in the Terminal.
Closes#1965
Adjusts the startup and shutdown behavior of most threads in the console host to alleviate race conditions that are either exacerbated or introduced by the VT PTY threads.
In the legacy console, it used to be possible to write out characters
from the C0 range of a PC code page (e.g. CP437), and get the actual
glyphs defined for those code points (at least those that weren't
processed as control codes). In the v2 console this stopped working so
you'd get an FFFD replacement glyph (�) for those characters instead.
This PR fixes the issue so the correct glyphs are displayed again.
There was already code in place to achieve this in the
`WriteCharsLegacy` method. It used the `GetStringTypeW` method to
determine the character type of the value being output, and if it was a
`C1_CNTRL` character it performed the appropriate mapping. The problem
was that the test of the character type flag was done as a direct
comparision, when it should have been a bit test, so the condition was
never met.
With this condition fixed, the code also needed to be reordered slightly
to handle the null character. That had a special-case mapping to space,
which was previously performed after the control test, but since a null
character now successfully matches `C1_CNTRL`, it no longer falls
through to that special case. To address that, I've had to move the null
check above the control test.
I've tested this manually, by trying to output all the characters in the
affected range (ASCII values 0 to 31, and 127, excluding the actual
control codes 8,9,10 and 13). In all cases they now match the output
that the legacy console produced.
Note that this only applies to PC code pages that have glyphs defined
for the C0 range, so it won't work with the UTF-8 code page, but that
was to be expected - the legacy console behaved the same way.
Also, note that this only works when the `ENABLE_PROCESSED_OUTPUT`
console mode is set. That seems wrong to me (I'd expect the glyphs to
work in both cases), but that's the way the legacy console behaved as
well, so if that's a bug it's a separate issue.
I haven't added any unit tests, because I expect the behaviour of some
of these characters to change over time (as support is added for more
control codes), which could then cause the tests to fail. But if that's
not a concern, I could probably add something to the ScreenBufferTests
(perhaps with a comment warning that the tests might be expected to fail
in the future).
Closes#166.
* fixes#411
* update this comment to actually match
* run this test in isolation so it doesn't break other tests, @dhowett-msft
* This fixes the test that's broken?
Kinda raises more questions tbh
CLS calls two functions:
- `SetConsoleCursorPositionImpl()`
- `ScrollConsoleScreenBufferWImpl()`
Both of these were not checking which buffer to apply to (main vs active buffer).
Now we get the active buffer and apply the changes to that one.
Also, we forgot to switch out of the alt buffer in the previous test. Added that in.
Closes#1189.
* Move cursor position to the left margin after execution of the IL and DL escape sequences.
* Update IL and DL screen buffer tests to account for the cursor moving to the left margin.
There are a number of VT escape sequences that rely on the `ScrollRegion`
function to scroll the viewport (RI, DL, IL, SU, SD, ICH, and DCH) , and all of
them have got the clipping rect or scroll boundaries wrong in some way,
resulting in content being scrolled off the screen that should have been
clipped, revealed areas not being correctly filled, or parts of the screen not
being moved that should have been. This PR attempts to fix all of those issues.
The `ScrollRegion` function is what ultimately handles the scrolling, but it's
typically called via the `ApiRoutines::ScrollConsoleScreenBufferWImpl` method,
and it's the callers of that method that have needed correcting.
One "mistake" that many of these operations made, was in setting a clipping
rect that was different from the scrolling rect. This should never have been
necessary, since the area being scrolled is also the boundary into which the
content needs to be clipped, so the easiest thing to do is just use the same
rect for both parameters.
Another common mistake was in clipping the horizontal boundaries to the width
of the viewport. But it's really the buffer width that represents the active
width of the screen - the viewport width and offset are merely a window on that
active area. As such, the viewport should only be used to clip vertically - the
horizontal extent should typically be the full buffer width.
On that note, there is really no need to actually calculate the buffer width
when we want to set any of the scrolling parameters to that width. The
`ScrollRegion` function already takes care of clipping everything within the
buffer boundary, so we can simply set the `Left` of the rect to `0` and the
`Right` to `SHORT_MAX`.
More details on individual commands:
* RI (the `DoSrvPrivateReverseLineFeed` function)
This now uses a single rect for both the scroll region and clipping boundary,
and the width is set to `SHORT_MAX` to cover the full buffer width. Also the
bottom of the scrolling region is now the bottom of the viewport (rather than
bottom-1), otherwise it would be off by one.
* DL and IL (the `DoSrvPrivateModifyLinesImpl` function)
Again this uses a single rect for both the scroll region and clipping
boundary, and the width is set to `SHORT_MAX` to cover the full width. The
most significant change, though, is that the bottom boundary is now the
viewport bottom rather than the buffer bottom. Using the buffer bottom
prevented it clipping the content that scrolled off screen when inserting,
and failed to fill the revealed area when deleting.
* SU and SD (the `AdaptDispatch::_ScrollMovement` method)
This was already using a single rect for both the scroll region and clipping
boundary, but it was previously constrained to the width of the viewport
rather than the buffer width, so some areas of the screen weren't correctly
scrolled. Also, the bottom boundary was off by 1, because it was using an
exclusive rect while the `ScrollRegion` function expects inclusive rects.
* ICH and DCH (the `AdaptDispatch::_InsertDeleteHelper` method)
This method has been considerably simplified, because it was reimplementing a
lot of functionality that was already provided by the `ScrollRegion`
function. And like many of the other cases, it has been updated to use a
single rect for both the scroll region and clipping boundary, and clip to the
full buffer width rather than the viewport width.
I should add that if we were following the specs exactly, then the SU and SD
commands should technically be panning the viewport over the buffer instead of
moving the buffer contents within the viewport boundary. So SU would be the
equivalent of a newline at the bottom of the viewport (assuming no margins).
And SD would assumedly do the opposite, scrolling the back buffer back into
view (an RI at the top of the viewport should do the same).
This doesn't seem to be something that is consistently implemented, though.
Some terminals do implement SU as a viewport pan, but I haven't seen anyone
implement SD or RI as a pan. If we do want to do something about this, I think
it's best addressed as a separate issue.
## Validation Steps Performed
There were already existing tests for the SU, SD, ICH, and DCH commands, but
they were implemented as adapter tests, which weren't effectively testing
anything - the `ScrollConsoleScreenBufferW` method used in those tests was just
a mock (an incomplete reimplementation of the `ScrollRegion` function), so
confirming that the mock produced the correct result told you nothing about the
validity of the real code.
To address that, I've now reimplemented those adapter tests as screen buffer
tests. For the most part I've tried to duplicate the functionality of the
original tests, but there are significant differences to account for the fact
that scrolling region now covers the full width of the buffer rather than just
the viewport width.
I've also extended those tests with additional coverage for the RI, DL, and IL
commands, which are really just a variation of the SU and SD functionality.
Closes#2174
## Summary of the Pull Request
When a user had "Disable Scroll Forward" enabled and switched to the alt buffer and maximized the console, then restored down, we'd crash. Now we don't.
## References
## PR Checklist
* [x] Closes#1206
* [x] I work here
* [x] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
The problem is that we'd previously try to "anchor" the viewport to the virtual bottom when resizing like this. This would also cause us to move the top of the viewport down, into the buffer. However, if the alt buffer is getting smaller, we don't want to do this - if we anchor to the old _virtualBottom, the bottom of the viewport will actually be outside the current buffer.
This could theoretically happen with the main buffer too, but it's much easier to repro with the alt buffer.
This pull request introduces a copy of the code from kernel32.dll that
implements CreatePseudoConsole, ClosePseudoConsole and
ResizePseudoConsole. Apart from some light modifications to fit into the
infrastructure in this project and support launching OpenConsole.exe, it
is intended to be 1:1 with the code that ships in Windows.
Any guideline violations in this code are likely intentional. Since this
was built into kernel32, it uses the STL only _very sparingly._
Consumers of this library must make sure that conpty.lib lives earlier
in the link line than onecoreuap_apiset, onecoreuap, onecore_apiset,
onecore or kernel32.
Refs #1130.
[Git2Git] Git Train: Merge of building/rs_onecore_dep_uxp/190820-1847 into official/rs_onecore_dep_uxp Retrieved from https://microsoft.visualstudio.com os OS official/rs_onecore_dep_uxp 73e964d4046c37df3030970cae1ae32e83103fb5
(cherry picked from commit 8c63dff982093db1af7e2bb46b49af884dfec0c5)
When the scrollback buffer is empty, the RIS escape sequence (Reset to Initial
State) will fail to clear the screen, or reset any of the state. And when there
is something in the scrollback, it doesn't get cleared completely, and the
screen may get filled with the wrong background color (it should use the
default color, but it actually uses the previously active background color).
This commit attempts to fix those issues.
The initial failure is caused by the `SCREEN_INFORMATION::WriteRect` method
throwing an exception when passed an empty viewport. And the reason it's passed
an empty viewport is because that's what the `Viewport::Subtract` method
returns when the result of the subtraction is nothing. The PR fixes the
problem by making the `Viewport::Subtract` method actually return nothing in
that situation.
This is a change in the defined behavior that also required the associated
viewport tests to be updated. However, it does seem a sensible change, since
the `Subtract` method never returns empty viewports under any other
circumstances. And the only place the method seems to be used is in the
`ScrollRegion` implementation, where the previous behavior is guaranteed to
throw an exception.
The other issues are fixed simply by changing the order in which things are
reset in the `AdaptDispatch::HardReset` method. The call to `SoftReset` needed
to be made first, so that the SGR attributes would be reset before the screen
was cleared, thus making sure that the default background color would be used.
And the screen needed to be cleared before the scrollback was erased, otherwise
the last view of the screen would be retained in the scrollback buffer.
These changes also required existing adapter tests to be updated, but not
because of a change in the expected behaviour. It's just that certain tests
relied on the `SoftReset` happening later in the order, so weren't expecting it
to be called if say the scrollback erase had failed. It doesn't seem like the
tests were deliberately trying to verify that the SoftReset _hadn't_ been
called.
In addition to the updates to existing tests, this PR also add a new screen
buffer test which verifies the display and scrollback are correctly cleared
under the conditions that were previously failing.
Fixes#2307.
Refactors the accessibility providers (ScreenInfoUiaProvider and UiaTextRange) into a better separated model between ConHost and Windows Terminal.
ScreenInfoUiaProviderBase and UiaTextRangeBase are introduced. ConHost and Windows Terminal implement their own versions of ScreenInfoUiaProvider and UiaTextRange that inherit from their respective base classes.
WindowsTerminal's ScreenInfoUiaProvider --> TermControlUiaProvider
This commit replaces CodepointWidthDetector's
dynamically-generated map with a static constexpr one that's compiled
into the binary.
It also almost totally removes the notion of an `Invalid` width. We
definitely had gaps in our character coverage where we'd report a
character as invalid, but we'd then flatten that down to `Narrow` when
asked. By combining the not-present state and the narrow state, we get
to save a significant chunk of data.
I've tested this by feeding it all 0x10FFFF codepoints (and then some)
and making sure they 100% match the old code's outputs.
|------------------------------|---------------|----------------|
| Metric | Then | Now |
|------------------------------|---------------|----------------|
| disk space | 56k (`.text`) | 3k (`.rdata`) |
| runtime memory (allocations) | 1088 | 0 |
| runtime memory (bytes) | 51k | ~0 |
| memory behavior | not shared | fully shared |
| lookup time | ~31ns | ~9ns |
| first hit penalty | ~170000ns | 0ns |
| lines of code | 1088 | 285 |
| clarity | extreme | slightly worse |
|------------------------------|---------------|----------------|
I also took a moment and cleaned up a stray boolean that we didn't need.
* Don't trigger a frame due to circling when in the middle of a resize operation
This fixes#1795, and shined quite a bit of light on the whole conpty resize process.
* Move the Begin/End to ResizeScreenBuffer, to catch more cases.
**The Basics of Accessibility**
- [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview)
- Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it.
- Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right.
**Accessibility in our Project**
There's a few important classes...
regarding Accessibility...
- **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree.
- **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer.
- **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them.
regarding the Windows Terminal window...
- **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff.
- **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal
- **NonClientIslandWindow**: An extension of the `IslandWindow`
regarding ConHost...
- **IConsoleWindow**: This is an interface for the console window.
- **Window**: This is the actual window for ConHost. Extends `IConsoleWindow`
- `IConsoleWindow` changes:
- move into `Microsoft::Console::Types` (a shared space)
- Have `IslandWindow` extend it
- `WindowUiaProvider` changes:
- move into `Microsoft::Console::Types` (a shared space)
- Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree)
### Changes to the WindowUiaProvider
As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized.
In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen.
So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen.
🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352.
### Moved to `Microsoft::Console::Types`
These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal.
This means that any references to the `ServiceLocator` had to be removed.
- `IConsoleWindow`
- Windows Terminal: `IslandWindow : IConsoleWindow`
- `ScreenInfoUiaProvider`
- all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details.
- `UiaTextRange`
- all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details.
- since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them.
### Changes to IRenderData
Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here:
- `bool IsAreaSelected()`
- `void ClearSelection()`
- `void SelectNewRegion(...)`
- `HRESULT SearchForText(...)`
`SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
* Refactors TerminalApp into two projects:
- TerminalAppLib, which builds a .lib, and includes all the code
- TerminalApp, which builds a dll by linking the lib
* Adds a TerminalApp.Unit.Tests project
- Includes the ability to test cppwinrt types we've authored using a SxS manifest for unpackaged winrt activation
- includes the ability to test types with XAML content using an appxmanifest
* Adds a giant doc explaining how this was all done. Really, just go read that doc, it'll really help you understand what's going on in this PR.
-------------------------
These are some previous commit messages. They may be helpful to future readers.
* Start adding unittests for json parsing, end up creating a TerminalAppLib project to make a lib. See #1042
* VS automatically did this for me
* This is a dead end
I tried including the idl-y things into the lib, but that way leads insanity
If you want to make a StaticLibrary, then suddenly the winrt toolchain forgets
that ProjectReferences can have winmd's in them, so it won't be able to
compile any types from the referenced projects. If you instead try to manually
reference the types, you'll get duplicate types up the wazoo, which of course
is insane, since we're referencing them the _one_ time
* Yea just follow #1042 on github for status
So current state:
1. If you try to add a `Reference` to all of MUX.Markup, TerminalControl and
TerminalSettings, then mdmerge will complain about all the types from
TerminalSettings being defined twice. In this magic scenario, the
dependencies of TerminalControl are used directly for some reason:
```
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalSettings\Microsoft.Terminal.Settings.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.Settings.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.TerminalConnection.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.TerminalControl.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\Microsoft.UI.Xaml.Markup\Microsoft.UI.Xaml.Markup.winmd.
```
2. If you don't add a `Reference` TerminalControl, then it'll complain about
being unable to find the type TitleChangedEventArgs, which is defined in
TerminalControl.
3. If you don't add a `Reference` TerminalSettings, then it'll complain about
being unable to find the type KeyChord and other types from
TerminalSettings. In this scenario, it doesn't recurse on the other
dependencies from TerminalControl for whatever reason.
4. If you instead try to add all 3 as a `ProjectReference`, then it'll
complain about being unable to find TitleChangedEventArgs, as in 2.
Presumably, it;ll have troubles with the other types too, as none of the 3
are actually included in the midlrt.rsp file.
5. If you add all 3 as a `ProjectReference`, then also add TerminalControl as
a `Reference`, you'll get a `MIDL2011: [msg] unresolved type declaration
Microsoft.UI.Xaml.Markup.XamlApplication`
6. If you add all 3 as a `ProjectReference`, then also add TerminalControl AND
MUX.Markup as a `Reference`, you'll get the same result as 3.
* what if we just don't idl
This seems to compile
* This compiles but I broke the MUX resources
look at the App.xaml change. in this changelist. That's what's broken right now. Lets fix that!
* lets do this
If I leave the MUX nuget out of the project, I'll get a compile error in
App.xaml:
```
...OpenConsole\src\cascadia\TerminalApp\App.xaml(21,40): XamlCompiler error WMC0001: Unknown type 'XamlControlsResources' in XML namespace 'using:Microsoft.UI.Xaml.Controls'
```
If I add it back to the project, it works
* Some cleanup from the previous commit
* This is busted again.
Doing a clean build didn't work.
A clean rebuild of the project, paired with some removal of dead code
revealed a problem with what I have so far.
TerminalAppLib depends on the generation of two headers,
`AppKeyBindings.g.h` and `App.g.h`, as those define some of bits of the
winrt types. They're needed to be able to compile the implementations.
Presumably that's not getting generated by the lib project, because the dll
project is the one to generate that file.
So we need to move the idl's to the lib project. This created maddness,
because of course the Duplicate Type thing. The solution to that is to
actually mark the winrt DLLs that we're chaining up through us as
```
<Private>false</Private>
<CopyLocalSatelliteAssemblies>false</CopyLocalSatelliteAssemblies>
```
This will prevent them from getting double-included.
This still doesn't work however, since
```
app.cpp(40): error C2039: 'XamlMetaDataProvider': is not a member of 'winrt::TerminalApp'
error C3861: 'XamlMetaDataProvider': identifier not found
```
So we need to figure that out. The dll project is still generating the right
header, so lets look there.
* Move the xaml stuff to the lib
This compiles, but when we launch, we fail to load the tabviewcontrol
resources again. So that's not what you want. Why is it not included?
* It works again!
* Use the pri, xbf files from TerminalAppLib, not TerminalApp
* Manually make TerminalApp include a reference to TerminalAppLib's
TerminalApp.winmd. This will force the build to copy TerminalApp.winmd to
TerminalApp/, which WindowsTerminal needs to be able to ProjectReference the
TerminalApp project (it's expecting it to have a winmd)
* Remove the module.g.cpp from TerminalApp, and move to TerminalAppLib. The
dll doesn't do any codegen anymore.
* Agressively clean up these files
* Clean up unnecessary includes in the dll pch.h
* This does NOT work.
The WindowsxamlManager call crashes. I'm thinking it has to do with activation
of winrt types from a dll.
Email out to @Austin-Lamb to see if he can assist
* This gets our cppwinrt types working, but xaml islands is still broken
* Split the tests apart, so they aren't insane
* These are the magic words to make xaml islands work
* All this witchcraft is necessary to make XAML+MUX work right
* Clean this up a bit and add comments
* Create an enormous doc explaining this madness
* Unsure how this got changed.
* Trying to get the CI build to work again.
This resolves the MUX issue. We need to manually include it, because their package's target doesn't mark it as CopyLocalSatelliteAssemblies=false, Private=false.
However, the TerminalApp project is still able to magically reason that the TerminalAppLib project should be included in the MdMerge step, because it think's it's a `GetCppWinRTStaticProjectReferences` reference.
* Update cppwinrt to the latest version - this fixes the MSBuild
* I still need to re-add the KeyModifiers checks from TermControl. I think
this update broke `operator&` for that enum.
* There needs to be some cleanup obviously
* The doc should be updated as well
* Clean up changes from cppwinrt update
* Try doing this, even though it seems wrong
* Lets try this (press x to doubt)
* Clean up vcxproj file, and remove appxmanifest change from previous commit
* Update to the latest TAEF release, maybe that'll work
* Let's try a prerelease version, shall we?
* Add notes about TAEF package, comment out tests
* Format the code
* Hopefully fix the arm64 and x86 builds
also a typo
* Fix PR nits
* Fix some bad merge conflicts
* Some cleanup from the merge
* Well I was close to getting the merge right
* I believe this will fix CI
* Apply suggestions from code review
Co-Authored-By: Carlos Zamora <carlos.zamora@microsoft.com>
* These definitely need to be fixed
* Try version detecting in the test
IDK if this will build, I'm letting the CI try while I clean rebuild locally
* Try blindly updating to the newest nuget version
* Revert "Try blindly updating to the newest nuget version"
This reverts commit b72bd9eb73.
* We're just going to see if these work in CI with this change
* Comment the tests back out. Windows Server 2019 is 10.0.17763.557
* Remove the nuget package
We don't need this package anymore now that we're hosting it
* Okay this _was_ important
* Fix margin boundary tests in the RI, DL, and IL sequences.
* Refactor the margin boundary tests into a reusable SCREEN_INFORMATION method.
* Add screen buffer unit tests for the RI, DL, and IL sequences.
* Add support for origin mode (DECOM).
* Added a state machine unit test for the origin mode.
* Prevent the cursor position moving below the bottom margin of the scrolling region if the origin mode is relative.
* Only adjust the relative cursor position for origin mode if the scrolling region is actually set.
* Add some screenbuffer unit tests for the origin mode.
* Enhance the soft reset screenbuffer tests to verify the origin mode is reset.
* Move the origin mode flag constructor assignments into the intializer list.
* Implement XTerm's private mode escape sequence for enabling DECCOLM support.
* Add output engine and screen buffer units test for the private mode 40 escape sequence.
…. It will return a replacement character at that point if it was given bad data. #788
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
This modifies the parser used while inserting text into the underlying data buffer to never return an empty sequence. The empty sequence is invalid as you can't insert a "nothing" into the buffer. The buffer asserted this with a fail fast crash. Now we will instead insert U+FFFD (the Unicode replacement character) � to symbolize that something was invalid and has been replaced.
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#788 and internal MSFT: 20990158
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [x] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #788
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
The solution here isn't perfect and isn't going to solve all of our problems. I was basically trying to stop the crash while not getting in the way of the other things coming down the pipe for the input channels.
I considered the following:
1. Remove the fail fast assertion from the buffer
- I didn't want to do this because it really is invalid to get all the way to placing the text down into the buffer and then request a string of 0 length get inserted. I feel the fail fast is a good indication that something is terribly wrong elsewhere that should be corrected.
2. Update the UTF16 parser in order to stop returning empty strings
- This is what I ultimately did. If it would ever return just a lead, it returns �. If it would ever return just a trail, it returns �. Otherwise it will return them as a pair if they're both there, or it will return a single valid codepoint. I am now assuming that if the parse function is being called in an Output Iterator and doesn't contain a string with all pieces of the data that are needed, that someone at a higher level messed up the data, it is in valid, and it should be repaired into replacements.
- This then will move the philosophy up out of the buffer layer to make folks inserting into the buffer identify half a sequence (if they're sitting on a stream where this circumstance could happen... one `wchar_t` at a time) and hold onto it until the next bit arrives. This is because there can be many different routes into the buffer from many different streams/channels. So buffering it low, right near the insertion point, is bad as it might pair loose `wchar_t` across stream entrypoints.
3. Update the iterator, on creating views, to disallow/transform empty strings.
- I considered this solution as well, but it would have required, under some circumstances, a second parsing of the string to identify lead/trail status from outside the `Utf16Parser` class to realize when to use the � character. So I avoided the double-parse.
4. Change the cooked read classes to identify that they pulled the lead `wchar_t` from a sequence then try to pull another one.
- I was going to attempt this, but @adiviness said that he tried it and it made all sorts of other weirdness happen with the edit line.
- Additionally, @adiviness has an outstanding series of effort to make cooked read significantly less horrible and disgusting. I didn't want to get in the way here.
5. Change the `GetChar` method off of the input buffer queue to return a `char32_t`, a `wstring_view`, transform a standalone lead/trail, etc.
- The `GetChar` method is used by several different accessors and API calls to retrieve information off of the input queue, transforming the Key events into straight up characters. To change this at that level would change them all. Long-term, it is probably warranted to do so as all of those consumers likely need to become aware of handling UTF-16 surrogates before we can declare victory. But two problems.
1. This gets in the way of @adiviness work on cooked read data
2. This goes WAY beyond the scope of what I want to accomplish here as the immediate goal is to stop the crash, not fix the world.
I've validated this by:
1. Writing some additional tests against the Utf16Parser to simulate some of the theoretical sequences that could arrive and need to be corrected into replacement characters per a verbal discussion and whiteboarding with @adiviness.
2. Manually triggered the emoji panel and inserted a bunch of emoji. Then seeked around left and right, deleted assorted points with the backspace key, pressed enter to commit, and used the up-arrow history to recommit them to see what happened. There were no crashes. The behavior is still weird and not great... but outside the scope of no crashy crashy.
Flushes the input queue on RawReadUnpacksCoalescedInputRecords test to ensure that other tests cannot cause failure by leaving extraneous input records behind after they run.
This only failed in the core operating system gate tests. This is because those tests run a subset of the complete test suite (subtracting the ones that do not make sense in a core environment). Apparently one of the tests that was skipped that normally runs prior to the UnpacksCoalesced test ensured that the input queue was clean enough for this test to succeed. But in the core environment, the test that ran prior left stuff behind.
To resolve this, I'm making the Coalesced test more resilient by cleaning out the queue prior to performing its operations.
(Also, bonus, I'm fixing the typo in the name Coalesced.)
This is less complicated/expensive than tracking down the tests that are leaving garbage behind, should prevent issues in the future related to ordering (since the tests run alphabetically, by default), and isn't as expensive as running the test in isolation (with its own conhost stood up for just the one test.)
Validated by running te.exe Microsoft.Console.Host.FeatureTests.dll /name:*InputTests* against a core operating system variant. Prior to change, this test failed. After the change, this test succeeded.
This will be automatically double-checked by the gates run after check-in.
Almost all functions in the Windows API that open or create objects and return HANDLEs to them return null on failure; only a few (mostly to do with the file system) return INVALID_HANDLE_VALUE on failure. This PR scrubs the repo of a few, but not necessarily all, cases where INVALID_HANDLE_VALUE was mistakenly used or tested against instead of null. In particular, it fixes 2 cases reported in issue #427 where the return value of CreateThread() was compared against INVALID_HANDLE_VALUE against null, causing the error handling code to run at the wrong time.
There are a lot of other uses of INVALID_HANDLE_VALUE I found that looked questionable, but which I left alone. Most of these were used to initialize HANDLE-typed variables and as a sentinel to see if those variables remained unset to a "real" value.
Fixes#427
* Removed using namespace directive from header files and put these in cpp files where they are used
* Fixed tabbing issues by replacing them with spaces.
Also regrouped the using directives.
* Update src/host/exemain.cpp
Co-Authored-By: Mike Griese <migrie@microsoft.com>
* Update src/interactivity/win32/find.cpp
Co-Authored-By: Mike Griese <migrie@microsoft.com>
* Apply [[nodiscard]] to functions returning error codes
- applied [[nodiscard]] for all HRESULT, LRESULT, and NTSTATUS functions
- fixed IntelliSense declaration complaints leading to function not
implemented warnings
- deleted declared but never implemented functions
- fixed unused parameter warnings
How verified:
- bcz dbg
- opencon
- testcon
- VS2019 debug build
* - use LOG_IF_FAILED where applicable
- remove use of goto
- make MakeAltRasterFont return void
* - add missing [[nodiscard]]
- remove vestigal function declarations
- fix inconsistent function declaration
* Support OSC to set default background and foreground colors
* Update the Terminal theme when the background changes
* Fix whitespace per code-review
* Add Documentation Comments
Also fix a few outdated comments and whitespace
* Update Telemetry codes per code review
* Add Unit Tests for OSC ForegroundColor and BackgroundColor
* Add a couple additional test cases
* Minor doc and whitespace change per PR review
* Update comment help per code review
* Add another OSC 10 & 11 test case, improve output
* Comments and syntax cleanup per code reviews
* static analysis fixes
* using C++ style casts
* explicit delete changed to reset(nullptr)
* fix for null apiMsg.OtherId during tracing in Compare()
* changed INVALID_ID macro to constexpr
* properly handle null ReplyMsg in ConsoleIoThread()
* Fixed wrong static_cast for State.InputBuffer
* compensate for null reply message to fix deref problem of ReplyMsg in srvinit.cpp by changing signature in DeviceComm.h
This encompasses a handful of problems with column counting.
The Terminal project didn't set a fallback column counter. Oops. I've fixed this to use the `DxEngine` as the fallback.
The `DxEngine` didn't implement its fallback method. Oops. I've fixed this to use the `CustomTextLayout` to figure out the advances based on the same font and fallback pattern as the real final layout, just without "rounding" it into cells yet.
- `CustomTextLayout` has been updated to move the advance-correction into a separate phase from glyph shaping. Previously, we corrected the advances to nice round cell counts during shaping, which is fine for drawing, but hard for column count analysis.
- Now that there are separate phases, an `Analyze` method was added to the `CustomTextLayout` which just performs the text analysis steps and the glyph shaping, but no advance correction to column boundaries nor actual drawing.
I've taken the caching code that I was working on to improve chafa, and I've brought it into this. Now that we're doing a lot of fallback and heavy lifting in terms of analysis via the layout, we should cache the results until the font changes.
I've adjusted how column counting is done overall. It's always been in these phases:
1. We used a quick-lookup of ranges of characters we knew to rapidly decide `Narrow`, `Wide` or `Invalid` (a.k.a. "I dunno")
2. If it was `Invalid`, we consulted a table based off of the Unicode standard that has either `Narrow`, `Wide`, or `Ambiguous` as a result.
3. If it's still `Ambiguous`, we consult a render engine fallback (usually GDI or now DX) to see how many columns it would take.
4. If we still don't know, then it's `Wide` to be safe.
- I've added an additional flow here. The quick-lookup can now return `Ambiguous` off the bat for some glyph characters in the x2000-x3000 range that used to just be simple shapes but have been retroactively recategorized as emoji and are frequently now using full width color glyphs.
- This new state causes the lookup to go immediately to the render engine if it is available instead of consulting the Unicode standard table first because the half/fullwidth table doesn't appear to have been updated for this nuance to reclass these characters as ambiguous, but we'd like to keep that table as a "generated from the spec" sort of table and keep our exceptions in the "quick lookup" function.
I have confirmed the following things "just work" now:
- The windows logo flag from the demo. (⚫⚪💖✅🌌😊)
- The dotted chart on the side of crossterm demo (•)
- The powerline characters that make arrows with the Consolas patched font (██)
- An accented é
- The warning and checkmark symbols appearing same size as the X. (✔⚠🔥)
Related work items: #21167256, #21237515, #21243859, #21274645, #21296827
