Adds a check before every UIA function call to ensure the terminal (specifically the buffer) is initialized before doing work. Both the `ScreenInfoUiaProvider` and the `UiaTextRange` are now covered.
## References
Closes#11135#10971 & #11042
## Detailed Description of the Pull Request / Additional comments
Originally, I tried applying this heuristic to all the `RuntimeClassInitialize` on `UiaTextRangeBase` with the philosophy of "a range pointing to an invalid buffer is invalid itself", but that caused a regression on [MSFT 33353327](https://microsoft.visualstudio.com/OS/_workitems/edit/33353327).
`IUiaData` also has `GetTextBuffer()` return a `TextBuffer&`, which cannot be checked for nullness. Instead, I decided to add a function to `IUiaData` that checks if we have a valid state. Since this is shared with Conhost and Conhost doesn't have this issue, I simply make that function say that it's always in a valid state.
## Validation Steps Performed
- [X] Narrator can detect newly created terminals
- [X] (On Windows Server 2022) Windows Terminal does not hang on launch
We were using std::basic_string_view as a stand-in for std::span so that
we could change over all at once when C++20 dropped with full span
support. That day's not here yet, but as of 54a7fce3e we're using GSL 3,
whose span is C++20-compliant.
This commit replaces every instance of basic_string_view that was not
referring to an actual string with a span of the appropriate type.
I moved the `const` qualifier into span's `T` because while
`basic_string_view.at()` returns `const T&`, `span.at()` returns `T&`
(without the const). I wanted to maintain the invariant that members of
the span were immutable.
* Mechanical Changes
* `sv.at(x)` -> `gsl::at(sp, x)`
* `sv.c{begin,end}` -> `sp.{begin,end}` (span's iterators are const)
I had to replace a `std::basic_string<>` with a `std::vector<>` in
ConImeInfo, and I chose to replace a manual array walk in
ScreenInfoUiaProviderBase with a ranged-for. Please review those
specifically.
This will almost certainly cause a code size regression in Windows
because I'm blowing out all the PGO counts. Whoops.
Related: #3956, #975.
Looking up the size of the viewport from the underlying dimensions of
the structures seemed like a good idea at the time (so it would only be
in one place), but it turns out to be more of a perf cost than we
expected. Not necessarily on any one hot path, but if we sort by
functions in WPR, it was the top consumer on the Terminal side. This
instead saves the size as a member of the `TextBuffer` and serves that
out. It only changes when it is constructed or resized traditionally, so
it's easy to update/keep track of. It impacted conhost/conpty to a
lesser degree but was still noticeable.
## Validation Steps Performed
- Run `time cat big.txt` under WPR. Checked before and after perf
metrics.
## PR Checklist
* [x] Closes perf itch
* [x] I work here
* [x] Manual test
* [x] Documentation irrelevant.
* [x] Schema irrelevant.
* [x] Am core contributor.
This pull request moves WindowUiaProvider back into Win32 interactivity
and deletes all mention of it from Windows Terminal. Terminal does not
have a single toplevel window that requires Console-like UIA, as each
Xaml control inside it is in charge of its own destiny.
I've also merged `IUiaWindow` and `IConsoleWindow` back together, as
well as `WindowUiaProviderBase` and `WindowUiaProvider`.
Things look a lot more like they did before we tore them apart.
## PR Checklist
* [x] Closes#3564
* [x] CLA
* [x] Tests added/passed (manual)
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already
## Validation
Carlos validated conhost and terminal on this branch.
## Summary of the Pull Request
In preparation for getting more accessibility-related issues, I added an ID to the `ScreenInfoUiaProvider` (SIUP) and abstracted the one from `UiaTextRange`. Using this, I noticed that we are creating SIUPs when a new tab/pane is created. This is _good_. This means that we need to somehow notify a UIA Client that out structure has changed, and we need to use the new SIUP because the old one has been removed.
I'll be investigating that more after this PR lands.
## Summary of the Pull Request
- Added better wide glyph support for UIA. We used to move one _cell_ at a time, so wide glyphs would be read twice.
- Converted a few things to use til::point since I'm already here.
- fixed telemetry for UIA
## PR Checklist
* [x] Closes#1354
## Detailed Description of the Pull Request / Additional comments
The text buffer has a concept of word boundaries, so it makes sense to have a concept of glyph boundaries too.
_start and _end in UiaTextRange are now til::point
## Validation Steps Performed
Verified using Narrator
[Git2Git] Merged PR 4314209: Fix some TVS warnings in Console UIA
1. We were doing `FAILED(X || FAILED(Y))` instead of `FAILED(X) || FAILED(Y)`.
Fixes MSFT:24904151. Fixes MSFT:24904224.
2. You cannot SAL-annotate a `gsl::not_null`.
Fixes MSFT:24904221
Related work items: #24904151, #24904221, #24904224 Retrieved from https://microsoft.visualstudio.com os OS official/rs_onecore_dep_uxp 46167d4415c888d4d6a52ea7d3e3cc57a0f5a78d
Related work items: #24904151, #24904221, #24904224
## Summary of the Pull Request
1) Improves the performance of word-recognition operations such as word
navigation in UIA and selection.
2) Fixes a bug where attempting to find the next word in UIA, when none
exists, would hang
3) TraceLogging code only runs when somebody is listening
## Detailed Description of the Pull Request / Additional comments
- The concept of a delimiter class got moved to the CharRow.
- The buffer iterator used to save a lot more information than we needed
- I missed updating a tracing function after making GetSelection return
one text range. That is fixed now.
## Validation Steps Performed
Performed Word Navigation under Narrator and NVDA.
NOTE: The release build should be used when testing to optimize
performance
Closes#4703
## Summary of the Pull Request
Debugging our custom UIA providers has been a painful experience because outputting content to VS may result in UIA Clients getting impatient and giving up on extracting data.
Adding tracing allows us to debug these providers without getting in the way of reproducing a bug. This will help immensely with developing accessibility features on Windows Terminal and Console.
This pull request additionally contains payload from #4526:
* Make GetVisibleRanges() return one range (and add tracing for it).
`ScreenInfoUiaProvider::GetVisibleRanges()` used to return one range per line of visible text. The documentation for this function says that we should return one per contiguous span of text. Since all of the text in the TermControl will always be contiguous (at least by our standards), we should only ever be returning one range.
## PR Checklist
* [x] Closes#1914. Closes#4507.
* [x] CLA signed
## Detailed Description of the Pull Request / Additional comments
`UiaTracing` is a singleton class that is in charge of registration for trace logging. `TextRange` is used to trace `UiaTextRange`, whereas `TextProvider` is used to trace `ScreenInfoUiaProviderBase`.
`_getValue()` is overloaded to transform complex objects and enums into a string for logging.
`_getTextValue()` had to be added to be able to trace the text a UiaTextRange included. This makes following UiaTextRanges much simpler.
## Validation Steps Performed
Performed a few operations when under NVDA/Narrator and manually checked the results.
## Summary of the Pull Request
We used to return multiple text ranges to represent one selection. We only support one selection at a time, so we should only return one range.
Additionally, I moved all TriggerSelection() calls to the renderer from Terminal to TermControl for consistency. This ensures we only call it _once_ when we make a change to our selection state.
## References
#2447 - helps polish Signaling for Selection
#4465 - This is more apparent as the problem holding back Signaling for Selection
## PR Checklist
* [x] Closes#4452
Tested using Accessibility Insights.
## Summary of the Pull Request
This pull request is intended to achieve the following goals...
1) reduce duplicate code
2) remove static functions
3) improve readability
4) improve reliability
5) improve code-coverage for testing
6) establish functioning text buffer navigation in Narrator and NVDA
This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA.
See below for additional context.
## References
#3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here.
#3895 - reduced the duplicate code. No need to separate into different files
#2160 - same as #3976 above
#1993 - I think just about everything is no longer static
## PR Checklist
* [x] Closes#3895, Closes#1993, Closes#3976, Closes#2160
* [x] CLA signed
* [x] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
### UiaTextRange
- converted endpoints into the COORD system in the TextBuffer coordinate space
- `start` is inclusive, `end` is exclusive. A degenerate range is when start == end.
- all functions are no longer static
- `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions
- removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc..
- relied more heavily on existing functionality from `TextBuffer` and `Viewport`
### XamlUiaTextRange
- `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA.
- `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module
### TextBuffer
- Word navigation functionality is entirely in `TextBuffer` for proper abstraction
- a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection
As an example, consider a buffer with this text in it:
" word other "
In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "].
In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "].
Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD.
Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection.
### Viewport
- the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive`
- Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds.
### Testing
- word navigation testing relies more heavily on TextBuffer tests
- additional testing was created for non-movement focused functions of UiaTextRange
- The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results.
## Validation Steps Performed
Tests pass
Narrator works
NVDA works
Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English.
# General "Word Movement" Expectations
The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source)
If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards)
# Word Expansion
Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer.
Then I built on top of it by adding an optional additional parameter that decides if you want to include...
- the delimiter run when moving forward
- the character run when moving backwards
It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange
# UiaTextRange
The code is based on character movement. This allows us to actually work with boundary conditions.
The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples:
- Endpoint 0 --> (0,0)
- Endpoint 79 --> (79,0) (when the buffer width is 80)
- Endpoint 80 -->(0,1) (when the buffer width is 80)
- When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards).
- When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end.
# Hooking it up
All we really had to do is add an enum. This part was super easy :)
I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable.
# Defining your own word delimiters
- Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP)
- This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context
- import a word delimiter into the UTR directly
- this provides more control over what a "word" is
- this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths)
The default param of " " is scattered throughout because this is the word delimiter used in the English language.
* Switch all of the UIA providers to WRL::RuntimeClass
Fixes#3209.
References #3051.
Co-authored-by: Carlos Zamora <cazamor@microsoft.com>
Co-authored-by: Dustin Howett <duhowett@microsoft.com>
Refactors the accessibility providers (ScreenInfoUiaProvider and UiaTextRange) into a better separated model between ConHost and Windows Terminal.
ScreenInfoUiaProviderBase and UiaTextRangeBase are introduced. ConHost and Windows Terminal implement their own versions of ScreenInfoUiaProvider and UiaTextRange that inherit from their respective base classes.
WindowsTerminal's ScreenInfoUiaProvider --> TermControlUiaProvider
2019-08-20 16:32:44 -07:00
Renamed from src/types/ScreenInfoUiaProvider.cpp (Browse further)
