## Summary of the Pull Request
Updates our `UiaTextRange` to no longer treat the end of the buffer as the "document end". Instead, we consider the "document end" to be the line beneath the cursor or last legible character (whichever is further down). In the event where the last legible character is on the last line of the buffer, we use the "end exclusive" position (left-most point on a line one past the end of the buffer).
When movement of any kind occurs, we clamp each endpoint to the document end. Since the document end is an actual spot in the buffer (most of the time), this should improve stability because we shouldn't be pointing out-of-bounds anymore.
The biggest benefit is that this significantly improves the performance of word navigation because screen readers no longer have to take into account the whitespace following the end of the prompt.
Word navigation tests were added to the `TestTableWriter` (see #10886). 24 of the 85 tests were failing, however, they don't seem to interact with the document end, so I've marked them as skip and will fix them in a follow-up. This PR is large enough as-is, so I'm hoping I can take time in the follow-up to clean some things on the side (aka `preventBoundary` and `allowBottomExclusive` being used interchangeably).
## References
#7000 - Epic
Closes#6986Closes#10925
## Validation Steps Performed
- [X] Tests pass
- [X] @codeofdusk has been personally testing this build (and others)
## Summary of the Pull Request
Follow-up for #10886. The new UIA movement tests found some failing cases. This PR fixes UiaTextRangeBase to have movement match that of MS Word. In total, this fixes 64 tests.
## PR Checklist
* [X] Closes#10924
* [X] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
Root causes include...
1. if we were a non-degenerate range and we failed to move, we should still expand to enclose the unit
2. non-degenerate ranges are treated as if they already encompassed their given unit.
- this one is a bit difficult to explain. Consider these examples:
1. document movement
- state: you have a 1-cell wide range on the buffer, and you try to move by document
- result: move by 0 (there is no next/prev document), but the range now encompasses the entire document
2. line movement
- state: you have a 1-cell wide range on a line, and you try to move back by a line
- result: you go to the previous line (not the beginning of this line)
- conversely, a degenerate range successfully moves to the beginning/end of the current unit (i.e. document/line)
- this (bizarre) behavior was confirmed using MS Word
As a bonus, occasionally, Narrator would get stuck when navigating by line. This issue now seems to be fixed.
## Updates to existing tests
- `CanMoveByCharacter`
- `can't move backward from (0, 0)` --> misauthored, result should be one character wide.
- `can't move past the last column in the last row` --> misauthored and already covered in generated tests
- `CanMoveByLine`
- `can't move backward from top row` --> misauthored, end should be on next line. Already covered by generated tests
- `can't move forward from bottom row` --> misauthored, end should be on next line
- `can't move backward when part of the top row is in the range` --> misauthored, should expand
- `can't move forward when part of the bottom row is in the range` --> misauthored, degenerate range moves to end of buffer
- `MovementAtExclusiveEnd`
- populate the text buffer _before_ we do a move by word operation
- update to match the now fixed behavior
Fixes a bug where interacting with Windows Terminal when using Narrator causes Windows Terminal to hang.
`UiaTextRangeBase::Move()` locks, but later calls `UiaTextRangeBase::ExpandToEnclosingUnit()` which attempts to lock again. The workaround for this is to introduce a `_expandToEnclosingUnit()` that _does not_ lock the console. Then, `Move()` calls this new method, thus only allowing one lock to be established at a time.
This bug is observed to be in v1.11.2221.0 and _not_ in v1.9.1942.0.
## Summary of the Pull Request
This implements `GetAttributeValue` and `FindAttribute` for `UiaTextRangeBase` (the shared `ITextRangeProvider` for Conhost and Windows Terminal). This also updates `UiaTracing` to collect more useful information on these function calls.
## References
#7000 - Epic
[Text Attribute Identifiers](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-textattribute-ids)
[ITextRangeProvider::GetAttributeValue](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-itextrangeprovider-getattributevalue)
[ITextRangeProvider::FindAttribute](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-itextrangeprovider-findattribute)
## PR Checklist
* [X] Closes#2161
* [X] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
- `TextBuffer`:
- Exposes a new `TextBufferCellIterator` that takes in an end position. This simplifies the logic drastically as we can now use this iterator to navigate through the text buffer. The iterator can also expose the position in the buffer.
- `UiaTextRangeBase`:
- Shared logic & helper functions:
- Most of the text attributes are stored as `TextAttribute`s in the text buffer. To extract them, we generate an attribute verification function via `_getAttrVerificationFn()`, then use that to verify if a given cell has the desired attribute.
- A few attributes are special (i.e. font name, font size, and "is read only"), in that they are (1) acquired differently and (2) consistent across the entire text buffer. These are handled separate from the attribute verification function.
- `GetAttributeValue`: Retrieve the attribute verification of the first cell in the range. Then, verify that the entire range has that attribute by iterating through the text range. If a cell does not have that attribute, return the "reserved mixed attribute value".
- `FindAttribute`: Iterate through the text range and leverage the attribute verification function to find the first contiguous range with that attribute. Then, make the end exclusive and output a `UiaTextRangeBase`. This function must be able to perform a search backwards, so we abstract the "start" and "end" into `resultFirstAnchor` and `resultSecondAnchor`, then perform post processing to output a valid `UiaTextRangeBase`.
- `UiaTracing`:
- `GetAttributeValue`: Log uia text range, desired attribute, resulting attribute metadata, and the type of the result.
- `FindAttribute`: Log uia text range, desired attribute and attribute metadata, if we were searching backwards, the type of the result, and the resulting text range.
- `AttributeType` is a nice way to understand/record if the result was either of the reserved UIA values, a normal result, or an error.
- `UiaTextRangeTests`:
- `GetAttributeValue`:
- verify that we know which attributes we support
- test each of the known text attributes (expecting 100% code coverage for `_getAttrVerificationFn()`)
- `FindAttribute`:
- test each of the known _special_ text attributes
- test `IsItalic`. NOTE: I'm explicitly only testing one of the standard text attributes because the logic is largely the same between all of them and they leverage `_getAttrVerificationFn()`.
## Validation Steps Performed
- @codeofdusk has been testing this Conhost build
- Tests added for Conhost and shared implementation
- Windows Terminal changes were manually verified using accessibility insights and NVDA
## Summary of the Pull Request
#7960 was caused by `UiaTextRangeBase::_blockRange` not being initialized, thus pointing to random memory. In most cases, we initialize it properly in `RuntimeClassInitialize`, however, the copying version of `RuntimeClassInitialize` doesn't actually copy it over, resulting in it still containing random memory.
NVDA (and other screen readers) occasionally use `Clone` (really just the copy initializer), resulting in this bug occurring randomly.
## PR Checklist
* [X] Closes#7960
* [X] Tests added/passed
## Validation Steps Performed
Test failed before the change, but passes after the change.
`ScrollIntoView` is responsible for scrolling the viewport to include
the UTR's start endpoint. The crash was caused by `start` being at the
exclusive end, and attempting to scroll to it. This is now fixed by
clamping the result to the bottom of the buffer.
Most of the work here is to allow a test for this. `ScrollIntoView`
relied on a virtual `ChangeViewport` function. By making that
non-virtual, the `DummyElementProvider` in the tests can now be a
`ScreenInfoUiaProviderBase`. This opens up the possibility of more
UiaTextRange tests in the future too.
Closes#7839
## Summary of the Pull Request
In preparation for getting more accessibility-related issues, I added an ID to the `ScreenInfoUiaProvider` (SIUP) and abstracted the one from `UiaTextRange`. Using this, I noticed that we are creating SIUPs when a new tab/pane is created. This is _good_. This means that we need to somehow notify a UIA Client that out structure has changed, and we need to use the new SIUP because the old one has been removed.
I'll be investigating that more after this PR lands.
## Summary of the Pull Request
- Added better wide glyph support for UIA. We used to move one _cell_ at a time, so wide glyphs would be read twice.
- Converted a few things to use til::point since I'm already here.
- fixed telemetry for UIA
## PR Checklist
* [x] Closes#1354
## Detailed Description of the Pull Request / Additional comments
The text buffer has a concept of word boundaries, so it makes sense to have a concept of glyph boundaries too.
_start and _end in UiaTextRange are now til::point
## Validation Steps Performed
Verified using Narrator
For some functions, the overriding implementation is set to default, but
the deletion is not explicitly set at all. For those functions, I
changed default to delete
## Summary of the Pull Request
Block selections were always read and displayed as line selections in UIA. This fixes that.
## PR Checklist
* [x] Closes#4509
## Detailed Description of the Pull Request / Additional comments
1. Expose `IsBlockSelection()` via IUiaData
2. Update the constructor to be able to take in a block selection parameter
3. Make ScreenInfoUiaProviders pass step 1 output into step 2 constructor
4. Update all instances of `UiaTextRange::GetTextRects()` to include this new flag
## Validation Steps Performed
Manually tested.
Additional tests would be redundant as GetTextRects() is tested in the text buffer.
[Git2Git] Merged PR 4314209: Fix some TVS warnings in Console UIA
1. We were doing `FAILED(X || FAILED(Y))` instead of `FAILED(X) || FAILED(Y)`.
Fixes MSFT:24904151. Fixes MSFT:24904224.
2. You cannot SAL-annotate a `gsl::not_null`.
Fixes MSFT:24904221
Related work items: #24904151, #24904221, #24904224 Retrieved from https://microsoft.visualstudio.com os OS official/rs_onecore_dep_uxp 46167d4415c888d4d6a52ea7d3e3cc57a0f5a78d
Related work items: #24904151, #24904221, #24904224
## Summary of the Pull Request
`GetTextForClipboard` already exists in the TextBuffer. It makes sense to use that for UIA as well. This changes the behavior or `GetText()` such that it does not remove leading/trailing whitespace anymore. That is more of an expected behavior.
## References
This also contributes to...
- #4509: UIA Box Selection
- #2447: UIA Signaling for Selection
- #1354: UIA support for Wide Glyphs
Now that the expansion occurs at before render-time, the selection anchors are an accurate representation of what is selected. We just need to move GetText to the TextBuffer. Then we can have those three issues just rely on code from the text buffer. This also means ConHost gets some of this stuff for free 😀
## Detailed Description of the Pull Request / Additional comments
- `TextBuffer::GetTextForClipboard()` --> `GetText()`
- `TextBuffer::GetText()` no longer requires GetForegroundColor/GetBackgroundColor. If either of these are not defined, we return a `TextAndColor` with only the `text` field populated.
- renamed a few parameters for copying text to the clipboard for clarity
- Updated `UiaTextRange::GetText()` to use `TextBuffer::GetText()`
## Validation Steps Performed
Manual tests for UIA using accessibility insights and Windows Terminal's copy action (w/ and w/out shift)
Added tests as well.
This PR hooks up the existing UIA implementation to the WPF control. Some existing code that was specific to the UWP terminal control could be shared so that has been refactored to a common location as well.
## Validation Steps Performed
WPF control was brought up in UISpy and the UIA tree was verified. NVDA was then used to check that screen readers were operating properly.
## Summary of the Pull Request
Debugging our custom UIA providers has been a painful experience because outputting content to VS may result in UIA Clients getting impatient and giving up on extracting data.
Adding tracing allows us to debug these providers without getting in the way of reproducing a bug. This will help immensely with developing accessibility features on Windows Terminal and Console.
This pull request additionally contains payload from #4526:
* Make GetVisibleRanges() return one range (and add tracing for it).
`ScreenInfoUiaProvider::GetVisibleRanges()` used to return one range per line of visible text. The documentation for this function says that we should return one per contiguous span of text. Since all of the text in the TermControl will always be contiguous (at least by our standards), we should only ever be returning one range.
## PR Checklist
* [x] Closes#1914. Closes#4507.
* [x] CLA signed
## Detailed Description of the Pull Request / Additional comments
`UiaTracing` is a singleton class that is in charge of registration for trace logging. `TextRange` is used to trace `UiaTextRange`, whereas `TextProvider` is used to trace `ScreenInfoUiaProviderBase`.
`_getValue()` is overloaded to transform complex objects and enums into a string for logging.
`_getTextValue()` had to be added to be able to trace the text a UiaTextRange included. This makes following UiaTextRanges much simpler.
## Validation Steps Performed
Performed a few operations when under NVDA/Narrator and manually checked the results.
## Summary of the Pull Request
In UIA Providers, update the concept of the size of the text buffer to just go down to the virtual bottom. This significantly increases performance to the point that it can even be used in the Debug build.
## PR Checklist
* [x] Closes#4485
* [x] CLA signed.
## Detailed Description of the Pull Request / Additional comments
We already actually have this concept exposed to us via the IUiaData. So we're just leveraging that and putting it in a helper function `_getBufferSize()`.
## Validation Steps Performed
Tested word nav on Narrator (previously hung). Now it works on the Debug build. Previously, using the release build was necessary to be able to test this feature.
Moved `FindText` to `UiaTextRangeBase`. Now that Search is a shared component (thanks #3279), I can just reuse it basically as-is.
#3279 - Make Search a shared component
#4018 - UiaTextRange Refactor
I removed it from the two different kinds of UiaTextRange and put it in the base class.
I needed a very minor change to ensure we convert from an inclusive end (from Search) to an exclusive end (in UTR).
Worked with `FindText` was globally messed with in windows.h. So we had to do a few weird things there (thanks Michael).
No need for additional tests because it _literally_ just sets up a Searcher and calls it.
## Summary of the Pull Request
This pull request is intended to achieve the following goals...
1) reduce duplicate code
2) remove static functions
3) improve readability
4) improve reliability
5) improve code-coverage for testing
6) establish functioning text buffer navigation in Narrator and NVDA
This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA.
See below for additional context.
## References
#3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here.
#3895 - reduced the duplicate code. No need to separate into different files
#2160 - same as #3976 above
#1993 - I think just about everything is no longer static
## PR Checklist
* [x] Closes#3895, Closes#1993, Closes#3976, Closes#2160
* [x] CLA signed
* [x] Tests added/passed
## Detailed Description of the Pull Request / Additional comments
### UiaTextRange
- converted endpoints into the COORD system in the TextBuffer coordinate space
- `start` is inclusive, `end` is exclusive. A degenerate range is when start == end.
- all functions are no longer static
- `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions
- removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc..
- relied more heavily on existing functionality from `TextBuffer` and `Viewport`
### XamlUiaTextRange
- `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA.
- `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module
### TextBuffer
- Word navigation functionality is entirely in `TextBuffer` for proper abstraction
- a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection
As an example, consider a buffer with this text in it:
" word other "
In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "].
In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "].
Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD.
Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection.
### Viewport
- the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive`
- Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds.
### Testing
- word navigation testing relies more heavily on TextBuffer tests
- additional testing was created for non-movement focused functions of UiaTextRange
- The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results.
## Validation Steps Performed
Tests pass
Narrator works
NVDA works
This pull request also includes build break fixes for things that do not build in the OSS repo
and disables the retro terminal effect in conhost.
Retrieved from https://microsoft.visualstudio.com os OS official/rs_onecore_dep_uxp f10445678e59197c1ae2ee29d8f009c9607c4e5d
Related work items: #24387718
Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English.
# General "Word Movement" Expectations
The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source)
If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards)
# Word Expansion
Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer.
Then I built on top of it by adding an optional additional parameter that decides if you want to include...
- the delimiter run when moving forward
- the character run when moving backwards
It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange
# UiaTextRange
The code is based on character movement. This allows us to actually work with boundary conditions.
The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples:
- Endpoint 0 --> (0,0)
- Endpoint 79 --> (79,0) (when the buffer width is 80)
- Endpoint 80 -->(0,1) (when the buffer width is 80)
- When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards).
- When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end.
# Hooking it up
All we really had to do is add an enum. This part was super easy :)
I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable.
# Defining your own word delimiters
- Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP)
- This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context
- import a word delimiter into the UTR directly
- this provides more control over what a "word" is
- this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths)
The default param of " " is scattered throughout because this is the word delimiter used in the English language.
* Switch all of the UIA providers to WRL::RuntimeClass
Fixes#3209.
References #3051.
Co-authored-by: Carlos Zamora <cazamor@microsoft.com>
Co-authored-by: Dustin Howett <duhowett@microsoft.com>
Refactors the accessibility providers (ScreenInfoUiaProvider and UiaTextRange) into a better separated model between ConHost and Windows Terminal.
ScreenInfoUiaProviderBase and UiaTextRangeBase are introduced. ConHost and Windows Terminal implement their own versions of ScreenInfoUiaProvider and UiaTextRange that inherit from their respective base classes.
WindowsTerminal's ScreenInfoUiaProvider --> TermControlUiaProvider
2019-08-20 16:32:44 -07:00
Renamed from src/types/UiaTextRange.hpp (Browse further)
