Commit graph

5 commits

Author SHA1 Message Date
Carlos Zamora a0e5085b49
Expose Text Attributes to UI Automation (#10336)
## Summary of the Pull Request
This implements `GetAttributeValue` and `FindAttribute` for `UiaTextRangeBase` (the shared `ITextRangeProvider` for Conhost and Windows Terminal). This also updates `UiaTracing` to collect more useful information on these function calls. 

## References
#7000 - Epic
[Text Attribute Identifiers](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-textattribute-ids)
[ITextRangeProvider::GetAttributeValue](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-itextrangeprovider-getattributevalue)
[ITextRangeProvider::FindAttribute](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-itextrangeprovider-findattribute)

## PR Checklist
* [X] Closes #2161 
* [X] Tests added/passed

## Detailed Description of the Pull Request / Additional comments
- `TextBuffer`:
   - Exposes a new `TextBufferCellIterator` that takes in an end position. This simplifies the logic drastically as we can now use this iterator to navigate through the text buffer. The iterator can also expose the position in the buffer.
- `UiaTextRangeBase`:
   - Shared logic & helper functions:
      - Most of the text attributes are stored as `TextAttribute`s in the text buffer. To extract them, we generate an attribute verification function via `_getAttrVerificationFn()`, then use that to verify if a given cell has the desired attribute.
      - A few attributes are special (i.e. font name, font size, and "is read only"), in that they are (1) acquired differently and (2) consistent across the entire text buffer. These are handled separate from the attribute verification function.
   - `GetAttributeValue`: Retrieve the attribute verification of the first cell in the range. Then, verify that the entire range has that attribute by iterating through the text range. If a cell does not have that attribute, return the "reserved mixed attribute value".
   - `FindAttribute`: Iterate through the text range and leverage the attribute verification function to find the first contiguous range with that attribute. Then, make the end exclusive and output a `UiaTextRangeBase`. This function must be able to perform a search backwards, so we abstract the "start" and "end" into `resultFirstAnchor` and `resultSecondAnchor`, then perform post processing to output a valid `UiaTextRangeBase`.
- `UiaTracing`:
   - `GetAttributeValue`: Log uia text range, desired attribute, resulting attribute metadata, and the type of the result.
   - `FindAttribute`: Log uia text range, desired attribute and attribute metadata, if we were searching backwards, the type of the result, and the resulting text range.
   - `AttributeType` is a nice way to understand/record if the result was either of the reserved UIA values, a normal result, or an error.
- `UiaTextRangeTests`:
   - `GetAttributeValue`:
      - verify that we know which attributes we support
      - test each of the known text attributes (expecting 100% code coverage for `_getAttrVerificationFn()`)
   - `FindAttribute`: 
      - test each of the known _special_ text attributes
      - test `IsItalic`. NOTE: I'm explicitly only testing one of the standard text attributes because the logic is largely the same between all of them and they leverage `_getAttrVerificationFn()`.

## Validation Steps Performed
- @codeofdusk has been testing this Conhost build
- Tests added for Conhost and shared implementation
- Windows Terminal changes were manually verified using accessibility insights and NVDA
2021-07-09 23:21:35 +00:00
Carlos Zamora 286af380c9
Add more object ID tracing for Accessibility (#5215)
## Summary of the Pull Request

In preparation for getting more accessibility-related issues, I added an ID to the `ScreenInfoUiaProvider` (SIUP) and abstracted the one from `UiaTextRange`. Using this, I noticed that we are creating SIUPs when a new tab/pane is created. This is _good_. This means that we need to somehow notify a UIA Client that out structure has changed, and we need to use the new SIUP because the old one has been removed.

I'll be investigating that more after this PR lands.
2020-04-03 20:06:47 +00:00
Carlos Zamora eca0c6b327
Fire UIA Events for Output and Cursor Movement (#4826)
## Summary of the Pull Request
This notifies automation clients (i.e.: NVDA, narrator, etc...) of new output being rendered to the screen.

## References
Close #2447 - Signaling for new output and cursor
Close #3791 - fixed by signaling cursor changes

## Detailed Description of the Pull Request / Additional comments
- Added tracing for UiaRenderer. This makes it easier to debug issues with notifying an automation client.
- Fire TextChanged automation events when new content is output to the screen.

## Validation Steps Performed
Verified with NVDA [1]

## Narrator
Narrator works _better_, but is unable to detect new output consistently. There is no harm for narrator when this change goes in.

[1] https://github.com/microsoft/terminal/issues/2447#issuecomment-595879890
2020-03-18 18:01:05 +00:00
msftbot[bot] 267deaaf70
Improve Word Navigation/Selection Performance (#4797)
## Summary of the Pull Request
1) Improves the performance of word-recognition operations such as word
   navigation in UIA and selection.

2) Fixes a bug where attempting to find the next word in UIA, when none
   exists, would hang

3) TraceLogging code only runs when somebody is listening

## Detailed Description of the Pull Request / Additional comments
- The concept of a delimiter class got moved to the CharRow.
- The buffer iterator used to save a lot more information than we needed
- I missed updating a tracing function after making GetSelection return
  one text range. That is fixed now.


## Validation Steps Performed
Performed Word Navigation under Narrator and NVDA.
NOTE: The release build should be used when testing to optimize
performance

Closes #4703
2020-03-04 23:10:10 +00:00
Carlos Zamora 360c655acc
UIA: Fix GetVisibleRanges() and add Tracing (#4495)
## Summary of the Pull Request
Debugging our custom UIA providers has been a painful experience because outputting content to VS may result in UIA Clients getting impatient and giving up on extracting data.

Adding tracing allows us to debug these providers without getting in the way of reproducing a bug. This will help immensely with developing accessibility features on Windows Terminal and Console.

This pull request additionally contains payload from #4526:
* Make GetVisibleRanges() return one range (and add tracing for it).
`ScreenInfoUiaProvider::GetVisibleRanges()` used to return one range per line of visible text. The documentation for this function says that we should return one per contiguous span of text. Since all of the text in the TermControl will always be contiguous (at least by our standards), we should only ever be returning one range.

## PR Checklist
* [x] Closes #1914. Closes #4507.
* [x] CLA signed

## Detailed Description of the Pull Request / Additional comments
`UiaTracing` is a singleton class that is in charge of registration for trace logging. `TextRange` is used to trace `UiaTextRange`, whereas `TextProvider` is used to trace `ScreenInfoUiaProviderBase`.

`_getValue()` is overloaded to transform complex objects and enums into a string for logging.

`_getTextValue()` had to be added to be able to trace the text a UiaTextRange included. This makes following UiaTextRanges much simpler.

## Validation Steps Performed
Performed a few operations when under NVDA/Narrator and manually checked the results.
2020-02-20 23:50:43 +00:00