terminal/src/cascadia/TerminalControl/XamlUiaTextRange.cpp
Carlos Zamora a0e5085b49
Expose Text Attributes to UI Automation (#10336)
## Summary of the Pull Request
This implements `GetAttributeValue` and `FindAttribute` for `UiaTextRangeBase` (the shared `ITextRangeProvider` for Conhost and Windows Terminal). This also updates `UiaTracing` to collect more useful information on these function calls. 

## References
#7000 - Epic
[Text Attribute Identifiers](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-textattribute-ids)
[ITextRangeProvider::GetAttributeValue](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-itextrangeprovider-getattributevalue)
[ITextRangeProvider::FindAttribute](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-itextrangeprovider-findattribute)

## PR Checklist
* [X] Closes #2161 
* [X] Tests added/passed

## Detailed Description of the Pull Request / Additional comments
- `TextBuffer`:
   - Exposes a new `TextBufferCellIterator` that takes in an end position. This simplifies the logic drastically as we can now use this iterator to navigate through the text buffer. The iterator can also expose the position in the buffer.
- `UiaTextRangeBase`:
   - Shared logic & helper functions:
      - Most of the text attributes are stored as `TextAttribute`s in the text buffer. To extract them, we generate an attribute verification function via `_getAttrVerificationFn()`, then use that to verify if a given cell has the desired attribute.
      - A few attributes are special (i.e. font name, font size, and "is read only"), in that they are (1) acquired differently and (2) consistent across the entire text buffer. These are handled separate from the attribute verification function.
   - `GetAttributeValue`: Retrieve the attribute verification of the first cell in the range. Then, verify that the entire range has that attribute by iterating through the text range. If a cell does not have that attribute, return the "reserved mixed attribute value".
   - `FindAttribute`: Iterate through the text range and leverage the attribute verification function to find the first contiguous range with that attribute. Then, make the end exclusive and output a `UiaTextRangeBase`. This function must be able to perform a search backwards, so we abstract the "start" and "end" into `resultFirstAnchor` and `resultSecondAnchor`, then perform post processing to output a valid `UiaTextRangeBase`.
- `UiaTracing`:
   - `GetAttributeValue`: Log uia text range, desired attribute, resulting attribute metadata, and the type of the result.
   - `FindAttribute`: Log uia text range, desired attribute and attribute metadata, if we were searching backwards, the type of the result, and the resulting text range.
   - `AttributeType` is a nice way to understand/record if the result was either of the reserved UIA values, a normal result, or an error.
- `UiaTextRangeTests`:
   - `GetAttributeValue`:
      - verify that we know which attributes we support
      - test each of the known text attributes (expecting 100% code coverage for `_getAttrVerificationFn()`)
   - `FindAttribute`: 
      - test each of the known _special_ text attributes
      - test `IsItalic`. NOTE: I'm explicitly only testing one of the standard text attributes because the logic is largely the same between all of them and they leverage `_getAttrVerificationFn()`.

## Validation Steps Performed
- @codeofdusk has been testing this Conhost build
- Tests added for Conhost and shared implementation
- Windows Terminal changes were manually verified using accessibility insights and NVDA
2021-07-09 23:21:35 +00:00

255 lines
9.8 KiB
C++

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "pch.h"
#include "XamlUiaTextRange.h"
#include "../types/TermControlUiaTextRange.hpp"
#include <UIAutomationClient.h>
#include <UIAutomationCoreApi.h>
// the same as COR_E_NOTSUPPORTED
// we don't want to import the CLR headers to get it
#define XAML_E_NOT_SUPPORTED 0x80131515L
namespace UIA
{
using ::ITextRangeProvider;
using ::SupportedTextSelection;
using ::TextPatternRangeEndpoint;
using ::TextUnit;
}
namespace XamlAutomation
{
using winrt::Windows::UI::Xaml::Automation::SupportedTextSelection;
using winrt::Windows::UI::Xaml::Automation::Provider::IRawElementProviderSimple;
using winrt::Windows::UI::Xaml::Automation::Provider::ITextRangeProvider;
using winrt::Windows::UI::Xaml::Automation::Text::TextPatternRangeEndpoint;
using winrt::Windows::UI::Xaml::Automation::Text::TextUnit;
}
namespace winrt::Microsoft::Terminal::Control::implementation
{
XamlAutomation::ITextRangeProvider XamlUiaTextRange::Clone() const
{
UIA::ITextRangeProvider* pReturn;
THROW_IF_FAILED(_uiaProvider->Clone(&pReturn));
auto xutr = winrt::make_self<XamlUiaTextRange>(pReturn, _parentProvider);
return xutr.as<XamlAutomation::ITextRangeProvider>();
}
bool XamlUiaTextRange::Compare(XamlAutomation::ITextRangeProvider pRange) const
{
auto self = winrt::get_self<XamlUiaTextRange>(pRange);
BOOL returnVal;
THROW_IF_FAILED(_uiaProvider->Compare(self->_uiaProvider.get(), &returnVal));
return returnVal;
}
int32_t XamlUiaTextRange::CompareEndpoints(XamlAutomation::TextPatternRangeEndpoint endpoint,
XamlAutomation::ITextRangeProvider pTargetRange,
XamlAutomation::TextPatternRangeEndpoint targetEndpoint)
{
auto self = winrt::get_self<XamlUiaTextRange>(pTargetRange);
int32_t returnVal;
THROW_IF_FAILED(_uiaProvider->CompareEndpoints(static_cast<UIA::TextPatternRangeEndpoint>(endpoint),
self->_uiaProvider.get(),
static_cast<UIA::TextPatternRangeEndpoint>(targetEndpoint),
&returnVal));
return returnVal;
}
void XamlUiaTextRange::ExpandToEnclosingUnit(XamlAutomation::TextUnit unit) const
{
THROW_IF_FAILED(_uiaProvider->ExpandToEnclosingUnit(static_cast<UIA::TextUnit>(unit)));
}
XamlAutomation::ITextRangeProvider XamlUiaTextRange::FindAttribute(int32_t /*textAttributeId*/,
winrt::Windows::Foundation::IInspectable /*val*/,
bool /*searchBackward*/)
{
// TODO GitHub #2161: potential accessibility improvement
// we don't support this currently
throw winrt::hresult_not_implemented();
}
XamlAutomation::ITextRangeProvider XamlUiaTextRange::FindText(winrt::hstring text,
bool searchBackward,
bool ignoreCase)
{
UIA::ITextRangeProvider* pReturn;
const auto queryText = wil::make_bstr(text.c_str());
THROW_IF_FAILED(_uiaProvider->FindText(queryText.get(), searchBackward, ignoreCase, &pReturn));
auto xutr = winrt::make_self<XamlUiaTextRange>(pReturn, _parentProvider);
return *xutr;
}
winrt::Windows::Foundation::IInspectable XamlUiaTextRange::GetAttributeValue(int32_t textAttributeId) const
{
// Call the function off of the underlying UiaTextRange.
VARIANT result;
THROW_IF_FAILED(_uiaProvider->GetAttributeValue(textAttributeId, &result));
// Convert the resulting VARIANT into a format that is consumable by XAML.
switch (result.vt)
{
case VT_BSTR:
{
return box_value(result.bstrVal);
}
case VT_I4:
{
// Surprisingly, `long` is _not_ a WinRT type.
// So we have to use `int32_t` to make sure this is output properly.
// Otherwise, you'll get "Attribute does not exist" out the other end.
return box_value<int32_t>(result.lVal);
}
case VT_R8:
{
return box_value(result.dblVal);
}
case VT_BOOL:
{
return box_value<bool>(result.boolVal);
}
case VT_UNKNOWN:
{
// This one is particularly special.
// We might return a special value like UiaGetReservedMixedAttributeValue
// or UiaGetReservedNotSupportedValue.
// Some text attributes may return a real value, however, none of those
// are supported at this time.
// So we need to figure out what was actually intended to be returned.
com_ptr<IUnknown> mixedAttributeVal;
UiaGetReservedMixedAttributeValue(mixedAttributeVal.put());
if (result.punkVal == mixedAttributeVal.get())
{
return Windows::UI::Xaml::DependencyProperty::UnsetValue();
}
[[fallthrough]];
}
default:
{
// We _need_ to return XAML_E_NOT_SUPPORTED here.
// Returning nullptr is an improper implementation of it being unsupported.
// UIA Clients rely on this HRESULT to signify that the requested attribute is undefined.
// Anything else will result in the UIA Client refusing to read when navigating by word
// Magically, this doesn't affect other forms of navigation...
winrt::throw_hresult(XAML_E_NOT_SUPPORTED);
}
}
}
void XamlUiaTextRange::GetBoundingRectangles(com_array<double>& returnValue) const
{
returnValue = {};
try
{
SAFEARRAY* pReturnVal;
THROW_IF_FAILED(_uiaProvider->GetBoundingRectangles(&pReturnVal));
double* pVals;
THROW_IF_FAILED(SafeArrayAccessData(pReturnVal, (void**)&pVals));
long lBound, uBound;
THROW_IF_FAILED(SafeArrayGetLBound(pReturnVal, 1, &lBound));
THROW_IF_FAILED(SafeArrayGetUBound(pReturnVal, 1, &uBound));
long count = uBound - lBound + 1;
std::vector<double> vec;
vec.reserve(count);
for (int i = 0; i < count; i++)
{
double element = pVals[i];
vec.push_back(element);
}
winrt::com_array<double> result{ vec };
returnValue = std::move(result);
}
catch (...)
{
}
}
XamlAutomation::IRawElementProviderSimple XamlUiaTextRange::GetEnclosingElement()
{
return _parentProvider;
}
winrt::hstring XamlUiaTextRange::GetText(int32_t maxLength) const
{
BSTR returnVal;
THROW_IF_FAILED(_uiaProvider->GetText(maxLength, &returnVal));
return winrt::to_hstring(returnVal);
}
int32_t XamlUiaTextRange::Move(XamlAutomation::TextUnit unit,
int32_t count)
{
int returnVal;
THROW_IF_FAILED(_uiaProvider->Move(static_cast<UIA::TextUnit>(unit),
count,
&returnVal));
return returnVal;
}
int32_t XamlUiaTextRange::MoveEndpointByUnit(XamlAutomation::TextPatternRangeEndpoint endpoint,
XamlAutomation::TextUnit unit,
int32_t count) const
{
int returnVal;
THROW_IF_FAILED(_uiaProvider->MoveEndpointByUnit(static_cast<UIA::TextPatternRangeEndpoint>(endpoint),
static_cast<UIA::TextUnit>(unit),
count,
&returnVal));
return returnVal;
}
void XamlUiaTextRange::MoveEndpointByRange(XamlAutomation::TextPatternRangeEndpoint endpoint,
XamlAutomation::ITextRangeProvider pTargetRange,
XamlAutomation::TextPatternRangeEndpoint targetEndpoint) const
{
auto self = winrt::get_self<XamlUiaTextRange>(pTargetRange);
THROW_IF_FAILED(_uiaProvider->MoveEndpointByRange(static_cast<UIA::TextPatternRangeEndpoint>(endpoint),
/*pTargetRange*/ self->_uiaProvider.get(),
static_cast<UIA::TextPatternRangeEndpoint>(targetEndpoint)));
}
void XamlUiaTextRange::Select() const
{
THROW_IF_FAILED(_uiaProvider->Select());
}
void XamlUiaTextRange::AddToSelection() const
{
// we don't support this
throw winrt::hresult_not_implemented();
}
void XamlUiaTextRange::RemoveFromSelection() const
{
// we don't support this
throw winrt::hresult_not_implemented();
}
void XamlUiaTextRange::ScrollIntoView(bool alignToTop) const
{
THROW_IF_FAILED(_uiaProvider->ScrollIntoView(alignToTop));
}
winrt::com_array<XamlAutomation::IRawElementProviderSimple> XamlUiaTextRange::GetChildren() const
{
// we don't have any children
return {};
}
}