## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
474 lines
15 KiB
C++
474 lines
15 KiB
C++
// Copyright (c) Microsoft Corporation.
|
|
// Licensed under the MIT license.
|
|
|
|
#include "precomp.h"
|
|
|
|
#include "ScreenInfoUiaProviderBase.h"
|
|
#include "WindowUiaProviderBase.hpp"
|
|
|
|
using namespace Microsoft::Console::Types;
|
|
using namespace Microsoft::Console::Types::ScreenInfoUiaProviderTracing;
|
|
|
|
// A helper function to create a SafeArray Version of an int array of a specified length
|
|
SAFEARRAY* BuildIntSafeArray(std::basic_string_view<int> data)
|
|
{
|
|
SAFEARRAY* psa = SafeArrayCreateVector(VT_I4, 0, gsl::narrow<ULONG>(data.size()));
|
|
if (psa != nullptr)
|
|
{
|
|
for (size_t i = 0; i < data.size(); i++)
|
|
{
|
|
LONG lIndex = 0;
|
|
if (FAILED(SizeTToLong(i, &lIndex) ||
|
|
FAILED(SafeArrayPutElement(psa, &lIndex, (void*)&(data.at(i))))))
|
|
{
|
|
SafeArrayDestroy(psa);
|
|
psa = nullptr;
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
|
|
return psa;
|
|
}
|
|
|
|
#pragma warning(suppress : 26434) // WRL RuntimeClassInitialize base is a no-op and we need this for MakeAndInitialize
|
|
HRESULT ScreenInfoUiaProviderBase::RuntimeClassInitialize(_In_ IUiaData* pData, _In_ std::wstring_view wordDelimiters) noexcept
|
|
try
|
|
{
|
|
RETURN_HR_IF_NULL(E_INVALIDARG, pData);
|
|
_pData = pData;
|
|
_wordDelimiters = wordDelimiters;
|
|
return S_OK;
|
|
}
|
|
CATCH_RETURN();
|
|
|
|
[[nodiscard]] HRESULT ScreenInfoUiaProviderBase::Signal(_In_ EVENTID id)
|
|
{
|
|
HRESULT hr = S_OK;
|
|
// check to see if we're already firing this particular event
|
|
if (_signalFiringMapping.find(id) != _signalFiringMapping.end() &&
|
|
_signalFiringMapping[id] == true)
|
|
{
|
|
return hr;
|
|
}
|
|
|
|
try
|
|
{
|
|
_signalFiringMapping[id] = true;
|
|
}
|
|
CATCH_RETURN();
|
|
|
|
IRawElementProviderSimple* pProvider = this;
|
|
hr = UiaRaiseAutomationEvent(pProvider, id);
|
|
_signalFiringMapping[id] = false;
|
|
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
// tracing
|
|
/*ApiMsgSignal apiMsg;
|
|
apiMsg.Signal = id;
|
|
Tracing::s_TraceUia(this, ApiCall::Signal, &apiMsg);*/
|
|
return hr;
|
|
}
|
|
|
|
#pragma region IRawElementProviderSimple
|
|
|
|
// Implementation of IRawElementProviderSimple::get_ProviderOptions.
|
|
// Gets UI Automation provider options.
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::get_ProviderOptions(_Out_ ProviderOptions* pOptions) noexcept
|
|
{
|
|
RETURN_HR_IF_NULL(E_INVALIDARG, pOptions);
|
|
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetProviderOptions, nullptr);
|
|
*pOptions = ProviderOptions_ServerSideProvider;
|
|
return S_OK;
|
|
}
|
|
|
|
// Implementation of IRawElementProviderSimple::get_PatternProvider.
|
|
// Gets the object that supports ISelectionPattern.
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetPatternProvider(_In_ PATTERNID patternId,
|
|
_COM_Outptr_result_maybenull_ IUnknown** ppInterface)
|
|
{
|
|
RETURN_HR_IF(E_INVALIDARG, ppInterface == nullptr);
|
|
*ppInterface = nullptr;
|
|
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetPatternProvider, nullptr);
|
|
|
|
*ppInterface = nullptr;
|
|
HRESULT hr = S_OK;
|
|
|
|
if (patternId == UIA_TextPatternId)
|
|
{
|
|
hr = QueryInterface(IID_PPV_ARGS(ppInterface));
|
|
if (FAILED(hr))
|
|
{
|
|
*ppInterface = nullptr;
|
|
}
|
|
}
|
|
return hr;
|
|
}
|
|
|
|
// Implementation of IRawElementProviderSimple::get_PropertyValue.
|
|
// Gets custom properties.
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetPropertyValue(_In_ PROPERTYID propertyId,
|
|
_Out_ VARIANT* pVariant) noexcept
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetPropertyValue, nullptr);
|
|
|
|
pVariant->vt = VT_EMPTY;
|
|
|
|
// Returning the default will leave the property as the default
|
|
// so we only really need to touch it for the properties we want to implement
|
|
if (propertyId == UIA_ControlTypePropertyId)
|
|
{
|
|
// This control is the Document control type, implying that it is
|
|
// a complex document that supports text pattern
|
|
pVariant->vt = VT_I4;
|
|
pVariant->lVal = UIA_DocumentControlTypeId;
|
|
}
|
|
else if (propertyId == UIA_NamePropertyId)
|
|
{
|
|
// TODO: MSFT: 7960168 - These strings should be localized text in the final UIA work
|
|
pVariant->bstrVal = SysAllocString(L"Text Area");
|
|
if (pVariant->bstrVal != nullptr)
|
|
{
|
|
pVariant->vt = VT_BSTR;
|
|
}
|
|
}
|
|
else if (propertyId == UIA_AutomationIdPropertyId)
|
|
{
|
|
pVariant->bstrVal = SysAllocString(L"Text Area");
|
|
if (pVariant->bstrVal != nullptr)
|
|
{
|
|
pVariant->vt = VT_BSTR;
|
|
}
|
|
}
|
|
else if (propertyId == UIA_IsControlElementPropertyId)
|
|
{
|
|
pVariant->vt = VT_BOOL;
|
|
pVariant->boolVal = VARIANT_TRUE;
|
|
}
|
|
else if (propertyId == UIA_IsContentElementPropertyId)
|
|
{
|
|
pVariant->vt = VT_BOOL;
|
|
pVariant->boolVal = VARIANT_TRUE;
|
|
}
|
|
else if (propertyId == UIA_IsKeyboardFocusablePropertyId)
|
|
{
|
|
pVariant->vt = VT_BOOL;
|
|
pVariant->boolVal = VARIANT_TRUE;
|
|
}
|
|
else if (propertyId == UIA_HasKeyboardFocusPropertyId)
|
|
{
|
|
pVariant->vt = VT_BOOL;
|
|
pVariant->boolVal = VARIANT_TRUE;
|
|
}
|
|
else if (propertyId == UIA_ProviderDescriptionPropertyId)
|
|
{
|
|
pVariant->bstrVal = SysAllocString(L"Microsoft Console Host: Screen Information Text Area");
|
|
if (pVariant->bstrVal != nullptr)
|
|
{
|
|
pVariant->vt = VT_BSTR;
|
|
}
|
|
}
|
|
else if (propertyId == UIA_IsEnabledPropertyId)
|
|
{
|
|
pVariant->vt = VT_BOOL;
|
|
pVariant->boolVal = VARIANT_TRUE;
|
|
}
|
|
|
|
return S_OK;
|
|
}
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::get_HostRawElementProvider(_COM_Outptr_result_maybenull_ IRawElementProviderSimple** ppProvider) noexcept
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetHostRawElementProvider, nullptr);
|
|
RETURN_HR_IF(E_INVALIDARG, ppProvider == nullptr);
|
|
*ppProvider = nullptr;
|
|
|
|
return S_OK;
|
|
}
|
|
#pragma endregion
|
|
|
|
#pragma region IRawElementProviderFragment
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetRuntimeId(_Outptr_result_maybenull_ SAFEARRAY** ppRuntimeId)
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetRuntimeId, nullptr);
|
|
|
|
// Root defers this to host, others must implement it...
|
|
RETURN_HR_IF(E_INVALIDARG, ppRuntimeId == nullptr);
|
|
*ppRuntimeId = nullptr;
|
|
|
|
// AppendRuntimeId is a magic Number that tells UIAutomation to Append its own Runtime ID(From the HWND)
|
|
const std::array<int, 2> rId{ UiaAppendRuntimeId, -1 };
|
|
|
|
const std::basic_string_view<int> span{ rId.data(), rId.size() };
|
|
// BuildIntSafeArray is a custom function to hide the SafeArray creation
|
|
*ppRuntimeId = BuildIntSafeArray(span);
|
|
RETURN_IF_NULL_ALLOC(*ppRuntimeId);
|
|
|
|
return S_OK;
|
|
}
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetEmbeddedFragmentRoots(_Outptr_result_maybenull_ SAFEARRAY** ppRoots) noexcept
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetEmbeddedFragmentRoots, nullptr);
|
|
|
|
RETURN_HR_IF(E_INVALIDARG, ppRoots == nullptr);
|
|
*ppRoots = nullptr;
|
|
return S_OK;
|
|
}
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::SetFocus()
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::SetFocus, nullptr);
|
|
|
|
return Signal(UIA_AutomationFocusChangedEventId);
|
|
}
|
|
|
|
#pragma endregion
|
|
|
|
#pragma region ITextProvider
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetSelection(_Outptr_result_maybenull_ SAFEARRAY** ppRetVal)
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//ApiMsgGetSelection apiMsg;
|
|
|
|
_LockConsole();
|
|
auto Unlock = wil::scope_exit([&]() noexcept {
|
|
_UnlockConsole();
|
|
});
|
|
|
|
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
HRESULT hr = S_OK;
|
|
|
|
if (!_pData->IsSelectionActive())
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//apiMsg.AreaSelected = false;
|
|
//apiMsg.SelectionRowCount = 1;
|
|
|
|
// return a degenerate range at the cursor position
|
|
const Cursor& cursor = _getTextBuffer().GetCursor();
|
|
|
|
// make a safe array
|
|
*ppRetVal = SafeArrayCreateVector(VT_UNKNOWN, 0, 1);
|
|
if (*ppRetVal == nullptr)
|
|
{
|
|
return E_OUTOFMEMORY;
|
|
}
|
|
|
|
if (FAILED(hr))
|
|
{
|
|
SafeArrayDestroy(*ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
return hr;
|
|
}
|
|
|
|
WRL::ComPtr<UiaTextRangeBase> range;
|
|
hr = CreateTextRange(this, cursor, _wordDelimiters, &range);
|
|
if (FAILED(hr))
|
|
{
|
|
SafeArrayDestroy(*ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
return hr;
|
|
}
|
|
|
|
LONG currentIndex = 0;
|
|
hr = SafeArrayPutElement(*ppRetVal, ¤tIndex, range.Detach());
|
|
if (FAILED(hr))
|
|
{
|
|
SafeArrayDestroy(*ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
return hr;
|
|
}
|
|
}
|
|
else
|
|
{
|
|
// get the selection ranges
|
|
std::deque<WRL::ComPtr<UiaTextRangeBase>> ranges;
|
|
RETURN_IF_FAILED(GetSelectionRanges(this, _wordDelimiters, ranges));
|
|
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//apiMsg.AreaSelected = true;
|
|
//apiMsg.SelectionRowCount = static_cast<unsigned int>(ranges.size());
|
|
|
|
// make a safe array
|
|
*ppRetVal = SafeArrayCreateVector(VT_UNKNOWN, 0, gsl::narrow<ULONG>(ranges.size()));
|
|
if (*ppRetVal == nullptr)
|
|
{
|
|
return E_OUTOFMEMORY;
|
|
}
|
|
|
|
// fill the safe array
|
|
for (LONG i = 0; i < gsl::narrow<LONG>(ranges.size()); ++i)
|
|
{
|
|
hr = SafeArrayPutElement(*ppRetVal, &i, ranges.at(i).Detach());
|
|
if (FAILED(hr))
|
|
{
|
|
SafeArrayDestroy(*ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
return hr;
|
|
}
|
|
}
|
|
}
|
|
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetSelection, &apiMsg);
|
|
return S_OK;
|
|
}
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetVisibleRanges(_Outptr_result_maybenull_ SAFEARRAY** ppRetVal)
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetVisibleRanges, nullptr);
|
|
|
|
_LockConsole();
|
|
auto Unlock = wil::scope_exit([&]() noexcept {
|
|
_UnlockConsole();
|
|
});
|
|
|
|
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
|
|
const auto bufferSize = _pData->GetTextBuffer().GetSize();
|
|
const auto viewport = bufferSize.ConvertToOrigin(_getViewport());
|
|
|
|
// make a safe array
|
|
const auto rowCount = viewport.Height();
|
|
*ppRetVal = SafeArrayCreateVector(VT_UNKNOWN, 0, gsl::narrow<ULONG>(rowCount));
|
|
if (*ppRetVal == nullptr)
|
|
{
|
|
return E_OUTOFMEMORY;
|
|
}
|
|
|
|
// stuff each visible line in the safearray
|
|
for (short i = 0; i < rowCount; ++i)
|
|
{
|
|
// end is exclusive so add 1
|
|
const COORD start{ viewport.Left(), viewport.Top() + i };
|
|
const COORD end{ start.X, start.Y + 1 };
|
|
|
|
HRESULT hr = S_OK;
|
|
WRL::ComPtr<UiaTextRangeBase> range;
|
|
hr = CreateTextRange(this,
|
|
start,
|
|
end,
|
|
_wordDelimiters,
|
|
&range);
|
|
if (FAILED(hr))
|
|
{
|
|
SafeArrayDestroy(*ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
return hr;
|
|
}
|
|
|
|
LONG currentIndex = gsl::narrow<LONG>(i);
|
|
hr = SafeArrayPutElement(*ppRetVal, ¤tIndex, range.Detach());
|
|
if (FAILED(hr))
|
|
{
|
|
SafeArrayDestroy(*ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
return hr;
|
|
}
|
|
}
|
|
return S_OK;
|
|
}
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::RangeFromChild(_In_ IRawElementProviderSimple* /*childElement*/,
|
|
_COM_Outptr_result_maybenull_ ITextRangeProvider** ppRetVal)
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::RangeFromChild, nullptr);
|
|
|
|
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
|
|
WRL::ComPtr<UiaTextRangeBase> utr;
|
|
RETURN_IF_FAILED(CreateTextRange(this, _wordDelimiters, &utr));
|
|
RETURN_IF_FAILED(utr.CopyTo(ppRetVal));
|
|
return S_OK;
|
|
}
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::RangeFromPoint(_In_ UiaPoint point,
|
|
_COM_Outptr_result_maybenull_ ITextRangeProvider** ppRetVal)
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::RangeFromPoint, nullptr);
|
|
|
|
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
|
|
WRL::ComPtr<UiaTextRangeBase> utr;
|
|
RETURN_IF_FAILED(CreateTextRange(this,
|
|
point,
|
|
_wordDelimiters,
|
|
&utr));
|
|
RETURN_IF_FAILED(utr.CopyTo(ppRetVal));
|
|
return S_OK;
|
|
}
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::get_DocumentRange(_COM_Outptr_result_maybenull_ ITextRangeProvider** ppRetVal)
|
|
{
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetDocumentRange, nullptr);
|
|
|
|
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
|
|
*ppRetVal = nullptr;
|
|
|
|
WRL::ComPtr<UiaTextRangeBase> utr;
|
|
RETURN_IF_FAILED(CreateTextRange(this, _wordDelimiters, &utr));
|
|
RETURN_IF_FAILED(utr->ExpandToEnclosingUnit(TextUnit::TextUnit_Document));
|
|
RETURN_IF_FAILED(utr.CopyTo(ppRetVal));
|
|
return S_OK;
|
|
}
|
|
|
|
IFACEMETHODIMP ScreenInfoUiaProviderBase::get_SupportedTextSelection(_Out_ SupportedTextSelection* pRetVal) noexcept
|
|
{
|
|
RETURN_HR_IF_NULL(E_INVALIDARG, pRetVal);
|
|
|
|
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
|
|
//Tracing::s_TraceUia(this, ApiCall::GetSupportedTextSelection, nullptr);
|
|
|
|
*pRetVal = SupportedTextSelection::SupportedTextSelection_Single;
|
|
return S_OK;
|
|
}
|
|
|
|
#pragma endregion
|
|
|
|
const COORD ScreenInfoUiaProviderBase::_getScreenBufferCoords() const
|
|
{
|
|
return _getTextBuffer().GetSize().Dimensions();
|
|
}
|
|
|
|
const TextBuffer& ScreenInfoUiaProviderBase::_getTextBuffer() const noexcept
|
|
{
|
|
return _pData->GetTextBuffer();
|
|
}
|
|
|
|
const Viewport ScreenInfoUiaProviderBase::_getViewport() const noexcept
|
|
{
|
|
return _pData->GetViewport();
|
|
}
|
|
|
|
void ScreenInfoUiaProviderBase::_LockConsole() noexcept
|
|
{
|
|
// TODO GitHub #2141: Lock and Unlock in conhost should decouple Ctrl+C dispatch and use smarter handling
|
|
_pData->LockConsole();
|
|
}
|
|
|
|
void ScreenInfoUiaProviderBase::_UnlockConsole() noexcept
|
|
{
|
|
// TODO GitHub #2141: Lock and Unlock in conhost should decouple Ctrl+C dispatch and use smarter handling
|
|
_pData->UnlockConsole();
|
|
}
|