terminal/src/types/ScreenInfoUiaProviderBase.cpp

474 lines
15 KiB
C++
Raw Normal View History

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "ScreenInfoUiaProviderBase.h"
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
#include "WindowUiaProviderBase.hpp"
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
using namespace Microsoft::Console::Types;
using namespace Microsoft::Console::Types::ScreenInfoUiaProviderTracing;
// A helper function to create a SafeArray Version of an int array of a specified length
SAFEARRAY* BuildIntSafeArray(std::basic_string_view<int> data)
{
SAFEARRAY* psa = SafeArrayCreateVector(VT_I4, 0, gsl::narrow<ULONG>(data.size()));
if (psa != nullptr)
{
for (size_t i = 0; i < data.size(); i++)
{
LONG lIndex = 0;
if (FAILED(SizeTToLong(i, &lIndex) ||
FAILED(SafeArrayPutElement(psa, &lIndex, (void*)&(data.at(i))))))
{
SafeArrayDestroy(psa);
psa = nullptr;
break;
}
}
}
return psa;
}
#pragma warning(suppress : 26434) // WRL RuntimeClassInitialize base is a no-op and we need this for MakeAndInitialize
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
HRESULT ScreenInfoUiaProviderBase::RuntimeClassInitialize(_In_ IUiaData* pData, _In_ std::wstring_view wordDelimiters) noexcept
try
{
RETURN_HR_IF_NULL(E_INVALIDARG, pData);
_pData = pData;
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
_wordDelimiters = wordDelimiters;
return S_OK;
}
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
CATCH_RETURN();
[[nodiscard]] HRESULT ScreenInfoUiaProviderBase::Signal(_In_ EVENTID id)
{
HRESULT hr = S_OK;
// check to see if we're already firing this particular event
if (_signalFiringMapping.find(id) != _signalFiringMapping.end() &&
_signalFiringMapping[id] == true)
{
return hr;
}
try
{
_signalFiringMapping[id] = true;
}
CATCH_RETURN();
IRawElementProviderSimple* pProvider = this;
hr = UiaRaiseAutomationEvent(pProvider, id);
_signalFiringMapping[id] = false;
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
// tracing
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
/*ApiMsgSignal apiMsg;
apiMsg.Signal = id;
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
Tracing::s_TraceUia(this, ApiCall::Signal, &apiMsg);*/
return hr;
}
#pragma region IRawElementProviderSimple
// Implementation of IRawElementProviderSimple::get_ProviderOptions.
// Gets UI Automation provider options.
IFACEMETHODIMP ScreenInfoUiaProviderBase::get_ProviderOptions(_Out_ ProviderOptions* pOptions) noexcept
{
RETURN_HR_IF_NULL(E_INVALIDARG, pOptions);
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetProviderOptions, nullptr);
*pOptions = ProviderOptions_ServerSideProvider;
return S_OK;
}
// Implementation of IRawElementProviderSimple::get_PatternProvider.
// Gets the object that supports ISelectionPattern.
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetPatternProvider(_In_ PATTERNID patternId,
_COM_Outptr_result_maybenull_ IUnknown** ppInterface)
{
RETURN_HR_IF(E_INVALIDARG, ppInterface == nullptr);
*ppInterface = nullptr;
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetPatternProvider, nullptr);
*ppInterface = nullptr;
HRESULT hr = S_OK;
if (patternId == UIA_TextPatternId)
{
hr = QueryInterface(IID_PPV_ARGS(ppInterface));
if (FAILED(hr))
{
*ppInterface = nullptr;
}
}
return hr;
}
// Implementation of IRawElementProviderSimple::get_PropertyValue.
// Gets custom properties.
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetPropertyValue(_In_ PROPERTYID propertyId,
_Out_ VARIANT* pVariant) noexcept
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetPropertyValue, nullptr);
pVariant->vt = VT_EMPTY;
// Returning the default will leave the property as the default
// so we only really need to touch it for the properties we want to implement
if (propertyId == UIA_ControlTypePropertyId)
{
// This control is the Document control type, implying that it is
// a complex document that supports text pattern
pVariant->vt = VT_I4;
pVariant->lVal = UIA_DocumentControlTypeId;
}
else if (propertyId == UIA_NamePropertyId)
{
// TODO: MSFT: 7960168 - These strings should be localized text in the final UIA work
pVariant->bstrVal = SysAllocString(L"Text Area");
if (pVariant->bstrVal != nullptr)
{
pVariant->vt = VT_BSTR;
}
}
else if (propertyId == UIA_AutomationIdPropertyId)
{
pVariant->bstrVal = SysAllocString(L"Text Area");
if (pVariant->bstrVal != nullptr)
{
pVariant->vt = VT_BSTR;
}
}
else if (propertyId == UIA_IsControlElementPropertyId)
{
pVariant->vt = VT_BOOL;
pVariant->boolVal = VARIANT_TRUE;
}
else if (propertyId == UIA_IsContentElementPropertyId)
{
pVariant->vt = VT_BOOL;
pVariant->boolVal = VARIANT_TRUE;
}
else if (propertyId == UIA_IsKeyboardFocusablePropertyId)
{
pVariant->vt = VT_BOOL;
pVariant->boolVal = VARIANT_TRUE;
}
else if (propertyId == UIA_HasKeyboardFocusPropertyId)
{
pVariant->vt = VT_BOOL;
pVariant->boolVal = VARIANT_TRUE;
}
else if (propertyId == UIA_ProviderDescriptionPropertyId)
{
pVariant->bstrVal = SysAllocString(L"Microsoft Console Host: Screen Information Text Area");
if (pVariant->bstrVal != nullptr)
{
pVariant->vt = VT_BSTR;
}
}
else if (propertyId == UIA_IsEnabledPropertyId)
{
pVariant->vt = VT_BOOL;
pVariant->boolVal = VARIANT_TRUE;
}
return S_OK;
}
IFACEMETHODIMP ScreenInfoUiaProviderBase::get_HostRawElementProvider(_COM_Outptr_result_maybenull_ IRawElementProviderSimple** ppProvider) noexcept
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetHostRawElementProvider, nullptr);
RETURN_HR_IF(E_INVALIDARG, ppProvider == nullptr);
*ppProvider = nullptr;
return S_OK;
}
#pragma endregion
#pragma region IRawElementProviderFragment
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetRuntimeId(_Outptr_result_maybenull_ SAFEARRAY** ppRuntimeId)
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetRuntimeId, nullptr);
// Root defers this to host, others must implement it...
RETURN_HR_IF(E_INVALIDARG, ppRuntimeId == nullptr);
*ppRuntimeId = nullptr;
// AppendRuntimeId is a magic Number that tells UIAutomation to Append its own Runtime ID(From the HWND)
2019-09-06 02:16:31 +02:00
const std::array<int, 2> rId{ UiaAppendRuntimeId, -1 };
2019-09-06 02:16:31 +02:00
const std::basic_string_view<int> span{ rId.data(), rId.size() };
// BuildIntSafeArray is a custom function to hide the SafeArray creation
*ppRuntimeId = BuildIntSafeArray(span);
RETURN_IF_NULL_ALLOC(*ppRuntimeId);
return S_OK;
}
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetEmbeddedFragmentRoots(_Outptr_result_maybenull_ SAFEARRAY** ppRoots) noexcept
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetEmbeddedFragmentRoots, nullptr);
RETURN_HR_IF(E_INVALIDARG, ppRoots == nullptr);
*ppRoots = nullptr;
return S_OK;
}
IFACEMETHODIMP ScreenInfoUiaProviderBase::SetFocus()
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::SetFocus, nullptr);
return Signal(UIA_AutomationFocusChangedEventId);
}
#pragma endregion
#pragma region ITextProvider
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetSelection(_Outptr_result_maybenull_ SAFEARRAY** ppRetVal)
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//ApiMsgGetSelection apiMsg;
_LockConsole();
auto Unlock = wil::scope_exit([&]() noexcept {
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
_UnlockConsole();
});
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
*ppRetVal = nullptr;
HRESULT hr = S_OK;
if (!_pData->IsSelectionActive())
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//apiMsg.AreaSelected = false;
//apiMsg.SelectionRowCount = 1;
// return a degenerate range at the cursor position
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
const Cursor& cursor = _getTextBuffer().GetCursor();
// make a safe array
*ppRetVal = SafeArrayCreateVector(VT_UNKNOWN, 0, 1);
if (*ppRetVal == nullptr)
{
return E_OUTOFMEMORY;
}
if (FAILED(hr))
{
SafeArrayDestroy(*ppRetVal);
*ppRetVal = nullptr;
return hr;
}
WRL::ComPtr<UiaTextRangeBase> range;
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
hr = CreateTextRange(this, cursor, _wordDelimiters, &range);
if (FAILED(hr))
{
SafeArrayDestroy(*ppRetVal);
*ppRetVal = nullptr;
return hr;
}
LONG currentIndex = 0;
hr = SafeArrayPutElement(*ppRetVal, &currentIndex, range.Detach());
if (FAILED(hr))
{
SafeArrayDestroy(*ppRetVal);
*ppRetVal = nullptr;
return hr;
}
}
else
{
// get the selection ranges
std::deque<WRL::ComPtr<UiaTextRangeBase>> ranges;
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
RETURN_IF_FAILED(GetSelectionRanges(this, _wordDelimiters, ranges));
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//apiMsg.AreaSelected = true;
//apiMsg.SelectionRowCount = static_cast<unsigned int>(ranges.size());
// make a safe array
*ppRetVal = SafeArrayCreateVector(VT_UNKNOWN, 0, gsl::narrow<ULONG>(ranges.size()));
if (*ppRetVal == nullptr)
{
return E_OUTOFMEMORY;
}
// fill the safe array
for (LONG i = 0; i < gsl::narrow<LONG>(ranges.size()); ++i)
{
hr = SafeArrayPutElement(*ppRetVal, &i, ranges.at(i).Detach());
if (FAILED(hr))
{
SafeArrayDestroy(*ppRetVal);
*ppRetVal = nullptr;
return hr;
}
}
}
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetSelection, &apiMsg);
return S_OK;
}
IFACEMETHODIMP ScreenInfoUiaProviderBase::GetVisibleRanges(_Outptr_result_maybenull_ SAFEARRAY** ppRetVal)
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetVisibleRanges, nullptr);
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
_LockConsole();
auto Unlock = wil::scope_exit([&]() noexcept {
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
_UnlockConsole();
});
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
*ppRetVal = nullptr;
Refactor UiaTextRange For Improved Navigation and Reliability (#4018) ## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
2020-01-31 21:59:39 +01:00
const auto bufferSize = _pData->GetTextBuffer().GetSize();
const auto viewport = bufferSize.ConvertToOrigin(_getViewport());
// make a safe array
Refactor UiaTextRange For Improved Navigation and Reliability (#4018) ## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
2020-01-31 21:59:39 +01:00
const auto rowCount = viewport.Height();
*ppRetVal = SafeArrayCreateVector(VT_UNKNOWN, 0, gsl::narrow<ULONG>(rowCount));
if (*ppRetVal == nullptr)
{
return E_OUTOFMEMORY;
}
// stuff each visible line in the safearray
Refactor UiaTextRange For Improved Navigation and Reliability (#4018) ## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
2020-01-31 21:59:39 +01:00
for (short i = 0; i < rowCount; ++i)
{
Refactor UiaTextRange For Improved Navigation and Reliability (#4018) ## Summary of the Pull Request This pull request is intended to achieve the following goals... 1) reduce duplicate code 2) remove static functions 3) improve readability 4) improve reliability 5) improve code-coverage for testing 6) establish functioning text buffer navigation in Narrator and NVDA This also required a change to the wrapper class `XamlUiaTextRange` that has been causing issues with Narrator and NVDA. See below for additional context. ## References #3976 - I believe this might have been a result of improperly handling degenerate ranges. Fixed here. #3895 - reduced the duplicate code. No need to separate into different files #2160 - same as #3976 above #1993 - I think just about everything is no longer static ## PR Checklist * [x] Closes #3895, Closes #1993, Closes #3976, Closes #2160 * [x] CLA signed * [x] Tests added/passed ## Detailed Description of the Pull Request / Additional comments ### UiaTextRange - converted endpoints into the COORD system in the TextBuffer coordinate space - `start` is inclusive, `end` is exclusive. A degenerate range is when start == end. - all functions are no longer static - `MoveByUnit()` functions now rely on `MoveEndpointByUnit()` functions - removed unnecessary typedefs like `Endpoint`, `ScreenInfoRow`, etc.. - relied more heavily on existing functionality from `TextBuffer` and `Viewport` ### XamlUiaTextRange - `GetAttributeValue()` must return a special HRESULT that signifies that the requested attribute is not supported. This was the cause of a number of inconsistencies between Narrator and NVDA. - `FindText()` should return `nullptr` if nothing was found. #4373 properly fixes this functionality now that Search is a shared module ### TextBuffer - Word navigation functionality is entirely in `TextBuffer` for proper abstraction - a total of 6 functions are now dedicated to word navigation to get a good understanding of the differences between a "word" in Accessibility and a "word" in selection As an example, consider a buffer with this text in it: " word other " In selection, a "word" is defined as the range between two delimiters, so the words in the example include [" ", "word", " ", "other", " "]. In accessibility , a "word" includes the delimiters after a range of readable characters, so the words in the example include ["word ", "other "]. Additionally, accessibility word navigation must be able to detect if it is on the first or last word. This resulted in a slight variant of word navigation functions that return a boolean instead of a COORD. Ideally, these functions can be consolidated, but that is too risky for a PR of this size as it can have an effect on selection. ### Viewport - the concept of `EndExclusive` is added. This is used by UiaTextRange's `end` anchor as it is exclusive. To signify that the last character in the buffer is included in this buffer, `end` must be one past the end of the buffer. This is `EndExclusive` - Since many functions check if the given `COORD` is in bounds, a flag must be set to allow `EndExclusive` as a valid `COORD` that is in bounds. ### Testing - word navigation testing relies more heavily on TextBuffer tests - additional testing was created for non-movement focused functions of UiaTextRange - The results have been compared to Microsoft Word and some have been verified by UiAutomation/Narrator contacts as expected results. ## Validation Steps Performed Tests pass Narrator works NVDA works
2020-01-31 21:59:39 +01:00
// end is exclusive so add 1
const COORD start{ viewport.Left(), viewport.Top() + i };
const COORD end{ start.X, start.Y + 1 };
HRESULT hr = S_OK;
WRL::ComPtr<UiaTextRangeBase> range;
hr = CreateTextRange(this,
start,
end,
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
_wordDelimiters,
&range);
if (FAILED(hr))
{
SafeArrayDestroy(*ppRetVal);
*ppRetVal = nullptr;
return hr;
}
LONG currentIndex = gsl::narrow<LONG>(i);
hr = SafeArrayPutElement(*ppRetVal, &currentIndex, range.Detach());
if (FAILED(hr))
{
SafeArrayDestroy(*ppRetVal);
*ppRetVal = nullptr;
return hr;
}
}
return S_OK;
}
IFACEMETHODIMP ScreenInfoUiaProviderBase::RangeFromChild(_In_ IRawElementProviderSimple* /*childElement*/,
_COM_Outptr_result_maybenull_ ITextRangeProvider** ppRetVal)
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::RangeFromChild, nullptr);
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
*ppRetVal = nullptr;
WRL::ComPtr<UiaTextRangeBase> utr;
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
RETURN_IF_FAILED(CreateTextRange(this, _wordDelimiters, &utr));
RETURN_IF_FAILED(utr.CopyTo(ppRetVal));
return S_OK;
}
IFACEMETHODIMP ScreenInfoUiaProviderBase::RangeFromPoint(_In_ UiaPoint point,
_COM_Outptr_result_maybenull_ ITextRangeProvider** ppRetVal)
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::RangeFromPoint, nullptr);
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
*ppRetVal = nullptr;
WRL::ComPtr<UiaTextRangeBase> utr;
RETURN_IF_FAILED(CreateTextRange(this,
point,
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
_wordDelimiters,
&utr));
RETURN_IF_FAILED(utr.CopyTo(ppRetVal));
return S_OK;
}
IFACEMETHODIMP ScreenInfoUiaProviderBase::get_DocumentRange(_COM_Outptr_result_maybenull_ ITextRangeProvider** ppRetVal)
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetDocumentRange, nullptr);
RETURN_HR_IF_NULL(E_INVALIDARG, ppRetVal);
*ppRetVal = nullptr;
WRL::ComPtr<UiaTextRangeBase> utr;
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
RETURN_IF_FAILED(CreateTextRange(this, _wordDelimiters, &utr));
RETURN_IF_FAILED(utr->ExpandToEnclosingUnit(TextUnit::TextUnit_Document));
RETURN_IF_FAILED(utr.CopyTo(ppRetVal));
return S_OK;
}
IFACEMETHODIMP ScreenInfoUiaProviderBase::get_SupportedTextSelection(_Out_ SupportedTextSelection* pRetVal) noexcept
{
RETURN_HR_IF_NULL(E_INVALIDARG, pRetVal);
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
// TODO GitHub #1914: Re-attach Tracing to UIA Tree
//Tracing::s_TraceUia(this, ApiCall::GetSupportedTextSelection, nullptr);
*pRetVal = SupportedTextSelection::SupportedTextSelection_Single;
return S_OK;
}
#pragma endregion
const COORD ScreenInfoUiaProviderBase::_getScreenBufferCoords() const
{
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
return _getTextBuffer().GetSize().Dimensions();
}
const TextBuffer& ScreenInfoUiaProviderBase::_getTextBuffer() const noexcept
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
{
return _pData->GetTextBuffer();
}
const Viewport ScreenInfoUiaProviderBase::_getViewport() const noexcept
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
{
return _pData->GetViewport();
}
void ScreenInfoUiaProviderBase::_LockConsole() noexcept
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
{
// TODO GitHub #2141: Lock and Unlock in conhost should decouple Ctrl+C dispatch and use smarter handling
_pData->LockConsole();
}
void ScreenInfoUiaProviderBase::_UnlockConsole() noexcept
Accessibility: Set-up UIA Tree (#1691) **The Basics of Accessibility** - [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview) - Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it. - Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right. **Accessibility in our Project** There's a few important classes... regarding Accessibility... - **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree. - **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer. - **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them. regarding the Windows Terminal window... - **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff. - **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal - **NonClientIslandWindow**: An extension of the `IslandWindow` regarding ConHost... - **IConsoleWindow**: This is an interface for the console window. - **Window**: This is the actual window for ConHost. Extends `IConsoleWindow` - `IConsoleWindow` changes: - move into `Microsoft::Console::Types` (a shared space) - Have `IslandWindow` extend it - `WindowUiaProvider` changes: - move into `Microsoft::Console::Types` (a shared space) - Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree) ### Changes to the WindowUiaProvider As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized. In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen. So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen. 🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352. ### Moved to `Microsoft::Console::Types` These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal. This means that any references to the `ServiceLocator` had to be removed. - `IConsoleWindow` - Windows Terminal: `IslandWindow : IConsoleWindow` - `ScreenInfoUiaProvider` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - `UiaTextRange` - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details. - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them. ### Changes to IRenderData Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here: - `bool IsAreaSelected()` - `void ClearSelection()` - `void SelectNewRegion(...)` - `HRESULT SearchForText(...)` `SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
2019-07-30 00:21:15 +02:00
{
// TODO GitHub #2141: Lock and Unlock in conhost should decouple Ctrl+C dispatch and use smarter handling
_pData->UnlockConsole();
}