terminal/src/types/TermControlUiaTextRange.cpp

145 lines
6.1 KiB
C++
Raw Permalink Normal View History

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "TermControlUiaTextRange.hpp"
#include "TermControlUiaProvider.hpp"
using namespace Microsoft::Terminal;
using namespace Microsoft::Console::Types;
using namespace Microsoft::WRL;
// degenerate range constructor.
HRESULT TermControlUiaTextRange::RuntimeClassInitialize(_In_ IUiaData* pData, _In_ IRawElementProviderSimple* const pProvider, _In_ const std::wstring_view wordDelimiters) noexcept
{
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
return UiaTextRangeBase::RuntimeClassInitialize(pData, pProvider, wordDelimiters);
}
HRESULT TermControlUiaTextRange::RuntimeClassInitialize(_In_ IUiaData* pData,
_In_ IRawElementProviderSimple* const pProvider,
const Cursor& cursor,
const std::wstring_view wordDelimiters) noexcept
{
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
return UiaTextRangeBase::RuntimeClassInitialize(pData, pProvider, cursor, wordDelimiters);
}
HRESULT TermControlUiaTextRange::RuntimeClassInitialize(_In_ IUiaData* pData,
_In_ IRawElementProviderSimple* const pProvider,
const COORD start,
const COORD end,
bool blockRange,
const std::wstring_view wordDelimiters) noexcept
{
return UiaTextRangeBase::RuntimeClassInitialize(pData, pProvider, start, end, blockRange, wordDelimiters);
}
// returns a degenerate text range of the start of the row closest to the y value of point
#pragma warning(suppress : 26434) // WRL RuntimeClassInitialize base is a no-op and we need this for MakeAndInitialize
HRESULT TermControlUiaTextRange::RuntimeClassInitialize(_In_ IUiaData* pData,
_In_ IRawElementProviderSimple* const pProvider,
const UiaPoint point,
const std::wstring_view wordDelimiters)
{
Enable Word Navigation in UiaTextRange (#3659) Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English. # General "Word Movement" Expectations The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source) If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards) # Word Expansion Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer. Then I built on top of it by adding an optional additional parameter that decides if you want to include... - the delimiter run when moving forward - the character run when moving backwards It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange # UiaTextRange The code is based on character movement. This allows us to actually work with boundary conditions. The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples: - Endpoint 0 --> (0,0) - Endpoint 79 --> (79,0) (when the buffer width is 80) - Endpoint 80 -->(0,1) (when the buffer width is 80) - When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards). - When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end. # Hooking it up All we really had to do is add an enum. This part was super easy :) I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable. # Defining your own word delimiters - Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP) - This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context - import a word delimiter into the UTR directly - this provides more control over what a "word" is - this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths) The default param of " " is scattered throughout because this is the word delimiter used in the English language.
2019-12-13 00:22:12 +01:00
RETURN_IF_FAILED(UiaTextRangeBase::RuntimeClassInitialize(pData, pProvider, wordDelimiters));
Initialize(point);
return S_OK;
}
#pragma warning(suppress : 26434) // WRL RuntimeClassInitialize base is a no-op and we need this for MakeAndInitialize
HRESULT TermControlUiaTextRange::RuntimeClassInitialize(const TermControlUiaTextRange& a) noexcept
{
return UiaTextRangeBase::RuntimeClassInitialize(a);
}
IFACEMETHODIMP TermControlUiaTextRange::Clone(_Outptr_result_maybenull_ ITextRangeProvider** ppRetVal)
{
RETURN_HR_IF(E_INVALIDARG, ppRetVal == nullptr);
*ppRetVal = nullptr;
const auto hr = MakeAndInitialize<TermControlUiaTextRange>(ppRetVal, *this);
if (hr != S_OK)
{
*ppRetVal = nullptr;
return hr;
}
return S_OK;
}
// Method Description:
// - Transform coordinates relative to the client to relative to the screen
// Arguments:
// - clientPoint: coordinates relative to the client where
// (0,0) is the top-left of the app window
// Return Value:
// - <none>
void TermControlUiaTextRange::_TranslatePointToScreen(LPPOINT clientPoint) const
{
const gsl::not_null<TermControlUiaProvider*> provider = static_cast<TermControlUiaProvider*>(_pProvider);
const auto includeOffsets = [](long clientPos, double termControlPos, double padding, double scaleFactor) {
Adjusts High DPI scaling to enable differential rendering (#5345) ## Summary of the Pull Request - Adjusts scaling practices in `DxEngine` (and related scaling practices in `TerminalControl`) for pixel-perfect row baselines and spacing at High DPI such that differential row-by-row rendering can be applied at High DPI. ## References - #5185 ## PR Checklist * [x] Closes #5320, closes #3515, closes #1064 * [x] I work here. * [x] Manually tested. * [x] No doc. * [x] Am core contributor. Also discussed with some of them already via Teams. ## Detailed Description of the Pull Request / Additional comments **WAS:** - We were using implicit DPI scaling on the `ID2D1RenderTarget` and running all of our processing in DIPs (Device-Independent Pixels). That's all well and good for getting things bootstrapped quickly, but it leaves the actual scaling of the draw commands up to the discretion of the rendering target. - When we don't get to explicitly choose exactly how many pixels tall/wide and our X/Y placement perfectly, the nature of floating point multiplication and division required to do the presentation can cause us to drift off slightly out of our control depending on what the final display resolution actually is. - Differential drawing cannot work unless we can know the exact integer pixels that need to be copied/moved/preserved/replaced between frames to give to the `IDXGISwapChain1::Present1` method. If things spill into fractional pixels or the sizes of rows/columns vary as they are rounded up and down implicitly, then we cannot do the differential rendering. **NOW:** - When deciding on a font, the `DxEngine` will take the scale factor into account and adjust the proposed height of the requested font. Then the remainder of the existing code that adjusts the baseline and integer-ifies each character cell will run naturally from there. That code already works correctly to align the height at normal DPI and scale out the font heights and advances to take an exact integer of pixels. - `TermControl` has to use the scale now, in some places, and stop scaling in other places. This has to do with how the target's nature used to be implicit and is now explicit. For instance, determining where the cursor click hits must be scaled now. And determining the pixel size of the display canvas must no longer be scaled. - `DxEngine` will no longer attempt to scale the invalid regions per my attempts in #5185 because the cell size is scaled. So it should work the same as at 96 DPI. - The block is removed from the `DxEngine` that was causing a full invalidate on every frame at High DPI. - A TODO was removed from `TermControl` that was invalidating everything when the DPI changed because the underlying renderer will already do that. ## Validation Steps Performed * [x] Check at 150% DPI. Print text, scroll text down and up, do selection. * [x] Check at 100% DPI. Print text, scroll text down and up, do selection. * [x] Span two different DPI monitors and drag between them. * [x] Giant pile of tests in https://github.com/microsoft/terminal/pull/5345#issuecomment-614127648 Co-authored-by: Dustin Howett <duhowett@microsoft.com> Co-authored-by: Mike Griese <migrie@microsoft.com>
2020-04-22 23:59:51 +02:00
auto result = base::ClampedNumeric<double>(padding);
// only the padding is in DIPs now
result *= scaleFactor;
Adjusts High DPI scaling to enable differential rendering (#5345) ## Summary of the Pull Request - Adjusts scaling practices in `DxEngine` (and related scaling practices in `TerminalControl`) for pixel-perfect row baselines and spacing at High DPI such that differential row-by-row rendering can be applied at High DPI. ## References - #5185 ## PR Checklist * [x] Closes #5320, closes #3515, closes #1064 * [x] I work here. * [x] Manually tested. * [x] No doc. * [x] Am core contributor. Also discussed with some of them already via Teams. ## Detailed Description of the Pull Request / Additional comments **WAS:** - We were using implicit DPI scaling on the `ID2D1RenderTarget` and running all of our processing in DIPs (Device-Independent Pixels). That's all well and good for getting things bootstrapped quickly, but it leaves the actual scaling of the draw commands up to the discretion of the rendering target. - When we don't get to explicitly choose exactly how many pixels tall/wide and our X/Y placement perfectly, the nature of floating point multiplication and division required to do the presentation can cause us to drift off slightly out of our control depending on what the final display resolution actually is. - Differential drawing cannot work unless we can know the exact integer pixels that need to be copied/moved/preserved/replaced between frames to give to the `IDXGISwapChain1::Present1` method. If things spill into fractional pixels or the sizes of rows/columns vary as they are rounded up and down implicitly, then we cannot do the differential rendering. **NOW:** - When deciding on a font, the `DxEngine` will take the scale factor into account and adjust the proposed height of the requested font. Then the remainder of the existing code that adjusts the baseline and integer-ifies each character cell will run naturally from there. That code already works correctly to align the height at normal DPI and scale out the font heights and advances to take an exact integer of pixels. - `TermControl` has to use the scale now, in some places, and stop scaling in other places. This has to do with how the target's nature used to be implicit and is now explicit. For instance, determining where the cursor click hits must be scaled now. And determining the pixel size of the display canvas must no longer be scaled. - `DxEngine` will no longer attempt to scale the invalid regions per my attempts in #5185 because the cell size is scaled. So it should work the same as at 96 DPI. - The block is removed from the `DxEngine` that was causing a full invalidate on every frame at High DPI. - A TODO was removed from `TermControl` that was invalidating everything when the DPI changed because the underlying renderer will already do that. ## Validation Steps Performed * [x] Check at 150% DPI. Print text, scroll text down and up, do selection. * [x] Check at 100% DPI. Print text, scroll text down and up, do selection. * [x] Span two different DPI monitors and drag between them. * [x] Giant pile of tests in https://github.com/microsoft/terminal/pull/5345#issuecomment-614127648 Co-authored-by: Dustin Howett <duhowett@microsoft.com> Co-authored-by: Mike Griese <migrie@microsoft.com>
2020-04-22 23:59:51 +02:00
result += clientPos;
result += termControlPos;
return result;
};
// update based on TermControl location (important for Panes)
UiaRect boundingRect;
THROW_IF_FAILED(provider->get_BoundingRectangle(&boundingRect));
// update based on TermControl padding
const auto padding = provider->GetPadding();
// Get scale factor for display
const auto scaleFactor = provider->GetScaleFactor();
clientPoint->x = includeOffsets(clientPoint->x, boundingRect.left, padding.left, scaleFactor);
clientPoint->y = includeOffsets(clientPoint->y, boundingRect.top, padding.top, scaleFactor);
}
// Method Description:
// - Transform coordinates relative to the screen to relative to the client
// Arguments:
// - screenPoint: coordinates relative to the screen where
// (0,0) is the top-left of the screen
// Return Value:
// - <none>
void TermControlUiaTextRange::_TranslatePointFromScreen(LPPOINT screenPoint) const
{
const gsl::not_null<TermControlUiaProvider*> provider = static_cast<TermControlUiaProvider*>(_pProvider);
const auto includeOffsets = [](long screenPos, double termControlPos, double padding, double scaleFactor) {
Adjusts High DPI scaling to enable differential rendering (#5345) ## Summary of the Pull Request - Adjusts scaling practices in `DxEngine` (and related scaling practices in `TerminalControl`) for pixel-perfect row baselines and spacing at High DPI such that differential row-by-row rendering can be applied at High DPI. ## References - #5185 ## PR Checklist * [x] Closes #5320, closes #3515, closes #1064 * [x] I work here. * [x] Manually tested. * [x] No doc. * [x] Am core contributor. Also discussed with some of them already via Teams. ## Detailed Description of the Pull Request / Additional comments **WAS:** - We were using implicit DPI scaling on the `ID2D1RenderTarget` and running all of our processing in DIPs (Device-Independent Pixels). That's all well and good for getting things bootstrapped quickly, but it leaves the actual scaling of the draw commands up to the discretion of the rendering target. - When we don't get to explicitly choose exactly how many pixels tall/wide and our X/Y placement perfectly, the nature of floating point multiplication and division required to do the presentation can cause us to drift off slightly out of our control depending on what the final display resolution actually is. - Differential drawing cannot work unless we can know the exact integer pixels that need to be copied/moved/preserved/replaced between frames to give to the `IDXGISwapChain1::Present1` method. If things spill into fractional pixels or the sizes of rows/columns vary as they are rounded up and down implicitly, then we cannot do the differential rendering. **NOW:** - When deciding on a font, the `DxEngine` will take the scale factor into account and adjust the proposed height of the requested font. Then the remainder of the existing code that adjusts the baseline and integer-ifies each character cell will run naturally from there. That code already works correctly to align the height at normal DPI and scale out the font heights and advances to take an exact integer of pixels. - `TermControl` has to use the scale now, in some places, and stop scaling in other places. This has to do with how the target's nature used to be implicit and is now explicit. For instance, determining where the cursor click hits must be scaled now. And determining the pixel size of the display canvas must no longer be scaled. - `DxEngine` will no longer attempt to scale the invalid regions per my attempts in #5185 because the cell size is scaled. So it should work the same as at 96 DPI. - The block is removed from the `DxEngine` that was causing a full invalidate on every frame at High DPI. - A TODO was removed from `TermControl` that was invalidating everything when the DPI changed because the underlying renderer will already do that. ## Validation Steps Performed * [x] Check at 150% DPI. Print text, scroll text down and up, do selection. * [x] Check at 100% DPI. Print text, scroll text down and up, do selection. * [x] Span two different DPI monitors and drag between them. * [x] Giant pile of tests in https://github.com/microsoft/terminal/pull/5345#issuecomment-614127648 Co-authored-by: Dustin Howett <duhowett@microsoft.com> Co-authored-by: Mike Griese <migrie@microsoft.com>
2020-04-22 23:59:51 +02:00
auto result = base::ClampedNumeric<double>(padding);
// only the padding is in DIPs now
result /= scaleFactor;
Adjusts High DPI scaling to enable differential rendering (#5345) ## Summary of the Pull Request - Adjusts scaling practices in `DxEngine` (and related scaling practices in `TerminalControl`) for pixel-perfect row baselines and spacing at High DPI such that differential row-by-row rendering can be applied at High DPI. ## References - #5185 ## PR Checklist * [x] Closes #5320, closes #3515, closes #1064 * [x] I work here. * [x] Manually tested. * [x] No doc. * [x] Am core contributor. Also discussed with some of them already via Teams. ## Detailed Description of the Pull Request / Additional comments **WAS:** - We were using implicit DPI scaling on the `ID2D1RenderTarget` and running all of our processing in DIPs (Device-Independent Pixels). That's all well and good for getting things bootstrapped quickly, but it leaves the actual scaling of the draw commands up to the discretion of the rendering target. - When we don't get to explicitly choose exactly how many pixels tall/wide and our X/Y placement perfectly, the nature of floating point multiplication and division required to do the presentation can cause us to drift off slightly out of our control depending on what the final display resolution actually is. - Differential drawing cannot work unless we can know the exact integer pixels that need to be copied/moved/preserved/replaced between frames to give to the `IDXGISwapChain1::Present1` method. If things spill into fractional pixels or the sizes of rows/columns vary as they are rounded up and down implicitly, then we cannot do the differential rendering. **NOW:** - When deciding on a font, the `DxEngine` will take the scale factor into account and adjust the proposed height of the requested font. Then the remainder of the existing code that adjusts the baseline and integer-ifies each character cell will run naturally from there. That code already works correctly to align the height at normal DPI and scale out the font heights and advances to take an exact integer of pixels. - `TermControl` has to use the scale now, in some places, and stop scaling in other places. This has to do with how the target's nature used to be implicit and is now explicit. For instance, determining where the cursor click hits must be scaled now. And determining the pixel size of the display canvas must no longer be scaled. - `DxEngine` will no longer attempt to scale the invalid regions per my attempts in #5185 because the cell size is scaled. So it should work the same as at 96 DPI. - The block is removed from the `DxEngine` that was causing a full invalidate on every frame at High DPI. - A TODO was removed from `TermControl` that was invalidating everything when the DPI changed because the underlying renderer will already do that. ## Validation Steps Performed * [x] Check at 150% DPI. Print text, scroll text down and up, do selection. * [x] Check at 100% DPI. Print text, scroll text down and up, do selection. * [x] Span two different DPI monitors and drag between them. * [x] Giant pile of tests in https://github.com/microsoft/terminal/pull/5345#issuecomment-614127648 Co-authored-by: Dustin Howett <duhowett@microsoft.com> Co-authored-by: Mike Griese <migrie@microsoft.com>
2020-04-22 23:59:51 +02:00
result -= screenPos;
result -= termControlPos;
return result;
};
// update based on TermControl location (important for Panes)
UiaRect boundingRect;
THROW_IF_FAILED(provider->get_BoundingRectangle(&boundingRect));
// update based on TermControl padding
const auto padding = provider->GetPadding();
// Get scale factor for display
const auto scaleFactor = provider->GetScaleFactor();
screenPoint->x = includeOffsets(screenPoint->x, boundingRect.left, padding.left, scaleFactor);
screenPoint->y = includeOffsets(screenPoint->y, boundingRect.top, padding.top, scaleFactor);
}
const COORD TermControlUiaTextRange::_getScreenFontSize() const noexcept
{
// Do NOT get the font info from IRenderData. It is a dummy font info.
// Instead, the font info is saved in the TermControl. So we have to
// ask our parent to get it for us.
const gsl::not_null<const TermControlUiaProvider*> provider = static_cast<TermControlUiaProvider*>(_pProvider);
return provider->GetFontSize();
}