Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919)
TerminalControl doesn't use any of the built in text input and edit
controls provided by XAML for text input, which means TermianlControl
needs to communicate with the Text Services Framework (TSF) in order to
provide Input Method Editor (IME) support. Just like the rest of
Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core)
namespace to provide support vs. the old TSF 1.0.
Windows.UI.Text.Core handles communication between a text edit control
and the text services primarily through a CoreTextEditContext object.
This change introduces a new UserControl TSFInputControl which is a
custom EditControl similar to the CustomEditControl sample[1].
TSFInputControl is similar (overlay with IME text) to how old console
(conimeinfo) handled IME.
# Details
TSFInputControl is a Windows.UI.Xaml.Controls.UserControl
TSFInputControl contains a Canvas control for absolution positioning a
TextBlock control within its containing control (TerminalControl).
The TextBlock control is used for displaying candidate text from the
IME. When the user makes a choice in the IME the TextBlock is cleared
and the text is written to the Terminal buffer like normal text.
TSFInputControl creates an instance of the CoreTextEditContext and
attaches appropriate event handlers to CoreTextEditContext in order to
interact with the IME.
A good write-up on how to interact with CoreTextEditContext can be found
here[2].
## Text Updates
Text updates from the IME come in on the TextUpdating event handler,
text updates are stored in an internal buffer (_inputBuffer).
## Completed Text
Once a user selects a text in the IME, the CompositionCompleted handler
is invoked. The input buffer (_inputBuffer) is written to the Terminal
buffer, _inputBuffer is cleared and Canvas and TextBlock controls are
hidden until the user starts a composition session again.
## Positioning
Telling the IME where to properly position itself was the hardest part
of this change. The IME expects to know it's location in screen
coordinates as supposed to client coordinates. This is pretty easy if
you are a pure UWP, but since we are hosted inside a XAMLIsland the
client to screen coordinate translation is a little harder.
### Calculating Screen Coordinates
1. Obtaining the Window position in Screen coordinates.
2. Determining the Client coordinate of the cursor.
3. Converting the Client coordinate of the cursor to Screen coordinates.
4. Offsetting the X and Y coordinate of the cursor by the position of
the TerminalControl within the window (tabs if present, margins, etc..).
5. Applying any scale factor of the display.
Once we have the right position in screen coordinates, this is supplied
in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets
the IME know where to position itself on the Screen.
## Font Information/Cursor/Writing to Terminal
3 events were added to the TSFInputControl to create a loosely-coupled
implementation between the TerminalControl and the TSFInputControl.
These events are used for obtaining Font information from the
TerminalControl, getting the Cursor position and writing to the terminal
buffer.
## Known Issues
- Width of TextBlock is hardcoded to 200 pixels and most likely should
adjust to the available width of the current input line on the console
(#3640)
- Entering text in the middle of an existing set of text has TextBlock
render under existing text. Current Console behavior here isn't good
experience either (writes over text)
- Text input at edges of window is clipped versus wrapping around to
next line. This isn't any worse than the original command line, but
Terminal should be better (#3657)
## Future Considerations
Ideally, we'd be able to interact with the console buffer directly and
replace characters as the user types.
## Validation
General steps to try functionality
- Open Console
- Switch to Simplified Chinese (Shortcut: Windows+Spacebar)
- Switch to Chinese mode on language bar
Scenarios validated:
- As user types unformatted candidates appear on command line and IME
renders in correct position under unformatted characters.
- User can dismiss IME and text doesn't appear on command line
- Switch back to English mode, functions like normal
- New tab has proper behavior
- Switching between tabs has proper behavior
- Switching away from Terminal Window with IME present causes IME to
disappear
[1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl
[2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input
Closes #459
Closes #2213
Closes #3641
2019-11-22 01:25:50 +01:00
|
|
|
// Copyright (c) Microsoft Corporation.
|
|
|
|
// Licensed under the MIT license.
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
|
|
|
// Method Description:
|
|
|
|
// - Scales a Rect based on a scale factor
|
|
|
|
// Arguments:
|
|
|
|
// - rect: Rect to scale by scale
|
|
|
|
// - scale: amount to scale rect by
|
|
|
|
// Return Value:
|
|
|
|
// - Rect scaled by scale
|
|
|
|
inline winrt::Windows::Foundation::Rect ScaleRect(winrt::Windows::Foundation::Rect rect, double scale)
|
|
|
|
{
|
Display Emojis, Kaomojis, and symbols while in IME mode (#4688)
## Summary of the Pull Request
Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to.
We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded.
I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue.
## PR Checklist
* [x] Closes #4148
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
## Validation Steps Performed
Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 19:36:02 +01:00
|
|
|
const auto scaleLocal = base::ClampedNumeric<float>(scale);
|
|
|
|
rect.X = base::ClampMul(rect.X, scaleLocal);
|
|
|
|
rect.Y = base::ClampMul(rect.Y, scaleLocal);
|
|
|
|
rect.Width = base::ClampMul(rect.Width, scaleLocal);
|
|
|
|
rect.Height = base::ClampMul(rect.Height, scaleLocal);
|
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919)
TerminalControl doesn't use any of the built in text input and edit
controls provided by XAML for text input, which means TermianlControl
needs to communicate with the Text Services Framework (TSF) in order to
provide Input Method Editor (IME) support. Just like the rest of
Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core)
namespace to provide support vs. the old TSF 1.0.
Windows.UI.Text.Core handles communication between a text edit control
and the text services primarily through a CoreTextEditContext object.
This change introduces a new UserControl TSFInputControl which is a
custom EditControl similar to the CustomEditControl sample[1].
TSFInputControl is similar (overlay with IME text) to how old console
(conimeinfo) handled IME.
# Details
TSFInputControl is a Windows.UI.Xaml.Controls.UserControl
TSFInputControl contains a Canvas control for absolution positioning a
TextBlock control within its containing control (TerminalControl).
The TextBlock control is used for displaying candidate text from the
IME. When the user makes a choice in the IME the TextBlock is cleared
and the text is written to the Terminal buffer like normal text.
TSFInputControl creates an instance of the CoreTextEditContext and
attaches appropriate event handlers to CoreTextEditContext in order to
interact with the IME.
A good write-up on how to interact with CoreTextEditContext can be found
here[2].
## Text Updates
Text updates from the IME come in on the TextUpdating event handler,
text updates are stored in an internal buffer (_inputBuffer).
## Completed Text
Once a user selects a text in the IME, the CompositionCompleted handler
is invoked. The input buffer (_inputBuffer) is written to the Terminal
buffer, _inputBuffer is cleared and Canvas and TextBlock controls are
hidden until the user starts a composition session again.
## Positioning
Telling the IME where to properly position itself was the hardest part
of this change. The IME expects to know it's location in screen
coordinates as supposed to client coordinates. This is pretty easy if
you are a pure UWP, but since we are hosted inside a XAMLIsland the
client to screen coordinate translation is a little harder.
### Calculating Screen Coordinates
1. Obtaining the Window position in Screen coordinates.
2. Determining the Client coordinate of the cursor.
3. Converting the Client coordinate of the cursor to Screen coordinates.
4. Offsetting the X and Y coordinate of the cursor by the position of
the TerminalControl within the window (tabs if present, margins, etc..).
5. Applying any scale factor of the display.
Once we have the right position in screen coordinates, this is supplied
in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets
the IME know where to position itself on the Screen.
## Font Information/Cursor/Writing to Terminal
3 events were added to the TSFInputControl to create a loosely-coupled
implementation between the TerminalControl and the TSFInputControl.
These events are used for obtaining Font information from the
TerminalControl, getting the Cursor position and writing to the terminal
buffer.
## Known Issues
- Width of TextBlock is hardcoded to 200 pixels and most likely should
adjust to the available width of the current input line on the console
(#3640)
- Entering text in the middle of an existing set of text has TextBlock
render under existing text. Current Console behavior here isn't good
experience either (writes over text)
- Text input at edges of window is clipped versus wrapping around to
next line. This isn't any worse than the original command line, but
Terminal should be better (#3657)
## Future Considerations
Ideally, we'd be able to interact with the console buffer directly and
replace characters as the user types.
## Validation
General steps to try functionality
- Open Console
- Switch to Simplified Chinese (Shortcut: Windows+Spacebar)
- Switch to Chinese mode on language bar
Scenarios validated:
- As user types unformatted candidates appear on command line and IME
renders in correct position under unformatted characters.
- User can dismiss IME and text doesn't appear on command line
- Switch back to English mode, functions like normal
- New tab has proper behavior
- Switching between tabs has proper behavior
- Switching away from Terminal Window with IME present causes IME to
disappear
[1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl
[2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input
Closes #459
Closes #2213
Closes #3641
2019-11-22 01:25:50 +01:00
|
|
|
return rect;
|
|
|
|
}
|
2021-10-01 20:33:22 +02:00
|
|
|
|
|
|
|
// Function Description:
|
|
|
|
// - This function presents a File Open "common dialog" and returns its selected file asynchronously.
|
|
|
|
// Parameters:
|
|
|
|
// - customize: A lambda that receives an IFileDialog* to customize.
|
|
|
|
// Return value:
|
|
|
|
// (async) path to the selected item.
|
|
|
|
template<typename TLambda>
|
|
|
|
winrt::Windows::Foundation::IAsyncOperation<winrt::hstring> FilePicker(HWND parentHwnd, bool saveDialog, TLambda&& customize)
|
|
|
|
{
|
|
|
|
auto fileDialog{ saveDialog ? winrt::create_instance<IFileDialog>(CLSID_FileSaveDialog) :
|
|
|
|
winrt::create_instance<IFileDialog>(CLSID_FileOpenDialog) };
|
|
|
|
DWORD flags{};
|
|
|
|
THROW_IF_FAILED(fileDialog->GetOptions(&flags));
|
|
|
|
THROW_IF_FAILED(fileDialog->SetOptions(flags | FOS_FORCEFILESYSTEM | FOS_NOCHANGEDIR | FOS_DONTADDTORECENT)); // filesystem objects only; no recent places
|
|
|
|
customize(fileDialog.get());
|
|
|
|
|
|
|
|
const auto hr{ fileDialog->Show(parentHwnd) };
|
|
|
|
if (!SUCCEEDED(hr))
|
|
|
|
{
|
|
|
|
if (hr == HRESULT_FROM_WIN32(ERROR_CANCELLED))
|
|
|
|
{
|
|
|
|
co_return winrt::hstring{};
|
|
|
|
}
|
|
|
|
THROW_HR(hr);
|
|
|
|
}
|
|
|
|
|
|
|
|
winrt::com_ptr<IShellItem> result;
|
|
|
|
THROW_IF_FAILED(fileDialog->GetResult(result.put()));
|
|
|
|
|
|
|
|
wil::unique_cotaskmem_string filePath;
|
|
|
|
THROW_IF_FAILED(result->GetDisplayName(SIGDN_FILESYSPATH, &filePath));
|
|
|
|
|
|
|
|
co_return winrt::hstring{ filePath.get() };
|
|
|
|
}
|
|
|
|
|
|
|
|
template<typename TLambda>
|
|
|
|
winrt::Windows::Foundation::IAsyncOperation<winrt::hstring> OpenFilePicker(HWND parentHwnd, TLambda&& customize)
|
|
|
|
{
|
|
|
|
return FilePicker(parentHwnd, false, customize);
|
|
|
|
}
|
|
|
|
template<typename TLambda>
|
|
|
|
winrt::Windows::Foundation::IAsyncOperation<winrt::hstring> SaveFilePicker(HWND parentHwnd, TLambda&& customize)
|
|
|
|
{
|
|
|
|
return FilePicker(parentHwnd, true, customize);
|
|
|
|
}
|
|
|
|
|
|
|
|
winrt::Windows::Foundation::IAsyncOperation<winrt::hstring> OpenImagePicker(HWND parentHwnd);
|