2019-05-03 00:29:04 +02:00
|
|
|
<?xml version="1.0" encoding="utf-8"?>
|
|
|
|
|
<Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
|
2019-11-05 23:29:11 +01:00
|
|
|
<PropertyGroup Label="Globals">
|
|
|
|
|
<ProjectGuid>{CA5CAD1A-44BD-4AC7-AC72-6CA5B3AB89ED}</ProjectGuid>
|
|
|
|
|
<ProjectName>TerminalControl</ProjectName>
|
|
|
|
|
<RootNamespace>Microsoft.Terminal.TerminalControl</RootNamespace>
|
2019-05-03 00:29:04 +02:00
|
|
|
<!-- cppwinrt.build.pre.props depends on these settings: -->
|
|
|
|
|
<!-- build a dll, not exe (Application) -->
|
|
|
|
|
<ConfigurationType>DynamicLibrary</ConfigurationType>
|
|
|
|
|
<SubSystem>Console</SubSystem>
|
|
|
|
|
<!-- sets a bunch of Windows Universal properties -->
|
|
|
|
|
<OpenConsoleUniversalApp>true</OpenConsoleUniversalApp>
|
Search - add search box control and implement search experience (#3590)
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
This is the PR for feature Search: #605
This PR includes the newly introduced SearchBoxControl in TermControl dir, which is the search bar for the search experience. And the codes that enable Search in Windows Terminal.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
The PR that migrates the Conhost search module: https://github.com/microsoft/terminal/pull/3279
Spec (still actively updating): https://github.com/microsoft/terminal/pull/3299
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes #605
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
These functionalities are included in the search experience.
1. Search in Terminal text buffer.
2. Automatic wrap-around.
3. Search up or down switch by clicking different buttons.
4. Search case sensitively/insensitively by clicking a button. S. Move the search box to the top/bottom by clicking a button.
6. Close by clicking 'X'.
7. Open search by ctrl + F.
When the searchbox is open, the user could still interact with the terminal by clicking the terminal input area.
While I already have the search functionalities, currently there are still some known to-do works and I will keep updating my PR:
1. Optimize the search box UI, this includes:
1) Theme adaptation. The search box background and font color
should change according to the theme,
2) Add background. Currently the elements in search box are all
transparent. However, we need a background.
3) Move button should be highlighted once clicked.
2. Accessibility: search process should be able to performed without mouse. Once the search box is focused, the user should be able to navigate between all interactive elements on the searchbox using keyboard.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
To test:
1. checkout this branch.
2. Build the project.
3. Start Windows Terminal and press Ctrl+F
4. The search box should appear on the top right corner.
2019-12-17 16:52:37 +01:00
|
|
|
<!-- C++/WinRT sets the depth to 1 if there is a XAML file in the project
|
|
|
|
|
Unfortunately for us, we need it to be 3. When the namespace merging
|
|
|
|
|
depth is 1, Microsoft.Terminal.TerminalControl becomes "Microsoft",
|
|
|
|
|
and our WinMD file becomes "Microsoft". Because WinRT is very
|
|
|
|
|
namespace-driven, this winmd is considered to contain the entire
|
|
|
|
|
Microsoft namespace. This is, obviously, not great. None of our other
|
|
|
|
|
projects compile properly when they depend on this "Microsoft.winmd."
|
|
|
|
|
-->
|
|
|
|
|
<CppWinRTNamespaceMergeDepth>3</CppWinRTNamespaceMergeDepth>
|
2019-11-05 23:29:11 +01:00
|
|
|
|
|
|
|
|
<!--
|
|
|
|
|
DON'T REDIRECT OUR OUTPUT.
|
|
|
|
|
Setting this will tell cppwinrt.build.post.props to copy our output from
|
|
|
|
|
the default OutDir up one level, so the wapproj will be able to find it.
|
|
|
|
|
-->
|
|
|
|
|
<NoOutputRedirection>true</NoOutputRedirection>
|
2020-03-20 21:08:32 +01:00
|
|
|
<XamlComponentResourceLocation>nested</XamlComponentResourceLocation>
|
2019-05-03 00:29:04 +02:00
|
|
|
</PropertyGroup>
|
2019-11-05 23:29:11 +01:00
|
|
|
|
|
|
|
|
<Import Project="..\..\..\common.openconsole.props" Condition="'$(OpenConsoleDir)'==''" />
|
2019-05-03 00:29:04 +02:00
|
|
|
<Import Project="$(OpenConsoleDir)src\cppwinrt.build.pre.props" />
|
2019-11-05 23:29:11 +01:00
|
|
|
|
2019-05-03 00:29:04 +02:00
|
|
|
<ItemGroup>
|
|
|
|
|
<ClInclude Include="pch.h" />
|
Search - add search box control and implement search experience (#3590)
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
This is the PR for feature Search: #605
This PR includes the newly introduced SearchBoxControl in TermControl dir, which is the search bar for the search experience. And the codes that enable Search in Windows Terminal.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
The PR that migrates the Conhost search module: https://github.com/microsoft/terminal/pull/3279
Spec (still actively updating): https://github.com/microsoft/terminal/pull/3299
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes #605
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
These functionalities are included in the search experience.
1. Search in Terminal text buffer.
2. Automatic wrap-around.
3. Search up or down switch by clicking different buttons.
4. Search case sensitively/insensitively by clicking a button. S. Move the search box to the top/bottom by clicking a button.
6. Close by clicking 'X'.
7. Open search by ctrl + F.
When the searchbox is open, the user could still interact with the terminal by clicking the terminal input area.
While I already have the search functionalities, currently there are still some known to-do works and I will keep updating my PR:
1. Optimize the search box UI, this includes:
1) Theme adaptation. The search box background and font color
should change according to the theme,
2) Add background. Currently the elements in search box are all
transparent. However, we need a background.
3) Move button should be highlighted once clicked.
2. Accessibility: search process should be able to performed without mouse. Once the search box is focused, the user should be able to navigate between all interactive elements on the searchbox using keyboard.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
To test:
1. checkout this branch.
2. Build the project.
3. Start Windows Terminal and press Ctrl+F
4. The search box should appear on the top right corner.
2019-12-17 16:52:37 +01:00
|
|
|
<ClInclude Include="SearchBoxControl.h">
|
|
|
|
|
<DependentUpon>SearchBoxControl.xaml</DependentUpon>
|
|
|
|
|
</ClInclude>
|
2019-05-03 00:29:04 +02:00
|
|
|
<ClInclude Include="TermControl.h">
|
2020-02-27 01:35:16 +01:00
|
|
|
<DependentUpon>TermControl.xaml</DependentUpon>
|
2019-05-03 00:29:04 +02:00
|
|
|
</ClInclude>
|
Accessibility: TermControl Automation Peer (#2083)
Builds on the work of #1691 and #1915
Let's start with the easy change:
- `TermControl`'s `controlRoot` was removed. `TermControl` is a `UserControl`
now.
Ok. Now we've got a story to tell here....
### TermControlAP - the Automation Peer
Here's an in-depth guide on custom automation peers:
https://docs.microsoft.com/en-us/windows/uwp/design/accessibility/custom-automation-peers
We have a custom XAML element (TermControl). So XAML can't really hold our
hands and determine an accessible behavior for us. So this automation peer is
responsible for enabling that interaction.
We made it a FrameworkElementAutomationPeer to get as much accessibility as
possible from it just being a XAML element (i.e.: where are we on the screen?
what are my dimensions?). This is recommended. Any functions with "Core" at the
end, are overwritten here to tweak this automation peer into what we really
need.
But what kind of interactions can a user expect from this XAML element?
Introducing ControlPatterns! There's a ton of interfaces that just define "what
can I do". Thankfully, we already know that we're supposed to be
`ScreenInfoUiaProvider` and that was an `ITextProvider`, so let's just make the
TermControlAP an `ITextProvider` too.
So now we have a way to define what accessible actions can be performed on us,
but what should those actions do? Well let's just use the automation providers
from ConHost that are now in a shared space! (Note: this is a great place to
stop and get some coffee. We're about to hop into the .cpp file in the next
section)
### Wrapping our shared Automation Providers
Unfortunately, we can't just use the automation providers from ConHost. Or, at
least not just hook them up as easily as we wish. ConHost's UIA Providers were
written using UIAutomationCore and ITextRangeProiuder. XAML's interfaces
ITextProvider and ITextRangeProvider are lined up to be exactly the same.
So we need to wrap our ConHost UIA Providers (UIAutomationCore) with the XAML
ones. We had two providers, so that means we have two wrappers.
#### TermControlAP (XAML) <----> ScreenInfoUiaProvider (UIAutomationCore)
Each of the functions in the pragma region `ITextProvider` for
TermControlAP.cpp is just wrapping what we do in `ScreenInfoUiaProvider`, and
returning an acceptable version of it.
Most of `ScreenInfoUiaProvider`'s functions return `UiaTextRange`s. So we need
to wrap that too. That's this next section...
#### XamlUiaTextRange (XAML) <----> UiaTextRange (UIAutomationCore)
Same idea. We're wrapping everything that we could do with `UiaTextRange` and
putting it inside of `XamlUiaTextRange`.
### Additional changes to `UiaTextRange` and `ScreenInfoUiaProvider`
If you don't know what I just said, please read this background:
- #1691: how accessibility works and the general responsibility of these two
classes
- #1915: how we pulled these Accessibility Providers into a shared area
TL;DR: `ScreenInfoUiaProvider` lets you interact with the displayed text.
`UiaTextRange` is specific ranges of text in the display and navigate the text.
Thankfully, we didn't do many changes here. I feel like some of it is hacked
together but now that we have a somewhat working system, making changes
shouldn't be too hard...I hope.
#### UiaTextRange
We don't have access to the window handle. We really only need it to draw the
bounding rects using WinUser's `ScreenToClient()` and `ClientToScreen()`. I
need to figure out how to get around this.
In the meantime, I made the window handle optional. And if we don't have
one....well, we need to figure that out. But other than that, we have a
`UiaTextRange`.
#### ScreenInfoUiaProvider
At some point, we need to hook up this automation provider to the
WindowUiaProvider. This should help with navigation of the UIA Tree and make
everything just look waaaay better. For now, let's just do the same approach
and make the pUiaParent optional.
This one's the one I'm not that proud of, but it works. We need the parent to
get a bounding rect of the terminal. While we figure out how to attach the
WindowUiaProvider, we should at the very least be able to get a bunch of info
from our xaml automation peer. So, I've added a _getBoundingRect optional
function. This is what's called when we don't have a WindowUiaProvider as our
parent.
## Validation Steps Performed
I've been using inspect.exe to see the UIA tree.
I was able to interact with the terminal mostly fine. A few known issues below.
Unfortunately, I tried running Narrator on this and it didn't seem to like it
(by that I mean WT crashed). Then again, I don't really know how to use
narrator other than "click on object" --> "listen voice". I feel like there's a
way to get the other interactions with narrator, but I'll be looking into more
of that soon. I bet if I fix the two issues below, Narrator will be happy.
## Miscellaneous Known Issues
- `GetSelection()` and `GetVisibleRanges()` crashes. I need to debug through
these. I want to include them in this PR.
Fixes #1353.
2019-07-31 01:43:10 +02:00
|
|
|
<ClInclude Include="TermControlAutomationPeer.h">
|
|
|
|
|
<DependentUpon>TermControlAutomationPeer.idl</DependentUpon>
|
|
|
|
|
</ClInclude>
|
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919)
TerminalControl doesn't use any of the built in text input and edit
controls provided by XAML for text input, which means TermianlControl
needs to communicate with the Text Services Framework (TSF) in order to
provide Input Method Editor (IME) support. Just like the rest of
Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core)
namespace to provide support vs. the old TSF 1.0.
Windows.UI.Text.Core handles communication between a text edit control
and the text services primarily through a CoreTextEditContext object.
This change introduces a new UserControl TSFInputControl which is a
custom EditControl similar to the CustomEditControl sample[1].
TSFInputControl is similar (overlay with IME text) to how old console
(conimeinfo) handled IME.
# Details
TSFInputControl is a Windows.UI.Xaml.Controls.UserControl
TSFInputControl contains a Canvas control for absolution positioning a
TextBlock control within its containing control (TerminalControl).
The TextBlock control is used for displaying candidate text from the
IME. When the user makes a choice in the IME the TextBlock is cleared
and the text is written to the Terminal buffer like normal text.
TSFInputControl creates an instance of the CoreTextEditContext and
attaches appropriate event handlers to CoreTextEditContext in order to
interact with the IME.
A good write-up on how to interact with CoreTextEditContext can be found
here[2].
## Text Updates
Text updates from the IME come in on the TextUpdating event handler,
text updates are stored in an internal buffer (_inputBuffer).
## Completed Text
Once a user selects a text in the IME, the CompositionCompleted handler
is invoked. The input buffer (_inputBuffer) is written to the Terminal
buffer, _inputBuffer is cleared and Canvas and TextBlock controls are
hidden until the user starts a composition session again.
## Positioning
Telling the IME where to properly position itself was the hardest part
of this change. The IME expects to know it's location in screen
coordinates as supposed to client coordinates. This is pretty easy if
you are a pure UWP, but since we are hosted inside a XAMLIsland the
client to screen coordinate translation is a little harder.
### Calculating Screen Coordinates
1. Obtaining the Window position in Screen coordinates.
2. Determining the Client coordinate of the cursor.
3. Converting the Client coordinate of the cursor to Screen coordinates.
4. Offsetting the X and Y coordinate of the cursor by the position of
the TerminalControl within the window (tabs if present, margins, etc..).
5. Applying any scale factor of the display.
Once we have the right position in screen coordinates, this is supplied
in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets
the IME know where to position itself on the Screen.
## Font Information/Cursor/Writing to Terminal
3 events were added to the TSFInputControl to create a loosely-coupled
implementation between the TerminalControl and the TSFInputControl.
These events are used for obtaining Font information from the
TerminalControl, getting the Cursor position and writing to the terminal
buffer.
## Known Issues
- Width of TextBlock is hardcoded to 200 pixels and most likely should
adjust to the available width of the current input line on the console
(#3640)
- Entering text in the middle of an existing set of text has TextBlock
render under existing text. Current Console behavior here isn't good
experience either (writes over text)
- Text input at edges of window is clipped versus wrapping around to
next line. This isn't any worse than the original command line, but
Terminal should be better (#3657)
## Future Considerations
Ideally, we'd be able to interact with the console buffer directly and
replace characters as the user types.
## Validation
General steps to try functionality
- Open Console
- Switch to Simplified Chinese (Shortcut: Windows+Spacebar)
- Switch to Chinese mode on language bar
Scenarios validated:
- As user types unformatted candidates appear on command line and IME
renders in correct position under unformatted characters.
- User can dismiss IME and text doesn't appear on command line
- Switch back to English mode, functions like normal
- New tab has proper behavior
- Switching between tabs has proper behavior
- Switching away from Terminal Window with IME present causes IME to
disappear
[1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl
[2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input
Closes #459
Closes #2213
Closes #3641
2019-11-22 01:25:50 +01:00
|
|
|
<ClInclude Include="TSFInputControl.h">
|
2020-02-28 20:32:19 +01:00
|
|
|
<DependentUpon>TSFInputControl.xaml</DependentUpon>
|
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919)
TerminalControl doesn't use any of the built in text input and edit
controls provided by XAML for text input, which means TermianlControl
needs to communicate with the Text Services Framework (TSF) in order to
provide Input Method Editor (IME) support. Just like the rest of
Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core)
namespace to provide support vs. the old TSF 1.0.
Windows.UI.Text.Core handles communication between a text edit control
and the text services primarily through a CoreTextEditContext object.
This change introduces a new UserControl TSFInputControl which is a
custom EditControl similar to the CustomEditControl sample[1].
TSFInputControl is similar (overlay with IME text) to how old console
(conimeinfo) handled IME.
# Details
TSFInputControl is a Windows.UI.Xaml.Controls.UserControl
TSFInputControl contains a Canvas control for absolution positioning a
TextBlock control within its containing control (TerminalControl).
The TextBlock control is used for displaying candidate text from the
IME. When the user makes a choice in the IME the TextBlock is cleared
and the text is written to the Terminal buffer like normal text.
TSFInputControl creates an instance of the CoreTextEditContext and
attaches appropriate event handlers to CoreTextEditContext in order to
interact with the IME.
A good write-up on how to interact with CoreTextEditContext can be found
here[2].
## Text Updates
Text updates from the IME come in on the TextUpdating event handler,
text updates are stored in an internal buffer (_inputBuffer).
## Completed Text
Once a user selects a text in the IME, the CompositionCompleted handler
is invoked. The input buffer (_inputBuffer) is written to the Terminal
buffer, _inputBuffer is cleared and Canvas and TextBlock controls are
hidden until the user starts a composition session again.
## Positioning
Telling the IME where to properly position itself was the hardest part
of this change. The IME expects to know it's location in screen
coordinates as supposed to client coordinates. This is pretty easy if
you are a pure UWP, but since we are hosted inside a XAMLIsland the
client to screen coordinate translation is a little harder.
### Calculating Screen Coordinates
1. Obtaining the Window position in Screen coordinates.
2. Determining the Client coordinate of the cursor.
3. Converting the Client coordinate of the cursor to Screen coordinates.
4. Offsetting the X and Y coordinate of the cursor by the position of
the TerminalControl within the window (tabs if present, margins, etc..).
5. Applying any scale factor of the display.
Once we have the right position in screen coordinates, this is supplied
in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets
the IME know where to position itself on the Screen.
## Font Information/Cursor/Writing to Terminal
3 events were added to the TSFInputControl to create a loosely-coupled
implementation between the TerminalControl and the TSFInputControl.
These events are used for obtaining Font information from the
TerminalControl, getting the Cursor position and writing to the terminal
buffer.
## Known Issues
- Width of TextBlock is hardcoded to 200 pixels and most likely should
adjust to the available width of the current input line on the console
(#3640)
- Entering text in the middle of an existing set of text has TextBlock
render under existing text. Current Console behavior here isn't good
experience either (writes over text)
- Text input at edges of window is clipped versus wrapping around to
next line. This isn't any worse than the original command line, but
Terminal should be better (#3657)
## Future Considerations
Ideally, we'd be able to interact with the console buffer directly and
replace characters as the user types.
## Validation
General steps to try functionality
- Open Console
- Switch to Simplified Chinese (Shortcut: Windows+Spacebar)
- Switch to Chinese mode on language bar
Scenarios validated:
- As user types unformatted candidates appear on command line and IME
renders in correct position under unformatted characters.
- User can dismiss IME and text doesn't appear on command line
- Switch back to English mode, functions like normal
- New tab has proper behavior
- Switching between tabs has proper behavior
- Switching away from Terminal Window with IME present causes IME to
disappear
[1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl
[2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input
Closes #459
Closes #2213
Closes #3641
2019-11-22 01:25:50 +01:00
|
|
|
</ClInclude>
|
Accessibility: TermControl Automation Peer (#2083)
Builds on the work of #1691 and #1915
Let's start with the easy change:
- `TermControl`'s `controlRoot` was removed. `TermControl` is a `UserControl`
now.
Ok. Now we've got a story to tell here....
### TermControlAP - the Automation Peer
Here's an in-depth guide on custom automation peers:
https://docs.microsoft.com/en-us/windows/uwp/design/accessibility/custom-automation-peers
We have a custom XAML element (TermControl). So XAML can't really hold our
hands and determine an accessible behavior for us. So this automation peer is
responsible for enabling that interaction.
We made it a FrameworkElementAutomationPeer to get as much accessibility as
possible from it just being a XAML element (i.e.: where are we on the screen?
what are my dimensions?). This is recommended. Any functions with "Core" at the
end, are overwritten here to tweak this automation peer into what we really
need.
But what kind of interactions can a user expect from this XAML element?
Introducing ControlPatterns! There's a ton of interfaces that just define "what
can I do". Thankfully, we already know that we're supposed to be
`ScreenInfoUiaProvider` and that was an `ITextProvider`, so let's just make the
TermControlAP an `ITextProvider` too.
So now we have a way to define what accessible actions can be performed on us,
but what should those actions do? Well let's just use the automation providers
from ConHost that are now in a shared space! (Note: this is a great place to
stop and get some coffee. We're about to hop into the .cpp file in the next
section)
### Wrapping our shared Automation Providers
Unfortunately, we can't just use the automation providers from ConHost. Or, at
least not just hook them up as easily as we wish. ConHost's UIA Providers were
written using UIAutomationCore and ITextRangeProiuder. XAML's interfaces
ITextProvider and ITextRangeProvider are lined up to be exactly the same.
So we need to wrap our ConHost UIA Providers (UIAutomationCore) with the XAML
ones. We had two providers, so that means we have two wrappers.
#### TermControlAP (XAML) <----> ScreenInfoUiaProvider (UIAutomationCore)
Each of the functions in the pragma region `ITextProvider` for
TermControlAP.cpp is just wrapping what we do in `ScreenInfoUiaProvider`, and
returning an acceptable version of it.
Most of `ScreenInfoUiaProvider`'s functions return `UiaTextRange`s. So we need
to wrap that too. That's this next section...
#### XamlUiaTextRange (XAML) <----> UiaTextRange (UIAutomationCore)
Same idea. We're wrapping everything that we could do with `UiaTextRange` and
putting it inside of `XamlUiaTextRange`.
### Additional changes to `UiaTextRange` and `ScreenInfoUiaProvider`
If you don't know what I just said, please read this background:
- #1691: how accessibility works and the general responsibility of these two
classes
- #1915: how we pulled these Accessibility Providers into a shared area
TL;DR: `ScreenInfoUiaProvider` lets you interact with the displayed text.
`UiaTextRange` is specific ranges of text in the display and navigate the text.
Thankfully, we didn't do many changes here. I feel like some of it is hacked
together but now that we have a somewhat working system, making changes
shouldn't be too hard...I hope.
#### UiaTextRange
We don't have access to the window handle. We really only need it to draw the
bounding rects using WinUser's `ScreenToClient()` and `ClientToScreen()`. I
need to figure out how to get around this.
In the meantime, I made the window handle optional. And if we don't have
one....well, we need to figure that out. But other than that, we have a
`UiaTextRange`.
#### ScreenInfoUiaProvider
At some point, we need to hook up this automation provider to the
WindowUiaProvider. This should help with navigation of the UIA Tree and make
everything just look waaaay better. For now, let's just do the same approach
and make the pUiaParent optional.
This one's the one I'm not that proud of, but it works. We need the parent to
get a bounding rect of the terminal. While we figure out how to attach the
WindowUiaProvider, we should at the very least be able to get a bunch of info
from our xaml automation peer. So, I've added a _getBoundingRect optional
function. This is what's called when we don't have a WindowUiaProvider as our
parent.
## Validation Steps Performed
I've been using inspect.exe to see the UIA tree.
I was able to interact with the terminal mostly fine. A few known issues below.
Unfortunately, I tried running Narrator on this and it didn't seem to like it
(by that I mean WT crashed). Then again, I don't really know how to use
narrator other than "click on object" --> "listen voice". I feel like there's a
way to get the other interactions with narrator, but I'll be looking into more
of that soon. I bet if I fix the two issues below, Narrator will be happy.
## Miscellaneous Known Issues
- `GetSelection()` and `GetVisibleRanges()` crashes. I need to debug through
these. I want to include them in this PR.
Fixes #1353.
2019-07-31 01:43:10 +02:00
|
|
|
<ClInclude Include="XamlUiaTextRange.h" />
|
2019-05-03 00:29:04 +02:00
|
|
|
</ItemGroup>
|
|
|
|
|
<ItemGroup>
|
|
|
|
|
<ClCompile Include="pch.cpp">
|
|
|
|
|
<PrecompiledHeader>Create</PrecompiledHeader>
|
|
|
|
|
</ClCompile>
|
2019-07-30 02:24:20 +02:00
|
|
|
<ClCompile Include="init.cpp" />
|
Search - add search box control and implement search experience (#3590)
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
This is the PR for feature Search: #605
This PR includes the newly introduced SearchBoxControl in TermControl dir, which is the search bar for the search experience. And the codes that enable Search in Windows Terminal.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
The PR that migrates the Conhost search module: https://github.com/microsoft/terminal/pull/3279
Spec (still actively updating): https://github.com/microsoft/terminal/pull/3299
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes #605
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
These functionalities are included in the search experience.
1. Search in Terminal text buffer.
2. Automatic wrap-around.
3. Search up or down switch by clicking different buttons.
4. Search case sensitively/insensitively by clicking a button. S. Move the search box to the top/bottom by clicking a button.
6. Close by clicking 'X'.
7. Open search by ctrl + F.
When the searchbox is open, the user could still interact with the terminal by clicking the terminal input area.
While I already have the search functionalities, currently there are still some known to-do works and I will keep updating my PR:
1. Optimize the search box UI, this includes:
1) Theme adaptation. The search box background and font color
should change according to the theme,
2) Add background. Currently the elements in search box are all
transparent. However, we need a background.
3) Move button should be highlighted once clicked.
2. Accessibility: search process should be able to performed without mouse. Once the search box is focused, the user should be able to navigate between all interactive elements on the searchbox using keyboard.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
To test:
1. checkout this branch.
2. Build the project.
3. Start Windows Terminal and press Ctrl+F
4. The search box should appear on the top right corner.
2019-12-17 16:52:37 +01:00
|
|
|
<ClCompile Include="SearchBoxControl.cpp">
|
|
|
|
|
<DependentUpon>SearchBoxControl.xaml</DependentUpon>
|
|
|
|
|
</ClCompile>
|
2019-05-03 00:29:04 +02:00
|
|
|
<ClCompile Include="TermControl.cpp">
|
2020-02-27 01:35:16 +01:00
|
|
|
<DependentUpon>TermControl.xaml</DependentUpon>
|
2019-05-03 00:29:04 +02:00
|
|
|
</ClCompile>
|
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919)
TerminalControl doesn't use any of the built in text input and edit
controls provided by XAML for text input, which means TermianlControl
needs to communicate with the Text Services Framework (TSF) in order to
provide Input Method Editor (IME) support. Just like the rest of
Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core)
namespace to provide support vs. the old TSF 1.0.
Windows.UI.Text.Core handles communication between a text edit control
and the text services primarily through a CoreTextEditContext object.
This change introduces a new UserControl TSFInputControl which is a
custom EditControl similar to the CustomEditControl sample[1].
TSFInputControl is similar (overlay with IME text) to how old console
(conimeinfo) handled IME.
# Details
TSFInputControl is a Windows.UI.Xaml.Controls.UserControl
TSFInputControl contains a Canvas control for absolution positioning a
TextBlock control within its containing control (TerminalControl).
The TextBlock control is used for displaying candidate text from the
IME. When the user makes a choice in the IME the TextBlock is cleared
and the text is written to the Terminal buffer like normal text.
TSFInputControl creates an instance of the CoreTextEditContext and
attaches appropriate event handlers to CoreTextEditContext in order to
interact with the IME.
A good write-up on how to interact with CoreTextEditContext can be found
here[2].
## Text Updates
Text updates from the IME come in on the TextUpdating event handler,
text updates are stored in an internal buffer (_inputBuffer).
## Completed Text
Once a user selects a text in the IME, the CompositionCompleted handler
is invoked. The input buffer (_inputBuffer) is written to the Terminal
buffer, _inputBuffer is cleared and Canvas and TextBlock controls are
hidden until the user starts a composition session again.
## Positioning
Telling the IME where to properly position itself was the hardest part
of this change. The IME expects to know it's location in screen
coordinates as supposed to client coordinates. This is pretty easy if
you are a pure UWP, but since we are hosted inside a XAMLIsland the
client to screen coordinate translation is a little harder.
### Calculating Screen Coordinates
1. Obtaining the Window position in Screen coordinates.
2. Determining the Client coordinate of the cursor.
3. Converting the Client coordinate of the cursor to Screen coordinates.
4. Offsetting the X and Y coordinate of the cursor by the position of
the TerminalControl within the window (tabs if present, margins, etc..).
5. Applying any scale factor of the display.
Once we have the right position in screen coordinates, this is supplied
in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets
the IME know where to position itself on the Screen.
## Font Information/Cursor/Writing to Terminal
3 events were added to the TSFInputControl to create a loosely-coupled
implementation between the TerminalControl and the TSFInputControl.
These events are used for obtaining Font information from the
TerminalControl, getting the Cursor position and writing to the terminal
buffer.
## Known Issues
- Width of TextBlock is hardcoded to 200 pixels and most likely should
adjust to the available width of the current input line on the console
(#3640)
- Entering text in the middle of an existing set of text has TextBlock
render under existing text. Current Console behavior here isn't good
experience either (writes over text)
- Text input at edges of window is clipped versus wrapping around to
next line. This isn't any worse than the original command line, but
Terminal should be better (#3657)
## Future Considerations
Ideally, we'd be able to interact with the console buffer directly and
replace characters as the user types.
## Validation
General steps to try functionality
- Open Console
- Switch to Simplified Chinese (Shortcut: Windows+Spacebar)
- Switch to Chinese mode on language bar
Scenarios validated:
- As user types unformatted candidates appear on command line and IME
renders in correct position under unformatted characters.
- User can dismiss IME and text doesn't appear on command line
- Switch back to English mode, functions like normal
- New tab has proper behavior
- Switching between tabs has proper behavior
- Switching away from Terminal Window with IME present causes IME to
disappear
[1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl
[2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input
Closes #459
Closes #2213
Closes #3641
2019-11-22 01:25:50 +01:00
|
|
|
<ClCompile Include="TSFInputControl.cpp">
|
2020-02-28 20:32:19 +01:00
|
|
|
<DependentUpon>TSFInputControl.xaml</DependentUpon>
|
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919)
TerminalControl doesn't use any of the built in text input and edit
controls provided by XAML for text input, which means TermianlControl
needs to communicate with the Text Services Framework (TSF) in order to
provide Input Method Editor (IME) support. Just like the rest of
Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core)
namespace to provide support vs. the old TSF 1.0.
Windows.UI.Text.Core handles communication between a text edit control
and the text services primarily through a CoreTextEditContext object.
This change introduces a new UserControl TSFInputControl which is a
custom EditControl similar to the CustomEditControl sample[1].
TSFInputControl is similar (overlay with IME text) to how old console
(conimeinfo) handled IME.
# Details
TSFInputControl is a Windows.UI.Xaml.Controls.UserControl
TSFInputControl contains a Canvas control for absolution positioning a
TextBlock control within its containing control (TerminalControl).
The TextBlock control is used for displaying candidate text from the
IME. When the user makes a choice in the IME the TextBlock is cleared
and the text is written to the Terminal buffer like normal text.
TSFInputControl creates an instance of the CoreTextEditContext and
attaches appropriate event handlers to CoreTextEditContext in order to
interact with the IME.
A good write-up on how to interact with CoreTextEditContext can be found
here[2].
## Text Updates
Text updates from the IME come in on the TextUpdating event handler,
text updates are stored in an internal buffer (_inputBuffer).
## Completed Text
Once a user selects a text in the IME, the CompositionCompleted handler
is invoked. The input buffer (_inputBuffer) is written to the Terminal
buffer, _inputBuffer is cleared and Canvas and TextBlock controls are
hidden until the user starts a composition session again.
## Positioning
Telling the IME where to properly position itself was the hardest part
of this change. The IME expects to know it's location in screen
coordinates as supposed to client coordinates. This is pretty easy if
you are a pure UWP, but since we are hosted inside a XAMLIsland the
client to screen coordinate translation is a little harder.
### Calculating Screen Coordinates
1. Obtaining the Window position in Screen coordinates.
2. Determining the Client coordinate of the cursor.
3. Converting the Client coordinate of the cursor to Screen coordinates.
4. Offsetting the X and Y coordinate of the cursor by the position of
the TerminalControl within the window (tabs if present, margins, etc..).
5. Applying any scale factor of the display.
Once we have the right position in screen coordinates, this is supplied
in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets
the IME know where to position itself on the Screen.
## Font Information/Cursor/Writing to Terminal
3 events were added to the TSFInputControl to create a loosely-coupled
implementation between the TerminalControl and the TSFInputControl.
These events are used for obtaining Font information from the
TerminalControl, getting the Cursor position and writing to the terminal
buffer.
## Known Issues
- Width of TextBlock is hardcoded to 200 pixels and most likely should
adjust to the available width of the current input line on the console
(#3640)
- Entering text in the middle of an existing set of text has TextBlock
render under existing text. Current Console behavior here isn't good
experience either (writes over text)
- Text input at edges of window is clipped versus wrapping around to
next line. This isn't any worse than the original command line, but
Terminal should be better (#3657)
## Future Considerations
Ideally, we'd be able to interact with the console buffer directly and
replace characters as the user types.
## Validation
General steps to try functionality
- Open Console
- Switch to Simplified Chinese (Shortcut: Windows+Spacebar)
- Switch to Chinese mode on language bar
Scenarios validated:
- As user types unformatted candidates appear on command line and IME
renders in correct position under unformatted characters.
- User can dismiss IME and text doesn't appear on command line
- Switch back to English mode, functions like normal
- New tab has proper behavior
- Switching between tabs has proper behavior
- Switching away from Terminal Window with IME present causes IME to
disappear
[1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl
[2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input
Closes #459
Closes #2213
Closes #3641
2019-11-22 01:25:50 +01:00
|
|
|
</ClCompile>
|
2019-05-03 00:29:04 +02:00
|
|
|
<ClCompile Include="$(GeneratedFilesDir)module.g.cpp" />
|
Accessibility: TermControl Automation Peer (#2083)
Builds on the work of #1691 and #1915
Let's start with the easy change:
- `TermControl`'s `controlRoot` was removed. `TermControl` is a `UserControl`
now.
Ok. Now we've got a story to tell here....
### TermControlAP - the Automation Peer
Here's an in-depth guide on custom automation peers:
https://docs.microsoft.com/en-us/windows/uwp/design/accessibility/custom-automation-peers
We have a custom XAML element (TermControl). So XAML can't really hold our
hands and determine an accessible behavior for us. So this automation peer is
responsible for enabling that interaction.
We made it a FrameworkElementAutomationPeer to get as much accessibility as
possible from it just being a XAML element (i.e.: where are we on the screen?
what are my dimensions?). This is recommended. Any functions with "Core" at the
end, are overwritten here to tweak this automation peer into what we really
need.
But what kind of interactions can a user expect from this XAML element?
Introducing ControlPatterns! There's a ton of interfaces that just define "what
can I do". Thankfully, we already know that we're supposed to be
`ScreenInfoUiaProvider` and that was an `ITextProvider`, so let's just make the
TermControlAP an `ITextProvider` too.
So now we have a way to define what accessible actions can be performed on us,
but what should those actions do? Well let's just use the automation providers
from ConHost that are now in a shared space! (Note: this is a great place to
stop and get some coffee. We're about to hop into the .cpp file in the next
section)
### Wrapping our shared Automation Providers
Unfortunately, we can't just use the automation providers from ConHost. Or, at
least not just hook them up as easily as we wish. ConHost's UIA Providers were
written using UIAutomationCore and ITextRangeProiuder. XAML's interfaces
ITextProvider and ITextRangeProvider are lined up to be exactly the same.
So we need to wrap our ConHost UIA Providers (UIAutomationCore) with the XAML
ones. We had two providers, so that means we have two wrappers.
#### TermControlAP (XAML) <----> ScreenInfoUiaProvider (UIAutomationCore)
Each of the functions in the pragma region `ITextProvider` for
TermControlAP.cpp is just wrapping what we do in `ScreenInfoUiaProvider`, and
returning an acceptable version of it.
Most of `ScreenInfoUiaProvider`'s functions return `UiaTextRange`s. So we need
to wrap that too. That's this next section...
#### XamlUiaTextRange (XAML) <----> UiaTextRange (UIAutomationCore)
Same idea. We're wrapping everything that we could do with `UiaTextRange` and
putting it inside of `XamlUiaTextRange`.
### Additional changes to `UiaTextRange` and `ScreenInfoUiaProvider`
If you don't know what I just said, please read this background:
- #1691: how accessibility works and the general responsibility of these two
classes
- #1915: how we pulled these Accessibility Providers into a shared area
TL;DR: `ScreenInfoUiaProvider` lets you interact with the displayed text.
`UiaTextRange` is specific ranges of text in the display and navigate the text.
Thankfully, we didn't do many changes here. I feel like some of it is hacked
together but now that we have a somewhat working system, making changes
shouldn't be too hard...I hope.
#### UiaTextRange
We don't have access to the window handle. We really only need it to draw the
bounding rects using WinUser's `ScreenToClient()` and `ClientToScreen()`. I
need to figure out how to get around this.
In the meantime, I made the window handle optional. And if we don't have
one....well, we need to figure that out. But other than that, we have a
`UiaTextRange`.
#### ScreenInfoUiaProvider
At some point, we need to hook up this automation provider to the
WindowUiaProvider. This should help with navigation of the UIA Tree and make
everything just look waaaay better. For now, let's just do the same approach
and make the pUiaParent optional.
This one's the one I'm not that proud of, but it works. We need the parent to
get a bounding rect of the terminal. While we figure out how to attach the
WindowUiaProvider, we should at the very least be able to get a bunch of info
from our xaml automation peer. So, I've added a _getBoundingRect optional
function. This is what's called when we don't have a WindowUiaProvider as our
parent.
## Validation Steps Performed
I've been using inspect.exe to see the UIA tree.
I was able to interact with the terminal mostly fine. A few known issues below.
Unfortunately, I tried running Narrator on this and it didn't seem to like it
(by that I mean WT crashed). Then again, I don't really know how to use
narrator other than "click on object" --> "listen voice". I feel like there's a
way to get the other interactions with narrator, but I'll be looking into more
of that soon. I bet if I fix the two issues below, Narrator will be happy.
## Miscellaneous Known Issues
- `GetSelection()` and `GetVisibleRanges()` crashes. I need to debug through
these. I want to include them in this PR.
Fixes #1353.
2019-07-31 01:43:10 +02:00
|
|
|
<ClCompile Include="TermControlAutomationPeer.cpp">
|
|
|
|
|
<DependentUpon>TermControlAutomationPeer.idl</DependentUpon>
|
|
|
|
|
</ClCompile>
|
|
|
|
|
<ClCompile Include="XamlUiaTextRange.cpp" />
|
2019-05-03 00:29:04 +02:00
|
|
|
</ItemGroup>
|
|
|
|
|
<ItemGroup>
|
Search - add search box control and implement search experience (#3590)
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
This is the PR for feature Search: #605
This PR includes the newly introduced SearchBoxControl in TermControl dir, which is the search bar for the search experience. And the codes that enable Search in Windows Terminal.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
The PR that migrates the Conhost search module: https://github.com/microsoft/terminal/pull/3279
Spec (still actively updating): https://github.com/microsoft/terminal/pull/3299
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes #605
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
These functionalities are included in the search experience.
1. Search in Terminal text buffer.
2. Automatic wrap-around.
3. Search up or down switch by clicking different buttons.
4. Search case sensitively/insensitively by clicking a button. S. Move the search box to the top/bottom by clicking a button.
6. Close by clicking 'X'.
7. Open search by ctrl + F.
When the searchbox is open, the user could still interact with the terminal by clicking the terminal input area.
While I already have the search functionalities, currently there are still some known to-do works and I will keep updating my PR:
1. Optimize the search box UI, this includes:
1) Theme adaptation. The search box background and font color
should change according to the theme,
2) Add background. Currently the elements in search box are all
transparent. However, we need a background.
3) Move button should be highlighted once clicked.
2. Accessibility: search process should be able to performed without mouse. Once the search box is focused, the user should be able to navigate between all interactive elements on the searchbox using keyboard.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
To test:
1. checkout this branch.
2. Build the project.
3. Start Windows Terminal and press Ctrl+F
4. The search box should appear on the top right corner.
2019-12-17 16:52:37 +01:00
|
|
|
<Midl Include="SearchBoxControl.idl">
|
|
|
|
|
<DependentUpon>SearchBoxControl.xaml</DependentUpon>
|
|
|
|
|
</Midl>
|
2020-02-27 01:35:16 +01:00
|
|
|
<Midl Include="TermControl.idl">
|
|
|
|
|
<DependentUpon>TermControl.xaml</DependentUpon>
|
|
|
|
|
</Midl>
|
Accessibility: TermControl Automation Peer (#2083)
Builds on the work of #1691 and #1915
Let's start with the easy change:
- `TermControl`'s `controlRoot` was removed. `TermControl` is a `UserControl`
now.
Ok. Now we've got a story to tell here....
### TermControlAP - the Automation Peer
Here's an in-depth guide on custom automation peers:
https://docs.microsoft.com/en-us/windows/uwp/design/accessibility/custom-automation-peers
We have a custom XAML element (TermControl). So XAML can't really hold our
hands and determine an accessible behavior for us. So this automation peer is
responsible for enabling that interaction.
We made it a FrameworkElementAutomationPeer to get as much accessibility as
possible from it just being a XAML element (i.e.: where are we on the screen?
what are my dimensions?). This is recommended. Any functions with "Core" at the
end, are overwritten here to tweak this automation peer into what we really
need.
But what kind of interactions can a user expect from this XAML element?
Introducing ControlPatterns! There's a ton of interfaces that just define "what
can I do". Thankfully, we already know that we're supposed to be
`ScreenInfoUiaProvider` and that was an `ITextProvider`, so let's just make the
TermControlAP an `ITextProvider` too.
So now we have a way to define what accessible actions can be performed on us,
but what should those actions do? Well let's just use the automation providers
from ConHost that are now in a shared space! (Note: this is a great place to
stop and get some coffee. We're about to hop into the .cpp file in the next
section)
### Wrapping our shared Automation Providers
Unfortunately, we can't just use the automation providers from ConHost. Or, at
least not just hook them up as easily as we wish. ConHost's UIA Providers were
written using UIAutomationCore and ITextRangeProiuder. XAML's interfaces
ITextProvider and ITextRangeProvider are lined up to be exactly the same.
So we need to wrap our ConHost UIA Providers (UIAutomationCore) with the XAML
ones. We had two providers, so that means we have two wrappers.
#### TermControlAP (XAML) <----> ScreenInfoUiaProvider (UIAutomationCore)
Each of the functions in the pragma region `ITextProvider` for
TermControlAP.cpp is just wrapping what we do in `ScreenInfoUiaProvider`, and
returning an acceptable version of it.
Most of `ScreenInfoUiaProvider`'s functions return `UiaTextRange`s. So we need
to wrap that too. That's this next section...
#### XamlUiaTextRange (XAML) <----> UiaTextRange (UIAutomationCore)
Same idea. We're wrapping everything that we could do with `UiaTextRange` and
putting it inside of `XamlUiaTextRange`.
### Additional changes to `UiaTextRange` and `ScreenInfoUiaProvider`
If you don't know what I just said, please read this background:
- #1691: how accessibility works and the general responsibility of these two
classes
- #1915: how we pulled these Accessibility Providers into a shared area
TL;DR: `ScreenInfoUiaProvider` lets you interact with the displayed text.
`UiaTextRange` is specific ranges of text in the display and navigate the text.
Thankfully, we didn't do many changes here. I feel like some of it is hacked
together but now that we have a somewhat working system, making changes
shouldn't be too hard...I hope.
#### UiaTextRange
We don't have access to the window handle. We really only need it to draw the
bounding rects using WinUser's `ScreenToClient()` and `ClientToScreen()`. I
need to figure out how to get around this.
In the meantime, I made the window handle optional. And if we don't have
one....well, we need to figure that out. But other than that, we have a
`UiaTextRange`.
#### ScreenInfoUiaProvider
At some point, we need to hook up this automation provider to the
WindowUiaProvider. This should help with navigation of the UIA Tree and make
everything just look waaaay better. For now, let's just do the same approach
and make the pUiaParent optional.
This one's the one I'm not that proud of, but it works. We need the parent to
get a bounding rect of the terminal. While we figure out how to attach the
WindowUiaProvider, we should at the very least be able to get a bunch of info
from our xaml automation peer. So, I've added a _getBoundingRect optional
function. This is what's called when we don't have a WindowUiaProvider as our
parent.
## Validation Steps Performed
I've been using inspect.exe to see the UIA tree.
I was able to interact with the terminal mostly fine. A few known issues below.
Unfortunately, I tried running Narrator on this and it didn't seem to like it
(by that I mean WT crashed). Then again, I don't really know how to use
narrator other than "click on object" --> "listen voice". I feel like there's a
way to get the other interactions with narrator, but I'll be looking into more
of that soon. I bet if I fix the two issues below, Narrator will be happy.
## Miscellaneous Known Issues
- `GetSelection()` and `GetVisibleRanges()` crashes. I need to debug through
these. I want to include them in this PR.
Fixes #1353.
2019-07-31 01:43:10 +02:00
|
|
|
<Midl Include="TermControlAutomationPeer.idl" />
|
Manually pass mouse wheel messages to TermControls (#5131)
## Summary of the Pull Request
As we've learned in #979, not all touchpads are created equal. Some of them have bad drivers that makes scrolling inactive windows not work. For whatever reason, these devices think the Terminal is all one giant inactive window, so we don't get the mouse wheel events through the XAML stack. We do however get the event as a `WM_MOUSEWHEEL` on those devices (a message we don't get on devices with normally functioning trackpads).
This PR attempts to take that `WM_MOUSEWHEEL` and manually dispatch it to the `TermControl`, so we can at least scroll the terminal content.
Unfortunately, this solution is not very general purpose. This only works to scroll controls that manually implement our own `IMouseWheelListener` interface. As we add more controls, we'll need to continue manually implementing this interface, until the underlying XAML Islands bug is fixed. **I don't love this**. I'd rather have a better solution, but it seems that we can't synthesize a more general-purpose `PointerWheeled` event that could get routed through the XAML tree as normal.
## References
* #2606 and microsoft/microsoft-ui-xaml#2101 - these bugs are also tracking a similar "inactive windows" / "scaled mouse events" issue in XAML
## PR Checklist
* [x] Closes #979
* [x] I work here
* [ ] Tests added/passed
* [n/a] Requires documentation to be updated
## Detailed Description of the Pull Request / Additional comments
I've also added a `til::point` conversion _to_ `winrt::Windows::Foundation::Point`, and some scaling operators for `point`
## Validation Steps Performed
* It works on my HP Spectre 2017 with a synaptics trackpad
- I also made sure to test that `tmux` works in panes on this laptop
* It works on my slaptop, and DOESN'T follow this hack codepath on this machine.
2020-04-01 18:58:16 +02:00
|
|
|
<Midl Include="IMouseWheelListener.idl" />
|
2020-02-28 20:32:19 +01:00
|
|
|
<Midl Include="TSFInputControl.idl">
|
|
|
|
|
<DependentUpon>TSFInputControl.xaml</DependentUpon>
|
|
|
|
|
</Midl>
|
2019-05-03 00:29:04 +02:00
|
|
|
</ItemGroup>
|
|
|
|
|
<ItemGroup>
|
|
|
|
|
<None Include="packages.config" />
|
|
|
|
|
<None Include="TerminalControl.def" />
|
|
|
|
|
</ItemGroup>
|
2019-12-21 02:35:31 +01:00
|
|
|
<ItemGroup>
|
2020-06-08 23:01:47 +02:00
|
|
|
<PRIResource Include="Resources\en-US\Resources.resw" />
|
|
|
|
|
<OCResourceDirectory Include="Resources" />
|
2019-12-21 02:35:31 +01:00
|
|
|
</ItemGroup>
|
2019-05-03 00:29:04 +02:00
|
|
|
<!-- ========================= Project References ======================== -->
|
|
|
|
|
<ItemGroup>
|
2019-09-30 19:39:55 +02:00
|
|
|
<ProjectReference Include="..\..\types\lib\types.vcxproj" />
|
2019-05-03 00:29:04 +02:00
|
|
|
<ProjectReference Include="..\..\buffer\out\lib\bufferout.vcxproj" />
|
|
|
|
|
<ProjectReference Include="$(OpenConsoleDir)src\renderer\base\lib\base.vcxproj" />
|
|
|
|
|
<ProjectReference Include="..\..\renderer\dx\lib\dx.vcxproj" />
|
2019-12-11 22:52:49 +01:00
|
|
|
<ProjectReference Include="..\..\renderer\uia\lib\uia.vcxproj" />
|
2019-05-03 00:29:04 +02:00
|
|
|
<ProjectReference Include="..\..\terminal\parser\lib\parser.vcxproj" />
|
|
|
|
|
<ProjectReference Include="$(OpenConsoleDir)src\terminal\input\lib\terminalinput.vcxproj" />
|
Refactor TerminalApp and Add Tests for Xaml Content (#1164)
* Refactors TerminalApp into two projects:
- TerminalAppLib, which builds a .lib, and includes all the code
- TerminalApp, which builds a dll by linking the lib
* Adds a TerminalApp.Unit.Tests project
- Includes the ability to test cppwinrt types we've authored using a SxS manifest for unpackaged winrt activation
- includes the ability to test types with XAML content using an appxmanifest
* Adds a giant doc explaining how this was all done. Really, just go read that doc, it'll really help you understand what's going on in this PR.
-------------------------
These are some previous commit messages. They may be helpful to future readers.
* Start adding unittests for json parsing, end up creating a TerminalAppLib project to make a lib. See #1042
* VS automatically did this for me
* This is a dead end
I tried including the idl-y things into the lib, but that way leads insanity
If you want to make a StaticLibrary, then suddenly the winrt toolchain forgets
that ProjectReferences can have winmd's in them, so it won't be able to
compile any types from the referenced projects. If you instead try to manually
reference the types, you'll get duplicate types up the wazoo, which of course
is insane, since we're referencing them the _one_ time
* Yea just follow #1042 on github for status
So current state:
1. If you try to add a `Reference` to all of MUX.Markup, TerminalControl and
TerminalSettings, then mdmerge will complain about all the types from
TerminalSettings being defined twice. In this magic scenario, the
dependencies of TerminalControl are used directly for some reason:
```
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalSettings\Microsoft.Terminal.Settings.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.Settings.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.TerminalConnection.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.TerminalControl.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\Microsoft.UI.Xaml.Markup\Microsoft.UI.Xaml.Markup.winmd.
```
2. If you don't add a `Reference` TerminalControl, then it'll complain about
being unable to find the type TitleChangedEventArgs, which is defined in
TerminalControl.
3. If you don't add a `Reference` TerminalSettings, then it'll complain about
being unable to find the type KeyChord and other types from
TerminalSettings. In this scenario, it doesn't recurse on the other
dependencies from TerminalControl for whatever reason.
4. If you instead try to add all 3 as a `ProjectReference`, then it'll
complain about being unable to find TitleChangedEventArgs, as in 2.
Presumably, it;ll have troubles with the other types too, as none of the 3
are actually included in the midlrt.rsp file.
5. If you add all 3 as a `ProjectReference`, then also add TerminalControl as
a `Reference`, you'll get a `MIDL2011: [msg] unresolved type declaration
Microsoft.UI.Xaml.Markup.XamlApplication`
6. If you add all 3 as a `ProjectReference`, then also add TerminalControl AND
MUX.Markup as a `Reference`, you'll get the same result as 3.
* what if we just don't idl
This seems to compile
* This compiles but I broke the MUX resources
look at the App.xaml change. in this changelist. That's what's broken right now. Lets fix that!
* lets do this
If I leave the MUX nuget out of the project, I'll get a compile error in
App.xaml:
```
...OpenConsole\src\cascadia\TerminalApp\App.xaml(21,40): XamlCompiler error WMC0001: Unknown type 'XamlControlsResources' in XML namespace 'using:Microsoft.UI.Xaml.Controls'
```
If I add it back to the project, it works
* Some cleanup from the previous commit
* This is busted again.
Doing a clean build didn't work.
A clean rebuild of the project, paired with some removal of dead code
revealed a problem with what I have so far.
TerminalAppLib depends on the generation of two headers,
`AppKeyBindings.g.h` and `App.g.h`, as those define some of bits of the
winrt types. They're needed to be able to compile the implementations.
Presumably that's not getting generated by the lib project, because the dll
project is the one to generate that file.
So we need to move the idl's to the lib project. This created maddness,
because of course the Duplicate Type thing. The solution to that is to
actually mark the winrt DLLs that we're chaining up through us as
```
<Private>false</Private>
<CopyLocalSatelliteAssemblies>false</CopyLocalSatelliteAssemblies>
```
This will prevent them from getting double-included.
This still doesn't work however, since
```
app.cpp(40): error C2039: 'XamlMetaDataProvider': is not a member of 'winrt::TerminalApp'
error C3861: 'XamlMetaDataProvider': identifier not found
```
So we need to figure that out. The dll project is still generating the right
header, so lets look there.
* Move the xaml stuff to the lib
This compiles, but when we launch, we fail to load the tabviewcontrol
resources again. So that's not what you want. Why is it not included?
* It works again!
* Use the pri, xbf files from TerminalAppLib, not TerminalApp
* Manually make TerminalApp include a reference to TerminalAppLib's
TerminalApp.winmd. This will force the build to copy TerminalApp.winmd to
TerminalApp/, which WindowsTerminal needs to be able to ProjectReference the
TerminalApp project (it's expecting it to have a winmd)
* Remove the module.g.cpp from TerminalApp, and move to TerminalAppLib. The
dll doesn't do any codegen anymore.
* Agressively clean up these files
* Clean up unnecessary includes in the dll pch.h
* This does NOT work.
The WindowsxamlManager call crashes. I'm thinking it has to do with activation
of winrt types from a dll.
Email out to @Austin-Lamb to see if he can assist
* This gets our cppwinrt types working, but xaml islands is still broken
* Split the tests apart, so they aren't insane
* These are the magic words to make xaml islands work
* All this witchcraft is necessary to make XAML+MUX work right
* Clean this up a bit and add comments
* Create an enormous doc explaining this madness
* Unsure how this got changed.
* Trying to get the CI build to work again.
This resolves the MUX issue. We need to manually include it, because their package's target doesn't mark it as CopyLocalSatelliteAssemblies=false, Private=false.
However, the TerminalApp project is still able to magically reason that the TerminalAppLib project should be included in the MdMerge step, because it think's it's a `GetCppWinRTStaticProjectReferences` reference.
* Update cppwinrt to the latest version - this fixes the MSBuild
* I still need to re-add the KeyModifiers checks from TermControl. I think
this update broke `operator&` for that enum.
* There needs to be some cleanup obviously
* The doc should be updated as well
* Clean up changes from cppwinrt update
* Try doing this, even though it seems wrong
* Lets try this (press x to doubt)
* Clean up vcxproj file, and remove appxmanifest change from previous commit
* Update to the latest TAEF release, maybe that'll work
* Let's try a prerelease version, shall we?
* Add notes about TAEF package, comment out tests
* Format the code
* Hopefully fix the arm64 and x86 builds
also a typo
* Fix PR nits
* Fix some bad merge conflicts
* Some cleanup from the merge
* Well I was close to getting the merge right
* I believe this will fix CI
* Apply suggestions from code review
Co-Authored-By: Carlos Zamora <carlos.zamora@microsoft.com>
* These definitely need to be fixed
* Try version detecting in the test
IDK if this will build, I'm letting the CI try while I clean rebuild locally
* Try blindly updating to the newest nuget version
* Revert "Try blindly updating to the newest nuget version"
This reverts commit b72bd9eb73cca9c3a9887e4d7a67ec2696dc6dba.
* We're just going to see if these work in CI with this change
* Comment the tests back out. Windows Server 2019 is 10.0.17763.557
* Remove the nuget package
We don't need this package anymore now that we're hosting it
* Okay this _was_ important
2019-07-15 21:27:56 +02:00
|
|
|
<ProjectReference Include="$(OpenConsoleDir)src\cascadia\TerminalSettings\TerminalSettings.vcxproj">
|
|
|
|
|
<Private>false</Private>
|
|
|
|
|
<CopyLocalSatelliteAssemblies>false</CopyLocalSatelliteAssemblies>
|
|
|
|
|
</ProjectReference>
|
2019-05-03 00:29:04 +02:00
|
|
|
<ProjectReference Include="$(OpenConsoleDir)src\cascadia\TerminalCore\lib\TerminalCore-lib.vcxproj" />
|
Accessibility: TermControl Automation Peer (#2083)
Builds on the work of #1691 and #1915
Let's start with the easy change:
- `TermControl`'s `controlRoot` was removed. `TermControl` is a `UserControl`
now.
Ok. Now we've got a story to tell here....
### TermControlAP - the Automation Peer
Here's an in-depth guide on custom automation peers:
https://docs.microsoft.com/en-us/windows/uwp/design/accessibility/custom-automation-peers
We have a custom XAML element (TermControl). So XAML can't really hold our
hands and determine an accessible behavior for us. So this automation peer is
responsible for enabling that interaction.
We made it a FrameworkElementAutomationPeer to get as much accessibility as
possible from it just being a XAML element (i.e.: where are we on the screen?
what are my dimensions?). This is recommended. Any functions with "Core" at the
end, are overwritten here to tweak this automation peer into what we really
need.
But what kind of interactions can a user expect from this XAML element?
Introducing ControlPatterns! There's a ton of interfaces that just define "what
can I do". Thankfully, we already know that we're supposed to be
`ScreenInfoUiaProvider` and that was an `ITextProvider`, so let's just make the
TermControlAP an `ITextProvider` too.
So now we have a way to define what accessible actions can be performed on us,
but what should those actions do? Well let's just use the automation providers
from ConHost that are now in a shared space! (Note: this is a great place to
stop and get some coffee. We're about to hop into the .cpp file in the next
section)
### Wrapping our shared Automation Providers
Unfortunately, we can't just use the automation providers from ConHost. Or, at
least not just hook them up as easily as we wish. ConHost's UIA Providers were
written using UIAutomationCore and ITextRangeProiuder. XAML's interfaces
ITextProvider and ITextRangeProvider are lined up to be exactly the same.
So we need to wrap our ConHost UIA Providers (UIAutomationCore) with the XAML
ones. We had two providers, so that means we have two wrappers.
#### TermControlAP (XAML) <----> ScreenInfoUiaProvider (UIAutomationCore)
Each of the functions in the pragma region `ITextProvider` for
TermControlAP.cpp is just wrapping what we do in `ScreenInfoUiaProvider`, and
returning an acceptable version of it.
Most of `ScreenInfoUiaProvider`'s functions return `UiaTextRange`s. So we need
to wrap that too. That's this next section...
#### XamlUiaTextRange (XAML) <----> UiaTextRange (UIAutomationCore)
Same idea. We're wrapping everything that we could do with `UiaTextRange` and
putting it inside of `XamlUiaTextRange`.
### Additional changes to `UiaTextRange` and `ScreenInfoUiaProvider`
If you don't know what I just said, please read this background:
- #1691: how accessibility works and the general responsibility of these two
classes
- #1915: how we pulled these Accessibility Providers into a shared area
TL;DR: `ScreenInfoUiaProvider` lets you interact with the displayed text.
`UiaTextRange` is specific ranges of text in the display and navigate the text.
Thankfully, we didn't do many changes here. I feel like some of it is hacked
together but now that we have a somewhat working system, making changes
shouldn't be too hard...I hope.
#### UiaTextRange
We don't have access to the window handle. We really only need it to draw the
bounding rects using WinUser's `ScreenToClient()` and `ClientToScreen()`. I
need to figure out how to get around this.
In the meantime, I made the window handle optional. And if we don't have
one....well, we need to figure that out. But other than that, we have a
`UiaTextRange`.
#### ScreenInfoUiaProvider
At some point, we need to hook up this automation provider to the
WindowUiaProvider. This should help with navigation of the UIA Tree and make
everything just look waaaay better. For now, let's just do the same approach
and make the pUiaParent optional.
This one's the one I'm not that proud of, but it works. We need the parent to
get a bounding rect of the terminal. While we figure out how to attach the
WindowUiaProvider, we should at the very least be able to get a bunch of info
from our xaml automation peer. So, I've added a _getBoundingRect optional
function. This is what's called when we don't have a WindowUiaProvider as our
parent.
## Validation Steps Performed
I've been using inspect.exe to see the UIA tree.
I was able to interact with the terminal mostly fine. A few known issues below.
Unfortunately, I tried running Narrator on this and it didn't seem to like it
(by that I mean WT crashed). Then again, I don't really know how to use
narrator other than "click on object" --> "listen voice". I feel like there's a
way to get the other interactions with narrator, but I'll be looking into more
of that soon. I bet if I fix the two issues below, Narrator will be happy.
## Miscellaneous Known Issues
- `GetSelection()` and `GetVisibleRanges()` crashes. I need to debug through
these. I want to include them in this PR.
Fixes #1353.
2019-07-31 01:43:10 +02:00
|
|
|
<ProjectReference Include="$(OpenConsoleDir)src\cascadia\TerminalConnection\TerminalConnection.vcxproj">
|
Refactor TerminalApp and Add Tests for Xaml Content (#1164)
* Refactors TerminalApp into two projects:
- TerminalAppLib, which builds a .lib, and includes all the code
- TerminalApp, which builds a dll by linking the lib
* Adds a TerminalApp.Unit.Tests project
- Includes the ability to test cppwinrt types we've authored using a SxS manifest for unpackaged winrt activation
- includes the ability to test types with XAML content using an appxmanifest
* Adds a giant doc explaining how this was all done. Really, just go read that doc, it'll really help you understand what's going on in this PR.
-------------------------
These are some previous commit messages. They may be helpful to future readers.
* Start adding unittests for json parsing, end up creating a TerminalAppLib project to make a lib. See #1042
* VS automatically did this for me
* This is a dead end
I tried including the idl-y things into the lib, but that way leads insanity
If you want to make a StaticLibrary, then suddenly the winrt toolchain forgets
that ProjectReferences can have winmd's in them, so it won't be able to
compile any types from the referenced projects. If you instead try to manually
reference the types, you'll get duplicate types up the wazoo, which of course
is insane, since we're referencing them the _one_ time
* Yea just follow #1042 on github for status
So current state:
1. If you try to add a `Reference` to all of MUX.Markup, TerminalControl and
TerminalSettings, then mdmerge will complain about all the types from
TerminalSettings being defined twice. In this magic scenario, the
dependencies of TerminalControl are used directly for some reason:
```
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalSettings\Microsoft.Terminal.Settings.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.Settings.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.TerminalConnection.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.TerminalControl.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\Microsoft.UI.Xaml.Markup\Microsoft.UI.Xaml.Markup.winmd.
```
2. If you don't add a `Reference` TerminalControl, then it'll complain about
being unable to find the type TitleChangedEventArgs, which is defined in
TerminalControl.
3. If you don't add a `Reference` TerminalSettings, then it'll complain about
being unable to find the type KeyChord and other types from
TerminalSettings. In this scenario, it doesn't recurse on the other
dependencies from TerminalControl for whatever reason.
4. If you instead try to add all 3 as a `ProjectReference`, then it'll
complain about being unable to find TitleChangedEventArgs, as in 2.
Presumably, it;ll have troubles with the other types too, as none of the 3
are actually included in the midlrt.rsp file.
5. If you add all 3 as a `ProjectReference`, then also add TerminalControl as
a `Reference`, you'll get a `MIDL2011: [msg] unresolved type declaration
Microsoft.UI.Xaml.Markup.XamlApplication`
6. If you add all 3 as a `ProjectReference`, then also add TerminalControl AND
MUX.Markup as a `Reference`, you'll get the same result as 3.
* what if we just don't idl
This seems to compile
* This compiles but I broke the MUX resources
look at the App.xaml change. in this changelist. That's what's broken right now. Lets fix that!
* lets do this
If I leave the MUX nuget out of the project, I'll get a compile error in
App.xaml:
```
...OpenConsole\src\cascadia\TerminalApp\App.xaml(21,40): XamlCompiler error WMC0001: Unknown type 'XamlControlsResources' in XML namespace 'using:Microsoft.UI.Xaml.Controls'
```
If I add it back to the project, it works
* Some cleanup from the previous commit
* This is busted again.
Doing a clean build didn't work.
A clean rebuild of the project, paired with some removal of dead code
revealed a problem with what I have so far.
TerminalAppLib depends on the generation of two headers,
`AppKeyBindings.g.h` and `App.g.h`, as those define some of bits of the
winrt types. They're needed to be able to compile the implementations.
Presumably that's not getting generated by the lib project, because the dll
project is the one to generate that file.
So we need to move the idl's to the lib project. This created maddness,
because of course the Duplicate Type thing. The solution to that is to
actually mark the winrt DLLs that we're chaining up through us as
```
<Private>false</Private>
<CopyLocalSatelliteAssemblies>false</CopyLocalSatelliteAssemblies>
```
This will prevent them from getting double-included.
This still doesn't work however, since
```
app.cpp(40): error C2039: 'XamlMetaDataProvider': is not a member of 'winrt::TerminalApp'
error C3861: 'XamlMetaDataProvider': identifier not found
```
So we need to figure that out. The dll project is still generating the right
header, so lets look there.
* Move the xaml stuff to the lib
This compiles, but when we launch, we fail to load the tabviewcontrol
resources again. So that's not what you want. Why is it not included?
* It works again!
* Use the pri, xbf files from TerminalAppLib, not TerminalApp
* Manually make TerminalApp include a reference to TerminalAppLib's
TerminalApp.winmd. This will force the build to copy TerminalApp.winmd to
TerminalApp/, which WindowsTerminal needs to be able to ProjectReference the
TerminalApp project (it's expecting it to have a winmd)
* Remove the module.g.cpp from TerminalApp, and move to TerminalAppLib. The
dll doesn't do any codegen anymore.
* Agressively clean up these files
* Clean up unnecessary includes in the dll pch.h
* This does NOT work.
The WindowsxamlManager call crashes. I'm thinking it has to do with activation
of winrt types from a dll.
Email out to @Austin-Lamb to see if he can assist
* This gets our cppwinrt types working, but xaml islands is still broken
* Split the tests apart, so they aren't insane
* These are the magic words to make xaml islands work
* All this witchcraft is necessary to make XAML+MUX work right
* Clean this up a bit and add comments
* Create an enormous doc explaining this madness
* Unsure how this got changed.
* Trying to get the CI build to work again.
This resolves the MUX issue. We need to manually include it, because their package's target doesn't mark it as CopyLocalSatelliteAssemblies=false, Private=false.
However, the TerminalApp project is still able to magically reason that the TerminalAppLib project should be included in the MdMerge step, because it think's it's a `GetCppWinRTStaticProjectReferences` reference.
* Update cppwinrt to the latest version - this fixes the MSBuild
* I still need to re-add the KeyModifiers checks from TermControl. I think
this update broke `operator&` for that enum.
* There needs to be some cleanup obviously
* The doc should be updated as well
* Clean up changes from cppwinrt update
* Try doing this, even though it seems wrong
* Lets try this (press x to doubt)
* Clean up vcxproj file, and remove appxmanifest change from previous commit
* Update to the latest TAEF release, maybe that'll work
* Let's try a prerelease version, shall we?
* Add notes about TAEF package, comment out tests
* Format the code
* Hopefully fix the arm64 and x86 builds
also a typo
* Fix PR nits
* Fix some bad merge conflicts
* Some cleanup from the merge
* Well I was close to getting the merge right
* I believe this will fix CI
* Apply suggestions from code review
Co-Authored-By: Carlos Zamora <carlos.zamora@microsoft.com>
* These definitely need to be fixed
* Try version detecting in the test
IDK if this will build, I'm letting the CI try while I clean rebuild locally
* Try blindly updating to the newest nuget version
* Revert "Try blindly updating to the newest nuget version"
This reverts commit b72bd9eb73cca9c3a9887e4d7a67ec2696dc6dba.
* We're just going to see if these work in CI with this change
* Comment the tests back out. Windows Server 2019 is 10.0.17763.557
* Remove the nuget package
We don't need this package anymore now that we're hosting it
* Okay this _was_ important
2019-07-15 21:27:56 +02:00
|
|
|
<Private>false</Private>
|
|
|
|
|
<CopyLocalSatelliteAssemblies>false</CopyLocalSatelliteAssemblies>
|
|
|
|
|
</ProjectReference>
|
2019-12-21 02:35:31 +01:00
|
|
|
<ProjectReference Include="$(OpenConsoleDir)src\cascadia\WinRTUtils\WinRTUtils.vcxproj">
|
|
|
|
|
<Project>{CA5CAD1A-039A-4929-BA2A-8BEB2E4106FE}</Project>
|
|
|
|
|
<ReferenceOutputAssembly>false</ReferenceOutputAssembly>
|
|
|
|
|
</ProjectReference>
|
2019-05-03 00:29:04 +02:00
|
|
|
</ItemGroup>
|
Search - add search box control and implement search experience (#3590)
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
This is the PR for feature Search: #605
This PR includes the newly introduced SearchBoxControl in TermControl dir, which is the search bar for the search experience. And the codes that enable Search in Windows Terminal.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
The PR that migrates the Conhost search module: https://github.com/microsoft/terminal/pull/3279
Spec (still actively updating): https://github.com/microsoft/terminal/pull/3299
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes #605
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
These functionalities are included in the search experience.
1. Search in Terminal text buffer.
2. Automatic wrap-around.
3. Search up or down switch by clicking different buttons.
4. Search case sensitively/insensitively by clicking a button. S. Move the search box to the top/bottom by clicking a button.
6. Close by clicking 'X'.
7. Open search by ctrl + F.
When the searchbox is open, the user could still interact with the terminal by clicking the terminal input area.
While I already have the search functionalities, currently there are still some known to-do works and I will keep updating my PR:
1. Optimize the search box UI, this includes:
1) Theme adaptation. The search box background and font color
should change according to the theme,
2) Add background. Currently the elements in search box are all
transparent. However, we need a background.
3) Move button should be highlighted once clicked.
2. Accessibility: search process should be able to performed without mouse. Once the search box is focused, the user should be able to navigate between all interactive elements on the searchbox using keyboard.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
To test:
1. checkout this branch.
2. Build the project.
3. Start Windows Terminal and press Ctrl+F
4. The search box should appear on the top right corner.
2019-12-17 16:52:37 +01:00
|
|
|
<ItemGroup>
|
|
|
|
|
<Page Include="SearchBoxControl.xaml">
|
|
|
|
|
<SubType>Designer</SubType>
|
|
|
|
|
</Page>
|
2020-02-27 01:35:16 +01:00
|
|
|
<Page Include="TermControl.xaml">
|
|
|
|
|
<SubType>Designer</SubType>
|
|
|
|
|
</Page>
|
2020-02-28 20:32:19 +01:00
|
|
|
<Page Include="TSFInputControl.xaml">
|
|
|
|
|
<SubType>Designer</SubType>
|
|
|
|
|
</Page>
|
Search - add search box control and implement search experience (#3590)
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
This is the PR for feature Search: #605
This PR includes the newly introduced SearchBoxControl in TermControl dir, which is the search bar for the search experience. And the codes that enable Search in Windows Terminal.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
The PR that migrates the Conhost search module: https://github.com/microsoft/terminal/pull/3279
Spec (still actively updating): https://github.com/microsoft/terminal/pull/3299
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes #605
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
These functionalities are included in the search experience.
1. Search in Terminal text buffer.
2. Automatic wrap-around.
3. Search up or down switch by clicking different buttons.
4. Search case sensitively/insensitively by clicking a button. S. Move the search box to the top/bottom by clicking a button.
6. Close by clicking 'X'.
7. Open search by ctrl + F.
When the searchbox is open, the user could still interact with the terminal by clicking the terminal input area.
While I already have the search functionalities, currently there are still some known to-do works and I will keep updating my PR:
1. Optimize the search box UI, this includes:
1) Theme adaptation. The search box background and font color
should change according to the theme,
2) Add background. Currently the elements in search box are all
transparent. However, we need a background.
3) Move button should be highlighted once clicked.
2. Accessibility: search process should be able to performed without mouse. Once the search box is focused, the user should be able to navigate between all interactive elements on the searchbox using keyboard.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
To test:
1. checkout this branch.
2. Build the project.
3. Start Windows Terminal and press Ctrl+F
4. The search box should appear on the top right corner.
2019-12-17 16:52:37 +01:00
|
|
|
</ItemGroup>
|
2019-11-05 23:29:11 +01:00
|
|
|
|
|
|
|
|
<Import Project="$(OpenConsoleDir)src\cppwinrt.build.post.props" />
|
|
|
|
|
|
2019-05-03 00:29:04 +02:00
|
|
|
<ItemDefinitionGroup>
|
|
|
|
|
<Link>
|
|
|
|
|
<AdditionalDependencies>dwrite.lib;dxgi.lib;d2d1.lib;d3d11.lib;shcore.lib;winmm.lib;pathcch.lib;propsys.lib;uiautomationcore.lib;Shlwapi.lib;ntdll.lib;user32.lib;kernel32.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
|
|
|
|
</Link>
|
|
|
|
|
<ClCompile>
|
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919)
TerminalControl doesn't use any of the built in text input and edit
controls provided by XAML for text input, which means TermianlControl
needs to communicate with the Text Services Framework (TSF) in order to
provide Input Method Editor (IME) support. Just like the rest of
Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core)
namespace to provide support vs. the old TSF 1.0.
Windows.UI.Text.Core handles communication between a text edit control
and the text services primarily through a CoreTextEditContext object.
This change introduces a new UserControl TSFInputControl which is a
custom EditControl similar to the CustomEditControl sample[1].
TSFInputControl is similar (overlay with IME text) to how old console
(conimeinfo) handled IME.
# Details
TSFInputControl is a Windows.UI.Xaml.Controls.UserControl
TSFInputControl contains a Canvas control for absolution positioning a
TextBlock control within its containing control (TerminalControl).
The TextBlock control is used for displaying candidate text from the
IME. When the user makes a choice in the IME the TextBlock is cleared
and the text is written to the Terminal buffer like normal text.
TSFInputControl creates an instance of the CoreTextEditContext and
attaches appropriate event handlers to CoreTextEditContext in order to
interact with the IME.
A good write-up on how to interact with CoreTextEditContext can be found
here[2].
## Text Updates
Text updates from the IME come in on the TextUpdating event handler,
text updates are stored in an internal buffer (_inputBuffer).
## Completed Text
Once a user selects a text in the IME, the CompositionCompleted handler
is invoked. The input buffer (_inputBuffer) is written to the Terminal
buffer, _inputBuffer is cleared and Canvas and TextBlock controls are
hidden until the user starts a composition session again.
## Positioning
Telling the IME where to properly position itself was the hardest part
of this change. The IME expects to know it's location in screen
coordinates as supposed to client coordinates. This is pretty easy if
you are a pure UWP, but since we are hosted inside a XAMLIsland the
client to screen coordinate translation is a little harder.
### Calculating Screen Coordinates
1. Obtaining the Window position in Screen coordinates.
2. Determining the Client coordinate of the cursor.
3. Converting the Client coordinate of the cursor to Screen coordinates.
4. Offsetting the X and Y coordinate of the cursor by the position of
the TerminalControl within the window (tabs if present, margins, etc..).
5. Applying any scale factor of the display.
Once we have the right position in screen coordinates, this is supplied
in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets
the IME know where to position itself on the Screen.
## Font Information/Cursor/Writing to Terminal
3 events were added to the TSFInputControl to create a loosely-coupled
implementation between the TerminalControl and the TSFInputControl.
These events are used for obtaining Font information from the
TerminalControl, getting the Cursor position and writing to the terminal
buffer.
## Known Issues
- Width of TextBlock is hardcoded to 200 pixels and most likely should
adjust to the available width of the current input line on the console
(#3640)
- Entering text in the middle of an existing set of text has TextBlock
render under existing text. Current Console behavior here isn't good
experience either (writes over text)
- Text input at edges of window is clipped versus wrapping around to
next line. This isn't any worse than the original command line, but
Terminal should be better (#3657)
## Future Considerations
Ideally, we'd be able to interact with the console buffer directly and
replace characters as the user types.
## Validation
General steps to try functionality
- Open Console
- Switch to Simplified Chinese (Shortcut: Windows+Spacebar)
- Switch to Chinese mode on language bar
Scenarios validated:
- As user types unformatted candidates appear on command line and IME
renders in correct position under unformatted characters.
- User can dismiss IME and text doesn't appear on command line
- Switch back to English mode, functions like normal
- New tab has proper behavior
- Switching between tabs has proper behavior
- Switching away from Terminal Window with IME present causes IME to
disappear
[1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl
[2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input
Closes #459
Closes #2213
Closes #3641
2019-11-22 01:25:50 +01:00
|
|
|
<AdditionalIncludeDirectories>$(OpenConsoleDir)src\cascadia\inc;$(OpenConsoleDir)src\types\inc;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
|
2019-05-03 00:29:04 +02:00
|
|
|
</ClCompile>
|
|
|
|
|
</ItemDefinitionGroup>
|
2020-06-08 23:01:47 +02:00
|
|
|
|
|
|
|
|
<Import Project="$(SolutionDir)build\rules\CollectWildcardResources.targets" />
|
2019-09-30 19:39:55 +02:00
|
|
|
</Project>
|
