Adds a `Microsoft.Terminal.Remoting.dll` to our solution. This DLL will
be responsible for all the Monarch/Peasant work that's been described in
#7240 & #8135.
This PR does _not_ implement the Monarch/Peasant architecture in any
significant way. The goal of this PR is to just to establish the project
layout, and the most basic connections. This should make reviewing the
actual meat of the implementation (in a later PR) easier. It will also
give us the opportunity to include some of the basic weird things we're
doing (with `CoRegisterClass`) in the Terminal _now_, and get them
selfhosted, before building on them too much.
This PR does have windows registering the `Monarch` class with COM. When
windows are created, they'll as the Monarch if they should create a new
window or not. In this PR, the Monarch will always reply "yes, please
make a new window".
Similar to other projects in our solution, we're adding 3 projects here:
* `Microsoft.Terminal.Remoting.lib`: the actual implementation, as a
static lib.
* `Microsoft.Terminal.Remoting.dll`: The implementation linked as a DLL,
for use in `WindowsTerminal.exe`.
* `Remoting.UnitTests.dll`: A unit test dll that links with the static
lib.
There are plenty of TODOs scattered about the code. Clearly, most of
this isn't implemented yet, but I do have more WIP branches. I'm using
[`projects/5`](https://github.com/microsoft/terminal/projects/5) as my
notation for TODOs that are too small for an issue, but are part of the
whole Process Model 2.0 work.
## References
* #5000 - this is the process model megathread
* #7240 - The process model 2.0 spec.
* #8135 - the window management spec. (please review me, I have 0/3
signoffs even after the discussion we had 😢)
* #8171 - the Monarch/peasant sample. (please review me, I have 1/2)
## PR Checklist
* [x] Closes nothing, this is just infrastructure
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
This is the same root cause as #8485. We moved the output of a bunch
of projects to be unified. We forgot to update all the test scripts.
In this case, the LocalTests were still using the old path that's
_not_ under `bin/`
This corrects the path to `Terminal.Core.Unit.Tests.dll` in the `runut`
unit test script and makes the `runft` feature test script capable of
working with _Release_ builds as well as _Debug_ builds.
The path to `Terminal.Core.Unit.Tests.dll` changed when the project was
restructured, and the `runut` script was never updated to reflect that
change. That has now been corrected.
And the `runft` script used to be hard coded to look for tests in the
_Debug_ directory, but it has now been updated to use the
`%_LAST_BUILD_CONF%` environment variable, so it should work for both
_Debug_ and _Release_ builds.
## Validation Steps Performed
I've manually confirmed that the `runut` and `runft` scripts now work as
expected.
Closes#8485
## Summary of the Pull Request
Does what the title says. Now while you're building terminal projects,
the taskbar will show indeterminate progress. If the build fails, it'll
blink the error state for 500ms before returning to normal.
## References
* Made possible by #8055 _and viewers like you_
## PR Checklist
* [x] scratches an itch I've had since at least 2018
* [x] I work here
* [x] this is a build script
## Validation Steps Performed
tested manually
This commit also adds an override UCD and migrates all of the overrides
from GetQuickCharWidth into it.
GetQuickCharWidth
-----------------
The removal of overrides from GQCW reduces the number of comparisons
required for looking up a single character's width from 41 (32
individual ranged comparisons from GQCW + 8+1 from the binary search in
CPWD) to 11 (2 from GQCW, 8+1 from CPWD).
GQCW also incorrectly marked 67 reserved codepoints as `Wide` when they
should have been `Narrow`.
The codepoints whose definitions have changed from `Wide` to `Narrow` are:
```
2E9A 2EF4 2EF5 2EF6 2EF7 2EF8 2EF9 2EFA 2EFB 2EFC 2EFD 2EFE 2EFF 2FD6
2FD7 2FD8 2FD9 2FDA 2FDB 2FDC 2FDD 2FDE 2FDF 2FE0 2FE1 2FE2 2FE3 2FE4
2FE5 2FE6 2FE7 2FE8 2FE9 2FEA 2FEB 2FEC 2FED 2FEE 2FEF 2FFC 2FFD 2FFE
2FFF 31E4 31E5 31E6 31E7 31E8 31E9 31EA 31EB 31EC 31ED 31EE 31EF 321F
A48D A48E A48F FE1A FE1B FE1C FE1D FE1E FE1F FE53 FE67
```
All of them are reserved, but those reserved regions are marked as narrow
in the UCD.
This change also offers us the chance to document exactly why we're
overriding a specific character range. Comments from the override
document will be copied to the generated CPWD table.
New in Unicode 13.0
------------------
Some widths have changed due to previously-reserved characters becoming
_used_ such as U+32FF SQUARE ERA NAME REIWA, the Tangut components
756-768, the entire Khitan Small Script character set, and the Tangut
Ideographs.
A number of the changes in this diff are due to better/worse comment
tracking and the removal of the Emoji/EPres comments. The script once
mistakenly applied comments to packed regions (and it has been updated
to not do so.)
Validation
----------
I build a test application that compared codepoints 0-FFFF for GQCW
against their new registered widths.
This commit removes our local copy of clang-format 8 and replaces it
with a newly-built nuget package containing clang-format 10.
This resolves the inconsistency between our version of clang-format and
the one shipped in Visual Studio.
A couple minor format changes were either required or erroneously forced
upon us--chief among them is a redistribution of `*`s around SAL
annotations in inline class members of COM classes. Don't ask why; I
couldn't figure it out.
We had some aspirational goals for our formatting, which were left in
but commented out. Enabling them changes our format a little more than
I'm comfortable with, so I uncommented them and locked them to the
format style we've been using for the past year. We may not love it, but
our aspirations may not matter here any longer. Consistent formatting is
better than perfect formatting.
This script takes a range of commits and generates a commit log with the
git2git-excluded file changes filtered out.
It also replaces GitHub issue numbers with GH-XXX so as to not confuse
Git2Git or Azure DevOps. Community contributions are tagged with CC- so
they can be detected later.
The output looks like this:
```
Carlos Zamora (2)
* Pass mouse button state into HandleMouse instead of asking win32 (GH-6765)
Dustin L. Howett (6)
* Disable MinimalCoreWin when OpenConsoleUniversalApp is false (GH-7203)
James Holderness (1)
* Add support for the "doubly underlined" graphic rendition attribute (CC-7223)
```
Yes, the numbers are wrong. No, it doesn't really matter.
Updates TAEF to 10.57.200731005-develop
## PR Checklist
* [x] Helps #6992 by bringing `wttlog.dll` along with the rest of TAEF.
* [x] I work here.
* [x] Automated tests in CI
* [x] No doc/schema update necessary (checked for docs in this repo)
* [x] Am core contributor.
Running `nuget restore` on every build is pretty unnecessary - usually, you _know_ when you need to run it. For the inner dev loop, this is a few seconds on every `bx` build.
This adds a environment variable you can set to skip the `nuget restore` part of a `bcz` build.
Add the following to your `.razzlerc.cmd`:
```cmd
set _SKIP_NUGET_RESTORE=1
```
and `bcz` (and the other helpers) _won't_ perform a nuget restore on every build.
This commit introduces Generate-CodepointWidthsFromUCD, a powershell
(7+) script that will parse a UCD XML database in the UAX 42 format from
https://www.unicode.org/Public/UCD/latest/ucdxml/ and generate
CodepointWidthDetector's giant width array.
By default, it will emit one UnicodeRange for every range of non-narrow
glyphs with a different Width + Emoji + Emoji Presentation class;
however, it can be run in "packing" and "full" mode.
* Packing mode: ignore the width/emoji/pres class and combine adjacent
runs that CPWD will treat the same.
* This is for optimizing the number of individual ranges emitted
into code.
* Full mode: include narrow codepoints (helpful for visualization)
It also supports overrides, provided in an XML document of the same format
as the UCD itself. Entries in the overrides files are applied after the
entire UCD is read and will replace any impacted ranges.
The output (when packing) looks like this:
```c++
// Generated by Generate-CodepointWidthsFromUCD -Pack:True -Full:False
// on 05/17/2020 02:47:55 (UTC) from Unicode 13.0.0.
// 66182 (0x10286) codepoints covered.
static constexpr std::array<UnicodeRange, 23> s_wideAndAmbiguousTable{
UnicodeRange{ 0xa1, 0xa1, CodepointWidth::Ambiguous },
UnicodeRange{ 0xa4, 0xa4, CodepointWidth::Ambiguous },
UnicodeRange{ 0xa7, 0xa8, CodepointWidth::Ambiguous },
.
.
.
UnicodeRange{ 0x1f210, 0x1f23b, CodepointWidth::Wide },
UnicodeRange{ 0x1f37e, 0x1f393, CodepointWidth::Wide },
UnicodeRange{ 0x100000, 0x10fffd, CodepointWidth::Ambiguous },
};
```
The output (when overriding) looks like this:
```c++
// Generated by Generate-CodepointWidthsFromUCD.ps1 -Pack:True -Full:False -NoOverrides:False
// on 5/22/2020 11:17:39 PM (UTC) from Unicode 13.0.0.
// 321205 (0x4E6B5) codepoints covered.
// 240 (0xF0) codepoints overridden.
static constexpr std::array<UnicodeRange, 23> s_wideAndAmbiguousTable{
UnicodeRange{ 0xa1, 0xa1, CodepointWidth::Ambiguous },
...
UnicodeRange{ 0xfe20, 0xfe2f, CodepointWidth::Narrow }, // narrow combining ligatures (split into left/right halves, which take 2 columns together)
...
UnicodeRange{ 0x100000, 0x10fffd, CodepointWidth::Ambiguous },
};
```
This PR introduces a new `ColorType` to allow us to distinguish between
`SGR` indexed colors from the 16 color table, the lower half of which
can be brightened, and the ISO/ITU indexed colors from the 256 color
table, which have a fixed brightness. Retaining the distinction between
these two types will enable us to forward the correct `SGR` sequences to
conpty when addressing issue #2661.
The other benefit of retaining the color index (which we didn't
previously do for ISO/ITU colors) is that it ensures that the colors are
updated correctly when the color scheme is changed.
## References
* This is another step towards fixing the conpty narrowing bugs in issue
#2661.
* This is technically a fix for issue #5384, but that won't be apparent
until #2661 is complete.
## PR Checklist
* [x] Closes#1223
* [x] CLA signed.
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already.
## Detailed Description of the Pull Request / Additional comments
The first part of this PR was the introduction of a new `ColorType` in
the `TextColor` class. Instead of just the one `IsIndex` type, there is
now an `IsIndex16` and an `IsIndex256`. `IsIndex16` covers the eight
original ANSI colors set with `SGR 3x` and `SGR 4x`, as well as the
brighter aixterm variants set with `SGR 9x` and `SGR 10x`. `IsIndex256`
covers the 256 ISO/ITU indexed colors set with `SGR 38;5` and `SGR
48;5`.
There are two reasons for this distinction. The first is that the ANSI
colors have the potential to be brightened by the `SGR 1` bold
attribute, while the ISO/ITO color do not. The second reason is that
when forwarding an attributes through conpty, we want to try and
preserve the original SGR sequence that generated each color (to the
extent that that is possible). By having the two separate types, we can
map the `IsIndex16` colors back to ANSI/aixterm values, and `IsIndex256`
to the ISO/ITU sequences.
In addition to the VT colors, we also have to deal with the legacy
colors set by the Windows console APIs, but we don't really need a
separate type for those. It seemed most appropriate to me to store them
as `IsIndex256` colors, since it doesn't make sense to have them
brightened by the `SGR 1` attribute (which is what would happen if they
were stored as `IsIndex16`). If a console app wanted a bright color it
would have selected one, so we shouldn't be messing with that choice.
The second part of the PR was the unification of the two color tables.
Originally we had a 16 color table for the legacy colors, and a separate
table for the 256 ISO/ITU colors. These have now been merged into one,
so color table lookups no longer need to decide which of the two tables
they should be referencing. I've also updated all the methods that took
a color table as a parameter to use a `basic_string_view` instead of
separate pointer and length variables, which I think makes them a lot
easier and safer to work with.
With this new architecture in place, I could now update the
`AdaptDispatch` SGR implementation to store the ISO/ITU indexed colors
as `IsIndex256` values, where before they were mapped to RGB values
(which prevented them reflecting any color scheme changes). I could also
update the `TerminalDispatch` implementation to differentiate between
the two index types, so that the `SGR 1` brightening would only be
applied to the ANSI colors.
I've also done a bit of code refactoring to try and minimise any direct
access to the color tables, getting rid of a lot of places that were
copying tables with `memmove` operations. I'm hoping this will make it
easier for us to update the code in the future if we want to reorder the
table entries (which is likely a requirement for unifying the
`AdaptDispatch` and `TerminalDispatch` implementations).
## Validation Steps Performed
For testing, I've just updated the existing unit tests to account for
the API changes. The `TextColorTests` required an extra parameter
specifying the index type when setting an index. And the `AdapterTest`
and `ScreenBufferTests` required the use of the new `SetIndexedXXX`
methods in order to be explicit about the index type, instead of relying
on the `TextAttribute` constructor and the old `SetForeground` and
`SetBackground` methods which didn't have a way to differentiate index
types.
I've manually tested the various console APIs
(`SetConsoleTextAttribute`, `ReadConsoleOutputAttribute`, and
`ReadConsoleOutput`), to make sure they are still setting and reading
the attributes as well as they used to. And I've tested the
`SetConsoleScreenBufferInfoEx` and `GetConsoleScreenBufferInfoEx` APIs
to make sure they can read and write the color table correctly. I've
also tested the color table in the properties dialog, made sure it was
saved and restored from the registry correctly, and similarly saved and
restored from a shortcut link.
Note that there are still a bunch of issues with the color table APIs,
but no new problems have been introduced by the changes in this PR, as
far as I could tell.
I've also done a bunch of manual tests of `OSC 4` to make sure it's
updating all the colors correctly (at least in conhost), and confirmed
that the test case in issue #1223 now works as expected.
The `bcz.cmd` script calls a powershell helper script (bx.ps1), but
(previous to this change) did not pass `-NoProfile`, which means that
powershell.exe would load and run one's personal profile script, which
can only slow things down, and worse, can change default behaviors (such
as turning on strict mode by default, which will cause scripts that
don't run cleanly with strict mode to generate lots of errors--such as
bx.ps1).
This change amends the powershell.exe command line to pass -NoProfile,
as well as to set the execution policy and ensure that interactive
prompts can't inadvertently show up (normal best practices for use of
powershell in build scripts).
This change will speed things up (probably negligibly, but still) and
(more importantly) prevent non-determinism and errors that could result
from running people's profile scripts when running the helper bx.ps1.
This commit introduces a NOTICE.html file that will be embedded into the
package. It will be stamped down with the real notices during a branded
release build (as part of the build pipeline.)
It, in part, reverts some of the really good work in determining the
commit hash at build time. That work will be preserved in history.
This is more compliant with our duties to the OSS we consume.
## Summary of the Pull Request
This PR will add a link to the version of `NOTICE.md` in GitHub at the commit that the build was on. It uses the same approach for generating our settings files, where we'll create a header file with the commit hash assigned to a `wstring_view` during build time.
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#5139
* [x] CLA signed.
* [x] Tests added/passed
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
The link is there and goes to the expected `NOTICE.md`.
This pull request migrates `profiles.json` to `settings.json` and removes the legacy roaming AppData settings migrator.
It also:
* separates the key bindings in defaults.json into logical groups
* syncs the universal terminal defaults with the primary defaults
* removes some stray newlines that ended up at the beginning of settings.json and defaults.json
Fixes#5186.
Fixes#3291.
### categorize key bindings
### sync universal with main
### kill stray newlines in template files
### move profiles.json to settings.json
This commit also changes Get*Settings from returning a string to
returning a std::filesystem::path. We gain in expressiveness without a
loss in clarity (since path still supports .c_str()).
NOTE: I tried to do an atomic rename with the handle open, but it didn't
work for reparse points (it moves the destination of a symbolic link
out into the settings folder directly.)
(snip for atomic rename code)
```c++
auto path{ pathToSettingsFile.wstring() };
auto renameBufferSize{ sizeof(FILE_RENAME_INFO) + (path.size() * sizeof(wchar_t)) };
auto renameBuffer{ std::make_unique<std::byte[]>(renameBufferSize) };
auto renameInfo{ reinterpret_cast<FILE_RENAME_INFO*>(renameBuffer.get()) };
renameInfo->Flags = FILE_RENAME_FLAG_REPLACE_IF_EXISTS | FILE_RENAME_FLAG_POSIX_SEMANTICS;
renameInfo->RootDirectory = nullptr;
renameInfo->FileNameLength = gsl::narrow_cast<DWORD>(path.size());
std::copy(path.cbegin(), path.cend(), std::begin(renameInfo->FileName));
THROW_IF_WIN32_BOOL_FALSE(SetFileInformationByHandle(hLegacyFile.get(),
FileRenameInfo,
renameBuffer.get(),
gsl::narrow_cast<DWORD>(renameBufferSize)));
```
(end snip)
### Stop resurrecting dead roaming profiles
## Summary of the Pull Request
Introduces convenience type `til::rectangle` which automatically implements our best practices for rectangle-related types and provides automatic conversions in/out of the relevant types.
## PR Checklist
* [x] In support of Differential Rendering #778
* [X] I work here.
* [x] Tests added/passed
* [x] I'm a core contributor.
## Detailed Description of the Pull Request / Additional comments
- Automatically converts in from anything with a Left/Top/Right/Bottom or left/top/right/bottom (Win32 `RECT`)
- Automatically converts Console type `SMALL_RECT` and shifts it from **inclusive** to **exclusive** on instantiation
- Automatically converts out to `SMALL_RECT` (converting back to **inclusive**), `RECT`, or `D2D1_RECT_F`.
- Constructs from bare integers written into source file
- Constructs from a single `til::point` as a 1x1 size rectangle with top-left corner (origin) at that point
- Constructs from a single `til::size` as a WxH size rectangle with top-left corner (origin) at 0,0
- Constructs from a `til::point` and a `til::size` representing the top-left corner and the width by height.
- Constructs from a `til::point` and another `til::point` representing the top-left corner and the **exclusive** bottom-right corner.
- Default constructs to empty
- Uses Chromium numerics for all basic math operations (+, -, *, /)
- Provides equality tests
- Provides `operator bool` to know when it's valid (has an area > 0) and `empty()` to know the contrary
- Accessors for left/top/right/bottom
- Type converting accessors (that use safe conversions and throw) for left/top/right/bottom
- Convenience methods for finding width/height (with Chromium numerics operations) and type-converting templates (with Chromium numerics conversions).
- Accessors for origin (top-left point) and the size/dimensions (as a `til::size`).
- Intersect operation on `operator &` to find where two `til::rectangle`s overlap, returned as a `til::rectangle`.
- Union operation on `operator |` to find the total area covered by two `til::rectangles`, returned as a `til::rectangle`.
- Subtract operation on `operator -` to find the area remaining after one `til::rectangle` is removed from another, returned as a `til::some<til::rectangle, 4>`.
- TAEF/WEX Output and Comparators so they will print very nicely with `VERIFY` and `Log` macros in our testing suite.
- Additional comparators, TAEF/WEX output, and tests written on `til::some` to support the Subtract operation.
- A natvis
## Validation Steps Performed
- See automated tests of functionality.
## Summary of the Pull Request
Introduces convenience type `til::point` which automatically implements our best practices for point-related types and provides automatic conversions in/out of the relevant types.
## PR Checklist
* [x] In support of Differential Rendering #778
* [X] I work here.
* [x] Tests added/passed
* [x] I'm a core contributor.
## Detailed Description of the Pull Request / Additional comments
- Automatically converts in from anything with a X/Y (console `COORD`) or x/y (Win32 `POINT`)
- Automatically converts out to `COORD`, `POINT`, or `D2D1_POINT_2F`.
- Constructs from bare integers written into source file
- Default constructs to empty
- Uses Chromium Math for all basic math operations (+, -, *, /)
- Provides equality tests
- Accessors for x/y
- Type converting accessors (that use safe conversions and throw) for x/y
- TAEF/WEX Output and Comparators so they will print very nicely with `VERIFY` and `Log` macros in our testing suite.
- A natvis
## Validation Steps Performed
- See automated tests of functionality.
## Summary of the Pull Request
Introduces convenience type `til::size` which automatically implements our best practices for size-related types and provides automatic conversions in/out of the relevant types.
## PR Checklist
* [x] In support of Differental Rendering #778
* [X] I work here.
* [x] Tests added/passed
* [x] I'm a core contributor.
## Detailed Description of the Pull Request / Additional comments
- Automatically converts in from anything with a X/Y (console `COORD`) or cx/cy (Win32 `SIZE`)
- Automatically converts out to `COORD`, `SIZE`, or `D2D1_SIZE_F`.
- Constructs from bare integers written into source file
- Default constructs to empty
- Uses Chromium Math for all basic math operations (+, -, *, /)
- Provides equality tests
- Adds initial proposal for division-to-ceiling (round up division) that attempts to `ceil` without any floating point math.
- Accessors for height/width
- Type converting accessors (that use safe conversions and throw) for height/width
- Convenience function for area calculation (as that's common with type) and uses safe math to do it.
- TAEF/WEX Output and Comparators so they will print very nicely with `VERIFY` and `Log` macros in our testing suite.
## Validation Steps Performed
- See automated tests of functionality.
til::color will help us move away from COLORREF internally. It supports
conversion to/from COLORREF, and from all types of structs containing
members named R, G, B and A (or r, g, b, and a).
## Validation Steps Performed
Tests; run through profile/colorScheme deserialization with `til::color`
instead of `uint32_t` or `COLORREF`.
Generated by https://github.com/jsoref/spelling `f`; to maintain your repo, please consider `fchurn`
I generally try to ignore upstream bits. I've accidentally included some items from the `deps/` directory. I expect someone will give me a list of items to drop, I'm happy to drop whole files/directories, or to split the PR into multiple items (E.g. comments/locals/public).
Closes#4294
## Summary of the Pull Request
New year, new unittests.
This PR introduces a new project, `TestHostApp`. This project is largely taken from the TAEF samples, and allows us to easily construct a helper executable and `resources.pri` for running TerminalApp unittests.
## References
## PR Checklist
* [x] Closes#3986
* [x] I work here
* [x] is Tests
* [n/a] Requires documentation to be updated
* [x] **Waiting for an updated version of TAEF to be available**
## Detailed Description of the Pull Request / Additional comments
Unittesting for the TerminalApp project has been a horrifying process to try getting everything pieced together just right. Dependencies need to get added to manifests, binplaced correctly, and XAML resources need to get compiled together as well. In addition, using a MUX `Application` (as opposed to the Windows.UI.Xaml `Application`) has led to additional problems.
This was always a horrifying house of cards for us. Turns out, the reason this was so horrible is that the test infrastructure for doing what we're doing _literally didn't exist_ when I started doing all that work last year.
So, with help from the TAEF team, I was able to get rid of our entire house of cards, and use a much simpler project to build and run the tests.
Unfortunately, the latest TAEF release has a minor bug in it's build rules, and only publishes the x86 version of a dll we need from them. But, the rest of this PR works for x86, and I'll bump this when that updated version is available. We should be able to review this even in the state it's in.
## Validation Steps Performed
ran the tests yo
## Summary of the Pull Request
Introduces a type that is basically an array (stack allocated, fixed size) that reports size based on how many elements are actually filled (from the front), iterates only the filled ones, and has some basic vector push/pop semantics.
## PR Checklist
* [x] I work here
* [x] I work here
* [x] I work here
* [ ] I'd love to roll this out to SomeViewports.... maybe in this commit or a follow on one.
* [ ] We need a TIL tests library and I should test this there.
## Detailed Description of the Pull Request / Additional comments
The original gist of this was used for `SomeViewports` which was a struct to hold between 0 and 4 viewports, based on how many were left after subtraction (since rectangle subtraction functions in Windows code simply fail for resultants that yield >=2 rectangle regions.)
I figured now that we're TIL-ifying useful common utility things that this would be best suited to a template because I'm certain there are other circumstances where we would like to iterate a partially filled array and want it to not auto-resize-up like a vector would.
## Validation Steps Performed
* [ ] TIL tests added
## Summary of the Pull Request
The InputStateMachineEngine was incorrectly not returning to the ground state after flushing the last sequence. That means that something like alt+backspace would leave us in the Escape state, not the ground state. This makes sure we return to ground.
Additionally removes the "Parser.UnitTests-common.vcxproj" file, which was originally used for a theoretical time when we only open-sourced the parser. It's unnecessary now, and we can get rid of it.
Also includes a small patch to bcz.cmd, to make sure bx works with projects with a space in their name.
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#2746
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
<hr>
* Return to ground when we flush the last char
The InputStateMachineEngine was incorrectly not returning to the ground state
after flushing the last sequence. That means that something like alt+backspace
would leave us in the Escape state, not the ground state. This makes sure we
return to ground.
Fixes#2746.
Additionally removes the "Parser.UnitTests-common.vcxproj" file, which was
originally used for a theoretical time when we only open-sourced the parser.
It's unnecessary now, and we can get rid of it.
Also includes a small patch to bcz.cmd, to make sure bx works with projects
with a space in their name.
* Update src/terminal/parser/stateMachine.cpp
Co-Authored-By: Dustin L. Howett (MSFT) <duhowett@microsoft.com>
* add the comment @miniksa wanted
* This fixes the registry path
What's happening is the console is writing the Forcev2 setting, then the v1
console is ignoring those settings, then when you check the checkbox to save
the v2 settings, we'll write the zeros out. That's obviously bad. So we'll
only write the v2 settings back to the registry if the propsheet was launched
from a v2 console.
This does not fix the shortcut path. That'll be the next commit.
* Fix the shortcut loading too
fixes#2319
* remove the redundant property I added
* add some notes to the bx.ps1 change
This PR represents the start of the work on Cascading User + default settings, #754.
Cascading settings will be done in two parts:
* [ ] Layered Default+User settings (this PR)
* [ ] Dynamic Profile Generation (#2603).
Until _both_ are done, _neither are going in. The dynamic profiles PR will target this PR when it's ready, but will go in as a separate commit into master.
This PR covers adding one primary feature: the settings are now in two separate files:
* a static `defaults.json` that ships with the package (the "default settings")
* a `profiles.json` with the user's customizations (the "user settings)
User settings are _layered_ upon the settings in the defaults settings.
## References
Other things that might be related here:
* #1378 - This seems like it's definitely fixed. The default keybindings are _much_ cleaner, and without the save-on-load behavior, the user's keybindings will be left in a good state
* #1398 - This might have honestly been solved by #2475
## PR Checklist
* [x] Closes#754
* [x] Closes#1378
* [x] Closes#2566
* [x] I work here
* [x] Tests added/passed
* [x] Requires documentation to be updated - it **ABSOLUTELY DOES**
## Detailed Description of the Pull Request / Additional comments
1. We start by taking all of the `FromJson` functions in Profile, ColorScheme, Globals, etc, and converting them to `LayerJson` methods. These are effectively the same, with the change that instead of building a new object, they are simply layering the values on top of `this` object.
2. Next, we add tests for layering properties like that.
3. Now, we add a `defaults.json` to the package. This is the file the users can refer to as our default settings.
4. We then take that `defaults.json` and stamp it into an auto generated `.h` file, so we can use it's data without having to worry about reading it from disk.
5. We then change the `LoadAll` function in `CascadiaSettings`. Now, the function does two loads - one from the defaults, and then a second load from the `profiles.json` file, layering the settings from each source upon the previous values.
6. If the `profiles.json` file doesn't exist, we'll create it from a hardcoded `userDefaults.json`, which is stamped in similar to how `defaults.json` is.
7. We also add support for _unbinding_ keybindings that might exist in the `defaults.json`, but the user doesn't want to be bound to anything.
8. We add support for _hiding_ a profile, which is useful if a user doesn't want one of the default profiles to appear in the list of profiles.
## TODO:
* [x] Still need to make Alt+Click work on the settings button
* [x] Need to write some user documentation on how the new settings model works
* [x] Fix the pair of tests I broke (re: Duplicate profiles)
<hr>
* Create profiles by layering them
* Update test to layer multiple times on the same profile
* Add support for layering an array of profiles, but break a couple tests
* Add a defaults.json to the package
* Layer colorschemes
* Moves tests into individual classes
* adds support for layering a colorscheme on top of another
* Layer an array of color schemes
* oh no, this was missed with #2481
must have committed without staging this change, uh oh. Not like those tests actually work so nbd
* Layer keybindings
* Read settings from defaults.json + profiles.json, layer appropriately
This is like 80% of #754. Needs tests.
* Add tests for keybindings
* add support to unbind a key with `null` or `"unbound"` or `"garbage"`
* Layer or clear optional properties
* Add a helper to get an optional variable for a bunch of different types
In the end, I think we need to ask _was this worth it_
* Do this with the stretch mode too
* Add back in the GUID check for profiles
* Add some tests for global settings layering
* M A D W I T H P O W E R
Add a MsBuild target to auto-generate a header with the defaults.json as a
string in the file. That way, we can _always_ load the defaults. Literally impossible to not.
* When the user's profile.json doesn't exist, create it from a template
* Re-order profiles to match the order set in the user's profiles.json
* Add tests for re-ordering profiles to match user ordering
* Add support for hiding profiles using `"hidden": true`
* Use the hardcoded defaults.json for the exception->"use defaults" case
* Somehow I messed up the git submodules?
* woo documentation
* Fix a Terminal.App.Unit.Tests failure
* signed/unsigned is hard
* Use Alt+Settings button to open the default settings
* Missed a signed/unsigned
* Some very preliminary PR feedback
* More PR feedback
Use the wil helper for the exe path
Move jsonutils into their own file
kill some dead code
* Add templates to these bois
* remove some code for generating defaults, reorder defaults.json a tad
* Make guid a std::optional
* Large block of PR feedback
* Remove some dead code
* add some comments
* tag some todos
* stl is love, stl is life
* add `-noprofile`
* Fix the crash that dustin found
* -Encoding ASCII
* Set a profile's default scheme to Campbell
* Fix the tests I regressed
* Update UsingJsonSetting.md to reflect that changes from these PRs
* Change how GenerateGuidForProfile works
* Make AppKeyBindings do its own serialization
* Remove leftover dead code from the previous commit
* Fix up an enormous number of PR nits
* Fix a typo; Update the defaults to match #2378
* Tiny nits
* Some typos, PR nits
* Fix this broken defaults case
In #1164 we learned that our CI doesn't support WinRT testing. This made us all sad. Since that merged, we haven't really added any TerminalApp tests, because it's a little too hard. You'd have to uncomment the entire file, and if the list of types changed you'd have to manually update the sxs manifest and appxmanifest.
Since that was all insane, I created a new Terminal App unittesting project without those problems.
1. The project is not named *Unit*Test*, so the CI won't run it, but it will run locally.
2. The project will auto-generate its SxS manifest, using the work from #1987.
3. We'll use the SxS manifest from step 2 to generate an AppxManifest for running packaged tests.
* This is the start of me trying to enable local unittesting again
* We've got a new unittests project that isn't named *unit*test*
* We're manually generating the SxS manifest for it. B/C we need to use it at runtime, we need to manually combine it into one manifest file
* the runas:UAP thing still doesn't work. We'll investigate.
* This shockingly works
but I'm still stuck with:
```
Summary of Errors Outside of Tests:
Error: TAEF: [HRESULT: 0x80270254] Failed to create the test host process for
out of process test execution. (The
IApplicationActivationManager::ActivateApplication call failed while using a
default host. TAEF's ETW logs which are gathered with the /enableEtwLogging
switch should contain events from relevant providers that may help to diagnose
the failure.)
```
* Cleaning this all up for review.
Frankly just pushing to see if it'll work in CI
* Couple things I noticed in the diff from master
* Apply @dhowett-msft's suggestions from code review
* Try createing a script to only build the current working directory
Inspired by #2078.
I wanted to use this for WindowsTerminal, but I can't generate the
resources.pri from just building WindowsTerminal. Maybe @dhowett-msft has
some ideas.
* Cleanup for PR
* fix some bugs with building outside a project directory.
* PR nits
In commit 0905140955 (PR #1164),
we updated the version of the Taef.Redist.Wlk NuGet package
for the TAEF test harness and framework. However, the helper commands
to run the various test cases hard-code the path to the TAEF executable,
which because of NuGet's design includes the TAEF NuGet package version.
These commands weren't updated to reflect the new TAEF version
and so have been broken since then.
This commit fixes the issue and makes running tests possible again.
* Refactors TerminalApp into two projects:
- TerminalAppLib, which builds a .lib, and includes all the code
- TerminalApp, which builds a dll by linking the lib
* Adds a TerminalApp.Unit.Tests project
- Includes the ability to test cppwinrt types we've authored using a SxS manifest for unpackaged winrt activation
- includes the ability to test types with XAML content using an appxmanifest
* Adds a giant doc explaining how this was all done. Really, just go read that doc, it'll really help you understand what's going on in this PR.
-------------------------
These are some previous commit messages. They may be helpful to future readers.
* Start adding unittests for json parsing, end up creating a TerminalAppLib project to make a lib. See #1042
* VS automatically did this for me
* This is a dead end
I tried including the idl-y things into the lib, but that way leads insanity
If you want to make a StaticLibrary, then suddenly the winrt toolchain forgets
that ProjectReferences can have winmd's in them, so it won't be able to
compile any types from the referenced projects. If you instead try to manually
reference the types, you'll get duplicate types up the wazoo, which of course
is insane, since we're referencing them the _one_ time
* Yea just follow #1042 on github for status
So current state:
1. If you try to add a `Reference` to all of MUX.Markup, TerminalControl and
TerminalSettings, then mdmerge will complain about all the types from
TerminalSettings being defined twice. In this magic scenario, the
dependencies of TerminalControl are used directly for some reason:
```
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalSettings\Microsoft.Terminal.Settings.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.Settings.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.TerminalConnection.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\TerminalControl\Microsoft.Terminal.TerminalControl.winmd.
12> Load input metadata file ...OpenConsole\x64\Debug\Microsoft.UI.Xaml.Markup\Microsoft.UI.Xaml.Markup.winmd.
```
2. If you don't add a `Reference` TerminalControl, then it'll complain about
being unable to find the type TitleChangedEventArgs, which is defined in
TerminalControl.
3. If you don't add a `Reference` TerminalSettings, then it'll complain about
being unable to find the type KeyChord and other types from
TerminalSettings. In this scenario, it doesn't recurse on the other
dependencies from TerminalControl for whatever reason.
4. If you instead try to add all 3 as a `ProjectReference`, then it'll
complain about being unable to find TitleChangedEventArgs, as in 2.
Presumably, it;ll have troubles with the other types too, as none of the 3
are actually included in the midlrt.rsp file.
5. If you add all 3 as a `ProjectReference`, then also add TerminalControl as
a `Reference`, you'll get a `MIDL2011: [msg] unresolved type declaration
Microsoft.UI.Xaml.Markup.XamlApplication`
6. If you add all 3 as a `ProjectReference`, then also add TerminalControl AND
MUX.Markup as a `Reference`, you'll get the same result as 3.
* what if we just don't idl
This seems to compile
* This compiles but I broke the MUX resources
look at the App.xaml change. in this changelist. That's what's broken right now. Lets fix that!
* lets do this
If I leave the MUX nuget out of the project, I'll get a compile error in
App.xaml:
```
...OpenConsole\src\cascadia\TerminalApp\App.xaml(21,40): XamlCompiler error WMC0001: Unknown type 'XamlControlsResources' in XML namespace 'using:Microsoft.UI.Xaml.Controls'
```
If I add it back to the project, it works
* Some cleanup from the previous commit
* This is busted again.
Doing a clean build didn't work.
A clean rebuild of the project, paired with some removal of dead code
revealed a problem with what I have so far.
TerminalAppLib depends on the generation of two headers,
`AppKeyBindings.g.h` and `App.g.h`, as those define some of bits of the
winrt types. They're needed to be able to compile the implementations.
Presumably that's not getting generated by the lib project, because the dll
project is the one to generate that file.
So we need to move the idl's to the lib project. This created maddness,
because of course the Duplicate Type thing. The solution to that is to
actually mark the winrt DLLs that we're chaining up through us as
```
<Private>false</Private>
<CopyLocalSatelliteAssemblies>false</CopyLocalSatelliteAssemblies>
```
This will prevent them from getting double-included.
This still doesn't work however, since
```
app.cpp(40): error C2039: 'XamlMetaDataProvider': is not a member of 'winrt::TerminalApp'
error C3861: 'XamlMetaDataProvider': identifier not found
```
So we need to figure that out. The dll project is still generating the right
header, so lets look there.
* Move the xaml stuff to the lib
This compiles, but when we launch, we fail to load the tabviewcontrol
resources again. So that's not what you want. Why is it not included?
* It works again!
* Use the pri, xbf files from TerminalAppLib, not TerminalApp
* Manually make TerminalApp include a reference to TerminalAppLib's
TerminalApp.winmd. This will force the build to copy TerminalApp.winmd to
TerminalApp/, which WindowsTerminal needs to be able to ProjectReference the
TerminalApp project (it's expecting it to have a winmd)
* Remove the module.g.cpp from TerminalApp, and move to TerminalAppLib. The
dll doesn't do any codegen anymore.
* Agressively clean up these files
* Clean up unnecessary includes in the dll pch.h
* This does NOT work.
The WindowsxamlManager call crashes. I'm thinking it has to do with activation
of winrt types from a dll.
Email out to @Austin-Lamb to see if he can assist
* This gets our cppwinrt types working, but xaml islands is still broken
* Split the tests apart, so they aren't insane
* These are the magic words to make xaml islands work
* All this witchcraft is necessary to make XAML+MUX work right
* Clean this up a bit and add comments
* Create an enormous doc explaining this madness
* Unsure how this got changed.
* Trying to get the CI build to work again.
This resolves the MUX issue. We need to manually include it, because their package's target doesn't mark it as CopyLocalSatelliteAssemblies=false, Private=false.
However, the TerminalApp project is still able to magically reason that the TerminalAppLib project should be included in the MdMerge step, because it think's it's a `GetCppWinRTStaticProjectReferences` reference.
* Update cppwinrt to the latest version - this fixes the MSBuild
* I still need to re-add the KeyModifiers checks from TermControl. I think
this update broke `operator&` for that enum.
* There needs to be some cleanup obviously
* The doc should be updated as well
* Clean up changes from cppwinrt update
* Try doing this, even though it seems wrong
* Lets try this (press x to doubt)
* Clean up vcxproj file, and remove appxmanifest change from previous commit
* Update to the latest TAEF release, maybe that'll work
* Let's try a prerelease version, shall we?
* Add notes about TAEF package, comment out tests
* Format the code
* Hopefully fix the arm64 and x86 builds
also a typo
* Fix PR nits
* Fix some bad merge conflicts
* Some cleanup from the merge
* Well I was close to getting the merge right
* I believe this will fix CI
* Apply suggestions from code review
Co-Authored-By: Carlos Zamora <carlos.zamora@microsoft.com>
* These definitely need to be fixed
* Try version detecting in the test
IDK if this will build, I'm letting the CI try while I clean rebuild locally
* Try blindly updating to the newest nuget version
* Revert "Try blindly updating to the newest nuget version"
This reverts commit b72bd9eb73.
* We're just going to see if these work in CI with this change
* Comment the tests back out. Windows Server 2019 is 10.0.17763.557
* Remove the nuget package
We don't need this package anymore now that we're hosting it
* Okay this _was_ important
* Fix OpenConsole.psm1 to use vswhere
I'm not sure this is good, since I don't write a lot of powershell, and
I don't know the project very well, but hopefully it's good!
* Do as @DHowett-MSFT says and use VSSetup
whee!
* try to do what @heaths is recommending
* fix `Import-LocalModule`
* fix openconsole.psm1 for hopefully the last time
Since version 5.0.2, NuGet has used the PATH environment variable
to find MSBuild.exe before looking in other file paths.
See NuGet change
21f2b07f2c
(https://github.com/NuGet/NuGet.Client/pull/2687 ).
Unfortunately, in PR
https://github.com/microsoft/terminal/pull/606 ,
`tools\razzle.cmd` was changed to add the MSBuild.exe folder path
in _quotes_ to the PATH environment variable.
Windows itself is fine with this (you can type `msbuild` and
MSBuild runs), but some tools are not, including NuGet itself,
so you would get errors like this:
```
D:\GitHub\metathinker\console> where nuget
C:\ProgramData\chocolatey\bin\nuget.exe
D:\GitHub\metathinker\console\dep\nuget\nuget.exe
D:\GitHub\metathinker\console> nuget restore OpenConsole.sln
Illegal characters in path.
```
`razzle.cmd` runs NuGet itself, but does so before adding
the MSBuild folder to the PATH, so it was not affected by this
problem.
This change fixes the issue by dequotifying the PATH,
so that if you already had a newer version of NuGet on your PATH
before running `tools\razzle.cmd`, that version will continue
to work should you need to run `nuget restore` again
(such as after a `git clean -dx`).
This commit switches the GUIDs for default profiles from being randomly generated to being version 5 UUIDs. More info in #870.
## PR Checklist
* [x] Closes#870
* [x] CLA signed
* [x] Tests added/passed
* [x] Requires documentation to be updated (#883)
* [x] I've discussed this with core contributors already.
## Detailed Description of the Pull Request / Additional comments
This commit has a number of changes that seem ancillary, but they're general goodness. Let me explain:
* I've added a whole new Types test library with only two tests in
* Since UUIDv5 generation requires SHA1, we needed to take a dependency on bcrypt
* I honestly don't think we should have to link bcrypt in conhost, but LTO should take care of that
* I considered adding a new Terminal-specific Utils/Types library, but that seemed like a waste
* The best way to link bcrypt turned out to be in line with a discussion @miniksa and I had, where we decided we both love APISets and think that the console should link against them exclusively... so I've added `onecore_apiset.lib` to the front of the link line, where it will deflect the linker away from most of the other libs automagically.
```
StartGroup: UuidTests::TestV5UuidU8String
Verify: AreEqual(uuidExpected, uuidActual)
EndGroup: UuidTests::TestV5UuidU8String [Passed]
StartGroup: UuidTests::TestV5UuidU16String
Verify: AreEqual(uuidExpected, uuidActual)
EndGroup: UuidTests::TestV5UuidU16String [Passed]
```
* make copying of files windows localization agnostic
On a german Windows when building I get the following error:
(D = Datei, V = Verzeichnis)? Ist das Ziel ...\Terminal\x64\Debug\TerminalSettings.pdb ein Dateiname
oder ein Verzeichnisname
(D = Datei, V = Verzeichnis)? f
The trick with piping 'f' for file into stdin does not work here, since
in german file is called 'Datei'. Due to the fact that the UI is
translated a 'd' is expected.
Lets use '*' at the end of the target filename which is a hack to trick
'xcopy' into assuming it is a filename her a target is a folder, if the
target does not exist.
* start fixing commandline tools to run new windows terminal
* opencas should do the same as openterm.
* correct the filename in openterm
openterm is able to start the terminal again, but it does not start
properly because of a missing dependency.
* remove openterm command
There is currently no plan on fixing this, because WindowsTerminal.exe
does not support unpackaged activation. Let's remove them for now.
* fix for historySize=32767 hang (except for historySize=0 case); tests still in progress
* tests run and almost pass - failure is a real bug in my change
* fixed bug that caused tests to fail, but it seems another bug causes the app to crash with a zero row count
* fix the additional bug (at a higher layer) mentioned in previous commit description
* Fix chk build assertion failures in new tests
It seems C++/WinRT doesn't like it when you implement a Windows Runtime
interface but then create instances of the implementing class
with function-call lifetime (aka stack allocation). That makes sense
given that WinRT objects are COM objects, but in my defense I was following
this example where they are just fine instantiating the `App` object
on the stack:
https://docs.microsoft.com/en-us/windows/uwp/cpp-and-winrt-apis/author-apis#if-youre-not-authoring-a-runtime-class
* tabs to spaces
* CR feedback
* fix minor CR feedback (incorrect test log message)
* Update razzle to use vswhere
* Make vswhere pickup build tools
* Make razzle handle errors better
* Make bcz handle MSBUILD with spaces
* Update readmes to use bcz and fix typo
