This commit adds a missing conversion utf8 to utf16 in decoding base64
for handling multibyte text in copying via OSC 52.
## Validation Steps Performed
* automatically
* Tests w/ multibyte characters
* manually
* case1
* Executed `printf "\x1b]52;;%s\x1b\\" "$(printf '👍👍🏻👍🏼👍🏽👍🏾👍🏿' | base64)"`
* Verified `👍👍🏻👍🏼👍🏽👍🏾👍🏿` in my clipboard
* case2
* Copied `👍👍🏻👍🏼👍🏽👍🏾👍🏿` by tmux 2.6 default copy function (OSC 52)
* Verified `👍👍🏻👍🏼👍🏽👍🏾👍🏿` in my clipboard
Closes#7819
* Correct the behaviour of parsing `rgb:R/G/B`. It should be interpreted
as `RR/GG/BB` instead of `0R/0G/0B`
* Add support for `rgb:RRR/GGG/BBB` and `rgb:RRRR/GGGG/BBBB`. The
behaviour of 12 bit variants is to repeat the first digit at the end,
e.g. `rgb:123/456/789` becomes `rgb:1231/4564/7897`.
* Add support for `#` formats. We are following the rules of
[XParseColor] by interpreting `#RGB` as `R000G000B000`.
* Add support for XOrg app color names, which are supported by xterm, VTE
and many other terminal emulators.
* Multi-parameter OSC 4 is now supported.
* The chaining of OSC 10-12 is not yet supported. But the parameter
validation is relaxed by parsing the parameters as multi-params but
only use the first one, which means `\e]10;rgb:R/G/B;` and
`\e]10:rgb:R/G/B;invalid` will execute `OSC 10` with the first color
correctly. This fixes some of the issues mentioned in #942 but not
all of them.
[XParseColor]: https://linux.die.net/man/3/xparsecolorCloses#3715
By default, the memory order on atomics is `seq_cst`. This is a relatively expensive ordering and it shows in situations where we're rapidly signaling a consumer to pick up something from a producer. I've instead attempted to switch these to `release` (producer) and `acquire` (consumer) to improve the performance of these signals.
## Validation Steps Performed
- Run `time cat big.txt` and `time cat ls.txt` under VS Performance Profiler.
## PR Checklist
* [x] Closes perf itch
* [x] I work here
* [x] Manual test
* [x] Documentation irrelevant.
* [x] Schema irrelevant.
* [x] Am core contributor.
With this commit, terminal will be able to copy text to the system
clipboard by using OSC 52 MANIPULATE SELECTION DAATA.
We chose not to implement the clipboard querying functionality offered
by OSC 52, as sending the clipboard text to an application without the
user's knowledge or consent is an immense security hole.
We do not currently support the clipboard specifier Pc to specify which
clipboard buffer should be filled
# Base64 encoded `foo`
$ echo -en "\e]52;;Zm9v\a"
# Multiple lines
# Base64 encoded `foo\r\nbar`
$ echo -en "\e]52;;Zm9vDQpiYXI=\a"
Closes#2946.
This PR improves our VT character set support, enabling the [`SCS`]
escape sequences to designate into all four G-sets with both 94- and
96-character sets, and supports invoking those G-sets into both the GL
and GR areas of the code table, with [locking shifts] and [single
shifts]. It also adds [`DOCS`] sequences to switch between UTF-8 and the
ISO-2022 coding system (which is what the VT character sets require),
and adds support for a lot more characters sets, up to around the level
of a VT510.
[`SCS`]: https://vt100.net/docs/vt510-rm/SCS.html
[locking shifts]: https://vt100.net/docs/vt510-rm/LS.html
[single shifts]: https://vt100.net/docs/vt510-rm/SS.html
[`DOCS`]: https://en.wikipedia.org/wiki/ISO/IEC_2022#Interaction_with_other_coding_systems
## Detailed Description of the Pull Request / Additional comments
To make it easier for us to declare a bunch of character sets, I've made
a little `constexpr` class that can build up a mapping table from a base
character set (ASCII or Latin1), along with a collection of mappings for
the characters the deviate from the base set. Many of the character sets
are simple variations of ASCII, so they're easy to define this way.
This class then casts directly to a `wstring_view` which is how the
translation tables are represented in most of the code. We have an array
of four of these tables representing the four G-sets, two instances for
the active left and right tables, and one instance for the single shift
table.
Initially we had just one `DesignateCharset` method, which could select
the active character set. We now have two designate methods (for 94- and
96- character sets), and each takes a G-set number specifying the target
of the designation, and a pair of characters identifying the character
set that will be designated (at the higher VT levels, character sets are
often identified by more than one character).
There are then two new `LockingShift` methods to invoke these G-sets
into either the GL or GR area of the code table, and a `SingleShift`
method which invokes a G-set temporarily (for just the next character
that is output).
I should mention here that I had to make some changes to the state
machine to make these single shift sequences work. The problem is that
the input state machine treats `SS3` as the start of a control sequence,
while the output state machine needs it to be dispatched immediately
(it's literally the _Single Shift 3_ escape sequence). To make that
work, I've added a `ParseControlSequenceAfterSs3` callback in the
`IStateMachineEngine` interface to decide which behavior is appropriate.
When it comes to mapping a character, it's simply an array reference
into the appropriate `wstring_view` table. If the single shift table is
set, that takes preference. Otherwise the GL table is used for
characters in the range 0x20 to 0x7F, and the GR table for characters
0xA0 to 0xFF (technically some character sets will only map up to 0x7E
and 0xFE, but that's easily controlled by the length of the
`wstring_view`).
The `DEL` character is a bit of a special case. By default it's meant to
be ignored like the `NUL` character (it's essentially a time-fill
character). However, it's possible that it could be remapped to a
printable character in a 96-character set, so we need to check for that
after the translation. This is handled in the `AdaptDispatch::Print`
method, so it doesn't interfere with the primary `PrintString` code
path.
The biggest problem with this whole process, though, is that the GR
mappings only really make sense if you have access to the raw output,
but by the time the output gets to us, it would already have been
translated to Unicode by the active code page. And in the case of UTF-8,
the characters we eventually receive may originally have been composed
from two or more code points.
The way I've dealt with this was to disable the GR translations by
default, and then added support for a pair of ISO-2022 `DOCS` sequences,
which can switch the code page between UTF-8 and ISO-8859-1. When the
code page is ISO-8859-1, we're essentially receiving the raw output
bytes, so it's safe to enable the GR translations. This is not strictly
correct ISO-2022 behavior, and there are edge cases where it's not going
to work, but it's the best solution I could come up with.
## Validation Steps Performed
As a result of the `SS3` changes in the state machine engine, I've had
to move the existing `SS3` tests from the `OutputEngineTest` to the
`InputEngineTest`, otherwise they would now fail (technically they
should never have been output tests).
I've added no additional unit tests, but I have done a lot of manual
testing, and made sure we passed all the character set tests in Vttest
(at least for the character sets we currently support). Note that this
required a slightly hacked version of the app, since by default it
doesn't expose a lot of the test to low-level terminals, and we
currently identify as a VT100.
Closes#3377Closes#3487
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
Updates the check spelling action to [0.0.16-a](https://github.com/check-spelling/check-spelling/releases/tag/0.0.16-alpha)
* update advice -- [sample](57fc13f6c6 (commitcomment-39489723)) -- I really do encourage others to adjust it as desired
* rename `expect` (there are consumers who were not a fan of the `whitelist` nomenclature)
* prune stale items
* some `patterns` improvements to reduce the number of items in `expect`
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
⚠️ Anyone with an inflight addition of a new file to the `whitelist` directory will be moderately unhappy as the action would only use items from there if it didn't find `expect` (and this PR includes the rename).
## References
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [ ] Closes #xxx
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
Runs should be ~30s faster.
I was hoping to be able to offer the ability to talk to the bot, but sadly that feature is still not quite ready -- and I suspect that I may want to let projects opt in/out of that feature.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
* I added a commit with misspellings: 57fc13f6c6❌ and ran the command it suggested (in bash).
* The commit [itself passes its own testing](78df00dcf6) ✔️
The commands were never `cmd`/`psh` friendly. This iteration is designed to make it easier for a bot to parse and eventually do the work in response to a GitHub request, sadly that feature is behind schedule.
I never got to fixing these in the original #3789 PR, but I messed up that branch way too many times already that I figured I'd just do it in post.
* [x] Fixes the typo bot in `master`
* [x] I work here
This commit introduces a context menu for Tab and a new item,
"Color...", which will display a color picker.
A flyout menu, containing a custom flyout, is attached to each tab. The
flyout displays a palette of 16 preset colors and includes a color
picker. When the user selects or clears color, an event is fired, which
is intercepted by the tab to which the flyout belongs.
The changing of the color is achieved by putting the selected color in
the resource dictionary of the tab, using well-defined dictionary keys
(e.g. TabViewItemHeaderBackground). Afterwards the visual state of the
tab is toggled, so that the color change is visible immediately.
Custom-colored tabs will be desaturated (somewhat) by alpha blending
them with the tab bar background.
The flyout menu also contains a 'Close' flyout item.
## Validation Steps Performed
I've validated the behavior manually: start the program via the start
menu. Right click on the tab -> Choose a tab color.
The color flyout is going to be shown. Click a color swatch or click
'Select a custom color' to use the color picker. Use the 'Clear the
current color' to remove the custom color.
Closes#2994. References #3327.
_This is literally just #1357, but moved to the `drafts/` folder_. Since
@dsafa doesn't have the time to finish this on their own, we'll take it
from here for 2.0 ☺️
## Summary of the Pull Request
Adds a spec describing jumplist integration and adding profiles to the jumplist. Includes details about previous investigations into adding the jumplist.
## PR Checklist
* [x] Specs #576
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [n/a] Tests added/passed
* [x] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #576
## Detailed Description of the Pull Request / Additional comments
Details in the spec.
## Validation Steps Performed
N/A
Co-authored-by: Brandon Chong <brndnchong@gmail.com>
Now that the Terminal is doing a better job of actually marking which
lines were and were not wrapped, we're not always copying lines as
"wrapped" when they should be. We're more correctly marking lines as not
wrapped, when previously we'd leave them marked wrapped.
The real problem is here in the `ScrollFrame` method - we'd manually
newline the cursor to make the terminal's viewport shift down to a new
line. If we had to scroll the viewport for a _wrapped_ line, this would
cause the Terminal to mark that line as broken, because conpty would
emit an extra `\n` that didn't actually exist.
This more correctly implements `ScrollFrame`. Now, well move where we
"thought" the cursor was, so when we get to the next `PaintBufferLine`,
if the cursor needs to newline for the next line, it'll newline, but if
we're in the middle of a wrapped line, we'll just keep printing the
wrapped line.
A couple follow up bugs were found to be caused by the same bad logic.
See #5039 and #5161 for more details on the investigations there.
## References
* #4741 RwR, which probably made this worse
* #5122, which I branched off of
* #1245, #357 - a pair of other conpty wrapped lines bugs
* #5228 - A followup issue for this PR
## PR Checklist
* [x] Closes#5113
* [x] Closes#5180 (by fixing DECRST 25)
* [x] Closes#5039
* [x] Closes#5161 (by ensuring we only `removeSpaces` on the actual
bottom line)
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
## Validation Steps Performed
* Checked the cases from #1245, #357 to validate that they still work
* Added more and more tests for these scenarios, and then I added MORE
tests
* The entire team played with this in selfhost builds
This commit introduces a github action to check our spelling and fixes
the following misspelled words so that we come up green.
It also renames TfEditSes to TfEditSession, because Ses is not a word.
currently, excerpt, fallthrough, identified, occurred, propagate,
provided, rendered, resetting, separate, succeeded, successfully,
terminal, transferred, adheres, breaks, combining, preceded,
architecture, populated, previous, setter, visible, window, within,
appxmanifest, hyphen, control, offset, powerpoint, suppress, parsing,
prioritized, aforementioned, check in, build, filling, indices, layout,
mapping, trying, scroll, terabyte, vetoes, viewport, whose
