terminal/src/host
James Holderness 701b421286 Add support for all the line feed control sequences (#3271)
## Summary of the Pull Request

This adds support for the `FF` (form feed) and `VT` (vertical tab) [control characters](https://vt100.net/docs/vt510-rm/chapter4.html#T4-1), as well as the [`NEL` (Next Line)](https://vt100.net/docs/vt510-rm/NEL.html) and [`IND` (Index)](https://vt100.net/docs/vt510-rm/IND.html) escape sequences.

## References

#976 discusses the conflict between VT100 Index sequence and the VT52 cursor back sequence.

## PR Checklist
* [x] Closes #3189
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3189

## Detailed Description of the Pull Request / Additional comments

I've added a `LineFeed` method to the `ITermDispatch` interface, with an enum parameter specifying the required line feed type (i.e. with carriage return, without carriage return, or dependent on the [`LNM` mode](https://vt100.net/docs/vt510-rm/LNM.html)). The output state machine can then call that method to handle the various line feed control characters (parsed in the `ActionExecute` method), as well the `NEL` and `IND` escape sequences (parsed in the `ActionEscDispatch` method).

The `AdaptDispatch` implementation of `LineFeed` then forwards the call to a new `PrivateLineFeed` method in the `ConGetSet` interface, which simply takes a bool parameter specifying whether a carriage return is required or not. In the case of mode-dependent line feeds, the `AdaptDispatch` implementation determines whether the return is necessary or not, based on the existing _AutoReturnOnNewLine_ setting (which I'm obtaining via another new `PrivateGetLineFeedMode` method).

Ultimately we'll want to support changing the mode via the [`LNM` escape sequence](https://vt100.net/docs/vt510-rm/LNM.html), but there's no urgent need for that now. And using the existing _AutoReturnOnNewLine_ setting as a substitute for the mode gives us backwards compatible behaviour, since that will be true for the Windows shells (which expect a linefeed to also generate a carriage return), and false in a WSL bash shell (which won't want the carriage return by default).

As for the actual `PrivateLineFeed` implementation, that is just a simplified version of how the line feed would previously have been executed in the `WriteCharsLegacy` function. This includes setting the cursor to "On" (with `Cursor::SetIsOn`), potentially clearing the wrap property of the line being left (with `CharRow::SetWrapForced` false), and then setting the new position using `AdjustCursorPosition` with the _fKeepCursorVisible_ parameter set to false.

I'm unsure whether the `SetIsOn` call is really necessary, and I think the way the forced wrap is handled needs a rethink in general, but for now this should at least be compatible with the existing behaviour.

Finally, in order to make this all work in the _Windows Terminal_ app, I also had to add a basic implementation of the `ITermDispatch::LineFeed` method in the `TerminalDispatch` class. There is currently no need to support mode-specific line feeds here, so this simply forwards a `\n` or `\r\n` to the `Execute` method, which is ultimately handled by the `Terminal::_WriteBuffer` implementation.

## Validation Steps Performed

I've added output engine tests which confirm that the various control characters and escape sequences trigger the dispatch method correctly. Then I've added adapter tests which confirm the various dispatch options trigger the `PrivateLineFeed` API correctly. And finally I added some screen buffer tests that check the actual results of the `NEL` and `IND` sequences, which covers both forms of the `PrivateLineFeed` API (i.e. with and without a carriage return).

I've also run the _Test of cursor movements_ in the [Vttest](https://invisible-island.net/vttest/) utility, and confirmed that screens 1, 2, and 5 are now working correctly. The first two depend on `NEL` and `IND` being supported, and screen 5 requires the `VT` control character.
2020-01-15 13:41:55 +00:00
..
dll Unify and clean up the common build properties (#3429) 2019-11-05 14:29:11 -08:00
exe Add experimental retro terminal effects (#3468) 2019-12-12 13:44:01 +00:00
ft_host Remove global namespaced min/max and replace it with STL min/max (#4173) 2020-01-10 13:27:05 +00:00
ft_integrity Integrate inbox changes up to 68d3b53286dd 2019-12-02 17:24:26 -08:00
ft_uia Fix unittesting our .xaml classes (#4105) 2020-01-10 18:55:31 +00:00
lib Migrate Search module as a shared component for Terminal Search (#3279) 2019-11-14 14:36:41 -08:00
ut_host Add support for all the line feed control sequences (#3271) 2020-01-15 13:41:55 +00:00
ut_lib Unify and clean up the common build properties (#3429) 2019-11-05 14:29:11 -08:00
_output.cpp Correct fill attributes when scrolling and erasing (#3100) 2019-12-10 23:14:40 +00:00
_output.h Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
_stream.cpp Reverse the behavior of the IS_GLYPH_CHAR macro so its function now matches its name. (#4209) 2020-01-14 16:43:38 +00:00
_stream.h Replace some macros with constexpr (#3362) 2019-11-01 10:33:09 -07:00
alias.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
alias.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
ApiRoutines.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
cmdline.cpp Replace macros with constexpr part 2 (#3416) 2019-11-04 07:37:47 -06:00
cmdline.h inbox: reflect changes from 20h1 branch (#2310) 2019-08-07 10:58:53 -07:00
CommandListPopup.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
CommandListPopup.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
CommandNumberPopup.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
CommandNumberPopup.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
conapi.h Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
conareainfo.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
conareainfo.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
conattrs.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
conddkrefs.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
conhost.rcv Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
conhostv2_traceviewpp.tvpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
conimeinfo.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
conimeinfo.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
conserv.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
ConsoleArguments.cpp Fix the conhost command line not being properly escaped (#1815) 2019-07-11 19:38:56 -07:00
ConsoleArguments.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
consoleInformation.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
conv.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
convarea.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
conwinuserrefs.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
CopyFromCharPopup.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
CopyFromCharPopup.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
CopyToCharPopup.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
CopyToCharPopup.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
CursorBlinker.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
CursorBlinker.hpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
dbcs.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
dbcs.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
directio.cpp Replace some macros with constexpr (#3362) 2019-11-01 10:33:09 -07:00
directio.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
dirs Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
exemain.cpp Force the use of v2 (non-legacy) conhost when in ConPTY mode (#1935) 2019-07-12 15:20:45 -07:00
getset.cpp Add support for all the line feed control sequences (#3271) 2020-01-15 13:41:55 +00:00
getset.h Add support for all the line feed control sequences (#3271) 2020-01-15 13:41:55 +00:00
globals.cpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
globals.h Revert locking changes (#3488) 2019-11-08 13:44:52 -08:00
handle.cpp Removed using namespace directive from header files (#955) 2019-05-30 11:14:21 -07:00
handle.h Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
history.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
history.h Replace macros with constexpr part 2 (#3416) 2019-11-04 07:37:47 -06:00
host-common.vcxitems Migrate Search module as a shared component for Terminal Search (#3279) 2019-11-14 14:36:41 -08:00
IIoProvider.hpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
init.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
init.hpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
input.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
input.h Replace macros with constexpr part 2 (#3416) 2019-11-04 07:37:47 -06:00
inputBuffer.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
inputBuffer.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
inputKeyInfo.cpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
inputReadHandleData.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
inputReadHandleData.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
misc.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
misc.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
ntprivapi.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
ntprivapi.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
output.cpp Correct fill attributes when scrolling and erasing (#3100) 2019-12-10 23:14:40 +00:00
output.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
outputStream.cpp Add support for all the line feed control sequences (#3271) 2020-01-15 13:41:55 +00:00
outputStream.hpp Add support for all the line feed control sequences (#3271) 2020-01-15 13:41:55 +00:00
popup.cpp Get rid of the SCREEN_INFORMATION::LineChar array (#3371) 2019-10-31 10:12:41 -05:00
popup.h Replace some macros with constexpr (#3362) 2019-11-01 10:33:09 -07:00
precomp.cpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
precomp.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
PtySignalInputThread.cpp Correct improper usage of THROW_IF_NULL_ALLOC (#4128) 2020-01-07 13:27:18 -08:00
PtySignalInputThread.hpp Revert locking changes (#3488) 2019-11-08 13:44:52 -08:00
readData.cpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
readData.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
readDataCooked.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
readDataCooked.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
readDataDirect.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
readDataDirect.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
readDataRaw.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
readDataRaw.hpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
registry.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
registry.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
renderData.cpp Search - add search box control and implement search experience (#3590) 2019-12-17 15:52:37 +00:00
renderData.hpp Search - add search box control and implement search experience (#3590) 2019-12-17 15:52:37 +00:00
renderFontDefaults.cpp Allow FontInfo{,Base,Desired} to store a font name > 32 wch (#3107) 2019-10-14 21:23:45 -07:00
renderFontDefaults.hpp Allow FontInfo{,Base,Desired} to store a font name > 32 wch (#3107) 2019-10-14 21:23:45 -07:00
res.rc Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
resource.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
runft.bat Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
runtests.bat Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
runut.bat Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
ScreenBufferRenderTarget.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
ScreenBufferRenderTarget.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
screenInfo.cpp Move reflowing the buffer to TextBuffer (#4197) 2020-01-14 21:34:43 +00:00
screenInfo.hpp Get rid of the SCREEN_INFORMATION::LineChar array (#3371) 2019-10-31 10:12:41 -05:00
scrolling.cpp Accessibility: Set-up UIA Tree (#1691) 2019-07-29 15:21:15 -07:00
scrolling.hpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
selection.cpp Allow FontInfo{,Base,Desired} to store a font name > 32 wch (#3107) 2019-10-14 21:23:45 -07:00
selection.hpp Allow FontInfo{,Base,Desired} to store a font name > 32 wch (#3107) 2019-10-14 21:23:45 -07:00
selectionInput.cpp Migrate Search module as a shared component for Terminal Search (#3279) 2019-11-14 14:36:41 -08:00
selectionState.cpp Removed using namespace directive from header files (#955) 2019-05-30 11:14:21 -07:00
server.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
settings.cpp Replace macros with constexpr part 2 (#3416) 2019-11-04 07:37:47 -06:00
settings.hpp Replace some macros with constexpr (#3362) 2019-11-01 10:33:09 -07:00
sources.inc Migrate Search module as a shared component for Terminal Search (#3279) 2019-11-14 14:36:41 -08:00
sources.test.inc Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00
srvinit.cpp Revert locking changes (#3488) 2019-11-08 13:44:52 -08:00
srvinit.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
stream.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
stream.h add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
telemetry.cpp Allow FontInfo{,Base,Desired} to store a font name > 32 wch (#3107) 2019-10-14 21:23:45 -07:00
telemetry.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
tracing.cpp Accessibility: Refactor Providers (#2414) 2019-08-20 16:32:44 -07:00
tracing.hpp Accessibility: Set-up UIA Tree (#1691) 2019-07-29 15:21:15 -07:00
utf8ToWideCharParser.cpp Compensate for non-minimal UTF-8 encodings (#3380) 2019-10-31 10:50:34 -07:00
utf8ToWideCharParser.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
utils.cpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
utils.hpp add clang-format conf to the project, format the c++ code (#1141) 2019-06-11 13:27:09 -07:00
VtInputThread.cpp Make the terminal parser/adapter and related classes use modern… (#3956) 2019-12-19 14:12:53 -08:00
VtInputThread.hpp Revert locking changes (#3488) 2019-11-08 13:44:52 -08:00
VtIo.cpp Revert locking changes (#3488) 2019-11-08 13:44:52 -08:00
VtIo.hpp Revert locking changes (#3488) 2019-11-08 13:44:52 -08:00
writeData.cpp Removed using namespace directive from header files (#955) 2019-05-30 11:14:21 -07:00
writeData.hpp Initial release of the Windows Terminal source code 2019-05-02 15:29:04 -07:00