Add support for VT100 Auto Wrap Mode (DECAWM) (#3943)

## Summary of the Pull Request

This adds support for the [`DECAWM`](https://vt100.net/docs/vt510-rm/DECAWM) private mode escape sequence, which controls whether or not the output wraps to the next line when the cursor reaches the right edge of the screen. Tested manually, with [Vttest](https://invisible-island.net/vttest/), and with some new unit tests.

## PR Checklist
* [x] Closes #3826
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #3826

## Detailed Description of the Pull Request / Additional comments

The idea was to repurpose the existing `ENABLE_WRAP_AT_EOL_OUTPUT` mode, but the problem with that was it didn't work in VT mode - specifically, disabling it didn't prevent the wrapping from happening. This was because in VT mode the `WC_DELAY_EOL_WRAP` behaviour takes affect, and that bypasses the usual codepath where `ENABLE_WRAP_AT_EOL_OUTPUT` is checked,

To fix this, I had to add additional checks in the `WriteCharsLegacy` function (7dbefe06e41f191a0e83cfefe4896b66094c4089) to make sure the `WC_DELAY_EOL_WRAP` mode is only activated when `ENABLE_WRAP_AT_EOL_OUTPUT`  is also set.

Once that was fixed, though, another issue came to light: the `ENABLE_WRAP_AT_EOL_OUTPUT` mode doesn't actually work as documented. According to the docs, "if this mode is disabled, the last character in the row is overwritten with any subsequent characters". What actually happens is the cursor jumps back to the position at the start of the write, which could be anywhere on the line.

This seems completely broken to me, but I've checked in the Windows XP, and it has the same behaviour, so it looks like that's the way it has always been. So I've added a fix for this (9df98497ca38f7d0ea42623b723a8e2ecf9a4ab9), but it is only applied in VT mode.

Once that basic functionality was in place, though, we just needed a private API in the `ConGetSet` interface to toggle the mode, and then that API could be called from the `AdaptDispatch` class when the `DECAWM` escape sequence was received.

One last thing was to reenable the mode in reponse to a `DECSTR` soft reset. Technically the auto wrap mode was disabled by default on many of the DEC terminals, and some documentation suggests that `DECSTR` should reset it to that state, But most modern terminals (including XTerm) expect the wrapping to be enabled by default, and `DECSTR` reenables that state, so that's the behaviour I've copied.

## Validation Steps Performed

I've add a state machine test to confirm the `DECAWM` escape is dispatched correctly, and a screen buffer test to make sure the output is wrapped or clamped as appropriate for the two states.

I've also confirmed that the "wrap around" test is now working correctly in the _Test of screen features_ in Vttest.

src/host/_stream.cpp

@@ -47,6 +47,7 @@ constexpr unsigned int LOCAL_BUFFER_SIZE = 100;
                                             const BOOL fKeepCursorVisible,
                                             _Inout_opt_ PSHORT psScrollY)
 {
+    const bool inVtMode = WI_IsFlagSet(screenInfo.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
     const COORD bufferSize = screenInfo.GetBufferSize().Dimensions();
     if (coordCursor.X < 0)
     {
@@ -70,7 +71,16 @@ constexpr unsigned int LOCAL_BUFFER_SIZE = 100;
         }
         else
         {
-            coordCursor.X = screenInfo.GetTextBuffer().GetCursor().GetPosition().X;
+            if (inVtMode)
+            {
+                // In VT mode, the cursor must be left in the last column.
+                coordCursor.X = bufferSize.X - 1;
+            }
+            else
+            {
+                // For legacy apps, it is left where it was at the start of the write.
+                coordCursor.X = screenInfo.GetTextBuffer().GetCursor().GetPosition().X;
+            }
         }
     }
 
@@ -85,7 +95,6 @@ constexpr unsigned int LOCAL_BUFFER_SIZE = 100;
     const bool fMarginsSet = srMargins.Bottom > srMargins.Top;
     COORD currentCursor = screenInfo.GetTextBuffer().GetCursor().GetPosition();
     const int iCurrentCursorY = currentCursor.Y;
-    const bool inVtMode = WI_IsFlagSet(screenInfo.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
 
     const bool fCursorInMargins = iCurrentCursorY <= srMargins.Bottom && iCurrentCursorY >= srMargins.Top;
     const bool cursorAboveViewport = coordCursor.Y < 0 && inVtMode;
@@ -308,6 +317,7 @@ constexpr unsigned int LOCAL_BUFFER_SIZE = 100;
     WCHAR LocalBuffer[LOCAL_BUFFER_SIZE];
     size_t TempNumSpaces = 0;
     const bool fUnprocessed = WI_IsFlagClear(screenInfo.OutputMode, ENABLE_PROCESSED_OUTPUT);
+    const bool fWrapAtEOL = WI_IsFlagSet(screenInfo.OutputMode, ENABLE_WRAP_AT_EOL_OUTPUT);
 
     // Must not adjust cursor here. It has to stay on for many write scenarios. Consumers should call for the
     // cursor to be turned off if they want that.
@@ -323,7 +333,7 @@ constexpr unsigned int LOCAL_BUFFER_SIZE = 100;
     while (*pcb < BufferSize)
     {
         // correct for delayed EOL
-        if (cursor.IsDelayedEOLWrap())
+        if (cursor.IsDelayedEOLWrap() && fWrapAtEOL)
         {
             const COORD coordDelayedAt = cursor.GetDelayedAtPosition();
             cursor.ResetDelayEOLWrap();
@@ -509,7 +519,7 @@ constexpr unsigned int LOCAL_BUFFER_SIZE = 100;
             CursorPosition.X = XPosition;
 
             // enforce a delayed newline if we're about to pass the end and the WC_DELAY_EOL_WRAP flag is set.
-            if (WI_IsFlagSet(dwFlags, WC_DELAY_EOL_WRAP) && CursorPosition.X >= coordScreenBufferSize.X)
+            if (WI_IsFlagSet(dwFlags, WC_DELAY_EOL_WRAP) && CursorPosition.X >= coordScreenBufferSize.X && fWrapAtEOL)
             {
                 // Our cursor position as of this time is going to remain on the last position in this column.
                 CursorPosition.X = coordScreenBufferSize.X - 1;
@@ -678,7 +688,7 @@ constexpr unsigned int LOCAL_BUFFER_SIZE = 100;
                 }
                 CATCH_LOG();
             }
-            if (cursor.GetPosition().X == 0 && (screenInfo.OutputMode & ENABLE_WRAP_AT_EOL_OUTPUT) && pwchBuffer > pwchBufferBackupLimit)
+            if (cursor.GetPosition().X == 0 && fWrapAtEOL && pwchBuffer > pwchBufferBackupLimit)
             {
                 if (CheckBisectProcessW(screenInfo,
                                         pwchBufferBackupLimit,
@@ -778,7 +788,7 @@ constexpr unsigned int LOCAL_BUFFER_SIZE = 100;
             if (Char >= UNICODE_SPACE &&
                 IsGlyphFullWidth(Char) &&
                 XPosition >= (coordScreenBufferSize.X - 1) &&
-                (screenInfo.OutputMode & ENABLE_WRAP_AT_EOL_OUTPUT))
+                fWrapAtEOL)
             {
                 const COORD TargetPoint = cursor.GetPosition();
                 ROW& Row = textBuffer.GetRowByOffset(TargetPoint.Y);

src/host/getset.cpp

@@ -1320,6 +1320,22 @@ void ApiRoutines::GetConsoleDisplayModeImpl(ULONG& flags) noexcept
     }
 }
 
+// Routine Description:
+// - A private API call for setting the ENABLE_WRAP_AT_EOL_OUTPUT mode.
+//     This controls whether the cursor moves to the beginning of the next row
+//     when it reaches the end of the current row.
+// Parameters:
+// - wrapAtEOL - set to true to wrap, false to overwrite the last character.
+// Return value:
+// - STATUS_SUCCESS if handled successfully.
+[[nodiscard]] NTSTATUS DoSrvPrivateSetAutoWrapMode(const bool wrapAtEOL)
+{
+    auto& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
+    auto& outputMode = gci.GetActiveOutputBuffer().GetActiveBuffer().OutputMode;
+    WI_UpdateFlag(outputMode, ENABLE_WRAP_AT_EOL_OUTPUT, wrapAtEOL);
+    return STATUS_SUCCESS;
+}
+
 // Routine Description:
 // - A private API call for making the cursor visible or not. Does not modify
 //      blinking state.

src/host/getset.h

@@ -30,6 +30,7 @@ void DoSrvPrivateSetDefaultAttributes(SCREEN_INFORMATION& screenInfo, const bool
 [[nodiscard]] NTSTATUS DoSrvPrivateSetKeypadMode(_In_ bool fApplicationMode);
 
 [[nodiscard]] NTSTATUS DoSrvPrivateSetScreenMode(const bool reverseMode);
+[[nodiscard]] NTSTATUS DoSrvPrivateSetAutoWrapMode(const bool wrapAtEOL);
 
 void DoSrvPrivateShowCursor(SCREEN_INFORMATION& screenInfo, const bool show) noexcept;
 void DoSrvPrivateAllowCursorBlinking(SCREEN_INFORMATION& screenInfo, const bool fEnable);

src/host/outputStream.cpp

@@ -358,6 +358,19 @@ bool ConhostInternalGetSet::PrivateSetScreenMode(const bool reverseMode)
     return NT_SUCCESS(DoSrvPrivateSetScreenMode(reverseMode));
 }
 
+// Routine Description:
+// - Connects the PrivateSetAutoWrapMode call directly into our Driver Message servicing call inside Conhost.exe
+//   PrivateSetAutoWrapMode is an internal-only "API" call that the vt commands can execute,
+//     but it is not represented as a function call on out public API surface.
+// Arguments:
+// - wrapAtEOL - set to true to wrap, false to overwrite the last character.
+// Return Value:
+// - true if successful (see DoSrvPrivateSetAutoWrapMode). false otherwise.
+bool ConhostInternalGetSet::PrivateSetAutoWrapMode(const bool wrapAtEOL)
+{
+    return NT_SUCCESS(DoSrvPrivateSetAutoWrapMode(wrapAtEOL));
+}
+
 // Routine Description:
 // - Connects the PrivateShowCursor call directly into our Driver Message servicing call inside Conhost.exe
 //   PrivateShowCursor is an internal-only "API" call that the vt commands can execute,

src/host/outputStream.hpp

@@ -94,6 +94,7 @@ public:
     bool PrivateSetKeypadMode(const bool applicationMode) override;
 
     bool PrivateSetScreenMode(const bool reverseMode) override;
+    bool PrivateSetAutoWrapMode(const bool wrapAtEOL) override;
 
     bool PrivateShowCursor(const bool show) noexcept override;
     bool PrivateAllowCursorBlinking(const bool enable) override;

src/host/ut_host/ScreenBufferTests.cpp

@@ -184,6 +184,7 @@ class ScreenBufferTests
 
     TEST_METHOD(SetScreenMode);
     TEST_METHOD(SetOriginMode);
+    TEST_METHOD(SetAutoWrapMode);
 
     TEST_METHOD(HardResetBuffer);
 
@@ -4600,6 +4601,50 @@ void ScreenBufferTests::SetOriginMode()
     stateMachine.ProcessString(L"\x1B[?6l");
 }
 
+void ScreenBufferTests::SetAutoWrapMode()
+{
+    auto& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
+    auto& si = gci.GetActiveOutputBuffer();
+    auto& stateMachine = si.GetStateMachine();
+    auto& cursor = si.GetTextBuffer().GetCursor();
+    const auto attributes = si.GetAttributes();
+    WI_SetFlag(si.OutputMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING);
+
+    const auto view = Viewport::FromDimensions({ 0, 0 }, { 80, 25 });
+    si.SetViewport(view, true);
+
+    Log::Comment(L"By default, output should wrap onto the next line.");
+    // Output 6 characters, 3 spaces from the end of the line.
+    short startLine = 0;
+    cursor.SetPosition({ 80 - 3, startLine });
+    stateMachine.ProcessString(L"abcdef");
+    // Half of the the content should wrap onto the next line.
+    VERIFY_IS_TRUE(_ValidateLineContains({ 80 - 3, startLine }, L"abc", attributes));
+    VERIFY_IS_TRUE(_ValidateLineContains({ 0, startLine + 1 }, L"def", attributes));
+    VERIFY_ARE_EQUAL(COORD({ 3, startLine + 1 }), cursor.GetPosition());
+
+    Log::Comment(L"When DECAWM is reset, output is clamped to the line width.");
+    stateMachine.ProcessString(L"\x1b[?7l");
+    // Output 6 characters, 3 spaces from the end of the line.
+    startLine = 2;
+    cursor.SetPosition({ 80 - 3, startLine });
+    stateMachine.ProcessString(L"abcdef");
+    // Content should be clamped to the line width, overwriting the last char.
+    VERIFY_IS_TRUE(_ValidateLineContains({ 80 - 3, startLine }, L"abf", attributes));
+    VERIFY_ARE_EQUAL(COORD({ 79, startLine }), cursor.GetPosition());
+
+    Log::Comment(L"When DECAWM is set, output is wrapped again.");
+    stateMachine.ProcessString(L"\x1b[?7h");
+    // Output 6 characters, 3 spaces from the end of the line.
+    startLine = 4;
+    cursor.SetPosition({ 80 - 3, startLine });
+    stateMachine.ProcessString(L"abcdef");
+    // Half of the the content should wrap onto the next line.
+    VERIFY_IS_TRUE(_ValidateLineContains({ 80 - 3, startLine }, L"abc", attributes));
+    VERIFY_IS_TRUE(_ValidateLineContains({ 0, startLine + 1 }, L"def", attributes));
+    VERIFY_ARE_EQUAL(COORD({ 3, startLine + 1 }), cursor.GetPosition());
+}
+
 void ScreenBufferTests::HardResetBuffer()
 {
     auto& gci = ServiceLocator::LocateGlobals().getConsoleInformation();

src/terminal/adapter/DispatchTypes.hpp

@@ -84,6 +84,7 @@ namespace Microsoft::Console::VirtualTerminal::DispatchTypes
         DECCOLM_SetNumberOfColumns = 3,
         DECSCNM_ScreenMode = 5,
         DECOM_OriginMode = 6,
+        DECAWM_AutoWrapMode = 7,
         ATT610_StartCursorBlink = 12,
         DECTCEM_TextCursorEnableMode = 25,
         XTERM_EnableDECCOLMSupport = 40,

src/terminal/adapter/ITermDispatch.hpp

@@ -56,6 +56,7 @@ public:
     virtual bool EnableCursorBlinking(const bool enable) = 0; // ATT610
     virtual bool SetScreenMode(const bool reverseMode) = 0; //DECSCNM
     virtual bool SetOriginMode(const bool relativeMode) = 0; // DECOM
+    virtual bool SetAutoWrapMode(const bool wrapAtEOL) = 0; // DECAWM
     virtual bool SetTopBottomScrollingMargins(const size_t topMargin, const size_t bottomMargin) = 0; // DECSTBM
     virtual bool WarningBell() = 0; // BEL
     virtual bool CarriageReturn() = 0; // CR

src/terminal/adapter/adaptDispatch.cpp

@@ -940,6 +940,9 @@ bool AdaptDispatch::_PrivateModeParamsHelper(const DispatchTypes::PrivateModePar
         // The cursor is also moved to the new home position when the origin mode is set or reset.
         success = SetOriginMode(enable) && CursorPosition(1, 1);
         break;
+    case DispatchTypes::PrivateModeParams::DECAWM_AutoWrapMode:
+        success = SetAutoWrapMode(enable);
+        break;
     case DispatchTypes::PrivateModeParams::ATT610_StartCursorBlink:
         success = EnableCursorBlinking(enable);
         break;
@@ -1108,6 +1111,19 @@ bool AdaptDispatch::SetOriginMode(const bool relativeMode) noexcept
     return true;
 }
 
+// Routine Description:
+// - DECAWM - Sets the Auto Wrap Mode.
+//    This controls whether the cursor moves to the beginning of the next row
+//    when it reaches the end of the current row.
+// Arguments:
+// - wrapAtEOL - set to true to wrap, false to overwrite the last character.
+// Return Value:
+// - True if handled successfully. False otherwise.
+bool AdaptDispatch::SetAutoWrapMode(const bool wrapAtEOL)
+{
+    return _pConApi->PrivateSetAutoWrapMode(wrapAtEOL);
+}
+
 // Routine Description:
 // - DECSTBM - Set Scrolling Region
 // This control function sets the top and bottom margins for the current page.
@@ -1391,7 +1407,7 @@ bool AdaptDispatch::DesignateCharset(const wchar_t wchCharset) noexcept
 //  X Text cursor enable          DECTCEM     Cursor enabled.
 //    Insert/replace              IRM         Replace mode.
 //  X Origin                      DECOM       Absolute (cursor origin at upper-left of screen.)
-//    Autowrap                    DECAWM      No autowrap.
+//  X Autowrap                    DECAWM      Autowrap enabled (matches XTerm behavior).
 //    National replacement        DECNRCM     Multinational set.
 //        character set
 //    Keyboard action             KAM         Unlocked.
@@ -1422,6 +1438,10 @@ bool AdaptDispatch::SoftReset()
         success = SetOriginMode(false); // Absolute cursor addressing.
     }
     if (success)
+    {
+        success = SetAutoWrapMode(true); // Wrap at end of line.
+    }
+    if (success)
     {
         success = SetCursorKeysMode(false); // Normal characters.
     }

src/terminal/adapter/adaptDispatch.hpp

@@ -70,6 +70,7 @@ namespace Microsoft::Console::VirtualTerminal
         bool EnableCursorBlinking(const bool enable) override; // ATT610
         bool SetScreenMode(const bool reverseMode) override; //DECSCNM
         bool SetOriginMode(const bool relativeMode) noexcept override; // DECOM
+        bool SetAutoWrapMode(const bool wrapAtEOL) override; // DECAWM
         bool SetTopBottomScrollingMargins(const size_t topMargin,
                                           const size_t bottomMargin) override; // DECSTBM
         bool WarningBell() override; // BEL

src/terminal/adapter/conGetSet.hpp

@@ -58,6 +58,7 @@ namespace Microsoft::Console::VirtualTerminal
         virtual bool PrivateSetKeypadMode(const bool applicationMode) = 0;
 
         virtual bool PrivateSetScreenMode(const bool reverseMode) = 0;
+        virtual bool PrivateSetAutoWrapMode(const bool wrapAtEOL) = 0;
 
         virtual bool PrivateShowCursor(const bool show) = 0;
         virtual bool PrivateAllowCursorBlinking(const bool enable) = 0;

src/terminal/adapter/termDispatch.hpp

@@ -50,6 +50,7 @@ public:
     bool EnableCursorBlinking(const bool /*enable*/) noexcept override { return false; } // ATT610
     bool SetScreenMode(const bool /*reverseMode*/) noexcept override { return false; } //DECSCNM
     bool SetOriginMode(const bool /*relativeMode*/) noexcept override { return false; }; // DECOM
+    bool SetAutoWrapMode(const bool /*wrapAtEOL*/) noexcept override { return false; }; // DECAWM
     bool SetTopBottomScrollingMargins(const size_t /*topMargin*/, const size_t /*bottomMargin*/) noexcept override { return false; } // DECSTBM
     bool WarningBell() noexcept override { return false; } // BEL
     bool CarriageReturn() noexcept override { return false; } // CR

src/terminal/adapter/ut_adapter/adapterTest.cpp

@@ -169,6 +169,13 @@ public:
         return true;
     }
 
+    bool PrivateSetAutoWrapMode(const bool /*wrapAtEOL*/) override
+    {
+        Log::Comment(L"PrivateSetAutoWrapMode MOCK called...");
+
+        return false;
+    }
+
     bool PrivateShowCursor(const bool show) override
     {
         Log::Comment(L"PrivateShowCursor MOCK called...");

src/terminal/parser/ut_parser/OutputEngineTest.cpp

@@ -680,6 +680,7 @@ public:
         _cursorBlinking{ true },
         _isScreenModeReversed{ false },
         _isOriginModeRelative{ false },
+        _isAutoWrapEnabled{ true },
         _warningBell{ false },
         _carriageReturn{ false },
         _lineFeed{ false },
@@ -865,6 +866,9 @@ public:
             // The cursor is also moved to the new home position when the origin mode is set or reset.
             fSuccess = SetOriginMode(fEnable) && CursorPosition(1, 1);
             break;
+        case DispatchTypes::PrivateModeParams::DECAWM_AutoWrapMode:
+            fSuccess = SetAutoWrapMode(fEnable);
+            break;
         case DispatchTypes::PrivateModeParams::ATT610_StartCursorBlink:
             fSuccess = EnableCursorBlinking(fEnable);
             break;
@@ -936,6 +940,12 @@ public:
         return true;
     }
 
+    bool SetAutoWrapMode(const bool wrapAtEOL) noexcept override
+    {
+        _isAutoWrapEnabled = wrapAtEOL;
+        return true;
+    }
+
     bool WarningBell() noexcept override
     {
         _warningBell = true;
@@ -1011,6 +1021,7 @@ public:
     bool _cursorBlinking;
     bool _isScreenModeReversed;
     bool _isOriginModeRelative;
+    bool _isAutoWrapEnabled;
     bool _warningBell;
     bool _carriageReturn;
     bool _lineFeed;
@@ -1345,6 +1356,25 @@ class StateMachineExternalTest final
         pDispatch->ClearState();
     }
 
+    TEST_METHOD(TestAutoWrapMode)
+    {
+        auto dispatch = std::make_unique<StatefulDispatch>();
+        auto pDispatch = dispatch.get();
+        auto engine = std::make_unique<OutputStateMachineEngine>(std::move(dispatch));
+        StateMachine mach(std::move(engine));
+
+        mach.ProcessString(L"\x1b[?7l");
+        VERIFY_IS_FALSE(pDispatch->_isAutoWrapEnabled);
+
+        pDispatch->ClearState();
+        pDispatch->_isAutoWrapEnabled = false;
+
+        mach.ProcessString(L"\x1b[?7h");
+        VERIFY_IS_TRUE(pDispatch->_isAutoWrapEnabled);
+
+        pDispatch->ClearState();
+    }
+
     TEST_METHOD(TestCursorBlinking)
     {
         auto dispatch = std::make_unique<StatefulDispatch>();