maxmustermann
/
terminal
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

First draft of a spec for VT52 escape sequences (#2017) 

* First draft of a spec for splitting off the existing VT52 escape sequences, and extending the VT52 support.

* Make the issue ID visible on GitHub.

* Added suggested mappings for the Graphics Mode character set.

* Add escape sequences for all the commands and clarify the use of the ESC < sequence when switching back to ANSI mode.

* Add details about the differing boundary rules of the VT100 CUP command and the VT52 Direct Cursor Address command.

* Specify the identifying sequence that the Identify command should return.

* Add details of the print commands.

* Add a list of keyboard sequences that are different in the VT52 mode, and make the description of the Keypad Mode commands a little clearer.

* Add a section describing the testing needed to cover the new functionality.
This commit is contained in:
James Holderness 2019-08-01 14:23:10 +01:00 committed by Mike Griese
parent 66044ca605
commit 6749ab03b8
1 changed files with 255 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

255
doc/specs/#976 - VT52 escape sequences.md Normal file
View File

@ -0,0 +1,255 @@
---
author: James Holderness @j4james
created on: 2019-07-17
last updated: 2019-07-28
issue id: 976
---
# VT52 Escape Sequences
## Abstract
This spec outlines the work required to split off the existing VT52 commands from the VT100 implementation, and extend the VT52 support to cover all of the core commands.
## Inspiration
The existing VT52 commands aren't currently implemented as a separate mode, so they conflict with sequences defined in the VT100 specification. This is blocking us from adding support for the VT100 Index (IND) escape sequence, which is one of the missing commands required to pass the test of cursor movements in Vttest.
## Solution Design
The basic idea is to add support for the [DECANM private mode sequence](https://vt100.net/docs/vt100-ug/chapter3.html#DECANM), which can then be used to switch from the default _ANSI_ mode, to a new _VT52_ mode. Once in _VT52_ mode, there is a separate [_Enter ANSI Mode_ sequence](https://vt100.net/docs/vt100-ug/chapter3.html#VT52ANSI) (`ESC <`) to switch back again.
In terms of implementation, there are a number of areas of the system that would need to be updated.
### The State Machine
In order to implement the VT52 compatibility mode correctly, we'll need to introduce a flag in the `StateMachine` class that indicates the mode that is currently active. When in VT52 mode, certain paths in the state diagram should not be followed - for example, you can't have CSI, OSC, or SS3 escape sequences. There would also need to be an additional state to handle VT52 parameters (for the _Direct Cursor Address_ command). These parameters take a different form to the typical VT100 parameters, as they follow the command character instead of preceding it.
It would probably be best to introduce a new dispatch method in the `IStateMachineEngine` interface to handle the parsed VT52 sequences, since the existing `ActionEscDispatch` does not support parameters (which are required for the _Direct Cursor Address_ command). I think it would also make for a cleaner implementation to have the VT52 commands separate from the VT100 code, and would likely have less impact on the performance that way.
### The Terminal Input
The escape sequences generated by the keyboard for function keys, cursor keys, and the numeric keypad, are not the same in VT52 mode as they are in ANSI mode. So there would need to be a flag in the `TerminalInput` class to keep track of the current mode, and thus be able to generate the appropriate sequences for that mode.
Technically the VT52 keyboard doesn't map directly to a typical PC keyboard, so we can't always work from the specs in deciding what sequences are required for each key. When in doubt, we should probably be trying to match the key sequences generated by XTerm. The sequences below are based on the default XTerm mappings.
**Function Keys**
The functions keys <kbd>F1</kbd> to <kbd>F4</kbd> generate a simple ESC prefix instead of SS3 (or CSI). These correspond with the four function keys on the VT100 keypad. In V52 mode they are not affected by modifiers.
Key | ANSI mode | VT52 mode
---------------|-----------|-----------
<kbd>F1</kbd> | `SS3 P` | `ESC P`
<kbd>F2</kbd> | `SS3 Q` | `ESC Q`
<kbd>F3</kbd> | `SS3 R` | `ESC R`
<kbd>F4</kbd> | `SS3 S` | `ESC S`
The function keys <kbd>F5</kbd> to <kbd>F12</kbd> generate the same sequences as they do in ANSI mode, except that they are not affected by modifiers. These correspond with a subset of the top-row functions keys on the VT220, along with the Windows <kbd>Menu</kbd> key mapping to the VT220 <kbd>DO</kbd> key.
Key | Sequence
----------------|-------------
<kbd>F5</kbd> | `CSI 1 5 ~`
<kbd>F6</kbd> | `CSI 1 7 ~`
<kbd>F7</kbd> | `CSI 1 8 ~`
<kbd>F8</kbd> | `CSI 1 9 ~`
<kbd>F9</kbd> | `CSI 2 0 ~`
<kbd>F10</kbd> | `CSI 2 1 ~`
<kbd>F11</kbd> | `CSI 2 3 ~`
<kbd>F12</kbd> | `CSI 2 4 ~`
<kbd>Menu</kbd> | `CSI 2 9 ~`
**Cursor and Editing Keys**
The cursor keys generate a simple ESC prefix instead of CSI or SS3. These correspond with the cursor keys on the VT100, except for <kbd>Home</kbd> and <kbd>End</kbd>, which are XTerm extensions. In V52 mode, they are not affected by modifiers, nor are they affected by the DECCKM _Cursor Keys_ mode.
Key | ANSI mode | VT52 mode
-----------------|-----------|-----------
<kbd>Up</kbd> | `CSI A` | `ESC A`
<kbd>Down</kbd> | `CSI B` | `ESC B`
<kbd>Right</kbd> | `CSI C` | `ESC C`
<kbd>Left</kbd> | `CSI D` | `ESC D`
<kbd>End</kbd> | `CSI F` | `ESC F`
<kbd>Home</kbd> | `CSI H` | `ESC H`
The "editing" keys generate the same sequences as they do in ANSI mode, except that they are not affected by modifiers. These correspond with a subset of the editing keys on the VT220.
Key | Sequence
----------------|-----------
<kbd>Ins</kbd> | `CSI 2 ~`
<kbd>Del</kbd> | `CSI 3 ~`
<kbd>PgUp</kbd> | `CSI 5 ~`
<kbd>PgDn</kbd> | `CSI 6 ~`
**Numeric Keypad**
With <kbd>Num Lock</kbd> disabled, most of the keys on the numeric keypad function the same as cursor keys or editing keys, but with the addition of a center <kbd>5</kbd> key. As a described above, the cursor keys generate a simple ESC prefix instead of CSI or SS3, while the editing keys remain unchanged (with the exception of modifiers).
In V52 mode, most modifiers are ignored, except for <kbd>Shift</kbd>, which is the equivalent of enabling <kbd>Num Lock</kbd> (i.e. the keys just generate their corresponding digit characters or `.`). With <kbd>Num Lock</kbd> enabled, it's the other way arround - the digits are generated by default, while <kbd>Shift</kbd> enables the cursor/editing functionality.
Key | Alias | ANSI mode | VT52 mode
-------------|-------|-----------|-----------
<kbd>.</kbd> | Del | `CSI 3 ~` | `CSI 3 ~`
<kbd>0</kbd> | Ins | `CSI 2 ~` | `CSI 2 ~`
<kbd>1</kbd> | End | `CSI F` | `ESC F`
<kbd>2</kbd> | Down | `CSI B` | `ESC B`
<kbd>3</kbd> | PgDn | `CSI 6 ~` | `CSI 6 ~`
<kbd>4</kbd> | Left | `CSI D` | `ESC D`
<kbd>4</kbd> | Clear | `CSI E` | `ESC E`
<kbd>6</kbd> | Right | `CSI C` | `ESC C`
<kbd>7</kbd> | Home | `CSI H` | `ESC H`
<kbd>8</kbd> | Up | `CSI A` | `ESC A`
<kbd>9</kbd> | PgUp | `CSI 5 ~` | `CSI 5 ~`
When the DECKPAM _Alternate/Application Keypad Mode_ is set, though, the <kbd>Shift</kbd> modifier has a different affect on the numeric keypad. The sequences generated now correspond with the VT100/V52 numeric keypad keys. In VT52 mode, these sequences are not affected by any other modifiers, and this mode only applies when <kbd>Num Lock</kbd> is disabled.
Key | Alias | ANSI mode | VT52 mode
-------------|-------|-----------|-----------
<kbd>.</kbd> | Del | `SS3 2 n` | `ESC ? n`
<kbd>0</kbd> | Ins | `SS3 2 p` | `ESC ? p`
<kbd>1</kbd> | End | `SS3 2 q` | `ESC ? q `
<kbd>2</kbd> | Down | `SS3 2 r` | `ESC ? r`
<kbd>3</kbd> | PgDn | `SS3 2 s` | `ESC ? s`
<kbd>4</kbd> | Left | `SS3 2 t` | `ESC ? t`
<kbd>4</kbd> | Clear | `SS3 2 u` | `ESC ? u`
<kbd>6</kbd> | Right | `SS3 2 v` | `ESC ? v`
<kbd>7</kbd> | Home | `SS3 2 w` | `ESC ? w`
<kbd>8</kbd> | Up | `SS3 2 x` | `ESC ? x`
<kbd>9</kbd> | PgUp | `SS3 2 y` | `ESC ? y`
When the DECKPAM _Alternate/Application Keypad Mode_ is set, the "arithmetic" keys on the numeric keypad are also affected (this includes the <kbd>Enter</kbd> key). The sequences generated again correspond with the VT100/VT52 numeric keys (more or less), but this mapping is active even without the <kbd>Shift</kbd> modifier (and in VT52 mode all other modifiers are ignored too). As above, the mode only applies when <kbd>Num Lock</kbd> is disabled.
Key | ANSI mode | VT52 mode
-----------------|-----------|-----------
<kbd>*</kbd> | `SS3 j` | `ESC ? j`
<kbd>+</kbd> | `SS3 k` | `ESC ? k`
<kbd>-</kbd> | `SS3 m` | `ESC ? m`
<kbd>/</kbd> | `SS3 o` | `ESC ? o`
<kbd>Enter</kbd> | `SS3 M` | `ESC ? M`
Note that the DECKPAM _Application Keypad Mode_ is not currently implemented in ANSI mode, so perhaps that needs to be addressed first, before trying to add support for the VT52 _Alternate Keypad Mode_.
### Changing Modes
The `_PrivateModeParamsHelper` method in the `AdaptDispatch` class would need to be extended to handle the DECANM mode parameter, and trigger a function to switch to VT52 mode. The typical pattern for this seems to be through a `PrivateXXX` method in the `ConGetSet` interface. Then the `ConhostInternalGetSet` implementation can pass that flag on to the active output buffer's `StateMachine`, and the active input buffer's `TerminalInput` instance.
Changing back from VT52 mode to ANSI mode would need to be achieved with a separate VT52 command (`ESC <`), since the VT100 CSI mode sequences would no longer be active. This would be handled in the same place as the other VT52 commands, in the `OutputStateMachineEngine`, and then passed on to the mode selection method in the `AdaptDispatch` class described above (essentially the equivalent of the DECANM private mode being set).
### Additional VT52 Commands
Most of the missing VT52 functionality can be implemented in terms of existing VT100 methods.
* The _Cursor Up_ (`ESC A`), _Cursor Down_ (`ESC B`), _Cursor Left_ (`ESC D`), and _Cursor Right_ (`ESC C`) commands are already implemented.
* The _Enter Graphics Mode_ (`ESC F`) and _Exit Graphics Mode_ (`ESC G`) commands can probably use the existing `DesignateCharset` method, although this would require a new `VTCharacterSets` option with a corresponding table of characters (see below).
* The _Reverse Line Feed_ (`ESC I`) command can use the existing `ReverseLineFeed` method.
* The _Erase to End of Display_ (`ESC J`) and _Erase to End of Line_ (`ESC K`) commands can use the existing `EraseInDisplay` and `EraseInLine` methods.
* The _Cursor Home_ (`ESC H`) and _Direct Cursor Address_ (`ESC Y`) commands can probably be implemented using the `CursorPosition` method. Technically the _Direct Cursor Address_ has different rules for the boundary conditions (the CUP command clamps out of range coordinates, while the _Direct Cursor Address_ command ignores them, judged individually - one may be ignored while the other is interpreted). Nobody seems to get that right, though, so it's probably not that big a deal.
* The _Identify_ (`ESC Z`) command may be the only one that doesn't build on existing functionality, but it should be a fairly trivial addition to the `AdaptDispatch` class. For a terminal emulating VT52, the identifying sequence should be `ESC / Z`.
* The _Enter Keypad Mode_ (`ESC =`) and _Exit Keypad Mode_ (`ESC >`) commands can use the existing `SetKeypadMode` method, assuming the `TerminalInput` class already knows to generate different sequences when in VT52 mode (as described in the _Terminal Input_ section above).
* The _Enter ANSI Mode_ (`ESC <`) command can just call through to the new mode selection method in the `AdaptDispatch` class as discussed in the _Changing Modes_ section above.
There are also a few VT52 print commands, but those are not technically part of the core command set, and considering we don't yet support any of the VT102 print commands, I think they can probably be considered out of scope for now. Briefly they are:
* _Auto Print_ on (`ESC ^`) and off (`ESC _`) commands. In auto print mode, a display line prints after you move the cursor off the line, or during an auto wrap.
* _Print Controller_ on (`ESC W`) and off (`ESC X`) commands. When enabled, the terminal transmits received characters to the printer without displaying them.
* The _Print Cursor Line_ (`ESC V`) command prints the display line with the cursor.
* The _Print Screen_ (`ESC ]`) command prints the screen (or at least the scrolling region).
I suspect most, if not all of these, would be direct equivalents of the VT102 print commands, if we ever implemented those.
### Graphic Mode Character Set
The table below lists suggested mappings for the _Graphics Mode_ character set, based on the descriptions in the [VT102 User Guide](https://vt100.net/docs/vt102-ug/table5-15.html).
Note that there is only the one _fraction numerator_ character in Unicode, so superscript digits have instead been used for the numerators 3, 5, and 7. There are also not enough _horizontal scan line_ characters (for the _bar at scan x_ characters), so each of them is used twice to cover the full range.
ASCII Character |Mapped Glyph |Unicode Value |Spec Description
----------------|---------------|---------------|----------------
_ | |U+0020 |Blank
` | |U+0020 |Reserved
a |█ |U+2588 |Solid rectangle
b |⅟ |U+215F |1/
c |³ |U+00B3 |3/
d |⁵ |U+2075 |5/
e |⁷ |U+2077 |7/
f |° |U+00B0 |Degrees
g |± |U+00B1 |Plus or minus
h |→ |U+2192 |Right arrow
i |… |U+2026 |Ellipsis (dots)
j |÷ |U+00F7 |Divide by
k |↓ |U+2193 |Down arrow
l |⎺ |U+23BA |Bar at scan 0
m |⎺ |U+23BA |Bar at scan 1
n |⎻ |U+23BB |Bar at scan 2
o |⎻ |U+23BB |Bar at scan 3
p |⎼ |U+23BC |Bar at scan 4
q |⎼ |U+23BC |Bar at scan 5
r |⎽ |U+23BD |Bar at scan 6
s |⎽ |U+23BD |Bar at scan 7
t |₀ |U+2080 |Subscript 0
u |₁ |U+2081 |Subscript 1
v |₂ |U+2082 |Subscript 2
w |₃ |U+2083 |Subscript 3
x |₄ |U+2084 |Subscript 4
y |₅ |U+2085 |Subscript 5
z |₆ |U+2086 |Subscript 6
{ |₇ |U+2087 |Subscript 7
\| |₈ |U+2088 |Subscript 8
} |₉ |U+2089 |Subscript 9
\~ |¶ |U+00B6 |Paragraph
### Testing
A simple unit test will need to be added to the `AdapterTest` class, to confirm that calls to toggle between the ANSI and VT52 modes in the `AdaptDispatch` class are correctly forwarded to the corresponding `PrivateXXX` handler in the `ConGetSet` interface.
The majority of the testing would be handled in the `StateMachineExternalTest` class though. These tests would confirm that the various VT52 sequences trigger the expected methods in the `ITermDispatch` interface when VT52 Mode is enabled, and also that they don't do anything when in ANSI mode.
There shouldn't really be any need for additional tests in the `ScreenBufferTests` class, since we're relying on existing VT100 functionality which should already be tested there.
For fuzzing support, we'll need to add the DECANM option to the `GeneratePrivateModeParamToken` method in the `VTCommandFuzzer` class, and also probably add two additional token generator methods - one specifically for the _Direct Cursor Address_ command, which requires parameters, and another to handle the remaining parameterless commands.
In terms of manual testing, it can be useful to run the _Test of VT52 mode_ option in Vttest, and confirm that everything looks correct there. It's also worth going through some of the options in the The _Test of keyboard_ section, since those tests aren't only intended for the later VT models - they do cover the VT52 keyboard as well.
## UI/UX Design
There is no additional UI associated with this feature.
## Capabilities
### Accessibility
This should not impact accessibility any more than the existing escape sequences.
### Security
This should not introduce any new security issues.
### Reliability
This should not introduce any new reliability issues.
### Compatibility
This could be a breaking change for code that relies on the few existing VT52 commands being available without a mode change. However, that functionality is non-standard, and has not been around for that long. There is almost certainly more benefit in being able to implement the missing VT100 functionality than there is in retaining that non-standard behaviour.
### Performance, Power, and Efficiency
The additional mode flags and associated processing in the `StateMachine` and `TerminalInput` classes could have some performance impact, but that is unlikely to be significant.
## Potential Issues
The only negative impacts I can think of would be the potential for breaking changes, and the possible impact on performance, as discussed in the _Compatibility_ and _Performance_ sections above. But as with any new code, there is always the possibility of new bugs being introduced as well.
## Future considerations
As mentioned in the _Inspiration_ section, having the VT52 functionality isolated with a new mode would enable us to implement the VT100 Index (IND) escape sequence, which currently conflicts with the VT52 _Cursor Left_ command.
## Resources
* [VT52 Mode Control Sequences](https://vt100.net/docs/vt100-ug/chapter3.html#S3.3.5)
* [VT100 ANSI/VT52 Mode (DECANM)](https://vt100.net/docs/vt100-ug/chapter3.html#DECANM)
* [VT100 Index Sequence (IND)](https://vt100.net/docs/vt100-ug/chapter3.html#IND)
* [VTTEST Test Utility](https://invisible-island.net/vttest/)
* [DEC STD 070 Video Systems Reference Manual](https://archive.org/details/bitsavers_decstandar0VideoSystemsReferenceManualDec91_74264381)