Write the entire exporting & logging spec
This commit is contained in:
parent
717ea85c9f
commit
b0006f4af2
|
@ -0,0 +1,380 @@
|
||||||
|
---
|
||||||
|
author: Mike Griese @zadjii-msft
|
||||||
|
created on: 2021-08-31
|
||||||
|
last updated: 2021-08-31
|
||||||
|
issue id: #642
|
||||||
|
---
|
||||||
|
|
||||||
|
# Buffer Exporting and Logging
|
||||||
|
|
||||||
|
## Abstract
|
||||||
|
|
||||||
|
A common user need is the ability to export the history of a terminal session to
|
||||||
|
a file, for later inspection or validation. This is something that could be
|
||||||
|
triggered manually. Many terminal emulators provide the ability to automatically
|
||||||
|
log the output of a session to a file, so the history is always captured. This
|
||||||
|
spec will address improvements to the Windows Terminal to enable these kinds of
|
||||||
|
exporting and logging scenarios.
|
||||||
|
|
||||||
|
## Background
|
||||||
|
|
||||||
|
### Inspiration
|
||||||
|
|
||||||
|
Below are screenshots from the settings pages of three different terminal
|
||||||
|
emulators with similar features - PuTTY, SecureCRT, and ConEmu:
|
||||||
|
|
||||||
|
![PuTTY settings](PuTTY-logging-settings.png)
|
||||||
|
|
||||||
|
_figure 1: PuTTY settings_
|
||||||
|
|
||||||
|
![SecureCRT settings](SecureCRT-logging-settings.png)
|
||||||
|
|
||||||
|
_figure 2: SecureCRT settings_
|
||||||
|
|
||||||
|
![ConEmu settings](ConEmu-logging-settings.png)
|
||||||
|
|
||||||
|
_figure 3: ConEmu settings_
|
||||||
|
|
||||||
|
These applications all offer some settings in common. Primarily, the important
|
||||||
|
feature is the ability to specify a path to a log file which contains some
|
||||||
|
special string formatting. This allows the user to log to different files based
|
||||||
|
on the time & date of the session, or based on the session name.
|
||||||
|
|
||||||
|
### User Stories
|
||||||
|
|
||||||
|
* **Story A**: The user is able to use a context menu entry on the tab to export
|
||||||
|
the contents of the buffer to a file, which they are prompted for.
|
||||||
|
- This is explicitly what was requested in [#642]
|
||||||
|
* **Story B**: The user can bind an action to export the contents of the buffer
|
||||||
|
to a file, which they are prompted for.
|
||||||
|
- Very similar to **A**, but via the command palette or a keybinding.
|
||||||
|
* **Story C**: The user can export to an explicit file via an action
|
||||||
|
- similar to **B**, but allowing for declaring the path to a file rather than
|
||||||
|
prompting at runtime.
|
||||||
|
* **Story D**: The user can choose to append to a file when exporting, rather
|
||||||
|
than overwritting.
|
||||||
|
* **Story E**: The user can specify a format string in the path to the file to
|
||||||
|
export to, which the Terminal will automatically replace with variables like
|
||||||
|
the time, date, and profile name.
|
||||||
|
* **Story F**: When opening a specific profile, the user can automatically log
|
||||||
|
to a file
|
||||||
|
* **Story G**: The user can execute an action to start or stop logging to a
|
||||||
|
given file.
|
||||||
|
|
||||||
|
|
||||||
|
## Solution Design
|
||||||
|
|
||||||
|
I'm proposing the following actions and profile settings
|
||||||
|
|
||||||
|
* New Action: `exportBuffer()`.
|
||||||
|
- Export the contents of the buffer to a file.
|
||||||
|
- `path` (string, defaults to `""`): When empty, prompt the user for a name of
|
||||||
|
a file to export to, using a file picker. This path accepts special
|
||||||
|
formatting strings that will be substituted with certain variables
|
||||||
|
(discussed [below](#path-formatting)).
|
||||||
|
- `append` (boolean, defaults to `false`): When `false`, the file's contents
|
||||||
|
will be overwritten. When `true`, the buffer contents will be appended to
|
||||||
|
the end of the file.
|
||||||
|
* New Profile Settings object: `logSettings`
|
||||||
|
- This is an object that describes a set of behavior for logging a profile.
|
||||||
|
- `path`: Same as the `path` in the `ExportBufferArgs` above
|
||||||
|
- `append`: Same as the `append` in the `ExportBufferArgs` above
|
||||||
|
- `captureAllOutput`: (boolean, defaults to `false`) When true, don't log only
|
||||||
|
printable characters, also log non-printable escape characters written to
|
||||||
|
the Terminal.
|
||||||
|
- `captureInput`: (boolean, defaults to `false`) Additionally log input to the
|
||||||
|
Terminal to the file. Input will be formatted as the traditional VT
|
||||||
|
sequences, rather than the full `win32-input` encoding.
|
||||||
|
- `newFileEveryDay`: (boolean, defaults to `false`) This requires the `day` to
|
||||||
|
be an element of the path format string. When logging with this setting,
|
||||||
|
opens a new file at midnight and starts writing that one.
|
||||||
|
<!-- TODO! - `flushFrequently`: (boolean, defaults to `true`) -->
|
||||||
|
* New Profile setting: `logAutomatically` (boolean, default `false`). When true,
|
||||||
|
terminals with this profile will begin logging automatically.
|
||||||
|
* New Action: `toggleLogging()`.
|
||||||
|
- Start or stop logging to the configured file. If the terminal is already
|
||||||
|
logging with different settings than in this action, then stop logging
|
||||||
|
regardless (don't just start logging to the new file)
|
||||||
|
- This action accepts all the same args the profile's `logSettings` object.
|
||||||
|
- If _any_ args are provided, use those args. If _none_ are provided, then use
|
||||||
|
the logging settings present in the profile (if there are any).
|
||||||
|
- If there's not path provided (either in the args to the action or in the
|
||||||
|
profile), prompt the user to pick a file to log to.
|
||||||
|
|
||||||
|
### Examples
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"actions": [
|
||||||
|
{ "keys": "f1", "command": "exportBuffer" },
|
||||||
|
{ "keys": "f2", "command": { "action": "exportBuffer", "path": "c:\\logs\\${year}-${month}-${date}\\{profile}.txt" } },
|
||||||
|
|
||||||
|
{ "keys": "f3", "command": "toggleLogging" },
|
||||||
|
{ "keys": "f4", "command": { "action": "toggleLogging", "path": "c:\\logs\\${profile}.log", "append": true } },
|
||||||
|
],
|
||||||
|
"profiles": [
|
||||||
|
{
|
||||||
|
"name": "foo",
|
||||||
|
"logging": {
|
||||||
|
"path": "c:\\foo.txt",
|
||||||
|
"append": true
|
||||||
|
},
|
||||||
|
"automaticallyLog": false
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "bar",
|
||||||
|
"logging": {
|
||||||
|
"path": "c:\\logs\\${date}\\bar.txt",
|
||||||
|
"append": false
|
||||||
|
},
|
||||||
|
"automaticallyLog": true
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Revisiting our original stories:
|
||||||
|
|
||||||
|
* **Story A**: This is already implemented in [#11062]
|
||||||
|
* **Story B**: This is the action bound to <kbd>f1</kbd>.
|
||||||
|
* **Story C**: This is the action bound to <kbd>f2</kbd>.
|
||||||
|
* **Story D**: This is the `append` property in the actions, profile settings.
|
||||||
|
* **Story E**: An example of this is in the action bound to <kbd>f2</kbd>,
|
||||||
|
<kbd>f4</kbd>, and in the profile "bar"'s logging settings.
|
||||||
|
* **Story F**: The profile "bar" is configured to automatically log when opened.
|
||||||
|
* **Story G**: This is the action bound to <kbd>f4</kbd>.
|
||||||
|
|
||||||
|
In addition,
|
||||||
|
* When opening the profile "foo", it will not automatically log to a file.
|
||||||
|
- Pressing <kbd>f3</kbd> will begin logging to `c:\foo.txt`
|
||||||
|
- Pressing <kbd>f4</kbd> will begin logging to `c:\logs\foo.log`
|
||||||
|
|
||||||
|
### Path formatting
|
||||||
|
|
||||||
|
[TODO!]: # TODO!
|
||||||
|
|
||||||
|
For discussion: What syntax do we want?
|
||||||
|
* PuTTY uses `&Y`, `&M`, `&D`, `&T`, `&H`, `&P` for year, month, day, time, host
|
||||||
|
and port respectively.
|
||||||
|
* SecureCRT uses:
|
||||||
|
- `%H` – hostname
|
||||||
|
- `%S` – session name
|
||||||
|
- `%Y` – four-digit year
|
||||||
|
- `%M` – two-digit month
|
||||||
|
- `%D` – two-digit day of the month
|
||||||
|
- `%h` – two-digit hour
|
||||||
|
- `%m` – two-digit minute
|
||||||
|
- `%s` – two-digit seconds
|
||||||
|
- `%t` – three-digit milliseconds
|
||||||
|
- `%%` – percent (%)
|
||||||
|
- `%envvar%` – environment variable (for instance `%USERNAME%`)
|
||||||
|
|
||||||
|
We have some precedent for formatting with `${braces}`, a la the iterable
|
||||||
|
command in the Command Palette (e.g `${profile.name}`). Additionally, [#9287]
|
||||||
|
implements support for environment variables in the Terminal with the
|
||||||
|
`${env:VARIABLE}` syntax.
|
||||||
|
|
||||||
|
What variables do we want exposed, and how do we want users to be able to format
|
||||||
|
them?
|
||||||
|
|
||||||
|
This doc was initially authored assuming we'd go with a `${braces}` syntax, like:
|
||||||
|
- `${profile}` – profile name
|
||||||
|
- `${year}` – four-digit year
|
||||||
|
- `${month}` – two-digit month
|
||||||
|
- `${day}` – two-digit day of the month
|
||||||
|
- `${hour}` – two-digit hour
|
||||||
|
- `${minute}` – two-digit minute
|
||||||
|
- `${second}` – two-digit second
|
||||||
|
- `${ms}` – three-digit milliseconds
|
||||||
|
- `${env:variable}` – environment variable (for instance `${env:USERPROFILE}`)
|
||||||
|
(inspired by [#9287])
|
||||||
|
|
||||||
|
### Exporting vs Logging
|
||||||
|
As far as specific implementation details goes, exporting is the easier work to
|
||||||
|
do. [#11062] already wires up the `TerminalApp` to retrieve the buffer contents
|
||||||
|
from the `TermControl`, so writing them at request is easy.
|
||||||
|
|
||||||
|
Logging is harder. We don't want the `TermControl` telling the `TerminalApp`
|
||||||
|
layer about every piece of output logged. Especially in the post-[#5000] world
|
||||||
|
where that's a cross-process hop. Instead, we'll want the `ControlCore` /
|
||||||
|
`ControlInteractivity` to do _logging_ themselves.
|
||||||
|
|
||||||
|
### Logging Mechanics
|
||||||
|
|
||||||
|
#### When do we log?
|
||||||
|
|
||||||
|
[TODO!]: # TODO!
|
||||||
|
|
||||||
|
When do we decide to actually log? Take for example typing in a `pwsh` or
|
||||||
|
`bash` prompt. Imagine the user types
|
||||||
|
<kbd>a</kbd><kbd>s</kbd><kbd>d</kbd><kbd>f</kbd>, then hits
|
||||||
|
<kbd>Bksp</kbd><kbd>Bksp</kbd>, such that the prompt is just `as`. What should
|
||||||
|
the log contain? `asdf^h ^h^h ^h`<sup>[[1]](#footnote-1)</sup>? `as`?
|
||||||
|
|
||||||
|
My worry with logging the backspaces is that conpty is sometimes a bit noisier
|
||||||
|
than it needs to be with using `^H` as a cursor positioning sequence. Should we
|
||||||
|
only log lines when the cursor newlines or otherwise moves from the line it is
|
||||||
|
currently on?
|
||||||
|
|
||||||
|
I'll need to look at what PuTTY emits for the "Printable output" option.
|
||||||
|
|
||||||
|
#### What happens when we _start_ logging?
|
||||||
|
|
||||||
|
If the user has a terminal that did not start with logging enabled, but then
|
||||||
|
started logging with `toggleLogging`, what should we log? All future output? Or
|
||||||
|
should we log the current buffer contents as well?
|
||||||
|
|
||||||
|
I'm inclined to lean towards simply "all future output", and ignore any current
|
||||||
|
buffer content. If the user rally wants to log the current buffer contents _and_
|
||||||
|
start logging, they can use a `multipleActions` action ([#11045]) to
|
||||||
|
`exportBuffer` to a file, then `toggleLogging` to that same file with
|
||||||
|
`"append":true`.
|
||||||
|
|
||||||
|
## Potential Issues
|
||||||
|
|
||||||
|
<table>
|
||||||
|
|
||||||
|
<tr>
|
||||||
|
<td><strong>Compatibility</strong></td>
|
||||||
|
<td>
|
||||||
|
|
||||||
|
Since this functionality is entirely new, nothing here should negatively affect
|
||||||
|
existing functionality.
|
||||||
|
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td><strong>Performance, Power, and Efficiency</strong></td>
|
||||||
|
<td>
|
||||||
|
|
||||||
|
When logging, it's expected there will be a measurable performance hit. We can
|
||||||
|
try to mitigate this by only writing to the file on a background thread,
|
||||||
|
separate from the connection or rendering thread. Since auto-logging will only
|
||||||
|
take place in the content process, we're not worried about the file writing
|
||||||
|
occuring on the UI thread.
|
||||||
|
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
</table>
|
||||||
|
|
||||||
|
Also frequently requested is the ability to log timestamps of when commands are
|
||||||
|
executed. I don't think that this is a valuable feature for the Terminal to
|
||||||
|
implement ourselves. Windows Terminal is fundamentally just a _terminal
|
||||||
|
emulator_, it doesn't really know what's going on behind the scenes with
|
||||||
|
whatever client application (`cmd`, `powershell`, `bash`, `vim`) that is
|
||||||
|
connected to it. WT doesn't know when the user is typing in commands to the
|
||||||
|
shell, or if the user is just typing in text in `emacs` or something. There's no
|
||||||
|
way for the terminal to know that. It's _typically_ the client application's
|
||||||
|
responsibility to save it's own command history. `bash` and `powershell` both do
|
||||||
|
a pretty good job of saving this to another file to restore across sessions,
|
||||||
|
while `cmd.exe` doesn't.
|
||||||
|
|
||||||
|
Windows is a messy world and this model gets a little tricky here. `cmd.exe`
|
||||||
|
isn't actually managing it's own command history _at all_. `conhost` is doing
|
||||||
|
that work on behalf of the client applications. Some long time ago someone
|
||||||
|
thought it would be a good idea to have the `readline` functionality baked
|
||||||
|
directly into the console host. Whether that was a good idea or not remains to
|
||||||
|
be seen - it's certainly made things like `python.exe`'s REPL easier to
|
||||||
|
implement, since they don't need to maintain their own history buffer, but it
|
||||||
|
makes it hard to de-tangle behavior like this from the console itself.
|
||||||
|
|
||||||
|
I'm not sure how it would be possible to add a keybinding to the Windows
|
||||||
|
Terminal that would be able to save the console's _command_ history. Especially
|
||||||
|
considering the Terminal might _not_ be connected to a console host session at
|
||||||
|
all. If the Windows Terminal were directly running a `wsl` instance (something
|
||||||
|
that's not possible today, but something we've considered adding in the future),
|
||||||
|
then there wouldn't be a `conhost` in the process tree at all, and now
|
||||||
|
requesting the command history from the console wouldn't work _mysteriously_.
|
||||||
|
|
||||||
|
Furthermore, shells can always be configured to emit timestamps in their prompts
|
||||||
|
themselves. Since the Terminal has no knowledge of when a command is actually
|
||||||
|
entered, but the _shell_ does, it makes the most sense to configure the user's
|
||||||
|
_shell_ to emit that information. The Terminal will then dutifully log that
|
||||||
|
output along with everything else.
|
||||||
|
|
||||||
|
## Implementation Plan
|
||||||
|
|
||||||
|
Below is a rough outline of how I'd go about implementing these features. Each
|
||||||
|
lop-level checkbox could be its own PR, following from [#11062].
|
||||||
|
|
||||||
|
### Buffer exporting
|
||||||
|
|
||||||
|
* [ ] Add an `exportBuffer()` action that opens the file picker
|
||||||
|
* [ ] Add a string `path` parameter to `exportBuffer()` that allows the user to
|
||||||
|
press a key and immediately export the buffer to a whole path
|
||||||
|
- default to `""`, which indicates "open the file picker"
|
||||||
|
* [ ] add a boolean `append` (default to `false`) parameter to `exportBuffer`.
|
||||||
|
When true, export to the file given by appending, not overwriting the file
|
||||||
|
* [ ] Enable string formatting in the `path` parameter.
|
||||||
|
- What format do we want? `yyyy-mm-dd`? `%Y-%m-%D`? `&Y-&m-&D`? `${year}-${month}-${day}`?
|
||||||
|
- What are all the variables we want?
|
||||||
|
- Year, month, day, hour, minute - those are easy
|
||||||
|
- `WT_SESSION`, for a uuid for eash session maybe?
|
||||||
|
- Profile name perhaps? Commandline?
|
||||||
|
* [ ] more...
|
||||||
|
|
||||||
|
### Automatic logging
|
||||||
|
|
||||||
|
* [ ] `toggleLogging()` Action for start/stop logging, with `path`, `append`
|
||||||
|
properties (like `exportBuffer()`)
|
||||||
|
- `ToggleLoggingArgs` contains a single member `LoggingSettings`, which
|
||||||
|
contains `path` and `append` properties. This will make sense below.
|
||||||
|
* [ ] add `LoggingSettings` property for "log all output" (default would just be
|
||||||
|
"log printable output")
|
||||||
|
* [ ] add `LoggingSettings` property for "log input" (Though, we'd probably want
|
||||||
|
to log it as normal VT encoded, not as `win32-input` encoded)
|
||||||
|
* [ ] Per-profile setting for `logSettings`, which can contain an entire
|
||||||
|
`LoggingSettings` (like the `ToggleLoggingArgs`). When `toggleLogging` with no
|
||||||
|
args, try to use the profile's `loggingSettings` instead.
|
||||||
|
* [ ] Per-profile setting for `automaticallyLog`, which would log by default
|
||||||
|
when the profile is opened
|
||||||
|
* [ ] `LoggingSettings` property for "New file every day", which only works when
|
||||||
|
the `{day}` is in the path string. When auto-logging with this setting, opens
|
||||||
|
a new file at midnight and starts writing that one.
|
||||||
|
<!-- * [ ] `LoggingSettings` property for "Flush log frequently", defaults to
|
||||||
|
`true`(?). This causes us to flush all output to the file, instead of just...
|
||||||
|
on close? on newline? It's unclear exactly when PuTTY flushes with this off.
|
||||||
|
Need more coffee. -->
|
||||||
|
|
||||||
|
### Future Considerations
|
||||||
|
|
||||||
|
* When logging begins, the Terminal could display a toast for "Logging to
|
||||||
|
{filename}", and a similar one for "Stopped logging to {filename}".
|
||||||
|
* There's no good way of displaying a UI element to indicate that a pane is
|
||||||
|
currently logging to a file. I don't believe PuTTY displays any sort of
|
||||||
|
indicator. SecureCRT only displays a checkbox within the context menus of the
|
||||||
|
application itself.
|
||||||
|
![securecrt context menu](securecrt-context-menu.png)
|
||||||
|
|
||||||
|
Maybe when logging to a file, we could replace the "Export Text" context menu
|
||||||
|
entry with "Stop Logging"
|
||||||
|
* We could maybe add a setting to disable logging from the alt buffer. This
|
||||||
|
might help make this setting more valuable for users who are using full-screen
|
||||||
|
applications like `vim`. Since those applications redraw the entire viewport
|
||||||
|
contents frequently, the log might be unnecessarily noisy. Disabling logging
|
||||||
|
while in the alt buffer would show that the user opened vim, and then they did
|
||||||
|
some things after vim exited.
|
||||||
|
* Logging all output will be VERY helpful to us in the future for trying to
|
||||||
|
recreate bugs on our end that users can repro but we can't!
|
||||||
|
|
||||||
|
## Resources
|
||||||
|
|
||||||
|
PuTTY Logging documentation: https://tartarus.org/~simon/putty-snapshots/htmldoc/Chapter4.html#config-logfilename
|
||||||
|
ConEmu Logging documentation: https://conemu.github.io/en/AnsiLogFiles.html
|
||||||
|
|
||||||
|
### Footnotes
|
||||||
|
|
||||||
|
<a name="footnote-1"><a>[1]: Remember that `^H` is non-desructive, so the
|
||||||
|
sequence `asdf^h ^h^h ^h` is can be read as:
|
||||||
|
* print "asdf"
|
||||||
|
* move the cursor back one
|
||||||
|
* print a space (overwriting 'f')
|
||||||
|
* move the cursor back one (now it's on the space where 'f' was)
|
||||||
|
* move the cursor back one
|
||||||
|
* print a space (overwriting 'd')
|
||||||
|
* move the cursor back one (now it's on the space where 'd' was)
|
||||||
|
|
||||||
|
[#642]: https://github.com/microsoft/terminal/issues/642
|
||||||
|
[#5000]: https://github.com/microsoft/terminal/issues/5000
|
||||||
|
[#9287]: https://github.com/microsoft/terminal/pull/9287
|
||||||
|
[#11045]: https://github.com/microsoft/terminal/pull/11045
|
||||||
|
[#11062]: https://github.com/microsoft/terminal/pull/11062
|
Binary file not shown.
After Width: | Height: | Size: 72 KiB |
Binary file not shown.
After Width: | Height: | Size: 14 KiB |
Binary file not shown.
After Width: | Height: | Size: 58 KiB |
Binary file not shown.
After Width: | Height: | Size: 42 KiB |
Loading…
Reference in a new issue