maxmustermann
/
terminal
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Introduce til::rle - a run length encoded vector (#10099) 

## Summary of the Pull Request

Introduces `til::rle`, a vector-like container which stores elements of
type T in a run length encoded format. This allows efficient compaction
of repeated elements within the vector.

## References

* #8000 - Supports buffer rewrite work. A re-use of `til::rle` will be
  useful as a column counter as we pursue NxM storage and presentation.
* #3075 - The new iterators allow skipping forward by multiple units,
  which wasn't possible under `TextBuffer-/OutputCellIterator`.
  Additionally it also allows a bulk insertions.
* #8787 and #410 - High probability this should be `pmr`-ified
  like `bitmap` for things like `chafa` and `cacafire`
  which are changing the run length frequently.

## PR Checklist

* [x] Closes #8741
* [x] I work here.
* [x] Tests added.
* [x] Tests passed.

## Validation Steps Performed

* [x] Ran `cacafire` in `OpenConsole.exe` and it looked beautiful
* [x] Ran new suite of `RunLengthEncodingTests.cpp`

Co-authored-by: Michael Niksa <miniksa@microsoft.com>
This commit is contained in:
Leonard Hecker 2021-05-20 19:27:50 +02:00 committed by GitHub
parent eaeab7a807
commit a8e4bedae3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
25 changed files with 1860 additions and 1691 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.github/actions/spelling/allow/allow.txt vendored
View File

@ -1,6 +1,7 @@
Apc
apc
clickable
copyable
dalet
Dcs
dcs

565
src/buffer/out/AttrRow.cpp
View File

@ -11,18 +11,8 @@
// - attr - the default text attribute
// Return Value:
// - constructed object
ATTR_ROW::ATTR_ROW(const UINT cchRowWidth, const TextAttribute attr) noexcept
{
try
{
_list.emplace_back(TextAttributeRun(cchRowWidth, attr));
}
catch (...)
{
FAIL_FAST_CAUGHT_EXCEPTION();
}
_cchRowWidth = cchRowWidth;
}
ATTR_ROW::ATTR_ROW(const uint16_t width, const TextAttribute attr) :
_data(width, attr) {}
// Routine Description:
// - Sets all properties of the ATTR_ROW to default values
@ -30,8 +20,7 @@ ATTR_ROW::ATTR_ROW(const UINT cchRowWidth, const TextAttribute attr) noexcept
// - attr - The default text attributes to use on text in this row.
void ATTR_ROW::Reset(const TextAttribute attr)
{
_list.clear();
_list.emplace_back(TextAttributeRun(_cchRowWidth, attr));
_data.replace(0, _data.size(), attr);
}
// Routine Description:
@ -43,48 +32,9 @@ void ATTR_ROW::Reset(const TextAttribute attr)
// - newWidth - The new width of the row.
// Return Value:
// - <none>, throws exceptions on failures.
void ATTR_ROW::Resize(const size_t newWidth)
void ATTR_ROW::Resize(const uint16_t newWidth)
{
THROW_HR_IF(E_INVALIDARG, 0 == newWidth);
// Easy case. If the new row is longer, increase the length of the last run by how much new space there is.
if (newWidth > _cchRowWidth)
{
// Get the attribute that covers the final column of old width.
const auto runPos = FindAttrIndex(_cchRowWidth - 1, nullptr);
auto& run = _list.at(runPos);
// Extend its length by the additional columns we're adding.
run.SetLength(run.GetLength() + newWidth - _cchRowWidth);
// Store that the new total width we represent is the new width.
_cchRowWidth = newWidth;
}
// harder case: new row is shorter.
else
{
// Get the attribute that covers the final column of the new width
size_t CountOfAttr = 0;
const auto runPos = FindAttrIndex(newWidth - 1, &CountOfAttr);
auto& run = _list.at(runPos);
// CountOfAttr was given to us as "how many columns left from this point forward are covered by the returned run"
// So if the original run was B5 covering a 5 size OldWidth and we have a NewWidth of 3
// then when we called FindAttrIndex, it returned the B5 as the pIndexedRun and a 2 for how many more segments it covers
// after and including the 3rd column.
// B5-2 = B3, which is what we desire to cover the new 3 size buffer.
run.SetLength(run.GetLength() - CountOfAttr + 1);
// Store that the new total width we represent is the new width.
_cchRowWidth = newWidth;
// Erase segments after the one we just updated.
_list.erase(_list.cbegin() + runPos + 1, _list.cend());
// NOTE: Under some circumstances here, we have leftover run segments in memory or blank run segments
// in memory. We're not going to waste time redimensioning the array in the heap. We're just noting that the useful
// portions of it have changed.
}
_data.resize_trailing_extent(newWidth);
}
// Routine Description:
@ -95,104 +45,23 @@ void ATTR_ROW::Resize(const size_t newWidth)
// - the text attribute at column
// Note:
// - will throw on error
TextAttribute ATTR_ROW::GetAttrByColumn(const size_t column) const
TextAttribute ATTR_ROW::GetAttrByColumn(const uint16_t column) const
{
return GetAttrByColumn(column, nullptr);
}
// Routine Description:
// - returns a copy of the TextAttribute at the specified column
// Arguments:
// - column - the column to get the attribute for
// - pApplies - if given, fills how long this attribute will apply for
// Return Value:
// - the text attribute at column
// Note:
// - will throw on error
TextAttribute ATTR_ROW::GetAttrByColumn(const size_t column,
size_t* const pApplies) const
{
THROW_HR_IF(E_INVALIDARG, column >= _cchRowWidth);
const auto runPos = FindAttrIndex(column, pApplies);
return _list.at(runPos).GetAttributes();
}
// Routine Description:
// - reports how many runs we have stored (to be used for some optimizations
// Return Value:
// - Count of runs. 1 means we have 1 color to represent the entire row.
size_t ATTR_ROW::GetNumberOfRuns() const noexcept
{
return _list.size();
}
// Routine Description:
// - This routine finds the nth attribute in this ATTR_ROW.
// Arguments:
// - index - which attribute to find
// - applies - on output, contains corrected length of indexed attr.
// for example, if the attribute string was { 5, BLUE } and the requested
// index was 3, CountOfAttr would be 2.
// Return Value:
// - const reference to attribute run object
size_t ATTR_ROW::FindAttrIndex(const size_t index, size_t* const pApplies) const
{
FAIL_FAST_IF(!(index < _cchRowWidth)); // The requested index cannot be longer than the total length described by this set of Attrs.
size_t cTotalLength = 0;
FAIL_FAST_IF(!(_list.size() > 0)); // There should be a non-zero and positive number of items in the array.
// Scan through the internal array from position 0 adding up the lengths that each attribute applies to
auto runPos = _list.cbegin();
do
{
cTotalLength += runPos->GetLength();
if (cTotalLength > index)
{
// If we've just passed up the requested index with the length we added, break early
break;
}
runPos++;
} while (runPos < _list.cend());
// we should have broken before falling out the while case.
// if we didn't break, then this ATTR_ROW wasn't filled with enough attributes for the entire row of characters
FAIL_FAST_IF(runPos >= _list.cend());
// The remaining iterator position is the position of the attribute that is applicable at the position requested (index)
// Calculate its remaining applicability if requested
// The length on which the found attribute applies is the total length seen so far minus the index we were searching for.
FAIL_FAST_IF(!(cTotalLength > index)); // The length of all attributes we counted up so far should be longer than the index requested or we'll underflow.
if (nullptr != pApplies)
{
const auto attrApplies = cTotalLength - index;
FAIL_FAST_IF(!(attrApplies > 0)); // An attribute applies for >0 characters
// MSFT: 17130145 - will restore this and add a better assert to catch the real issue.
//FAIL_FAST_IF(!(attrApplies <= _cchRowWidth)); // An attribute applies for a maximum of the total length available to us
*pApplies = attrApplies;
}
return runPos - _list.cbegin();
return _data.at(column);
}
// Routine Description:
// - Finds the hyperlink IDs present in this row and returns them
// Return value:
// - The hyperlink IDs present in this row
std::vector<uint16_t> ATTR_ROW::GetHyperlinks()
std::vector<uint16_t> ATTR_ROW::GetHyperlinks() const
{
std::vector<uint16_t> ids;
for (const auto& run : _list)
for (const auto& run : _data.runs())
{
if (run.GetAttributes().IsHyperlink())
if (run.value.IsHyperlink())
{
ids.emplace_back(run.GetAttributes().GetHyperlinkId());
ids.emplace_back(run.value.GetHyperlinkId());
}
}
return ids;
@ -205,12 +74,10 @@ std::vector<uint16_t> ATTR_ROW::GetHyperlinks()
// - attr - Attribute (color) to fill remaining characters with
// Return Value:
// - <none>
bool ATTR_ROW::SetAttrToEnd(const UINT iStart, const TextAttribute attr)
bool ATTR_ROW::SetAttrToEnd(const uint16_t beginIndex, const TextAttribute attr)
{
size_t const length = _cchRowWidth - iStart;
const TextAttributeRun run(length, attr);
return SUCCEEDED(InsertAttrRuns({ &run, 1 }, iStart, _cchRowWidth - 1, _cchRowWidth));
_data.replace(gsl::narrow<uint16_t>(beginIndex), _data.size(), attr);
return true;
}
// Method Description:
@ -221,419 +88,47 @@ bool ATTR_ROW::SetAttrToEnd(const UINT iStart, const TextAttribute attr)
// - replaceWith - the new value for the matching runs' attributes.
// Return Value:
// - <none>
void ATTR_ROW::ReplaceAttrs(const TextAttribute& toBeReplacedAttr, const TextAttribute& replaceWith) noexcept
void ATTR_ROW::ReplaceAttrs(const TextAttribute& toBeReplacedAttr, const TextAttribute& replaceWith)
{
for (auto& run : _list)
{
if (run.GetAttributes() == toBeReplacedAttr)
{
run.SetAttributes(replaceWith);
}
}
_data.replace_values(toBeReplacedAttr, replaceWith);
}
// Routine Description:
// - Takes a array of attribute runs, and inserts them into this row from startIndex to endIndex.
// - For example, if the current row was was [{4, BLUE}], the merge string
// was [{ 2, RED }], with (StartIndex, EndIndex) = (1, 2),
// then the row would modified to be = [{ 1, BLUE}, {2, RED}, {1, BLUE}].
// - Takes an attribute, and merges it into this row from beginIndex (inclusive) to endIndex (exclusive).
// - For example, if the current row was was [{4, BLUE}], the merge arguments were
// { beginIndex = 1, endIndex = 3, newAttr = RED }, then the row would modified to be
// [{ 1, BLUE}, {2, RED}, {1, BLUE}].
// Arguments:
// - rgInsertAttrs - The array of attrRuns to merge into this row.
// - cInsertAttrs - The number of elements in rgInsertAttrs
// - iStart - The index in the row to place the array of runs.
// - iEnd - the final index of the merge runs
// - BufferWidth - the width of the row.
// - beginIndex, endIndex: The [beginIndex, endIndex) range that's to be replaced with newAttr.
// - newAttr: The attribute to merge into this row.
// Return Value:
// - STATUS_NO_MEMORY if there wasn't enough memory to insert the runs
// otherwise STATUS_SUCCESS if we were successful.
[[nodiscard]] HRESULT ATTR_ROW::InsertAttrRuns(const gsl::span<const TextAttributeRun> newAttrs,
const size_t iStart,
const size_t iEnd,
const size_t cBufferWidth)
// - <none>
void ATTR_ROW::Replace(const uint16_t beginIndex, const uint16_t endIndex, const TextAttribute& newAttr)
{
// Definitions:
// Existing Run = The run length encoded color array we're already storing in memory before this was called.
// Insert Run = The run length encoded color array that someone is asking us to inject into our stored memory run.
// New Run = The run length encoded color array that we have to allocate and rebuild to store internally
// which will replace Existing Run at the end of this function.
// Example:
// cBufferWidth = 10.
// Existing Run: R3 -> G5 -> B2
// Insert Run: Y1 -> N1 at iStart = 5 and iEnd = 6
// (rgInsertAttrs is a 2 length array with Y1->N1 in it and cInsertAttrs = 2)
// Final Run: R3 -> G2 -> Y1 -> N1 -> G1 -> B2
// We'll need to know what the last valid column is for some calculations versus iEnd
// because iEnd is specified to us as an inclusive index value.
// Do the -1 math here now so we don't have to have -1s scattered all over this function.
const size_t iLastBufferCol = cBufferWidth - 1;
// If the insertion size is 1, do some pre-processing to
// see if we can get this done quickly.
if (newAttrs.size() == 1)
{
// Get the new color attribute we're trying to apply
const TextAttribute NewAttr = til::at(newAttrs, 0).GetAttributes();
// If the existing run was only 1 element...
// ...and the new color is the same as the old, we don't have to do anything and can exit quick.
if (_list.size() == 1 && _list.at(0).GetAttributes() == NewAttr)
{
return S_OK;
}
// .. otherwise if we internally have a list of 2 or more and we're about to insert a single color
// it's possible that we just walk left-to-right through the row and find a quick exit.
else if (iStart >= 0 && iStart == iEnd)
{
// First we try to find the run where the insertion happens, using lowerBound and upperBound to track
// where we are currently at.
const auto begin = _list.begin();
size_t lowerBound = 0;
size_t upperBound = 0;
for (size_t i = 0; i < _list.size(); i++)
{
const auto curr = begin + i;
upperBound += curr->GetLength();
if (iStart >= lowerBound && iStart < upperBound)
{
// The run that we try to insert into has the same color as the new one.
// e.g.
// AAAAABBBBBBBCCC
// ^
// AAAAABBBBBBBCCC
//
// 'B' is the new color and '^' represents where iStart is. We don't have to
// do anything.
if (curr->GetAttributes() == NewAttr)
{
return S_OK;
}
// If the current run has length of exactly one, we can simply change the attribute
// of the current run.
// e.g.
// AAAAABCCCCCCCCC
// ^
// AAAAADCCCCCCCCC
//
// Here 'D' is the new color.
if (curr->GetLength() == 1)
{
curr->SetAttributes(NewAttr);
return S_OK;
}
// If the insertion happens at current run's lower boundary...
if (iStart == lowerBound && i > 0)
{
const auto prev = std::prev(curr, 1);
// ... and the previous run has the same color as the new one, we can
// just adjust the counts in the existing two elements in our internal list.
// e.g.
// AAAAABBBBBBBCCC
// ^
// AAAAAABBBBBBCCC
//
// Here 'A' is the new color.
if (NewAttr == prev->GetAttributes())
{
prev->IncrementLength();
curr->DecrementLength();
// If we just reduced the right half to zero, just erase it out of the list.
if (curr->GetLength() == 0)
{
_list.erase(curr);
}
return S_OK;
}
}
// If the insertion happens at current run's upper boundary...
if (iStart == upperBound - 1 && i + 1 < _list.size())
{
// ...then let's try our luck with the next run if possible. This is basically the opposite
// of what we did with the previous run.
// e.g.
// AAAAAABBBBBBCCC
// ^
// AAAAABBBBBBBCCC
//
// Here 'B' is the new color.
const auto next = std::next(curr, 1);
if (NewAttr == next->GetAttributes())
{
curr->DecrementLength();
next->IncrementLength();
if (curr->GetLength() == 0)
{
_list.erase(curr);
}
return S_OK;
}
}
}
// Advance one run in the _list.
lowerBound = upperBound;
// The lowerBound is larger than iStart, which means we fail to find an early exit at the run
// where the insertion happens. We can just break out.
if (lowerBound > iStart)
{
break;
}
}
}
}
// If we're about to cover the entire existing run with a new one, we can also make an optimization.
if (iStart == 0 && iEnd == iLastBufferCol)
{
// Just dump what we're given over what we have and call it a day.
_list.assign(newAttrs.begin(), newAttrs.end());
return S_OK;
}
// In the worst case scenario, we will need a new run that is the length of
// The existing run in memory + The new run in memory + 1.
// This worst case occurs when we inject a new item in the middle of an existing run like so
// Existing R3->B5->G2, Insertion Y2 starting at 5 (in the middle of the B5)
// becomes R3->B2->Y2->B1->G2.
// The original run was 3 long. The insertion run was 1 long. We need 1 more for the
// fact that an existing piece of the run was split in half (to hold the latter half).
const size_t cNewRun = _list.size() + newAttrs.size() + 1;
decltype(_list) newRun;
newRun.reserve(cNewRun);
// We will start analyzing from the beginning of our existing run.
// Use some pointers to keep track of where we are in walking through our runs.
// Get the existing run that we'll be updating/manipulating.
const auto existingRun = _list.begin();
auto pExistingRunPos = existingRun;
const auto pExistingRunEnd = _list.end();
auto pInsertRunPos = newAttrs.begin();
size_t cInsertRunRemaining = newAttrs.size();
size_t iExistingRunCoverage = 0;
// Copy the existing run into the new buffer up to the "start index" where the new run will be injected.
// If the new run starts at 0, we have nothing to copy from the beginning.
if (iStart != 0)
{
// While we're less than the desired insertion position...
while (iExistingRunCoverage < iStart)
{
// Add up how much length we can cover by copying an item from the existing run.
iExistingRunCoverage += pExistingRunPos->GetLength();
// Copy it to the new run buffer and advance both pointers.
newRun.push_back(*pExistingRunPos++);
}
// When we get to this point, we've copied full segments from the original existing run
// into our new run buffer. We will have 1 or more full segments of color attributes and
// we MIGHT have to cut the last copied segment's length back depending on where the inserted
// attributes will fall in the final/new run.
// Some examples:
// - Starting with the original string R3 -> G5 -> B2
// - 1. If the insertion is Y5 at start index 3
// We are trying to get a result/final/new run of R3 -> Y5 -> B2.
// We just copied R3 to the new destination buffer and we cang skip down and start inserting the new attrs.
// - 2. If the insertion is Y3 at start index 5
// We are trying to get a result/final/new run of R3 -> G2 -> Y3 -> B2.
// We just copied R3 -> G5 to the new destination buffer with the code above.
// But the insertion is going to cut out some of the length of the G5.
// We need to fix this up below so it says G2 instead to leave room for the Y3 to fit in
// the new/final run.
// Fetch out the length so we can fix it up based on the below conditions.
size_t length = newRun.back().GetLength();
// If we've covered more cells already than the start of the attributes to be inserted...
if (iExistingRunCoverage > iStart)
{
// ..then subtract some of the length of the final cell we copied.
// We want to take remove the difference in distance between the cells we've covered in the new
// run and the insertion point.
// (This turns G5 into G2 from Example 2 just above)
length -= (iExistingRunCoverage - iStart);
}
// Now we're still on that "last cell copied" into the new run.
// If the color of that existing copied cell matches the color of the first segment
// of the run we're about to insert, we can just increment the length to extend the coverage.
if (newRun.back().GetAttributes() == pInsertRunPos->GetAttributes())
{
length += pInsertRunPos->GetLength();
// Since the color matched, we have already "used up" part of the insert run
// and can skip it in our big "memcopy" step below that will copy the bulk of the insert run.
cInsertRunRemaining--;
pInsertRunPos++;
}
// We're done manipulating the length. Store it back.
newRun.back().SetLength(length);
}
// Bulk copy the majority (or all, depending on circumstance) of the insert run into the final run buffer.
std::copy_n(pInsertRunPos, cInsertRunRemaining, std::back_inserter(newRun));
// We're technically done with the insert run now and have 0 remaining, but won't bother updating its pointers
// and counts any further because we won't use them.
// Now we need to move our pointer for the original existing run forward and update our counts
// on how many cells we could have copied from the source before finishing off the new run.
while (iExistingRunCoverage <= iEnd)
{
FAIL_FAST_IF(!(pExistingRunPos != pExistingRunEnd));
iExistingRunCoverage += pExistingRunPos->GetLength();
pExistingRunPos++;
}
// If we still have original existing run cells remaining, copy them into the final new run.
if (pExistingRunPos != pExistingRunEnd || iExistingRunCoverage != (iEnd + 1))
{
// We advanced the existing run pointer and its count to on or past the end of what the insertion run filled in.
// If this ended up being past the end of what the insertion run covers, we have to account for the cells after
// the insertion run but before the next piece of the original existing run.
// The example in this case is if we had...
// Existing Run = R3 -> G5 -> B2 -> X5
// Insert Run = Y2 @ iStart = 7 and iEnd = 8
// ... then at this point in time, our states would look like...
// New Run so far = R3 -> G4 -> Y2
// Existing Run Pointer is at X5
// Existing run coverage count at 3 + 5 + 2 = 10.
// However, in order to get the final desired New Run
// (which is R3 -> G4 -> Y2 -> B1 -> X5)
// we would need to grab a piece of that B2 we already skipped past.
// iExistingRunCoverage = 10. iEnd = 8. iEnd+1 = 9. 10 > 9. So we skipped something.
if (iExistingRunCoverage > (iEnd + 1))
{
// Back up the existing run pointer so we can grab the piece we skipped.
pExistingRunPos--;
// If the color matches what's already in our run, just increment the count value.
// This case is slightly off from the example above. This case is for if the B2 above was actually Y2.
// That Y2 from the existing run is the same color as the Y2 we just filled a few columns left in the final run
// so we can just adjust the final run's column count instead of adding another segment here.
if (newRun.back().GetAttributes() == pExistingRunPos->GetAttributes())
{
size_t length = newRun.back().GetLength();
length += (iExistingRunCoverage - (iEnd + 1));
newRun.back().SetLength(length);
}
else
{
// If the color didn't match, then we just need to copy the piece we skipped and adjust
// its length for the discrepancy in columns not yet covered by the final/new run.
// Move forward to a blank spot in the new run
newRun.emplace_back();
// Copy the existing run's color information to the new run
newRun.back().SetAttributes(pExistingRunPos->GetAttributes());
// Adjust the length of that copied color to cover only the reduced number of columns needed
// now that some have been replaced by the insert run.
newRun.back().SetLength(iExistingRunCoverage - (iEnd + 1));
}
// Now that we're done recovering a piece of the existing run we skipped, move the pointer forward again.
pExistingRunPos++;
}
// OK. In this case, we didn't skip anything. The end of the insert run fell right at a boundary
// in columns that was in the original existing run.
// However, the next piece of the original existing run might happen to have the same color attribute
// as the final piece of what we just copied.
// As an example...
// Existing Run = R3 -> G5 -> B2.
// Insert Run = B5 @ iStart = 3 and iEnd = 7
// New Run so far = R3 -> B5
// New Run desired when done = R3 -> B7
// Existing run pointer is on B2.
// We want to merge the 2 from the B2 into the B5 so we get B7.
else if (newRun.back().GetAttributes() == pExistingRunPos->GetAttributes())
{
// Add the value from the existing run into the current new run position.
size_t length = newRun.back().GetLength();
length += pExistingRunPos->GetLength();
newRun.back().SetLength(length);
// Advance the existing run position since we consumed its value and merged it in.
pExistingRunPos++;
}
// Now bulk copy any segments left in the original existing run
if (pExistingRunPos < pExistingRunEnd)
{
std::copy_n(pExistingRunPos, (pExistingRunEnd - pExistingRunPos), std::back_inserter(newRun));
}
}
// OK, phew. We're done. Now we just need to free the existing run and store the new run in its place.
_list.swap(newRun);
return S_OK;
}
// Routine Description:
// - packs a vector of TextAttribute into a vector of TextAttributeRun
// Arguments:
// - attrs - text attributes to pack
// Return Value:
// - packed text attribute run
std::vector<TextAttributeRun> ATTR_ROW::PackAttrs(const std::vector<TextAttribute>& attrs)
{
std::vector<TextAttributeRun> runs;
if (attrs.empty())
{
return runs;
}
for (auto attr : attrs)
{
if (runs.empty() || runs.back().GetAttributes() != attr)
{
runs.emplace_back(TextAttributeRun(1, attr));
}
else
{
runs.back().SetLength(runs.back().GetLength() + 1);
}
}
return runs;
_data.replace(beginIndex, endIndex, newAttr);
}
ATTR_ROW::const_iterator ATTR_ROW::begin() const noexcept
{
return AttrRowIterator(this);
return _data.begin();
}
ATTR_ROW::const_iterator ATTR_ROW::end() const noexcept
{
return AttrRowIterator::CreateEndIterator(this);
return _data.end();
}
ATTR_ROW::const_iterator ATTR_ROW::cbegin() const noexcept
{
return AttrRowIterator(this);
return _data.cbegin();
}
ATTR_ROW::const_iterator ATTR_ROW::cend() const noexcept
{
return AttrRowIterator::CreateEndIterator(this);
return _data.cend();
}
bool operator==(const ATTR_ROW& a, const ATTR_ROW& b) noexcept
{
return (a._list.size() == b._list.size() &&
a._list.data() == b._list.data() &&
a._cchRowWidth == b._cchRowWidth);
return a._data == b._data;
}

45
src/buffer/out/AttrRow.hpp
View File

@ -20,16 +20,17 @@ Revision History:
#pragma once
#include "TextAttributeRun.hpp"
#include "AttrRowIterator.hpp"
#include "til/rle.h"
#include "TextAttribute.hpp"
class ATTR_ROW final
{
public:
using const_iterator = typename AttrRowIterator;
using rle_vector = til::small_rle<TextAttribute, uint16_t, 1>;
ATTR_ROW(const UINT cchRowWidth, const TextAttribute attr)
noexcept;
public:
using const_iterator = rle_vector::const_iterator;
ATTR_ROW(uint16_t width, TextAttribute attr);
~ATTR_ROW() = default;
@ -39,28 +40,13 @@ public:
noexcept = default;
ATTR_ROW& operator=(ATTR_ROW&&) noexcept = default;
TextAttribute GetAttrByColumn(const size_t column) const;
TextAttribute GetAttrByColumn(const size_t column,
size_t* const pApplies) const;
TextAttribute GetAttrByColumn(uint16_t column) const;
std::vector<uint16_t> GetHyperlinks() const;
size_t GetNumberOfRuns() const noexcept;
size_t FindAttrIndex(const size_t index,
size_t* const pApplies) const;
std::vector<uint16_t> GetHyperlinks();
bool SetAttrToEnd(const UINT iStart, const TextAttribute attr);
void ReplaceAttrs(const TextAttribute& toBeReplacedAttr, const TextAttribute& replaceWith) noexcept;
void Resize(const size_t newWidth);
[[nodiscard]] HRESULT InsertAttrRuns(const gsl::span<const TextAttributeRun> newAttrs,
const size_t iStart,
const size_t iEnd,
const size_t cBufferWidth);
static std::vector<TextAttributeRun> PackAttrs(const std::vector<TextAttribute>& attrs);
bool SetAttrToEnd(uint16_t beginIndex, TextAttribute attr);
void ReplaceAttrs(const TextAttribute& toBeReplacedAttr, const TextAttribute& replaceWith);
void Resize(uint16_t newWidth);
void Replace(uint16_t beginIndex, uint16_t endIndex, const TextAttribute& newAttr);
const_iterator begin() const noexcept;
const_iterator end() const noexcept;
@ -69,17 +55,14 @@ public:
const_iterator cend() const noexcept;
friend bool operator==(const ATTR_ROW& a, const ATTR_ROW& b) noexcept;
friend class AttrRowIterator;
friend class ROW;
private:
void Reset(const TextAttribute attr);
boost::container::small_vector<TextAttributeRun, 1> _list;
size_t _cchRowWidth;
rle_vector _data;
#ifdef UNIT_TESTING
friend class AttrRowTests;
friend class CommonState;
#endif
};

136
src/buffer/out/AttrRowIterator.cpp
View File

@ -1,136 +0,0 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "AttrRowIterator.hpp"
#include "AttrRow.hpp"
AttrRowIterator AttrRowIterator::CreateEndIterator(const ATTR_ROW* const attrRow) noexcept
{
AttrRowIterator it{ attrRow };
it._setToEnd();
return it;
}
AttrRowIterator::AttrRowIterator(const ATTR_ROW* const attrRow) noexcept :
_pAttrRow{ attrRow },
_run{ attrRow->_list.cbegin() },
_currentAttributeIndex{ 0 },
_exceeded{ false }
{
}
AttrRowIterator::operator bool() const noexcept
{
return !_exceeded && _run < _pAttrRow->_list.cend();
}
bool AttrRowIterator::operator==(const AttrRowIterator& it) const noexcept
{
return (_pAttrRow == it._pAttrRow &&
_run == it._run &&
_currentAttributeIndex == it._currentAttributeIndex &&
_exceeded == it._exceeded);
}
bool AttrRowIterator::operator!=(const AttrRowIterator& it) const noexcept
{
return !(*this == it);
}
AttrRowIterator& AttrRowIterator::operator+=(const ptrdiff_t& movement)
{
if (!_exceeded)
{
if (movement >= 0)
{
_increment(gsl::narrow<size_t>(movement));
}
else
{
_decrement(gsl::narrow<size_t>(-movement));
}
}
return *this;
}
AttrRowIterator& AttrRowIterator::operator-=(const ptrdiff_t& movement)
{
return this->operator+=(-movement);
}
const TextAttribute* AttrRowIterator::operator->() const
{
THROW_HR_IF(E_BOUNDS, _exceeded);
return &_run->GetAttributes();
}
const TextAttribute& AttrRowIterator::operator*() const
{
THROW_HR_IF(E_BOUNDS, _exceeded);
return _run->GetAttributes();
}
// Routine Description:
// - increments the index the iterator points to
// Arguments:
// - count - the amount to increment by
void AttrRowIterator::_increment(size_t count) noexcept
{
while (count > 0)
{
const size_t runLength = _run->GetLength();
if (count + _currentAttributeIndex < runLength)
{
_currentAttributeIndex += count;
return;
}
else
{
count -= runLength - _currentAttributeIndex;
++_run;
_currentAttributeIndex = 0;
}
}
}
// Routine Description:
// - decrements the index the iterator points to
// Arguments:
// - count - the amount to decrement by
void AttrRowIterator::_decrement(size_t count) noexcept
{
while (count > 0)
{
// If there's still space within this color attribute to move left, do so.
if (count <= _currentAttributeIndex)
{
_currentAttributeIndex -= count;
return;
}
// If there's not space, move to the previous attribute run
// We'll walk through above on the if branch to move left further (if necessary)
else
{
// make sure we don't go out of bounds
if (_run == _pAttrRow->_list.cbegin())
{
_exceeded = true;
return;
}
count -= _currentAttributeIndex + 1;
--_run;
_currentAttributeIndex = _run->GetLength() - 1;
}
}
}
// Routine Description:
// - sets fields on the iterator to describe the end() state of the ATTR_ROW
void AttrRowIterator::_setToEnd() noexcept
{
_run = _pAttrRow->_list.cend();
_currentAttributeIndex = 0;
}

80
src/buffer/out/AttrRowIterator.hpp
View File

@ -1,80 +0,0 @@
/*++
Copyright (c) Microsoft Corporation
Licensed under the MIT license.
Module Name:
- AttrRowIterator.hpp
Abstract:
- iterator for ATTR_ROW to walk the TextAttributes of the run
- read only iterator
Author(s):
- Austin Diviness (AustDi) 04-Jun-2018
--*/
#pragma once
#include "TextAttribute.hpp"
#include "TextAttributeRun.hpp"
class ATTR_ROW;
class AttrRowIterator final
{
public:
using iterator_category = std::bidirectional_iterator_tag;
using value_type = TextAttribute;
using difference_type = std::ptrdiff_t;
using pointer = TextAttribute*;
using reference = TextAttribute&;
static AttrRowIterator CreateEndIterator(const ATTR_ROW* const attrRow) noexcept;
AttrRowIterator(const ATTR_ROW* const attrRow) noexcept;
operator bool() const noexcept;
bool operator==(const AttrRowIterator& it) const noexcept;
bool operator!=(const AttrRowIterator& it) const noexcept;
AttrRowIterator& operator++() noexcept
{
_increment(1);
return *this;
}
AttrRowIterator operator++(int) noexcept
{
auto copy = *this;
_increment(1);
return copy;
}
AttrRowIterator& operator+=(const ptrdiff_t& movement);
AttrRowIterator& operator-=(const ptrdiff_t& movement);
AttrRowIterator& operator--() noexcept
{
_decrement(1);
return *this;
}
AttrRowIterator operator--(int) noexcept
{
auto copy = *this;
_decrement(1);
return copy;
}
const TextAttribute* operator->() const;
const TextAttribute& operator*() const;
private:
boost::container::small_vector_base<TextAttributeRun>::const_iterator _run;
const ATTR_ROW* _pAttrRow;
size_t _currentAttributeIndex; // index of TextAttribute within the current TextAttributeRun
bool _exceeded;
void _increment(size_t count) noexcept;
void _decrement(size_t count) noexcept;
void _setToEnd() noexcept;
};

154
src/buffer/out/Row.cpp
View File

@ -16,7 +16,7 @@
// - pParent - the text buffer that this row belongs to
// Return Value:
// - constructed object
ROW::ROW(const SHORT rowId, const unsigned short rowWidth, const TextAttribute fillAttribute, TextBuffer* const pParent) noexcept :
ROW::ROW(const SHORT rowId, const unsigned short rowWidth, const TextAttribute fillAttribute, TextBuffer* const pParent) :
_id{ rowId },
_rowWidth{ rowWidth },
_charRow{ rowWidth, this },
@ -107,103 +107,91 @@ OutputCellIterator ROW::WriteCells(OutputCellIterator it, const size_t index, co
{
THROW_HR_IF(E_INVALIDARG, index >= _charRow.size());
THROW_HR_IF(E_INVALIDARG, limitRight.value_or(0) >= _charRow.size());
size_t currentIndex = index;
// If we're given a right-side column limit, use it. Otherwise, the write limit is the final column index available in the char row.
const auto finalColumnInRow = limitRight.value_or(_charRow.size() - 1);
if (it)
auto currentColor = it->TextAttr();
uint16_t colorUses = 0;
uint16_t colorStarts = gsl::narrow_cast<uint16_t>(index);
uint16_t currentIndex = colorStarts;
while (it && currentIndex <= finalColumnInRow)
{
// Accumulate usages of the same color so we can spend less time in InsertAttrRuns rewriting it.
auto currentColor = it->TextAttr();
size_t colorUses = 0;
size_t colorStarts = index;
while (it && currentIndex <= finalColumnInRow)
// Fill the color if the behavior isn't set to keeping the current color.
if (it->TextAttrBehavior() != TextAttributeBehavior::Current)
{
// Fill the color if the behavior isn't set to keeping the current color.
if (it->TextAttrBehavior() != TextAttributeBehavior::Current)
// If the color of this cell is the same as the run we're currently on,
// just increment the counter.
if (currentColor == it->TextAttr())
{
// If the color of this cell is the same as the run we're currently on,
// just increment the counter.
if (currentColor == it->TextAttr())
{
++colorUses;
}
else
{
// Otherwise, commit this color into the run and save off the new one.
const TextAttributeRun run{ colorUses, currentColor };
// Now commit the new color runs into the attr row.
LOG_IF_FAILED(_attrRow.InsertAttrRuns({ &run, 1 },
colorStarts,
currentIndex - 1,
_charRow.size()));
currentColor = it->TextAttr();
colorUses = 1;
colorStarts = currentIndex;
}
}
// Fill the text if the behavior isn't set to saying there's only a color stored in this iterator.
if (it->TextAttrBehavior() != TextAttributeBehavior::StoredOnly)
{
const bool fillingLastColumn = currentIndex == finalColumnInRow;
// TODO: MSFT: 19452170 - We need to ensure when writing any trailing byte that the one to the left
// is a matching leading byte. Likewise, if we're writing a leading byte, we need to make sure we still have space in this loop
// for the trailing byte coming up before writing it.
// If we're trying to fill the first cell with a trailing byte, pad it out instead by clearing it.
// Don't increment iterator. We'll advance the index and try again with this value on the next round through the loop.
if (currentIndex == 0 && it->DbcsAttr().IsTrailing())
{
_charRow.ClearCell(currentIndex);
}
// If we're trying to fill the last cell with a leading byte, pad it out instead by clearing it.
// Don't increment iterator. We'll exit because we couldn't write a lead at the end of a line.
else if (fillingLastColumn && it->DbcsAttr().IsLeading())
{
_charRow.ClearCell(currentIndex);
SetDoubleBytePadded(true);
}
// Otherwise, copy the data given and increment the iterator.
else
{
_charRow.DbcsAttrAt(currentIndex) = it->DbcsAttr();
_charRow.GlyphAt(currentIndex) = it->Chars();
++it;
}
// If we're asked to (un)set the wrap status and we just filled the last column with some text...
// NOTE:
// - wrap = std::nullopt --> don't change the wrap value
// - wrap = true --> we're filling cells as a steam, consider this a wrap
// - wrap = false --> we're filling cells as a block, unwrap
if (wrap.has_value() && fillingLastColumn)
{
// set wrap status on the row to parameter's value.
SetWrapForced(*wrap);
}
++colorUses;
}
else
{
// Otherwise, commit this color into the run and save off the new one.
// Now commit the new color runs into the attr row.
_attrRow.Replace(colorStarts, currentIndex, currentColor);
currentColor = it->TextAttr();
colorUses = 1;
colorStarts = currentIndex;
}
}
// Fill the text if the behavior isn't set to saying there's only a color stored in this iterator.
if (it->TextAttrBehavior() != TextAttributeBehavior::StoredOnly)
{
const bool fillingLastColumn = currentIndex == finalColumnInRow;
// TODO: MSFT: 19452170 - We need to ensure when writing any trailing byte that the one to the left
// is a matching leading byte. Likewise, if we're writing a leading byte, we need to make sure we still have space in this loop
// for the trailing byte coming up before writing it.
// If we're trying to fill the first cell with a trailing byte, pad it out instead by clearing it.
// Don't increment iterator. We'll advance the index and try again with this value on the next round through the loop.
if (currentIndex == 0 && it->DbcsAttr().IsTrailing())
{
_charRow.ClearCell(currentIndex);
}
// If we're trying to fill the last cell with a leading byte, pad it out instead by clearing it.
// Don't increment iterator. We'll exit because we couldn't write a lead at the end of a line.
else if (fillingLastColumn && it->DbcsAttr().IsLeading())
{
_charRow.ClearCell(currentIndex);
SetDoubleBytePadded(true);
}
// Otherwise, copy the data given and increment the iterator.
else
{
_charRow.DbcsAttrAt(currentIndex) = it->DbcsAttr();
_charRow.GlyphAt(currentIndex) = it->Chars();
++it;
}
// Move to the next cell for the next time through the loop.
++currentIndex;
// If we're asked to (un)set the wrap status and we just filled the last column with some text...
// NOTE:
// - wrap = std::nullopt --> don't change the wrap value
// - wrap = true --> we're filling cells as a steam, consider this a wrap
// - wrap = false --> we're filling cells as a block, unwrap
if (wrap.has_value() && fillingLastColumn)
{
// set wrap status on the row to parameter's value.
SetWrapForced(*wrap);
}
}
else
{
++it;
}
// Now commit the final color into the attr row
if (colorUses)
{
const TextAttributeRun run{ colorUses, currentColor };
LOG_IF_FAILED(_attrRow.InsertAttrRuns({ &run, 1 },
colorStarts,
currentIndex - 1,
_charRow.size()));
}
// Move to the next cell for the next time through the loop.
++currentIndex;
}
// Now commit the final color into the attr row
if (colorUses)
{
_attrRow.Replace(colorStarts, currentIndex, currentColor);
}
return it;

3
src/buffer/out/Row.hpp
View File

@ -32,8 +32,7 @@ class TextBuffer;
class ROW final
{
public:
ROW(const SHORT rowId, const unsigned short rowWidth, const TextAttribute fillAttribute, TextBuffer* const pParent)
noexcept;
ROW(const SHORT rowId, const unsigned short rowWidth, const TextAttribute fillAttribute, TextBuffer* const pParent);
size_t size() const noexcept { return _rowWidth; }

7
src/buffer/out/TextAttribute.cpp
View File

@ -5,6 +5,13 @@
#include "TextAttribute.hpp"
#include "../../inc/conattrs.hpp"
// Keeping TextColor compact helps us keeping TextAttribute compact,
// which in turn ensures that our buffer memory usage is low.
static_assert(sizeof(TextAttribute) == 14);
static_assert(alignof(TextAttribute) == 2);
// Ensure that we can memcpy() and memmove() the struct for performance.
static_assert(std::is_trivially_copyable_v<TextAttribute>);
BYTE TextAttribute::s_legacyDefaultForeground = 7;
BYTE TextAttribute::s_legacyDefaultBackground = 0;

20
src/buffer/out/TextAttribute.hpp
View File

@ -27,8 +27,6 @@ Revision History:
#include "WexTestClass.h"
#endif
#pragma pack(push, 1)
class TextAttribute final
{
public:
@ -176,12 +174,11 @@ private:
static BYTE s_legacyDefaultForeground;
static BYTE s_legacyDefaultBackground;
WORD _wAttrLegacy;
TextColor _foreground;
TextColor _background;
ExtendedAttributes _extendedAttrs;
uint16_t _hyperlinkId;
uint16_t _wAttrLegacy; // sizeof: 2, alignof: 2
uint16_t _hyperlinkId; // sizeof: 2, alignof: 2
TextColor _foreground; // sizeof: 4, alignof: 1
TextColor _background; // sizeof: 4, alignof: 1
ExtendedAttributes _extendedAttrs; // sizeof: 1, alignof: 1
#ifdef UNIT_TESTING
friend class TextBufferTests;
@ -191,13 +188,6 @@ private:
#endif
};
#pragma pack(pop)
// 2 for _wAttrLegacy
// 4 for _foreground
// 4 for _background
// 1 for _extendedAttrs
static_assert(sizeof(TextAttribute) <= 13 * sizeof(BYTE), "We should only need 13B for an entire TextAttribute. We may need to increment this in the future as we add additional attributes");
enum class TextAttributeBehavior
{
Stored, // use contained text attribute

50
src/buffer/out/TextAttributeRun.hpp
View File

@ -1,50 +0,0 @@
/*++
Copyright (c) Microsoft Corporation
Licensed under the MIT license.
Module Name:
- TextAttributeRun.hpp
Abstract:
- contains data structure for run-length-encoding of text attribute data
Author(s):
- Michael Niksa (miniksa) 10-Apr-2014
- Paul Campbell (paulcam) 10-Apr-2014
Revision History:
- From components of output.h/.c
by Therese Stowell (ThereseS) 1990-1991
- Pulled into its own file from textBuffer.hpp/cpp (AustDi, 2017)
--*/
#pragma once
#include "TextAttribute.hpp"
class TextAttributeRun final
{
public:
TextAttributeRun() = default;
TextAttributeRun(const size_t cchLength, const TextAttribute attr) noexcept :
_cchLength(gsl::narrow<unsigned int>(cchLength))
{
SetAttributes(attr);
}
size_t GetLength() const noexcept { return _cchLength; }
void SetLength(const size_t cchLength) noexcept { _cchLength = gsl::narrow<unsigned int>(cchLength); }
void IncrementLength() noexcept { _cchLength++; }
void DecrementLength() noexcept { _cchLength--; }
const TextAttribute& GetAttributes() const noexcept { return _attributes; }
void SetAttributes(const TextAttribute textAttribute) noexcept { _attributes = textAttribute; }
private:
unsigned int _cchLength{ 0 };
TextAttribute _attributes{ 0 };
#ifdef UNIT_TESTING
friend class AttrRowTests;
#endif
};

6
src/buffer/out/TextColor.h
View File

@ -37,8 +37,6 @@ Revision History:
#include "WexTestClass.h"
#endif
#pragma pack(push, 1)
enum class ColorType : BYTE
{
IsIndex256 = 0x0,
@ -102,13 +100,13 @@ public:
COLORREF GetRGB() const noexcept;
private:
ColorType _meta : 2;
union
{
BYTE _red, _index;
};
BYTE _green;
BYTE _blue;
ColorType _meta;
#ifdef UNIT_TESTING
friend class TextBufferTests;
@ -117,8 +115,6 @@ private:
#endif
};
#pragma pack(pop)
bool constexpr operator==(const TextColor& a, const TextColor& b) noexcept
{
return a._meta == b._meta &&

3
src/buffer/out/lib/bufferout.vcxproj
View File

@ -11,7 +11,6 @@
<Import Project="$(SolutionDir)src\common.build.pre.props" />
<ItemGroup>
<ClCompile Include="..\AttrRow.cpp" />
<ClCompile Include="..\AttrRowIterator.cpp" />
<ClCompile Include="..\cursor.cpp" />
<ClCompile Include="..\OutputCell.cpp" />
<ClCompile Include="..\OutputCellIterator.cpp" />
@ -34,7 +33,6 @@
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\AttrRow.hpp" />
<ClInclude Include="..\AttrRowIterator.hpp" />
<ClInclude Include="..\cursor.h" />
<ClInclude Include="..\DbcsAttribute.hpp" />
<ClInclude Include="..\ICharRow.hpp" />
@ -47,7 +45,6 @@
<ClInclude Include="..\search.h" />
<ClInclude Include="..\TextColor.h" />
<ClInclude Include="..\TextAttribute.h" />
<ClInclude Include="..\TextAttributeRun.h" />
<ClInclude Include="..\textBuffer.hpp" />
<ClInclude Include="..\textBufferCellIterator.hpp" />
<ClInclude Include="..\textBufferTextIterator.hpp" />

4
src/buffer/out/textBufferCellIterator.hpp
View File

@ -15,8 +15,8 @@ Author(s):
#pragma once
#include "AttrRowIterator.hpp"
#include "CharRow.hpp"
#include "AttrRow.hpp"
#include "OutputCellView.hpp"
#include "../../types/inc/viewport.hpp"
@ -55,7 +55,7 @@ protected:
OutputCellView _view;
const ROW* _pRow;
AttrRowIterator _attrIter;
ATTR_ROW::const_iterator _attrIter;
const TextBuffer& _buffer;
const Microsoft::Console::Types::Viewport _bounds;
bool _exceeded;

731
src/buffer/out/ut_textbuffer/AttrRowTests.cpp
View File

@ -1,731 +0,0 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "WexTestClass.h"
#include "../../../inc/consoletaeftemplates.hpp"
#include "../textBuffer.hpp"
using namespace WEX::Common;
using namespace WEX::Logging;
using namespace WEX::TestExecution;
namespace WEX
{
namespace TestExecution
{
template<>
class VerifyOutputTraits<TextAttributeRun>
{
public:
static WEX::Common::NoThrowString ToString(const TextAttributeRun& tar)
{
return WEX::Common::NoThrowString().Format(
L"Length:%d, attr:%s",
tar.GetLength(),
VerifyOutputTraits<TextAttribute>::ToString(tar.GetAttributes()).GetBuffer());
}
};
template<>
class VerifyCompareTraits<TextAttributeRun, TextAttributeRun>
{
public:
static bool AreEqual(const TextAttributeRun& expected, const TextAttributeRun& actual)
{
return expected.GetAttributes() == actual.GetAttributes() &&
expected.GetLength() == actual.GetLength();
}
static bool AreSame(const TextAttributeRun& expected, const TextAttributeRun& actual)
{
return &expected == &actual;
}
static bool IsLessThan(const TextAttributeRun&, const TextAttributeRun&) = delete;
static bool IsGreaterThan(const TextAttributeRun&, const TextAttributeRun&) = delete;
static bool IsNull(const TextAttributeRun& object)
{
return object.GetAttributes().IsLegacy() && object.GetAttributes().GetLegacyAttributes() == 0 &&
object.GetLength() == 0;
}
};
}
}
class AttrRowTests
{
ATTR_ROW* pSingle;
ATTR_ROW* pChain;
short _sDefaultLength = 80;
short _sDefaultChainLength = 6;
short sChainSegLength;
short sChainLeftover;
short sChainSegmentsNeeded;
WORD __wDefaultAttr = FOREGROUND_BLUE | FOREGROUND_GREEN | FOREGROUND_RED;
WORD __wDefaultChainAttr = BACKGROUND_BLUE | BACKGROUND_GREEN | BACKGROUND_RED | BACKGROUND_INTENSITY;
TextAttribute _DefaultAttr = TextAttribute(__wDefaultAttr);
TextAttribute _DefaultChainAttr = TextAttribute(__wDefaultChainAttr);
TEST_CLASS(AttrRowTests);
TEST_METHOD_SETUP(MethodSetup)
{
pSingle = new ATTR_ROW(_sDefaultLength, _DefaultAttr);
// Segment length is the expected length divided by the row length
// E.g. row of 80, 4 segments, 20 segment length each
sChainSegLength = _sDefaultLength / _sDefaultChainLength;
// Leftover is spaces that don't fit evenly
// E.g. row of 81, 4 segments, 1 leftover length segment
sChainLeftover = _sDefaultLength % _sDefaultChainLength;
// Start with the number of segments we expect
sChainSegmentsNeeded = _sDefaultChainLength;
// If we had a remainder, add one more segment
if (sChainLeftover)
{
sChainSegmentsNeeded++;
}
// Create the chain
pChain = new ATTR_ROW(_sDefaultLength, _DefaultAttr);
pChain->_list.resize(sChainSegmentsNeeded);
// Attach all chain segments that are even multiples of the row length
for (short iChain = 0; iChain < _sDefaultChainLength; iChain++)
{
TextAttributeRun* pRun = &pChain->_list[iChain];
pRun->SetAttributes(TextAttribute{ gsl::narrow_cast<WORD>(iChain) }); // Just use the chain position as the value
pRun->SetLength(sChainSegLength);
}
if (sChainLeftover > 0)
{
// If we had a leftover, then this chain is one longer than we expected (the default length)
// So use it as the index (because indices start at 0)
TextAttributeRun* pRun = &pChain->_list[_sDefaultChainLength];
pRun->SetAttributes(_DefaultChainAttr);
pRun->SetLength(sChainLeftover);
}
return true;
}
TEST_METHOD_CLEANUP(MethodCleanup)
{
delete pSingle;
delete pChain;
return true;
}
TEST_METHOD(TestInitialize)
{
// Properties needed for test
const WORD wAttr = FOREGROUND_RED | BACKGROUND_BLUE;
TextAttribute attr = TextAttribute(wAttr);
// Cases to test
ATTR_ROW* pTestItems[]{ pSingle, pChain };
// Loop cases
for (UINT iIndex = 0; iIndex < ARRAYSIZE(pTestItems); iIndex++)
{
ATTR_ROW* pUnderTest = pTestItems[iIndex];
pUnderTest->Reset(attr);
VERIFY_ARE_EQUAL(pUnderTest->_list.size(), 1u);
VERIFY_ARE_EQUAL(pUnderTest->_list[0].GetAttributes(), attr);
VERIFY_ARE_EQUAL(pUnderTest->_list[0].GetLength(), (unsigned int)_sDefaultLength);
}
}
// Routine Description:
// - Packs an array of words representing attributes into the more compact storage form used by the row.
// Arguments:
// - rgAttrs - Array of words representing the attribute associated with each character position in the row.
// - cRowLength - Length of preceding array.
// - outAttrRun - reference to unique_ptr that will contain packed attr run on success.
// Return Value:
// - Success if success. Buffer too small if row length is incorrect.
HRESULT PackAttrs(_In_reads_(cRowLength) const TextAttribute* const rgAttrs,
const size_t cRowLength,
_Inout_ std::unique_ptr<TextAttributeRun[]>& outAttrRun,
_Out_ size_t* const cOutAttrRun)
{
RETURN_HR_IF(E_NOT_SUFFICIENT_BUFFER, cRowLength == 0);
// first count up the deltas in the array
size_t cDeltas = 1;
const TextAttribute* pPrevAttr = &rgAttrs[0];
for (size_t i = 1; i < cRowLength; i++)
{
const TextAttribute* pCurAttr = &rgAttrs[i];
if (*pCurAttr != *pPrevAttr)
{
cDeltas++;
}
pPrevAttr = pCurAttr;
}
// This whole situation was too complicated with a one off holder for one row run
// new method:
// delete the old buffer
// make a new buffer, one run + one run for each change
// set the values for each run one run index at a time
std::unique_ptr<TextAttributeRun[]> attrRun = std::make_unique<TextAttributeRun[]>(cDeltas);
RETURN_HR_IF_NULL(E_OUTOFMEMORY, attrRun);
TextAttributeRun* pCurrentRun = attrRun.get();
pCurrentRun->SetAttributes(rgAttrs[0]);
pCurrentRun->SetLength(1);
for (size_t i = 1; i < cRowLength; i++)
{
if (pCurrentRun->GetAttributes() == rgAttrs[i])
{
pCurrentRun->SetLength(pCurrentRun->GetLength() + 1);
}
else
{
pCurrentRun++;
pCurrentRun->SetAttributes(rgAttrs[i]);
pCurrentRun->SetLength(1);
}
}
attrRun.swap(outAttrRun);
*cOutAttrRun = cDeltas;
return S_OK;
}
NoThrowString LogRunElement(_In_ TextAttributeRun& run)
{
return NoThrowString().Format(L"%wc%d", run.GetAttributes().GetLegacyAttributes(), run.GetLength());
}
void LogChain(_In_ PCWSTR pwszPrefix,
boost::container::small_vector_base<TextAttributeRun>& chain)
{
NoThrowString str(pwszPrefix);
if (chain.size() > 0)
{
str.Append(LogRunElement(chain[0]));
for (size_t i = 1; i < chain.size(); i++)
{
str.AppendFormat(L"->%s", (const wchar_t*)(LogRunElement(chain[i])));
}
}
Log::Comment(str);
}
void LogChain(_In_ PCWSTR pwszPrefix,
std::vector<TextAttributeRun>& chain)
{
NoThrowString str(pwszPrefix);
if (chain.size() > 0)
{
str.Append(LogRunElement(chain[0]));
for (size_t i = 1; i < chain.size(); i++)
{
str.AppendFormat(L"->%s", (const wchar_t*)(LogRunElement(chain[i])));
}
}
Log::Comment(str);
}
void DoTestInsertAttrRuns(UINT& uiStartPos, WORD& ch1, UINT& uiChar1Length, WORD& ch2, UINT& uiChar2Length)
{
Log::Comment(String().Format(L"StartPos: %d, Char1: %wc, Char1Length: %d, Char2: %wc, Char2Length: %d",
uiStartPos,
ch1,
uiChar1Length,
ch2,
uiChar2Length));
bool const fUseStr2 = (ch2 != L'0');
// Set up our "original row" that we are going to try to insert into.
// This will represent a 10 column run of R3->B5->G2 that we will use for all tests.
ATTR_ROW originalRow{ static_cast<UINT>(_sDefaultLength), _DefaultAttr };
originalRow._list.resize(3);
originalRow._cchRowWidth = 10;
originalRow._list[0].SetAttributes(TextAttribute{ 'R' });
originalRow._list[0].SetLength(3);
originalRow._list[1].SetAttributes(TextAttribute{ 'B' });
originalRow._list[1].SetLength(5);
originalRow._list[2].SetAttributes(TextAttribute{ 'G' });
originalRow._list[2].SetLength(2);
LogChain(L"Original: ", originalRow._list);
// Set up our "insertion run"
size_t cInsertRow = 1;
if (fUseStr2)
{
cInsertRow++;
}
std::vector<TextAttributeRun> insertRow;
insertRow.resize(cInsertRow);
insertRow[0].SetAttributes(TextAttribute{ ch1 });
insertRow[0].SetLength(uiChar1Length);
if (fUseStr2)
{
insertRow[1].SetAttributes(TextAttribute{ ch2 });
insertRow[1].SetLength(uiChar2Length);
}
LogChain(L"Insert: ", insertRow);
Log::Comment(NoThrowString().Format(L"At Index: %d", uiStartPos));
UINT uiTotalLength = uiChar1Length;
if (fUseStr2)
{
uiTotalLength += uiChar2Length;
}
VERIFY_IS_TRUE((uiStartPos + uiTotalLength) >= 1); // assert we won't underflow.
UINT const uiEndPos = uiStartPos + uiTotalLength - 1;
// Calculate our expected final/result run by unpacking original, laying our insertion on it at the index
// then using our pack function to repack it.
// This method is easy to understand and very reliable, but its performance is bad.
// The InsertAttrRuns method we test against below is hard to understand but very high performance in production.
// - 1. Unpack
std::vector<TextAttribute> unpackedOriginal = { originalRow.begin(), originalRow.end() };
// - 2. Overlay insertion
UINT uiInsertedCount = 0;
UINT uiInsertIndex = 0;
// --- Walk through the unpacked run from start to end....
for (UINT uiUnpackedIndex = uiStartPos; uiUnpackedIndex <= uiEndPos; uiUnpackedIndex++)
{
// Pull the item from the insert run to analyze.
TextAttributeRun run = insertRow[uiInsertIndex];
// Copy the attribute from the run into the unpacked array
unpackedOriginal[uiUnpackedIndex] = run.GetAttributes();
// Increment how many times we've copied this particular portion of the run
uiInsertedCount++;
// If we've now inserted enough of them to match the length, advance the insert index and reset the counter.
if (uiInsertedCount >= run.GetLength())
{
uiInsertIndex++;
uiInsertedCount = 0;
}
}
// - 3. Pack.
std::unique_ptr<TextAttributeRun[]> packedRun;
size_t cPackedRun = 0;
VERIFY_SUCCEEDED(PackAttrs(unpackedOriginal.data(), originalRow._cchRowWidth, packedRun, &cPackedRun));
// Now send parameters into InsertAttrRuns and get its opinion on the subject.
VERIFY_SUCCEEDED(originalRow.InsertAttrRuns({ insertRow.data(), insertRow.size() }, uiStartPos, uiEndPos, (UINT)originalRow._cchRowWidth));
// Compare and ensure that the expected and actual match.
VERIFY_ARE_EQUAL(cPackedRun, originalRow._list.size(), L"Ensure that number of array elements required for RLE are the same.");
std::vector<TextAttributeRun> packedRunExpected;
std::copy_n(packedRun.get(), cPackedRun, std::back_inserter(packedRunExpected));
LogChain(L"Expected: ", packedRunExpected);
LogChain(L"Actual: ", originalRow._list);
for (size_t testIndex = 0; testIndex < cPackedRun; testIndex++)
{
VERIFY_ARE_EQUAL(packedRun[testIndex], originalRow._list[testIndex]);
}
}
TEST_METHOD(TestInsertAttrRunsSingle)
{
UINT const uiTestRunLength = 10;
UINT uiStartPos = 0;
WORD ch1 = L'0';
UINT uiChar1Length = 0;
WORD ch2 = L'0';
UINT uiChar2Length = 0;
Log::Comment(L"Test inserting a single item of a variable length into the run.");
WORD rgch1Options[] = { L'X', L'R', L'G', L'B' };
for (size_t iCh1Option = 0; iCh1Option < ARRAYSIZE(rgch1Options); iCh1Option++)
{
ch1 = rgch1Options[iCh1Option];
for (UINT iCh1Length = 1; iCh1Length <= uiTestRunLength; iCh1Length++)
{
uiChar1Length = iCh1Length;
// We can't try to insert a run that's longer than would fit.
// If the run is of length 10 and we're trying to insert a length of 10,
// we can only insert at position 0.
// For the run length of 10 and an insert length of 9, we can try positions 0 and 1.
// And so on...
UINT const uiMaxPos = uiTestRunLength - uiChar1Length;
for (UINT iStartPos = 0; iStartPos < uiMaxPos; iStartPos++)
{
uiStartPos = iStartPos;
DoTestInsertAttrRuns(uiStartPos, ch1, uiChar1Length, ch2, uiChar2Length);
}
}
}
}
TEST_METHOD(TestInsertAttrRunsMultiple)
{
UINT const uiTestRunLength = 10;
UINT uiStartPos = 0;
WORD ch1 = L'0';
UINT uiChar1Length = 0;
WORD ch2 = L'0';
UINT uiChar2Length = 0;
Log::Comment(L"Test inserting a multiple item run with each piece having variable length into the existing run.");
WORD rgch1Options[] = { L'X', L'R', L'G', L'B' };
for (size_t iCh1Option = 0; iCh1Option < ARRAYSIZE(rgch1Options); iCh1Option++)
{
ch1 = rgch1Options[iCh1Option];
UINT const uiMaxCh1Length = uiTestRunLength - 1; // leave at least 1 space for the second piece of the insert run.
for (UINT iCh1Length = 1; iCh1Length <= uiMaxCh1Length; iCh1Length++)
{
uiChar1Length = iCh1Length;
WORD rgch2Options[] = { L'Y' };
for (size_t iCh2Option = 0; iCh2Option < ARRAYSIZE(rgch2Options); iCh2Option++)
{
ch2 = rgch2Options[iCh2Option];
// When choosing the length of the second item, it can't be bigger than the remaining space in the run
// when accounting for the length of the first piece chosen.
// For example if the total run length is 10 and the first piece chosen was 8 long,
// the second piece can only be 1 or 2 long.
UINT const uiMaxCh2Length = uiTestRunLength - uiMaxCh1Length;
for (UINT iCh2Length = 1; iCh2Length <= uiMaxCh2Length; iCh2Length++)
{
uiChar2Length = iCh2Length;
// We can't try to insert a run that's longer than would fit.
// If the run is of length 10 and we're trying to insert a total length of 10,
// we can only insert at position 0.
// For the run length of 10 and an insert length of 9, we can try positions 0 and 1.
// And so on...
UINT const uiMaxPos = uiTestRunLength - (uiChar1Length + uiChar2Length);
for (UINT iStartPos = 0; iStartPos <= uiMaxPos; iStartPos++)
{
uiStartPos = iStartPos;
DoTestInsertAttrRuns(uiStartPos, ch1, uiChar1Length, ch2, uiChar2Length);
}
}
}
}
}
}
TEST_METHOD(TestUnpackAttrs)
{
Log::Comment(L"Checking unpack of a single color for the entire length");
{
const std::vector<TextAttribute> attrs{ pSingle->begin(), pSingle->end() };
for (auto& attr : attrs)
{
VERIFY_ARE_EQUAL(attr, _DefaultAttr);
}
}
Log::Comment(L"Checking unpack of the multiple color chain");
const std::vector<TextAttribute> attrs{ pChain->begin(), pChain->end() };
short cChainRun = 0; // how long we've been looking at the current piece of the chain
short iChainSegIndex = 0; // which piece of the chain we should be on right now
for (auto& attr : attrs)
{
// by default the chain was assembled above to have the chain segment index be the attribute
TextAttribute MatchingAttr = TextAttribute(iChainSegIndex);
// However, if the index is greater than the expected chain length, a remainder piece was made with a default attribute
if (iChainSegIndex >= _sDefaultChainLength)
{
MatchingAttr = _DefaultChainAttr;
}
VERIFY_ARE_EQUAL(attr, MatchingAttr);
// Add to the chain run
cChainRun++;
// If the chain run is greater than the length the segments were specified to be
if (cChainRun >= sChainSegLength)
{
// reset to 0
cChainRun = 0;
// move to the next chain segment down the line
iChainSegIndex++;
}
}
}
TEST_METHOD(TestReverseIteratorWalkFromMiddle)
{
// GH #3409, walking backwards through color range runs out of bounds
// We're going to create an attribute row with assorted colors and varying lengths
// just like the row of text on the Ubuntu prompt line that triggered this bug being found.
// Then we're going to walk backwards through the iterator like a selection-expand-to-left
// operation and ensure we don't run off the bounds.
// walk the chain, from index, stepSize at a time
// ensure we don't crash
auto testWalk = [](ATTR_ROW* chain, size_t index, int stepSize) {
// move to starting index
auto iter = chain->cbegin();
iter += index;
// Now walk backwards in a loop until 0.
while (iter)
{
iter -= stepSize;
}
Log::Comment(L"We made it through without crashing!");
};
// take one step of size stepSize on the chain
// index is where we start from
// expectedAttribute is what we expect to read here
auto verifyStep = [](ATTR_ROW* chain, size_t index, int stepSize, TextAttribute expectedAttribute) {
// move to starting index
auto iter = chain->cbegin();
iter += index;
// Now step backwards
iter -= stepSize;
VERIFY_ARE_EQUAL(expectedAttribute, *iter);
};
Log::Comment(L"Reverse iterate through ubuntu prompt");
{
// Create attr row representing a buffer that's 121 wide.
auto chain = std::make_unique<ATTR_ROW>(121, _DefaultAttr);
// The repro case had 4 chain segments.
chain->_list.resize(4);
// The color 10 went for the first 18.
chain->_list[0].SetAttributes(TextAttribute(0xA));
chain->_list[0].SetLength(18);
// Default color for the next 1
chain->_list[1].SetAttributes(TextAttribute());
chain->_list[1].SetLength(1);
// Color 12 for the next 29
chain->_list[2].SetAttributes(TextAttribute(0xC));
chain->_list[2].SetLength(29);
// Then default color to end the run
chain->_list[3].SetAttributes(TextAttribute());
chain->_list[3].SetLength(73);
// The sum of the lengths should be 121.
VERIFY_ARE_EQUAL(chain->_cchRowWidth, chain->_list[0]._cchLength + chain->_list[1]._cchLength + chain->_list[2]._cchLength + chain->_list[3]._cchLength);
auto index = chain->_list[0].GetLength();
auto stepSize = 1;
testWalk(chain.get(), index, stepSize);
}
Log::Comment(L"Reverse iterate across a text run in the chain");
{
// Create attr row representing a buffer that's 3 wide.
auto chain = std::make_unique<ATTR_ROW>(3, _DefaultAttr);
// The repro case had 3 chain segments.
chain->_list.resize(3);
// The color 10 went for the first 1.
chain->_list[0].SetAttributes(TextAttribute(0xA));
chain->_list[0].SetLength(1);
// The color 11 for the next 1
chain->_list[1].SetAttributes(TextAttribute(0xB));
chain->_list[1].SetLength(1);
// Color 12 for the next 1
chain->_list[2].SetAttributes(TextAttribute(0xC));
chain->_list[2].SetLength(1);
// The sum of the lengths should be 3.
VERIFY_ARE_EQUAL(chain->_cchRowWidth, chain->_list[0]._cchLength + chain->_list[1]._cchLength + chain->_list[2]._cchLength);
// on 'ABC', step from B to A
auto index = 1;
auto stepSize = 1;
verifyStep(chain.get(), index, stepSize, TextAttribute(0xA));
}
Log::Comment(L"Reverse iterate across two text runs in the chain");
{
// Create attr row representing a buffer that's 3 wide.
auto chain = std::make_unique<ATTR_ROW>(3, _DefaultAttr);
// The repro case had 3 chain segments.
chain->_list.resize(3);
// The color 10 went for the first 1.
chain->_list[0].SetAttributes(TextAttribute(0xA));
chain->_list[0].SetLength(1);
// The color 11 for the next 1
chain->_list[1].SetAttributes(TextAttribute(0xB));
chain->_list[1].SetLength(1);
// Color 12 for the next 1
chain->_list[2].SetAttributes(TextAttribute(0xC));
chain->_list[2].SetLength(1);
// The sum of the lengths should be 3.
VERIFY_ARE_EQUAL(chain->_cchRowWidth, chain->_list[0]._cchLength + chain->_list[1]._cchLength + chain->_list[2]._cchLength);
// on 'ABC', step from C to A
auto index = 2;
auto stepSize = 2;
verifyStep(chain.get(), index, stepSize, TextAttribute(0xA));
}
}
TEST_METHOD(TestSetAttrToEnd)
{
const WORD wTestAttr = FOREGROUND_BLUE | BACKGROUND_GREEN;
TextAttribute TestAttr = TextAttribute(wTestAttr);
Log::Comment(L"FIRST: Set index to > 0 to test making/modifying chains");
const short iTestIndex = 50;
VERIFY_IS_TRUE(iTestIndex >= 0 && iTestIndex < _sDefaultLength);
Log::Comment(L"SetAttrToEnd for single color applied to whole string.");
pSingle->SetAttrToEnd(iTestIndex, TestAttr);
// Was 1 (single), should now have 2 segments
VERIFY_ARE_EQUAL(pSingle->_list.size(), 2u);
VERIFY_ARE_EQUAL(pSingle->_list[0].GetAttributes(), _DefaultAttr);
VERIFY_ARE_EQUAL(pSingle->_list[0].GetLength(), (unsigned int)(_sDefaultLength - (_sDefaultLength - iTestIndex)));
VERIFY_ARE_EQUAL(pSingle->_list[1].GetAttributes(), TestAttr);
VERIFY_ARE_EQUAL(pSingle->_list[1].GetLength(), (unsigned int)(_sDefaultLength - iTestIndex));
Log::Comment(L"SetAttrToEnd for existing chain of multiple colors.");
pChain->SetAttrToEnd(iTestIndex, TestAttr);
// From 7 segments down to 5.
VERIFY_ARE_EQUAL(pChain->_list.size(), 5u);
// Verify chain colors and lengths
VERIFY_ARE_EQUAL(TextAttribute(0), pChain->_list[0].GetAttributes());
VERIFY_ARE_EQUAL(pChain->_list[0].GetLength(), (unsigned int)13);
VERIFY_ARE_EQUAL(TextAttribute(1), pChain->_list[1].GetAttributes());
VERIFY_ARE_EQUAL(pChain->_list[1].GetLength(), (unsigned int)13);
VERIFY_ARE_EQUAL(TextAttribute(2), pChain->_list[2].GetAttributes());
VERIFY_ARE_EQUAL(pChain->_list[2].GetLength(), (unsigned int)13);
VERIFY_ARE_EQUAL(TextAttribute(3), pChain->_list[3].GetAttributes());
VERIFY_ARE_EQUAL(pChain->_list[3].GetLength(), (unsigned int)11);
VERIFY_ARE_EQUAL(TestAttr, pChain->_list[4].GetAttributes());
VERIFY_ARE_EQUAL(pChain->_list[4].GetLength(), (unsigned int)30);
Log::Comment(L"SECOND: Set index to 0 to test replacing anything with a single");
ATTR_ROW* pTestItems[]{ pSingle, pChain };
for (UINT iIndex = 0; iIndex < ARRAYSIZE(pTestItems); iIndex++)
{
ATTR_ROW* pUnderTest = pTestItems[iIndex];
pUnderTest->SetAttrToEnd(0, TestAttr);
// should be down to 1 attribute set from beginning to end of string
VERIFY_ARE_EQUAL(pUnderTest->_list.size(), 1u);
// singular pair should contain the color
VERIFY_ARE_EQUAL(pUnderTest->_list[0].GetAttributes(), TestAttr);
// and its length should be the length of the whole string
VERIFY_ARE_EQUAL(pUnderTest->_list[0].GetLength(), (unsigned int)_sDefaultLength);
}
}
TEST_METHOD(TestTotalLength)
{
ATTR_ROW* pTestItems[]{ pSingle, pChain };
for (UINT iIndex = 0; iIndex < ARRAYSIZE(pTestItems); iIndex++)
{
ATTR_ROW* pUnderTest = pTestItems[iIndex];
const size_t Result = pUnderTest->_cchRowWidth;
VERIFY_ARE_EQUAL((short)Result, _sDefaultLength);
}
}
TEST_METHOD(TestResize)
{
pSingle->Resize(240);
pChain->Resize(240);
pSingle->Resize(255);
pChain->Resize(255);
pSingle->Resize(255);
pChain->Resize(255);
pSingle->Resize(60);
pChain->Resize(60);
pSingle->Resize(60);
pChain->Resize(60);
VERIFY_THROWS_SPECIFIC(pSingle->Resize(0), wil::ResultException, [](wil::ResultException& e) { return e.GetErrorCode() == E_INVALIDARG; });
VERIFY_THROWS_SPECIFIC(pChain->Resize(0), wil::ResultException, [](wil::ResultException& e) { return e.GetErrorCode() == E_INVALIDARG; });
}
};

3
src/buffer/out/ut_textbuffer/TextBuffer.Unit.Tests.vcxproj
View File

@ -6,11 +6,10 @@
<RootNamespace>TextBufferUnitTests</RootNamespace>
<ProjectName>TextBuffer.Unit.Tests</ProjectName>
<TargetName>TextBuffer.Unit.Tests</TargetName>
<ConfigurationType>DynamicLibrary</ConfigurationType>
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<Import Project="$(SolutionDir)src\common.build.pre.props" />
<ItemGroup>
<ClCompile Include="AttrRowTests.cpp" />
<ClCompile Include="ReflowTests.cpp" />
<ClCompile Include="TextColorTests.cpp" />
<ClCompile Include="TextAttributeTests.cpp" />

1
src/buffer/out/ut_textbuffer/sources
View File

@ -14,7 +14,6 @@ DLLDEF =
SOURCES = \
$(SOURCES) \
AttrRowTests.cpp \
ReflowTests.cpp \
TextColorTests.cpp \
TextAttributeTests.cpp \

2
src/cascadia/UnitTests_TerminalCore/ConptyRoundtripTests.cpp
View File

@ -3680,7 +3680,7 @@ void ConptyRoundtripTests::HyperlinkIdConsistency()
// Check that all the linked cells still have the same ID
auto& attrRow = tb.GetRowByOffset(0).GetAttrRow();
auto id = attrRow.GetAttrByColumn(0).GetHyperlinkId();
for (auto i = 1; i < 4; ++i)
for (uint16_t i = 1; i < 4; ++i)
{
VERIFY_ARE_EQUAL(id, attrRow.GetAttrByColumn(i).GetHyperlinkId());
}

3
src/host/ut_host/Host.UnitTests.vcxproj.filters
View File

@ -111,6 +111,9 @@
<ClCompile Include="ObjectTests.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="ConptyOutputTests.cpp">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="UnicodeLiteral.hpp">

38
src/inc/consoletaeftemplates.hpp
View File

@ -568,37 +568,59 @@ namespace WEX::TestExecution
}
};
template<>
class VerifyOutputTraits<std::string_view>
{
public:
static WEX::Common::NoThrowString ToString(const std::string_view& view)
{
if (view.empty())
{
return L"<empty>";
}
WEX::Common::NoThrowString s;
s.AppendFormat(L"%.*hs", gsl::narrow_cast<unsigned int>(view.size()), view.data());
return s;
}
};
template<>
class VerifyOutputTraits<std::wstring_view>
{
public:
static WEX::Common::NoThrowString ToString(const std::wstring_view& view)
{
if (view.empty())
{
return L"<empty>";
}
return WEX::Common::NoThrowString(view.data(), gsl::narrow<int>(view.size()));
}
};
template<>
class VerifyCompareTraits<std::wstring_view, std::wstring_view>
template<typename Elem>
class VerifyCompareTraits<std::basic_string_view<Elem>, std::basic_string_view<Elem>>
{
public:
static bool AreEqual(const std::wstring_view& expected, const std::wstring_view& actual)
static bool AreEqual(const std::basic_string_view<Elem>& expected, const std::basic_string_view<Elem>& actual)
{
return expected == actual;
}
static bool AreSame(const std::wstring_view& expected, const std::wstring_view& actual)
static bool AreSame(const std::basic_string_view<Elem>& expected, const std::basic_string_view<Elem>& actual)
{
return expected.data() == actual.data();
}
static bool IsLessThan(const std::wstring_view&, const std::wstring_view&) = delete;
static bool IsLessThan(const std::basic_string_view<Elem>&, const std::basic_string_view<Elem>&) = delete;
static bool IsGreaterThan(const std::wstring_view&, const std::wstring_view&) = delete;
static bool IsGreaterThan(const std::basic_string_view<Elem>&, const std::basic_string_view<Elem>&) = delete;
static bool IsNull(const std::wstring_view& object)
static bool IsNull(const std::basic_string_view<Elem>& object)
{
return object.size() == 0;
return object.empty();
}
};
}

1
src/inc/til.h
View File

@ -13,6 +13,7 @@
#include "til/point.h"
#include "til/operators.h"
#include "til/rectangle.h"
#include "til/rle.h"
#include "til/bitmap.h"
#include "til/u8u16convert.h"
#include "til/spsc.h"

1063
src/inc/til/rle.h Normal file
View File

File diff suppressed because it is too large Load Diff

604
src/til/ut_til/RunLengthEncodingTests.cpp Normal file
View File

@ -0,0 +1,604 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "til/rle.h"
#include "consoletaeftemplates.hpp"
using namespace std::literals;
using namespace WEX::Common;
using namespace WEX::Logging;
using namespace WEX::TestExecution;
namespace WEX::TestExecution
{
template<typename T, typename S, typename Container>
class VerifyCompareTraits<::std::string_view, ::til::basic_rle<T, S, Container>>
{
using rle_vector = ::til::basic_rle<T, S, Container>;
using value_type = typename rle_vector::value_type;
public:
static bool AreEqual(const ::std::string_view& expected, const rle_vector& actual) noexcept
{
auto it = expected.data();
const auto end = it + expected.size();
size_t expected_size = 0;
for (const auto& run : actual.runs())
{
const auto actual_value = run.value;
const auto length = run.length;
if (length == 0)
{
return false;
}
for (size_t i = 0; i < length; ++it)
{
if (it == end)
{
return false;
}
const auto ch = *it;
if (ch == '|' && i != 0)
{
return false;
}
if (ch == ' ' && i == 0)
{
return false;
}
if (ch < '0' || ch > '9')
{
continue;
}
const value_type expected_value = ch - '0';
if (expected_value != actual_value)
{
return false;
}
++i;
++expected_size;
}
}
return it == end && expected_size == actual.size();
}
static bool AreSame(const ::std::string_view& expected, const rle_vector& actual) noexcept
{
return false;
}
static bool IsLessThan(const ::std::string_view& expectedLess, const rle_vector& expectedGreater) = delete;
static bool IsGreaterThan(const ::std::string_view& expectedGreater, const rle_vector& expectedLess) = delete;
static bool IsNull(const ::std::string_view& object) = delete;
};
}
class RunLengthEncodingTests
{
using rle_vector = til::small_rle<uint16_t, uint16_t, 16>;
using value_type = rle_vector::value_type;
using size_type = rle_vector::size_type;
using rle_type = rle_vector::rle_type;
using basic_container = std::basic_string<value_type>;
using basic_container_view = std::basic_string_view<value_type>;
using rle_container = rle_vector::container;
static rle_container rle_encode(const std::string_view& from)
{
if (from.empty())
{
return {};
}
rle_container to;
value_type value = from.front() - '0';
size_type length = 0;
for (auto ch : from)
{
if (ch < '0' || ch > '9')
{
continue;
}
const value_type val = ch - '0';
if (val != value)
{
to.emplace_back(value, length);
value = val;
length = 0;
}
length++;
}
if (length)
{
to.emplace_back(value, length);
}
return to;
}
static rle_container rle_encode(const basic_container_view& from)
{
if (from.empty())
{
return {};
}
rle_container to;
value_type value = from.front();
size_type length = 0;
for (auto v : from)
{
if (v != value)
{
to.emplace_back(value, length);
value = v;
length = 0;
}
length++;
}
if (length)
{
to.emplace_back(value, length);
}
return to;
}
static basic_container rle_decode(const rle_container& from)
{
basic_container to;
to.reserve(from.size());
for (const auto& run : from)
{
for (size_t i = 0; i < run.length; ++i)
{
to.push_back(run.value);
}
}
return to;
}
TEST_CLASS(RunLengthEncodingTests)
TEST_METHOD(ConstructDefault)
{
rle_vector rle{};
VERIFY_ARE_EQUAL(0u, rle.size());
VERIFY_IS_TRUE(rle.empty());
// We're testing replace() elsewhere, but this is special:
// This ensures that even if we're default constructed we can add data.
rle.replace(0, 0, { 1, 5 });
VERIFY_ARE_EQUAL(5u, rle.size());
VERIFY_IS_FALSE(rle.empty());
}
TEST_METHOD(ConstructWithInitializerList)
{
rle_vector rle{ { { 1, 3 }, { 2, 2 }, { 1, 3 } } };
VERIFY_ARE_EQUAL("1 1 1|2 2|1 1 1"sv, rle);
}
TEST_METHOD(ConstructWithLengthAndValue)
{
rle_vector rle(5, 1);
VERIFY_ARE_EQUAL("1 1 1 1 1"sv, rle);
}
TEST_METHOD(CopyAndMove)
{
constexpr auto expected_full = "1 1 1|2 2|1 1 1"sv;
constexpr auto expected_empty = ""sv;
rle_vector rle1{ { { 1, 3 }, { 2, 2 }, { 1, 3 } } };
rle_vector rle2;
VERIFY_ARE_EQUAL(expected_full, rle1);
VERIFY_ARE_EQUAL(expected_empty, rle2);
// swap
rle1.swap(rle2);
VERIFY_ARE_EQUAL(expected_empty, rle1);
VERIFY_ARE_EQUAL(expected_full, rle2);
// copy
rle1 = rle2;
VERIFY_ARE_EQUAL(expected_full, rle1);
VERIFY_ARE_EQUAL(expected_full, rle2);
// make sure we can detect whether the upcoming move failed
rle1 = { { { 1, 1 } } };
// move
rle1 = std::move(rle2);
VERIFY_ARE_EQUAL(expected_full, rle1);
}
TEST_METHOD(At)
{
rle_vector rle{
{
{ 1, 1 },
{ 3, 2 },
{ 2, 1 },
{ 1, 3 },
{ 5, 2 },
}
};
VERIFY_ARE_EQUAL(1u, rle.at(0));
VERIFY_ARE_EQUAL(3u, rle.at(1));
VERIFY_ARE_EQUAL(3u, rle.at(2));
VERIFY_ARE_EQUAL(2u, rle.at(3));
VERIFY_ARE_EQUAL(1u, rle.at(4));
VERIFY_ARE_EQUAL(1u, rle.at(5));
VERIFY_ARE_EQUAL(1u, rle.at(6));
VERIFY_ARE_EQUAL(5u, rle.at(7));
VERIFY_ARE_EQUAL(5u, rle.at(8));
VERIFY_THROWS(rle.at(9), std::out_of_range);
}
TEST_METHOD(Slice)
{
rle_vector rle{
{
{ 1, 1 },
{ 3, 2 },
{ 2, 1 },
{ 1, 3 },
{ 5, 2 },
}
};
VERIFY_ARE_EQUAL("1|3 3|2|1 1 1|5 5"sv, rle);
// empty
VERIFY_ARE_EQUAL(""sv, rle.slice(0, 0)); // begin
VERIFY_ARE_EQUAL(""sv, rle.slice(1, 1)); // between two runs
VERIFY_ARE_EQUAL(""sv, rle.slice(2, 2)); // within a run
VERIFY_ARE_EQUAL(""sv, rle.slice(rle.size(), rle.size())); // end
VERIFY_ARE_EQUAL(""sv, rle.slice(5, 0)); // end_index > begin_index
VERIFY_ARE_EQUAL(""sv, rle.slice(1000, 900)); // end_index > begin_index
// full copy
VERIFY_ARE_EQUAL("1|3 3|2|1 1 1|5 5"sv, rle.slice(0, rle.size()));
// between two runs -> between two runs
VERIFY_ARE_EQUAL("1|3 3|2|1 1 1"sv, rle.slice(0, 7));
VERIFY_ARE_EQUAL("2|1 1 1"sv, rle.slice(3, 7));
// between two runs -> within a run
VERIFY_ARE_EQUAL("3 3|2|1"sv, rle.slice(1, 5));
VERIFY_ARE_EQUAL("3 3|2|1 1"sv, rle.slice(1, 6));
// within a run -> between two runs
VERIFY_ARE_EQUAL("3|2|1 1 1|5 5"sv, rle.slice(2, rle.size()));
VERIFY_ARE_EQUAL("3|2|1 1 1"sv, rle.slice(2, 7));
// within a run -> within a run
VERIFY_ARE_EQUAL("3|2|1"sv, rle.slice(2, 5));
VERIFY_ARE_EQUAL("3|2|1 1"sv, rle.slice(2, 6));
}
TEST_METHOD(Replace)
{
struct TestCase
{
std::string_view source;
size_type start_index;
size_type end_index;
std::string_view change;
std::string_view expected;
};
std::array<TestCase, 29> test_cases{
{
// empty source
{ "", 0, 0, "", "" },
{ "", 0, 0, "1|2|3", "1|2|3" },
// empty change
{ "1|2|3", 0, 0, "", "1|2|3" },
{ "1|2|3", 2, 2, "", "1|2|3" },
{ "1|2|3", 3, 3, "", "1|2|3" },
// remove
{ "1|3 3|2|1 1 1|5 5", 0, 9, "", "" }, // all
{ "1|3 3|2|1 1 1|5 5", 0, 6, "", "1|5 5" }, // beginning
{ "1|3 3|2|1 1 1|5 5", 6, 9, "", "1|3 3|2|1 1" }, // end
{ "1|3 3|2|1 1 1|5 5", 3, 7, "", "1|3 3|5 5" }, // middle, between runs
{ "1|3 3|2|1 1 1|5 5", 2, 6, "", "1|3|1|5 5" }, // middle, within runs
// insert
{ "1|3 3|2|1 1 1|5 5", 0, 0, "6|7 7|8", "6|7 7|8|1|3 3|2|1 1 1|5 5" }, // beginning
{ "1|3 3|2|1 1 1|5 5", 9, 9, "6|7 7|8", "1|3 3|2|1 1 1|5 5|6|7 7|8" }, // end
{ "1|3 3|2|1 1 1|5 5", 4, 4, "6|7 7|8", "1|3 3|2|6|7 7|8|1 1 1|5 5" }, // middle, between runs
{ "1|3 3|2|1 1 1|5 5", 5, 5, "6|7 7|8", "1|3 3|2|1|6|7 7|8|1 1|5 5" }, // middle, within runs
{ "1|3 3|2|1 1 1|5 5", 6, 6, "6", "1|3 3|2|1 1|6|1|5 5" }, // middle, within runs, single run
// replace
{ "1|3 3|2|1 1 1|5 5", 0, 9, "6|7 7|8", "6|7 7|8" }, // all
{ "1|3 3|2|1 1 1|5 5", 0, 6, "6|7 7|8", "6|7 7|8|1|5 5" }, // beginning
{ "1|3 3|2|1 1 1|5 5", 6, 9, "6|7 7|8", "1|3 3|2|1 1|6|7 7|8" }, // end
{ "1|3 3|2|1 1 1|5 5", 3, 7, "6|7 7|8", "1|3 3|6|7 7|8|5 5" }, // middle, between runs
{ "1|3 3|2|1 1 1|5 5", 3, 7, "6|7 7 7", "1|3 3|6|7 7 7|5 5" }, // middle, between runs, same size
{ "1|3 3|2|1 1 1|5 5", 2, 6, "6|7 7|8", "1|3|6|7 7|8|1|5 5" }, // middle, within runs
{ "1|3 3|2|1 1 1|5 5", 2, 6, "6", "1|3|6|1|5 5" }, // middle, within runs, single run
// join with predecessor/successor run
{ "1|3 3|2|1 1 1|5 5", 0, 3, "1|2 2", "1|2 2 2|1 1 1|5 5" }, // beginning
{ "1|3 3|2|1 1 1|5 5", 7, 9, "1|5", "1|3 3|2|1 1 1 1|5" }, // end
{ "1|3 3|2|1 1 1|5 5", 1, 4, "1|2|1", "1 1|2|1 1 1 1|5 5" }, // middle, between runs
{ "1|3 3|2|1 1 1|5 5", 2, 6, "3 3|1", "1|3 3 3|1 1|5 5" }, // middle, within runs
{ "1|3 3|2|1 1 1|5 5", 1, 6, "1", "1 1 1|5 5" }, // middle, within runs, single run
{ "1|3 3|2|1 1 1|5 5", 1, 4, "", "1 1 1 1|5 5" }, // middle, within runs, no runs
}
};
int idx = 0;
for (const auto& test_case : test_cases)
{
rle_vector rle{ rle_encode(test_case.source) };
const auto change = rle_encode(test_case.change);
rle.replace(test_case.start_index, test_case.end_index, change);
VERIFY_ARE_EQUAL(
test_case.expected,
rle,
NoThrowString().Format(
L"test case: %d\nsource: %hs\nstart_index: %u\nend_index: %u\nchange: %hs\nexpected: %hs\nactual: %s",
idx,
test_case.source.data(),
test_case.start_index,
test_case.end_index,
test_case.change.data(),
test_case.expected.data(),
rle.to_string().c_str()));
++idx;
}
}
TEST_METHOD(ReplaceValues)
{
struct TestCase
{
std::string_view source;
value_type old_value;
value_type new_value;
std::string_view expected;
};
std::array<TestCase, 6> test_cases{
{
// empty source
{ "", 1, 2, "" },
// no changes
{ "3|4|5", 1, 2, "3|4|5" },
// begin
{ "1 1|2|3|4", 1, 2, "2 2 2|3|4" },
// end
{ "4|3|2|1 1", 1, 2, "4|3|2 2 2" },
// middle
{ "3|2|1|2|4", 1, 2, "3|2 2 2|4" },
// middle
{ "3|1|2|1|4", 1, 2, "3|2 2 2|4" },
}
};
int idx = 0;
for (const auto& test_case : test_cases)
{
rle_vector rle{ rle_encode(test_case.source) };
rle.replace_values(test_case.old_value, test_case.new_value);
VERIFY_ARE_EQUAL(
test_case.expected,
rle,
NoThrowString().Format(
L"test case: %d\nsource: %hs\nold_value: %u\nnew_value: %u\nexpected: %hs\nactual: %s",
idx,
test_case.source.data(),
test_case.old_value,
test_case.new_value,
test_case.expected.data(),
rle.to_string().c_str()));
++idx;
}
}
TEST_METHOD(ResizeTrailingExtent)
{
constexpr std::string_view data{ "133211155" };
// shrink
for (size_type length = 0; length <= data.size(); length++)
{
rle_vector rle{ rle_encode(data) };
rle.resize_trailing_extent(length);
VERIFY_ARE_EQUAL(data.substr(0, length), rle);
}
// grow
{
std::string expected{ data };
expected.insert(expected.end(), 5, expected.back());
rle_vector actual{ rle_encode(data) };
actual.resize_trailing_extent(static_cast<size_type>(expected.size()));
VERIFY_ARE_EQUAL(std::string_view{ expected }, actual);
}
}
TEST_METHOD(Comparison)
{
rle_vector rle1{ { { 1, 1 }, { 3, 2 }, { 2, 1 } } };
rle_vector rle2{ rle1 };
VERIFY_IS_TRUE(rle1 == rle2);
VERIFY_IS_FALSE(rle1 != rle2);
rle2.replace(0, 1, 2);
VERIFY_IS_FALSE(rle1 == rle2);
VERIFY_IS_TRUE(rle1 != rle2);
}
TEST_METHOD(Iterators)
{
using difference_type = rle_vector::const_iterator::difference_type;
constexpr std::string_view expected{ "133211155" };
rle_vector rle{ rle_encode(expected) };
// linear forward iteration (the most common use case)
{
std::string actual;
actual.reserve(expected.size());
for (auto v : rle)
{
actual.push_back(static_cast<char>(v + '0'));
}
VERIFY_ARE_EQUAL(expected, actual);
}
// linear backward iteration
{
std::string reverse_expectation{ expected };
std::reverse(reverse_expectation.begin(), reverse_expectation.end());
std::string actual;
actual.reserve(reverse_expectation.size());
for (auto it = rle.rbegin(); it != rle.rend(); ++it)
{
actual.push_back(static_cast<char>(*it + '0'));
}
VERIFY_ARE_EQUAL(reverse_expectation, actual);
}
// random forward iteration
{
auto it = rle.begin();
const auto end = rle.end();
// 133211155
// ^
it += 2;
VERIFY_ARE_EQUAL(3u, *it);
// 133211155
// ^
it += 1;
VERIFY_ARE_EQUAL(2u, *it);
// 133211155
// ^
it += 1;
VERIFY_ARE_EQUAL(1u, *it);
// 133211155
// ^
it += 2;
VERIFY_ARE_EQUAL(1u, *it);
// 133211155
// ^
it += 2;
VERIFY_ARE_EQUAL(5u, *it);
// 133211155
// ^
++it;
VERIFY_ARE_EQUAL(end, it);
}
// random backward iteration
{
auto it = rle.end();
// 133211155
// ^
--it;
VERIFY_ARE_EQUAL(5u, *it);
// 133211155
// ^
it -= 2;
VERIFY_ARE_EQUAL(1u, *it);
// 133211155
// ^
it -= 2;
VERIFY_ARE_EQUAL(1u, *it);
// 133211155
// ^
it -= 1;
VERIFY_ARE_EQUAL(2u, *it);
// 133211155
// ^
it -= 1;
VERIFY_ARE_EQUAL(3u, *it);
}
// difference (basic test)
{
const auto beg = rle.begin();
auto it = beg;
for (size_t i = 0; i <= expected.size(); ++i, ++it)
{
VERIFY_ARE_EQUAL(static_cast<difference_type>(i), it - beg);
VERIFY_ARE_EQUAL(-static_cast<difference_type>(i), beg - it);
}
}
// difference (in the middle of the vector)
{
const auto beg = rle.begin();
const auto lower = beg + 2;
const auto upper = beg + 5;
VERIFY_ARE_EQUAL(static_cast<difference_type>(3), upper - lower);
VERIFY_ARE_EQUAL(-static_cast<difference_type>(3), lower - upper);
}
// difference (in the middle of a run)
{
const auto beg = rle.begin();
const auto lower = beg + 5;
const auto upper = beg + 6;
VERIFY_ARE_EQUAL(static_cast<difference_type>(1), upper - lower);
VERIFY_ARE_EQUAL(-static_cast<difference_type>(1), lower - upper);
}
}
};

1
src/til/ut_til/sources
View File

@ -21,6 +21,7 @@ SOURCES = \
PointTests.cpp \
MathTests.cpp \
RectangleTests.cpp \
RunLengthEncodingTests.cpp \
SizeTests.cpp \
SomeTests.cpp \
u8u16convertTests.cpp \

1
src/til/ut_til/til.unit.tests.vcxproj
View File

@ -17,6 +17,7 @@
<ClCompile Include="StaticMapTests.cpp" />
<ClCompile Include="MathTests.cpp" />
<ClCompile Include="RectangleTests.cpp" />
<ClCompile Include="RunLengthEncodingTests.cpp" />
<ClCompile Include="SizeTests.cpp" />
<ClCompile Include="ColorTests.cpp" />
<ClCompile Include="CoalesceTests.cpp" />

29
tools/ConsoleTypes.natvis
View File

@ -22,10 +22,6 @@
</Expand>
</Type>
<Type Name="TextAttributeRun">
<DisplayString>Length={_cchLength} Attr={_attributes}</DisplayString>
</Type>
<Type Name="Microsoft::Console::Types::Viewport">
<!-- Can't call functions in here -->
<DisplayString>{{LT({_sr.Left}, {_sr.Top}) RB({_sr.Right}, {_sr.Bottom}) [{_sr.Right-_sr.Left+1} x { _sr.Bottom-_sr.Top+1}]}}</DisplayString>
@ -50,9 +46,8 @@
</Type>
<Type Name="ATTR_ROW">
<DisplayString>{{ size={_cchRowWidth} }}</DisplayString>
<Expand>
<ExpandedItem>_list</ExpandedItem>
<ExpandedItem>_data</ExpandedItem>
</Expand>
</Type>
@ -97,4 +92,26 @@
<Type Name="til::color">
<DisplayString>{{RGB: {(int)r,d}, {(int)g,d}, {(int)b,d}; α: {(int)a,d}}}</DisplayString>
</Type>
<Type Name="til::rle&lt;*&gt;">
<DisplayString>{{Size: {_total_length,d}}}</DisplayString>
<Expand>
<Item Name="[size]">_total_length</Item>
<ExpandedItem>_runs</ExpandedItem>
</Expand>
</Type>
<Type Name="til::details::rle_const_iterator&lt;*&gt;">
<DisplayString>{{run of {_it->first,d} for {_it->second,d} at {_usage,d}}}</DisplayString>
</Type>
<Type Name="boost::container::small_vector&lt;*&gt;">
<Expand>
<Item Name="[size]">m_holder.m_size</Item>
<ArrayItems>
<Size>m_holder.m_size</Size>
<ValuePointer>m_holder.m_start</ValuePointer>
</ArrayItems>
</Expand>
</Type>
</AutoVisualizer>