MirrorHub/construct
0
0
Fork 0
mirror of https://github.com/matrix-construct/construct synced 2025-03-16 22:41:46 +01:00
Code Issues Releases Wiki Activity

ircd:Ⓜ️ Update README.

Browse source
This commit is contained in:
Jason Volk 2018-02-08 11:41:31 -08:00
parent f5adae46d1
commit 775f410e63
1 changed files with 32 additions and 54 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

86
include/ircd/m/README.md
View file

@ -265,6 +265,33 @@ An example of this is the search-terms database which specializes in indexing
individual words to the events where they are found so content searches can be
efficient.
#### Technique
The Matrix room, as described earlier, is a state machine underwritten by
timelines of events in a directed acyclic graph with eventual consistency.
To operate effectively, the state machine must respond to queries about
the state of the room at the point of any event in the past. This is similar
to a `git reset` to some specific commit where one can browse the work tree
as it appeared at that commit.
> Was X a member of room Y when event Z happened?
The naive approach is to trace the graph from the event backward collecting
all of the state events to then satisfy the query. Sequences of specific state
event types can be held by implementations to hasten such a trace.
Alternatively, a complete list of all state events can be created for each
modification to the state to avoid the reference chasing of a trace at the
cost of space.
Our approach is more efficient. We create a b-tree to represent the complete
state to satisfy any query in logarithmic time. When the state is updated,
only one path in the tree is modified and the root is stored with that event.
This structure is actually immutable: the previous versions of the affected
nodes are not discarded allowing past configurations of the tree to be
represented. We further benefit from the fact that each node is referenced by
the hash of its content for efficient reuse, as well as our database being
well compressed.
#### Flow
This is a single-writer/multiple-reader approach. The "core" is the only writer.
@ -289,43 +316,6 @@ This also gives us the benefit of a total serialization at this point.
//|// \\|\\
::::::::::::: <-- release sequence propagation cone
The evaluation phase ensures the event commitment will work: that the event
is valid, and that the event is a valid transition of the machine according
to the rules. This process may take some time and many yields and IO, even
network IO -- if the server lacks a warm cache. During the evaluation phase
locks and exclusions may be acquired to maintain the validity of the
evaluation state through writing at the expense of other contexts contending
for that resource.
> Many ircd::ctx are concurrently working their way through the core. The
> "velocity" is low when an ircd::ctx on this path may yield a lot for various
> IO and allow other events to be processed. The velocity increases when
> concurrent evaluation and reordering is no longer viable to maintain
> coherence. Any yielding of an ircd::ctx at a higher velocity risks stalling
> the whole core.
::::::: <-- event input (low velocity)
||||||| <-- evaluation process (low velocity)
\|/ <-- serialization process (higher velocity)
The write commitment saves the event to the database. This is a relatively
fast operation which probably won't even yield the ircd::ctx, and all
future reads to the database will see this write.
! <-- serial write commitment (highest velocity)
The release sequence broadcasts the event so its effects can be consumed.
This works by yielding the ircd::ctx so all consumers can view the event
and apply its effects for their feature module or send the event out to
clients. This is usually faster than it sounds, as the consumers try not to
hold up the release sequence for more than their first execution-slice,
and copy the event if their output rate is slower.
* <-- event revelation (higher velocity)
//|||\\
//|// \\|\\
::::::::::::: <-- release sequence propagation cone (low velocity)
The entire core commitment process relative to an event riding through it
on an ircd::ctx has a duration tolerable for something like a REST interface,
so the response to the user can wait for the commitment to succeed or fail
@ -352,9 +342,9 @@ them acquire any exclusivity which impede the others.
:::::::::::::
Close up of the charybdis's write head when tight to one schwarzschild-radius of
matrix room surface which propagates only one event through at a time.
Vertical tracks are contexts on their journey through each evaluation and exclusion
step to the core.
matrix room surface which propagates only one event through at a time. Vertical
tracks are contexts on their journey through each evaluation and exclusion step
to the core.
Input Events Phase
:::::::::::::::::::::::::::::::::::::::::::::::::::::: validation / dupcheck
@ -363,7 +353,7 @@ step to the core.
|||| ||||||||||||||| ||||||||||||||| ||||||||||||||||| head resolution
--|--|----|-|---|--|--|---|---|---|---------|---|---|- graph resolutions
----------|-|---|---------|-------|-----------------|- module evaluations
\ | | | | /
\ | | | | / post-commit prefetching
== ==============| | == Lowest velocity locks
\ | | /
== | | == Mid velocity locks
@ -378,10 +368,7 @@ step to the core.
Above, two contexts are illustrated as contending for the highest velocity
lock. The highest velocity lock is not held for significant time, as the
holder has very little work left to be done within the core, and will
release the lock to the other context quickly. The lower velocity locks
may have to be held longer, but are also less exclusive to all contexts.
lock. The highest velocity lock is not held for significant time.
* Singularity
[ ]
@ -404,15 +391,6 @@ each consumer context separated by the horizontal lines representing a context
switch from the perspective of the event travelling down. Each consumer
performs its task for how to propagate the commissioned event.
Each consumer has a shared-lock of the event which will hold up the completion
of the commitment until all consumers release that. The ideal consumer will only
hold their lock for a single context-slice while they play their part in applying
the event, like non-blocking copies to sockets etc. These consumers then go on
to do the rest of their output without the original event data which was memory
supplied by the evaluator (like an HTTP client). Then all locks acquired on
the entry side of the core can be released. The evaluator then gets the result
of the successful commitment.
#### Scaling
Scaling beyond the limit of a single CPU core can be done with multiple instances