this has been merged into matrix-doc/specification/30_server_server_api.rst
This commit is contained in:
Matthew Hodgson
parent
61ecb13bf0
commit
bebca337c4
|
|
@ -1,231 +0,0 @@
|
|||
===========================
|
||||
Matrix Server-to-Server API
|
||||
===========================
|
||||
|
||||
A description of the protocol used to communicate between Matrix home servers;
|
||||
also known as Federation.
|
||||
|
||||
|
||||
Overview
|
||||
========
|
||||
|
||||
The server-server API is a mechanism by which two home servers can exchange
|
||||
Matrix event messages, both as a real-time push of current events, and as a
|
||||
historic fetching mechanism to synchronise past history for clients to view. It
|
||||
uses HTTP connections between each pair of servers involved as the underlying
|
||||
transport. Messages are exchanged between servers in real-time by active pushing
|
||||
from each server's HTTP client into the server of the other. Queries to fetch
|
||||
historic data for the purpose of back-filling scrollback buffers and the like
|
||||
can also be performed.
|
||||
|
||||
|
||||
{ Matrix clients } { Matrix clients }
|
||||
^ | ^ |
|
||||
| events | | events |
|
||||
| V | V
|
||||
+------------------+ +------------------+
|
||||
| |---------( HTTP )---------->| |
|
||||
| Home Server | | Home Server |
|
||||
| |<--------( HTTP )-----------| |
|
||||
+------------------+ +------------------+
|
||||
|
||||
There are three main kinds of communication that occur between home servers:
|
||||
|
||||
* Queries
|
||||
These are single request/response interactions between a given pair of
|
||||
servers, initiated by one side sending an HTTP request to obtain some
|
||||
information, and responded by the other. They are not persisted and contain
|
||||
no long-term significant history. They simply request a snapshot state at the
|
||||
instant the query is made.
|
||||
|
||||
* EDUs - Ephemeral Data Units
|
||||
These are notifications of events that are pushed from one home server to
|
||||
another. They are not persisted and contain no long-term significant history,
|
||||
nor does the receiving home server have to reply to them.
|
||||
|
||||
* PDUs - Persisted Data Units
|
||||
These are notifications of events that are broadcast from one home server to
|
||||
any others that are interested in the same "context" (namely, a Room ID).
|
||||
They are persisted to long-term storage and form the record of history for
|
||||
that context.
|
||||
|
||||
Where Queries are presented directly across the HTTP connection as GET requests
|
||||
to specific URLs, EDUs and PDUs are further wrapped in an envelope called a
|
||||
Transaction, which is transferred from the origin to the destination home server
|
||||
using a PUT request.
|
||||
|
||||
|
||||
Transactions and EDUs/PDUs
|
||||
==========================
|
||||
|
||||
The transfer of EDUs and PDUs between home servers is performed by an exchange
|
||||
of Transaction messages, which are encoded as JSON objects with a dict as the
|
||||
top-level element, passed over an HTTP PUT request. A Transaction is meaningful
|
||||
only to the pair of home servers that exchanged it; they are not globally-
|
||||
meaningful.
|
||||
|
||||
Each transaction has an opaque ID and timestamp (UNIX epoch time in
|
||||
milliseconds) generated by its origin server, an origin and destination server
|
||||
name, a list of "previous IDs", and a list of PDUs - the actual message payload
|
||||
that the Transaction carries.
|
||||
|
||||
{"transaction_id":"916d630ea616342b42e98a3be0b74113",
|
||||
"ts":1404835423000,
|
||||
"origin":"red",
|
||||
"destination":"blue",
|
||||
"prev_ids":["e1da392e61898be4d2009b9fecce5325"],
|
||||
"pdus":[...],
|
||||
"edus":[...]}
|
||||
|
||||
The "previous IDs" field will contain a list of previous transaction IDs that
|
||||
the origin server has sent to this destination. Its purpose is to act as a
|
||||
sequence checking mechanism - the destination server can check whether it has
|
||||
successfully received that Transaction, or ask for a retransmission if not.
|
||||
|
||||
The "pdus" field of a transaction is a list, containing zero or more PDUs.[*]
|
||||
Each PDU is itself a dict containing a number of keys, the exact details of
|
||||
which will vary depending on the type of PDU. Similarly, the "edus" field is
|
||||
another list containing the EDUs. This key may be entirely absent if there are
|
||||
no EDUs to transfer.
|
||||
|
||||
(* Normally the PDU list will be non-empty, but the server should cope with
|
||||
receiving an "empty" transaction, as this is useful for informing peers of other
|
||||
transaction IDs they should be aware of. This effectively acts as a push
|
||||
mechanism to encourage peers to continue to replicate content.)
|
||||
|
||||
All PDUs have an ID, a context, a declaration of their type, a list of other PDU
|
||||
IDs that have been seen recently on that context (regardless of which origin
|
||||
sent them), and a nested content field containing the actual event content.
|
||||
|
||||
[[TODO(paul): Update this structure so that 'pdu_id' is a two-element
|
||||
[origin,ref] pair like the prev_pdus are]]
|
||||
|
||||
{"pdu_id":"a4ecee13e2accdadf56c1025af232176",
|
||||
"context":"#example.green",
|
||||
"origin":"green",
|
||||
"ts":1404838188000,
|
||||
"pdu_type":"m.text",
|
||||
"prev_pdus":[["blue","99d16afbc857975916f1d73e49e52b65"]],
|
||||
"content":...
|
||||
"is_state":false}
|
||||
|
||||
In contrast to the transaction layer, it is important to note that the prev_pdus
|
||||
field of a PDU refers to PDUs that any origin server has sent, rather than
|
||||
previous IDs that this origin has sent. This list may refer to other PDUs sent
|
||||
by the same origin as the current one, or other origins.
|
||||
|
||||
Because of the distributed nature of participants in a Matrix conversation, it
|
||||
is impossible to establish a globally-consistent total ordering on the events.
|
||||
However, by annotating each outbound PDU at its origin with IDs of other PDUs it
|
||||
has received, a partial ordering can be constructed allowing causallity
|
||||
relationships to be preserved. A client can then display these messages to the
|
||||
end-user in some order consistent with their content and ensure that no message
|
||||
that is semantically in reply of an earlier one is ever displayed before it.
|
||||
|
||||
PDUs fall into two main categories: those that deliver Events, and those that
|
||||
synchronise State. For PDUs that relate to State synchronisation, additional
|
||||
keys exist to support this:
|
||||
|
||||
{...,
|
||||
"is_state":true,
|
||||
"state_key":TODO
|
||||
"power_level":TODO
|
||||
"prev_state_id":TODO
|
||||
"prev_state_origin":TODO}
|
||||
|
||||
[[TODO(paul): At this point we should probably have a long description of how
|
||||
State management works, with descriptions of clobbering rules, power levels, etc
|
||||
etc... But some of that detail is rather up-in-the-air, on the whiteboard, and
|
||||
so on. This part needs refining. And writing in its own document as the details
|
||||
relate to the server/system as a whole, not specifically to server-server
|
||||
federation.]]
|
||||
|
||||
EDUs, by comparison to PDUs, do not have an ID, a context, or a list of
|
||||
"previous" IDs. The only mandatory fields for these are the type, origin and
|
||||
destination home server names, and the actual nested content.
|
||||
|
||||
{"edu_type":"m.presence",
|
||||
"origin":"blue",
|
||||
"destination":"orange",
|
||||
"content":...}
|
||||
|
||||
|
||||
Protocol URLs
|
||||
=============
|
||||
|
||||
All these URLs are namespaced within a prefix of
|
||||
|
||||
/_matrix/federation/v1/...
|
||||
|
||||
For active pushing of messages representing live activity "as it happens":
|
||||
|
||||
PUT .../send/:transaction_id/
|
||||
Body: JSON encoding of a single Transaction
|
||||
|
||||
Response: [[TODO(paul): I don't actually understand what
|
||||
ReplicationLayer.on_transaction() is doing here, so I'm not sure what the
|
||||
response ought to be]]
|
||||
|
||||
The transaction_id path argument will override any ID given in the JSON body.
|
||||
The destination name will be set to that of the receiving server itself. Each
|
||||
embedded PDU in the transaction body will be processed.
|
||||
|
||||
|
||||
To fetch a particular PDU:
|
||||
|
||||
GET .../pdu/:origin/:pdu_id/
|
||||
|
||||
Response: JSON encoding of a single Transaction containing one PDU
|
||||
|
||||
Retrieves a given PDU from the server. The response will contain a single new
|
||||
Transaction, inside which will be the requested PDU.
|
||||
|
||||
|
||||
To fetch all the state of a given context:
|
||||
|
||||
GET .../state/:context/
|
||||
|
||||
Response: JSON encoding of a single Transaction containing multiple PDUs
|
||||
|
||||
Retrieves a snapshot of the entire current state of the given context. The
|
||||
response will contain a single Transaction, inside which will be a list of
|
||||
PDUs that encode the state.
|
||||
|
||||
|
||||
To backfill events on a given context:
|
||||
|
||||
GET .../backfill/:context/
|
||||
Query args: v, limit
|
||||
|
||||
Response: JSON encoding of a single Transaction containing multiple PDUs
|
||||
|
||||
Retrieves a sliding-window history of previous PDUs that occurred on the
|
||||
given context. Starting from the PDU ID(s) given in the "v" argument, the
|
||||
PDUs that preceeded it are retrieved, up to a total number given by the
|
||||
"limit" argument. These are then returned in a new Transaction containing all
|
||||
off the PDUs.
|
||||
|
||||
|
||||
To stream events all the events:
|
||||
|
||||
GET .../pull/
|
||||
Query args: origin, v
|
||||
|
||||
Response: JSON encoding of a single Transaction consisting of multiple PDUs
|
||||
|
||||
Retrieves all of the transactions later than any version given by the "v"
|
||||
arguments. [[TODO(paul): I'm not sure what the "origin" argument does because
|
||||
I think at some point in the code it's got swapped around.]]
|
||||
|
||||
|
||||
To make a query:
|
||||
|
||||
GET .../query/:query_type
|
||||
Query args: as specified by the individual query types
|
||||
|
||||
Response: JSON encoding of a response object
|
||||
|
||||
Performs a single query request on the receiving home server. The Query Type
|
||||
part of the path specifies the kind of query being made, and its query
|
||||
arguments have a meaning specific to that kind of query. The response is a
|
||||
JSON-encoded object whose meaning also depends on the kind of query.
|
||||
Loading…
Reference in a new issue