Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Make lock better handle process being killed
If the process gets killed and restarted (so that it didn't have a
chance to drop its locks gracefully) then there may still be locks in
the DB that are for the same instance that haven't yet timed out but are
safe to delete.
We handle this case by a) checking if the current instance already has
taken out the lock, and b) if not then ignoring locks that are for the
same instance.
* Periodically check for old staged events
This is to protect against other instances dying and their locks timing
out.
* Remove unused Vagrant scripts
* Change package Architecture to any
* Preinstall the wheel package when building venvs.
Addresses the following warnings during Debian builds:
Using legacy 'setup.py install' for jaeger-client, since package 'wheel' is not installed.
Using legacy 'setup.py install' for matrix-synapse-ldap3, since package 'wheel' is not installed.
Using legacy 'setup.py install' for opentracing, since package 'wheel' is not installed.
Using legacy 'setup.py install' for psycopg2, since package 'wheel' is not installed.
Using legacy 'setup.py install' for systemd-python, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pympler, since package 'wheel' is not installed.
Using legacy 'setup.py install' for threadloop, since package 'wheel' is not installed.
Using legacy 'setup.py install' for thrift, since package 'wheel' is not installed.
* Allow /etc/default/matrix-synapse to be missing
Per the systemd.exec manpage, prefixing an EnvironmentFile with "-":
> indicates that if the file does not exist, it will not be read and no
> error or warning message is logged.
Signed-off-by: Dan Callahan <danc@element.io>
When an event fetcher aborts due to an exception, `_event_fetch_ongoing`
must be decremented, otherwise the event fetcher would never be
replaced. If enough event fetchers were to fail, no more events would be
fetched and requests would get stuck waiting for events.
Before, we could rely on the `connected_insertion_event_query`
to navigate the and find the historical branch. But now
we solely rely on the marker event to point out the
historical branch. So we need to make sure to add
the insertion event extremeties whenever we see a marker event.
Whether it be a live event or backfilled.
We now rely on the marker events to backfill the base insertion event
which puts it as a insertion event extremity. This functionality was
already in place (see `handle_marker_event`) and was an easy transition.
This way, remote federated homeserver will have the insertion extremity
to ask about in backfill and goes down the historical branch no problem
because of the depth order and the rest of the DAG navigation happens as
normal. Yay simplification!
The key breakthrough was discussing all the ways we can find connected insertion events.
https://docs.google.com/document/d/1KCEmpnGr4J-I8EeaVQ8QJZKBDu53ViI7V62y5BzfXr0/edit#bookmark=id.1hbt9acs963h
The three options we came up were:
- Find by insertion event prev_events (this is what we were doing before)
- Find connected insertion events by depth
- Find connected insertion events by the marker event
- This made the most sense since we already backfill the insertion event
when a marker event is processed (see `handle_marker_event`).
- Gets rid of the extra insertion event lookup in backfill because we
know it's already backfilled from the marker processing.
- And gets rid of the extra federated lookup we added in this to PR
to ask whether the homeserver requesting backfill already has the
insertion event (deciding whether we fork
to the history branch before we go down the "live" DAG)
