MirrorHub/synapse
0
0
Fork 1
mirror of https://mau.dev/maunium/synapse.git synced 2024-11-17 07:21:37 +01:00
Code Issues Projects Releases Wiki Activity
16616 commits 5 branches 731 tags 91 MiB
Commit graph

2698 commits

Author SHA1 Message Date
Patrick Cloke
c2db6599c8
Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors (#7089). 2020-03-19 08:22:56 -04:00
Erik Johnston
a319cb1dd1
Change device list streams to have one row per ID (#7010) 
* Add 'device_lists_outbound_pokes' as extra table.

This makes sure we check all the relevant tables to get the current max
stream ID.

Currently not doing so isn't problematic as the max stream ID in
`device_lists_outbound_pokes` is the same as in `device_lists_stream`,
however that will change.

* Change device lists stream to have one row per id.

This will make it possible to process the streams more incrementally,
avoiding having to process large chunks at once.

* Change device list replication to match new semantics.

Instead of sending down batches of user ID/host tuples, send down a row
per entity (user ID or host).

* Newsfile

* Remove handling of multiple rows per ID

* Fix worker handling

* Comments from review
 2020-03-19 11:36:53 +00:00
Richard van der Hoff
e43e78b985 1.12.0rc1 2020-03-19 11:07:16 +00:00
Richard van der Hoff
e913823a22
Fix concurrent modification errors in pusher metrics (#7106) 
add a lock to try to make this metric actually work
 2020-03-19 10:28:49 +00:00
Richard van der Hoff
8c75667ad7
Add prometheus metrics for the number of active pushers (#7103) 2020-03-19 10:00:24 +00:00
Richard van der Hoff
443162e577
Move pusherpool startup into _base.setup (#7104) 
This should be safe to do on all workers/masters because it is guarded by
a config option which will ensure it is only actually done on the worker
assigned as a pusher.
 2020-03-19 09:48:45 +00:00
Erik Johnston
4a17a647a9
Improve get auth chain difference algorithm. (#7095) 
It was originally implemented by pulling the full auth chain of all
state sets out of the database and doing set comparison. However, that
can take a lot work if the state and auth chains are large.

Instead, lets try and fetch the auth chains at the same time and
calculate the difference on the fly, allowing us to bail early if all
the auth chains converge. Assuming that the auth chains do converge more
often than not, this should improve performance. Hopefully.
 2020-03-18 16:46:41 +00:00
Patrick Cloke
88b41986db
Add an option to the set password API to choose whether to logout other devices. (#7085) 2020-03-18 07:50:00 -04:00
Richard von Kellner
6d110ddea4
Update INSTALL.md updated CentOS8 install instructions (#6925) 2020-03-17 21:48:23 +00:00
Richard van der Hoff
c37db0211e
Share SSL contexts for non-federation requests (#7094) 
Extends #5794 etc to the SimpleHttpClient so that it also applies to non-federation requests.

Fixes #7092.
 2020-03-17 21:32:25 +00:00
The Stranjer
5e477c1deb
Set charset to utf-8 when adding headers for certain text content types (#7044) 
Fixes #7043
 2020-03-17 13:29:09 +00:00
Patrick Cloke
7581d30e9f
Remove unused federation endpoint (query_auth) (#7026) 2020-03-17 08:04:49 -04:00
Patrick Cloke
60724c46b7
Remove special casing of m.room.aliases events (#7034) 2020-03-17 07:37:04 -04:00
Richard van der Hoff
6a35046363 Revert "Add options to disable setting profile info for prevent changes. (#7053)" 
This reverts commit 54dd28621b, reversing
changes made to 6640460d05.
 2020-03-17 11:25:01 +00:00
Brendan Abolivier
7df04ca0e6
Populate the room version from state events (#7070) 
Fixes #7065 

This is basically the same as https://github.com/matrix-org/synapse/pull/6847 except it tries to populate events from `state_events` rather than `current_state_events`, since the latter might have been cleared from the state of some rooms too early, leaving them with a `NULL` room version.
 2020-03-16 22:31:47 +00:00
Brendan Abolivier
beb19cf61a
Fix buggy condition in account validity handler (#7074) 2020-03-16 12:16:30 +00:00
Brendan Abolivier
d8d91983bc
Merge pull request #7067 from matrix-org/babolivier/saml_error_moar 
Move the default SAML2 error HTML to a dedicated file
 2020-03-13 19:53:19 +00:00
Patrick Cloke
77d0a4507b
Add type annotations and comments to auth handler (#7063) 2020-03-12 11:36:27 -04:00
Brendan Abolivier
e55a240681
Changelog 2020-03-11 19:37:04 +00:00
Brendan Abolivier
1cde4cf3f1
Changelog 2020-03-11 18:03:56 +00:00
Brendan Abolivier
54dd28621b
Add options to disable setting profile info for prevent changes. (#7053) 2020-03-10 22:23:01 +00:00
Brendan Abolivier
156f271867
Changelog 2020-03-10 14:01:24 +00:00
dklimpel
885134529f updates after review 2020-03-09 22:09:29 +01:00
Brendan Abolivier
14b2ebe767
Merge pull request #7055 from matrix-org/babolivier/get_time_of_last_push_action_before 
Move get_time_of_last_push_action_before to the EventPushActionsWorkerStore
 2020-03-09 14:53:50 +00:00
Brendan Abolivier
f9e3a3f4d0
Changelog 
It's the same as in #6964 since it's the most likely cause of the bug
and that change hasn't been released yet.
 2020-03-09 14:21:01 +00:00
Patrick Cloke
06eb5cae08
Remove special auth and redaction rules for aliases events in experimental room ver. (#7037) 2020-03-09 08:58:25 -04:00
Patrick Cloke
66315d862f
Update routing of fallback auth in the worker docs. (#7048) 2020-03-09 07:19:24 -04:00
Brendan Abolivier
bbf725e7da
Merge pull request #7045 from matrix-org/babolivier/room_keys_check 
Make sure that is_verified is a boolean when processing room keys
 2020-03-09 09:54:48 +00:00
dklimpel
fb078f921b changelog 2020-03-08 15:19:07 +01:00
Neil Pilgrim
2bff4457d9
Add type hints to logging/context.py (#6309) 
* Add type hints to logging/context.py

Signed-off-by: neiljp (Neil Pilgrim) <github@kepier.clara.net>
 2020-03-07 17:57:26 +00:00
Neil Johnson
1d66dce83e
Break down monthly active users by appservice_id (#7030) 
* Break down monthly active users by appservice_id and emit via prometheus.

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2020-03-06 18:14:19 +00:00
Brendan Abolivier
a27056d539
Changelog 2020-03-06 11:06:47 +00:00
Patrick Cloke
87972f07e5
Convert remote key resource REST layer to async/await. (#7020) 2020-03-05 11:29:56 -05:00
Richard van der Hoff
78a15b1f9d
Store room_versions in EventBase objects (#6875) 
This is a bit fiddly because it all has to be done on one fell swoop:

* Wherever we create a new event, pass in the room version (and check it matches the format version)
* When we prune an event, use the room version of the unpruned event to create the pruned version.
* When we pass an event over the replication protocol, pass the room version over alongside it, and use it when deserialising the event again.
 2020-03-05 15:46:44 +00:00
Brendan Abolivier
83b6c69d3d
Changelog 2020-03-04 17:29:09 +00:00
Patrick Cloke
13892776ef
Allow deleting an alias if the user has sufficient power level (#6986) 2020-03-04 11:30:46 -05:00
Richard van der Hoff
8ef8fb2c1c
Read the room version from database when fetching events (#6874) 
This is a precursor to giving EventBase objects the knowledge of which room version they belong to.
 2020-03-04 13:11:04 +00:00
Brendan Abolivier
43f874055d
Merge branch 'master' into develop 2020-03-03 15:20:49 +00:00
Brendan Abolivier
fd983fad96 v1.11.1 2020-03-03 14:58:37 +00:00
Patrick Cloke
7dcbc33a1b
Validate the alt_aliases property of canonical alias events (#6971) 2020-03-03 07:12:45 -05:00
Erik Johnston
65a941d1f8 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fixup_devices_stream 2020-03-02 16:55:55 +00:00
Erik Johnston
b29474e0aa
Always return a deferred from get_current_state_deltas. (#7019) 
This currently causes presence notify code to log exceptions when there
is no state changes to process. This doesn't actually cause any problems
as we'd simply do nothing anyway.
 2020-03-02 16:52:15 +00:00
Erik Johnston
3ab8e9c293
Fix py35-old CI by using native tox. (#7018) 
I'm not really sure how this was going wrong, but this seems like the
right approach anyway.
 2020-03-02 16:17:11 +00:00
Richard van der Hoff
174aaa1d62 remove spurious changelog 2020-03-02 14:53:56 +00:00
Dirk Klimpel
bbeee33d63 Fixed set a user as an admin with the new API (#6928) 
Fix #6910
 2020-03-02 13:28:50 +00:00
Uday Bansal
e4ffb14d57
Fix last date for ACMEv1 install (#7015) 
Support for getting TLS certificates through ACMEv1 ended on November 2019.

Signed-off-by: Uday Bansal <43824981+udaybansal19@users.noreply.github.com>
 2020-02-29 23:37:23 +00:00
Patrick Cloke
12d4259000
Add some type annotations to the federation base & client classes (#6995) 2020-02-28 07:31:07 -05:00
Erik Johnston
59ad93d2a4 Newsfile 2020-02-28 11:27:37 +00:00
Dirk Klimpel
9b06d8f8a6
Fixed set a user as an admin with the new API (#6928) 
Fix #6910
 2020-02-28 09:58:05 +00:00
Patrick Cloke
ab0073a6c0 Merge remote-tracking branch 'origin/release-v1.11.1' into develop 2020-02-27 13:47:44 -05:00
Erik Johnston
2201bc9795
Don't refuse to start worker if media listener configured. (#7002) 
Instead lets just warn if the worker has a media listener configured but
has the media repository disabled.

Previously non media repository workers would just ignore the media
listener.
 2020-02-27 16:33:21 +00:00
Richard van der Hoff
cab4a52535
set worker_app for frontend proxy test (#7003) 
to stop the federationhandler trying to do master stuff
 2020-02-27 13:08:43 +00:00
James
b32ac60c22
Expose common commands via snap run interface to allow easier invocation (#6315) 
Signed-off-by: James Hebden <james@ec0.io>
 2020-02-27 12:47:40 +00:00
Richard van der Hoff
132b673dbe
Add some type annotations in synapse.storage (#6987) 
I cracked, and added some type definitions in synapse.storage.
 2020-02-27 11:53:40 +00:00
Richard van der Hoff
3e99528f2b
Store room version on invite (#6983) 
When we get an invite over federation, store the room version in the rooms table.

The general idea here is that, when we pull the invite out again, we'll want to know what room_version it belongs to (so that we can later redact it if need be). So we need to store it somewhere...
 2020-02-26 16:58:33 +00:00
Patrick Cloke
380122866f
Cast a coroutine into a Deferred in the federation base (#6996) 
Properly convert a coroutine into a Deferred in federation_base to fix an error when joining a room.
 2020-02-26 11:32:13 -05:00
Erik Johnston
1f773eec91
Port PresenceHandler to async/await (#6991) 2020-02-26 15:33:26 +00:00
Uday Bansal
7728d87fd7
Updated warning for incorrect database collation/ctype (#6985) 
Signed-off-by: Uday Bansal <43824981+udaybansal19@users.noreply.github.com>
 2020-02-26 15:17:03 +00:00
Andrew Morgan
8c75b621bf
Ensure 'deactivated' parameter is a boolean on user admin API, Fix error handling of call to deactivate user (#6990) 2020-02-26 12:22:55 +00:00
Richard van der Hoff
e66f099ca9
Sanity-check database before running upgrades (#6982) 
Some of the database deltas rely on `config.server_name` being set correctly,
so we should check that it is before running the deltas.

Fixes #6870.
 2020-02-25 17:46:00 +00:00
Erik Johnston
bbf8886a05
Merge worker apps into one. (#6964) 2020-02-25 16:56:55 +00:00
Fridtjof Mund
4aea0bd292
contrib/docker: remove quotes for POSTGRES_INITDB_ARGS (#6984) 
I made a mistake in https://github.com/matrix-org/synapse/pull/6921 - the quotes break the postgres container's startup script (or docker-compose), which makes initdb fail: https://github.com/matrix-org/synapse/pull/6921#issuecomment-590657154

Signed-off-by: Fridtjof Mund <fridtjof@das-labor.org>
 2020-02-25 10:48:13 +00:00
Richard van der Hoff
691659568f
Remove redundant store_room call (#6979) 
`_process_received_pdu` is only called by `on_receive_pdu`, which ignores any
events for unknown rooms, so this is redundant.
 2020-02-24 17:20:44 +00:00
Richard van der Hoff
a301934f46
Upsert room version when we join over federation (#6968) 
This is intended as a precursor to storing room versions when we receive an
invite over federation, but has the happy side-effect of fixing #3374 at last.

In short: change the store_room with try/except to a proper upsert which
updates the right columns.
 2020-02-24 15:46:41 +00:00
Richard van der Hoff
4c2ed3f20e
Fix minor issues with email config (#6962) 
* Give `notif_template_html`, `notif_template_text` default values (fixes #6960)
 * Don't complain if `smtp_host` and `smtp_port` are unset, since they have sensible defaults (fixes #6961)
 * Set the example for `enable_notifs` to `True`, for consistency and because it's more useful
 * Raise errors as ConfigError rather than RuntimeError for nicer formatting
 2020-02-24 15:18:38 +00:00
Patrick Cloke
af6c389501
No longer use room alias events to calculate room names for push notifications. (#6966) 2020-02-21 12:50:48 -05:00
Dirk Klimpel
7b0e2d961c
Change displayname of user as admin in rooms (#6876) 2020-02-21 17:44:03 +00:00
Patrick Cloke
fcf4599488
Stop returning aliases as part of the room list. (#6970) 2020-02-21 12:40:23 -05:00
Patrick Cloke
7936d2a96e
Publishing/removing from the directory requires a power level greater than canonical aliases. 2020-02-21 07:18:33 -05:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) 
Ensure good comprehension hygiene using flake8-comprehensions.
 2020-02-21 07:15:07 -05:00
Richard van der Hoff
1fcb9a1a7a changelog 2020-02-21 09:06:18 +00:00
Richard van der Hoff
9c1b83b007 1.11.0 2020-02-21 08:56:04 +00:00
Andrew Morgan
8f6d9c4cf0
Small grammar fixes to the ACME v1 deprecation notice (#6944) 
Some small fixes to the copy in #6907.
 2020-02-21 08:53:01 +00:00
Patrick Cloke
99eed85a77
Do not send alias events when creating / upgrading a room (#6941) 
Stop emitting room alias update events during room creation/upgrade.
 2020-02-20 16:24:04 -05:00
Hubert Chathi
a90d0dc5c2
don't insert into the device table for remote cross-signing keys (#6956) 2020-02-20 09:59:00 -05:00
Ruben Barkow-Kuder
4fb5f4d0ce
Add some clarifications to README.md in the database schema directory. (#6615) 
Signed-off-by: Ruben Barkow-Kuder <github@r.z11.de>
 2020-02-20 10:37:57 +00:00
Erik Johnston
7b7c3cedf2 Minor perf fixes to get_auth_chain_ids. 2020-02-19 15:47:11 +00:00
Erik Johnston
fc87d2ffb3
Freeze allocated objects on startup. (#6953) 
This may make gc go a bit faster as the gc will know things like
caches/data stores etc. are frozen without having to check.
 2020-02-19 15:09:00 +00:00
Erik Johnston
2b37eabca1
Reduce auth chains fetched during v2 state res. (#6952) 
The state res v2 algorithm only cares about the difference between auth
chains, so we can pass in the known common state to the `get_auth_chain`
storage function so that it can ignore those events.
 2020-02-19 15:04:47 +00:00
Richard van der Hoff
197b08de35 1.11.0rc1 2020-02-19 13:48:32 +00:00
Erik Johnston
099c96b89b
Revert get_auth_chain_ids changes (#6951) 2020-02-19 11:37:35 +00:00
Richard van der Hoff
2fb7794e60
Merge pull request #6949 from matrix-org/rav/list_room_aliases_peekable 
Make room alias lists peekable
 2020-02-19 11:19:11 +00:00
Brendan Abolivier
bbe39f808c
Merge pull request #6940 from matrix-org/babolivier/federate.md 
Clean up and update federation docs
 2020-02-19 10:58:59 +00:00
Richard van der Hoff
880aaac1d8
Move MSC2432 stuff onto unstable prefix (#6948) 
it's not in the spec yet, so needs to be unstable. Also add a feature flag for it. Also add a test for admin users.
 2020-02-19 10:40:27 +00:00
Richard van der Hoff
abf1e5c526
Tiny optimisation for _get_handler_for_request (#6950) 
we have hundreds of path_regexes (see #5118), so let's not convert the same
bytes to str for each of them.
 2020-02-19 10:38:20 +00:00
Erik Johnston
0d0bc35792
Increase DB/CPU perf of _is_server_still_joined check. (#6936) 
* Increase DB/CPU perf of `_is_server_still_joined` check.

For rooms with large amount of state a single user leaving could cause
us to go and load a lot of membership events and then pull out
membership state in a large number of batches.

* Newsfile

* Update synapse/storage/persist_events.py

Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Fix adding if too soon

* Update docstring

* Review comments

* Woops typo

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2020-02-19 10:15:49 +00:00
Brendan Abolivier
5e4a438556
Merge pull request #6945 from matrix-org/babolivier/fix-retention-debug-log 
Fix log in message retention purge jobs
 2020-02-19 10:12:55 +00:00
Brendan Abolivier
fa64f836ec
Update changelog.d/6945.bugfix 
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2020-02-19 09:54:13 +00:00
Erik Johnston
5a5abd55e8
Limit size of get_auth_chain_ids query (#6947) 2020-02-19 09:39:26 +00:00
Richard van der Hoff
603618c002 changelog 2020-02-19 08:53:32 +00:00
Brendan Abolivier
771d70e89c
Changelog 2020-02-18 17:31:02 +00:00
Richard van der Hoff
adfaea8c69
Implement GET /_matrix/client/r0/rooms/{roomId}/aliases (#6939) 
per matrix-org/matrix-doc#2432
 2020-02-18 16:23:25 +00:00
Richard van der Hoff
3f1cd14791
Merge pull request #6872 from matrix-org/rav/dictproperty 
Rewrite _EventInternalMetadata to back it with a dict
 2020-02-18 16:21:02 +00:00
Brendan Abolivier
d484126bf7
Merge pull request #6907 from matrix-org/babolivier/acme-config 
Add mention and warning about ACME v1 deprecation to the TLS config
 2020-02-18 16:11:31 +00:00
Erik Johnston
8a380d0fe2
Increase perf of get_auth_chain_ids used in state res v2. (#6937) 
We do this by moving the recursive query to be fully in the DB.
 2020-02-18 15:39:09 +00:00
Erik Johnston
818def8248
Fix worker docs to point /publicised_groups API correctly. (#6938) 2020-02-18 15:27:45 +00:00
Brendan Abolivier
7a782c32a2
Merge pull request #6909 from matrix-org/babolivier/acme-install 
Update INSTALL.md to recommend reverse proxying and warn about ACMEv1 deprecation
 2020-02-18 15:06:06 +00:00
Brendan Abolivier
b1255077f5 Changelog 2020-02-18 14:35:51 +00:00
Patrick Cloke
fe3941f6e3
Stop sending events when creating or deleting aliases (#6904) 
Stop sending events when creating or deleting associations (room aliases). Send an updated canonical alias event if one of the alt_aliases is deleted.
 2020-02-18 07:29:44 -05:00
Richard van der Hoff
3be2abd0a9
Kill off deprecated "config-on-the-fly" docker mode (#6918) 
Lots of people seem to get confused by this mode, and it's been deprecated
since Synapse 1.1.0. It's time for it to go.
 2020-02-18 11:41:53 +00:00