Patrick Cloke
f594e434c3
Switch the JSON byte producer from a pull to a push producer. ( #8116 )
2020-08-19 08:07:57 -04:00
Patrick Cloke
d294f0e7e1
Remove the unused inlineCallbacks code-paths in the caching code ( #8119 )
2020-08-19 07:09:07 -04:00
Erik Johnston
76d21d14a0
Separate get_current_token into two. ( #8113 )
...
The function is used for two purposes: 1) for subscribers of streams to
get a token they can use to get further updates with, and 2) for
replication to track position of the writers of the stream.
For streams with a single writer the two scenarios produce the same
result, however the situation becomes complicated for streams with
multiple writers. The current `MultiWriterIdGenerator` does not
correctly handle the first case (which is not an issue as its only used
for the `caches` stream which nothing subscribes to outside of
replication).
2020-08-19 10:39:31 +01:00
Patrick Cloke
f40645e60b
Convert events worker database to async/await. ( #8071 )
2020-08-18 16:20:49 -04:00
Andrew Morgan
5cf7c12995
Remove : from allowed client_secret chars ( #8101 )
...
Closes: https://github.com/matrix-org/synapse/issues/6766
Equivalent Sydent PR: https://github.com/matrix-org/sydent/pull/309
I believe it's now time to remove the extra allowed `:` from `client_secret` parameters.
2020-08-18 14:14:27 +01:00
Patrick Cloke
2f4d60a5ba
Iteratively encode JSON responses to avoid blocking the reactor. ( #8013 )
2020-08-18 08:49:59 -04:00
Patrick Cloke
050e20e7ca
Convert some of the general database methods to async ( #8100 )
2020-08-17 12:18:01 -04:00
Andrew Morgan
e04e465b4d
Use the default templates when a custom template file cannot be found ( #8037 )
...
Fixes https://github.com/matrix-org/synapse/issues/6583
2020-08-17 17:05:00 +01:00
Patrick Cloke
ad6190c925
Convert stream database to async/await. ( #8074 )
2020-08-17 07:24:46 -04:00
Patrick Cloke
ac77cdb64e
Add a shadow-banned flag to users. ( #8092 )
2020-08-14 12:37:59 -04:00
Patrick Cloke
dd8f28bd3f
Fix unawaited coroutine error in tests. ( #8072 )
2020-08-13 07:11:39 -04:00
Patrick Cloke
fbe930dad2
Convert the roommember database to async/await. ( #8070 )
2020-08-12 12:14:34 -04:00
Patrick Cloke
5ecc8b5825
Convert devices database to async/await. ( #8069 )
2020-08-12 10:51:42 -04:00
Erik Johnston
5dd73d029e
Add type hints to handlers.message and events.builder ( #8067 )
2020-08-12 15:05:50 +01:00
Patrick Cloke
d68e10f308
Convert account data, device inbox, and censor events databases to async/await ( #8063 )
2020-08-12 09:29:06 -04:00
Patrick Cloke
a3a59bab7b
Convert appservice, group server, profile and more databases to async ( #8066 )
2020-08-12 09:28:48 -04:00
Patrick Cloke
04faa0bfa9
Convert tags and metrics databases to async/await ( #8062 )
2020-08-11 17:21:20 -04:00
Patrick Cloke
a0acdfa9e9
Converts event_federation and registration databases to async/await ( #8061 )
2020-08-11 17:21:13 -04:00
Richard van der Hoff
fcbab08cbd
Add an assertion on prev_events in create_new_client_event ( #8041 )
...
I think this would have caught all the cases in
https://github.com/matrix-org/synapse/issues/7642 - and I think a 500 makes
more sense here than a 403
2020-08-10 12:29:47 +01:00
Patrick Cloke
7f837959ea
Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async ( #8042 )
2020-08-07 13:36:29 -04:00
Patrick Cloke
f3fe6961b2
Convert additional database stores to async/await ( #8045 )
2020-08-07 12:17:17 -04:00
Erik Johnston
7620912d84
Add health check endpoint ( #8048 )
2020-08-07 14:21:24 +01:00
Brendan Abolivier
2ffd6783c7
Revert #7736 ( #8039 )
2020-08-06 17:15:35 +01:00
Patrick Cloke
fe6cfc80ec
Convert some util functions to async ( #8035 )
2020-08-06 08:39:35 -04:00
Patrick Cloke
d4a7829b12
Convert synapse.api to async/await ( #8031 )
2020-08-06 08:30:06 -04:00
Patrick Cloke
66f24449dd
Improve performance of the register endpoint ( #8009 )
2020-08-06 08:09:55 -04:00
Erik Johnston
a7bdf98d01
Rename database classes to make some sense ( #8033 )
2020-08-05 21:38:57 +01:00
Patrick Cloke
e19de43eb5
Convert streams to async. ( #8014 )
2020-08-04 07:21:47 -04:00
Andrew Morgan
5d92a1428c
Prevent join->join membership transitions changing member count ( #7977 )
...
`StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc.
However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information.
This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point.
Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update).
Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338 .
2020-08-03 21:54:24 +01:00
Patrick Cloke
6812509807
Implement handling of HTTP HEAD requests. ( #7999 )
2020-08-03 08:45:42 -04:00
Patrick Cloke
2a89ce8cd4
Convert the crypto module to async/await. ( #8003 )
2020-08-03 08:29:01 -04:00
Erik Johnston
faba873d4b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/add_rate_limiting_to_joins
2020-07-31 15:07:01 +01:00
Erik Johnston
18de00adb4
Add ratelimiting on joins
2020-07-31 15:06:56 +01:00
Patrick Cloke
c978f6c451
Convert federation client to async/await. ( #7975 )
2020-07-30 08:01:33 -04:00
Patrick Cloke
4cce8ef74e
Convert appservice to async. ( #7973 )
2020-07-30 07:27:39 -04:00
Patrick Cloke
b3a97d6dac
Convert some of the data store to async. ( #7976 )
2020-07-30 07:20:41 -04:00
Brendan Abolivier
8dff4a1242
Re-implement unread counts ( #7736 )
2020-07-29 18:26:55 +01:00
Patrick Cloke
3345c166a4
Convert storage layer to async/await. ( #7963 )
2020-07-28 16:09:53 -04:00
Dirk Klimpel
e866e3b896
Add an option to disable purge in delete room admin API ( #7964 )
...
Add option ```purge``` to ```POST /_synapse/admin/v1/rooms/<room_id>/delete```
Fixes : #3761
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-07-28 20:08:23 +01:00
lugino-emeritus
3857de2194
Option to allow server admins to join complex rooms ( #7902 )
...
Fixes #7901 .
Signed-off-by: Niklas Tittjung <nik_t.01@web.de>
2020-07-28 13:41:44 +01:00
Patrick Cloke
8553f46498
Convert a synapse.events to async/await. ( #7949 )
2020-07-27 13:40:22 -04:00
Patrick Cloke
5f65e62681
Convert groups and visibility code to async / await. ( #7951 )
2020-07-27 12:32:08 -04:00
Patrick Cloke
8144bc26a7
Convert push to async/await. ( #7948 )
2020-07-27 12:21:34 -04:00
Andrew Morgan
c4268e3da6
Convert tests/rest/admin/test_room.py to unix file endings ( #7953 )
...
Converts tests/rest/admin/test_room.py to have unix file endings after they were accidentally changed in #7613 .
Keeping the same changelog as #7613 as it hasn't gone out in a release yet.
2020-07-27 13:22:52 +01:00
Patrick Cloke
3fc8fdd150
Support oEmbed for media previews. ( #7920 )
...
Fixes previews of Twitter URLs by using their oEmbed endpoint to grab content.
2020-07-27 07:50:44 -04:00
Patrick Cloke
b975fa2e99
Convert state resolution to async/await ( #7942 )
2020-07-24 10:59:51 -04:00
Patrick Cloke
5ea29d7f85
Convert more of the media code to async/await ( #7873 )
2020-07-24 09:39:02 -04:00
Patrick Cloke
6a080ea184
Return an empty body for OPTIONS requests. ( #7886 )
2020-07-24 07:08:07 -04:00
Richard van der Hoff
1ec688bf21
Downgrade warning on client disconnect to INFO ( #7928 )
...
Clients disconnecting before we finish processing the request happens from time
to time. We don't need to yell about it
2020-07-24 09:55:47 +01:00
Patrick Cloke
68cd935826
Convert the federation agent and related code to async/await. ( #7874 )
2020-07-23 07:05:57 -04:00
Patrick Cloke
cc9bb3dc3f
Convert the message handler to async/await. ( #7884 )
2020-07-22 12:29:15 -04:00
Patrick Cloke
6b3ac3b8cd
Convert device handler to async/await ( #7871 )
2020-07-17 07:09:25 -04:00
Patrick Cloke
6fca1b3506
Convert _base, profile, and _receipts handlers to async/await ( #7860 )
2020-07-17 07:08:30 -04:00
Michael Albert
fff483ea96
Add admin endpoint to get members in a room. ( #7842 )
2020-07-16 16:43:23 -04:00
Erik Johnston
649a7ead5c
Add ability to run multiple pusher instances ( #7855 )
...
This reuses the same scheme as federation sender sharding
2020-07-16 14:06:28 +01:00
Erik Johnston
97e1159ac1
Merge branch 'erikj/faster_typing' of github.com:matrix-org/synapse into develop
2020-07-15 16:54:30 +01:00
Patrick Cloke
8c7d0f163d
Allow accounts to be re-activated from the admin APIs. ( #7847 )
2020-07-15 11:00:21 -04:00
Erik Johnston
9006e125af
Fix tests
2020-07-15 15:47:27 +01:00
Erik Johnston
f13061d515
Fix client reader sharding tests ( #7853 )
...
* Fix client reader sharding tests
* Newsfile
* Fix typing
* Update changelog.d/7853.misc
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Move mocking of http_client to tests
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2020-07-15 15:27:35 +01:00
Patrick Cloke
b11450dedc
Convert E2E key and room key handlers to async/await. ( #7851 )
2020-07-15 08:48:58 -04:00
Patrick Cloke
111e70d75c
Return the proper 403 Forbidden error during errors with JWT logins. ( #7844 )
2020-07-15 07:10:21 -04:00
Dirk Klimpel
491f0dab1b
Add delete room admin endpoint ( #7613 )
...
The Delete Room admin API allows server admins to remove rooms from server
and block these rooms.
`DELETE /_synapse/admin/v1/rooms/<room_id>`
It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md )" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md )" API.
Fixes : #6425
It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`.
It should return `None` if the room is unknown. But it returns an `IndexError`.
901b1fa561/synapse/storage/data_stores/main/room.py (L99-L105)#5575
- https://github.com/Awesome-Technologies/synapse-admin/issues/17
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-07-14 12:36:23 +01:00
Patrick Cloke
77d2c05410
Add the option to validate the iss and aud claims for JWT logins. ( #7827 )
2020-07-14 07:16:43 -04:00
Patrick Cloke
457096e6df
Support handling registration requests across multiple client readers. ( #7830 )
2020-07-13 13:31:46 -04:00
Patrick Cloke
66a4af8d96
Do not use canonicaljson to magically handle decoding bytes from JSON. ( #7802 )
2020-07-10 14:30:08 -04:00
Erik Johnston
f299441cc6
Add ability to shard the federation sender ( #7798 )
2020-07-10 18:26:36 +01:00
Patrick Cloke
38e1fac886
Fix some spelling mistakes / typos. ( #7811 )
2020-07-09 09:52:58 -04:00
Richard van der Hoff
67593b1728
Add HomeServer.signing_key property ( #7805 )
...
... instead of duplicating `config.signing_key[0]` everywhere
2020-07-08 17:51:56 +01:00
Richard van der Hoff
76dbd7b8d6
Stop populating unused table local_invites. ( #7793 )
...
This table is no longer used, so we may as well stop populating it. Removing it
would prevent people rolling back to older releases of Synapse, so that can
happen in a future release.
2020-07-07 14:20:40 +01:00
Patrick Cloke
6d687ebba1
Convert the appservice handler to async/await. ( #7775 )
2020-07-06 07:40:35 -04:00
reivilibre
57feeab364
Don't ignore set_tweak actions with no explicit value. ( #7766 )
...
* Fix spec compliance; tweaks without values are valid
(default to True, which is only concretely specified for
`highlight`, but it seems only reasonable to generalise)
* Changelog for 7766.
* Add documentation to `tweaks_for_actions`
May as well tidy up when I'm here.
* Add a test for `tweaks_for_actions`
2020-07-06 11:43:41 +01:00
Will Hunt
62b1ce8539
isort 5 compatibility ( #7786 )
...
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
Erik Johnston
5cdca53aa0
Merge different Resource implementation classes ( #7732 )
2020-07-03 19:02:19 +01:00
Dirk Klimpel
21a212f8e5
Fix inconsistent handling of upper and lower cases of email addresses. ( #7021 )
...
fixes #7016
2020-07-03 14:03:13 +01:00
reivilibre
e5808c4cfb
Hack to add push priority to push notifications ( #7765 )
...
* Remove obsolete comment about ancient temporary code
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Implement hack to set push priority
based on whether the tweaks indicate the event might cause
effects.
* Changelog for 7765
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Antilint
* Add tests for push priority
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Update synapse/push/httppusher.py
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Antilint
* Remove needless invites from tests.
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2020-07-01 17:02:31 +01:00
Brendan Abolivier
74d3e177f0
Back out MSC2625 implementation ( #7761 )
2020-07-01 11:08:25 +01:00
Patrick Cloke
71cccf1593
Additional configuration options for auto-join rooms ( #7763 )
2020-06-30 15:41:36 -04:00
Erik Johnston
0e0a2817a2
Yield during large v2 state res. ( #7735 )
...
State res v2 across large data sets can be very CPU intensive, and if
all the relevant events are in the cache the algorithm will run from
start to finish within a single reactor tick. This can result in
blocking the reactor tick for several seconds, which can have major
repercussions on other requests.
To fix this we simply add the occaisonal `sleep(0)` during iterations to
yield execution until the next reactor tick. The aim is to only do this
for large data sets so that we don't impact otherwise quick resolutions.=
2020-06-24 18:48:18 +01:00
Sorunome
6920e58136
add org.matrix.login.jwt so that m.login.jwt can be deprecated ( #7675 )
2020-06-24 10:23:55 +01:00
Patrick Cloke
95e41f368b
Allow local media to be marked as safe from being quarantined. ( #7718 )
2020-06-22 08:04:14 -04:00
Patrick Cloke
3630825612
Convert the typing handler to async/await. ( #7679 )
2020-06-17 10:37:59 -04:00
Patrick Cloke
434716e1d3
Fetch from the r0 media path instead of the unspecced v1. ( #7714 )
2020-06-17 08:36:46 -04:00
Brendan Abolivier
46613aaf79
Implement unread counter (MSC2625) ( #7673 )
...
Implementation of https://github.com/matrix-org/matrix-doc/pull/2625
2020-06-17 10:58:32 +01:00
Patrick Cloke
ac51bd581a
Include a user agent in federation requests. ( #7677 )
2020-06-16 10:43:29 -04:00
Dagfinn Ilmari Mannsåker
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents ( #7704 )
2020-06-16 08:51:47 -04:00
Richard van der Hoff
03619324fc
Create a ListenerConfig object ( #7681 )
...
This ended up being a bit more invasive than I'd hoped for (not helped by
generic_worker duplicating some of the code from homeserver), but hopefully
it's an improvement.
The idea is that, rather than storing unstructured `dict`s in the config for
the listener configurations, we instead parse it into a structured
`ListenerConfig` object.
2020-06-16 12:44:07 +01:00
Brendan Abolivier
789606577a
Merge branch 'master' into develop
2020-06-16 10:36:29 +01:00
Andrew Morgan
3e6b5bba71
Wrap register_device coroutine in an ensureDeferred ( #7684 )
...
Fixes https://github.com/matrix-org/synapse/issues/7683
Broke in: #7649
We had a `yield` acting on a coroutine. To be fair this one is a bit difficult to notice as there's a function in the middle that just passes the coroutine along.
2020-06-16 10:13:59 +01:00
Patrick Cloke
cc32fa7358
Ensure the body is a string before comparing push rules. ( #7701 )
2020-06-15 16:20:34 -04:00
Hubert Chathi
2b2344652b
Ensure etag is a string for GET room_keys/version response ( #7691 )
2020-06-15 13:42:44 -04:00
Brendan Abolivier
6efb2b0ad4
Merge branch 'develop' into babolivier/mark_unread
2020-06-15 16:37:52 +01:00
Brendan Abolivier
c2b4621630
Merge branch 'develop' into babolivier/mark_unread
2020-06-15 16:37:08 +01:00
Patrick Cloke
7d2532be36
Discard RDATA from already seen positions. ( #7648 )
2020-06-15 08:44:54 -04:00
Brendan Abolivier
7e80c84902
Lint
2020-06-12 11:31:11 +01:00
Brendan Abolivier
6b1fa3293d
Test that a mark_unread action updates the right counter when using a slave store
2020-06-12 11:28:26 +01:00
Brendan Abolivier
63d9a00bf1
Remove debug logging
2020-06-12 11:13:30 +01:00
Brendan Abolivier
2a07c5ded6
Test that a mark_unread action updates the right counter
2020-06-12 11:08:05 +01:00
Brendan Abolivier
476a89707a
Fix tests
2020-06-10 17:55:03 +01:00
Andrew Morgan
fcd6961441
Add option to enable encryption by default for new rooms ( #7639 )
...
Fixes https://github.com/matrix-org/synapse/issues/2431
Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used.
Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637
Based on #7637
2020-06-10 17:44:34 +01:00
Patrick Cloke
737b4a936e
Convert user directory handler and related classes to async/await. ( #7640 )
2020-06-05 14:42:55 -04:00
