MirrorHub/synapse
0
0
Fork 1
mirror of https://mau.dev/maunium/synapse.git synced 2024-12-21 10:14:07 +01:00
Code Issues Projects Releases Wiki Activity
20575 commits 5 branches 731 tags 91 MiB
Commit graph

678 commits

Author SHA1 Message Date
Erik Johnston
2b7c180879
Start fewer opentracing spans (#8640) 
#8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously.

This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary.

I don't really know how much this will help.
 2020-10-26 09:30:19 +00:00
Erik Johnston
c850dd9a8e
Fix handling of User-Agent headers with bad utf-8. (#8632) 2020-10-23 17:12:59 +01:00
Richard van der Hoff
4433d01519
Merge pull request #8537 from matrix-org/rav/simplify_locally_reject_invite 
Simplify `_locally_reject_invite`
 2020-10-15 10:20:19 +01:00
Erik Johnston
921a3f8a59
Fix not sending events over federation when using sharded event persisters (#8536) 
* Fix outbound federaion with multiple event persisters.

We incorrectly notified federation senders that the minimum persisted
stream position had advanced when we got an `RDATA` from an event
persister.

Notifying of federation senders already correctly happens in the
notifier, so we just delete the offending line.

* Change some interfaces to use RoomStreamToken.

By enforcing use of `RoomStreamTokens` we make it less likely that
people pass in random ints that they got from somewhere random.
 2020-10-14 13:27:51 +01:00
Richard van der Hoff
a34b17e492 Simplify _locally_reject_invite 
Update `EventCreationHandler.create_event` to accept an auth_events param, and
use it in `_locally_reject_invite` instead of reinventing the wheel.
 2020-10-13 23:58:48 +01:00
Richard van der Hoff
d9d86c2996 Remove redundant token_id parameter to create_event 
this is always the same as requester.access_token_id.
 2020-10-13 23:06:36 +01:00
Patrick Cloke
629a951b49
Move additional tasks to the background worker, part 4 (#8513) 2020-10-13 08:20:32 -04:00
Erik Johnston
b2486f6656
Fix message duplication if something goes wrong after persisting the event (#8476) 
Should fix #3365.
 2020-10-13 12:07:56 +01:00
Patrick Cloke
c9c0ad5e20
Remove the deprecated Handlers object (#8494) 
All handlers now available via get_*_handler() methods on the HomeServer.
 2020-10-09 07:24:34 -04:00
Hubert Chathi
a97cec18bb
Invalidate the cache when an olm fallback key is uploaded (#8501) 2020-10-08 13:24:46 -04:00
Hubert Chathi
4cb44a1585
Add support for MSC2697: Dehydrated devices (#8380) 
This allows a user to store an offline device on the server and
then restore it at a subsequent login.
 2020-10-07 08:00:17 -04:00
Richard van der Hoff
43c622885c
Merge pull request #8463 from matrix-org/rav/clean_up_event_handling 
Reduce inconsistencies between codepaths for membership and non-membership events.
 2020-10-07 12:20:44 +01:00
Hubert Chathi
3cd78bbe9e
Add support for MSC2732: olm fallback keys (#8312) 2020-10-06 13:26:29 -04:00
Richard van der Hoff
e775b5bb5b kill off send_nonmember_event 
This is now redundant, and we can just call `handle_new_client_event` directly.
 2020-10-05 19:04:10 +01:00
BBBSnowball
05ee048f2c
Add config option for always using "userinfo endpoint" for OIDC (#7658) 
This allows for connecting to certain IdPs, e.g. GitLab.
 2020-10-01 13:54:35 -04:00
Patrick Cloke
8b40843392
Allow additional SSO properties to be passed to the client (#8413) 2020-09-30 13:02:43 -04:00
Tdxdxoz
abd04b6af0
Allow existing users to login via OpenID Connect. (#8345) 
Co-authored-by: Benjamin Koch <bbbsnowball@gmail.com>

This adds configuration flags that will match a user to pre-existing users
when logging in via OpenID Connect. This is useful when switching to
an existing SSO system.
 2020-09-25 07:01:45 -04:00
Dionysis Grigoropoulos
37ca5924bd
Create function to check for long names in devices (#8364) 
* Create a new function to verify that the length of a device name is
under a certain threshold.
* Refactor old code and tests to use said function.
* Verify device name length during registration of device
* Add a test for the above

Signed-off-by: Dionysis Grigoropoulos <dgrig@erethon.com>
 2020-09-22 11:42:55 +01:00
Patrick Cloke
8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344) 
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
 2020-09-18 09:56:44 -04:00
reivilibre
576bc37d31
Catch-up after Federation Outage (split, 4): catch-up loop (#8272) 2020-09-15 09:07:19 +01:00
Erik Johnston
c9dbee50ae
Fixup pusher pool notifications (#8287) 
`pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed.

I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.
 2020-09-09 16:56:08 +01:00
Erik Johnston
dc9dcdbd59 Revert "Fixup pusher pool notifications" 
This reverts commit e7fd336a53.
 2020-09-09 16:19:22 +01:00
Erik Johnston
e7fd336a53 Fixup pusher pool notifications 2020-09-09 16:17:50 +01:00
Patrick Cloke
cef00211c8
Allow for make_awaitable's return value to be re-used. (#8261) 2020-09-08 07:26:55 -04:00
Andrew Morgan
68cdb3708e
Rename 'populate_stats_process_rooms_2' background job back to 'populate_stats_process_rooms' again (#8243) 
Fixes https://github.com/matrix-org/synapse/issues/8238

Alongside the delta file, some changes were also necessary to the codebase to remove references to the now defunct `populate_stats_process_rooms_2` background job. Thankfully the latter doesn't seem to have made it into any documentation yet :)
 2020-09-08 11:05:59 +01:00
Patrick Cloke
c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Patrick Cloke
b055dc9322
Ensure that the OpenID Connect remote ID is a string. (#8190) 2020-08-28 08:56:36 -04:00
Patrick Cloke
e00816ad98
Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00
Patrick Cloke
b49a5b9307
Convert stats and related calls to async/await (#8192) 2020-08-27 17:24:37 -04:00
Patrick Cloke
4a739c73b4
Convert simple_update* and simple_select* to async (#8173) 2020-08-27 07:08:38 -04:00
Andrew Morgan
a466b67972
Reduce run-times of tests by advancing the reactor less (#7757) 2020-08-27 11:39:53 +01:00
Patrick Cloke
6fe12c9512
Do not propagate typing notifications from shadow-banned users. (#8176) 2020-08-26 12:05:36 -04:00
Patrick Cloke
4c6c56dc58
Convert simple_select_one and simple_select_one_onecol to async (#8162) 2020-08-26 07:19:32 -04:00
Patrick Cloke
3f91638da6
Allow denying or shadow banning registrations via the spam checker (#8034) 2020-08-20 15:42:58 -04:00
Patrick Cloke
050e20e7ca
Convert some of the general database methods to async (#8100) 2020-08-17 12:18:01 -04:00
Patrick Cloke
ad6190c925
Convert stream database to async/await. (#8074) 2020-08-17 07:24:46 -04:00
Patrick Cloke
5ecc8b5825
Convert devices database to async/await. (#8069) 2020-08-12 10:51:42 -04:00
Patrick Cloke
d68e10f308
Convert account data, device inbox, and censor events databases to async/await (#8063) 2020-08-12 09:29:06 -04:00
Patrick Cloke
a0acdfa9e9
Converts event_federation and registration databases to async/await (#8061) 2020-08-11 17:21:13 -04:00
Patrick Cloke
7f837959ea
Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async (#8042) 2020-08-07 13:36:29 -04:00
Patrick Cloke
d4a7829b12
Convert synapse.api to async/await (#8031) 2020-08-06 08:30:06 -04:00
Erik Johnston
a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Andrew Morgan
5d92a1428c
Prevent join->join membership transitions changing member count (#7977) 
`StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc.

However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information.

This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point.

Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update).

Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338.
 2020-08-03 21:54:24 +01:00
Patrick Cloke
c978f6c451
Convert federation client to async/await. (#7975) 2020-07-30 08:01:33 -04:00
Patrick Cloke
4cce8ef74e
Convert appservice to async. (#7973) 2020-07-30 07:27:39 -04:00
Patrick Cloke
6b3ac3b8cd
Convert device handler to async/await (#7871) 2020-07-17 07:09:25 -04:00
Patrick Cloke
6fca1b3506
Convert _base, profile, and _receipts handlers to async/await (#7860) 2020-07-17 07:08:30 -04:00
Erik Johnston
9006e125af Fix tests 2020-07-15 15:47:27 +01:00
Patrick Cloke
b11450dedc
Convert E2E key and room key handlers to async/await. (#7851) 2020-07-15 08:48:58 -04:00
Patrick Cloke
6d687ebba1
Convert the appservice handler to async/await. (#7775) 2020-07-06 07:40:35 -04:00
Will Hunt
62b1ce8539
isort 5 compatibility (#7786) 
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
 2020-07-05 16:32:02 +01:00
Patrick Cloke
71cccf1593
Additional configuration options for auto-join rooms (#7763) 2020-06-30 15:41:36 -04:00
Patrick Cloke
3630825612
Convert the typing handler to async/await. (#7679) 2020-06-17 10:37:59 -04:00
Hubert Chathi
2b2344652b
Ensure etag is a string for GET room_keys/version response (#7691) 2020-06-15 13:42:44 -04:00
Andrew Morgan
fcd6961441
Add option to enable encryption by default for new rooms (#7639) 
Fixes https://github.com/matrix-org/synapse/issues/2431

Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used.

Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637

Based on #7637
 2020-06-10 17:44:34 +01:00
Patrick Cloke
737b4a936e
Convert user directory handler and related classes to async/await. (#7640) 2020-06-05 14:42:55 -04:00
Travis Ralston
09099313e6
Add an option to disable autojoin for guest accounts (#6637) 
Fixes https://github.com/matrix-org/synapse/issues/3177
 2020-06-05 18:18:15 +01:00
Andrew Morgan
f4e6495b5d
Performance improvements and refactor of Ratelimiter (#7595) 
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:

* Rather undocumented, and
* causing a *lot* of config checks

This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. 

Best to be reviewed commit-by-commit.
 2020-06-05 10:47:20 +01:00
Andrew Morgan
c389bfb6ea
Fix encryption algorithm typos in tests/comments (#7637) 
@uhoreg has confirmed these were both typos. They are only in comments and tests though, rather than anything critical.

Introduced in:

* https://github.com/matrix-org/synapse/pull/7157
* https://github.com/matrix-org/synapse/pull/5726
 2020-06-04 20:03:40 +01:00
Erik Johnston
1531b214fc
Add ability to wait for replication streams (#7542) 
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).

Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.

People probably want to look at this commit by commit.
 2020-05-22 14:21:54 +01:00
Patrick Cloke
a3cf36f76e
Support UI Authentication for OpenID Connect accounts (#7457) 2020-05-15 12:26:02 -04:00
Patrick Cloke
08bc80ef09
Implement room version 6 (MSC2240). (#7506) 2020-05-15 09:30:10 -04:00
Patrick Cloke
56b66db78a
Strictly enforce canonicaljson requirements in a new room version (#7381) 2020-05-14 13:24:01 -04:00
Andrew Morgan
67feea8044
Extend spam checker to allow for multiple modules (#7435) 2020-05-08 19:25:48 +01:00
Quentin Gliech
616af44137
Implement OpenID Connect-based login (#7256) 2020-05-08 08:30:40 -04:00
Andrew Morgan
aee9130a83
Stop Auth methods from polling the config on every req. (#7420) 2020-05-06 15:54:58 +01:00
Andrew Morgan
6b22921b19
async/await is_server_admin (#7363) 2020-05-01 15:15:36 +01:00
Patrick Cloke
eed7c5b89e
Convert auth handler to async/await (#7261) 2020-04-15 12:40:18 -04:00
Patrick Cloke
468dcc767b
Allow admins to create aliases when they are not in the room (#7191) 2020-04-01 08:27:05 -04:00
Richard van der Hoff
6486c96b65
Merge pull request #7157 from matrix-org/rev.outbound_device_pokes_tests 
Add tests for outbound device pokes
 2020-03-30 13:59:07 +01:00
Dirk Klimpel
8327eb9280
Add options to prevent users from changing their profile. (#7096) 2020-03-27 19:15:23 +00:00
Richard van der Hoff
28d9d6e8a9 Remove spurious "name" parameter to default_config 
this is never set to anything other than "test", and is a source of unnecessary
boilerplate.
 2020-03-24 18:33:49 +00:00
Richard van der Hoff
6a35046363 Revert "Add options to disable setting profile info for prevent changes. (#7053)" 
This reverts commit 54dd28621b, reversing
changes made to 6640460d05.
 2020-03-17 11:25:01 +00:00
Brendan Abolivier
54dd28621b
Add options to disable setting profile info for prevent changes. (#7053) 2020-03-10 22:23:01 +00:00
dklimpel
885134529f updates after review 2020-03-09 22:09:29 +01:00
dklimpel
7e5f40e771 fix tests 2020-03-09 21:00:36 +01:00
dklimpel
04f4b5f6f8 add tests 2020-03-09 19:51:31 +01:00
dklimpel
1f5f3ae8b1 Add options to disable setting profile info for prevent changes. 2020-03-08 14:49:33 +01:00
Patrick Cloke
13892776ef
Allow deleting an alias if the user has sufficient power level (#6986) 2020-03-04 11:30:46 -05:00
Patrick Cloke
7dcbc33a1b
Validate the alt_aliases property of canonical alias events (#6971) 2020-03-03 07:12:45 -05:00
Richard van der Hoff
3e99528f2b
Store room version on invite (#6983) 
When we get an invite over federation, store the room version in the rooms table.

The general idea here is that, when we pull the invite out again, we'll want to know what room_version it belongs to (so that we can later redact it if need be). So we need to store it somewhere...
 2020-02-26 16:58:33 +00:00
Erik Johnston
1f773eec91
Port PresenceHandler to async/await (#6991) 2020-02-26 15:33:26 +00:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) 
Ensure good comprehension hygiene using flake8-comprehensions.
 2020-02-21 07:15:07 -05:00
Richard van der Hoff
b58d17e44f Refactor the membership check methods in Auth 
these were getting a bit unwieldy, so let's combine `check_joined_room` and
`check_user_was_in_room` into a single `check_user_in_room`.
 2020-02-18 23:21:44 +00:00
Patrick Cloke
fe3941f6e3
Stop sending events when creating or deleting aliases (#6904) 
Stop sending events when creating or deleting associations (room aliases). Send an updated canonical alias event if one of the alt_aliases is deleted.
 2020-02-18 07:29:44 -05:00
Patrick Cloke
02e89021f5
Convert the directory handler tests to use HomeserverTestCase (#6919) 
Convert directory handler tests to use HomeserverTestCase.
 2020-02-14 09:05:43 -05:00
Patrick Cloke
49f877d32e
Filter the results of user directory searching via the spam checker (#6888) 
Add a method to the spam checker to filter the user directory results.
 2020-02-14 07:17:54 -05:00
Patrick Cloke
a92e703ab9
Reject device display names that are too long (#6882) 
* Reject device display names that are too long.

Too long is currently defined as 100 characters in length.

* Add a regression test for rejecting a too long device display name.
 2020-02-10 16:35:26 -05:00
Erik Johnston
b08b0a22d5
Add typing to synapse.federation.sender (#6871) 2020-02-07 13:56:38 +00:00
Richard van der Hoff
928edef979 Pass room_version into event_from_pdu_json 
It's called from all over the shop, so this one's a bit messy.
 2020-02-06 16:08:27 +00:00
Richard van der Hoff
a5afdd15e5
Merge pull request #6806 from matrix-org/rav/redact_changes/3 
Pass room_version into add_hashes_and_signatures
 2020-01-31 10:57:03 +00:00
Richard van der Hoff
d7bf793cc1 s/get_room_version/get_room_version_id/ 
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
 2020-01-31 10:06:21 +00:00
Richard van der Hoff
ef6bdafb29 Store the room version in EventBuilder 2020-01-30 22:15:50 +00:00
Erik Johnston
b660327056
Resync remote device list when detected as stale. (#6786) 2020-01-30 17:06:38 +00:00
Erik Johnston
28c98e51ff
Add local_current_membership table (#6655) 
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
 2020-01-15 14:59:33 +00:00
Richard van der Hoff
326c893d24
Kill off RegistrationError (#6691) 
This is pretty pointless. Let's just use SynapseError.
 2020-01-13 12:48:22 +00:00
Erik Johnston
2284eb3a53
Add database config class (#6513) 
This encapsulates config for a given database and is the way to get new
connections.
 2019-12-18 10:45:12 +00:00
Hubert Chathi
cb2db17994
look up cross-signing keys from the DB in bulk (#6486) 2019-12-12 12:03:28 -05:00
Neil Johnson
adfdd82b21
Back out perf regression from get_cross_signing_keys_from_cache. (#6494) 
Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression.
 2019-12-09 13:59:27 +00:00
Erik Johnston
a9b393340f
Merge pull request #6484 from matrix-org/erikj/port_sync_handler 
Port SyncHandler to async/await
 2019-12-09 11:32:44 +00:00
Erik Johnston
b3a4e35ca8 Fixup functions to consistently return deferreds 2019-12-06 10:40:05 +00:00
Erik Johnston
8437e2383e Port SyncHandler to async/await 2019-12-05 17:58:25 +00:00
Erik Johnston
4a33a6dd19 Move background update handling out of store 2019-12-05 11:11:26 +00:00
Erik Johnston
756d4942f5 Move DB pool and helper functions into dedicated Database class 2019-12-05 10:46:37 +00:00
Erik Johnston
ee86abb2d6 Remove underscore from SQLBaseStore functions 2019-12-04 16:23:43 +00:00
Amber Brown
0f87b912ab
Implementation of MSC2314 (#6176) 2019-11-28 08:54:07 +11:00
Hubert Chathi
0d27aba900
add etag and count to key backup endpoints (#5858) 2019-11-27 16:14:44 -05:00
Richard van der Hoff
807ec3bd99
Fix bug which caused rejected events to be stored with the wrong room state (#6320) 
Fixes a bug where rejected events were persisted with the wrong state group.

Also fixes an occasional internal-server-error when receiving events over
federation which are rejected and (possibly because they are
backwards-extremities) have no prev_group.

Fixes #6289.
 2019-11-06 10:01:39 +00:00
Hubert Chathi
bb6cec27a5 rename get_devices_by_remote to get_device_updates_by_remote 2019-10-30 14:57:34 -04:00
Erik Johnston
3f33879be4 Port federation_server to async/await 2019-10-29 14:13:08 +00:00
Erik Johnston
7e7a1461f6 Fix tests 2019-10-25 10:57:37 +01:00
Erik Johnston
c17efdc01c Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores 2019-10-22 18:44:29 +01:00
Erik Johnston
bb6264be0b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores 2019-10-22 10:41:18 +01:00
Erik Johnston
c66a06ac6b Move storage classes into a main "data store". 
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
 2019-10-21 16:05:06 +01:00
Hubert Chathi
36adfaedab Merge branch 'develop' into cross-signing_sig_upload 2019-10-18 18:34:42 +01:00
Hubert Chathi
cfc28325a6 Merge branch 'develop' into uhoreg/e2e_cross-signing_merged 2019-10-18 11:39:40 +01:00
Erik Johnston
5859a5c569 Fix presence timeouts when synchrotron restarts. (#6212) 
* Fix presence timeouts when synchrotron restarts.

Handling timeouts would fail if there was an external process that had
timed out, e.g. a synchrotron restarting. This was due to a couple of
variable name typoes.

Fixes #3715.
 2019-10-18 06:42:26 +01:00
Hubert Chathi
691dd67fcd
Merge pull request #6189 from matrix-org/uhoreg/e2e_backup_optional_version 
make version optional in body of e2e backup version update
 2019-10-11 10:11:59 -04:00
Hubert Chathi
5373de6cce change test name to be unique 2019-10-10 08:54:07 -04:00
Richard van der Hoff
a139420a3c
Fix races in room stats (and other) updates. (#6187) 
Hopefully this will fix the occasional failures we were seeing in the room directory.

The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8.

We can solve this easily by only processing up to the stream_id where we know all events have been persisted.
 2019-10-10 11:29:01 +01:00
Hubert Chathi
4535a07f4a make version optional in body of e2e backup version update 
to agree with latest version of the MSC
 2019-10-09 17:54:03 -04:00
Brendan Abolivier
ae0b78cb1f
Merge pull request #6147 from matrix-org/babolivier/3pid-invite-revoked 
Don't 500 when trying to exchange a revoked 3PID invite
 2019-10-04 12:09:05 +01:00
Brendan Abolivier
8a5e8e829b
Lint (again) 2019-10-03 11:30:43 +01:00
Brendan Abolivier
ebcb6a30d7
Lint 2019-10-03 11:29:07 +01:00
Brendan Abolivier
6527fa18c1 Add test case 2019-10-03 11:24:36 +01:00
Erik Johnston
a5166e4d5f
Land improved room list based on room stats (#6019) 
Use room_stats and room_state for room directory search
 2019-10-02 14:08:35 +01:00
Richard van der Hoff
1e19ce00bf
Add 'failure_ts' column to 'destinations' table (#6016) 
Track the time that a server started failing at, for general analysis purposes.
 2019-09-17 11:41:54 +01:00
Jason Robinson
be618e0551 Only count real users when checking for auto-creation of auto-join room 
Previously if the first registered user was a "support" or "bot" user,
when the first real user registers, the auto-join rooms were not
created.

Fix to exclude non-real (ie users with a special user type) users
when counting how many users there are to determine whether we should
auto-create a room.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2019-09-09 14:48:08 +03:00
Hubert Chathi
ab729e31cf use something that's the right type for user_id 2019-09-06 17:52:37 -04:00
Hubert Chathi
e47af0f086 fix test 2019-09-05 17:03:14 -04:00
Hubert Chathi
5914fd09c7 add test 2019-09-04 22:05:02 -04:00
Hubert Chathi
9061b4198a make isort happy 2019-09-04 22:04:41 -04:00
Hubert Chathi
7d6c70fc7a make black happy 2019-09-04 22:04:12 -04:00
Hubert Chathi
ac4746ac4b allow uploading signatures of master key signed by devices 2019-09-04 22:03:41 -04:00
Hubert Chathi
faf72a4c40 Merge branch 'develop' into cross-signing_keys 2019-09-04 19:12:29 -04:00
Erik Johnston
6e834e94fc
Fix and refactor room and user stats (#5971) 
Previously the stats were not being correctly populated.
 2019-09-04 13:04:27 +01:00
Andrew Morgan
4548d1f87e
Remove unnecessary parentheses around return statements (#5931) 
Python will return a tuple whether there are parentheses around the returned values or not.

I'm just sick of my editor complaining about this all over the place :)
 2019-08-30 16:28:26 +01:00
Hubert Chathi
8c9adcc95d fix formatting 2019-08-01 22:09:05 -04:00
Hubert Chathi
336c546d6a Merge branch 'cross-signing_hidden' into cross-signing_keys 2019-08-01 16:31:40 -04:00
Richard van der Hoff
8c97f6414c
Remove non-functional 'expire_access_token' setting (#5782) 
The `expire_access_token` didn't do what it sounded like it should do. What it
actually did was make Synapse enforce the 'time' caveat on macaroons used as
access tokens, but since our access token macaroons never contained such a
caveat, it was always a no-op.

(The code to add 'time' caveats was removed back in v0.18.5, in #1656)
 2019-07-30 08:25:02 +01:00
Hubert Chathi
c659b9f94f allow uploading keys for cross-signing 2019-07-25 11:08:24 -04:00
Amber Brown
4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Erik Johnston
d86321300a
Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_data 
Add basic function to get all data for a user out of synapse
 2019-07-15 10:04:02 +01:00
Richard van der Hoff
5f158ec039
Implement access token expiry (#5660) 
Record how long an access token is valid for, and raise a soft-logout once it
expires.
 2019-07-12 17:26:02 +01:00
Richard van der Hoff
953dbb7980
Remove access-token support from RegistrationStore.register (#5642) 
The 'token' param is no longer used anywhere except the tests, so let's kill
that off too.
 2019-07-10 16:26:49 +01:00
Richard van der Hoff
824707383b
Remove access-token support from RegistrationHandler.register (#5641) 
Nothing uses this now, so we can remove the dead code, and clean up the
API.

Since we're changing the shape of the return value anyway, we take the
opportunity to give the method a better name.
 2019-07-08 19:01:08 +01:00
Richard van der Hoff
1af2fcd492 Move get_or_create_user to test code (#5628) 
This is only used in tests, so...
 2019-07-08 23:52:26 +10:00
Erik Johnston
c061d4f237 Fixup from review comments. 2019-07-04 11:41:06 +01:00
Erik Johnston
8ee69f299c Add basic function to get all data for a user out of synapse 2019-07-02 12:09:04 +01:00
Amber Brown
32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Erik Johnston
75538813fc Fix background updates to handle redactions/rejections (#5352) 
* Fix background updates to handle redactions/rejections

In background updates based on current state delta stream we need to
handle that we may not have all the events (or at least that
`get_events` may raise an exception).
 2019-06-06 00:45:46 +10:00
Amber Brown
4a30e4acb4
Room Statistics (#4338) 2019-05-21 11:36:50 -05:00
ReidAnderson
3787133c9e Limit UserIds to a length that fits in a state key (#5198) 2019-05-20 11:20:08 +01:00
Amber Brown
df2ebd75d3
Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig (#5171) 2019-05-13 15:01:14 -05:00
Amber Brown
b36c82576e
Run Black on the tests again (#5170) 2019-05-10 00:12:11 -05:00
Richard van der Hoff
12f9d51e82
Add admin api for sending server_notices (#5121) 2019-05-02 11:59:16 +01:00
Richard van der Hoff
40e576e29c Move admin api impl to its own package 
It doesn't really belong under rest/client/v1 any more.
 2019-05-01 15:44:30 +01:00
Andrew Morgan
4a4d5c4fd6
Fix grammar and document get_current_users_in_room (#4998) 2019-04-03 14:32:20 +01:00
Erik Johnston
40e56997bc Review comments 2019-03-28 13:48:41 +00:00
Erik Johnston
b7fa834c40 Add unit tests 2019-03-27 13:41:36 +00:00
Andrew Morgan
4a125be138
Make federation endpoints more tolerant of trailing slashes v2 (#4935) 
Redo of https://github.com/matrix-org/synapse/pull/4840
 2019-03-26 11:35:29 +00:00
Amber Brown
a68e00fca8
Some more porting to HomeserverTestCase and remove old RESTHelper (#4913) 2019-03-22 02:10:21 +11:00
Andrew Morgan
7bef97dfb7
Remove trailing slashes from outbound federation requests and retry on 400 (#4840) 
As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
 2019-03-21 15:07:28 +00:00
Erik Johnston
09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory 
Add option to disable search room lists
 2019-03-21 10:16:54 +00:00
Erik Johnston
3660d24ebe Add test 2019-03-20 15:16:36 +00:00
Erik Johnston
cc09685830 Add test 2019-03-20 14:53:44 +00:00
Richard van der Hoff
b5d48560c7 Fix RegistrationTestCase 
turns out this relies on there being a `user_consent_version` set.
 2019-03-19 12:05:05 +00:00
Neil Johnson
88f0675967 fix test_auto_create_auto_join_where_no_consent (#4886) 2019-03-19 11:38:59 +00:00
Amber Brown
282c97327f
Migrate the user directory initial population to a background task (#4864) 2019-03-19 04:50:24 +11:00
Erik Johnston
271cb1998b Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793)" 
This reverts commit 290552fd83.
 2019-03-14 14:30:54 +00:00
Erik Johnston
9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search 
Improve searching in the userdir
 2019-03-14 13:54:15 +00:00
Andrew Morgan
09626bfd39 Switch to wrapper function around _send_request 2019-03-13 18:26:06 +00:00
Andrew Morgan
97653ef1f4 Correct argument name 2019-03-12 14:30:26 +00:00
Amber Brown
797b6a63fc fixup 2019-03-13 01:17:51 +11:00
Andrew Morgan
0ea8582f8b Cleaner way of implementing trailing slashes 2019-03-12 14:11:11 +00:00
Amber Brown
10480c4348 fixup 2019-03-12 21:47:14 +11:00
Andrew Morgan
4868b12029 and again 2019-03-11 18:22:26 +00:00
Andrew Morgan
802cb5dcf0 Fix syntax error 2019-03-11 18:08:28 +00:00
Andrew Morgan
66f205e93d We're calling different functions now 2019-03-11 18:01:58 +00:00
Andrew Morgan
290552fd83
Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793) 
Server side of a solution towards #3622.
 2019-03-11 17:44:03 +00:00
Amber Brown
5ba8ceab4c fixes 2019-03-12 00:35:31 +11:00
Amber Brown
4abc988c6a
initial 2019-03-11 21:11:36 +11:00
Amber Brown
f6135d06cf
Rewrite userdir to be faster (#4537) 2019-03-07 01:22:53 -08:00
Brendan Abolivier
a4c3a361b7
Add rate-limiting on registration (#4735) 
* Rate-limiting for registration

* Add unit test for registration rate limiting

* Add config parameters for rate limiting on auth endpoints

* Doc

* Fix doc of rate limiting function

Co-Authored-By: babolivier <contact@brendanabolivier.com>

* Incorporate review

* Fix config parsing

* Fix linting errors

* Set default config for auth rate limiting

* Fix tests

* Add changelog

* Advance reactor instead of mocked clock

* Move parameters to registration specific config and give them more sensible default values

* Remove unused config options

* Don't mock the rate limiter un MAU tests

* Rename _register_with_store into register_with_store

* Make CI happy

* Remove unused import

* Update sample config

* Fix ratelimiting test for py2

* Add non-guest test
 2019-03-05 14:25:33 +00:00
Richard van der Hoff
2c3548d9d8
Update test_typing to use HomeserverTestCase. (#4771) 2019-03-04 10:05:39 +00:00
Erik Johnston
dbdc565dfd Fix registration on workers (#4682) 
* Move RegistrationHandler init to HomeServer

* Move post registration actions to RegistrationHandler

* Add post regisration replication endpoint

* Newsfile
 2019-02-20 18:47:31 +11:00
Erik Johnston
eaf4d11af9 Add configurable room list publishing rules 
This allows specifying who and what is allowed to be published onto the
public room list
 2019-02-14 16:02:23 +00:00
Hubert Chathi
afae8442b5 make sure version is in body and wrap in linearizer queue 
also add tests
 2019-02-08 01:32:45 -05:00
Neil Johnson
d2f7c4e6b1
create support user (#4141) 
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
 2018-12-14 18:20:59 +00:00
Richard van der Hoff
b0c24a66ec Rip out half-implemented m.login.saml2 support (#4265) 
* Rip out half-implemented m.login.saml2 support

This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.

* drop undocumented dependency on dateutil

It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
 2018-12-06 19:44:38 +11:00
Neil Johnson
7039ece8fb Neilj/fix autojoin (#4223) 
* Fix auto join failures for servers that require user consent

* Fix auto join failures for servers that require user consent
 2018-11-28 22:24:57 +11:00
Hubert Chathi
f1087106cf
handle empty backups according to latest spec proposal (#4123) 
fixes #4056
 2018-11-05 17:59:29 -05:00
Erik Johnston
b94a43d5b5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list 2018-10-25 15:25:31 +01:00
Erik Johnston
e5481b22aa Use allow/deny 2018-10-25 15:25:21 +01:00
Neil Johnson
f7f487e14c Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin 2018-10-25 14:40:06 +01:00
Erik Johnston
3c580c2b47 Add tests for alias creation rules 2018-10-19 10:22:45 +01:00
Will Hunt
d6a7797dd1 Fix roomlist since tokens on Python 3 (#4046) 
Thanks @Half-Shot !!!
 2018-10-17 23:04:55 +11:00
Neil Johnson
a2bfb778c8 improve auto room join logic, comments and tests 2018-10-12 18:17:36 +01:00
David Baker
dc045ef202 Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups 2018-10-09 10:05:02 +01:00
Neil Johnson
2dadc092b8 move logic into register, fix room alias localpart bug, tests 2018-10-04 17:00:27 +01:00
Amber Brown
52ec6e9dfa
Port tests/ to Python 3 (#3808) 2018-09-07 02:58:18 +10:00
Hubert Chathi
3801b8aa03 try to make flake8 and isort happy 2018-09-06 11:35:19 -04:00
Richard van der Hoff
4f8baab0c4 Merge branch 'master' into develop 2018-09-06 13:05:22 +01:00
Richard van der Hoff
c127c8d042 Fix origin handling for pushed transactions 
Use the actual origin for push transactions, rather than whatever the remote
server claimed.
 2018-09-05 13:08:07 +01:00
Amber Brown
77055dba92
Fix tests on postgresql (#3740) 2018-09-04 02:21:48 +10:00
Hubert Chathi
16a31c6fce update to newer Synapse APIs 2018-08-24 22:51:25 -04:00
Hubert Chathi
83caead95a
Merge branch 'develop' into e2e_backups 2018-08-24 11:44:26 -04:00
Erik Johnston
fd2dbf1836 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-22 17:06:10 +01:00
Erik Johnston
764030cf63
Merge pull request #3659 from matrix-org/erikj/split_profiles 
Allow profile updates to happen on workers
 2018-08-22 11:35:55 +01:00
Neil Johnson
e07970165f rename error code 2018-08-18 14:39:45 +01:00
Erik Johnston
782689bd40 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles 2018-08-17 14:15:48 +01:00
Erik Johnston
ca87ad1def Split ProfileHandler into master and worker 2018-08-17 14:15:14 +01:00
Neil Johnson
13ad9930c8 add new error type ResourceLimit 2018-08-16 18:02:02 +01:00
Neil Johnson
86a00e05e1 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus 2018-08-15 16:27:08 +01:00
Neil Johnson
75c663c7b9 update error codes 2018-08-15 11:27:48 +01:00
Neil Johnson
19b433e3f4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_email 2018-08-14 17:44:46 +01:00
Neil Johnson
b586b8b986 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus 2018-08-14 17:43:22 +01:00
Neil Johnson
7277216d01 fix setup_test_homeserver to be postgres compatible 2018-08-14 17:14:39 +01:00
Neil Johnson
414d54b61a
Merge pull request #3670 from matrix-org/neilj/mau_sync_block 
Block ability to read via sync if mau limit exceeded
 2018-08-14 15:21:31 +00:00
Neil Johnson
06b331ff40 fix off by 1 errors 2018-08-14 15:28:15 +01:00
Neil Johnson
8f9a7eb58d support admin_email config and pass through into blocking errors, return AuthError in all cases 2018-08-14 15:11:54 +01:00
Neil Johnson
ed4bc3d2fc fix off by 1s on mau 2018-08-14 15:04:48 +01:00
Neil Johnson
9b5bf3d858 Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus 2018-08-14 14:51:38 +01:00
Neil Johnson
e25d87d97b Merge branch 'neilj/mau_sync_block' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus 2018-08-14 14:32:18 +01:00
Neil Johnson
f4b49152e2 support admin_email config and pass through into blocking errors, return AuthError in all cases 2018-08-13 21:09:47 +01:00
Neil Johnson
ce7de9ae6b Revert "support admin_email config and pass through into blocking errors, return AuthError in all cases" 
This reverts commit 0d43f991a1.
 2018-08-13 18:06:18 +01:00
Neil Johnson
0d43f991a1 support admin_email config and pass through into blocking errors, return AuthError in all cases 2018-08-13 18:00:23 +01:00
Amber Brown
99dd975dae
Run tests under PostgreSQL (#3423) 2018-08-13 16:47:46 +10:00
Matthew Hodgson
66a4ca1d28 404 nicely if you try to interact with a missing current version 2018-08-12 19:14:31 -04:00
Matthew Hodgson
edc427a351 flake8 2018-08-12 19:14:31 -04:00
Matthew Hodgson
fe87890b18 implement remaining tests and make them work 2018-08-12 19:14:31 -04:00
Matthew Hodgson
f6a3067868 linting 2018-08-12 19:14:31 -04:00
Matthew Hodgson
15d513f16f fix idiocies and so make tests pass 2018-08-12 19:14:31 -04:00
Matthew Hodgson
174be586e5 first cut at a UT 2018-08-12 19:14:31 -04:00
black
8b3d9b6b19 Run black. 2018-08-10 23:54:09 +10:00
Neil Johnson
04df714259 fix imports 2018-08-09 17:41:52 +01:00
Neil Johnson
09cf130898 only block on sync where user is not part of the mau cohort 2018-08-09 17:39:12 +01:00
Neil Johnson
69ce057ea6 block sync if auth checks fail 2018-08-09 12:26:27 +01:00
Neil Johnson
a5ef110749 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block 2018-08-09 11:40:37 +01:00
Amber Brown
2511f3f8a0
Test fixes for Python 3 (#3647) 2018-08-09 12:22:01 +10:00
Neil Johnson
e92fb00f32 sync auth blocking 2018-08-08 17:54:49 +01:00
Neil Johnson
886be75ad1 bug fixes 2018-08-03 22:29:03 +01:00
Neil Johnson
74b1d46ad9 do mau checks based on monthly_active_users table 2018-08-02 16:57:35 +01:00
Neil Johnson
6eed16d8a2 fix test for py3 2018-08-01 14:02:10 +01:00
Neil Johnson
303f1c851f Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits 2018-08-01 13:42:50 +01:00
Neil Johnson
7931393495 make count_monthly_users async synapse/handlers/auth.py 2018-08-01 10:21:56 +01:00
Neil Johnson
df2235e7fa coding style 2018-07-31 13:16:20 +01:00
Travis Ralston
e908b86832 Remove pdu_failures from transactions 
The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.
 2018-07-30 16:28:47 -06:00
Neil Johnson
251e6c1210 limit register and sign in on number of monthly users 2018-07-30 15:55:57 +01:00
Amber Brown
49af402019 run isort 2018-07-09 16:09:20 +10:00