Commit graph

19592 commits

Author SHA1 Message Date
Eric Eastwood 4a12304cf7 Put empty prev_events behind new room version
See https://github.com/matrix-org/synapse/pull/11114#discussion_r733475645
2021-10-28 01:42:48 -05:00
Eric Eastwood 1d3f417046 Revert back to string interpolation for SQL boolean value
Maybe fixes the `sqlite3.OperationalError: no such column: FALSE,` problem in CI,
https://github.com/matrix-org/synapse/runs/3962382283#step:4:11038
2021-10-21 18:49:30 -05:00
Eric Eastwood 2fbe3f13ae Merge branch 'develop' into madlittlemods/return-historical-events-in-order-from-backfill 2021-10-21 05:26:04 -05:00
Eric Eastwood 5bfde7bd33 Merge branch 'master' into madlittlemods/return-historical-events-in-order-from-backfill 2021-10-21 04:51:57 -05:00
Eric Eastwood ee47878439 Fix mypy lints 2021-10-21 04:49:28 -05:00
Eric Eastwood 1263c7e2a9 Remove unused import 2021-10-21 04:39:34 -05:00
Eric Eastwood 83474d9158 Put MSC2716 backfill logic behind experimental feature flag 2021-10-21 04:31:10 -05:00
Eric Eastwood 69dfa16dcb Move to sorting the backfill events in the existing sorted 2021-10-21 04:19:36 -05:00
Eric Eastwood 7da8012f31 Remove extra event info 2021-10-21 04:06:03 -05:00
Eric Eastwood f39c1da083 Remove debug logging 2021-10-21 04:05:24 -05:00
Dirk Klimpel ef7fe09778
Fix setting a user's external_id via the admin API returns 500 and deletes users existing external mappings if that external ID is already mapped (#11051)
Fixes #10846
2021-10-21 09:52:32 +01:00
Eric Eastwood 4191f5615f Remove fake prev events from historical state chain
Fix https://github.com/matrix-org/synapse/issues/11091

We have to allow creation of events with no prev_events
but do have auth_events.

And since the historical member events are outliers
with no prev_events to resolve them, we want to avoid
putting them as backward extremeties.
2021-10-21 03:44:27 -05:00
Eric Eastwood 477c15df72 Some more debug logging 2021-10-20 22:11:12 -05:00
Eric Eastwood 886071b66b Fix backfill not picking up batch events connected to non-base insertion events
Previously, we would only look for a batch event if the insertion event
was connected to something else by prev_event. This is only the case
for the base insertion event. And instead, we need to look for
a batch event whenever we come across an insertion event.
2021-10-20 16:48:28 -05:00
Richard van der Hoff 57501d9194
Update sign_json to support inline key config (#11139)
It's been possible to configure a key inline in the homeserver.yaml since
13bc1e0746. Update `sign_json` to work with this.
2021-10-20 18:49:20 +01:00
Robert Edström 62db603fa0
Consider IP whitelist for identity server resolution (#11120)
Signed-off-by: Robert Edström <github@legogris.se>
2021-10-20 18:43:49 +01:00
Richard van der Hoff 0930e9ae12
Clean up _update_auth_events_and_context_for_auth (#11122)
Remove some redundant code, and generally simplify.
2021-10-20 18:22:40 +01:00
Aaron R 2c61a318cc
Show error when timestamp in seconds is provided to the /purge_media_cache API (#11101) 2021-10-20 14:41:48 +00:00
Sean Quah ee2cee5f52 Merge branch 'master' into develop 2021-10-20 13:43:02 +01:00
Travis Ralston 106d99b8cd
Remove false warning about copying the log config to a homeserver.yaml (#11092)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-10-20 11:48:15 +00:00
Brendan Abolivier 78d5896d19
Document the version of Synapse each module callback was introduced in (#11132)
* Mention callbacks introduced in v1.37.0

According to the documentation introduced in https://github.com/matrix-org/synapse/pull/10062

* Mention callbacks introduced in v1.39.0

According to https://github.com/matrix-org/synapse/pull/10386 and https://github.com/matrix-org/synapse/pull/9884

* Mention callbacks introduced in v1.42.0

According to https://github.com/matrix-org/synapse/pull/10524

* Mention callbacks introduced in v1.44.0 and v1.45.0

As per https://github.com/matrix-org/synapse/pull/10898, https://github.com/matrix-org/synapse/pull/10910 and https://github.com/matrix-org/synapse/pull/10894

* Mention callbacks introduced in v1.46.0

According to https://github.com/matrix-org/synapse/pull/10548
2021-10-20 11:04:27 +00:00
Sean Quah 9b016a0fb4 Remove link to #10947 from changelog 2021-10-20 12:04:20 +01:00
Sean Quah 522489fbcd 1.45.1 2021-10-20 12:00:03 +01:00
Sean Quah df95d3aec2
Revert change to counting of deactivated users towards the monthly active users limit (#11127)
Temporarily revert "Add functionality to remove deactivated users from the monthly_active_users table (#10947)".

This reverts commit eda8c88b84.
2021-10-20 11:05:29 +01:00
Patrick Cloke 0dd0c40329
Add missing type hints to event fetching. (#11121)
Updates the event rows returned from the database to be
attrs classes instead of dictionaries.
2021-10-19 14:29:03 +00:00
Andrew Morgan 5e0e683541
Fix instances of [example]{.title-ref} in the upgrade notes (#11118) 2021-10-19 14:13:56 +01:00
David Robertson a6c318735d
Merge branch 'master' into develop 2021-10-19 12:44:33 +01:00
David Robertson 95813ff43c
Be less inconsistent about v1.2.3 versus 1.2.3 2021-10-19 11:30:16 +01:00
David Robertson a21f8c4b41
Duplicate known issues under 1.45 release 2021-10-19 11:21:21 +01:00
David Robertson 8b1185347a
1.45.0 2021-10-19 11:19:55 +01:00
Dan Callahan 191396f4ba
Reword changelog regarding a suspected regression (#11117)
Signed-off-by: Dan Callahan <danc@element.io>
2021-10-19 11:16:52 +01:00
Richard van der Hoff f3efa0036b
Move _persist_auth_tree into FederationEventHandler (#11115)
This is just a lift-and-shift, because it fits more naturally here. We do
rename it to `process_remote_join` at the same time though.
2021-10-19 10:24:09 +01:00
Richard van der Hoff 0170774b19
Rename _auth_and_persist_fetched_events (#11116)
... to `_auth_and_persist_outliers`, since that reflects its purpose better.
2021-10-19 10:23:55 +01:00
Richard van der Hoff d85bc9a4a7
Include rejected status when we log events. (#11008)
If we find ourselves dealing with rejected events, we proably want to know
about it. Let's include it in the stringification of the event so that it gets
logged.
2021-10-19 10:21:50 +01:00
Eric Eastwood 260ca06f0f Some more trials of trying to get many many events to backfill in order on remote 2021-10-19 01:55:40 -05:00
Eric Eastwood a64bb2e811 Add changelog 2021-10-18 19:30:43 -05:00
Eric Eastwood 4983739156 Avoid constant missing prev_event fetching while backfilling
Persist backfilled event response from oldest -> newest to avoid
having to go fetch missing prev_events which de-outliers every
other event and screws up the stream_ordering. Missing prev_events
aren't fetched as "backfilled" so the stream_ordering was incrementing.

This helps us in MSC2716 land where we can more easily copy a similar
stream_ordering that the originating homeserver has.
2021-10-18 19:27:27 -05:00
Eric Eastwood 438e2226cc Use OrderedDict to gurantee order returned is the same as we were building the list
We are using a Dict over a list to gurantee we don't duplicate the event if it's already in there.
I assume this is why we were using a Set before.
2021-10-18 16:40:04 -05:00
Eric Eastwood f30302db58 Scratch debugging why events appear out of order on remote homeservers 2021-10-18 15:24:19 -05:00
Patrick Cloke 3ab55d43bd
Add missing type hints to synapse.api. (#11109)
* Convert UserPresenceState to attrs.
* Remove args/kwargs from error classes and explicitly pass msg/errorcode.
2021-10-18 15:01:10 -04:00
Richard van der Hoff cc33d9eee2
Check auth on received events' auth_events (#11001)
Currently, when we receive an event whose auth_events differ from those we expect, we state-resolve between the two state sets, and check that the event passes auth based on the resolved state.

This means that it's possible for us to accept events which don't pass auth at their declared auth_events (or where the auth events themselves were rejected), leading to problems down the line like #10083.

This change means we will:

 * ignore any events where we cannot find the auth events
 * reject any events whose auth events were rejected
 * reject any events which do not pass auth at their declared auth_events.

Together with a whole raft of previous work, this is a partial fix to #9595.

Fixes #6643.

Based on #11009.
2021-10-18 18:29:37 +01:00
Richard van der Hoff a5d2ea3d08
Check *all* auth events for room id and rejection (#11009)
This fixes a bug where we would accept an event whose `auth_events` include
rejected events, if the rejected event was shadowed by another `auth_event`
with same `(type, state_key)`.

The approach is to pass a list of auth events into
`check_auth_rules_for_event` instead of a dict, which of course means updating
the call sites.

This is an extension of #10956.
2021-10-18 18:28:30 +01:00
Brendan Abolivier 73743b8ad1
Document Synapse's behaviour when dealing with multiple modules (#11096)
Document Synapse's behaviour when multiple modules register the same
callback/web resource/etc.

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-18 18:26:52 +02:00
Richard van der Hoff e8f24b6c35
_run_push_actions_and_persist_event: handle no min_depth (#11014)
Make sure that we correctly handle rooms where we do not yet have a
`min_depth`, and also add some comments and logging.
2021-10-18 17:17:15 +01:00
Hillery Shay 7d70582eb0
Fix broken export-data admin command and add a test for it to CI (#11078)
Fix broken export-data admin command and add a test for it to CI
2021-10-18 08:14:12 -07:00
David Robertson 37b845dabc
Don't remove local users from dir when the leave their last room (#11103) 2021-10-18 13:20:04 +00:00
David Robertson e09be0c87a
Correctly exclude users when making a room public or private (#11075)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-10-15 15:53:05 +01:00
Patrick Cloke 5573133348
Move experimental & retention config out of the server module. (#11070) 2021-10-15 14:30:48 +00:00
Sean Quah 6a67f3786a
Fix logging context warnings when losing replication connection (#10984)
Instead of triggering `__exit__` manually on the replication handler's
logging context, use it as a context manager so that there is an
`__enter__` call to balance the `__exit__`.
2021-10-15 13:10:58 +01:00
Dirk Klimpel 013e0f9cae
Update doc of the allowed characters for registration tokens (#11093)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-15 09:56:39 +00:00