Commit graph

916 commits

Author SHA1 Message Date
Nick Mills-Barrett
982fe29655
Optimise room creation event lookups part 2 (#13224) 2022-07-13 19:32:46 +01:00
Thomas Weston
0312ff44c6
Fix "add user" admin api error when request contains a "msisdn" threepid (#13263)
Co-authored-by: Thomas Weston <thomas.weston@clearspancloud.com>
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-07-13 11:33:21 +01:00
Richard van der Hoff
fa71bb18b5
Drop support for delegating email validation (#13192)
* Drop support for delegating email validation

Delegating email validation to an IS is insecure (since it allows the owner of
the IS to do a password reset on your HS), and has long been deprecated. It
will now cause a config error at startup.

* Update unit test which checks for email verification

Give it an `email` config instead of a threepid delegate

* Remove unused method `requestEmailToken`

* Simplify config handling for email verification

Rather than an enum and a boolean, all we need here is a single bool, which
says whether we are or are not doing email verification.

* update docs

* changelog

* upgrade.md: fix typo

* update version number

this will be in 1.64, not 1.63

* update version number

this one too
2022-07-12 19:18:53 +01:00
andrew do
2d82cdafd2
expose whether a room is a space in the Admin API (#13208) 2022-07-12 15:30:53 +01:00
Erik Johnston
e5716b631c
Don't pull out the full state when calculating push actions (#13078) 2022-07-11 20:08:39 +00:00
Nick Mills-Barrett
92202ce867
Reduce event lookups during room creation by passing known event IDs (#13210)
Inspired by the room batch handler, this uses previous event inserts to
pre-populate prev events during room creation, reducing the number of
queries required to create a room.

Signed off by Nick @ Beeper (@Fizzadar)
2022-07-11 18:00:12 +01:00
David Teller
11f811470f
Uniformize spam-checker API, part 5: expand other spam-checker callbacks to return Tuple[Codes, dict] (#13044)
Signed-off-by: David Teller <davidt@element.io>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-07-11 16:52:10 +00:00
David Robertson
d102ad67fd
annotate tests.server.FakeChannel (#13136) 2022-07-04 18:08:56 +01:00
Šimon Brandner
13e359aec8
Implement MSC3827: Filtering of /publicRooms by room type (#13031)
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
2022-06-29 17:12:45 +00:00
Andrew Morgan
6cba6a51af Merge branch 'master' into develop 2022-06-28 15:19:48 +01:00
reivilibre
fa13080618
Merge pull request from GHSA-22p3-qrh9-cx32
* Make _iterate_over_text easier to read by using simple data structures

* Prefer a set of tags to ignore

In my tests, it's 4x faster to check for containment in a set of this size

* Add a stack size limit to _iterate_over_text

* Continue accepting the case where there is no body element

* Use an early return instead for None

Co-authored-by: Richard van der Hoff <richard@matrix.org>
2022-06-28 14:29:08 +01:00
Robert Long
9b683ea80f
Add Cross-Origin-Resource-Policy header to thumbnail and download media endpoints (#12944) 2022-06-27 14:44:05 +01:00
santhoshivan23
d54909956e
validate room alias before interacting with the room directory (#13106) 2022-06-22 15:32:18 +01:00
Quentin Gliech
73af10f419
Simplify the alias deletion logic as an application service. (#13093) 2022-06-17 12:19:22 +01:00
Patrick Cloke
0fcc0ae37c
Improve URL previews for sites with only Twitter card information. (#13056)
Pull out `twitter:` meta tags when generating a preview and
use it to augment any `og:` meta tags.

Prefers Open Graph information over Twitter card information.
2022-06-16 07:41:57 -04:00
Jacek Kuśnierz
0ef1307619
Add custom well-known (#13035)
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-06-16 11:48:18 +01:00
Sean Quah
99d3931974
Add more tests for room upgrades (#13074)
Signed-off-by: Sean Quah <seanq@element.io>
2022-06-15 18:58:23 +01:00
Hannes Lerchl
7d99414edf
Replace pyjwt with authlib in org.matrix.login.jwt (#13011) 2022-06-15 16:45:16 +00:00
David Robertson
c99b511db9
Fix destination_is errors seen in sentry. (#13041)
* Rename test_fedclient to match its source file
* Require at least one destination to be truthy
* Explicitly validate user ID in profile endpoint GETs
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-06-14 18:28:26 +01:00
David Teller
a164a46038
Uniformize spam-checker API, part 4: port other spam-checker callbacks to return Union[Allow, Codes]. (#12857)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
Patrick Cloke
dd2d66b0c9
Move the (unstable) dir parameter for /relations behind an experimental flag. (#12984)
MSC3715 defines this parameter, but the unstable version of it should be
behind an experimental flag.
2022-06-08 09:00:35 -04:00
Sean Quah
a10cc5f824
Test cancellation at every await during request handling (#12674)
* Add tests for `/rooms/<room_id>/members` cancellation.
* Add tests for `/rooms/<room_id>/state` cancellation.

Signed-off-by: Sean Quah <seanq@element.io>
2022-06-07 17:14:47 +01:00
Andrew Morgan
a47636c570
Prevent local quarantined media from being claimed by media retention (#12972) 2022-06-07 10:53:47 +00:00
Patrick Cloke
f7baffd8ec
Remove remaining pieces of groups code. (#12966)
* Remove an unused stream ID generator.
* Remove the now unused remote profile cache.
2022-06-06 13:20:05 -04:00
Patrick Cloke
148fe58a24
Do not break URL previews if an image is unreachable. (#12950)
Avoid breaking a URL preview completely if the chosen image 404s
or is unreachable for some other reason (e.g. DNS).
2022-06-06 07:46:04 -04:00
Patrick Cloke
1acc897c31
Implement MSC3816, consider the root event for thread participation. (#12766)
As opposed to only considering a user to have "participated" if they
replied to the thread.
2022-06-06 07:18:04 -04:00
Patrick Cloke
01df5bacac
Improve URL previews for some pages (#12951)
* Skip `og` and `meta` tags where the value is empty.
* Fallback to the favicon if there are no other images.
* Ignore tags meant for navigation.
2022-06-03 12:09:12 -04:00
Erik Johnston
888a29f412
Wait for lazy join to complete when getting current state (#12872) 2022-06-01 16:02:53 +01:00
Patrick Cloke
7bc08f3201
Remove remaining bits of groups code. (#12936)
* Update worker docs to remove group endpoints.
* Removes an unused parameter to `ApplicationService`.
* Break dependency between media repo and groups.
* Avoid copying `m.room.related_groups` state events during room upgrades.
2022-06-01 09:41:25 -04:00
Andrew Morgan
2fc787c341
Add config options for media retention (#12732) 2022-05-31 16:35:29 +00:00
Erik Johnston
1e453053cb
Rename storage classes (#12913) 2022-05-31 12:17:50 +00:00
DeepBlueV7.X
b10211871f
Fix invite notifications for users without pushers (#12840)
Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
2022-05-30 13:14:43 +02:00
Jonathan de Jong
6be4953b99
Mutual rooms: Remove dependency on user directory (#12836) 2022-05-30 10:05:31 +01:00
Brendan Abolivier
28989cb301
Add a background job to automatically delete stale devices (#12855)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-05-27 17:47:32 +02:00
Patrick Cloke
c52abc1cfd
Additional constants for EDU types. (#12884)
Instead of hard-coding strings in many places.
2022-05-27 07:14:36 -04:00
Sean Quah
053ca5f3ca Synapse 1.60.0rc2 (2022-05-27)
==============================
 
 This release of Synapse adds a unique index to the `state_group_edges` table, in
 order to prevent accidentally introducing duplicate information (for example,
 because a database backup was restored multiple times). If your Synapse database
 already has duplicate rows in this table, this could fail with an error and
 require manual remediation.
 
 Additionally, the signature of the `check_event_for_spam` module callback has changed.
 The previous signature has been deprecated and remains working for now. Module authors
 should update their modules to use the new signature where possible.
 
 See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1600)
 for more details.
 
 Features
 --------
 
 - Add an option allowing users to use their password to reauthenticate for privileged actions even though password login is disabled. ([\#12883](https://github.com/matrix-org/synapse/issues/12883))
 
 Bugfixes
 --------
 
 - Explicitly close `ijson` coroutines once we are done with them, instead of leaving the garbage collector to close them. ([\#12875](https://github.com/matrix-org/synapse/issues/12875))
 
 Internal Changes
 ----------------
 
 - Improve URL previews by not including the content of media tags in the generated description. ([\#12887](https://github.com/matrix-org/synapse/issues/12887))
 -----BEGIN PGP SIGNATURE-----
 
 iQGzBAABCgAdFiEEWMTnW8Z8khaaf90R+84KzgcyGG8FAmKQqMcACgkQ+84Kzgcy
 GG9Z2Av+N+b/fvaB3D56UkFqTW/xLmCEyri65njcXU8625bWiLSPM6hssmyJB1FA
 xlc2RBKr8QxlnHRS/v31wDtONC8YZ2O3fyzYPFfY1fF5Ul7Kg3XCzLeUH4/j1/Ar
 5bqriDqaN9FQ/6QJybShXlA4l7lY1Fs0C4P23jDBgqfKjnlToeVLqhVA70dDaFu/
 ir+vVprKCkQI1iqnYXwIxGRmgBzLWGoVqQFGbSI6hugGwXpGIyX7+2I+0v8tI6vA
 SZ99vLFWcvnd6DJTyBhIeV22Ff4qA7eQsyPvSrMETdsaZmrxGlG+t332HNCgplv8
 gv2gUpJL0br++5WTAX+nRc7HpfKo/74vKeTktqPmlvFP8kUOg+PbzmoJFUu21PhA
 rnq5TzgsPHK0dqBhM1RC2vtOiJ5v3ZBqzJJzSRXl6lsFpWxxFmwesEcIDAYS0Nmh
 QoJb7/L8cPCHksHvZM76bzB465tSH9NhuFYZQoLGHcpxa0kYekrdlYasP8U0FU7L
 nF3C0Pgw
 =D3F+
 -----END PGP SIGNATURE-----

Merge tag 'v1.60.0rc2' into develop

Synapse 1.60.0rc2 (2022-05-27)
==============================

This release of Synapse adds a unique index to the `state_group_edges` table, in
order to prevent accidentally introducing duplicate information (for example,
because a database backup was restored multiple times). If your Synapse database
already has duplicate rows in this table, this could fail with an error and
require manual remediation.

Additionally, the signature of the `check_event_for_spam` module callback has changed.
The previous signature has been deprecated and remains working for now. Module authors
should update their modules to use the new signature where possible.

See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1600)
for more details.

Features
--------

- Add an option allowing users to use their password to reauthenticate for privileged actions even though password login is disabled. ([\#12883](https://github.com/matrix-org/synapse/issues/12883))

Bugfixes
--------

- Explicitly close `ijson` coroutines once we are done with them, instead of leaving the garbage collector to close them. ([\#12875](https://github.com/matrix-org/synapse/issues/12875))

Internal Changes
----------------

- Improve URL previews by not including the content of media tags in the generated description. ([\#12887](https://github.com/matrix-org/synapse/issues/12887))
2022-05-27 12:07:18 +01:00
reivilibre
7b88f5a107
Add an option allowing users to use their password to reauthenticate even though password authentication is disabled. (#12883) 2022-05-27 09:44:51 +00:00
Patrick Cloke
a8db8c6eba
Remove user-visible groups/communities code (#12553)
Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove:

* All API endpoints (including Client-Server, Server-Server, and admin).
* Documented configuration options (and the experimental flag, which is now unused).
* Special handling during room upgrades.
* The `groups` section of the `/sync` response.
2022-05-25 07:53:40 -04:00
Brendan Abolivier
4cc4229cd7
Prevent expired events from being filtered out when retention is disabled (#12611)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-05-23 17:18:23 +00:00
Andrew Morgan
96df31239c
Add a unit test for copying over arbitrary room types when upgrading a room (#12792) 2022-05-19 18:32:48 +01:00
Mathieu Velten
182ca78a12
Delete events from federation_inbound_events_staging table on purge (#12770) 2022-05-17 17:01:06 +00:00
Patrick Cloke
86a515ccbf
Consolidate logic for parsing relations. (#12693)
Parse the `m.relates_to` event content field (which describes relations)
in a single place, this is used during:

* Event persistence.
* Validation of the Client-Server API.
* Fetching bundled aggregations.
* Processing of push rules.

Each of these separately implement the logic and each made slightly
different assumptions about what was valid. Some had minor / potential
bugs.
2022-05-16 12:42:45 +00:00
Jess Porter
39bed28b28
SpamChecker metrics (#12513)
* add Measure blocks all over SpamChecker

Signed-off-by: jesopo <github@lolnerd.net>

* fix test_spam_checker_may_join_room and test_threepid_invite_spamcheck

* better changelog entry
2022-05-13 12:17:38 +01:00
Andy Balaam
de1e599b9d
add default_power_level_content_override config option. (#12618)
Co-authored-by: Matthew Hodgson <matthew@matrix.org>
2022-05-12 10:41:35 +00:00
David Robertson
db10f2c037
No longer permit empty body when sending receipts (#12709) 2022-05-11 15:34:17 +00:00
David Robertson
051a1c3f22
Convert stringy power levels to integers on room upgrade (#12657) 2022-05-07 13:37:29 +01:00
Šimon Brandner
9ae0253f4e
Use private instead of hidden in MSC2285 related code. (#12635) 2022-05-05 12:31:25 +00:00
Patrick Cloke
f90d381c7b
Edits/annotations should not have any bundled aggregations calculated. (#12633)
Fixes a regression from 8b309adb43 (#11660)
and b65acead42 (#11752) where events which
themselves were an edit or an annotation could have bundled aggregations calculated,
which is not allowed.
2022-05-05 08:15:12 -04:00
Šimon Brandner
116a4c8340
Implement changes to MSC2285 (hidden read receipts) (#12168)
* Changes hidden read receipts to be a separate receipt type
  (instead of a field on `m.read`).
* Updates the `/receipts` endpoint to accept `m.fully_read`.
2022-05-04 11:59:22 -04:00
Patrick Cloke
ba3fd54bad
Remove unstable/unspecced login types. (#12597)
* `m.login.jwt`, which was never specced and has been deprecated
  since Synapse 1.16.0. (`org.matrix.login.jwt` can be used instead.)
* `uk.half-shot.msc2778.login.application_service`, which was
  stabilized as part of the Matrix spec v1.2 release.
2022-05-04 13:53:21 +00:00