MirrorHub/synapse
0
0
Fork 1
mirror of https://mau.dev/maunium/synapse.git synced 2024-11-18 07:52:56 +01:00
Code Issues Projects Releases Wiki Activity
20688 commits 5 branches 731 tags 91 MiB
Commit graph

5874 commits

Author SHA1 Message Date
Shay
cde8af9a49
Add config flags to allow for cache auto-tuning (#12701) 2022-05-13 12:32:39 -07:00
Till
e8ae472d3b
Update configs used by Complement to allow more invites (#12731) 2022-05-13 16:45:47 +01:00
Brendan Abolivier
9013104429
Don't create an empty room when checking for MAU limits (#12713) 2022-05-13 15:30:15 +02:00
David Robertson
aec69d2481
Another batch of type annotations (#12726) 2022-05-13 12:35:31 +01:00
Jess Porter
39bed28b28
SpamChecker metrics (#12513) 
* add Measure blocks all over SpamChecker

Signed-off-by: jesopo <github@lolnerd.net>

* fix test_spam_checker_may_join_room and test_threepid_invite_spamcheck

* better changelog entry
 2022-05-13 12:17:38 +01:00
Niklas
c9fc2c0d22
Update issuer URL in example OIDC Keycloak config (#12727) 
* Update openid.md

Newer versions of keycloak returning a 404 when using the `/auth` prefix.

Related: https://github.com/matrix-org/synapse/issues/12714
 2022-05-13 10:15:51 +00:00
Andrew Morgan
57f6c496d0
URL preview cache expiry logs: INFO -> DEBUG, text clarifications (#12720) 2022-05-12 18:16:32 +01:00
David Robertson
17e1eb7749
Reduce the number of "untyped defs" (#12716) 2022-05-12 14:33:50 +00:00
Andy Balaam
de1e599b9d
add default_power_level_content_override config option. (#12618) 
Co-authored-by: Matthew Hodgson <matthew@matrix.org>
 2022-05-12 10:41:35 +00:00
Andrew Morgan
409573f6d0
Fix reference to the wrong symbol in the media admin api docs (#12715) 2022-05-12 09:29:37 +01:00
Sean Quah
bf7ce92bf7
Enable cancellation of GET /members and GET /state requests (#12708) 
Enable cancellation of `GET /rooms/$room_id/members`,
`GET /rooms/$room_id/state` and
`GET /rooms/$room_id/state/$state_key/*` requests.

Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-11 17:22:34 +01:00
David Robertson
db10f2c037
No longer permit empty body when sending receipts (#12709) 2022-05-11 15:34:17 +00:00
Sean Quah
6ee61b9052
Complain if a federation endpoint has the @cancellable flag (#12705) 
`BaseFederationServlet` wraps its endpoints in a bunch of async code
that has not been vetted for compatibility with cancellation.
Fail CI if a `@cancellable` flag is applied to a federation endpoint.

Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-11 14:52:26 +01:00
David Robertson
d38d242411
Reload cache factors from disk on SIGHUP (#12673) 2022-05-11 13:43:22 +00:00
Sean Quah
a559c8b0d9
Respect the @cancellable flag for ReplicationEndpoints (#12700) 
While `ReplicationEndpoint`s register themselves via `JsonResource`,
they pass a method that calls the handler, instead of the handler itself,
to `register_paths`. As a result, `JsonResource` will not correctly pick
up the `@cancellable` flag and we have to apply it ourselves.

Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-11 12:25:39 +01:00
Sean Quah
9d8e380d2e
Respect the @cancellable flag for RestServlets and BaseFederationServlets (#12699) 
Both `RestServlet`s and `BaseFederationServlet`s register their handlers
with `HttpServer.register_paths` / `JsonResource.register_paths`. Update
`JsonResource` to respect the `@cancellable` flag on handlers registered
in this way.

Although `ReplicationEndpoint` also registers itself using
`register_paths`, it does not pass the handler method that would have the
`@cancellable` flag directly, and so needs separate handling.

Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-11 12:25:13 +01:00
Sean Quah
dffecade7d
Respect the @cancellable flag for DirectServe{Html,Json}Resources (#12698) 
`DirectServeHtmlResource` and `DirectServeJsonResource` both inherit
from `_AsyncResource`. These classes expect to be subclassed with
`_async_render_*` methods.

This commit has no effect on `JsonResource`, despite inheriting from
`_AsyncResource`. `JsonResource` has its own `_async_render` override
which will need to be updated separately.

Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-11 12:24:48 +01:00
Patrick Cloke
a4c75918b3
Remove unneeded ActionGenerator class. (#12691) 
It simply passes through to `BulkPushRuleEvaluator`, which can be
called directly instead.
 2022-05-11 07:15:21 -04:00
Eric Eastwood
84facf769e
Fix /messages throwing a 500 when querying for non-existent room (#12683) 
Fix https://github.com/matrix-org/synapse/issues/12678

Complement test added:  https://github.com/matrix-org/complement/pull/369

**Before:** 500 internal server error

**After:** According to the [spec](https://spec.matrix.org/latest/client-server-api/#get_matrixclientv3roomsroomidmessages), calling `/messages` against a non-existent `room_id` should throw a 403 forbidden (since you're not part of the room). This also matches the behavior before https://github.com/matrix-org/synapse/pull/12370 which regressed Synapse to the 500 behavior.
```json
{
    "errcode": "M_FORBIDDEN",
    "error": "User @test:my.synapse.server not in room !dne:my.synapse.server, and room previews are disabled"
}
```
 2022-05-10 23:39:14 -05:00
Erik Johnston
c72d26c1e1
Refactor EventContext (#12689) 
Refactor how the `EventContext` class works, with the intention of reducing the amount of state we fetch from the DB during event processing.

The idea here is to get rid of the cached `current_state_ids` and `prev_state_ids` that live in the `EventContext`, and instead defer straight to the database (and its caching). 

One change that may have a noticeable effect is that we now no longer prefill the `get_current_state_ids` cache on a state change. However, that query is relatively light, since its just a case of reading a table from the DB (unlike fetching state at an event which is more heavyweight). For deployments with workers this cache isn't even used.


Part of #12684
 2022-05-10 19:43:13 +00:00
Sean Quah
c997bfb926
Capture the Deferred for request cancellation in _AsyncResource (#12694) 
All async request processing goes through `_AsyncResource`, so this is
the only place where a `Deferred` needs to be captured for cancellation.

Unfortunately, the same isn't true for determining whether a request
can be cancelled. Each of `RestServlet`, `BaseFederationServlet`,
`DirectServe{Html,Json}Resource` and `ReplicationEndpoint` have
different wrappers around the method doing the request handling and they
all need to be handled separately.

Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-10 20:39:05 +01:00
Patrick Cloke
29f06704b8
Fix incorrect type hint in filtering code. (#12695) 2022-05-10 14:10:22 -04:00
Dirk Klimpel
989fa33096
Add some type hints to datastore. (#12477) 2022-05-10 14:07:48 -04:00
Richard van der Hoff
147f098fb4
Stop writing to event_reference_hashes (#12679) 
This table is never read, since #11794. We stop writing to it; in future we can
drop it altogether.
 2022-05-10 15:35:08 +01:00
Sean Quah
dbb12a0b54
Add helper class for testing request cancellation (#12630) 
Also expose the `SynapseRequest` from `FakeChannel` in tests, so that
we can call `Request.connectionLost` to simulate a client disconnecting.

Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-10 14:06:56 +01:00
Sean Quah
5cfb004595
Add ability to cancel disconnected requests to SynapseRequest (#12588) 
Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-10 14:06:08 +01:00
Sean Quah
5c00151c28
Add @cancellable decorator, for use on request handlers (#12586) 
Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-10 14:05:22 +01:00
David Robertson
2aad0ae57f Synapse 1.59.0rc1 (2022-05-10) 
==============================
 
 This release makes several changes that server administrators should be aware of:
 
 - Device name lookup over federation is now disabled by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616))
 - The `synapse.app.appservice` and `synapse.app.user_dir` worker application types are now deprecated. ([\#12452](https://github.com/matrix-org/synapse/issues/12452), [\#12654](https://github.com/matrix-org/synapse/issues/12654))
 
 See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1590) for more details.
 
 Additionally, this release removes the non-standard `m.login.jwt` login type from Synapse. It can be replaced with `org.matrix.login.jwt` for identical behaviour. This is only used if `jwt_config.enabled` is set to `true` in the configuration. ([\#12597](https://github.com/matrix-org/synapse/issues/12597))
 
 Features
 --------
 
 - Support [MSC3266](https://github.com/matrix-org/matrix-doc/pull/3266) room summaries over federation. ([\#11507](https://github.com/matrix-org/synapse/issues/11507))
 - Implement [changes](4a77139249) to [MSC2285 (hidden read receipts)](https://github.com/matrix-org/matrix-spec-proposals/pull/2285). Contributed by @SimonBrandner. ([\#12168](https://github.com/matrix-org/synapse/issues/12168), [\#12635](https://github.com/matrix-org/synapse/issues/12635), [\#12636](https://github.com/matrix-org/synapse/issues/12636), [\#12670](https://github.com/matrix-org/synapse/issues/12670))
 - Extend the [module API](https://github.com/matrix-org/synapse/blob/release-v1.59/synapse/module_api/__init__.py) to allow modules to change actions for existing push rules of local users. ([\#12406](https://github.com/matrix-org/synapse/issues/12406))
 - Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. ([\#12452](https://github.com/matrix-org/synapse/issues/12452))
 - Add the `update_user_directory_from_worker` configuration option (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. ([\#12654](https://github.com/matrix-org/synapse/issues/12654))
 - Add new `enable_registration_token_3pid_bypass` configuration option to allow registrations via token as an alternative to verifying a 3pid. ([\#12526](https://github.com/matrix-org/synapse/issues/12526))
 - Implement [MSC3786](https://github.com/matrix-org/matrix-spec-proposals/pull/3786): Add a default push rule to ignore `m.room.server_acl` events. ([\#12601](https://github.com/matrix-org/synapse/issues/12601))
 - Add new `mau_appservice_trial_days` configuration option to specify a different trial period for users registered via an appservice. ([\#12619](https://github.com/matrix-org/synapse/issues/12619))
 
 Bugfixes
 --------
 
 - Fix a bug introduced in Synapse 1.48.0 where the latest thread reply provided failed to include the proper bundled aggregations. ([\#12273](https://github.com/matrix-org/synapse/issues/12273))
 - Fix a bug introduced in Synapse 1.22.0 where attempting to send a large amount of read receipts to an application service all at once would result in duplicate content and abnormally high memory usage. Contributed by Brad & Nick @ Beeper. ([\#12544](https://github.com/matrix-org/synapse/issues/12544))
 - Fix a bug introduced in Synapse 1.57.0 which could cause `Failed to calculate hosts in room` errors to be logged for outbound federation. ([\#12570](https://github.com/matrix-org/synapse/issues/12570))
 - Fix a long-standing bug where status codes would almost always get logged as `200!`, irrespective of the actual status code, when clients disconnect before a request has finished processing. ([\#12580](https://github.com/matrix-org/synapse/issues/12580))
 - Fix race when persisting an event and deleting a room that could lead to outbound federation breaking. ([\#12594](https://github.com/matrix-org/synapse/issues/12594))
 - Fix a bug introduced in Synapse 1.53.0 where bundled aggregations for annotations/edits were incorrectly calculated. ([\#12633](https://github.com/matrix-org/synapse/issues/12633))
 - Fix a long-standing bug where rooms containing power levels with string values could not be upgraded. ([\#12657](https://github.com/matrix-org/synapse/issues/12657))
 - Prevent memory leak from reoccurring when presence is disabled. ([\#12656](https://github.com/matrix-org/synapse/issues/12656))
 
 Updates to the Docker image
 ---------------------------
 
 - Explicitly opt-in to using [BuildKit-specific features](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md) in the Dockerfile. This fixes issues with building images in some GitLab CI environments. ([\#12541](https://github.com/matrix-org/synapse/issues/12541))
 - Update the "Build docker images" GitHub Actions workflow to use `docker/metadata-action` to generate docker image tags, instead of a custom shell script. Contributed by @henryclw. ([\#12573](https://github.com/matrix-org/synapse/issues/12573))
 
 Improved Documentation
 ----------------------
 
 - Update SQL statements and replace use of old table `user_stats_historical` in docs for Synapse Admins. ([\#12536](https://github.com/matrix-org/synapse/issues/12536))
 - Add missing linebreak to `pipx` install instructions. ([\#12579](https://github.com/matrix-org/synapse/issues/12579))
 - Add information about the TCP replication module to docs. ([\#12621](https://github.com/matrix-org/synapse/issues/12621))
 - Fixes to the formatting of `README.rst`. ([\#12627](https://github.com/matrix-org/synapse/issues/12627))
 - Fix docs on how to run specific Complement tests using the `complement.sh` test runner. ([\#12664](https://github.com/matrix-org/synapse/issues/12664))
 
 Deprecations and Removals
 -------------------------
 
 - Remove unstable identifiers from [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069). ([\#12596](https://github.com/matrix-org/synapse/issues/12596))
 - Remove the unspecified `m.login.jwt` login type and the unstable `uk.half-shot.msc2778.login.application_service` from
   [MSC2778](https://github.com/matrix-org/matrix-doc/pull/2778). ([\#12597](https://github.com/matrix-org/synapse/issues/12597))
 - Synapse now requires at least Python 3.7.1 (up from 3.7.0), for compatibility with the latest Twisted trunk. ([\#12613](https://github.com/matrix-org/synapse/issues/12613))
 
 Internal Changes
 ----------------
 
 - Use supervisord to supervise Postgres and Caddy in the Complement image to reduce restart time. ([\#12480](https://github.com/matrix-org/synapse/issues/12480))
 - Immediately retry any requests that have backed off when a server comes back online. ([\#12500](https://github.com/matrix-org/synapse/issues/12500))
 - Use `make_awaitable` instead of `defer.succeed` for return values of mocks in tests. ([\#12505](https://github.com/matrix-org/synapse/issues/12505))
 - Consistently check if an object is a `frozendict`. ([\#12564](https://github.com/matrix-org/synapse/issues/12564))
 - Protect module callbacks with read semantics against cancellation. ([\#12568](https://github.com/matrix-org/synapse/issues/12568))
 - Improve comments and error messages around access tokens. ([\#12577](https://github.com/matrix-org/synapse/issues/12577))
 - Improve docstrings for the receipts store. ([\#12581](https://github.com/matrix-org/synapse/issues/12581))
 - Use constants for read-receipts in tests. ([\#12582](https://github.com/matrix-org/synapse/issues/12582))
 - Log status code of cancelled requests as 499 and avoid logging stack traces for them. ([\#12587](https://github.com/matrix-org/synapse/issues/12587), [\#12663](https://github.com/matrix-org/synapse/issues/12663))
 - Remove special-case for `twisted` logger from default log config. ([\#12589](https://github.com/matrix-org/synapse/issues/12589))
 - Use `getClientAddress` instead of the deprecated `getClientIP`. ([\#12599](https://github.com/matrix-org/synapse/issues/12599))
 - Add link to documentation in Grafana Dashboard. ([\#12602](https://github.com/matrix-org/synapse/issues/12602))
 - Reduce log spam when running multiple event persisters. ([\#12610](https://github.com/matrix-org/synapse/issues/12610))
 - Add extra debug logging to federation sender. ([\#12614](https://github.com/matrix-org/synapse/issues/12614))
 - Prevent remote homeservers from requesting local user device names by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616))
 - Add a consistency check on events which we read from the database. ([\#12620](https://github.com/matrix-org/synapse/issues/12620))
 - Remove use of the `constantly` library and switch to enums for `EventRedactBehaviour`. Contributed by @andrewdoh. ([\#12624](https://github.com/matrix-org/synapse/issues/12624))
 - Remove unused code related to receipts. ([\#12632](https://github.com/matrix-org/synapse/issues/12632))
 - Minor improvements to the scripts for running Synapse in worker mode under Complement. ([\#12637](https://github.com/matrix-org/synapse/issues/12637))
 - Move `pympler` back in to the `all` extras. ([\#12652](https://github.com/matrix-org/synapse/issues/12652))
 - Fix spelling of `M_UNRECOGNIZED` in comments. ([\#12665](https://github.com/matrix-org/synapse/issues/12665))
 - Release script: confirm the commit to be tagged before tagging. ([\#12556](https://github.com/matrix-org/synapse/issues/12556))
 - Fix a typo in the announcement text generated by the Synapse release development script. ([\#12612](https://github.com/matrix-org/synapse/issues/12612))
 
 - Fix scripts-dev to pass typechecking. ([\#12356](https://github.com/matrix-org/synapse/issues/12356))
 - Add some type hints to datastore. ([\#12485](https://github.com/matrix-org/synapse/issues/12485))
 - Remove unused `# type: ignore`s. ([\#12531](https://github.com/matrix-org/synapse/issues/12531))
 - Allow unused `# type: ignore` comments in bleeding edge CI jobs. ([\#12576](https://github.com/matrix-org/synapse/issues/12576))
 - Remove redundant lines of config from `mypy.ini`. ([\#12608](https://github.com/matrix-org/synapse/issues/12608))
 - Update to mypy 0.950. ([\#12650](https://github.com/matrix-org/synapse/issues/12650))
 - Use `Concatenate` to better annotate `_do_execute`. ([\#12666](https://github.com/matrix-org/synapse/issues/12666))
 - Use `ParamSpec` to refine type hints. ([\#12667](https://github.com/matrix-org/synapse/issues/12667))
 - Fix mypy against latest pillow stubs. ([\#12671](https://github.com/matrix-org/synapse/issues/12671))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmJ6UDoACgkQkD7OEIo5
 3t3XMw/9GfROfC///lio77EhdI0lh39bkgvfqkFE/wRYtTs4n3ZSiclgVBnNmgnN
 WnLvjL/4wXRjzJZ8vcO+qG0OasT3H9l5xdGE+7h8oUMIYCG7Xdyx37xnhv5IC8Y5
 dAjqs2mMnIWnAqyOAVZQVgmNLDhRAb9YGCJ3fJY9cV8dmzr8/RzR8gXrFPaBT9Nj
 w5Xag0jkiJuff4+Y5oan2yBxMvHT/zAb8rQf5wNjen00lwYHSAr/5aiAsbnHcMYM
 q0ZahVB/hVaA5GqxwrySlE0mqrPP6M/CpbcLr1h8LlmLdgA6f3vcf/sREGq6wCRK
 Z8yYXM4RyvNTyYirQfnP5QseVazJqcCY0pCXrMnQxwXda7nSDKjXo0WFBmyIQpK4
 YqDvlOhECr+Pcac/EAviH2OEvTMBg4cUs1xdm4/CBElIOZQPyXHJ0DLku8R0CRx/
 i01fN0I/r/wX1Oliy4wFJgSYt/uUQzZaJmRieK8fznlFibM3Gk1GvniX0M5Hu6cZ
 PQuHdPqRxQzvBmEAA1hxTfyOaDeP3g29NnOz9cjsBiAVNqUbEV5KFusjwksI4kcT
 Bsdk+v/i8/Kx6t0ovRI2i3JJAFNWmUzpY0iDzY3kJCn1YmEMZ7rs+e5qRmv2eBLW
 oS9dhLGsW7A/mJC6fLM+CATwbFiL/SqB5ocnl4POodZ/b27eDyw=
 =NPHq
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmJ6V/YACgkQkD7OEIo5
 3t0ANA/7BZ762LAjLNsFvJVxnH8LjDEjsjCyjHQ/zzVkAete3354zcW/mqN2YzMl
 shaxHucfZ9sKCXlOSuwY9N+1Q24ALMAb8QebCQZjNamxKqddpMD0+ijeaKRoFq8P
 6VY5saL+rjW0g2QaKE/IHc99UKnw/cNN/vGa3xc3dJFRiiNkK95jS8R7aaaT2mXB
 9frbbN0Q45Q6JTU9qJTP6jqdYmKZNjvIHBJcEXkstGqS8NPJXpfVKMZIgTj/8nGL
 RVJqOStsnHdx4vywZ76GPTAxQ+ZjXMilIbz+bF/6Zw1SCJEZfwFOelYApwMi0qU7
 yDK7xeokKxzXyykzmptmiqmePCbr9fHbk+BJtE7VblMdj7OaM7jX7foSX6J1elIp
 zwDwBK0VH3CJzk9SRoy4JW3PI9K45DZodev33CT+sT39/wYTiV3mbGIYkgP8oSDH
 rd588QZDovVDy6NWzxkT0MsutHE5Rhx2aP+hkIFDHL4WWnKeFRD5HbiZfg/eC6r0
 0nPRY2CCikvqLRqefTi4dgxb+twVBnioPZiDBTEKA/M1/qVg978SIpdHsfmaDJf9
 BzWsvfsns0t54J2QnazOjKx2yxIYH6pZ2VFnrb47404O5O3+TuKZL9eX8oNZeBFl
 JKZKD2xii/ktkw5p27RxEpBJ13hrssN+RMd0JrvS3LTa5G9aIUM=
 =2gaD
 -----END PGP SIGNATURE-----

Merge tag 'v1.59.0rc1' into develop

Synapse 1.59.0rc1 (2022-05-10)
==============================

This release makes several changes that server administrators should be aware of:

- Device name lookup over federation is now disabled by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616))
- The `synapse.app.appservice` and `synapse.app.user_dir` worker application types are now deprecated. ([\#12452](https://github.com/matrix-org/synapse/issues/12452), [\#12654](https://github.com/matrix-org/synapse/issues/12654))

See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1590) for more details.

Additionally, this release removes the non-standard `m.login.jwt` login type from Synapse. It can be replaced with `org.matrix.login.jwt` for identical behaviour. This is only used if `jwt_config.enabled` is set to `true` in the configuration. ([\#12597](https://github.com/matrix-org/synapse/issues/12597))

Features
--------

- Support [MSC3266](https://github.com/matrix-org/matrix-doc/pull/3266) room summaries over federation. ([\#11507](https://github.com/matrix-org/synapse/issues/11507))
- Implement [changes](4a77139249) to [MSC2285 (hidden read receipts)](https://github.com/matrix-org/matrix-spec-proposals/pull/2285). Contributed by @SimonBrandner. ([\#12168](https://github.com/matrix-org/synapse/issues/12168), [\#12635](https://github.com/matrix-org/synapse/issues/12635), [\#12636](https://github.com/matrix-org/synapse/issues/12636), [\#12670](https://github.com/matrix-org/synapse/issues/12670))
- Extend the [module API](https://github.com/matrix-org/synapse/blob/release-v1.59/synapse/module_api/__init__.py) to allow modules to change actions for existing push rules of local users. ([\#12406](https://github.com/matrix-org/synapse/issues/12406))
- Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. ([\#12452](https://github.com/matrix-org/synapse/issues/12452))
- Add the `update_user_directory_from_worker` configuration option (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. ([\#12654](https://github.com/matrix-org/synapse/issues/12654))
- Add new `enable_registration_token_3pid_bypass` configuration option to allow registrations via token as an alternative to verifying a 3pid. ([\#12526](https://github.com/matrix-org/synapse/issues/12526))
- Implement [MSC3786](https://github.com/matrix-org/matrix-spec-proposals/pull/3786): Add a default push rule to ignore `m.room.server_acl` events. ([\#12601](https://github.com/matrix-org/synapse/issues/12601))
- Add new `mau_appservice_trial_days` configuration option to specify a different trial period for users registered via an appservice. ([\#12619](https://github.com/matrix-org/synapse/issues/12619))

Bugfixes
--------

- Fix a bug introduced in Synapse 1.48.0 where the latest thread reply provided failed to include the proper bundled aggregations. ([\#12273](https://github.com/matrix-org/synapse/issues/12273))
- Fix a bug introduced in Synapse 1.22.0 where attempting to send a large amount of read receipts to an application service all at once would result in duplicate content and abnormally high memory usage. Contributed by Brad & Nick @ Beeper. ([\#12544](https://github.com/matrix-org/synapse/issues/12544))
- Fix a bug introduced in Synapse 1.57.0 which could cause `Failed to calculate hosts in room` errors to be logged for outbound federation. ([\#12570](https://github.com/matrix-org/synapse/issues/12570))
- Fix a long-standing bug where status codes would almost always get logged as `200!`, irrespective of the actual status code, when clients disconnect before a request has finished processing. ([\#12580](https://github.com/matrix-org/synapse/issues/12580))
- Fix race when persisting an event and deleting a room that could lead to outbound federation breaking. ([\#12594](https://github.com/matrix-org/synapse/issues/12594))
- Fix a bug introduced in Synapse 1.53.0 where bundled aggregations for annotations/edits were incorrectly calculated. ([\#12633](https://github.com/matrix-org/synapse/issues/12633))
- Fix a long-standing bug where rooms containing power levels with string values could not be upgraded. ([\#12657](https://github.com/matrix-org/synapse/issues/12657))
- Prevent memory leak from reoccurring when presence is disabled. ([\#12656](https://github.com/matrix-org/synapse/issues/12656))

Updates to the Docker image
---------------------------

- Explicitly opt-in to using [BuildKit-specific features](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md) in the Dockerfile. This fixes issues with building images in some GitLab CI environments. ([\#12541](https://github.com/matrix-org/synapse/issues/12541))
- Update the "Build docker images" GitHub Actions workflow to use `docker/metadata-action` to generate docker image tags, instead of a custom shell script. Contributed by @henryclw. ([\#12573](https://github.com/matrix-org/synapse/issues/12573))

Improved Documentation
----------------------

- Update SQL statements and replace use of old table `user_stats_historical` in docs for Synapse Admins. ([\#12536](https://github.com/matrix-org/synapse/issues/12536))
- Add missing linebreak to `pipx` install instructions. ([\#12579](https://github.com/matrix-org/synapse/issues/12579))
- Add information about the TCP replication module to docs. ([\#12621](https://github.com/matrix-org/synapse/issues/12621))
- Fixes to the formatting of `README.rst`. ([\#12627](https://github.com/matrix-org/synapse/issues/12627))
- Fix docs on how to run specific Complement tests using the `complement.sh` test runner. ([\#12664](https://github.com/matrix-org/synapse/issues/12664))

Deprecations and Removals
-------------------------

- Remove unstable identifiers from [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069). ([\#12596](https://github.com/matrix-org/synapse/issues/12596))
- Remove the unspecified `m.login.jwt` login type and the unstable `uk.half-shot.msc2778.login.application_service` from
  [MSC2778](https://github.com/matrix-org/matrix-doc/pull/2778). ([\#12597](https://github.com/matrix-org/synapse/issues/12597))
- Synapse now requires at least Python 3.7.1 (up from 3.7.0), for compatibility with the latest Twisted trunk. ([\#12613](https://github.com/matrix-org/synapse/issues/12613))

Internal Changes
----------------

- Use supervisord to supervise Postgres and Caddy in the Complement image to reduce restart time. ([\#12480](https://github.com/matrix-org/synapse/issues/12480))
- Immediately retry any requests that have backed off when a server comes back online. ([\#12500](https://github.com/matrix-org/synapse/issues/12500))
- Use `make_awaitable` instead of `defer.succeed` for return values of mocks in tests. ([\#12505](https://github.com/matrix-org/synapse/issues/12505))
- Consistently check if an object is a `frozendict`. ([\#12564](https://github.com/matrix-org/synapse/issues/12564))
- Protect module callbacks with read semantics against cancellation. ([\#12568](https://github.com/matrix-org/synapse/issues/12568))
- Improve comments and error messages around access tokens. ([\#12577](https://github.com/matrix-org/synapse/issues/12577))
- Improve docstrings for the receipts store. ([\#12581](https://github.com/matrix-org/synapse/issues/12581))
- Use constants for read-receipts in tests. ([\#12582](https://github.com/matrix-org/synapse/issues/12582))
- Log status code of cancelled requests as 499 and avoid logging stack traces for them. ([\#12587](https://github.com/matrix-org/synapse/issues/12587), [\#12663](https://github.com/matrix-org/synapse/issues/12663))
- Remove special-case for `twisted` logger from default log config. ([\#12589](https://github.com/matrix-org/synapse/issues/12589))
- Use `getClientAddress` instead of the deprecated `getClientIP`. ([\#12599](https://github.com/matrix-org/synapse/issues/12599))
- Add link to documentation in Grafana Dashboard. ([\#12602](https://github.com/matrix-org/synapse/issues/12602))
- Reduce log spam when running multiple event persisters. ([\#12610](https://github.com/matrix-org/synapse/issues/12610))
- Add extra debug logging to federation sender. ([\#12614](https://github.com/matrix-org/synapse/issues/12614))
- Prevent remote homeservers from requesting local user device names by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616))
- Add a consistency check on events which we read from the database. ([\#12620](https://github.com/matrix-org/synapse/issues/12620))
- Remove use of the `constantly` library and switch to enums for `EventRedactBehaviour`. Contributed by @andrewdoh. ([\#12624](https://github.com/matrix-org/synapse/issues/12624))
- Remove unused code related to receipts. ([\#12632](https://github.com/matrix-org/synapse/issues/12632))
- Minor improvements to the scripts for running Synapse in worker mode under Complement. ([\#12637](https://github.com/matrix-org/synapse/issues/12637))
- Move `pympler` back in to the `all` extras. ([\#12652](https://github.com/matrix-org/synapse/issues/12652))
- Fix spelling of `M_UNRECOGNIZED` in comments. ([\#12665](https://github.com/matrix-org/synapse/issues/12665))
- Release script: confirm the commit to be tagged before tagging. ([\#12556](https://github.com/matrix-org/synapse/issues/12556))
- Fix a typo in the announcement text generated by the Synapse release development script. ([\#12612](https://github.com/matrix-org/synapse/issues/12612))

- Fix scripts-dev to pass typechecking. ([\#12356](https://github.com/matrix-org/synapse/issues/12356))
- Add some type hints to datastore. ([\#12485](https://github.com/matrix-org/synapse/issues/12485))
- Remove unused `# type: ignore`s. ([\#12531](https://github.com/matrix-org/synapse/issues/12531))
- Allow unused `# type: ignore` comments in bleeding edge CI jobs. ([\#12576](https://github.com/matrix-org/synapse/issues/12576))
- Remove redundant lines of config from `mypy.ini`. ([\#12608](https://github.com/matrix-org/synapse/issues/12608))
- Update to mypy 0.950. ([\#12650](https://github.com/matrix-org/synapse/issues/12650))
- Use `Concatenate` to better annotate `_do_execute`. ([\#12666](https://github.com/matrix-org/synapse/issues/12666))
- Use `ParamSpec` to refine type hints. ([\#12667](https://github.com/matrix-org/synapse/issues/12667))
- Fix mypy against latest pillow stubs. ([\#12671](https://github.com/matrix-org/synapse/issues/12671))
 2022-05-10 13:17:56 +01:00
Patrick Cloke
b44fbdffa4
Move free functions into PushRuleEvaluatorForEvent. (#12677) 
* Move `_condition_checker` into `PushRuleEvaluatorForEvent`.
* Move the condition cache into `PushRuleEvaluatorForEvent`.
* Improve docstrings.
* Inline a method which is only called once.
 2022-05-10 07:54:30 -04:00
Patrick Cloke
02cdace707
Add class-diagrams and notes for push. (#12676) 2022-05-10 07:43:34 -04:00
David Robertson
2bae6d93c9
I manually added O's change, remove newsfile 2022-05-10 11:17:42 +01:00
reivilibre
699192fc1a
Add the update_user_directory_from_worker configuration option (superseding update_user_directory) to allow a generic worker to be designated as the worker to update the user directory. (#12654) 
Co-authored-by: Shay <hillerys@element.io>
 2022-05-10 11:08:45 +01:00
David Robertson
e5fd23fb6f
1.59.0rc1 2022-05-10 10:45:13 +01:00
Erik Johnston
8dd3e0e084
Immediately retry any requests that have backed off when a server comes back online. (#12500) 
Otherwise it can take up to a minute for any in-flight `/send` requests to be retried.
 2022-05-10 10:39:54 +01:00
Šimon Brandner
ade3008821
Implement MSC3786: Add a default push rule to ignore m.room.server_acl events (#12601) 
Fixes vector-im/element-web#20788
Implements matrix-org/matrix-spec-proposals#3786
 2022-05-10 08:57:36 +01:00
Shay
d80a7ab151
Update replication.md with info on TCP module structure (#12621) 2022-05-09 14:46:43 -07:00
Dirk Klimpel
615d96ad6e
Update SQL statements in docs for Synapse Admins (#12536) 2022-05-09 14:43:02 -07:00
Richard van der Hoff
34e84fee68
Tweaks to workers-under-complement (#12637) 
* Bump the HS startup timeout
* Log prefixes for more processes
* Bump the overall timeout
 2022-05-09 22:41:06 +01:00
Val Lorentz
bf0c3ca20a
Fix inconsistent spelling of 'M_UNRECOGNIZED'. (#12665) 2022-05-09 20:29:07 +00:00
Sean Quah
a00462dd99
Implement cancellation support/protection for module callbacks (#12568) 
There's no guarantee that module callbacks will handle cancellation
appropriately. Protect module callbacks with read semantics from
cancellation and avoid swallowing `CancelledError`s that arise.

Other module callbacks, such as the `on_*` callbacks, are presumed to
live on code paths that involve writes and aren't cancellation-friendly.
These module callbacks have been left alone.

Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-09 12:31:14 +01:00
David Robertson
8de0facaae
Fix mypy against latest pillow stubs (#12671) 2022-05-09 10:48:14 +00:00
Sean Quah
41a882e62d
Update changelog for #12587 to be more accurate (#12663) 
#12587 has fallen on the wrong side of the release cutoff to the rest of
the related PRs.

Signed-off-by: Sean Quah <seanq@element.io>
 2022-05-09 11:34:39 +01:00
David Robertson
fa0eab9c8e
Use ParamSpec in a few places (#12667) 2022-05-09 10:27:39 +00:00
Erik Johnston
c5969b346d
Don't error on unknown receipt types (#12670) 
Fixes #12669
 2022-05-09 11:09:19 +01:00
Sheogorath
77258b6725
docs(contrib): Add link to documentation in dashboard (#12602) 2022-05-09 10:08:31 +00:00
Eric Eastwood
18d6c18aa1
Fix docs on how to run specific Complement tests after recent complement.sh change (#12664) 2022-05-09 10:38:32 +01:00
David Robertson
26c1ad71c5
Use Concatenate to annotate do_execute (#12666) 2022-05-09 10:28:38 +01:00
David Robertson
0ce2201932
Move pympler back into the all extras (#12652) 
* Move `pympler` back into the `all` extras

Undoes a change I made in #12381. I can't fully remember my reasoning,
but this changed the contents of the debian packages in a backwards
incompatible way. We're not aware of anyone who's been bitten by this,
but we still want to fix it.

To the reviewer: please be convinced that the debian packages will still
contain pympler after this change.

* Debian changelog entry to keep the linter happy
 2022-05-07 13:40:58 +01:00
David Robertson
051a1c3f22
Convert stringy power levels to integers on room upgrade (#12657) 2022-05-07 13:37:29 +01:00
Erik Johnston
4337d33a73
Prevent memory leak from reoccurring when presence is disabled. (#12656) 2022-05-06 16:41:57 +00:00