tileradotorg/synapse
1
0
Fork 0
forked from MirrorHub/synapse
Code Pull requests Activity
19023 commits 4 branches 629 tags 119 MiB
Commit graph

19023 commits

This branch
This branch
All branches
Author SHA1 Message Date
Brendan Abolivier
d1473f7362
Use link to advisory rather than to the CVE repo 2021-05-11 14:09:46 +01:00
Aaron Raimist
dc6366a9bd
Add config option to hide device names over federation (#9945) 
Now that cross signing exists there is much less of a need for other people to look at devices and verify them individually. This PR adds a config option to allow you to prevent device display names from being shared with other servers.

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2021-05-11 14:03:23 +01:00
Brendan Abolivier
86fb71431c
1.33.2 2021-05-11 14:01:32 +01:00
Richard van der Hoff
b378d98c8f
Add debug logging for issue #9533 (#9959) 
Hopefully this will help us track down where to-device messages are getting
lost/delayed.
 2021-05-11 11:04:03 +01:00
Richard van der Hoff
7967b36efe
Fix m.room_key_request to-device messages (#9961) 
fixes #9960
 2021-05-11 11:02:56 +01:00
Richard van der Hoff
03318a766c
Merge pull request from GHSA-x345-32rc-8h85 
* tests for push rule pattern matching

* tests for acl pattern matching

* factor out common `re.escape`

* Factor out common re.compile

* Factor out common anchoring code

* add word_boundary support to `glob_to_regex`

* Use `glob_to_regex` in push rule evaluator

NB that this drops support for character classes. I don't think anyone ever
used them.

* Improve efficiency of globs with multiple wildcards

The idea here is that we compress multiple `*` globs into a single `.*`. We
also need to consider `?`, since `*?*` is as hard to implement efficiently as
`**`.

* add assertion on regex pattern

* Fix mypy

* Simplify glob_to_regex

* Inline the glob_to_regex helper function

Signed-off-by: Dan Callahan <danc@element.io>

* Moar comments

Signed-off-by: Dan Callahan <danc@element.io>

Co-authored-by: Dan Callahan <danc@element.io>
 2021-05-11 11:47:23 +02:00
Erik Johnston
2b2985b5cf
Improve performance of backfilling in large rooms. (#9935) 
We were pulling the full auth chain for the room out of the DB each time
we backfilled, which can be *huge* for large rooms and is totally
unnecessary.
 2021-05-10 13:29:02 +01:00
Richard van der Hoff
51065c44bb
Fix port_db on empty db (#9930) 
... and test it.
 2021-05-10 13:02:55 +01:00
Erik Johnston
6c84778549
Always cache 'event_to_prev_state_group' (#9950) 
Fixes regression in send PDU times introduced in #9905.
 2021-05-07 14:54:09 +01:00
Richard van der Hoff
765473567c
Fix make_full_schema to create the db with the right options and user (#9931) 2021-05-07 14:01:57 +01:00
Erik Johnston
b65ecaff9b Merge remote-tracking branch 'origin/release-v1.33.2' into develop 2021-05-07 13:27:19 +01:00
Erik Johnston
4df26abf28
Unpin attrs dep after new version has been released (#9946) 
c.f. #9936
 2021-05-07 12:57:21 +01:00
Richard van der Hoff
25f43faa70
Reorganise the database schema directories (#9932) 
The hope here is that by moving all the schema files into synapse/storage/schema, it gets a bit easier for newcomers to navigate.

It certainly got easier for me to write a helpful README. There's more to do on that front, but I'll follow up with other PRs for that.
 2021-05-07 10:22:05 +01:00
Erik Johnston
8771b1337d
Export jemalloc stats to prometheus when used (#9882) 2021-05-06 15:54:07 +01:00
Erik Johnston
eba431c539
Revert "Leave out optional keys from /sync (#9919)" (#9940) 
This reverts commit e9eb3549d3.
 2021-05-06 15:06:35 +01:00
Erik Johnston
a8803e2b6e Merge remote-tracking branch 'origin/master' into develop 2021-05-06 14:36:59 +01:00
Erik Johnston
ac88aca7f7 1.33.1 2021-05-06 14:06:38 +01:00
Erik Johnston
24f07a83e6
Pin attrs to <21.1.0 (#9937) 
Fixes #9936
 2021-05-06 14:06:06 +01:00
Patrick Cloke
70f0ffd2fc Follow-up to #9915 to correct the identifier for room types. 2021-05-05 16:31:16 -04:00
Patrick Cloke
d783880083
Include the time of the create event in Spaces Summary. (#9928) 
This is an update based on changes to MSC2946. The origin_server_ts
of the m.room.create event is copied into the creation_ts field for each
room returned from the spaces summary.
 2021-05-05 13:33:05 -04:00
Erik Johnston
37623e3382
Increase perf of handling presence when joining large rooms. (#9916) 2021-05-05 17:27:05 +01:00
Patrick Cloke
e2a443550e
Support stable MSC1772 spaces identifiers. (#9915) 
Support both the unstable and stable identifiers. A future release
will disable the unstable identifiers.
 2021-05-05 11:56:51 -04:00
Erik Johnston
ef889c98a6
Optionally track memory usage of each LruCache (#9881) 
This will double count slightly in the presence of interned strings. It's off by default as it can consume a lot of resources.
 2021-05-05 16:54:36 +01:00
Erik Johnston
1fb9a2d0bf
Limit how often GC happens by time. (#9902) 
Synapse can be quite memory intensive, and unless care is taken to tune
the GC thresholds it can end up thrashing, causing noticable performance
problems for large servers. We fix this by limiting how often we GC a
given generation, regardless of current counts/thresholds.

This does not help with the reverse problem where the thresholds are set
too high, but that should only happen in situations where they've been
manually configured.

Adds a `gc_min_seconds_between` config option to override the defaults.

Fixes #9890.
 2021-05-05 16:53:45 +01:00
Erik Johnston
de8f0a03a3
Don't set the external cache if its been done recently (#9905) 2021-05-05 16:53:22 +01:00
Erik Johnston
d0aee697ac
Use get_current_users_in_room from store and not StateHandler (#9910) 2021-05-05 16:49:34 +01:00
Christopher May-Townsend
d5305000f1
Docker healthcheck timings - add startup delay and changed interval (#9913) 
* Add healthcheck startup delay by 5secs and reduced interval check to 15s
to reduce waiting time for docker aware edge routers bringing an
instance online
 2021-05-05 17:33:04 +02:00
DeepBlueV7.X
e9eb3549d3
Leave out optional keys from /sync (#9919) 
This leaves out all optional keys from /sync. This should be fine for all clients tested against conduit already, but it may break some clients, as such we should check, that at least most of them don't break horribly and maybe back out some of the individual changes. (We can probably always leave out groups for example, while the others may cause more issues.)

Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
 2021-05-05 14:37:56 +01:00
Brendan Abolivier
a61b13c0a1 Merge branch 'master' into develop 2021-05-05 14:25:49 +01:00
Brendan Abolivier
0644ac0989 1.33.0 2021-05-05 14:15:54 +01:00
Erik Johnston
e3bc4617fc
Time external cache response time (#9904) 2021-05-04 15:14:22 +01:00
Andrew Morgan
b85821aca2
Add port parameter to the sample config for psycopg2 args (#9911) 
Adds the `port` option with the default value to the sample config file.
 2021-05-04 13:28:59 +01:00
Dan Callahan
56c4b47df3
Build Debian packages for Ubuntu 21.04 Hirsute (#9909) 
Signed-off-by: Dan Callahan <danc@element.io>
 2021-04-30 15:36:05 +01:00
Andrew Morgan
4d624f467a Synapse 1.33.0rc2 (2021-04-29) 
==============================
 
 Bugfixes
 --------
 
 - Fix tight loop when handling presence replication when using workers. Introduced in v1.33.0rc1. ([\#9900](https://github.com/matrix-org/synapse/issues/9900))
 -----BEGIN PGP SIGNATURE-----
 
 iQJHBAABCgAxFiEEgQG31Z317NrSMt0QiISIDS7+X/QFAmCKtVATHGFuZHJld0Bh
 bW9yZ2FuLnh5egAKCRCIhIgNLv5f9Ok1D/40l93bxGBFtSNEHlqem7lZRLY2On9j
 AUnzIMqtJTLUgXf67vKIYqtTWkRjTdLIhIuINwhRmB0K23vcRrONnnyRLdhUC4Hl
 v6qLlPBHzupAJQPRcy3FGIVwsloAZ3JqVhV8AG0gXBRA0Dc34qYXh1DlA8EQ8mW/
 l2umQIJI562xLNss7c5tpbCQxi7mq6OmTZ+Nwh45VRDavKEQQzSb9sPfs5w8IGLA
 ViYosUoXoXEWus/j60E38LSA/v1YcEE+Uyf5+LieV4b2c305xKj8gIbZqjLHR2K5
 U9IrUp4QlYpPiebfGn7EmiPnOnlWdR9soIVZMzJ/m29tlyRSL5MBagtWA2fFlXo6
 PywW1R/VsnLTdOuWY/t+et11iI4lVa7VSpywEcbMKGgmGKexDnfB10K7jHw0YICq
 0pyLgy8JWyGUk/xpH2W1pvak8vyyN/zI/H09bDgGu721Qr/EPiHWF6RADNASpUVW
 w10SnO1udVc8ajugNwpX2oOQA7i5uiEuTdyFUHYfZ/aSpaCvdxM+rL3v0K955bNc
 /AGjSbs9ITe+JJQDiAJsWCYc+XLkR4h5KHVaZm2XlusbYb4iobK1DflkWXnQFjsi
 rlTcUH3sIFot3i9mjBBvXdvmzuI9qyx81/Sh8gRunPYT5TPhwaXKfyMBKE93/kif
 4bKoHIXVcJvAqA==
 =sboq
 -----END PGP SIGNATURE-----

Merge tag 'v1.33.0rc2' into develop

Synapse 1.33.0rc2 (2021-04-29)
==============================

Bugfixes
--------

- Fix tight loop when handling presence replication when using workers. Introduced in v1.33.0rc1. ([\#9900](https://github.com/matrix-org/synapse/issues/9900))
 2021-04-29 14:35:14 +01:00
Andrew Morgan
d11f2dfee5 typo in changelog 2021-04-29 14:31:14 +01:00
Patrick Cloke
bb4b11846f
Add missing type hints to handlers and fix a Spam Checker type hint. (#9896) 
The user_may_create_room_alias method on spam checkers
declared the room_alias parameter as a str when in reality it is
passed a RoomAlias object.
 2021-04-29 07:17:28 -04:00
Andrew Morgan
e9444cc74d 1.33.0rc2 2021-04-29 11:45:37 +01:00
ThibF
0085dc5abc
Delete room endpoint (#9889) 
Support the delete of a room through DELETE request and mark
previous request as deprecated through documentation.

Signed-off-by: Thibault Ferrante <thibault.ferrante@pm.me>
 2021-04-29 10:31:45 +01:00
Erik Johnston
802560211a Merge remote-tracking branch 'origin/release-v1.33.0' into develop 2021-04-28 14:43:10 +01:00
Erik Johnston
e4ab8676b4
Fix tight loop handling presence replication. (#9900) 
Only affects workers. Introduced in #9819.

Fixes #9899.
 2021-04-28 14:42:50 +01:00
Patrick Cloke
10a08ab88a
Use the parent's logging context name for runWithConnection. (#9895) 
This fixes a regression where the logging context for runWithConnection
was reported as runWithConnection instead of the connection name,
e.g. "POST-XYZ".
 2021-04-28 07:44:52 -04:00
Andrew Morgan
fa6679e794 Synapse 1.33.0rc1 (2021-04-28) 
==============================
 
 Features
 --------
 
 - Update experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership. ([\#9800](https://github.com/matrix-org/synapse/issues/9800), [\#9814](https://github.com/matrix-org/synapse/issues/9814))
 - Add experimental support for handling presence on a worker. ([\#9819](https://github.com/matrix-org/synapse/issues/9819), [\#9820](https://github.com/matrix-org/synapse/issues/9820), [\#9828](https://github.com/matrix-org/synapse/issues/9828), [\#9850](https://github.com/matrix-org/synapse/issues/9850))
 - Return a new template when an user attempts to renew their account multiple times with the same token, stating that their account is set to expire. This replaces the invalid token template that would previously be shown in this case. This change concerns the optional account validity feature. ([\#9832](https://github.com/matrix-org/synapse/issues/9832))
 
 Bugfixes
 --------
 
 - Fixes the OIDC SSO flow when using a `public_baseurl` value including a non-root URL path. ([\#9726](https://github.com/matrix-org/synapse/issues/9726))
 - Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg. ([\#9788](https://github.com/matrix-org/synapse/issues/9788))
 - Add some sanity checks to identity server passed to 3PID bind/unbind endpoints. ([\#9802](https://github.com/matrix-org/synapse/issues/9802))
 - Limit the size of HTTP responses read over federation. ([\#9833](https://github.com/matrix-org/synapse/issues/9833))
 - Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists. ([\#9867](https://github.com/matrix-org/synapse/issues/9867))
 - Fix a long-standing bug where errors from federation did not propagate to the client. ([\#9868](https://github.com/matrix-org/synapse/issues/9868))
 
 Improved Documentation
 ----------------------
 
 - Add a note to the docker docs mentioning that we mirror upstream's supported Docker platforms. ([\#9801](https://github.com/matrix-org/synapse/issues/9801))
 
 Internal Changes
 ----------------
 
 - Add a dockerfile for running Synapse in worker-mode under Complement. ([\#9162](https://github.com/matrix-org/synapse/issues/9162))
 - Apply `pyupgrade` across the codebase. ([\#9786](https://github.com/matrix-org/synapse/issues/9786))
 - Move some replication processing out of `generic_worker`. ([\#9796](https://github.com/matrix-org/synapse/issues/9796))
 - Replace `HomeServer.get_config()` with inline references. ([\#9815](https://github.com/matrix-org/synapse/issues/9815))
 - Rename some handlers and config modules to not duplicate the top-level module. ([\#9816](https://github.com/matrix-org/synapse/issues/9816))
 - Fix a long-standing bug which caused `max_upload_size` to not be correctly enforced. ([\#9817](https://github.com/matrix-org/synapse/issues/9817))
 - Reduce CPU usage of the user directory by reusing existing calculated room membership. ([\#9821](https://github.com/matrix-org/synapse/issues/9821))
 - Small speed up for joining large remote rooms. ([\#9825](https://github.com/matrix-org/synapse/issues/9825))
 - Introduce flake8-bugbear to the test suite and fix some of its lint violations. ([\#9838](https://github.com/matrix-org/synapse/issues/9838))
 - Only store the raw data in the in-memory caches, rather than objects that include references to e.g. the data stores. ([\#9845](https://github.com/matrix-org/synapse/issues/9845))
 - Limit length of accepted email addresses. ([\#9855](https://github.com/matrix-org/synapse/issues/9855))
 - Remove redundant `synapse.types.Collection` type definition. ([\#9856](https://github.com/matrix-org/synapse/issues/9856))
 - Handle recently added rate limits correctly when using `--no-rate-limit` with the demo scripts. ([\#9858](https://github.com/matrix-org/synapse/issues/9858))
 - Disable invite rate-limiting by default when running the unit tests. ([\#9871](https://github.com/matrix-org/synapse/issues/9871))
 - Pass a reactor into `SynapseSite` to make testing easier. ([\#9874](https://github.com/matrix-org/synapse/issues/9874))
 - Make `DomainSpecificString` an `attrs` class. ([\#9875](https://github.com/matrix-org/synapse/issues/9875))
 - Add type hints to `synapse.api.auth` and `synapse.api.auth_blocking` modules. ([\#9876](https://github.com/matrix-org/synapse/issues/9876))
 - Remove redundant `_PushHTTPChannel` test class. ([\#9878](https://github.com/matrix-org/synapse/issues/9878))
 - Remove backwards-compatibility code for Python versions < 3.6. ([\#9879](https://github.com/matrix-org/synapse/issues/9879))
 - Small performance improvement around handling new local presence updates. ([\#9887](https://github.com/matrix-org/synapse/issues/9887))
 -----BEGIN PGP SIGNATURE-----
 
 iQJHBAABCgAxFiEEgQG31Z317NrSMt0QiISIDS7+X/QFAmCJQmgTHGFuZHJld0Bh
 bW9yZ2FuLnh5egAKCRCIhIgNLv5f9EKWD/9MZG4ngjECrk3QMBlYqaipGl/l7wcl
 8vBOTxiEqIgVJxHLCRFBbtFOItdxB4YkrorIo2fqshkk+lv4CRtTD9jEKHRvS4T6
 7p1icRdRTv2K94OkJ8R9jMFlmywZFU87oHPfI2xUYg6hvOKrR+RwPvIjA7c24UZt
 6MJqDhgGDlZD7/hQdQof9O4oOJzzIgLJPk6o8E42y6c0bLlPPKgH3sh0vlenDLfE
 15thRCOeiP237YYvXSdbr7G3PI66Efhq4BwQowSrgFg+B0BR68l4747iSIeWmLQJ
 Ow6QLLFPCOhgAPC4amp9PdaaV/9NYiBcvNxlOvyQAVl/+ioEiATNHnzdNtDouCQo
 nDdSHw0Mt9D3i+rxpu5Pf0gZN9dXRGiczqnq5QKL8+EvT/4FLYeluEeduuy3rE+G
 o5OTCd3EajxynzjftuopeysNAw6zeDpbulZoTCeCumxBL2+wAod2PsyY5Ei9Gapn
 iJvExNOJX4OlkFc67jO2CK8o3sUTNEDeIDWCQ0fVKAwIt7T45ebTA/UzDNWg7YzN
 EyUp+3NZcUZBskgMB5hpcijPJoXNYzWZPews73vMPV7AfQRxDzU5xrM9AdbadBDd
 Idv6wT9ssDUA+M0aKnafvoSJZ+qE85mi2x0rRsueZNd9uO9/QHIXKIb+4PvVpj3C
 BJnZf34m568AGQ==
 =smAk
 -----END PGP SIGNATURE-----

Merge tag 'v1.33.0rc1' into develop

Synapse 1.33.0rc1 (2021-04-28)
==============================

Features
--------

- Update experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership. ([\#9800](https://github.com/matrix-org/synapse/issues/9800), [\#9814](https://github.com/matrix-org/synapse/issues/9814))
- Add experimental support for handling presence on a worker. ([\#9819](https://github.com/matrix-org/synapse/issues/9819), [\#9820](https://github.com/matrix-org/synapse/issues/9820), [\#9828](https://github.com/matrix-org/synapse/issues/9828), [\#9850](https://github.com/matrix-org/synapse/issues/9850))
- Return a new template when an user attempts to renew their account multiple times with the same token, stating that their account is set to expire. This replaces the invalid token template that would previously be shown in this case. This change concerns the optional account validity feature. ([\#9832](https://github.com/matrix-org/synapse/issues/9832))

Bugfixes
--------

- Fixes the OIDC SSO flow when using a `public_baseurl` value including a non-root URL path. ([\#9726](https://github.com/matrix-org/synapse/issues/9726))
- Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg. ([\#9788](https://github.com/matrix-org/synapse/issues/9788))
- Add some sanity checks to identity server passed to 3PID bind/unbind endpoints. ([\#9802](https://github.com/matrix-org/synapse/issues/9802))
- Limit the size of HTTP responses read over federation. ([\#9833](https://github.com/matrix-org/synapse/issues/9833))
- Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists. ([\#9867](https://github.com/matrix-org/synapse/issues/9867))
- Fix a long-standing bug where errors from federation did not propagate to the client. ([\#9868](https://github.com/matrix-org/synapse/issues/9868))

Improved Documentation
----------------------

- Add a note to the docker docs mentioning that we mirror upstream's supported Docker platforms. ([\#9801](https://github.com/matrix-org/synapse/issues/9801))

Internal Changes
----------------

- Add a dockerfile for running Synapse in worker-mode under Complement. ([\#9162](https://github.com/matrix-org/synapse/issues/9162))
- Apply `pyupgrade` across the codebase. ([\#9786](https://github.com/matrix-org/synapse/issues/9786))
- Move some replication processing out of `generic_worker`. ([\#9796](https://github.com/matrix-org/synapse/issues/9796))
- Replace `HomeServer.get_config()` with inline references. ([\#9815](https://github.com/matrix-org/synapse/issues/9815))
- Rename some handlers and config modules to not duplicate the top-level module. ([\#9816](https://github.com/matrix-org/synapse/issues/9816))
- Fix a long-standing bug which caused `max_upload_size` to not be correctly enforced. ([\#9817](https://github.com/matrix-org/synapse/issues/9817))
- Reduce CPU usage of the user directory by reusing existing calculated room membership. ([\#9821](https://github.com/matrix-org/synapse/issues/9821))
- Small speed up for joining large remote rooms. ([\#9825](https://github.com/matrix-org/synapse/issues/9825))
- Introduce flake8-bugbear to the test suite and fix some of its lint violations. ([\#9838](https://github.com/matrix-org/synapse/issues/9838))
- Only store the raw data in the in-memory caches, rather than objects that include references to e.g. the data stores. ([\#9845](https://github.com/matrix-org/synapse/issues/9845))
- Limit length of accepted email addresses. ([\#9855](https://github.com/matrix-org/synapse/issues/9855))
- Remove redundant `synapse.types.Collection` type definition. ([\#9856](https://github.com/matrix-org/synapse/issues/9856))
- Handle recently added rate limits correctly when using `--no-rate-limit` with the demo scripts. ([\#9858](https://github.com/matrix-org/synapse/issues/9858))
- Disable invite rate-limiting by default when running the unit tests. ([\#9871](https://github.com/matrix-org/synapse/issues/9871))
- Pass a reactor into `SynapseSite` to make testing easier. ([\#9874](https://github.com/matrix-org/synapse/issues/9874))
- Make `DomainSpecificString` an `attrs` class. ([\#9875](https://github.com/matrix-org/synapse/issues/9875))
- Add type hints to `synapse.api.auth` and `synapse.api.auth_blocking` modules. ([\#9876](https://github.com/matrix-org/synapse/issues/9876))
- Remove redundant `_PushHTTPChannel` test class. ([\#9878](https://github.com/matrix-org/synapse/issues/9878))
- Remove backwards-compatibility code for Python versions < 3.6. ([\#9879](https://github.com/matrix-org/synapse/issues/9879))
- Small performance improvement around handling new local presence updates. ([\#9887](https://github.com/matrix-org/synapse/issues/9887))
 2021-04-28 12:12:29 +01:00
Andrew Morgan
8ba086980d Reword account validity template change to sound less like a bugfix 2021-04-28 12:07:49 +01:00
Erik Johnston
391bfe9a7b
Reduce memory footprint of caches (#9886) 2021-04-28 11:59:28 +01:00
Andrew Morgan
787de3190f 1.33.0rc1 2021-04-28 11:43:33 +01:00
Andrew Morgan
4e0fd35bc9 Revert "Experimental Federation Speedup (#9702)" 
This reverts commit 05e8c70c05.
 2021-04-28 11:38:33 +01:00
Erik Johnston
dd2d32dcdb
Add type hints to presence handler (#9885) 2021-04-28 11:07:47 +01:00
Andrew Morgan
fe604a022a
Remove various bits of compatibility code for Python <3.6 (#9879) 
I went through and removed a bunch of cruft that was lying around for compatibility with old Python versions. This PR also will now prevent Synapse from starting unless you're running Python 3.6+.
 2021-04-27 13:13:07 +01:00
Patrick Cloke
1350b053da
Pass errors back to the client when trying multiple federation destinations. (#9868) 
This ensures that something like an auth error (403) will be
returned to the requester instead of attempting to try more
servers, which will likely result in the same error, and then
passing back a generic 400 error.
 2021-04-27 07:30:34 -04:00
Erik Johnston
0ffa5fb935
Use current state table for presence.get_interested_remotes (#9887) 
This should be a lot quicker than asking the state handler.
 2021-04-27 10:09:41 +01:00