maxmustermann/synapse
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
18527 commits 636 branches 519 tags 164 MiB
Commit graph

18527 commits

This branch
This branch
All branches
Author SHA1 Message Date
Erik Johnston 82a91208d6 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2021-01-07 13:04:45 +00:00
Patrick Cloke 23d701864f
Improve the performance of calculating ignored users in large rooms (#9024) 
This allows for efficiently finding which users ignore a particular
user.

Co-authored-by: Erik Johnston <erik@matrix.org>
 2021-01-07 13:03:38 +00:00
Richard van der Hoff 1d5c021a45
tox: Add a -noextras factor (#9030) 
... for running the tests with no optional deps.
 2021-01-07 11:41:54 +00:00
Richard van der Hoff 8d3d264052
Skip unit tests which require optional dependencies (#9031) 
If we are lacking an optional dependency, skip the tests that rely on it.
 2021-01-07 11:41:28 +00:00
Erik Johnston eee3c3c52f
Handle updating schema version without any deltas. (#9033) 
This can happen when using a split out state database and we've upgraded
the schema version without there being any changes in the state schema.
 2021-01-07 11:33:36 +00:00
Erik Johnston 91fd180be1 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2021-01-07 10:35:04 +00:00
Patrick Cloke 1b4d5d6acf
Empty iterables should count towards cache usage. (#9028) 2021-01-06 12:33:20 -05:00
Patrick Cloke 0312266ee3 Synapse 1.25.0rc1 (2021-01-06) 
==============================
 
 Removal warning
 ---------------
 
 The old [Purge Room API](https://github.com/matrix-org/synapse/tree/master/docs/admin_api/purge_room.md)
 and [Shutdown Room API](https://github.com/matrix-org/synapse/tree/master/docs/admin_api/shutdown_room.md)
 are deprecated and will be removed in a future release. They will be replaced by the
 [Delete Room API](https://github.com/matrix-org/synapse/tree/master/docs/admin_api/rooms.md#delete-room-api).
 
 `POST /_synapse/admin/v1/rooms/<room_id>/delete` replaces `POST /_synapse/admin/v1/purge_room` and
 `POST /_synapse/admin/v1/shutdown_room/<room_id>`.
 
 Features
 --------
 
 - Add an admin API that lets server admins get power in rooms in which local users have power. ([\#8756](https://github.com/matrix-org/synapse/issues/8756))
 - Add optional HTTP authentication to replication endpoints. ([\#8853](https://github.com/matrix-org/synapse/issues/8853))
 - Improve the error messages printed as a result of configuration problems for extension modules. ([\#8874](https://github.com/matrix-org/synapse/issues/8874))
 - Add the number of local devices to Room Details Admin API. Contributed by @dklimpel. ([\#8886](https://github.com/matrix-org/synapse/issues/8886))
 - Add `X-Robots-Tag` header to stop web crawlers from indexing media. Contributed by Aaron Raimist. ([\#8887](https://github.com/matrix-org/synapse/issues/8887))
 - Spam-checkers may now define their methods as `async`. ([\#8890](https://github.com/matrix-org/synapse/issues/8890))
 - Add support for allowing users to pick their own user ID during a single-sign-on login. ([\#8897](https://github.com/matrix-org/synapse/issues/8897), [\#8900](https://github.com/matrix-org/synapse/issues/8900), [\#8911](https://github.com/matrix-org/synapse/issues/8911), [\#8938](https://github.com/matrix-org/synapse/issues/8938), [\#8941](https://github.com/matrix-org/synapse/issues/8941), [\#8942](https://github.com/matrix-org/synapse/issues/8942), [\#8951](https://github.com/matrix-org/synapse/issues/8951))
 - Add an `email.invite_client_location` configuration option to send a web client location to the invite endpoint on the identity server which allows customisation of the email template. ([\#8930](https://github.com/matrix-org/synapse/issues/8930))
 - The search term in the list room and list user Admin APIs is now treated as case-insensitive. ([\#8931](https://github.com/matrix-org/synapse/issues/8931))
 - Apply an IP range blacklist to push and key revocation requests. ([\#8821](https://github.com/matrix-org/synapse/issues/8821), [\#8870](https://github.com/matrix-org/synapse/issues/8870), [\#8954](https://github.com/matrix-org/synapse/issues/8954))
 - Add an option to allow re-use of user-interactive authentication sessions for a period of time. ([\#8970](https://github.com/matrix-org/synapse/issues/8970))
 - Allow running the redact endpoint on workers. ([\#8994](https://github.com/matrix-org/synapse/issues/8994))
 
 Bugfixes
 --------
 
 - Fix bug where we might not correctly calculate the current state for rooms with multiple extremities. ([\#8827](https://github.com/matrix-org/synapse/issues/8827))
 - Fix a long-standing bug in the register admin endpoint (`/_synapse/admin/v1/register`) when the `mac` field was not provided. The endpoint now properly returns a 400 error. Contributed by @edwargix. ([\#8837](https://github.com/matrix-org/synapse/issues/8837))
 - Fix a long-standing bug on Synapse instances supporting Single-Sign-On, where users would be prompted to enter their password to confirm certain actions, even though they have not set a password. ([\#8858](https://github.com/matrix-org/synapse/issues/8858))
 - Fix a longstanding bug where a 500 error would be returned if the `Content-Length` header was not provided to the upload media resource. ([\#8862](https://github.com/matrix-org/synapse/issues/8862))
 - Add additional validation to pusher URLs to be compliant with the specification. ([\#8865](https://github.com/matrix-org/synapse/issues/8865))
 - Fix the error code that is returned when a user tries to register on a homeserver on which new-user registration has been disabled. ([\#8867](https://github.com/matrix-org/synapse/issues/8867))
 - Fix a bug where `PUT /_synapse/admin/v2/users/<user_id>` failed to create a new user when `avatar_url` is specified. Bug introduced in Synapse v1.9.0. ([\#8872](https://github.com/matrix-org/synapse/issues/8872))
 - Fix a 500 error when attempting to preview an empty HTML file. ([\#8883](https://github.com/matrix-org/synapse/issues/8883))
 - Fix occasional deadlock when handling SIGHUP. ([\#8918](https://github.com/matrix-org/synapse/issues/8918))
 - Fix login API to not ratelimit application services that have ratelimiting disabled. ([\#8920](https://github.com/matrix-org/synapse/issues/8920))
 - Fix bug where we ratelimited auto joining of rooms on registration (using `auto_join_rooms` config). ([\#8921](https://github.com/matrix-org/synapse/issues/8921))
 - Fix a bug where deactivated users appeared in the user directory when their profile information was updated. ([\#8933](https://github.com/matrix-org/synapse/issues/8933), [\#8964](https://github.com/matrix-org/synapse/issues/8964))
 - Fix bug introduced in Synapse v1.24.0 which would cause an exception on startup if both `enabled` and `localdb_enabled` were set to `False` in the `password_config` setting of the configuration file. ([\#8937](https://github.com/matrix-org/synapse/issues/8937))
 - Fix a bug where 500 errors would be returned if the `m.room_history_visibility` event had invalid content. ([\#8945](https://github.com/matrix-org/synapse/issues/8945))
 - Fix a bug causing common English words to not be considered for a user directory search. ([\#8959](https://github.com/matrix-org/synapse/issues/8959))
 - Fix bug where application services couldn't register new ghost users if the server had reached its MAU limit. ([\#8962](https://github.com/matrix-org/synapse/issues/8962))
 - Fix a long-standing bug where a `m.image` event without a `url` would cause errors on push. ([\#8965](https://github.com/matrix-org/synapse/issues/8965))
 - Fix a small bug in v2 state resolution algorithm, which could also cause performance issues for rooms with large numbers of power levels. ([\#8971](https://github.com/matrix-org/synapse/issues/8971))
 - Add validation to the `sendToDevice` API to raise a missing parameters error instead of a 500 error. ([\#8975](https://github.com/matrix-org/synapse/issues/8975))
 - Add validation of group IDs to raise a 400 error instead of a 500 eror. ([\#8977](https://github.com/matrix-org/synapse/issues/8977))
 
 Improved Documentation
 ----------------------
 
 - Fix the "Event persist rate" section of the included grafana dashboard by adding missing prometheus rules. ([\#8802](https://github.com/matrix-org/synapse/issues/8802))
 - Combine related media admin API docs. ([\#8839](https://github.com/matrix-org/synapse/issues/8839))
 - Fix an error in the documentation for the SAML username mapping provider. ([\#8873](https://github.com/matrix-org/synapse/issues/8873))
 - Clarify comments around template directories in `sample_config.yaml`. ([\#8891](https://github.com/matrix-org/synapse/issues/8891))
 - Moved instructions for database setup, adjusted heading levels and improved syntax highlighting in [INSTALL.md](../INSTALL.md). Contributed by fossterer. ([\#8987](https://github.com/matrix-org/synapse/issues/8987))
 - Update the example value of `group_creation_prefix` in the sample configuration. ([\#8992](https://github.com/matrix-org/synapse/issues/8992))
 - Link the Synapse developer room to the development section in the docs. ([\#9002](https://github.com/matrix-org/synapse/issues/9002))
 
 Deprecations and Removals
 -------------------------
 
 - Deprecate Shutdown Room and Purge Room Admin APIs. ([\#8829](https://github.com/matrix-org/synapse/issues/8829))
 
 Internal Changes
 ----------------
 
 - Properly store the mapping of external ID to Matrix ID for CAS users. ([\#8856](https://github.com/matrix-org/synapse/issues/8856), [\#8958](https://github.com/matrix-org/synapse/issues/8958))
 - Remove some unnecessary stubbing from unit tests. ([\#8861](https://github.com/matrix-org/synapse/issues/8861))
 - Remove unused `FakeResponse` class from unit tests. ([\#8864](https://github.com/matrix-org/synapse/issues/8864))
 - Pass `room_id` to `get_auth_chain_difference`. ([\#8879](https://github.com/matrix-org/synapse/issues/8879))
 - Add type hints to push module. ([\#8880](https://github.com/matrix-org/synapse/issues/8880), [\#8882](https://github.com/matrix-org/synapse/issues/8882), [\#8901](https://github.com/matrix-org/synapse/issues/8901), [\#8940](https://github.com/matrix-org/synapse/issues/8940), [\#8943](https://github.com/matrix-org/synapse/issues/8943), [\#9020](https://github.com/matrix-org/synapse/issues/9020))
 - Simplify logic for handling user-interactive-auth via single-sign-on servers. ([\#8881](https://github.com/matrix-org/synapse/issues/8881))
 - Skip the SAML tests if the requirements (`pysaml2` and `xmlsec1`) aren't available. ([\#8905](https://github.com/matrix-org/synapse/issues/8905))
 - Fix multiarch docker image builds. ([\#8906](https://github.com/matrix-org/synapse/issues/8906))
 - Don't publish `latest` docker image until all archs are built. ([\#8909](https://github.com/matrix-org/synapse/issues/8909))
 - Various clean-ups to the structured logging and logging context code. ([\#8916](https://github.com/matrix-org/synapse/issues/8916), [\#8935](https://github.com/matrix-org/synapse/issues/8935))
 - Automatically drop stale forward-extremities under some specific conditions. ([\#8929](https://github.com/matrix-org/synapse/issues/8929))
 - Refactor test utilities for injecting HTTP requests. ([\#8946](https://github.com/matrix-org/synapse/issues/8946))
 - Add a maximum size of 50 kilobytes to .well-known lookups. ([\#8950](https://github.com/matrix-org/synapse/issues/8950))
 - Fix bug in `generate_log_config` script which made it write empty files. ([\#8952](https://github.com/matrix-org/synapse/issues/8952))
 - Clean up tox.ini file; disable coverage checking for non-test runs. ([\#8963](https://github.com/matrix-org/synapse/issues/8963))
 - Add type hints to the admin and room list handlers. ([\#8973](https://github.com/matrix-org/synapse/issues/8973))
 - Add type hints to the receipts and user directory handlers. ([\#8976](https://github.com/matrix-org/synapse/issues/8976))
 - Drop the unused `local_invites` table. ([\#8979](https://github.com/matrix-org/synapse/issues/8979))
 - Add type hints to the base storage code. ([\#8980](https://github.com/matrix-org/synapse/issues/8980))
 - Support using PyJWT v2.0.0 in the test suite. ([\#8986](https://github.com/matrix-org/synapse/issues/8986))
 - Fix `tests.federation.transport.RoomDirectoryFederationTests` and ensure it runs in CI. ([\#8998](https://github.com/matrix-org/synapse/issues/8998))
 - Add type hints to the crypto module. ([\#8999](https://github.com/matrix-org/synapse/issues/8999))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl/1rv4ACgkQM/xY9qcR
 MEiIGw/+PytgC2yOpaR7cIQl0K1v+y4X/AmNbshdQch7GtC3vtKxdmo4PJp4KwY4
 Sxiiykq46Xx48w/Hqd3hoC7OqsTyfmNjRKW0InsTarbaXMrSv9umOtScItzMMWq8
 SCmDPQ+cF0SP2GhOMwB/N5JpRS12OpdBzHXtcbwxULilTuO/EKD2/W1zq3wiSP7r
 BxhHHYsx69hufUGC3OGD0I8e26TqjgkLiVm0FeM6cC5TubozO3qrJDlmwTPSpFrL
 WZEBEbnYUVz4ydMKRq4ALiZzPzxxBfgQmITb79CNqim4+fc+M+AqV6mRez2ULrqh
 d4/UEMTaA0yh54i0oqei3gwf2ucns/NDftQ9cYwF1+a9LCcWAyJZY7+/9Hk4VKSU
 KqQUzGdBfxyefsLc2hNGANzS/VbThdejC09IjaxKtHSgaT+bb5LxS8GcbvcG5FxY
 LyD8yeazgXPub+Ixr6dvD51roCuBtpCYvGZtHHYOiX/S1MYbq22Duet2Z1zhy7qB
 Jl8r3L32x5wAG3lR8aLqDVNTrC1ur0TW9wUJ88bDjcwn0PsseQG41DHbQSoXQIlk
 p6/IKjJjfnbKJFTwzfC8RpL29NtyTPqA/d2JDJ7yqUbpEkAnc5GN+pO0B/736aOH
 aTRu8nVdP7GOoP4whZ9C7r5kYjiCR3jPtpZHRYXaPJtkw57dWS8=
 =/g7u
 -----END PGP SIGNATURE-----

Merge tag 'v1.25.0rc1' into develop

Synapse 1.25.0rc1 (2021-01-06)
==============================

Removal warning
---------------

The old [Purge Room API](https://github.com/matrix-org/synapse/tree/master/docs/admin_api/purge_room.md)
and [Shutdown Room API](https://github.com/matrix-org/synapse/tree/master/docs/admin_api/shutdown_room.md)
are deprecated and will be removed in a future release. They will be replaced by the
[Delete Room API](https://github.com/matrix-org/synapse/tree/master/docs/admin_api/rooms.md#delete-room-api).

`POST /_synapse/admin/v1/rooms/<room_id>/delete` replaces `POST /_synapse/admin/v1/purge_room` and
`POST /_synapse/admin/v1/shutdown_room/<room_id>`.

Features
--------

- Add an admin API that lets server admins get power in rooms in which local users have power. ([\#8756](https://github.com/matrix-org/synapse/issues/8756))
- Add optional HTTP authentication to replication endpoints. ([\#8853](https://github.com/matrix-org/synapse/issues/8853))
- Improve the error messages printed as a result of configuration problems for extension modules. ([\#8874](https://github.com/matrix-org/synapse/issues/8874))
- Add the number of local devices to Room Details Admin API. Contributed by @dklimpel. ([\#8886](https://github.com/matrix-org/synapse/issues/8886))
- Add `X-Robots-Tag` header to stop web crawlers from indexing media. Contributed by Aaron Raimist. ([\#8887](https://github.com/matrix-org/synapse/issues/8887))
- Spam-checkers may now define their methods as `async`. ([\#8890](https://github.com/matrix-org/synapse/issues/8890))
- Add support for allowing users to pick their own user ID during a single-sign-on login. ([\#8897](https://github.com/matrix-org/synapse/issues/8897), [\#8900](https://github.com/matrix-org/synapse/issues/8900), [\#8911](https://github.com/matrix-org/synapse/issues/8911), [\#8938](https://github.com/matrix-org/synapse/issues/8938), [\#8941](https://github.com/matrix-org/synapse/issues/8941), [\#8942](https://github.com/matrix-org/synapse/issues/8942), [\#8951](https://github.com/matrix-org/synapse/issues/8951))
- Add an `email.invite_client_location` configuration option to send a web client location to the invite endpoint on the identity server which allows customisation of the email template. ([\#8930](https://github.com/matrix-org/synapse/issues/8930))
- The search term in the list room and list user Admin APIs is now treated as case-insensitive. ([\#8931](https://github.com/matrix-org/synapse/issues/8931))
- Apply an IP range blacklist to push and key revocation requests. ([\#8821](https://github.com/matrix-org/synapse/issues/8821), [\#8870](https://github.com/matrix-org/synapse/issues/8870), [\#8954](https://github.com/matrix-org/synapse/issues/8954))
- Add an option to allow re-use of user-interactive authentication sessions for a period of time. ([\#8970](https://github.com/matrix-org/synapse/issues/8970))
- Allow running the redact endpoint on workers. ([\#8994](https://github.com/matrix-org/synapse/issues/8994))

Bugfixes
--------

- Fix bug where we might not correctly calculate the current state for rooms with multiple extremities. ([\#8827](https://github.com/matrix-org/synapse/issues/8827))
- Fix a long-standing bug in the register admin endpoint (`/_synapse/admin/v1/register`) when the `mac` field was not provided. The endpoint now properly returns a 400 error. Contributed by @edwargix. ([\#8837](https://github.com/matrix-org/synapse/issues/8837))
- Fix a long-standing bug on Synapse instances supporting Single-Sign-On, where users would be prompted to enter their password to confirm certain actions, even though they have not set a password. ([\#8858](https://github.com/matrix-org/synapse/issues/8858))
- Fix a longstanding bug where a 500 error would be returned if the `Content-Length` header was not provided to the upload media resource. ([\#8862](https://github.com/matrix-org/synapse/issues/8862))
- Add additional validation to pusher URLs to be compliant with the specification. ([\#8865](https://github.com/matrix-org/synapse/issues/8865))
- Fix the error code that is returned when a user tries to register on a homeserver on which new-user registration has been disabled. ([\#8867](https://github.com/matrix-org/synapse/issues/8867))
- Fix a bug where `PUT /_synapse/admin/v2/users/<user_id>` failed to create a new user when `avatar_url` is specified. Bug introduced in Synapse v1.9.0. ([\#8872](https://github.com/matrix-org/synapse/issues/8872))
- Fix a 500 error when attempting to preview an empty HTML file. ([\#8883](https://github.com/matrix-org/synapse/issues/8883))
- Fix occasional deadlock when handling SIGHUP. ([\#8918](https://github.com/matrix-org/synapse/issues/8918))
- Fix login API to not ratelimit application services that have ratelimiting disabled. ([\#8920](https://github.com/matrix-org/synapse/issues/8920))
- Fix bug where we ratelimited auto joining of rooms on registration (using `auto_join_rooms` config). ([\#8921](https://github.com/matrix-org/synapse/issues/8921))
- Fix a bug where deactivated users appeared in the user directory when their profile information was updated. ([\#8933](https://github.com/matrix-org/synapse/issues/8933), [\#8964](https://github.com/matrix-org/synapse/issues/8964))
- Fix bug introduced in Synapse v1.24.0 which would cause an exception on startup if both `enabled` and `localdb_enabled` were set to `False` in the `password_config` setting of the configuration file. ([\#8937](https://github.com/matrix-org/synapse/issues/8937))
- Fix a bug where 500 errors would be returned if the `m.room_history_visibility` event had invalid content. ([\#8945](https://github.com/matrix-org/synapse/issues/8945))
- Fix a bug causing common English words to not be considered for a user directory search. ([\#8959](https://github.com/matrix-org/synapse/issues/8959))
- Fix bug where application services couldn't register new ghost users if the server had reached its MAU limit. ([\#8962](https://github.com/matrix-org/synapse/issues/8962))
- Fix a long-standing bug where a `m.image` event without a `url` would cause errors on push. ([\#8965](https://github.com/matrix-org/synapse/issues/8965))
- Fix a small bug in v2 state resolution algorithm, which could also cause performance issues for rooms with large numbers of power levels. ([\#8971](https://github.com/matrix-org/synapse/issues/8971))
- Add validation to the `sendToDevice` API to raise a missing parameters error instead of a 500 error. ([\#8975](https://github.com/matrix-org/synapse/issues/8975))
- Add validation of group IDs to raise a 400 error instead of a 500 eror. ([\#8977](https://github.com/matrix-org/synapse/issues/8977))

Improved Documentation
----------------------

- Fix the "Event persist rate" section of the included grafana dashboard by adding missing prometheus rules. ([\#8802](https://github.com/matrix-org/synapse/issues/8802))
- Combine related media admin API docs. ([\#8839](https://github.com/matrix-org/synapse/issues/8839))
- Fix an error in the documentation for the SAML username mapping provider. ([\#8873](https://github.com/matrix-org/synapse/issues/8873))
- Clarify comments around template directories in `sample_config.yaml`. ([\#8891](https://github.com/matrix-org/synapse/issues/8891))
- Moved instructions for database setup, adjusted heading levels and improved syntax highlighting in [INSTALL.md](../INSTALL.md). Contributed by fossterer. ([\#8987](https://github.com/matrix-org/synapse/issues/8987))
- Update the example value of `group_creation_prefix` in the sample configuration. ([\#8992](https://github.com/matrix-org/synapse/issues/8992))
- Link the Synapse developer room to the development section in the docs. ([\#9002](https://github.com/matrix-org/synapse/issues/9002))

Deprecations and Removals
-------------------------

- Deprecate Shutdown Room and Purge Room Admin APIs. ([\#8829](https://github.com/matrix-org/synapse/issues/8829))

Internal Changes
----------------

- Properly store the mapping of external ID to Matrix ID for CAS users. ([\#8856](https://github.com/matrix-org/synapse/issues/8856), [\#8958](https://github.com/matrix-org/synapse/issues/8958))
- Remove some unnecessary stubbing from unit tests. ([\#8861](https://github.com/matrix-org/synapse/issues/8861))
- Remove unused `FakeResponse` class from unit tests. ([\#8864](https://github.com/matrix-org/synapse/issues/8864))
- Pass `room_id` to `get_auth_chain_difference`. ([\#8879](https://github.com/matrix-org/synapse/issues/8879))
- Add type hints to push module. ([\#8880](https://github.com/matrix-org/synapse/issues/8880), [\#8882](https://github.com/matrix-org/synapse/issues/8882), [\#8901](https://github.com/matrix-org/synapse/issues/8901), [\#8940](https://github.com/matrix-org/synapse/issues/8940), [\#8943](https://github.com/matrix-org/synapse/issues/8943), [\#9020](https://github.com/matrix-org/synapse/issues/9020))
- Simplify logic for handling user-interactive-auth via single-sign-on servers. ([\#8881](https://github.com/matrix-org/synapse/issues/8881))
- Skip the SAML tests if the requirements (`pysaml2` and `xmlsec1`) aren't available. ([\#8905](https://github.com/matrix-org/synapse/issues/8905))
- Fix multiarch docker image builds. ([\#8906](https://github.com/matrix-org/synapse/issues/8906))
- Don't publish `latest` docker image until all archs are built. ([\#8909](https://github.com/matrix-org/synapse/issues/8909))
- Various clean-ups to the structured logging and logging context code. ([\#8916](https://github.com/matrix-org/synapse/issues/8916), [\#8935](https://github.com/matrix-org/synapse/issues/8935))
- Automatically drop stale forward-extremities under some specific conditions. ([\#8929](https://github.com/matrix-org/synapse/issues/8929))
- Refactor test utilities for injecting HTTP requests. ([\#8946](https://github.com/matrix-org/synapse/issues/8946))
- Add a maximum size of 50 kilobytes to .well-known lookups. ([\#8950](https://github.com/matrix-org/synapse/issues/8950))
- Fix bug in `generate_log_config` script which made it write empty files. ([\#8952](https://github.com/matrix-org/synapse/issues/8952))
- Clean up tox.ini file; disable coverage checking for non-test runs. ([\#8963](https://github.com/matrix-org/synapse/issues/8963))
- Add type hints to the admin and room list handlers. ([\#8973](https://github.com/matrix-org/synapse/issues/8973))
- Add type hints to the receipts and user directory handlers. ([\#8976](https://github.com/matrix-org/synapse/issues/8976))
- Drop the unused `local_invites` table. ([\#8979](https://github.com/matrix-org/synapse/issues/8979))
- Add type hints to the base storage code. ([\#8980](https://github.com/matrix-org/synapse/issues/8980))
- Support using PyJWT v2.0.0 in the test suite. ([\#8986](https://github.com/matrix-org/synapse/issues/8986))
- Fix `tests.federation.transport.RoomDirectoryFederationTests` and ensure it runs in CI. ([\#8998](https://github.com/matrix-org/synapse/issues/8998))
- Add type hints to the crypto module. ([\#8999](https://github.com/matrix-org/synapse/issues/8999))
 2021-01-06 07:42:38 -05:00
Patrick Cloke 8f08021e86 More updates to changes for consistency. 2021-01-06 07:36:52 -05:00
Patrick Cloke 62b5f13768 A few more tweaks to changes. 2021-01-06 07:34:11 -05:00
Patrick Cloke 0248409bfa
Handle a display name / avatar URL not included in a federation request. (#9023) 
These may be omitted if not set, but Synapse assumed they would
be in the response.
 2021-01-06 07:29:03 -05:00
Patrick Cloke bde6705ad1 Some manual tweaks to the changes file. 2021-01-06 07:20:12 -05:00
Patrick Cloke 2fe0fb21f6 1.25.0rc1 2021-01-06 07:08:13 -05:00
Patrick Cloke fb4a4f9f15 Merge branch 'release-v1.25.0' of github.com:matrix-org/synapse into matrix-org-hotfixes 2021-01-05 12:12:07 -05:00
Patrick Cloke 37eaf9c272
Fix-up assertions about last stream token in push (#9020) 
The last stream token is always known and we do not need to handle none.
 2021-01-05 10:53:15 -05:00
Richard van der Hoff 5a4f09228d Remove cache from room directory query results 
This reverts a285fe0. Hopefully the cache is no longer required, thanks to
 2021-01-05 13:52:36 +00:00
Eric Eastwood c027a199f3
Ignore date-rotated logs (#9018) 
Ex.

 - homeserver.log.2020-12-29
 - homeserver.log.2020-12-31
 2021-01-05 13:09:15 +00:00
Patrick Cloke 06fefe0bb1
Add type hints to the logging context code. (#8939) 2021-01-05 08:06:55 -05:00
Patrick Cloke 9dde9c9f01
Implement MSC2176: Updated redaction rules (#8984) 
An experimental room version ("org.matrix.msc2176") contains
the new redaction rules for testing.
 2021-01-05 07:41:48 -05:00
Richard van der Hoff 97d12dcf56 Merge remote-tracking branch 'origin/release-v1.25.0' into matrix-org-hotfixes 2021-01-05 11:32:29 +00:00
Richard van der Hoff 111b673fc1
Add initial support for a "pick your IdP" page (#9017) 
During login, if there are multiple IdPs enabled, offer the user a choice of
IdPs.
 2021-01-05 11:25:28 +00:00
Richard van der Hoff d2c616a413
Combine the SSO Redirect Servlets (#9015) 
* Implement CasHandler.handle_redirect_request

... to make it match OidcHandler and SamlHandler

* Clean up interface for OidcHandler.handle_redirect_request

Make it accept `client_redirect_url=None`.

* Clean up interface for `SamlHandler.handle_redirect_request`

... bring it into line with CAS and OIDC by making it take a Request parameter,
move the magic for `client_redirect_url` for UIA into the handler, and fix the
return type to be a `str` rather than a `bytes`.

* Define a common protocol for SSO auth provider impls

* Give SsoIdentityProvider an ID and register them

* Combine the SSO Redirect servlets

Now that the SsoHandler knows about the identity providers, we can combine the
various *RedirectServlets into a single implementation which delegates to the
right IdP.

* changelog
 2021-01-04 18:13:49 +00:00
Patrick Cloke 31b1905e13
Add type hints to the receipts and user directory handlers. (#8976) 2021-01-04 10:05:12 -05:00
Patrick Cloke 1c9a850562
Add type hints to the crypto module. (#8999) 2021-01-04 10:04:50 -05:00
Eric Eastwood a685bbb018
Add link to Synapse dev room to the relevant README section (#9002) 2021-01-04 08:59:19 -05:00
Patrick Cloke 0eccf53146
Use the SSO handler helpers for CAS registration/login. (#8856) 2021-01-03 16:25:44 +00:00
Andrew Morgan 168ba00d01
Fix RoomDirectoryFederationTests and make them actually run (#8998) 
The `RoomDirectoryFederationTests` tests were not being run unless explicitly called as an `__init__.py` file was not present in `tests/federation/transport/`. Thus the folder was not a python module, and `trial` did not look inside for any test cases to run. This was found while working on #6739.

This PR adds a `__init__.py` and also fixes the test in a couple ways:

- Switch to subclassing `unittest.FederatingHomeserverTestCase` instead, which sets up federation endpoints for us.
- Supply a `federation_auth_origin` to `make_request` in order to more act like the request is coming from another server, instead of just an unauthenicated client requesting a federation endpoint.

I found that the second point makes no difference to the test passing, but felt like the right thing to do if we're testing over federation.
 2020-12-30 19:27:32 +00:00
Patrick Cloke b7c580e333
Check if group IDs are valid before using them. (#8977) 2020-12-30 08:39:59 -05:00
Patrick Cloke 637282bb50
Add additional type hints to the storage module. (#8980) 2020-12-30 08:09:53 -05:00
Shashank Sabniveesu b8591899ab
Doc/move database setup instructions in install md (#8987) 2020-12-30 11:33:03 +00:00
Patrick Cloke 9999eb2d02
Add type hints to admin and room list handlers. (#8973) 2020-12-29 17:42:10 -05:00
Patrick Cloke 14a7371375
Validate input parameters for the sendToDevice API. (#8975) 
This makes the "messages" key in the content required. This is currently
optional in the spec, but that seems to be an error.
 2020-12-29 12:47:45 -05:00
Patrick Cloke f4f65f4e99 Allow redacting events on workers (#8994) 
Adds the redacts endpoint to workers that have the client listener.
 2020-12-29 11:06:10 -05:00
Jerin J Titus cfcf5541b4
Update the value of group_creation_prefix in sample config. (#8992) 
Removes the trailing slash with causes issues with matrix.to/Element.
 2020-12-29 09:30:48 -05:00
Patrick Cloke 68bb26da69
Allow redacting events on workers (#8994) 
Adds the redacts endpoint to workers that have the client listener.
 2020-12-29 07:40:12 -05:00
Patrick Cloke d0c3c24eb2
Drop the unused local_invites table. (#8979) 
This table has been unused since Synapse v1.17.0.
 2020-12-29 07:26:29 -05:00
Patrick Cloke a802606475
Support PyJWT v2.0.0. (#8986) 
Tests were broken due to an API changing. The code used in Synapse
proper should be compatible with both versions already.
 2020-12-22 13:00:14 -05:00
Patrick Cloke 4218473f9e
Refactor the CAS handler in prep for using the abstracted SSO code. (#8958) 
This makes the CAS handler look more like the SAML/OIDC handlers:

* Render errors to users instead of throwing JSON errors.
* Internal reorganization.
 2020-12-18 13:09:45 -05:00
Patrick Cloke 56e00ca85e
Send the location of the web client to the IS when inviting via 3PIDs. (#8930) 
Adds a new setting `email.invite_client_location` which, if defined, is
passed to the identity server during invites.
 2020-12-18 11:01:57 -05:00
Erik Johnston d781a81e69
Allow server admin to get admin bit in rooms where local user is an admin (#8756) 
This adds an admin API that allows a server admin to get power in a room if a local user has power in a room. Will also invite the user if they're not in the room and its a private room. Can specify another user (rather than the admin user) to be granted power.

Co-authored-by: Matthew Hodgson <matthew@matrix.org>
 2020-12-18 15:37:19 +00:00
Erik Johnston 5e7d75daa2
Fix mainline ordering in state res v2 (#8971) 
This had two effects 1) it'd give the wrong answer and b) would iterate
*all* power levels in the auth chain of each event. The latter of which
can be *very* expensive for certain types of IRC bridge rooms that have
large numbers of power level changes.
 2020-12-18 15:00:34 +00:00
Richard van der Hoff 28877fade9
Implement a username picker for synapse (#8942) 
The final part (for now) of my work to implement a username picker in synapse itself. The idea is that we allow
`UsernameMappingProvider`s to return `localpart=None`, in which case, rather than redirecting the browser
back to the client, we redirect to a username-picker resource, which allows the user to enter a username.
We *then* complete the SSO flow (including doing the client permission checks).

The static resources for the username picker itself (in 
https://github.com/matrix-org/synapse/tree/rav/username_picker/synapse/res/username_picker)
are essentially lifted wholesale from
https://github.com/matrix-org/matrix-synapse-saml-mozilla/tree/master/matrix_synapse_saml_mozilla/res. 
As the comment says, we might want to think about making them customisable, but that can be a follow-up. 

Fixes #8876.
 2020-12-18 14:19:46 +00:00
Patrick Cloke 5d4c330ed9
Allow re-using a UI auth validation for a period of time (#8970) 2020-12-18 07:33:57 -05:00
Patrick Cloke 4136255d3c
Ensure that a URL exists in the content during push. (#8965) 
This fixes an KeyError exception, after this PR the content
is just considered unknown.
 2020-12-18 07:26:15 -05:00
Erik Johnston a7a913918c Merge remote-tracking branch 'origin/erikj/as_mau_block' into develop 2020-12-18 09:51:56 +00:00
Erik Johnston 70586aa63e
Try and drop stale extremities. (#8929) 
If we see stale extremities while persisting events, and notice that
they don't change the result of state resolution, we drop them.
 2020-12-18 09:49:18 +00:00
Richard van der Hoff f1db20b5a5
Clean up tox.ini (#8963) 
... and disable coverage tracking for mypy and friends.
 2020-12-17 22:58:00 +00:00
Erik Johnston 14eab1b4d2
Update tests/test_mau.py 
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2020-12-17 16:14:13 +00:00
Richard van der Hoff c9c1c9d82f
Fix UsersListTestCase (#8964) 2020-12-17 10:46:40 -05:00
Brendan Abolivier f2783fc201
Use the simple dictionary in full text search for the user directory (#8959) 
* Use the simple dictionary in fts for the user directory

* Clarify naming
 2020-12-17 14:42:30 +01:00