Merge remote-tracking branch 'origin/release-v1.46' into develop

2021-10-27 14:45:40 +01:00 · 2021-10-27 14:45:40 +01:00 · 179dc8ae9e
commit 179dc8ae9e
parent c7a5e49664 e0ef8fe58d
63 changed files with 77 additions and 60 deletions
--- a/CHANGES.md
+++ b/CHANGES.md
@ -1,3 +1,71 @@
+Synapse 1.46.0rc1 (2021-10-27)
+==============================
+
+The cause of the [performance regression affecting Synapse 1.44](https://github.com/matrix-org/synapse/issues/11049) has been identified and fixed. ([\#11177](https://github.com/matrix-org/synapse/issues/11177))
+
+Features
+--------
+
+- Add support for Ubuntu 21.10 "Impish Indri". ([\#11024](https://github.com/matrix-org/synapse/issues/11024))
+- Port the Password Auth Providers module interface to the new generic interface. ([\#10548](https://github.com/matrix-org/synapse/issues/10548), [\#11180](https://github.com/matrix-org/synapse/issues/11180))
+- Experimental support for the thread relation defined in [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440). ([\#11088](https://github.com/matrix-org/synapse/issues/11088), [\#11181](https://github.com/matrix-org/synapse/issues/11181), [\#11192](https://github.com/matrix-org/synapse/issues/11192))
+- Users admin API can now also modify user type in addition to allowing it to be set on user creation. ([\#11174](https://github.com/matrix-org/synapse/issues/11174))
+
+
+Bugfixes
+--------
+
+- Newly-created public rooms are now only assigned an alias if the room's creation has not been blocked by permission settings. Contributed by @AndrewFerr. ([\#10930](https://github.com/matrix-org/synapse/issues/10930))
+- Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state. ([\#11001](https://github.com/matrix-org/synapse/issues/11001), [\#11009](https://github.com/matrix-org/synapse/issues/11009), [\#11012](https://github.com/matrix-org/synapse/issues/11012))
+- Fix 500 error on `/messages` when the server accumulates more than 5 backwards extremities at a given depth for a room. ([\#11027](https://github.com/matrix-org/synapse/issues/11027))
+- Fix a bug where setting a user's `external_id` via the admin API returns 500 and deletes user's existing external mappings if that external ID is already mapped. ([\#11051](https://github.com/matrix-org/synapse/issues/11051))
+- Fix a long-standing bug where users excluded from the user directory were added into the directory if they belonged to a room which became public or private. ([\#11075](https://github.com/matrix-org/synapse/issues/11075))
+- Fix a long-standing bug when attempting to preview URLs which are in the `windows-1252` character encoding. ([\#11077](https://github.com/matrix-org/synapse/issues/11077), [\#11089](https://github.com/matrix-org/synapse/issues/11089))
+- Fix broken export-data admin command and add test script checking the command to CI. ([\#11078](https://github.com/matrix-org/synapse/issues/11078))
+- Show an error when timestamp in seconds is provided to the `/purge_media_cache` Admin API. ([\#11101](https://github.com/matrix-org/synapse/issues/11101))
+- Fix local users who left all their rooms being removed from the user directory, even if the `search_all_users` config option was enabled. ([\#11103](https://github.com/matrix-org/synapse/issues/11103))
+- Fix a bug which caused the module API's `get_user_ip_and_agents` function to always fail on workers. `get_user_ip_and_agents` was introduced in 1.44.0 and did not function correctly on worker processes at the time. ([\#11112](https://github.com/matrix-org/synapse/issues/11112))
+- Identity server connection is no longer ignoring `ip_range_whitelist`. ([\#11120](https://github.com/matrix-org/synapse/issues/11120))
+- Fix a bug introduced in Synapse 1.45.0 breaking the configuration file parsing script. ([\#11145](https://github.com/matrix-org/synapse/issues/11145))
+- Fix a performance regression introduced in 1.44.0 which could cause client requests to time out when making large numbers of outbound requests. ([\#11177](https://github.com/matrix-org/synapse/issues/11177), [\#11190](https://github.com/matrix-org/synapse/issues/11190))
+- Resolve and share `state_groups` for all [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) historical events in batch. ([\#10975](https://github.com/matrix-org/synapse/issues/10975))
+
+
+Improved Documentation
+----------------------
+
+- Fix broken links relating to module API deprecation in the upgrade notes. ([\#11069](https://github.com/matrix-org/synapse/issues/11069))
+- Add more information about what happens when a user is deactivated. ([\#11083](https://github.com/matrix-org/synapse/issues/11083))
+- Clarify the the sample log config can be copied from the documentation without issue. ([\#11092](https://github.com/matrix-org/synapse/issues/11092))
+- Update the admin API documentation with an updated list of the characters allowed in registration tokens. ([\#11093](https://github.com/matrix-org/synapse/issues/11093))
+- Document Synapse's behaviour when dealing with multiple modules registering the same callbacks and/or handlers for the same HTTP endpoints. ([\#11096](https://github.com/matrix-org/synapse/issues/11096))
+- Fix instances of `[example]{.title-ref}` in the upgrade documentation as a result of prior RST to Markdown conversion. ([\#11118](https://github.com/matrix-org/synapse/issues/11118))
+- Document the version of Synapse each module callback was introduced in. ([\#11132](https://github.com/matrix-org/synapse/issues/11132))
+- Document the version of Synapse that introduced each module API method. ([\#11183](https://github.com/matrix-org/synapse/issues/11183))
+
+
+Internal Changes
+----------------
+- Fix spurious warnings about losing the logging context on the `ReplicationCommandHandler` when losing the replication connection. ([\#10984](https://github.com/matrix-org/synapse/issues/10984))
+- Include rejected status when we log events. ([\#11008](https://github.com/matrix-org/synapse/issues/11008))
+- Add some extra logging to the event persistence code. ([\#11014](https://github.com/matrix-org/synapse/issues/11014))
+- Rearrange the internal workings of the incremental user directory updates. ([\#11035](https://github.com/matrix-org/synapse/issues/11035))
+- Fix a long-standing bug where users excluded from the directory could still be added to the `users_who_share_private_rooms` table after a regular user joins a private room. ([\#11143](https://github.com/matrix-org/synapse/issues/11143))
+- Add and improve type hints. ([\#10972](https://github.com/matrix-org/synapse/issues/10972), [\#11055](https://github.com/matrix-org/synapse/issues/11055), [\#11066](https://github.com/matrix-org/synapse/issues/11066), [\#11076](https://github.com/matrix-org/synapse/issues/11076), [\#11095](https://github.com/matrix-org/synapse/issues/11095), [\#11109](https://github.com/matrix-org/synapse/issues/11109), [\#11121](https://github.com/matrix-org/synapse/issues/11121), [\#11146](https://github.com/matrix-org/synapse/issues/11146))
+- Mark the Synapse package as containing type annotations and fix export declarations so that Synapse pluggable modules may be type checked against Synapse. ([\#11054](https://github.com/matrix-org/synapse/issues/11054))
+- Remove dead code from `MediaFilePaths`. ([\#11056](https://github.com/matrix-org/synapse/issues/11056))
+- Be more lenient when parsing oEmbed response versions. ([\#11065](https://github.com/matrix-org/synapse/issues/11065))
+- Create a separate module for the retention configuration. ([\#11070](https://github.com/matrix-org/synapse/issues/11070))
+- Clean up some of the federation event authentication code for clarity. ([\#11115](https://github.com/matrix-org/synapse/issues/11115), [\#11116](https://github.com/matrix-org/synapse/issues/11116), [\#11122](https://github.com/matrix-org/synapse/issues/11122))
+- Add docstrings and comments to the application service ephemeral event sending code. ([\#11138](https://github.com/matrix-org/synapse/issues/11138))
+- Update the `sign_json` script to support inline configuration of the signing key. ([\#11139](https://github.com/matrix-org/synapse/issues/11139))
+- Fix broken link in the docker image README. ([\#11144](https://github.com/matrix-org/synapse/issues/11144))
+- Always dump logs from unit tests during CI runs. ([\#11068](https://github.com/matrix-org/synapse/issues/11068))
+- Add tests for `MediaFilePaths` class. ([\#11057](https://github.com/matrix-org/synapse/issues/11057))
+- Simplify the user admin API tests. ([\#11048](https://github.com/matrix-org/synapse/issues/11048))
+- Add a test for the workaround introduced in [\#11042](https://github.com/matrix-org/synapse/pull/11042) concerning the behaviour of third-party rule modules and `SynapseError`s. ([\#11071](https://github.com/matrix-org/synapse/issues/11071))
+
+
 Synapse 1.45.1 (2021-10-20)
 ===========================

--- a/changelog.d/10548.feature
+++ b/changelog.d/10548.feature
@ -1 +0,0 @@
-Port the Password Auth Providers module interface to the new generic interface.
--- a/changelog.d/10930.bugfix
+++ b/changelog.d/10930.bugfix
@ -1 +0,0 @@
-Newly-created public rooms are now only assigned an alias if the room's creation has not been blocked by permission settings. Contributed by @AndrewFerr.
--- a/changelog.d/10972.misc
+++ b/changelog.d/10972.misc
@ -1 +0,0 @@
-Add type hints to `synapse.storage.databases.main.client_ips`.
--- a/changelog.d/10975.feature
+++ b/changelog.d/10975.feature
@ -1 +0,0 @@
-Resolve and share `state_groups` for all [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) historical events in batch.
--- a/changelog.d/10984.misc
+++ b/changelog.d/10984.misc
@ -1 +0,0 @@
-Fix spurious warnings about losing the logging context on the `ReplicationCommandHandler` when losing the replication connection.
--- a/changelog.d/11001.bugfix
+++ b/changelog.d/11001.bugfix
@ -1 +0,0 @@
- Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state.
--- a/changelog.d/11008.misc
+++ b/changelog.d/11008.misc
@ -1 +0,0 @@
-Include rejected status when we log events.
--- a/changelog.d/11009.bugfix
+++ b/changelog.d/11009.bugfix
@ -1 +0,0 @@
-Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state.
--- a/changelog.d/11012.bugfix
+++ b/changelog.d/11012.bugfix
@ -1 +0,0 @@
-Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state.
--- a/changelog.d/11014.misc
+++ b/changelog.d/11014.misc
@ -1 +0,0 @@
-Add some extra logging to the event persistence code.
--- a/changelog.d/11024.misc
+++ b/changelog.d/11024.misc
@ -1 +0,0 @@
-Add support for Ubuntu 21.10 "Impish Indri".
--- a/changelog.d/11027.bugfix
+++ b/changelog.d/11027.bugfix
@ -1 +0,0 @@
-Fix 500 error on `/messages` when the server accumulates more than 5 backwards extremities at a given depth for a room.
--- a/changelog.d/11035.misc
+++ b/changelog.d/11035.misc
@ -1 +0,0 @@
-Rearrange the internal workings of the incremental user directory updates.
--- a/changelog.d/11048.misc
+++ b/changelog.d/11048.misc
@ -1 +0,0 @@
-Simplify the user admin API tests.
--- a/changelog.d/11051.bugfix
+++ b/changelog.d/11051.bugfix
@ -1 +0,0 @@
-Fix a bug where setting a user's external_id via the admin API returns 500 and deletes users existing external mappings if that external ID is already mapped.
--- a/changelog.d/11054.misc
+++ b/changelog.d/11054.misc
@ -1 +0,0 @@
-Mark the Synapse package as containing type annotations and fix export declarations so that Synapse pluggable modules may be type checked against Synapse.
--- a/changelog.d/11055.misc
+++ b/changelog.d/11055.misc
@ -1 +0,0 @@
-Improve type hints for `_wrap_in_base_path` decorator used by `MediaFilePaths`.
--- a/changelog.d/11056.misc
+++ b/changelog.d/11056.misc
@ -1 +0,0 @@
-Remove dead code from `MediaFilePaths`.
--- a/changelog.d/11057.misc
+++ b/changelog.d/11057.misc
@ -1 +0,0 @@
-Add tests for `MediaFilePaths` class.
--- a/changelog.d/11065.misc
+++ b/changelog.d/11065.misc
@ -1 +0,0 @@
-Be more lenient when parsing oEmbed response versions.
--- a/changelog.d/11066.misc
+++ b/changelog.d/11066.misc
@ -1 +0,0 @@
-Add type hints to `synapse.events`.
--- a/changelog.d/11068.misc
+++ b/changelog.d/11068.misc
@ -1 +0,0 @@
-Always dump logs from unit tests during CI runs.
--- a/changelog.d/11069.doc
+++ b/changelog.d/11069.doc
@ -1 +0,0 @@
-Fix broken links relating to module API deprecation in the upgrade notes.
--- a/changelog.d/11070.misc
+++ b/changelog.d/11070.misc
@ -1 +0,0 @@
-Create a separate module for the retention configuration.
--- a/changelog.d/11071.misc
+++ b/changelog.d/11071.misc
@ -1 +0,0 @@
-Add a test for the workaround introduced in [\#11042](https://github.com/matrix-org/synapse/pull/11042) concerning the behaviour of third-party rule modules and `SynapseError`s.
--- a/changelog.d/11075.bugfix
+++ b/changelog.d/11075.bugfix
@ -1 +0,0 @@
-Fix a long-standing bug where users excluded from the user directory were added into the directory if they belonged to a room which became public or private.
--- a/changelog.d/11076.misc
+++ b/changelog.d/11076.misc
@ -1 +0,0 @@
-Fix type hints in the relations tests.
--- a/changelog.d/11077.bugfix
+++ b/changelog.d/11077.bugfix
@ -1 +0,0 @@
-Fix a long-standing bug when attempting to preview URLs which are in the `windows-1252` character encoding.
--- a/changelog.d/11078.bugfix
+++ b/changelog.d/11078.bugfix
@ -1 +0,0 @@
-Fix broken export-data admin command and add test script checking the command to CI.
--- a/changelog.d/11083.doc
+++ b/changelog.d/11083.doc
@ -1 +0,0 @@
-Add more information about what happens when a user is deactivated.
--- a/changelog.d/11088.feature
+++ b/changelog.d/11088.feature
@ -1 +0,0 @@
-Experimental support for the thread relation defined in [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440).
--- a/changelog.d/11089.bugfix
+++ b/changelog.d/11089.bugfix
@ -1 +0,0 @@
-Fix a long-standing bug when attempting to preview URLs which are in the `windows-1252` character encoding.
--- a/changelog.d/11092.doc
+++ b/changelog.d/11092.doc
@ -1 +0,0 @@
-Clarify the the sample log config can be copied from the documentation without issue.
--- a/changelog.d/11093.doc
+++ b/changelog.d/11093.doc
@ -1 +0,0 @@
-Update the admin API documentation with an updated list of the characters allowed in registration tokens.
--- a/changelog.d/11095.misc
+++ b/changelog.d/11095.misc
@ -1 +0,0 @@
-Add type hints to most `HomeServer` parameters.
--- a/changelog.d/11096.doc
+++ b/changelog.d/11096.doc
@ -1 +0,0 @@
-Document Synapse's behaviour when dealing with multiple modules registering the same callbacks and/or handlers for the same HTTP endpoints.
--- a/changelog.d/11101.bugfix
+++ b/changelog.d/11101.bugfix
@ -1 +0,0 @@
-Show an error when timestamp in seconds is provided to the `/purge_media_cache` Admin API.
--- a/changelog.d/11103.bugfix
+++ b/changelog.d/11103.bugfix
@ -1 +0,0 @@
-Fix local users who left all their rooms being removed from the user directory, even if the "search_all_users" config option was enabled.
--- a/changelog.d/11109.misc
+++ b/changelog.d/11109.misc
@ -1 +0,0 @@
-Add missing type hints to `synapse.api` module.
--- a/changelog.d/11112.bugfix
+++ b/changelog.d/11112.bugfix
@ -1 +0,0 @@
-Fix a bug which caused the module API's `get_user_ip_and_agents` function to always fail on workers. `get_user_ip_and_agents` was introduced in 1.44.0 and did not function correctly on worker processes at the time.
--- a/changelog.d/11115.misc
+++ b/changelog.d/11115.misc
@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
--- a/changelog.d/11116.misc
+++ b/changelog.d/11116.misc
@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
--- a/changelog.d/11118.doc
+++ b/changelog.d/11118.doc
@ -1 +0,0 @@
-Fix instances of `[example]{.title-ref}` in the upgrade documentation as a result of prior RST to Markdown conversion.
--- a/changelog.d/11120.bugfix
+++ b/changelog.d/11120.bugfix
@ -1 +0,0 @@
-Identity server connection is no longer ignoring `ip_range_whitelist`.
--- a/changelog.d/11121.misc
+++ b/changelog.d/11121.misc
@ -1 +0,0 @@
-Add type hints for event fetching.
--- a/changelog.d/11122.misc
+++ b/changelog.d/11122.misc
@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
--- a/changelog.d/11132.doc
+++ b/changelog.d/11132.doc
@ -1 +0,0 @@
-Document the version of Synapse each module callback was introduced in.
--- a/changelog.d/11138.misc
+++ b/changelog.d/11138.misc
@ -1 +0,0 @@
-Add docstrings and comments to the application service ephemeral event sending code.
--- a/changelog.d/11139.misc
+++ b/changelog.d/11139.misc
@ -1 +0,0 @@
-Update the `sign_json` script to support inline configuration of the signing key.
--- a/changelog.d/11143.misc
+++ b/changelog.d/11143.misc
@ -1 +0,0 @@
-Fix a long-standing bug where users excluded from the directory could still be added to the `users_who_share_private_rooms` table after a regular user joins a private room.
--- a/changelog.d/11144.misc
+++ b/changelog.d/11144.misc
@ -1 +0,0 @@
-Fix broken link in the docker image README.
--- a/changelog.d/11145.bugfix
+++ b/changelog.d/11145.bugfix
@ -1 +0,0 @@
-Fix a bug introduced in Synapse v1.45.0 breaking the configuration file parsing script.
--- a/changelog.d/11146.misc
+++ b/changelog.d/11146.misc
@ -1 +0,0 @@
-Add missing type hints to `synapse.crypto`.
--- a/changelog.d/11174.feature
+++ b/changelog.d/11174.feature
@ -1 +0,0 @@
-Users admin API can now also modify user type in addition to allowing it to be set on user creation.
--- a/changelog.d/11177.bugfix
+++ b/changelog.d/11177.bugfix
@ -1 +0,0 @@
-Fix a performance regression introduced in v1.44.0 which could cause client requests to time out when making large numbers of outbound requests.
--- a/changelog.d/11180.feature
+++ b/changelog.d/11180.feature
@ -1 +0,0 @@
-Port the Password Auth Providers module interface to the new generic interface.
--- a/changelog.d/11181.feature
+++ b/changelog.d/11181.feature
@ -1 +0,0 @@
-Experimental support for the thread relation defined in [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440).
--- a/changelog.d/11183.doc
+++ b/changelog.d/11183.doc
@ -1 +0,0 @@
-Document the version of Synapse that introduced each module API method.
--- a/debian/changelog
+++ b/debian/changelog
@ -1,3 +1,9 @@
+matrix-synapse-py3 (1.46.0~rc1) stable; urgency=medium
+
+  * New synapse release 1.46.0~rc1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 26 Oct 2021 14:04:04 +0100
+
 matrix-synapse-py3 (1.45.1) stable; urgency=medium

  * New synapse release 1.45.1.
--- a/synapse/init.py
+++ b/synapse/init.py
@ -47,7 +47,7 @@ try:
 except ImportError:
    pass

-__version__ = "1.45.1"
+__version__ = "1.46.0rc1"

 if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
    # We import here so that we don't have to install a bunch of deps when
--- a/synapse/app/_base.py
+++ b/synapse/app/_base.py
@ -346,6 +346,7 @@ async def start(hs: "HomeServer"):
    # numbers of DNS requests don't starve out other users of the threadpool.
    resolver_threadpool = ThreadPool(name="gai_resolver")
    resolver_threadpool.start()
+    reactor.addSystemEventTrigger("during", "shutdown", resolver_threadpool.stop)
    reactor.installNameResolver(
        GAIResolver(reactor, getThreadPool=lambda: resolver_threadpool)
    )
--- a/synapse/storage/databases/main/events_bg_updates.py
+++ b/synapse/storage/databases/main/events_bg_updates.py
@ -1108,7 +1108,7 @@ class EventsBackgroundUpdatesStore(SQLBaseStore):
                """
                SELECT event_id, json FROM event_json
                LEFT JOIN event_relations USING (event_id)
-                WHERE event_id > ? AND relates_to_id IS NULL
+                WHERE event_id > ? AND event_relations.event_id IS NULL
                ORDER BY event_id LIMIT ?
                """,
                (last_event_id, batch_size),
				`@ -1 +0,0 @@`
				`Port the Password Auth Providers module interface to the new generic interface.`
				`@ -1 +0,0 @@`
				`Newly-created public rooms are now only assigned an alias if the room's creation has not been blocked by permission settings. Contributed by @AndrewFerr.`
				`@ -1 +0,0 @@`
				Add type hints to `synapse.storage.databases.main.client_ips`.
				`@ -1 +0,0 @@`
				Resolve and share `state_groups` for all [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) historical events in batch.
				`@ -1 +0,0 @@`
				Fix spurious warnings about losing the logging context on the `ReplicationCommandHandler` when losing the replication connection.
				`@ -1 +0,0 @@`
				`Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state.`
				`@ -1 +0,0 @@`
				`Add some extra logging to the event persistence code.`
				`@ -1 +0,0 @@`
				`Add support for Ubuntu 21.10 "Impish Indri".`
				`@ -1 +0,0 @@`
				Fix 500 error on `/messages` when the server accumulates more than 5 backwards extremities at a given depth for a room.
				`@ -1 +0,0 @@`
				`Rearrange the internal workings of the incremental user directory updates.`