Merge branch 'release-v1.42' into matrix-org-hotfixes

This commit is contained in:
Brendan Abolivier 2021-09-06 15:38:03 +01:00
commit 92bbe0ebe1
57 changed files with 216 additions and 53 deletions

View file

@ -1,4 +1,84 @@
Users will stop receiving message updates via email for addresses that were previously linked to their account
Synapse 1.42.0rc2 (2021-09-06)
==============================
Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).
Features
--------
- Support room version 9 from [MSC3375](https://github.com/matrix-org/matrix-doc/pull/3375). ([\#10747](https://github.com/matrix-org/synapse/issues/10747))
Internal Changes
----------------
- Print a warning when using one of the deprecated `template_dir` settings. ([\#10768](https://github.com/matrix-org/synapse/issues/10768))
Synapse 1.42.0rc1 (2021-09-01)
==============================
Features
--------
- Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142))
- Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452))
- Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524))
- Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10613](https://github.com/matrix-org/synapse/issues/10613), [\#10725](https://github.com/matrix-org/synapse/issues/10725))
Bugfixes
--------
- Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232))
- Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561))
- Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581), [\#10734](https://github.com/matrix-org/synapse/issues/10734))
- Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593))
- Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644))
- Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654))
- Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677))
- Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679))
- Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684))
- Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703))
- Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713))
Improved Documentation
----------------------
- Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192))
- Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595))
- Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639))
- Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708))
- Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711))
Deprecations and Removals
-------------------------
- Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830))
Internal Changes
----------------
- Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608))
- Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614), [\#10615](https://github.com/matrix-org/synapse/issues/10615), [\#10624](https://github.com/matrix-org/synapse/issues/10624), [\#10640](https://github.com/matrix-org/synapse/issues/10640))
- Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621))
- Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627))
- Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629), [\#10642](https://github.com/matrix-org/synapse/issues/10642))
- Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630))
- Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645))
- Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647))
- Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651), [\#10672](https://github.com/matrix-org/synapse/issues/10672))
- Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662))
- Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664))
- Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665), [\#10666](https://github.com/matrix-org/synapse/issues/10666), [\#10674](https://github.com/matrix-org/synapse/issues/10674))
- Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667))
- Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686))
- Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692))
- Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706))
Synapse 1.41.1 (2021-08-31)
===========================

View file

@ -1 +0,0 @@
Add support for [MSC3231 - Token authenticated registration](https://github.com/matrix-org/matrix-doc/pull/3231). Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown.

View file

@ -1 +0,0 @@
Add documentation on how to connect Django with synapse using oidc and django-oauth-toolkit. Contributed by @HugoDelval.

View file

@ -1 +0,0 @@
Validate new `m.room.power_levels` events. Contributed by @aaronraimist.

View file

@ -1 +0,0 @@
Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose enable_set_displayname in capabilities.

View file

@ -1 +0,0 @@
Port the PresenceRouter module interface to the new generic interface.

View file

@ -1 +0,0 @@
Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown.

View file

@ -1 +0,0 @@
Remove pushers when deleting a 3pid from an account. Pushers for old unlinked emails will also be deleted.

View file

@ -1 +0,0 @@
Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly.

View file

@ -1 +0,0 @@
Advertise https://matrix-org.github.io/synapse docs in README and CONTRIBUTING files.

View file

@ -1 +0,0 @@
Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel.

View file

@ -1 +0,0 @@
Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946).

View file

@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.

View file

@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.

View file

@ -1 +0,0 @@
Add a comment asking developers to leave a reason when bumping the database schema version.

View file

@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.

View file

@ -1 +0,0 @@
Remove not needed database updates in modify user admin API.

View file

@ -1 +0,0 @@
Convert room member storage tuples to `attrs` classes.

View file

@ -1 +0,0 @@
Use auto-attribs for the attrs classes used in sync.

View file

@ -1 +0,0 @@
Fix some of the titles not rendering in the OIDC documentation.

View file

@ -1 +0,0 @@
Clean up some of the federation event authentication code for clarity.

View file

@ -1 +0,0 @@
Convert room member storage tuples to `attrs` classes.

View file

@ -1 +0,0 @@
Rooms with unsupported room versions are no longer returned via `/sync`.

View file

@ -1 +0,0 @@
Make `backfill` and `get_missing_events` use the same codepath.

View file

@ -1 +0,0 @@
Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation.

View file

@ -1 +0,0 @@
Run a nightly CI build against Twisted trunk.

View file

@ -1 +0,0 @@
Enforce the maximum length for per-room display names and avatar URLs.

View file

@ -1 +0,0 @@
Do not print out stack traces for network errors when fetching data over federation.

View file

@ -1 +0,0 @@
Simplify tests for device admin rest API.

View file

@ -1 +0,0 @@
Add missing type hints to REST servlets.

View file

@ -1 +0,0 @@
Add missing type hints to REST servlets.

View file

@ -1 +0,0 @@
Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent.

View file

@ -1 +0,0 @@
Run a nightly CI build against Twisted trunk.

View file

@ -1 +0,0 @@
Add missing type hints to REST servlets.

View file

@ -1 +0,0 @@
Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart.

View file

@ -1 +0,0 @@
Improve ServerNoticeServlet to avoid duplicate requests and add unit tests.

View file

@ -1 +0,0 @@
Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating.

View file

@ -1 +0,0 @@
Update `complement.sh` to rebuild the base Docker image when run with workers.

View file

@ -1 +0,0 @@
Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`.

View file

@ -1 +0,0 @@
Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang.

View file

@ -1 +0,0 @@
Remove unused `compare_digest` function.

View file

@ -1 +0,0 @@
Minor clarifications to the documentation for reverse proxies.

View file

@ -1 +0,0 @@
Removed table of contents from the top of installation and contributing documentation pages.

View file

@ -1 +0,0 @@
Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library.

View file

@ -1 +0,0 @@
Fix unauthorised exposure of room metadata to communities.

View file

@ -1 +0,0 @@
Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946).

View file

@ -1 +0,0 @@
Remove pushers when deleting a 3pid from an account. Pushers for old unlinked emails will also be deleted.

View file

@ -1 +0,0 @@
Remove deprecated Shutdown Room and Purge Room Admin API.

12
debian/changelog vendored
View file

@ -1,3 +1,15 @@
matrix-synapse-py3 (1.42.0~rc2) stable; urgency=medium
* New synapse release 1.42.0~rc2.
-- Synapse Packaging team <packages@matrix.org> Mon, 06 Sep 2021 15:25:13 +0100
matrix-synapse-py3 (1.42.0~rc1) stable; urgency=medium
* New synapse release 1.42.0rc1.
-- Synapse Packaging team <packages@matrix.org> Wed, 01 Sep 2021 11:37:48 +0100
matrix-synapse-py3 (1.41.1) stable; urgency=high
* New synapse release 1.41.1.

View file

@ -85,7 +85,7 @@ process, for example:
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
```
# Upgrading to v1.xx.0
# Upgrading to v1.42.0
## Removal of old Room Admin API
@ -107,12 +107,12 @@ This may affect you if you make use of custom HTML templates for the
The template is now provided an `error` variable if the authentication
process failed. See the default templates linked above for an example.
# Upgrading to v1.42.0
## Removal of out-of-date email pushers
Users will stop receiving message updates via email for addresses that were
once, but not still, linked to their account.
# Upgrading to v1.41.0
## Add support for routing outbound HTTP requests via a proxy for federation

View file

@ -47,7 +47,7 @@ try:
except ImportError:
pass
__version__ = "1.41.1"
__version__ = "1.42.0rc2"
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
# We import here so that we don't have to install a bunch of deps when

View file

@ -70,6 +70,9 @@ class RoomVersion:
msc2176_redaction_rules = attr.ib(type=bool)
# MSC3083: Support the 'restricted' join_rule.
msc3083_join_rules = attr.ib(type=bool)
# MSC3375: Support for the proper redaction rules for MSC3083. This mustn't
# be enabled if MSC3083 is not.
msc3375_redaction_rules = attr.ib(type=bool)
# MSC2403: Allows join_rules to be set to 'knock', changes auth rules to allow sending
# m.room.membership event with membership 'knock'.
msc2403_knocking = attr.ib(type=bool)
@ -92,6 +95,7 @@ class RoomVersions:
limit_notifications_power_levels=False,
msc2176_redaction_rules=False,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=False,
msc2716_historical=False,
msc2716_redactions=False,
@ -107,6 +111,7 @@ class RoomVersions:
limit_notifications_power_levels=False,
msc2176_redaction_rules=False,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=False,
msc2716_historical=False,
msc2716_redactions=False,
@ -122,6 +127,7 @@ class RoomVersions:
limit_notifications_power_levels=False,
msc2176_redaction_rules=False,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=False,
msc2716_historical=False,
msc2716_redactions=False,
@ -137,6 +143,7 @@ class RoomVersions:
limit_notifications_power_levels=False,
msc2176_redaction_rules=False,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=False,
msc2716_historical=False,
msc2716_redactions=False,
@ -152,6 +159,7 @@ class RoomVersions:
limit_notifications_power_levels=False,
msc2176_redaction_rules=False,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=False,
msc2716_historical=False,
msc2716_redactions=False,
@ -167,6 +175,7 @@ class RoomVersions:
limit_notifications_power_levels=True,
msc2176_redaction_rules=False,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=False,
msc2716_historical=False,
msc2716_redactions=False,
@ -182,6 +191,7 @@ class RoomVersions:
limit_notifications_power_levels=True,
msc2176_redaction_rules=True,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=False,
msc2716_historical=False,
msc2716_redactions=False,
@ -197,6 +207,7 @@ class RoomVersions:
limit_notifications_power_levels=True,
msc2176_redaction_rules=False,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=True,
msc2716_historical=False,
msc2716_redactions=False,
@ -212,6 +223,23 @@ class RoomVersions:
limit_notifications_power_levels=True,
msc2176_redaction_rules=False,
msc3083_join_rules=True,
msc3375_redaction_rules=False,
msc2403_knocking=True,
msc2716_historical=False,
msc2716_redactions=False,
)
V9 = RoomVersion(
"9",
RoomDisposition.STABLE,
EventFormatVersions.V3,
StateResolutionVersions.V2,
enforce_key_validity=True,
special_case_aliases_auth=False,
strict_canonicaljson=True,
limit_notifications_power_levels=True,
msc2176_redaction_rules=False,
msc3083_join_rules=True,
msc3375_redaction_rules=True,
msc2403_knocking=True,
msc2716_historical=False,
msc2716_redactions=False,
@ -227,6 +255,7 @@ class RoomVersions:
limit_notifications_power_levels=True,
msc2176_redaction_rules=False,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=True,
msc2716_historical=True,
msc2716_redactions=False,
@ -242,6 +271,7 @@ class RoomVersions:
limit_notifications_power_levels=True,
msc2176_redaction_rules=False,
msc3083_join_rules=False,
msc3375_redaction_rules=False,
msc2403_knocking=True,
msc2716_historical=True,
msc2716_redactions=True,
@ -261,6 +291,7 @@ KNOWN_ROOM_VERSIONS: Dict[str, RoomVersion] = {
RoomVersions.V7,
RoomVersions.MSC2716,
RoomVersions.V8,
RoomVersions.V9,
)
}

View file

@ -11,8 +11,20 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from synapse.config._base import Config, ConfigError
logger = logging.getLogger(__name__)
LEGACY_TEMPLATE_DIR_WARNING = """
This server's configuration file is using the deprecated 'template_dir' setting in the
'account_validity' section. Support for this setting has been deprecated and will be
removed in a future version of Synapse. Server admins should instead use the new
'custom_templates_directory' setting documented here:
https://matrix-org.github.io/synapse/latest/templates.html
---------------------------------------------------------------------------------------"""
class AccountValidityConfig(Config):
section = "account_validity"
@ -69,6 +81,8 @@ class AccountValidityConfig(Config):
# Load account validity templates.
account_validity_template_dir = account_validity_config.get("template_dir")
if account_validity_template_dir is not None:
logger.warning(LEGACY_TEMPLATE_DIR_WARNING)
account_renewed_template_filename = account_validity_config.get(
"account_renewed_html_path", "account_renewed.html"

View file

@ -16,6 +16,7 @@
# This file can't be called email.py because if it is, we cannot:
import email.utils
import logging
import os
from enum import Enum
from typing import Optional
@ -24,6 +25,8 @@ import attr
from ._base import Config, ConfigError
logger = logging.getLogger(__name__)
MISSING_PASSWORD_RESET_CONFIG_ERROR = """\
Password reset emails are enabled on this homeserver due to a partial
'email' block. However, the following required keys are missing:
@ -44,6 +47,14 @@ DEFAULT_SUBJECTS = {
"email_validation": "[%(server_name)s] Validate your email",
}
LEGACY_TEMPLATE_DIR_WARNING = """
This server's configuration file is using the deprecated 'template_dir' setting in the
'email' section. Support for this setting has been deprecated and will be removed in a
future version of Synapse. Server admins should instead use the new
'custom_templates_directory' setting documented here:
https://matrix-org.github.io/synapse/latest/templates.html
---------------------------------------------------------------------------------------"""
@attr.s(slots=True, frozen=True)
class EmailSubjectConfig:
@ -105,6 +116,9 @@ class EmailConfig(Config):
# A user-configurable template directory
template_dir = email_config.get("template_dir")
if template_dir is not None:
logger.warning(LEGACY_TEMPLATE_DIR_WARNING)
if isinstance(template_dir, str):
# We need an absolute path, because we change directory after starting (and
# we don't yet know what auxiliary templates like mail.css we will need).

View file

@ -11,12 +11,23 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from typing import Any, Dict, Optional
import attr
from ._base import Config
logger = logging.getLogger(__name__)
LEGACY_TEMPLATE_DIR_WARNING = """
This server's configuration file is using the deprecated 'template_dir' setting in the
'sso' section. Support for this setting has been deprecated and will be removed in a
future version of Synapse. Server admins should instead use the new
'custom_templates_directory' setting documented here:
https://matrix-org.github.io/synapse/latest/templates.html
---------------------------------------------------------------------------------------"""
@attr.s(frozen=True)
class SsoAttributeRequirement:
@ -43,6 +54,8 @@ class SSOConfig(Config):
# The sso-specific template_dir
self.sso_template_dir = sso_config.get("template_dir")
if self.sso_template_dir is not None:
logger.warning(LEGACY_TEMPLATE_DIR_WARNING)
# Read templates from disk
custom_template_directories = (

View file

@ -104,6 +104,8 @@ def prune_event_dict(room_version: RoomVersion, event_dict: dict) -> dict:
if event_type == EventTypes.Member:
add_fields("membership")
if room_version.msc3375_redaction_rules:
add_fields("join_authorised_via_users_server")
elif event_type == EventTypes.Create:
# MSC2176 rules state that create events cannot be redacted.
if room_version.msc2176_redaction_rules:

View file

@ -322,7 +322,7 @@ class PruneEventTestCase(unittest.TestCase):
},
)
# After MSC3083, alias events have no special behavior.
# After MSC3083, the allow key is protected from redaction.
self.run_test(
{
"type": "m.room.join_rules",
@ -344,6 +344,50 @@ class PruneEventTestCase(unittest.TestCase):
room_version=RoomVersions.V8,
)
def test_member(self):
"""Member events have changed behavior starting with MSC3375."""
self.run_test(
{
"type": "m.room.member",
"event_id": "$test:domain",
"content": {
"membership": "join",
"join_authorised_via_users_server": "@user:domain",
"other_key": "stripped",
},
},
{
"type": "m.room.member",
"event_id": "$test:domain",
"content": {"membership": "join"},
"signatures": {},
"unsigned": {},
},
)
# After MSC3375, the join_authorised_via_users_server key is protected
# from redaction.
self.run_test(
{
"type": "m.room.member",
"content": {
"membership": "join",
"join_authorised_via_users_server": "@user:domain",
"other_key": "stripped",
},
},
{
"type": "m.room.member",
"content": {
"membership": "join",
"join_authorised_via_users_server": "@user:domain",
},
"signatures": {},
"unsigned": {},
},
room_version=RoomVersions.V9,
)
class SerializeEventTestCase(unittest.TestCase):
def serialize(self, ev, fields):