MirrorHub/synapse
0
0
Fork 1
mirror of https://mau.dev/maunium/synapse.git synced 2024-11-18 07:52:56 +01:00
Code Issues Projects Releases Wiki Activity

Synapse 1.58.0rc2 (2022-04-26)

Browse source 
==============================
 
 This release candidate fixes bugs related to Synapse 1.58.0rc1's logic for handling device list updates.
 
 Bugfixes
 --------
 
 - Fix a bug introduced in Synapse 1.58.0rc1 where the main process could consume excessive amounts of CPU and memory while handling sentry logging failures. ([\#12554](https://github.com/matrix-org/synapse/issues/12554))
 - Fix a bug introduced in Synapse 1.58.0rc1 where opentracing contexts were not correctly sent to whitelisted remote servers with device lists updates. ([\#12555](https://github.com/matrix-org/synapse/issues/12555))
 
 Internal Changes
 ----------------
 
 - Reduce unnecessary work when handling remote device list updates. ([\#12557](https://github.com/matrix-org/synapse/issues/12557))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmJoINsACgkQkD7OEIo5
 3t0ETA//VANpmz0ykLeBA0XRW0eifQkHnQyBoZQtXL+IUmV9FZcX2q/akuQ5toJL
 p+AAJ6bKfYSwxyxgw3bJhftOVsIpG546auqVHvRXznlHk9vXKnuYevOz1ifnYSih
 j58fYTWnYZYv04uADbTX4fnJjiPDhiRYCZF6RiB5u1HGl1gr+Q1kJI7CL289/d1V
 MgBsJfzjHw+M7Vl2doddINnG65oPzrntMysv+PAma+WFxjQvLQoiaui96kaDM/2W
 XdFQh4k3OeyhBHKVk3+KrV9zPEh16JuymKDWoAnaMeIBBPwtngZp8tkEr4rSW8gP
 fOGDOMfqu896WqyT20fVhWzrsi24UjCw2TxY1C2UdCOoNqVJvMMMZ/aB129+8S/5
 65FOI77Oqmn2KGQ5+75hun3a9UcsRD+9iitg2wajxlle5yk6ZFi095aZVf94ls8d
 uSWakqvFr6PAY8TnvthQNsAj00LJVqKKH3RrNw7pDOX7QWE1qbcbPHFrtCMHC5Bz
 Z1/LDqNCxhwruPEghmg+0ToRama5uYdeCMzqECZ8GmzhlouezMaFK6bFH1SGwNbo
 Idx60qwCXK3vHka3MVSqLg15gBGj1dQEJt8q/FVrYDemEFlNshPydJTbtbIOfj/R
 VEZa12gbtqFenPMQyDuzbFqgwirbeYQQHIRltQB3yDeggg8j4ng=
 =FAvF
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmJoJswACgkQkD7OEIo5
 3t3u0RAApTbw1tPIFXN9aQWVzBfaYRlw4zFKJlAl4JF6J7QT5qDny9NnhLTKqUPh
 bMj+IfcPN2nlUM6U6F9Fnu0HQXP9/hyVNLIauvqCnRa2GCb6F7fymSTJSsyUGSZW
 0i3FXaR+OaIOqhjphGYx6oaVtGpgg7cKsp8VhVMTZrt0/sjFXxDV6oeaKfe2OpIE
 6D4wYh13zS9CzzoNoMUkkqwUr2e+DbsCp5BkjbU8F2abRRpC14QSF4/qH0dfAQ4i
 A6Lj02Gzv/AcE5e6e3bRYQkt4L8+/di+aY3vbj4jsCqJYFTP4yvdc6iScFALyxXn
 QNGTZlctuNbR/v6WVQbiFtSyF1GnLos1+7jpEBpGRUk30H68zOwsi+qLZh9r98mB
 vQ0G2HA/klzdnIvq9DxWLL0PxQZKocdu5CT2ujV+6J7vkKAXmC/0rkDRWdi2qooU
 uRb3CbAHo4A6GnoqsrhY1i1JuUNbX7Nu2BZsLaiIdp2JTt6OzS/0zfUgCZzPw0u0
 1586SlmiSdtXNMOlgpHPVtD71CCq/48tbccsomI1hk5dmiqlZX4hqsHSQY97k422
 NV9LNpPo0K5l2Qmegera94Qz0FzCv4TCSDEAXn//lHCZ6eAVARk6Wym0UCOXToqw
 Xg1igJEkGOkIcwXj6GpG+r8yMdYD52xG6Ud8ZbYTX1glDfVWN4U=
 =YYmi
 -----END PGP SIGNATURE-----

Merge tag 'v1.58.0rc2' into develop

Synapse 1.58.0rc2 (2022-04-26)
==============================

This release candidate fixes bugs related to Synapse 1.58.0rc1's logic for handling device list updates.

Bugfixes
--------

- Fix a bug introduced in Synapse 1.58.0rc1 where the main process could consume excessive amounts of CPU and memory while handling sentry logging failures. ([\#12554](https://github.com/matrix-org/synapse/issues/12554))
- Fix a bug introduced in Synapse 1.58.0rc1 where opentracing contexts were not correctly sent to whitelisted remote servers with device lists updates. ([\#12555](https://github.com/matrix-org/synapse/issues/12555))

Internal Changes
----------------

- Reduce unnecessary work when handling remote device list updates. ([\#12557](https://github.com/matrix-org/synapse/issues/12557))
This commit is contained in:
David Robertson 2022-04-26 18:07:15 +01:00
commit 9986621bc8
59 changed files with 144 additions and 172 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

90
CHANGES.md
View file

@ -1,3 +1,93 @@
Synapse 1.58.0rc2 (2022-04-26)
==============================
This release candidate fixes bugs related to Synapse 1.58.0rc1's logic for handling device list updates.
Bugfixes
--------
- Fix a bug introduced in Synapse 1.58.0rc1 where the main process could consume excessive amounts of CPU and memory while handling sentry logging failures. ([\#12554](https://github.com/matrix-org/synapse/issues/12554))
- Fix a bug introduced in Synapse 1.58.0rc1 where opentracing contexts were not correctly sent to whitelisted remote servers with device lists updates. ([\#12555](https://github.com/matrix-org/synapse/issues/12555))
Internal Changes
----------------
- Reduce unnecessary work when handling remote device list updates. ([\#12557](https://github.com/matrix-org/synapse/issues/12557))
Synapse 1.58.0rc1 (2022-04-26)
==============================
As of this release, the groups/communities feature in Synapse is now disabled by default. See [\#11584](https://github.com/matrix-org/synapse/issues/11584) for details. As mentioned in [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1580), this feature will be removed in Synapse 1.61.
Features
--------
- Implement [MSC3383](https://github.com/matrix-org/matrix-spec-proposals/pull/3383) for including the destination in server-to-server authentication headers. Contributed by @Bubu and @jcgruenhage for Famedly. ([\#11398](https://github.com/matrix-org/synapse/issues/11398))
- Docker images and Debian packages from matrix.org now contain a locked set of Python dependencies, greatly improving build reproducibility. ([Board](https://github.com/orgs/matrix-org/projects/54), [\#11537](https://github.com/matrix-org/synapse/issues/11537))
- Enable processing of device list updates asynchronously. ([\#12365](https://github.com/matrix-org/synapse/issues/12365), [\#12465](https://github.com/matrix-org/synapse/issues/12465))
- Implement [MSC2815](https://github.com/matrix-org/matrix-spec-proposals/pull/2815) to allow room moderators to view redacted event content. Contributed by @tulir @ Beeper. ([\#12427](https://github.com/matrix-org/synapse/issues/12427))
- Build Debian packages for Ubuntu 22.04 "Jammy Jellyfish". ([\#12543](https://github.com/matrix-org/synapse/issues/12543))
Bugfixes
--------
- Prevent a sync request from removing a user's busy presence status. ([\#12213](https://github.com/matrix-org/synapse/issues/12213))
- Fix bug with incremental sync missing events when rejoining/backfilling. Contributed by Nick @ Beeper. ([\#12319](https://github.com/matrix-org/synapse/issues/12319))
- Fix a long-standing bug which incorrectly caused `GET /_matrix/client/v3/rooms/{roomId}/event/{eventId}` to return edited events rather than the original. ([\#12476](https://github.com/matrix-org/synapse/issues/12476))
- Fix a bug introduced in Synapse 1.27.0 where the admin API for [deleting forward extremities](https://github.com/matrix-org/synapse/blob/erikj/fix_delete_event_response_count/docs/admin_api/rooms.md#deleting-forward-extremities) would always return a count of 1, no matter how many extremities were deleted. ([\#12496](https://github.com/matrix-org/synapse/issues/12496))
- Fix a long-standing bug where the image thumbnails embedded into email notifications were broken. ([\#12510](https://github.com/matrix-org/synapse/issues/12510))
- Fix a bug in the implementation of [MSC3202](https://github.com/matrix-org/matrix-spec-proposals/pull/3202) where Synapse would use the field name `device_unused_fallback_keys`, rather than `device_unused_fallback_key_types`. ([\#12520](https://github.com/matrix-org/synapse/issues/12520))
- Fix a bug introduced in Synapse 0.99.3 which could cause Synapse to consume large amounts of RAM when back-paginating in a large room. ([\#12522](https://github.com/matrix-org/synapse/issues/12522))
Improved Documentation
----------------------
- Fix rendering of the documentation site when using the 'print' feature. ([\#12340](https://github.com/matrix-org/synapse/issues/12340))
- Add a manual documenting config file options. ([\#12368](https://github.com/matrix-org/synapse/issues/12368), [\#12527](https://github.com/matrix-org/synapse/issues/12527))
- Update documentation to reflect that both the `run_background_tasks_on` option and the options for moving stream writers off of the main process are no longer experimental. ([\#12451](https://github.com/matrix-org/synapse/issues/12451))
- Update worker documentation and replace old `federation_reader` with `generic_worker`. ([\#12457](https://github.com/matrix-org/synapse/issues/12457))
- Strongly recommend [Poetry](https://python-poetry.org/) for development. ([\#12475](https://github.com/matrix-org/synapse/issues/12475))
- Add some example configurations for workers and update architectural diagram. ([\#12492](https://github.com/matrix-org/synapse/issues/12492))
- Fix a broken link in `README.rst`. ([\#12495](https://github.com/matrix-org/synapse/issues/12495))
- Add HAProxy delegation example with CORS headers to docs. ([\#12501](https://github.com/matrix-org/synapse/issues/12501))
- Remove extraneous comma in User Admin API's device deletion section so that the example JSON is actually valid and works. Contributed by @olmari. ([\#12533](https://github.com/matrix-org/synapse/issues/12533))
Deprecations and Removals
-------------------------
- The groups/communities feature in Synapse is now disabled by default. ([\#12344](https://github.com/matrix-org/synapse/issues/12344))
- Remove unstable identifiers from [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440). ([\#12382](https://github.com/matrix-org/synapse/issues/12382))
Internal Changes
----------------
- Preparation for faster-room-join work: start a background process to resynchronise the room state after a room join. ([\#12394](https://github.com/matrix-org/synapse/issues/12394))
- Preparation for faster-room-join work: Implement a tracking mechanism to allow functions to wait for full room state to arrive. ([\#12399](https://github.com/matrix-org/synapse/issues/12399))
- Remove an unstable identifier from [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083). ([\#12395](https://github.com/matrix-org/synapse/issues/12395))
- Run CI in the locked [Poetry](https://python-poetry.org/) environment, and remove corresponding `tox` jobs. ([\#12425](https://github.com/matrix-org/synapse/issues/12425), [\#12434](https://github.com/matrix-org/synapse/issues/12434), [\#12438](https://github.com/matrix-org/synapse/issues/12438), [\#12441](https://github.com/matrix-org/synapse/issues/12441), [\#12449](https://github.com/matrix-org/synapse/issues/12449), [\#12478](https://github.com/matrix-org/synapse/issues/12478), [\#12514](https://github.com/matrix-org/synapse/issues/12514), [\#12472](https://github.com/matrix-org/synapse/issues/12472))
- Change Mutual Rooms' `unstable_features` flag to `uk.half-shot.msc2666.mutual_rooms` which matches the current iteration of [MSC2666](https://github.com/matrix-org/matrix-spec-proposals/pull/2666). ([\#12445](https://github.com/matrix-org/synapse/issues/12445))
- Fix typo in the release script help string. ([\#12450](https://github.com/matrix-org/synapse/issues/12450))
- Fix a minor typo in the Debian changelogs generated by the release script. ([\#12497](https://github.com/matrix-org/synapse/issues/12497))
- Reintroduce the list of targets to the linter script, to avoid linting unwanted local-only directories during development. ([\#12455](https://github.com/matrix-org/synapse/issues/12455))
- Limit length of `device_id` to less than 512 characters. ([\#12454](https://github.com/matrix-org/synapse/issues/12454))
- Dockerfile-workers: reduce the amount we install in the image. ([\#12464](https://github.com/matrix-org/synapse/issues/12464))
- Dockerfile-workers: give the master its own log config. ([\#12466](https://github.com/matrix-org/synapse/issues/12466))
- complement-synapse-workers: factor out separate entry point script. ([\#12467](https://github.com/matrix-org/synapse/issues/12467))
- Back out experimental implementation of [MSC2314](https://github.com/matrix-org/matrix-spec-proposals/pull/2314). ([\#12474](https://github.com/matrix-org/synapse/issues/12474))
- Fix grammatical error in federation error response when the room version of a room is unknown. ([\#12483](https://github.com/matrix-org/synapse/issues/12483))
- Remove unnecessary configuration overrides in tests. ([\#12511](https://github.com/matrix-org/synapse/issues/12511))
- Refactor the relations code for clarity. ([\#12519](https://github.com/matrix-org/synapse/issues/12519))
- Add type hints so `docker` and `stubs` directories pass `mypy --disallow-untyped-defs`. ([\#12528](https://github.com/matrix-org/synapse/issues/12528))
- Update `delay_cancellation` to accept any awaitable, rather than just `Deferred`s. ([\#12468](https://github.com/matrix-org/synapse/issues/12468))
- Handle cancellation in `EventsWorkerStore._get_events_from_cache_or_db`. ([\#12529](https://github.com/matrix-org/synapse/issues/12529))
Synapse 1.57.1 (2022-04-20) Synapse 1.57.1 (2022-04-20)
=========================== ===========================

1
changelog.d/11398.feature
View file

@ -1 +0,0 @@
Implement [MSC3383](https://github.com/matrix-org/matrix-spec-proposals/pull/3383) for including the destination in server-to-server authentication headers. Contributed by @Bubu and @jcgruenhage for Famedly GmbH.

1
changelog.d/12213.bugfix
View file

@ -1 +0,0 @@
Prevent a sync request from removing a user's busy presence status.

1
changelog.d/12319.bugfix
View file

@ -1 +0,0 @@
Fix bug with incremental sync missing events when rejoining/backfilling. Contributed by Nick @ Beeper.

1
changelog.d/12337.feature
View file

@ -1 +0,0 @@
Use poetry to manage Synapse's dependencies.

1
changelog.d/12340.doc
View file

@ -1 +0,0 @@
Fix rendering of the documentation site when using the 'print' feature.

1
changelog.d/12344.removal
View file

@ -1 +0,0 @@
The groups/communities feature in Synapse has been disabled by default.

1
changelog.d/12365.feature
View file

@ -1 +0,0 @@
Enable processing of device list updates asynchronously.

1
changelog.d/12368.doc
View file

@ -1 +0,0 @@
Add a manual documenting config file options.

1
changelog.d/12382.removal
View file

@ -1 +0,0 @@
Remove unstable identifiers from [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440).

1
changelog.d/12394.misc
View file

@ -1 +0,0 @@
Preparation for faster-room-join work: start a background process to resynchronise the room state after a room join.

1
changelog.d/12395.misc
View file

@ -1 +0,0 @@
Remove an unstable identifier from [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083).

1
changelog.d/12399.misc
View file

@ -1 +0,0 @@
Preparation for faster-room-join work: Implement a tracking mechanism to allow functions to wait for full room state to arrive.

1
changelog.d/12425.misc
View file

@ -1 +0,0 @@
Run twisted trunk CI job in the locked poetry environment.

1
changelog.d/12427.feature
View file

@ -1 +0,0 @@
Implement [MSC2815](https://github.com/matrix-org/matrix-spec-proposals/pull/2815) to allow room moderators to view redacted event content. Contributed by @tulir.

1
changelog.d/12434.misc
View file

@ -1 +0,0 @@
Run lints under poetry in CI, and remove corresponding tox lint jobs.

1
changelog.d/12438.misc
View file

@ -1 +0,0 @@
Run "main" trial tests under `poetry`.

1
changelog.d/12441.misc
View file

@ -1 +0,0 @@
Bump twisted version in `poetry.lock` to work around [pip bug #9644](https://github.com/pypa/pip/issues/9644).

1
changelog.d/12445.misc
View file

@ -1 +0,0 @@
Change Mutual Rooms' `unstable_features` flag to `uk.half-shot.msc2666.mutual_rooms` which matches the current MSC iteration.

1
changelog.d/12449.misc
View file

@ -1 +0,0 @@
Use `poetry` to manage the virtualenv in debian packages.

1
changelog.d/12450.misc
View file

@ -1 +0,0 @@
Fix typo in the release script help string.

1
changelog.d/12451.doc
View file

@ -1 +0,0 @@
Update documentation to reflect that both the `run_background_tasks_on` option and the options for moving stream writers off of the main process are no longer experimental.

1
changelog.d/12454.misc
View file

@ -1 +0,0 @@
Limit length of device_id to less than 512 characters.

1
changelog.d/12455.misc
View file

@ -1 +0,0 @@
Reintroduce the list of targets to the linter script, to avoid linting unwanted local-only directories during development.

1
changelog.d/12457.doc
View file

@ -1 +0,0 @@
Update worker documentation and replace old `federation_reader` with `generic_worker`.

1
changelog.d/12464.misc
View file

@ -1 +0,0 @@
Dockerfile-workers: reduce the amount we install in the image.

1
changelog.d/12465.feature
View file

@ -1 +0,0 @@
Enable processing of device list updates asynchronously.

1
changelog.d/12466.misc
View file

@ -1 +0,0 @@
Dockerfile-workers: give the master its own log config.

1
changelog.d/12467.misc
View file

@ -1 +0,0 @@
complement-synapse-workers: factor out separate entry point script.

1
changelog.d/12468.misc
View file

@ -1 +0,0 @@
Update `delay_cancellation` to accept any awaitable, rather than just `Deferred`s.

1
changelog.d/12472.misc
View file

@ -1 +0,0 @@
Add a CI job which tests Synapse against the latest version of all dependencies.

1
changelog.d/12474.misc
View file

@ -1 +0,0 @@
Back out experimental implementation of [MSC2314](https://github.com/matrix-org/matrix-spec-proposals/pull/2314).

1
changelog.d/12475.doc
View file

@ -1 +0,0 @@
Strongly recommend `poetry` for development.

1
changelog.d/12476.bugfix
View file

@ -1 +0,0 @@
Fix a long-standing bug which incorrectly caused `GET /_matrix/client/r3/rooms/{roomId}/event/{eventId}` to return edited events rather than the original.

1
changelog.d/12478.misc
View file

@ -1 +0,0 @@
Use poetry-core instead of setuptools to build wheels.

1
changelog.d/12483.misc
View file

@ -1 +0,0 @@
Fix grammatical error in federation error response when the room version of a room is unknown.

1
changelog.d/12492.doc
View file

@ -1 +0,0 @@
Add some example configurations for workers and update architectural diagram.

1
changelog.d/12495.doc
View file

@ -1 +0,0 @@
Fix a broken link in `README.rst`.

1
changelog.d/12496.bugfix
View file

@ -1 +0,0 @@
Fix bug where the admin API for [deleting forward extremities](https://github.com/matrix-org/synapse/blob/erikj/fix_delete_event_response_count/docs/admin_api/rooms.md#deleting-forward-extremities) would always return a count of 1 no matter how many extremities were deleted. Broke in v1.27.0.

1
changelog.d/12497.misc
View file

@ -1 +0,0 @@
Fix a minor typo in the Debian changelogs generated by the release script.

1
changelog.d/12501.doc
View file

@ -1 +0,0 @@
Add HAProxy delegation example with CORS headers to docs.

1
changelog.d/12510.bugfix
View file

@ -1 +0,0 @@
Fix a long-standing bug where the image thumbanils embedded into email notifications were broken.

1
changelog.d/12511.misc
View file

@ -1 +0,0 @@
Remove unnecessary configuration overrides in tests.

1
changelog.d/12514.misc
View file

@ -1 +0,0 @@
Use poetry-core instead of setuptools to build wheels.

1
changelog.d/12519.misc
View file

@ -1 +0,0 @@
Refactor the relations code for clarity.

1
changelog.d/12520.bugfix
View file

@ -1 +0,0 @@
Fix a bug in the implementation of MSC3202 where Synapse would use the field name `device_unused_fallback_keys`, rather than `device_unused_fallback_key_types`.

1
changelog.d/12522.bugfix
View file

@ -1 +0,0 @@
Fix a bug introduced in Synapse 0.99.3 which could cause Synapse to consume large amounts of RAM when back-paginating in a large room.

2
changelog.d/12527.doc
View file

@ -1,2 +0,0 @@
Add an index to the configuration manual.

1
changelog.d/12528.misc
View file

@ -1 +0,0 @@
Add type hints so `docker` and `stubs` directories pass `mypy --disallow-untyped-defs`.

1
changelog.d/12529.misc
View file

@ -1 +0,0 @@
Handle cancellation in `EventsWorkerStore._get_events_from_cache_or_db`.

1
changelog.d/12533.doc
View file

@ -1 +0,0 @@
Remove extraneous comma in User Admin API's device deletion section so that the example JSON is actually valid and works. Contributed by @olmari.

1
changelog.d/12543.misc
View file

@ -1 +0,0 @@
Build debian packages for Ubuntu 22.04 "Jammy Jellyfish".

11
debian/changelog vendored
View file

@ -1,8 +1,15 @@
matrix-synapse-py3 (1.58.0+nmu1) UNRELEASED; urgency=medium matrix-synapse-py3 (1.58.0~rc2) stable; urgency=medium
* New Synapse release 1.58.0rc2.
-- Synapse Packaging team <packages@matrix.org> Tue, 26 Apr 2022 17:14:56 +0100
matrix-synapse-py3 (1.58.0~rc1) stable; urgency=medium
* Use poetry to manage the bundled virtualenv included with this package. * Use poetry to manage the bundled virtualenv included with this package.
* New Synapse release 1.58.0rc1.
-- Synapse Packaging team <packages@matrix.org> Wed, 30 Mar 2022 12:21:43 +0100 -- Synapse Packaging team <packages@matrix.org> Tue, 26 Apr 2022 11:15:20 +0100
matrix-synapse-py3 (1.57.1) stable; urgency=medium matrix-synapse-py3 (1.57.1) stable; urgency=medium

62
poetry.lock generated
View file

@ -1,11 +1,3 @@
[[package]]
name = "appdirs"
version = "1.4.4"
description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"."
category = "dev"
optional = false
python-versions = "*"
[[package]] [[package]]
name = "attrs" name = "attrs"
version = "21.4.0" version = "21.4.0"
@ -49,17 +41,6 @@ six = "*"
[package.extras] [package.extras]
visualize = ["graphviz (>0.5.1)", "Twisted (>=16.1.1)"] visualize = ["graphviz (>0.5.1)", "Twisted (>=16.1.1)"]
[[package]]
name = "baron"
version = "0.10.1"
description = "Full Syntax Tree for python to make writing refactoring code a realist task"
category = "dev"
optional = false
python-versions = "*"
[package.dependencies]
rply = "*"
[[package]] [[package]]
name = "bcrypt" name = "bcrypt"
version = "3.2.0" version = "3.2.0"
@ -984,20 +965,6 @@ Pygments = ">=2.5.1"
[package.extras] [package.extras]
md = ["cmarkgfm (>=0.8.0)"] md = ["cmarkgfm (>=0.8.0)"]
[[package]]
name = "redbaron"
version = "0.9.2"
description = "Abstraction on top of baron, a FST for python to make writing refactoring code a realistic task"
category = "dev"
optional = false
python-versions = "*"
[package.dependencies]
baron = ">=0.7"
[package.extras]
notebook = ["pygments"]
[[package]] [[package]]
name = "requests" name = "requests"
version = "2.27.1" version = "2.27.1"
@ -1038,17 +1005,6 @@ python-versions = ">=3.7"
[package.extras] [package.extras]
idna2008 = ["idna"] idna2008 = ["idna"]
[[package]]
name = "rply"
version = "0.7.8"
description = "A pure Python Lex/Yacc that works with RPython"
category = "dev"
optional = false
python-versions = "*"
[package.dependencies]
appdirs = "*"
[[package]] [[package]]
name = "secretstorage" name = "secretstorage"
version = "3.3.1" version = "3.3.1"
@ -1597,13 +1553,9 @@ url_preview = ["lxml"]
[metadata] [metadata]
lock-version = "1.1" lock-version = "1.1"
python-versions = "^3.7" python-versions = "^3.7"
content-hash = "964ad29eaf7fd02749a4e735818f3bc0ba729c2f4b9e3213f0daa02643508b16" content-hash = "f482a4f594a165dfe01ce253a22510d5faf38647ab0dcebc35789350cafd9bf0"
[metadata.files] [metadata.files]
appdirs = [
{file = "appdirs-1.4.4-py2.py3-none-any.whl", hash = "sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128"},
{file = "appdirs-1.4.4.tar.gz", hash = "sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41"},
]
attrs = [ attrs = [
{file = "attrs-21.4.0-py2.py3-none-any.whl", hash = "sha256:2d27e3784d7a565d36ab851fe94887c5eccd6a463168875832a1be79c82828b4"}, {file = "attrs-21.4.0-py2.py3-none-any.whl", hash = "sha256:2d27e3784d7a565d36ab851fe94887c5eccd6a463168875832a1be79c82828b4"},
{file = "attrs-21.4.0.tar.gz", hash = "sha256:626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd"}, {file = "attrs-21.4.0.tar.gz", hash = "sha256:626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd"},
@ -1616,10 +1568,6 @@ automat = [
{file = "Automat-20.2.0-py2.py3-none-any.whl", hash = "sha256:b6feb6455337df834f6c9962d6ccf771515b7d939bca142b29c20c2376bc6111"}, {file = "Automat-20.2.0-py2.py3-none-any.whl", hash = "sha256:b6feb6455337df834f6c9962d6ccf771515b7d939bca142b29c20c2376bc6111"},
{file = "Automat-20.2.0.tar.gz", hash = "sha256:7979803c74610e11ef0c0d68a2942b152df52da55336e0c9d58daf1831cbdf33"}, {file = "Automat-20.2.0.tar.gz", hash = "sha256:7979803c74610e11ef0c0d68a2942b152df52da55336e0c9d58daf1831cbdf33"},
] ]
baron = [
{file = "baron-0.10.1-py2.py3-none-any.whl", hash = "sha256:befb33f4b9e832c7cd1e3cf0eafa6dd3cb6ed4cb2544245147c019936f4e0a8a"},
{file = "baron-0.10.1.tar.gz", hash = "sha256:af822ad44d4eb425c8516df4239ac4fdba9fdb398ef77e4924cd7c9b4045bc2f"},
]
bcrypt = [ bcrypt = [
{file = "bcrypt-3.2.0-cp36-abi3-macosx_10_10_universal2.whl", hash = "sha256:b589229207630484aefe5899122fb938a5b017b0f4349f769b8c13e78d99a8fd"}, {file = "bcrypt-3.2.0-cp36-abi3-macosx_10_10_universal2.whl", hash = "sha256:b589229207630484aefe5899122fb938a5b017b0f4349f769b8c13e78d99a8fd"},
{file = "bcrypt-3.2.0-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:c95d4cbebffafcdd28bd28bb4e25b31c50f6da605c81ffd9ad8a3d1b2ab7b1b6"}, {file = "bcrypt-3.2.0-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:c95d4cbebffafcdd28bd28bb4e25b31c50f6da605c81ffd9ad8a3d1b2ab7b1b6"},
@ -2412,10 +2360,6 @@ readme-renderer = [
{file = "readme_renderer-33.0-py3-none-any.whl", hash = "sha256:f02cee0c4de9636b5a62b6be50c9742427ba1b956aad1d938bfb087d0d72ccdf"}, {file = "readme_renderer-33.0-py3-none-any.whl", hash = "sha256:f02cee0c4de9636b5a62b6be50c9742427ba1b956aad1d938bfb087d0d72ccdf"},
{file = "readme_renderer-33.0.tar.gz", hash = "sha256:e3b53bc84bd6af054e4cc1fe3567dc1ae19f554134221043a3f8c674e22209db"}, {file = "readme_renderer-33.0.tar.gz", hash = "sha256:e3b53bc84bd6af054e4cc1fe3567dc1ae19f554134221043a3f8c674e22209db"},
] ]
redbaron = [
{file = "redbaron-0.9.2-py2.py3-none-any.whl", hash = "sha256:d01032b6a848b5521a8d6ef72486315c2880f420956870cdd742e2b5a09b9bab"},
{file = "redbaron-0.9.2.tar.gz", hash = "sha256:472d0739ca6b2240bb2278ae428604a75472c9c12e86c6321e8c016139c0132f"},
]
requests = [ requests = [
{file = "requests-2.27.1-py2.py3-none-any.whl", hash = "sha256:f22fa1e554c9ddfd16e6e41ac79759e17be9e492b3587efa038054674760e72d"}, {file = "requests-2.27.1-py2.py3-none-any.whl", hash = "sha256:f22fa1e554c9ddfd16e6e41ac79759e17be9e492b3587efa038054674760e72d"},
{file = "requests-2.27.1.tar.gz", hash = "sha256:68d7c56fd5a8999887728ef304a6d12edc7be74f1cfa47714fc8b414525c9a61"}, {file = "requests-2.27.1.tar.gz", hash = "sha256:68d7c56fd5a8999887728ef304a6d12edc7be74f1cfa47714fc8b414525c9a61"},
@ -2428,10 +2372,6 @@ rfc3986 = [
{file = "rfc3986-2.0.0-py2.py3-none-any.whl", hash = "sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd"}, {file = "rfc3986-2.0.0-py2.py3-none-any.whl", hash = "sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd"},
{file = "rfc3986-2.0.0.tar.gz", hash = "sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c"}, {file = "rfc3986-2.0.0.tar.gz", hash = "sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c"},
] ]
rply = [
{file = "rply-0.7.8-py2.py3-none-any.whl", hash = "sha256:28ffd11d656c48aeb8c508eb382acd6a0bd906662624b34388751732a27807e7"},
{file = "rply-0.7.8.tar.gz", hash = "sha256:2a808ac25a4580a9991fc304d64434e299a8fc75760574492f242cbb5bb301c9"},
]
secretstorage = [ secretstorage = [
{file = "SecretStorage-3.3.1-py3-none-any.whl", hash = "sha256:422d82c36172d88d6a0ed5afdec956514b189ddbfb72fefab0c8a1cee4eaf71f"}, {file = "SecretStorage-3.3.1-py3-none-any.whl", hash = "sha256:422d82c36172d88d6a0ed5afdec956514b189ddbfb72fefab0c8a1cee4eaf71f"},
{file = "SecretStorage-3.3.1.tar.gz", hash = "sha256:fd666c51a6bf200643495a04abb261f83229dcb6fd8472ec393df7ffc8b6f195"}, {file = "SecretStorage-3.3.1.tar.gz", hash = "sha256:fd666c51a6bf200643495a04abb261f83229dcb6fd8472ec393df7ffc8b6f195"},

3
pyproject.toml
View file

@ -54,7 +54,7 @@ skip_gitignore = true
[tool.poetry] [tool.poetry]
name = "matrix-synapse" name = "matrix-synapse"
version = "1.57.1" version = "1.58.0rc2"
description = "Homeserver for the Matrix decentralised comms protocol" description = "Homeserver for the Matrix decentralised comms protocol"
authors = ["Matrix.org Team and Contributors <packages@matrix.org>"] authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
license = "Apache-2.0" license = "Apache-2.0"
@ -270,7 +270,6 @@ idna = ">=2.5"
# The following are used by the release script # The following are used by the release script
click = "==8.1.0" click = "==8.1.0"
redbaron = "==0.9.2"
GitPython = "==3.1.14" GitPython = "==3.1.14"
commonmark = "==0.9.1" commonmark = "==0.9.1"
pygithub = "==1.55" pygithub = "==1.55"

67
scripts-dev/release.py
View file

@ -25,13 +25,12 @@ import sys
import urllib.request import urllib.request
from os import path from os import path
from tempfile import TemporaryDirectory from tempfile import TemporaryDirectory
from typing import List, Optional, Tuple from typing import List, Optional
import attr import attr
import click import click
import commonmark import commonmark
import git import git
import redbaron
from click.exceptions import ClickException from click.exceptions import ClickException
from github import Github from github import Github
from packaging import version from packaging import version
@ -100,7 +99,7 @@ def prepare():
repo.remote().fetch() repo.remote().fetch()
# Get the current version and AST from root Synapse module. # Get the current version and AST from root Synapse module.
current_version, parsed_synapse_ast, version_node = parse_version_from_module() current_version = get_package_version()
# Figure out what sort of release we're doing and calcuate the new version. # Figure out what sort of release we're doing and calcuate the new version.
rc = click.confirm("RC", default=True) rc = click.confirm("RC", default=True)
@ -162,7 +161,7 @@ def prepare():
click.get_current_context().abort() click.get_current_context().abort()
# Switch to the release branch. # Switch to the release branch.
parsed_new_version = version.parse(new_version) parsed_new_version: version.Version = version.parse(new_version)
# We assume for debian changelogs that we only do RCs or full releases. # We assume for debian changelogs that we only do RCs or full releases.
assert not parsed_new_version.is_devrelease assert not parsed_new_version.is_devrelease
@ -207,17 +206,15 @@ def prepare():
# Create the new release branch # Create the new release branch
release_branch = repo.create_head(release_branch_name, commit=base_branch) release_branch = repo.create_head(release_branch_name, commit=base_branch)
# Switch to the release branch and ensure its up to date. # Switch to the release branch and ensure it's up to date.
repo.git.checkout(release_branch_name) repo.git.checkout(release_branch_name)
update_branch(repo) update_branch(repo)
# Update the `__version__` variable and write it back to the file. # Update the version specified in pyproject.toml.
version_node.value = '"' + new_version + '"' subprocess.check_output(["poetry", "version", new_version])
with open("synapse/__init__.py", "w") as f:
f.write(parsed_synapse_ast.dumps())
# Generate changelogs. # Generate changelogs.
generate_and_write_changelog(current_version) generate_and_write_changelog(current_version, new_version)
# Generate debian changelogs # Generate debian changelogs
if parsed_new_version.pre is not None: if parsed_new_version.pre is not None:
@ -284,7 +281,7 @@ def tag(gh_token: Optional[str]):
repo.remote().fetch() repo.remote().fetch()
# Find out the version and tag name. # Find out the version and tag name.
current_version, _, _ = parse_version_from_module() current_version = get_package_version()
tag_name = f"v{current_version}" tag_name = f"v{current_version}"
# Check we haven't released this version. # Check we haven't released this version.
@ -362,7 +359,7 @@ def publish(gh_token: str):
if repo.is_dirty(): if repo.is_dirty():
raise click.ClickException("Uncommitted changes exist.") raise click.ClickException("Uncommitted changes exist.")
current_version, _, _ = parse_version_from_module() current_version = get_package_version()
tag_name = f"v{current_version}" tag_name = f"v{current_version}"
if not click.confirm(f"Publish {tag_name}?", default=True): if not click.confirm(f"Publish {tag_name}?", default=True):
@ -396,7 +393,7 @@ def publish(gh_token: str):
def upload(): def upload():
"""Upload release to pypi.""" """Upload release to pypi."""
current_version, _, _ = parse_version_from_module() current_version = get_package_version()
tag_name = f"v{current_version}" tag_name = f"v{current_version}"
pypi_asset_names = [ pypi_asset_names = [
@ -424,7 +421,7 @@ def upload():
def announce(): def announce():
"""Generate markdown to announce the release.""" """Generate markdown to announce the release."""
current_version, _, _ = parse_version_from_module() current_version = get_package_version()
tag_name = f"v{current_version}" tag_name = f"v{current_version}"
click.echo( click.echo(
@ -455,37 +452,11 @@ Announce the release in
) )
def parse_version_from_module() -> Tuple[ def get_package_version() -> version.Version:
version.Version, redbaron.RedBaron, redbaron.Node version_string = subprocess.check_output(["poetry", "version", "--short"]).decode(
]: "utf-8"
# Parse the AST and load the `__version__` node so that we can edit it )
# later. return version.Version(version_string)
with open("synapse/__init__.py") as f:
red = redbaron.RedBaron(f.read())
version_node = None
for node in red:
if node.type != "assignment":
continue
if node.target.type != "name":
continue
if node.target.value != "__version__":
continue
version_node = node
break
if not version_node:
print("Failed to find '__version__' definition in synapse/__init__.py")
sys.exit(1)
# Parse the current version.
current_version = version.parse(version_node.value.value.strip('"'))
assert isinstance(current_version, version.Version)
return current_version, red, version_node
def find_ref(repo: git.Repo, ref_name: str) -> Optional[git.HEAD]: def find_ref(repo: git.Repo, ref_name: str) -> Optional[git.HEAD]:
@ -565,11 +536,13 @@ def get_changes_for_version(wanted_version: version.Version) -> str:
return "\n".join(version_changelog) return "\n".join(version_changelog)
def generate_and_write_changelog(current_version: version.Version): def generate_and_write_changelog(current_version: version.Version, new_version: str):
# We do this by getting a draft so that we can edit it before writing to the # We do this by getting a draft so that we can edit it before writing to the
# changelog. # changelog.
result = run_until_successful( result = run_until_successful(
"python3 -m towncrier --draft", shell=True, capture_output=True f"python3 -m towncrier build --draft --version {new_version}",
shell=True,
capture_output=True,
) )
new_changes = result.stdout.decode("utf-8") new_changes = result.stdout.decode("utf-8")
new_changes = new_changes.replace( new_changes = new_changes.replace(

12
synapse/handlers/device.py
View file

@ -505,7 +505,8 @@ class DeviceHandler(DeviceWorkerHandler):
"device_list_key", position, users={user_id}, rooms=room_ids "device_list_key", position, users={user_id}, rooms=room_ids
) )
# We may need to do some processing asynchronously. # We may need to do some processing asynchronously for local user IDs.
if self.hs.is_mine_id(user_id):
self._handle_new_device_update_async() self._handle_new_device_update_async()
async def notify_user_signature_update( async def notify_user_signature_update(
@ -683,9 +684,12 @@ class DeviceHandler(DeviceWorkerHandler):
self.federation_sender.send_device_messages( self.federation_sender.send_device_messages(
host, immediate=False host, immediate=False
) )
log_kv( # TODO: when called, this isn't in a logging context.
{"message": "sent device update to host", "host": host} # This leads to log spam, sentry event spam, and massive
) # memory usage. See #12552.
# log_kv(
# {"message": "sent device update to host", "host": host}
# )
if current_stream_id != stream_id: if current_stream_id != stream_id:
# Clear the set of hosts we've already sent to as we're # Clear the set of hosts we've already sent to as we're

15
synapse/storage/databases/main/devices.py
View file

@ -1748,7 +1748,8 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore):
device_id, device_id,
room_id, room_id,
stream_id, stream_id,
False, # We only need to calculate outbound pokes for local users
not self.hs.is_mine_id(user_id),
encoded_context, encoded_context,
) )
for room_id in room_ids for room_id in room_ids
@ -1776,7 +1777,17 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore):
def get_uncoverted_outbound_room_pokes_txn(txn): def get_uncoverted_outbound_room_pokes_txn(txn):
txn.execute(sql, (limit,)) txn.execute(sql, (limit,))
return txn.fetchall()
return [
(
user_id,
device_id,
room_id,
stream_id,
db_to_json(opentracing_context),
)
for user_id, device_id, room_id, stream_id, opentracing_context in txn
]
return await self.db_pool.runInteraction( return await self.db_pool.runInteraction(
"get_uncoverted_outbound_room_pokes", get_uncoverted_outbound_room_pokes_txn "get_uncoverted_outbound_room_pokes", get_uncoverted_outbound_room_pokes_txn

2
tests/storage/test_devices.py
View file

@ -29,7 +29,7 @@ class DeviceStoreTestCase(HomeserverTestCase):
for device_id in device_ids: for device_id in device_ids:
stream_id = self.get_success( stream_id = self.get_success(
self.store.add_device_change_to_streams( self.store.add_device_change_to_streams(
"user_id", [device_id], ["!some:room"] user_id, [device_id], ["!some:room"]
) )
) )