Merge branch 'release-v1.33.0' of github.com:matrix-org/synapse into matrix-org-hotfixes
This commit is contained in:
Andrew Morgan
commit
696e7df2e8
53
CHANGES.md
53
CHANGES.md
|
|
@ -1,3 +1,56 @@
|
|||
Synapse 1.33.0rc1 (2021-04-28)
|
||||
==============================
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Update experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership. ([\#9800](https://github.com/matrix-org/synapse/issues/9800), [\#9814](https://github.com/matrix-org/synapse/issues/9814))
|
||||
- Add experimental support for handling presence on a worker. ([\#9819](https://github.com/matrix-org/synapse/issues/9819), [\#9820](https://github.com/matrix-org/synapse/issues/9820), [\#9828](https://github.com/matrix-org/synapse/issues/9828), [\#9850](https://github.com/matrix-org/synapse/issues/9850))
|
||||
- Return a new template when an user attempts to renew their account multiple times with the same token, stating that their account is set to expire. This replaces the invalid token template that would previously be shown in this case. This change concerns the optional account validity feature. ([\#9832](https://github.com/matrix-org/synapse/issues/9832))
|
||||
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fixes the OIDC SSO flow when using a `public_baseurl` value including a non-root URL path. ([\#9726](https://github.com/matrix-org/synapse/issues/9726))
|
||||
- Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg. ([\#9788](https://github.com/matrix-org/synapse/issues/9788))
|
||||
- Add some sanity checks to identity server passed to 3PID bind/unbind endpoints. ([\#9802](https://github.com/matrix-org/synapse/issues/9802))
|
||||
- Limit the size of HTTP responses read over federation. ([\#9833](https://github.com/matrix-org/synapse/issues/9833))
|
||||
- Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists. ([\#9867](https://github.com/matrix-org/synapse/issues/9867))
|
||||
- Fix a long-standing bug where errors from federation did not propagate to the client. ([\#9868](https://github.com/matrix-org/synapse/issues/9868))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Add a note to the docker docs mentioning that we mirror upstream's supported Docker platforms. ([\#9801](https://github.com/matrix-org/synapse/issues/9801))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Add a dockerfile for running Synapse in worker-mode under Complement. ([\#9162](https://github.com/matrix-org/synapse/issues/9162))
|
||||
- Apply `pyupgrade` across the codebase. ([\#9786](https://github.com/matrix-org/synapse/issues/9786))
|
||||
- Move some replication processing out of `generic_worker`. ([\#9796](https://github.com/matrix-org/synapse/issues/9796))
|
||||
- Replace `HomeServer.get_config()` with inline references. ([\#9815](https://github.com/matrix-org/synapse/issues/9815))
|
||||
- Rename some handlers and config modules to not duplicate the top-level module. ([\#9816](https://github.com/matrix-org/synapse/issues/9816))
|
||||
- Fix a long-standing bug which caused `max_upload_size` to not be correctly enforced. ([\#9817](https://github.com/matrix-org/synapse/issues/9817))
|
||||
- Reduce CPU usage of the user directory by reusing existing calculated room membership. ([\#9821](https://github.com/matrix-org/synapse/issues/9821))
|
||||
- Small speed up for joining large remote rooms. ([\#9825](https://github.com/matrix-org/synapse/issues/9825))
|
||||
- Introduce flake8-bugbear to the test suite and fix some of its lint violations. ([\#9838](https://github.com/matrix-org/synapse/issues/9838))
|
||||
- Only store the raw data in the in-memory caches, rather than objects that include references to e.g. the data stores. ([\#9845](https://github.com/matrix-org/synapse/issues/9845))
|
||||
- Limit length of accepted email addresses. ([\#9855](https://github.com/matrix-org/synapse/issues/9855))
|
||||
- Remove redundant `synapse.types.Collection` type definition. ([\#9856](https://github.com/matrix-org/synapse/issues/9856))
|
||||
- Handle recently added rate limits correctly when using `--no-rate-limit` with the demo scripts. ([\#9858](https://github.com/matrix-org/synapse/issues/9858))
|
||||
- Disable invite rate-limiting by default when running the unit tests. ([\#9871](https://github.com/matrix-org/synapse/issues/9871))
|
||||
- Pass a reactor into `SynapseSite` to make testing easier. ([\#9874](https://github.com/matrix-org/synapse/issues/9874))
|
||||
- Make `DomainSpecificString` an `attrs` class. ([\#9875](https://github.com/matrix-org/synapse/issues/9875))
|
||||
- Add type hints to `synapse.api.auth` and `synapse.api.auth_blocking` modules. ([\#9876](https://github.com/matrix-org/synapse/issues/9876))
|
||||
- Remove redundant `_PushHTTPChannel` test class. ([\#9878](https://github.com/matrix-org/synapse/issues/9878))
|
||||
- Remove backwards-compatibility code for Python versions < 3.6. ([\#9879](https://github.com/matrix-org/synapse/issues/9879))
|
||||
- Small performance improvement around handling new local presence updates. ([\#9887](https://github.com/matrix-org/synapse/issues/9887))
|
||||
|
||||
|
||||
Synapse 1.32.2 (2021-04-22)
|
||||
===========================
|
||||
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
Add a dockerfile for running Synapse in worker-mode under Complement.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fixes the OIDC SSO flow when using a `public_baseurl` value including a non-root URL path.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Apply `pyupgrade` across the codebase.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Move some replication processing out of `generic_worker`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add a note to the docker docs mentioning that we mirror upstream's supported Docker platforms.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add some sanity checks to identity server passed to 3PID bind/unbind endpoints.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Replace `HomeServer.get_config()` with inline references.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Rename some handlers and config modules to not duplicate the top-level module.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a long-standing bug which caused `max_upload_size` to not be correctly enforced.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add experimental support for handling presence on a worker.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add experimental support for handling presence on a worker.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reduce CPU usage of the user directory by reusing existing calculated room membership.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Small speed up for joining large remote rooms.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add experimental support for handling presence on a worker.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Don't return an error when a user attempts to renew their account multiple times with the same token. Instead, state when their account is set to expire. This change concerns the optional account validity feature.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Limit the size of HTTP responses read over federation.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Introduce flake8-bugbear to the test suite and fix some of its lint violations.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Only store the raw data in the in-memory caches, rather than objects that include references to e.g. the data stores.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add experimental support for handling presence on a worker.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Limit length of accepted email addresses.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove redundant `synapse.types.Collection` type definition.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Handle recently added rate limits correctly when using `--no-rate-limit` with the demo scripts.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a long-standing bug where errors from federation did not propagate to the client.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Disable invite rate-limiting by default when running the unit tests.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Pass a reactor into `SynapseSite` to make testing easier.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Make `DomainSpecificString` an `attrs` class.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add type hints to `synapse.api.auth` and `synapse.api.auth_blocking` modules.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove redundant `_PushHTTPChannel` test class.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove backwards-compatibility code for Python versions < 3.6.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Small performance improvement around handling new local presence updates.
|
||||
1
changelog.d/9900.bugfix
Normal file
1
changelog.d/9900.bugfix
Normal file
|
|
@ -0,0 +1 @@
|
|||
Fix tight loop handling presence replication when using workers. Introduced in v1.33.0rc1.
|
||||
|
|
@ -47,7 +47,7 @@ try:
|
|||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "1.32.2"
|
||||
__version__ = "1.33.0rc1"
|
||||
|
||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||
# We import here so that we don't have to install a bunch of deps when
|
||||
|
|
|
|||
|
|
@ -14,16 +14,7 @@
|
|||
|
||||
import abc
|
||||
import logging
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Dict,
|
||||
Hashable,
|
||||
Iterable,
|
||||
List,
|
||||
Optional,
|
||||
Set,
|
||||
Tuple,
|
||||
)
|
||||
from typing import TYPE_CHECKING, Dict, Hashable, Iterable, List, Optional, Set, Tuple
|
||||
|
||||
from prometheus_client import Counter
|
||||
|
||||
|
|
@ -35,10 +26,7 @@ from synapse.events import EventBase
|
|||
from synapse.federation.sender.per_destination_queue import PerDestinationQueue
|
||||
from synapse.federation.sender.transaction_manager import TransactionManager
|
||||
from synapse.federation.units import Edu
|
||||
from synapse.logging.context import (
|
||||
make_deferred_yieldable,
|
||||
run_in_background,
|
||||
)
|
||||
from synapse.logging.context import make_deferred_yieldable, run_in_background
|
||||
from synapse.metrics import (
|
||||
LaterGauge,
|
||||
event_processing_loop_counter,
|
||||
|
|
|
|||
|
|
@ -2026,18 +2026,40 @@ class PresenceFederationQueue:
|
|||
)
|
||||
return result["updates"], result["upto_token"], result["limited"]
|
||||
|
||||
# If the from_token is the current token then there's nothing to return
|
||||
# and we can trivially no-op.
|
||||
if from_token == self._next_id - 1:
|
||||
return [], upto_token, False
|
||||
|
||||
# We can find the correct position in the queue by noting that there is
|
||||
# exactly one entry per stream ID, and that the last entry has an ID of
|
||||
# `self._next_id - 1`, so we can count backwards from the end.
|
||||
#
|
||||
# Since we are returning all states in the range `from_token < stream_id
|
||||
# <= upto_token` we look for the index with a `stream_id` of `from_token
|
||||
# + 1`.
|
||||
#
|
||||
# Since the start of the queue is periodically truncated we need to
|
||||
# handle the case where `from_token` stream ID has already been dropped.
|
||||
start_idx = max(from_token - self._next_id, -len(self._queue))
|
||||
start_idx = max(from_token + 1 - self._next_id, -len(self._queue))
|
||||
|
||||
to_send = [] # type: List[Tuple[int, Tuple[str, str]]]
|
||||
limited = False
|
||||
new_id = upto_token
|
||||
for _, stream_id, destinations, user_ids in self._queue[start_idx:]:
|
||||
if stream_id <= from_token:
|
||||
# Paranoia check that we are actually only sending states that
|
||||
# are have stream_id strictly greater than from_token. We should
|
||||
# never hit this.
|
||||
logger.warning(
|
||||
"Tried returning presence federation stream ID: %d less than from_token: %d (next_id: %d, len: %d)",
|
||||
stream_id,
|
||||
from_token,
|
||||
self._next_id,
|
||||
len(self._queue),
|
||||
)
|
||||
continue
|
||||
|
||||
if stream_id > upto_token:
|
||||
break
|
||||
|
||||
|
|
|
|||
|
|
@ -509,6 +509,14 @@ class PresenceFederationQueueTestCase(unittest.HomeserverTestCase):
|
|||
|
||||
self.assertCountEqual(rows, expected_rows)
|
||||
|
||||
now_token = self.queue.get_current_token(self.instance_name)
|
||||
rows, upto_token, limited = self.get_success(
|
||||
self.queue.get_replication_rows("master", upto_token, now_token, 10)
|
||||
)
|
||||
self.assertEqual(upto_token, now_token)
|
||||
self.assertFalse(limited)
|
||||
self.assertCountEqual(rows, [])
|
||||
|
||||
def test_send_and_get_split(self):
|
||||
state1 = UserPresenceState.default("@user1:test")
|
||||
state2 = UserPresenceState.default("@user2:test")
|
||||
|
|
@ -538,6 +546,20 @@ class PresenceFederationQueueTestCase(unittest.HomeserverTestCase):
|
|||
|
||||
self.assertCountEqual(rows, expected_rows)
|
||||
|
||||
now_token = self.queue.get_current_token(self.instance_name)
|
||||
rows, upto_token, limited = self.get_success(
|
||||
self.queue.get_replication_rows("master", upto_token, now_token, 10)
|
||||
)
|
||||
|
||||
self.assertEqual(upto_token, now_token)
|
||||
self.assertFalse(limited)
|
||||
|
||||
expected_rows = [
|
||||
(2, ("dest3", "@user3:test")),
|
||||
]
|
||||
|
||||
self.assertCountEqual(rows, expected_rows)
|
||||
|
||||
def test_clear_queue_all(self):
|
||||
state1 = UserPresenceState.default("@user1:test")
|
||||
state2 = UserPresenceState.default("@user2:test")
|
||||
|
|
|
|||
Loading…
Reference in a new issue