Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes
This commit is contained in:
Erik Johnston
commit
6942377f90
65
CHANGES.md
65
CHANGES.md
|
|
@ -1,3 +1,68 @@
|
|||
Synapse 1.34.0rc1 (2021-05-12)
|
||||
==============================
|
||||
|
||||
This release deprecates the `room_invite_state_types` configuration setting. See the [upgrade notes](https://github.com/matrix-org/synapse/blob/release-v1.34.0/UPGRADE.rst#upgrading-to-v1340) for instructions on updating your configuration file to use the new `room_prejoin_state` setting.
|
||||
|
||||
This release also deprecates the `POST /_synapse/admin/v1/rooms/<room_id>/delete` admin API route. Server administrators are encouraged to update their scripts to use the new `DELETE /_synapse/admin/v1/rooms/<room_id>` route instead.
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Add experimental option to track memory usage of the caches. ([\#9881](https://github.com/matrix-org/synapse/issues/9881))
|
||||
- Add support for `DELETE /_synapse/admin/v1/rooms/<room_id>`. ([\#9889](https://github.com/matrix-org/synapse/issues/9889))
|
||||
- Add limits to how often Synapse will GC, ensuring that large servers do not end up GC thrashing if `gc_thresholds` has not been correctly set. ([\#9902](https://github.com/matrix-org/synapse/issues/9902))
|
||||
- Improve performance of sending events for worker-based deployments using Redis. ([\#9905](https://github.com/matrix-org/synapse/issues/9905), [\#9950](https://github.com/matrix-org/synapse/issues/9950), [\#9951](https://github.com/matrix-org/synapse/issues/9951))
|
||||
- Improve performance after joining a large room when presence is enabled. ([\#9910](https://github.com/matrix-org/synapse/issues/9910), [\#9916](https://github.com/matrix-org/synapse/issues/9916))
|
||||
- Support stable identifiers for [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) Spaces. `m.space.child` events will now be taken into account when populating the experimental spaces summary response. Please see [the upgrade notes](https://github.com/matrix-org/synapse/blob/release-v1.34.0/UPGRADE.rst#upgrading-to-v1340) if you have customised `room_invite_state_types` in your configuration. ([\#9915](https://github.com/matrix-org/synapse/issues/9915), [\#9966](https://github.com/matrix-org/synapse/issues/9966))
|
||||
- Improve performance of backfilling in large rooms. ([\#9935](https://github.com/matrix-org/synapse/issues/9935))
|
||||
- Add a config option to allow you to prevent device display names from being shared over federation. Contributed by @aaronraimist. ([\#9945](https://github.com/matrix-org/synapse/issues/9945))
|
||||
- Update support for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946): Spaces Summary. ([\#9947](https://github.com/matrix-org/synapse/issues/9947), [\#9954](https://github.com/matrix-org/synapse/issues/9954))
|
||||
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix a bug introduced in v1.32.0 where the associated connection was improperly logged for SQL logging statements. ([\#9895](https://github.com/matrix-org/synapse/issues/9895))
|
||||
- Correct the type hint for the `user_may_create_room_alias` method of spam checkers. It is provided a `RoomAlias`, not a `str`. ([\#9896](https://github.com/matrix-org/synapse/issues/9896))
|
||||
- Fix bug where user directory could get out of sync if room visibility and membership changed in quick succession. ([\#9910](https://github.com/matrix-org/synapse/issues/9910))
|
||||
- Include the `origin_server_ts` property in the experimental [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946) support to allow clients to properly sort rooms. ([\#9928](https://github.com/matrix-org/synapse/issues/9928))
|
||||
- Fix bugs introduced in v1.23.0 which made the PostgreSQL port script fail when run with a newly-created SQLite database. ([\#9930](https://github.com/matrix-org/synapse/issues/9930))
|
||||
- Fix a bug introduced in Synapse 1.29.0 which caused `m.room_key_request` to-device messages sent from one user to another to be dropped. ([\#9961](https://github.com/matrix-org/synapse/issues/9961), [\#9965](https://github.com/matrix-org/synapse/issues/9965))
|
||||
- Fix a bug introduced in v1.27.0 preventing users and appservices exempt from ratelimiting from creating rooms with many invitees. ([\#9968](https://github.com/matrix-org/synapse/issues/9968))
|
||||
|
||||
|
||||
Updates to the Docker image
|
||||
---------------------------
|
||||
|
||||
- Add `startup_delay` to docker healthcheck to reduce waiting time for coming online and update the documentation with extra options. Contributed by @Maquis196. ([\#9913](https://github.com/matrix-org/synapse/issues/9913))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Add `port` argument to the Postgres database sample config section. ([\#9911](https://github.com/matrix-org/synapse/issues/9911))
|
||||
|
||||
|
||||
Deprecations and Removals
|
||||
-------------------------
|
||||
|
||||
- Mark as deprecated `POST /_synapse/admin/v1/rooms/<room_id>/delete`. ([\#9889](https://github.com/matrix-org/synapse/issues/9889))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Reduce the length of Synapse's access tokens. ([\#5588](https://github.com/matrix-org/synapse/issues/5588))
|
||||
- Export jemalloc stats to Prometheus if it is being used. ([\#9882](https://github.com/matrix-org/synapse/issues/9882))
|
||||
- Add type hints to presence handler. ([\#9885](https://github.com/matrix-org/synapse/issues/9885))
|
||||
- Reduce memory usage of the LRU caches. ([\#9886](https://github.com/matrix-org/synapse/issues/9886))
|
||||
- Add type hints to the `synapse.handlers` module. ([\#9896](https://github.com/matrix-org/synapse/issues/9896))
|
||||
- Time response time for external cache requests. ([\#9904](https://github.com/matrix-org/synapse/issues/9904))
|
||||
- Minor fixes to the `make_full_schema.sh` script. ([\#9931](https://github.com/matrix-org/synapse/issues/9931))
|
||||
- Move database schema files into a common directory. ([\#9932](https://github.com/matrix-org/synapse/issues/9932))
|
||||
- Add debug logging for lost/delayed to-device messages. ([\#9959](https://github.com/matrix-org/synapse/issues/9959))
|
||||
|
||||
|
||||
Synapse 1.33.2 (2021-05-11)
|
||||
===========================
|
||||
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
Reduce the length of Synapse's access tokens.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add experimental option to track memory usage of the caches.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Export jemalloc stats to Prometheus if it is being used.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add type hints to presence handler.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reduce memory usage of the LRU caches.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add support for `DELETE /_synapse/admin/v1/rooms/<room_id>`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Mark as deprecated `POST /_synapse/admin/v1/rooms/<room_id>/delete`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a bug introduced in v1.32.0 where the associated connection was improperly logged for SQL logging statements.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Correct the type hint for the `user_may_create_room_alias` method of spam checkers. It is provided a `RoomAlias`, not a `str`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add type hints to the `synapse.handlers` module.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add limits to how often Synapse will GC, ensuring that large servers do not end up GC thrashing if `gc_thresholds` has not been correctly set.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Time response time for external cache requests.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve performance of sending events for worker-based deployments using Redis.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix bug where user directory could get out of sync if room visibility and membership changed in quick succession.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve performance after joining a large room when presence is enabled.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add `port` argument to the Postgres database sample config section.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Added startup_delay to docker healthcheck to reduce waiting time for coming online, updated readme for extra options, contributed by @Maquis196.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Support stable identifiers for [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) Spaces. `m.space.child` events will now be taken into account when populating the experimental spaces summary response. Please see `UPGRADE.rst` if you have customised `room_invite_state_types` in your configuration.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve performance after joining a large room when presence is enabled.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Include the `origin_server_ts` property in the experimental [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946) support to allow clients to properly sort rooms.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix bugs introduced in v1.23.0 which made the PostgreSQL port script fail when run with a newly-created SQLite database.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Minor fixes to the `make_full_schema.sh` script.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Move database schema files into a common directory.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve performance of backfilling in large rooms.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add a config option to allow you to prevent device display names from being shared over federation. Contributed by @aaronraimist.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update support for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946): Spaces Summary.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve performance of sending events for worker-based deployments using Redis.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve performance of sending events for worker-based deployments using Redis.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update support for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946): Spaces Summary.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add debug logging for lost/delayed to-device messages.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a bug introduced in Synapse 1.29.0 which caused `m.room_key_request` to-device messages sent from one user to another to be dropped.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a bug introduced in Synapse 1.29.0 which caused `m.room_key_request` to-device messages sent from one user to another to be dropped.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Support stable identifiers for [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) Spaces. `m.space.child` events will now be taken into account when populating the experimental spaces summary response. Please see `UPGRADE.rst` if you have customised `room_invite_state_types` in your configuration.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a bug introduced in v1.27.0 preventing users and appservices exempt from ratelimiting from creating rooms with many invitees.
|
||||
1
changelog.d/9975.misc
Normal file
1
changelog.d/9975.misc
Normal file
|
|
@ -0,0 +1 @@
|
|||
Minor enhancements to the `@cachedList` descriptor.
|
||||
1
changelog.d/9978.feature
Normal file
1
changelog.d/9978.feature
Normal file
|
|
@ -0,0 +1 @@
|
|||
Add a configuration option which allows enabling opentracing by user id.
|
||||
1
changelog.d/9980.doc
Normal file
1
changelog.d/9980.doc
Normal file
|
|
@ -0,0 +1 @@
|
|||
Clarify documentation around SSO mapping providers generating unique IDs and localparts.
|
||||
1
changelog.d/9984.misc
Normal file
1
changelog.d/9984.misc
Normal file
|
|
@ -0,0 +1 @@
|
|||
Simplify a few helper functions.
|
||||
1
changelog.d/9985.misc
Normal file
1
changelog.d/9985.misc
Normal file
|
|
@ -0,0 +1 @@
|
|||
Simplify a few helper functions.
|
||||
1
changelog.d/9986.misc
Normal file
1
changelog.d/9986.misc
Normal file
|
|
@ -0,0 +1 @@
|
|||
Simplify a few helper functions.
|
||||
1
changelog.d/9987.misc
Normal file
1
changelog.d/9987.misc
Normal file
|
|
@ -0,0 +1 @@
|
|||
Remove unnecessary property from SQLBaseStore.
|
||||
1
changelog.d/9988.doc
Normal file
1
changelog.d/9988.doc
Normal file
|
|
@ -0,0 +1 @@
|
|||
Fix outdated minimum PostgreSQL version in postgres.md.
|
||||
|
|
@ -42,17 +42,17 @@ To receive OpenTracing spans, start up a Jaeger server. This can be done
|
|||
using docker like so:
|
||||
|
||||
```sh
|
||||
docker run -d --name jaeger
|
||||
docker run -d --name jaeger \
|
||||
-p 6831:6831/udp \
|
||||
-p 6832:6832/udp \
|
||||
-p 5778:5778 \
|
||||
-p 16686:16686 \
|
||||
-p 14268:14268 \
|
||||
jaegertracing/all-in-one:1.13
|
||||
jaegertracing/all-in-one:1
|
||||
```
|
||||
|
||||
Latest documentation is probably at
|
||||
<https://www.jaegertracing.io/docs/1.13/getting-started/>
|
||||
https://www.jaegertracing.io/docs/latest/getting-started.
|
||||
|
||||
## Enable OpenTracing in Synapse
|
||||
|
||||
|
|
@ -62,7 +62,7 @@ as shown in the [sample config](./sample_config.yaml). For example:
|
|||
|
||||
```yaml
|
||||
opentracing:
|
||||
tracer_enabled: true
|
||||
enabled: true
|
||||
homeserver_whitelist:
|
||||
- "mytrustedhomeserver.org"
|
||||
- "*.myotherhomeservers.com"
|
||||
|
|
@ -90,4 +90,4 @@ to two problems, namely:
|
|||
## Configuring Jaeger
|
||||
|
||||
Sampling strategies can be set as in this document:
|
||||
<https://www.jaegertracing.io/docs/1.13/sampling/>
|
||||
<https://www.jaegertracing.io/docs/latest/sampling/>.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Using Postgres
|
||||
|
||||
Postgres version 9.5 or later is known to work.
|
||||
Synapse supports PostgreSQL versions 9.6 or later.
|
||||
|
||||
## Install postgres client libraries
|
||||
|
||||
|
|
|
|||
|
|
@ -2845,7 +2845,8 @@ opentracing:
|
|||
#enabled: true
|
||||
|
||||
# The list of homeservers we wish to send and receive span contexts and span baggage.
|
||||
# See docs/opentracing.rst
|
||||
# See docs/opentracing.rst.
|
||||
#
|
||||
# This is a list of regexes which are matched against the server_name of the
|
||||
# homeserver.
|
||||
#
|
||||
|
|
@ -2854,19 +2855,26 @@ opentracing:
|
|||
#homeserver_whitelist:
|
||||
# - ".*"
|
||||
|
||||
# A list of the matrix IDs of users whose requests will always be traced,
|
||||
# even if the tracing system would otherwise drop the traces due to
|
||||
# probabilistic sampling.
|
||||
#
|
||||
# By default, the list is empty.
|
||||
#
|
||||
#force_tracing_for_users:
|
||||
# - "@user1:server_name"
|
||||
# - "@user2:server_name"
|
||||
|
||||
# Jaeger can be configured to sample traces at different rates.
|
||||
# All configuration options provided by Jaeger can be set here.
|
||||
# Jaeger's configuration mostly related to trace sampling which
|
||||
# Jaeger's configuration is mostly related to trace sampling which
|
||||
# is documented here:
|
||||
# https://www.jaegertracing.io/docs/1.13/sampling/.
|
||||
# https://www.jaegertracing.io/docs/latest/sampling/.
|
||||
#
|
||||
#jaeger_config:
|
||||
# sampler:
|
||||
# type: const
|
||||
# param: 1
|
||||
|
||||
# Logging whether spans were started and reported
|
||||
#
|
||||
# logging:
|
||||
# false
|
||||
|
||||
|
|
|
|||
|
|
@ -67,8 +67,8 @@ A custom mapping provider must specify the following methods:
|
|||
- Arguments:
|
||||
- `userinfo` - A `authlib.oidc.core.claims.UserInfo` object to extract user
|
||||
information from.
|
||||
- This method must return a string, which is the unique identifier for the
|
||||
user. Commonly the ``sub`` claim of the response.
|
||||
- This method must return a string, which is the unique, immutable identifier
|
||||
for the user. Commonly the `sub` claim of the response.
|
||||
* `map_user_attributes(self, userinfo, token, failures)`
|
||||
- This method must be async.
|
||||
- Arguments:
|
||||
|
|
@ -87,7 +87,9 @@ A custom mapping provider must specify the following methods:
|
|||
`localpart` value, such as `john.doe1`.
|
||||
- Returns a dictionary with two keys:
|
||||
- `localpart`: A string, used to generate the Matrix ID. If this is
|
||||
`None`, the user is prompted to pick their own username.
|
||||
`None`, the user is prompted to pick their own username. This is only used
|
||||
during a user's first login. Once a localpart has been associated with a
|
||||
remote user ID (see `get_remote_user_id`) it cannot be updated.
|
||||
- `displayname`: An optional string, the display name for the user.
|
||||
* `get_extra_attributes(self, userinfo, token)`
|
||||
- This method must be async.
|
||||
|
|
@ -153,8 +155,8 @@ A custom mapping provider must specify the following methods:
|
|||
information from.
|
||||
- `client_redirect_url` - A string, the URL that the client will be
|
||||
redirected to.
|
||||
- This method must return a string, which is the unique identifier for the
|
||||
user. Commonly the ``uid`` claim of the response.
|
||||
- This method must return a string, which is the unique, immutable identifier
|
||||
for the user. Commonly the `uid` claim of the response.
|
||||
* `saml_response_to_user_attributes(self, saml_response, failures, client_redirect_url)`
|
||||
- Arguments:
|
||||
- `saml_response` - A `saml2.response.AuthnResponse` object to extract user
|
||||
|
|
@ -172,8 +174,10 @@ A custom mapping provider must specify the following methods:
|
|||
redirected to.
|
||||
- This method must return a dictionary, which will then be used by Synapse
|
||||
to build a new user. The following keys are allowed:
|
||||
* `mxid_localpart` - The mxid localpart of the new user. If this is
|
||||
`None`, the user is prompted to pick their own username.
|
||||
* `mxid_localpart` - A string, the mxid localpart of the new user. If this is
|
||||
`None`, the user is prompted to pick their own username. This is only used
|
||||
during a user's first login. Once a localpart has been associated with a
|
||||
remote user ID (see `get_remote_user_id`) it cannot be updated.
|
||||
* `displayname` - The displayname of the new user. If not provided, will default to
|
||||
the value of `mxid_localpart`.
|
||||
* `emails` - A list of emails for the new user. If not provided, will
|
||||
|
|
|
|||
|
|
@ -47,7 +47,7 @@ try:
|
|||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "1.33.2"
|
||||
__version__ = "1.34.0rc1"
|
||||
|
||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||
# We import here so that we don't have to install a bunch of deps when
|
||||
|
|
|
|||
|
|
@ -87,6 +87,7 @@ class Auth:
|
|||
)
|
||||
self._track_appservice_user_ips = hs.config.track_appservice_user_ips
|
||||
self._macaroon_secret_key = hs.config.macaroon_secret_key
|
||||
self._force_tracing_for_users = hs.config.tracing.force_tracing_for_users
|
||||
|
||||
async def check_from_context(
|
||||
self, room_version: str, event, context, do_sig_check=True
|
||||
|
|
@ -208,6 +209,8 @@ class Auth:
|
|||
opentracing.set_tag("authenticated_entity", user_id)
|
||||
opentracing.set_tag("user_id", user_id)
|
||||
opentracing.set_tag("appservice_id", app_service.id)
|
||||
if user_id in self._force_tracing_for_users:
|
||||
opentracing.set_tag(opentracing.tags.SAMPLING_PRIORITY, 1)
|
||||
|
||||
return requester
|
||||
|
||||
|
|
@ -260,6 +263,8 @@ class Auth:
|
|||
opentracing.set_tag("user_id", user_info.user_id)
|
||||
if device_id:
|
||||
opentracing.set_tag("device_id", device_id)
|
||||
if user_info.token_owner in self._force_tracing_for_users:
|
||||
opentracing.set_tag(opentracing.tags.SAMPLING_PRIORITY, 1)
|
||||
|
||||
return requester
|
||||
except KeyError:
|
||||
|
|
|
|||
|
|
@ -349,4 +349,4 @@ class RegistrationConfig(Config):
|
|||
|
||||
def read_arguments(self, args):
|
||||
if args.enable_registration is not None:
|
||||
self.enable_registration = bool(strtobool(str(args.enable_registration)))
|
||||
self.enable_registration = strtobool(str(args.enable_registration))
|
||||
|
|
|
|||
|
|
@ -12,6 +12,8 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from typing import Set
|
||||
|
||||
from synapse.python_dependencies import DependencyException, check_requirements
|
||||
|
||||
from ._base import Config, ConfigError
|
||||
|
|
@ -32,6 +34,8 @@ class TracerConfig(Config):
|
|||
{"sampler": {"type": "const", "param": 1}, "logging": False},
|
||||
)
|
||||
|
||||
self.force_tracing_for_users: Set[str] = set()
|
||||
|
||||
if not self.opentracer_enabled:
|
||||
return
|
||||
|
||||
|
|
@ -48,6 +52,19 @@ class TracerConfig(Config):
|
|||
if not isinstance(self.opentracer_whitelist, list):
|
||||
raise ConfigError("Tracer homeserver_whitelist config is malformed")
|
||||
|
||||
force_tracing_for_users = opentracing_config.get("force_tracing_for_users", [])
|
||||
if not isinstance(force_tracing_for_users, list):
|
||||
raise ConfigError(
|
||||
"Expected a list", ("opentracing", "force_tracing_for_users")
|
||||
)
|
||||
for i, u in enumerate(force_tracing_for_users):
|
||||
if not isinstance(u, str):
|
||||
raise ConfigError(
|
||||
"Expected a string",
|
||||
("opentracing", "force_tracing_for_users", f"index {i}"),
|
||||
)
|
||||
self.force_tracing_for_users.add(u)
|
||||
|
||||
def generate_config_section(cls, **kwargs):
|
||||
return """\
|
||||
## Opentracing ##
|
||||
|
|
@ -64,7 +81,8 @@ class TracerConfig(Config):
|
|||
#enabled: true
|
||||
|
||||
# The list of homeservers we wish to send and receive span contexts and span baggage.
|
||||
# See docs/opentracing.rst
|
||||
# See docs/opentracing.rst.
|
||||
#
|
||||
# This is a list of regexes which are matched against the server_name of the
|
||||
# homeserver.
|
||||
#
|
||||
|
|
@ -73,19 +91,26 @@ class TracerConfig(Config):
|
|||
#homeserver_whitelist:
|
||||
# - ".*"
|
||||
|
||||
# A list of the matrix IDs of users whose requests will always be traced,
|
||||
# even if the tracing system would otherwise drop the traces due to
|
||||
# probabilistic sampling.
|
||||
#
|
||||
# By default, the list is empty.
|
||||
#
|
||||
#force_tracing_for_users:
|
||||
# - "@user1:server_name"
|
||||
# - "@user2:server_name"
|
||||
|
||||
# Jaeger can be configured to sample traces at different rates.
|
||||
# All configuration options provided by Jaeger can be set here.
|
||||
# Jaeger's configuration mostly related to trace sampling which
|
||||
# Jaeger's configuration is mostly related to trace sampling which
|
||||
# is documented here:
|
||||
# https://www.jaegertracing.io/docs/1.13/sampling/.
|
||||
# https://www.jaegertracing.io/docs/latest/sampling/.
|
||||
#
|
||||
#jaeger_config:
|
||||
# sampler:
|
||||
# type: const
|
||||
# param: 1
|
||||
|
||||
# Logging whether spans were started and reported
|
||||
#
|
||||
# logging:
|
||||
# false
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -14,7 +14,6 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import logging
|
||||
import random
|
||||
from abc import ABCMeta
|
||||
from typing import TYPE_CHECKING, Any, Collection, Iterable, Optional, Union
|
||||
|
||||
|
|
@ -44,7 +43,6 @@ class SQLBaseStore(metaclass=ABCMeta):
|
|||
self._clock = hs.get_clock()
|
||||
self.database_engine = database.engine
|
||||
self.db_pool = database
|
||||
self.rand = random.SystemRandom()
|
||||
|
||||
def process_replication_rows(
|
||||
self,
|
||||
|
|
|
|||
|
|
@ -665,7 +665,7 @@ class DeviceWorkerStore(SQLBaseStore):
|
|||
cached_method_name="get_device_list_last_stream_id_for_remote",
|
||||
list_name="user_ids",
|
||||
)
|
||||
async def get_device_list_last_stream_id_for_remotes(self, user_ids: str):
|
||||
async def get_device_list_last_stream_id_for_remotes(self, user_ids: Iterable[str]):
|
||||
rows = await self.db_pool.simple_select_many_batch(
|
||||
table="device_lists_remote_extremeties",
|
||||
column="user_id",
|
||||
|
|
|
|||
|
|
@ -473,7 +473,7 @@ class EndToEndKeyWorkerStore(EndToEndKeyBackgroundStore):
|
|||
num_args=1,
|
||||
)
|
||||
async def _get_bare_e2e_cross_signing_keys_bulk(
|
||||
self, user_ids: List[str]
|
||||
self, user_ids: Iterable[str]
|
||||
) -> Dict[str, Dict[str, dict]]:
|
||||
"""Returns the cross-signing keys for a set of users. The output of this
|
||||
function should be passed to _get_e2e_cross_signing_signatures_txn if
|
||||
|
|
@ -497,7 +497,7 @@ class EndToEndKeyWorkerStore(EndToEndKeyBackgroundStore):
|
|||
def _get_bare_e2e_cross_signing_keys_bulk_txn(
|
||||
self,
|
||||
txn: Connection,
|
||||
user_ids: List[str],
|
||||
user_ids: Iterable[str],
|
||||
) -> Dict[str, Dict[str, dict]]:
|
||||
"""Returns the cross-signing keys for a set of users. The output of this
|
||||
function should be passed to _get_e2e_cross_signing_signatures_txn if
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import logging
|
||||
import random
|
||||
import re
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple, Union
|
||||
|
||||
|
|
@ -997,7 +998,7 @@ class RegistrationWorkerStore(CacheInvalidationWorkerStore):
|
|||
expiration_ts = now_ms + self._account_validity_period
|
||||
|
||||
if use_delta:
|
||||
expiration_ts = self.rand.randrange(
|
||||
expiration_ts = random.randrange(
|
||||
expiration_ts - self._account_validity_startup_job_max_delta,
|
||||
expiration_ts,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -12,6 +12,8 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from typing import Dict, Iterable
|
||||
|
||||
from synapse.storage._base import SQLBaseStore
|
||||
from synapse.util.caches.descriptors import cached, cachedList
|
||||
|
||||
|
|
@ -37,21 +39,16 @@ class UserErasureWorkerStore(SQLBaseStore):
|
|||
return bool(result)
|
||||
|
||||
@cachedList(cached_method_name="is_user_erased", list_name="user_ids")
|
||||
async def are_users_erased(self, user_ids):
|
||||
async def are_users_erased(self, user_ids: Iterable[str]) -> Dict[str, bool]:
|
||||
"""
|
||||
Checks which users in a list have requested erasure
|
||||
|
||||
Args:
|
||||
user_ids (iterable[str]): full user id to check
|
||||
user_ids: full user ids to check
|
||||
|
||||
Returns:
|
||||
dict[str, bool]:
|
||||
for each user, whether the user has requested erasure.
|
||||
for each user, whether the user has requested erasure.
|
||||
"""
|
||||
# this serves the dual purpose of (a) making sure we can do len and
|
||||
# iterate it multiple times, and (b) avoiding duplicates.
|
||||
user_ids = tuple(set(user_ids))
|
||||
|
||||
rows = await self.db_pool.simple_select_many_batch(
|
||||
table="erased_users",
|
||||
column="user_id",
|
||||
|
|
|
|||
|
|
@ -322,8 +322,8 @@ class DeferredCacheDescriptor(_CacheDescriptorBase):
|
|||
class DeferredCacheListDescriptor(_CacheDescriptorBase):
|
||||
"""Wraps an existing cache to support bulk fetching of keys.
|
||||
|
||||
Given a list of keys it looks in the cache to find any hits, then passes
|
||||
the list of missing keys to the wrapped function.
|
||||
Given an iterable of keys it looks in the cache to find any hits, then passes
|
||||
the tuple of missing keys to the wrapped function.
|
||||
|
||||
Once wrapped, the function returns a Deferred which resolves to the list
|
||||
of results.
|
||||
|
|
@ -437,7 +437,9 @@ class DeferredCacheListDescriptor(_CacheDescriptorBase):
|
|||
return f
|
||||
|
||||
args_to_call = dict(arg_dict)
|
||||
args_to_call[self.list_name] = list(missing)
|
||||
# copy the missing set before sending it to the callee, to guard against
|
||||
# modification.
|
||||
args_to_call[self.list_name] = tuple(missing)
|
||||
|
||||
cached_defers.append(
|
||||
defer.maybeDeferred(
|
||||
|
|
@ -522,14 +524,14 @@ def cachedList(
|
|||
|
||||
Used to do batch lookups for an already created cache. A single argument
|
||||
is specified as a list that is iterated through to lookup keys in the
|
||||
original cache. A new list consisting of the keys that weren't in the cache
|
||||
get passed to the original function, the result of which is stored in the
|
||||
original cache. A new tuple consisting of the (deduplicated) keys that weren't in
|
||||
the cache gets passed to the original function, the result of which is stored in the
|
||||
cache.
|
||||
|
||||
Args:
|
||||
cached_method_name: The name of the single-item lookup method.
|
||||
This is only used to find the cache to use.
|
||||
list_name: The name of the argument that is the list to use to
|
||||
list_name: The name of the argument that is the iterable to use to
|
||||
do batch lookups in the cache.
|
||||
num_args: Number of arguments to use as the key in the cache
|
||||
(including list_name). Defaults to all named parameters.
|
||||
|
|
|
|||
|
|
@ -13,8 +13,8 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import itertools
|
||||
import random
|
||||
import re
|
||||
import secrets
|
||||
import string
|
||||
from collections.abc import Iterable
|
||||
from typing import Optional, Tuple
|
||||
|
|
@ -35,26 +35,27 @@ CLIENT_SECRET_REGEX = re.compile(r"^[0-9a-zA-Z\.=_\-]+$")
|
|||
#
|
||||
MXC_REGEX = re.compile("^mxc://([^/]+)/([^/#?]+)$")
|
||||
|
||||
# random_string and random_string_with_symbols are used for a range of things,
|
||||
# some cryptographically important, some less so. We use SystemRandom to make sure
|
||||
# we get cryptographically-secure randoms.
|
||||
rand = random.SystemRandom()
|
||||
|
||||
|
||||
def random_string(length: int) -> str:
|
||||
return "".join(rand.choice(string.ascii_letters) for _ in range(length))
|
||||
"""Generate a cryptographically secure string of random letters.
|
||||
|
||||
Drawn from the characters: `a-z` and `A-Z`
|
||||
"""
|
||||
return "".join(secrets.choice(string.ascii_letters) for _ in range(length))
|
||||
|
||||
|
||||
def random_string_with_symbols(length: int) -> str:
|
||||
return "".join(rand.choice(_string_with_symbols) for _ in range(length))
|
||||
"""Generate a cryptographically secure string of random letters/numbers/symbols.
|
||||
|
||||
Drawn from the characters: `a-z`, `A-Z`, `0-9`, and `.,;:^&*-_+=#~@`
|
||||
"""
|
||||
return "".join(secrets.choice(_string_with_symbols) for _ in range(length))
|
||||
|
||||
|
||||
def is_ascii(s: bytes) -> bool:
|
||||
try:
|
||||
s.decode("ascii").encode("ascii")
|
||||
except UnicodeDecodeError:
|
||||
return False
|
||||
except UnicodeEncodeError:
|
||||
except UnicodeError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
|
|
|||
|
|
@ -666,18 +666,20 @@ class CachedListDescriptorTestCase(unittest.TestCase):
|
|||
with LoggingContext("c1") as c1:
|
||||
obj = Cls()
|
||||
obj.mock.return_value = {10: "fish", 20: "chips"}
|
||||
|
||||
# start the lookup off
|
||||
d1 = obj.list_fn([10, 20], 2)
|
||||
self.assertEqual(current_context(), SENTINEL_CONTEXT)
|
||||
r = yield d1
|
||||
self.assertEqual(current_context(), c1)
|
||||
obj.mock.assert_called_once_with([10, 20], 2)
|
||||
obj.mock.assert_called_once_with((10, 20), 2)
|
||||
self.assertEqual(r, {10: "fish", 20: "chips"})
|
||||
obj.mock.reset_mock()
|
||||
|
||||
# a call with different params should call the mock again
|
||||
obj.mock.return_value = {30: "peas"}
|
||||
r = yield obj.list_fn([20, 30], 2)
|
||||
obj.mock.assert_called_once_with([30], 2)
|
||||
obj.mock.assert_called_once_with((30,), 2)
|
||||
self.assertEqual(r, {20: "chips", 30: "peas"})
|
||||
obj.mock.reset_mock()
|
||||
|
||||
|
|
@ -692,6 +694,15 @@ class CachedListDescriptorTestCase(unittest.TestCase):
|
|||
obj.mock.assert_not_called()
|
||||
self.assertEqual(r, {10: "fish", 20: "chips", 30: "peas"})
|
||||
|
||||
# we should also be able to use a (single-use) iterable, and should
|
||||
# deduplicate the keys
|
||||
obj.mock.reset_mock()
|
||||
obj.mock.return_value = {40: "gravy"}
|
||||
iterable = (x for x in [10, 40, 40])
|
||||
r = yield obj.list_fn(iterable, 2)
|
||||
obj.mock.assert_called_once_with((40,), 2)
|
||||
self.assertEqual(r, {10: "fish", 40: "gravy"})
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def test_invalidate(self):
|
||||
"""Make sure that invalidation callbacks are called."""
|
||||
|
|
@ -717,7 +728,7 @@ class CachedListDescriptorTestCase(unittest.TestCase):
|
|||
# cache miss
|
||||
obj.mock.return_value = {10: "fish", 20: "chips"}
|
||||
r1 = yield obj.list_fn([10, 20], 2, on_invalidate=invalidate0)
|
||||
obj.mock.assert_called_once_with([10, 20], 2)
|
||||
obj.mock.assert_called_once_with((10, 20), 2)
|
||||
self.assertEqual(r1, {10: "fish", 20: "chips"})
|
||||
obj.mock.reset_mock()
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue