Merge branch 'develop' into cross-signing_federation

This commit is contained in:
Hubert Chathi 2019-10-24 22:31:25 -04:00
commit c40d7244f8
350 changed files with 5090 additions and 4592 deletions

View file

@ -1,3 +1,85 @@
Synapse 1.5.0rc1 (2019-10-24)
==========================
This release includes a database migration step **which may take a long time to complete**:
- Allow devices to be marked as hidden, for use by features such as cross-signing.
This adds a new field with a default value to the devices field in the database,
and so the database upgrade may take a long time depending on how many devices
are in the database. ([\#5759](https://github.com/matrix-org/synapse/issues/5759))
Features
--------
- Improve quality of thumbnails for 1-bit/8-bit color palette images. ([\#2142](https://github.com/matrix-org/synapse/issues/2142))
- Add ability to upload cross-signing signatures. ([\#5726](https://github.com/matrix-org/synapse/issues/5726))
- Allow uploading of cross-signing keys. ([\#5769](https://github.com/matrix-org/synapse/issues/5769))
- CAS login now provides a default display name for users if a `displayname_attribute` is set in the configuration file. ([\#6114](https://github.com/matrix-org/synapse/issues/6114))
- Reject all pending invites for a user during deactivation. ([\#6125](https://github.com/matrix-org/synapse/issues/6125))
- Add config option to suppress client side resource limit alerting. ([\#6173](https://github.com/matrix-org/synapse/issues/6173))
Bugfixes
--------
- Return an HTTP 404 instead of 400 when requesting a filter by ID that is unknown to the server. Thanks to @krombel for contributing this! ([\#2380](https://github.com/matrix-org/synapse/issues/2380))
- Fix a bug where users could be invited twice to the same group. ([\#3436](https://github.com/matrix-org/synapse/issues/3436))
- Fix `/createRoom` failing with badly-formatted MXIDs in the invitee list. Thanks to @wener291! ([\#4088](https://github.com/matrix-org/synapse/issues/4088))
- Make the `synapse_port_db` script create the right indexes on a new PostgreSQL database. ([\#6102](https://github.com/matrix-org/synapse/issues/6102), [\#6178](https://github.com/matrix-org/synapse/issues/6178), [\#6243](https://github.com/matrix-org/synapse/issues/6243))
- Fix bug when uploading a large file: Synapse responds with `M_UNKNOWN` while it should be `M_TOO_LARGE` according to spec. Contributed by Anshul Angaria. ([\#6109](https://github.com/matrix-org/synapse/issues/6109))
- Fix user push rules being deleted from a room when it is upgraded. ([\#6144](https://github.com/matrix-org/synapse/issues/6144))
- Don't 500 when trying to exchange a revoked 3PID invite. ([\#6147](https://github.com/matrix-org/synapse/issues/6147))
- Fix transferring notifications and tags when joining an upgraded room that is new to your server. ([\#6155](https://github.com/matrix-org/synapse/issues/6155))
- Fix bug where guest account registration can wedge after restart. ([\#6161](https://github.com/matrix-org/synapse/issues/6161))
- Fix monthly active user reaping when reserved users are specified. ([\#6168](https://github.com/matrix-org/synapse/issues/6168))
- Fix `/federation/v1/state` endpoint not supporting newer room versions. ([\#6170](https://github.com/matrix-org/synapse/issues/6170))
- Fix bug where we were updating censored events as bytes rather than text, occaisonally causing invalid JSON being inserted breaking APIs that attempted to fetch such events. ([\#6186](https://github.com/matrix-org/synapse/issues/6186))
- Fix occasional missed updates in the room and user directories. ([\#6187](https://github.com/matrix-org/synapse/issues/6187))
- Fix tracing of non-JSON APIs, `/media`, `/key` etc. ([\#6195](https://github.com/matrix-org/synapse/issues/6195))
- Fix bug where presence would not get timed out correctly if a synchrotron worker is used and restarted. ([\#6212](https://github.com/matrix-org/synapse/issues/6212))
- synapse_port_db: Add 2 additional BOOLEAN_COLUMNS to be able to convert from database schema v56. ([\#6216](https://github.com/matrix-org/synapse/issues/6216))
- Fix a bug where the Synapse demo script blacklisted `::1` (ipv6 localhost) from receiving federation traffic. ([\#6229](https://github.com/matrix-org/synapse/issues/6229))
Updates to the Docker image
---------------------------
- Fix logging getting lost for the docker image. ([\#6197](https://github.com/matrix-org/synapse/issues/6197))
Internal Changes
----------------
- Update `user_filters` table to have a unique index, and non-null columns. Thanks to @pik for contributing this. ([\#1172](https://github.com/matrix-org/synapse/issues/1172), [\#6175](https://github.com/matrix-org/synapse/issues/6175), [\#6184](https://github.com/matrix-org/synapse/issues/6184))
- Move lookup-related functions from RoomMemberHandler to IdentityHandler. ([\#5978](https://github.com/matrix-org/synapse/issues/5978))
- Improve performance of the public room list directory. ([\#6019](https://github.com/matrix-org/synapse/issues/6019), [\#6152](https://github.com/matrix-org/synapse/issues/6152), [\#6153](https://github.com/matrix-org/synapse/issues/6153), [\#6154](https://github.com/matrix-org/synapse/issues/6154))
- Edit header dicts docstrings in `SimpleHttpClient` to note that `str` or `bytes` can be passed as header keys. ([\#6077](https://github.com/matrix-org/synapse/issues/6077))
- Add snapcraft packaging information. Contributed by @devec0. ([\#6084](https://github.com/matrix-org/synapse/issues/6084), [\#6191](https://github.com/matrix-org/synapse/issues/6191))
- Kill off half-implemented password-reset via sms. ([\#6101](https://github.com/matrix-org/synapse/issues/6101))
- Remove `get_user_by_req` opentracing span and add some tags. ([\#6108](https://github.com/matrix-org/synapse/issues/6108))
- Drop some unused database tables. ([\#6115](https://github.com/matrix-org/synapse/issues/6115))
- Add env var to turn on tracking of log context changes. ([\#6127](https://github.com/matrix-org/synapse/issues/6127))
- Refactor configuration loading to allow better typechecking. ([\#6137](https://github.com/matrix-org/synapse/issues/6137))
- Log responder when responding to media request. ([\#6139](https://github.com/matrix-org/synapse/issues/6139))
- Improve performance of `find_next_generated_user_id` DB query. ([\#6148](https://github.com/matrix-org/synapse/issues/6148))
- Expand type-checking on modules imported by `synapse.config`. ([\#6150](https://github.com/matrix-org/synapse/issues/6150))
- Use Postgres ANY for selecting many values. ([\#6156](https://github.com/matrix-org/synapse/issues/6156))
- Add more caching to `_get_joined_users_from_context` DB query. ([\#6159](https://github.com/matrix-org/synapse/issues/6159))
- Add some metrics on the federation sender. ([\#6160](https://github.com/matrix-org/synapse/issues/6160))
- Add some logging to the rooms stats updates, to try to track down a flaky test. ([\#6167](https://github.com/matrix-org/synapse/issues/6167))
- Remove unused `timeout` parameter from `_get_public_room_list`. ([\#6179](https://github.com/matrix-org/synapse/issues/6179))
- Reject (accidental) attempts to insert bytes into postgres tables. ([\#6186](https://github.com/matrix-org/synapse/issues/6186))
- Make `version` optional in body of `PUT /room_keys/version/{version}`, since it's redundant. ([\#6189](https://github.com/matrix-org/synapse/issues/6189))
- Make storage layer responsible for adding device names to key, rather than the handler. ([\#6193](https://github.com/matrix-org/synapse/issues/6193))
- Port `synapse.rest.admin` module to use async/await. ([\#6196](https://github.com/matrix-org/synapse/issues/6196))
- Enforce that all boolean configuration values are lowercase in CI. ([\#6203](https://github.com/matrix-org/synapse/issues/6203))
- Remove some unused event-auth code. ([\#6214](https://github.com/matrix-org/synapse/issues/6214))
- Remove `Auth.check` method. ([\#6217](https://github.com/matrix-org/synapse/issues/6217))
- Remove `format_tap.py` script in favour of a perl reimplementation in Sytest's repo. ([\#6219](https://github.com/matrix-org/synapse/issues/6219))
- Refactor storage layer in preparation to support having multiple databases. ([\#6231](https://github.com/matrix-org/synapse/issues/6231))
- Remove some extra quotation marks across the codebase. ([\#6236](https://github.com/matrix-org/synapse/issues/6236))
Synapse 1.4.1 (2019-10-18) Synapse 1.4.1 (2019-10-18)
========================== ==========================

View file

@ -8,11 +8,12 @@ include demo/demo.tls.dh
include demo/*.py include demo/*.py
include demo/*.sh include demo/*.sh
recursive-include synapse/storage/schema *.sql recursive-include synapse/storage *.sql
recursive-include synapse/storage/schema *.sql.postgres recursive-include synapse/storage *.sql.postgres
recursive-include synapse/storage/schema *.sql.sqlite recursive-include synapse/storage *.sql.sqlite
recursive-include synapse/storage/schema *.py recursive-include synapse/storage *.py
recursive-include synapse/storage/schema *.txt recursive-include synapse/storage *.txt
recursive-include synapse/storage *.md
recursive-include docs * recursive-include docs *
recursive-include scripts * recursive-include scripts *

View file

@ -1 +0,0 @@
Update `user_filters` table to have a unique index, and non-null columns. Thanks to @pik for contributing this.

View file

@ -1 +0,0 @@
Improve quality of thumbnails for 1-bit/8-bit color palette images.

View file

@ -1 +0,0 @@
Return an HTTP 404 instead of 400 when requesting a filter by ID that is unknown to the server. Thanks to @krombel for contributing this!

View file

@ -1 +0,0 @@
Fix a problem where users could be invited twice to the same group.

View file

@ -1 +0,0 @@
Added domain validation when including a list of invitees upon room creation.

View file

@ -1 +0,0 @@
Add ability to upload cross-signing signatures.

View file

@ -1,4 +0,0 @@
Allow devices to be marked as hidden, for use by features such as cross-signing.
This adds a new field with a default value to the devices field in the database,
and so the database upgrade may take a long time depending on how many devices
are in the database.

View file

@ -1 +0,0 @@
Allow uploading of cross-signing keys.

View file

@ -1 +0,0 @@
Move lookup-related functions from RoomMemberHandler to IdentityHandler.

View file

@ -1 +0,0 @@
Improve performance of the public room list directory.

View file

@ -1 +0,0 @@
Edit header dicts docstrings in SimpleHttpClient to note that `str` or `bytes` can be passed as header keys.

View file

@ -1 +0,0 @@
Add snapcraft packaging information. Contributed by @devec0.

View file

@ -1 +0,0 @@
Kill off half-implemented password-reset via sms.

View file

@ -1 +0,0 @@
Remove `get_user_by_req` opentracing span and add some tags.

View file

@ -1 +0,0 @@
Fix bug when uploading a large file: Synapse responds with `M_UNKNOWN` while it should be `M_TOO_LARGE` according to spec. Contributed by Anshul Angaria.

View file

@ -1 +0,0 @@
CAS login now provides a default display name for users if a `displayname_attribute` is set in the configuration file.

View file

@ -1 +0,0 @@
Drop some unused database tables.

View file

@ -1 +0,0 @@
Reject all pending invites for a user during deactivation.

View file

@ -1 +0,0 @@
Add env var to turn on tracking of log context changes.

View file

@ -1 +0,0 @@
Refactor configuration loading to allow better typechecking.

View file

@ -1 +0,0 @@
Log responder when responding to media request.

View file

@ -1 +0,0 @@
Prevent user push rules being deleted from a room when it is upgraded.

View file

@ -1 +0,0 @@
Don't 500 when trying to exchange a revoked 3PID invite.

View file

@ -1 +0,0 @@
Improve performance of `find_next_generated_user_id` DB query.

View file

@ -1 +0,0 @@
Expand type-checking on modules imported by synapse.config.

View file

@ -1 +0,0 @@
Improve performance of the public room list directory.

View file

@ -1 +0,0 @@
Improve performance of the public room list directory.

View file

@ -1 +0,0 @@
Improve performance of the public room list directory.

View file

@ -1 +0,0 @@
Fix transferring notifications and tags when joining an upgraded room that is new to your server.

View file

@ -1 +0,0 @@
Use Postgres ANY for selecting many values.

View file

@ -1 +0,0 @@
Add more caching to `_get_joined_users_from_context` DB query.

View file

@ -1 +0,0 @@
Add some metrics on the federation sender.

View file

@ -1 +0,0 @@
Fix bug where guest account registration can wedge after restart.

View file

@ -1 +0,0 @@
Add some logging to the rooms stats updates, to try to track down a flaky test.

View file

@ -1 +0,0 @@
Fix monthly active user reaping where reserved users are specified.

View file

@ -1 +0,0 @@
Fix /federation/v1/state endpoint for recent room versions.

View file

@ -1 +0,0 @@
Update `user_filters` table to have a unique index, and non-null columns. Thanks to @pik for contributing this.

View file

@ -1 +0,0 @@
Make the `synapse_port_db` script create the right indexes on a new PostgreSQL database.

View file

@ -1 +0,0 @@
Remove unused `timeout` parameter from `_get_public_room_list`.

View file

@ -1 +0,0 @@
Update `user_filters` table to have a unique index, and non-null columns. Thanks to @pik for contributing this.

View file

@ -1 +0,0 @@
Fix bug where we were updating censored events as bytes rather than text, occaisonally causing invalid JSON being inserted breaking APIs that attempted to fetch such events.

View file

@ -1 +0,0 @@
Reject (accidental) attempts to insert bytes into postgres tables.

View file

@ -1 +0,0 @@
Fix occasional missed updates in the room and user directories.

View file

@ -1 +0,0 @@
Make `version` optional in body of `PUT /room_keys/version/{version}`, since it's redundant.

View file

@ -1 +0,0 @@
Add snapcraft packaging information. Contributed by @devec0.

View file

@ -1 +0,0 @@
Make storage layer responsible for adding device names to key, rather than the handler.

View file

@ -1 +0,0 @@
Fix tracing of non-JSON APIs, /media, /key etc.

View file

@ -1 +0,0 @@
Port synapse.rest.admin module to use async/await.

View file

@ -1 +0,0 @@
Fix logging getting lost for the docker image.

View file

@ -1 +0,0 @@
Fix bug where presence would not get timed out correctly if a synchrotron worker is used and restarted.

View file

@ -1 +0,0 @@
Remove some unused event-auth code.

View file

@ -1 +0,0 @@
synapse_port_db: Add 2 additional BOOLEAN_COLUMNS to be able to convert from database schema v56.

View file

@ -1 +0,0 @@
Remove Auth.check method.

View file

@ -1 +0,0 @@
Remove `format_tap.py` script in favour of a perl reimplementation in Sytest's repo.

View file

@ -1 +0,0 @@
Prevent the demo Synapse's from blacklisting `::1`.

1
changelog.d/6247.bugfix Normal file
View file

@ -0,0 +1 @@
Update list of boolean columns in `synapse_port_db`.

1
changelog.d/6248.misc Normal file
View file

@ -0,0 +1 @@
Move schema delta files to the correct data store.

View file

@ -339,7 +339,7 @@ def main(stdscr):
root_logger = logging.getLogger() root_logger = logging.getLogger()
formatter = logging.Formatter( formatter = logging.Formatter(
"%(asctime)s - %(name)s - %(lineno)d - " "%(levelname)s - %(message)s" "%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(message)s"
) )
if not os.path.exists("logs"): if not os.path.exists("logs"):
os.makedirs("logs") os.makedirs("logs")

View file

@ -36,7 +36,7 @@ def make_graph(db_name, room_id, file_prefix, limit):
args = [room_id] args = [room_id]
if limit: if limit:
sql += " ORDER BY topological_ordering DESC, stream_ordering DESC " "LIMIT ?" sql += " ORDER BY topological_ordering DESC, stream_ordering DESC LIMIT ?"
args.append(limit) args.append(limit)
@ -53,7 +53,7 @@ def make_graph(db_name, room_id, file_prefix, limit):
for event in events: for event in events:
c = conn.execute( c = conn.execute(
"SELECT state_group FROM event_to_state_groups " "WHERE event_id = ?", "SELECT state_group FROM event_to_state_groups WHERE event_id = ?",
(event.event_id,), (event.event_id,),
) )

View file

@ -86,7 +86,7 @@ pid_file: DATADIR/homeserver.pid
# Whether room invites to users on this server should be blocked # Whether room invites to users on this server should be blocked
# (except those sent by local server admins). The default is False. # (except those sent by local server admins). The default is False.
# #
#block_non_admin_invites: True #block_non_admin_invites: true
# Room searching # Room searching
# #
@ -239,9 +239,8 @@ listeners:
# Global blocking # Global blocking
# #
#hs_disabled: False #hs_disabled: false
#hs_disabled_message: 'Human readable reason for why the HS is blocked' #hs_disabled_message: 'Human readable reason for why the HS is blocked'
#hs_disabled_limit_type: 'error code(str), to help clients decode reason'
# Monthly Active User Blocking # Monthly Active User Blocking
# #
@ -261,15 +260,22 @@ listeners:
# sign up in a short space of time never to return after their initial # sign up in a short space of time never to return after their initial
# session. # session.
# #
#limit_usage_by_mau: False # 'mau_limit_alerting' is a means of limiting client side alerting
# should the mau limit be reached. This is useful for small instances
# where the admin has 5 mau seats (say) for 5 specific people and no
# interest increasing the mau limit further. Defaults to True, which
# means that alerting is enabled
#
#limit_usage_by_mau: false
#max_mau_value: 50 #max_mau_value: 50
#mau_trial_days: 2 #mau_trial_days: 2
#mau_limit_alerting: false
# If enabled, the metrics for the number of monthly active users will # If enabled, the metrics for the number of monthly active users will
# be populated, however no one will be limited. If limit_usage_by_mau # be populated, however no one will be limited. If limit_usage_by_mau
# is true, this is implied to be true. # is true, this is implied to be true.
# #
#mau_stats_only: False #mau_stats_only: false
# Sometimes the server admin will want to ensure certain accounts are # Sometimes the server admin will want to ensure certain accounts are
# never blocked by mau checking. These accounts are specified here. # never blocked by mau checking. These accounts are specified here.
@ -294,7 +300,7 @@ listeners:
# #
# Uncomment the below lines to enable: # Uncomment the below lines to enable:
#limit_remote_rooms: #limit_remote_rooms:
# enabled: True # enabled: true
# complexity: 1.0 # complexity: 1.0
# complexity_error: "This room is too complex." # complexity_error: "This room is too complex."
@ -411,7 +417,7 @@ acme:
# ACME support is disabled by default. Set this to `true` and uncomment # ACME support is disabled by default. Set this to `true` and uncomment
# tls_certificate_path and tls_private_key_path above to enable it. # tls_certificate_path and tls_private_key_path above to enable it.
# #
enabled: False enabled: false
# Endpoint to use to request certificates. If you only want to test, # Endpoint to use to request certificates. If you only want to test,
# use Let's Encrypt's staging url: # use Let's Encrypt's staging url:
@ -786,7 +792,7 @@ uploads_path: "DATADIR/uploads"
# connect to arbitrary endpoints without having first signed up for a # connect to arbitrary endpoints without having first signed up for a
# valid account (e.g. by passing a CAPTCHA). # valid account (e.g. by passing a CAPTCHA).
# #
#turn_allow_guests: True #turn_allow_guests: true
## Registration ## ## Registration ##
@ -829,7 +835,7 @@ uploads_path: "DATADIR/uploads"
# where d is equal to 10% of the validity period. # where d is equal to 10% of the validity period.
# #
#account_validity: #account_validity:
# enabled: True # enabled: true
# period: 6w # period: 6w
# renew_at: 1w # renew_at: 1w
# renew_email_subject: "Renew your %(app)s account" # renew_email_subject: "Renew your %(app)s account"
@ -971,7 +977,7 @@ account_threepid_delegates:
# Enable collection and rendering of performance metrics # Enable collection and rendering of performance metrics
# #
#enable_metrics: False #enable_metrics: false
# Enable sentry integration # Enable sentry integration
# NOTE: While attempts are made to ensure that the logs don't contain # NOTE: While attempts are made to ensure that the logs don't contain
@ -1023,7 +1029,7 @@ metrics_flags:
# Uncomment to enable tracking of application service IP addresses. Implicitly # Uncomment to enable tracking of application service IP addresses. Implicitly
# enables MAU tracking for application service users. # enables MAU tracking for application service users.
# #
#track_appservice_user_ips: True #track_appservice_user_ips: true
# a secret which is used to sign access tokens. If none is specified, # a secret which is used to sign access tokens. If none is specified,
@ -1149,7 +1155,7 @@ saml2_config:
# - url: https://our_idp/metadata.xml # - url: https://our_idp/metadata.xml
# #
# # By default, the user has to go to our login page first. If you'd like # # By default, the user has to go to our login page first. If you'd like
# # to allow IdP-initiated login, set 'allow_unsolicited: True' in a # # to allow IdP-initiated login, set 'allow_unsolicited: true' in a
# # 'service.sp' section: # # 'service.sp' section:
# # # #
# #service: # #service:
@ -1263,13 +1269,13 @@ password_config:
# smtp_port: 25 # SSL: 465, STARTTLS: 587 # smtp_port: 25 # SSL: 465, STARTTLS: 587
# smtp_user: "exampleusername" # smtp_user: "exampleusername"
# smtp_pass: "examplepassword" # smtp_pass: "examplepassword"
# require_transport_security: False # require_transport_security: false
# notif_from: "Your Friendly %(app)s Home Server <noreply@example.com>" # notif_from: "Your Friendly %(app)s Home Server <noreply@example.com>"
# app_name: Matrix # app_name: Matrix
# #
# # Enable email notifications by default # # Enable email notifications by default
# # # #
# notif_for_new_users: True # notif_for_new_users: true
# #
# # Defining a custom URL for Riot is only needed if email notifications # # Defining a custom URL for Riot is only needed if email notifications
# # should contain links to a self-hosted installation of Riot; when set # # should contain links to a self-hosted installation of Riot; when set
@ -1447,11 +1453,11 @@ password_config:
# body: >- # body: >-
# To continue using this homeserver you must review and agree to the # To continue using this homeserver you must review and agree to the
# terms and conditions at %(consent_uri)s # terms and conditions at %(consent_uri)s
# send_server_notice_to_guests: True # send_server_notice_to_guests: true
# block_events_error: >- # block_events_error: >-
# To continue using this homeserver you must review and agree to the # To continue using this homeserver you must review and agree to the
# terms and conditions at %(consent_uri)s # terms and conditions at %(consent_uri)s
# require_at_registration: False # require_at_registration: false
# policy_name: Privacy Policy # policy_name: Privacy Policy
# #

9
scripts-dev/config-lint.sh Executable file
View file

@ -0,0 +1,9 @@
#!/bin/bash
# Find linting errors in Synapse's default config file.
# Exits with 0 if there are no problems, or another code otherwise.
# Fix non-lowercase true/false values
sed -i -E "s/: +True/: true/g; s/: +False/: false/g;" docs/sample_config.yaml
# Check if anything changed
git diff --exit-code docs/sample_config.yaml

View file

@ -10,3 +10,4 @@ set -e
isort -y -rc synapse tests scripts-dev scripts isort -y -rc synapse tests scripts-dev scripts
flake8 synapse tests flake8 synapse tests
python3 -m black synapse tests scripts-dev scripts python3 -m black synapse tests scripts-dev scripts
./scripts-dev/config-lint.sh

View file

@ -2,6 +2,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# Copyright 2015, 2016 OpenMarket Ltd # Copyright 2015, 2016 OpenMarket Ltd
# Copyright 2018 New Vector Ltd # Copyright 2018 New Vector Ltd
# Copyright 2019 The Matrix.org Foundation C.I.C.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
@ -29,9 +30,33 @@ import yaml
from twisted.enterprise import adbapi from twisted.enterprise import adbapi
from twisted.internet import defer, reactor from twisted.internet import defer, reactor
from synapse.storage._base import LoggingTransaction, SQLBaseStore from synapse.config.homeserver import HomeServerConfig
from synapse.logging.context import PreserveLoggingContext
from synapse.storage._base import LoggingTransaction
from synapse.storage.data_stores.main.client_ips import ClientIpBackgroundUpdateStore
from synapse.storage.data_stores.main.deviceinbox import (
DeviceInboxBackgroundUpdateStore,
)
from synapse.storage.data_stores.main.devices import DeviceBackgroundUpdateStore
from synapse.storage.data_stores.main.events_bg_updates import (
EventsBackgroundUpdatesStore,
)
from synapse.storage.data_stores.main.media_repository import (
MediaRepositoryBackgroundUpdateStore,
)
from synapse.storage.data_stores.main.registration import (
RegistrationBackgroundUpdateStore,
)
from synapse.storage.data_stores.main.roommember import RoomMemberBackgroundUpdateStore
from synapse.storage.data_stores.main.search import SearchBackgroundUpdateStore
from synapse.storage.data_stores.main.state import StateBackgroundUpdateStore
from synapse.storage.data_stores.main.stats import StatsStore
from synapse.storage.data_stores.main.user_directory import (
UserDirectoryBackgroundUpdateStore,
)
from synapse.storage.engines import create_engine from synapse.storage.engines import create_engine
from synapse.storage.prepare_database import prepare_database from synapse.storage.prepare_database import prepare_database
from synapse.util import Clock
logger = logging.getLogger("synapse_port_db") logger = logging.getLogger("synapse_port_db")
@ -43,6 +68,7 @@ BOOLEAN_COLUMNS = {
"presence_list": ["accepted"], "presence_list": ["accepted"],
"presence_stream": ["currently_active"], "presence_stream": ["currently_active"],
"public_room_list_stream": ["visibility"], "public_room_list_stream": ["visibility"],
"devices": ["hidden"],
"device_lists_outbound_pokes": ["sent"], "device_lists_outbound_pokes": ["sent"],
"users_who_share_rooms": ["share_private"], "users_who_share_rooms": ["share_private"],
"groups": ["is_public"], "groups": ["is_public"],
@ -98,33 +124,24 @@ APPEND_ONLY_TABLES = [
end_error_exec_info = None end_error_exec_info = None
class Store(object): class Store(
"""This object is used to pull out some of the convenience API from the ClientIpBackgroundUpdateStore,
Storage layer. DeviceInboxBackgroundUpdateStore,
DeviceBackgroundUpdateStore,
*All* database interactions should go through this object. EventsBackgroundUpdatesStore,
""" MediaRepositoryBackgroundUpdateStore,
RegistrationBackgroundUpdateStore,
def __init__(self, db_pool, engine): RoomMemberBackgroundUpdateStore,
self.db_pool = db_pool SearchBackgroundUpdateStore,
self.database_engine = engine StateBackgroundUpdateStore,
UserDirectoryBackgroundUpdateStore,
_simple_insert_txn = SQLBaseStore.__dict__["_simple_insert_txn"] StatsStore,
_simple_insert = SQLBaseStore.__dict__["_simple_insert"] ):
def __init__(self, db_conn, hs):
_simple_select_onecol_txn = SQLBaseStore.__dict__["_simple_select_onecol_txn"] super().__init__(db_conn, hs)
_simple_select_onecol = SQLBaseStore.__dict__["_simple_select_onecol"] self.db_pool = hs.get_db_pool()
_simple_select_one = SQLBaseStore.__dict__["_simple_select_one"]
_simple_select_one_txn = SQLBaseStore.__dict__["_simple_select_one_txn"]
_simple_select_one_onecol = SQLBaseStore.__dict__["_simple_select_one_onecol"]
_simple_select_one_onecol_txn = SQLBaseStore.__dict__[
"_simple_select_one_onecol_txn"
]
_simple_update_one = SQLBaseStore.__dict__["_simple_update_one"]
_simple_update_one_txn = SQLBaseStore.__dict__["_simple_update_one_txn"]
_simple_update_txn = SQLBaseStore.__dict__["_simple_update_txn"]
@defer.inlineCallbacks
def runInteraction(self, desc, func, *args, **kwargs): def runInteraction(self, desc, func, *args, **kwargs):
def r(conn): def r(conn):
try: try:
@ -150,7 +167,8 @@ class Store(object):
logger.debug("[TXN FAIL] {%s} %s", desc, e) logger.debug("[TXN FAIL] {%s} %s", desc, e)
raise raise
return self.db_pool.runWithConnection(r) with PreserveLoggingContext():
return (yield self.db_pool.runWithConnection(r))
def execute(self, f, *args, **kwargs): def execute(self, f, *args, **kwargs):
return self.runInteraction(f.__name__, f, *args, **kwargs) return self.runInteraction(f.__name__, f, *args, **kwargs)
@ -176,6 +194,25 @@ class Store(object):
raise raise
class MockHomeserver:
def __init__(self, config, database_engine, db_conn, db_pool):
self.database_engine = database_engine
self.db_conn = db_conn
self.db_pool = db_pool
self.clock = Clock(reactor)
self.config = config
self.hostname = config.server_name
def get_db_conn(self):
return self.db_conn
def get_db_pool(self):
return self.db_pool
def get_clock(self):
return self.clock
class Porter(object): class Porter(object):
def __init__(self, **kwargs): def __init__(self, **kwargs):
self.__dict__.update(kwargs) self.__dict__.update(kwargs)
@ -447,31 +484,75 @@ class Porter(object):
db_conn.commit() db_conn.commit()
return db_conn
@defer.inlineCallbacks
def build_db_store(self, config):
"""Builds and returns a database store using the provided configuration.
Args:
config: The database configuration, i.e. a dict following the structure of
the "database" section of Synapse's configuration file.
Returns:
The built Store object.
"""
engine = create_engine(config)
self.progress.set_state("Preparing %s" % config["name"])
conn = self.setup_db(config, engine)
db_pool = adbapi.ConnectionPool(
config["name"], **config["args"]
)
hs = MockHomeserver(self.hs_config, engine, conn, db_pool)
store = Store(conn, hs)
yield store.runInteraction(
"%s_engine.check_database" % config["name"],
engine.check_database,
)
return store
@defer.inlineCallbacks
def run_background_updates_on_postgres(self):
# Manually apply all background updates on the PostgreSQL database.
postgres_ready = yield self.postgres_store.has_completed_background_updates()
if not postgres_ready:
# Only say that we're running background updates when there are background
# updates to run.
self.progress.set_state("Running background updates on PostgreSQL")
while not postgres_ready:
yield self.postgres_store.do_next_background_update(100)
postgres_ready = yield (
self.postgres_store.has_completed_background_updates()
)
@defer.inlineCallbacks @defer.inlineCallbacks
def run(self): def run(self):
try: try:
sqlite_db_pool = adbapi.ConnectionPool( self.sqlite_store = yield self.build_db_store(self.sqlite_config)
self.sqlite_config["name"], **self.sqlite_config["args"]
# Check if all background updates are done, abort if not.
updates_complete = yield self.sqlite_store.has_completed_background_updates()
if not updates_complete:
sys.stderr.write(
"Pending background updates exist in the SQLite3 database."
" Please start Synapse again and wait until every update has finished"
" before running this script.\n"
)
defer.returnValue(None)
self.postgres_store = yield self.build_db_store(
self.hs_config.database_config
) )
postgres_db_pool = adbapi.ConnectionPool( yield self.run_background_updates_on_postgres()
self.postgres_config["name"], **self.postgres_config["args"]
)
sqlite_engine = create_engine(sqlite_config)
postgres_engine = create_engine(postgres_config)
self.sqlite_store = Store(sqlite_db_pool, sqlite_engine)
self.postgres_store = Store(postgres_db_pool, postgres_engine)
yield self.postgres_store.execute(postgres_engine.check_database)
# Step 1. Set up databases.
self.progress.set_state("Preparing SQLite3")
self.setup_db(sqlite_config, sqlite_engine)
self.progress.set_state("Preparing PostgreSQL")
self.setup_db(postgres_config, postgres_engine)
self.progress.set_state("Creating port tables") self.progress.set_state("Creating port tables")
@ -563,6 +644,8 @@ class Porter(object):
def conv(j, col): def conv(j, col):
if j in bool_cols: if j in bool_cols:
return bool(col) return bool(col)
if isinstance(col, bytes):
return bytearray(col)
elif isinstance(col, string_types) and "\0" in col: elif isinstance(col, string_types) and "\0" in col:
logger.warn( logger.warn(
"DROPPING ROW: NUL value in table %s col %s: %r", "DROPPING ROW: NUL value in table %s col %s: %r",
@ -926,18 +1009,24 @@ if __name__ == "__main__":
}, },
} }
postgres_config = yaml.safe_load(args.postgres_config) hs_config = yaml.safe_load(args.postgres_config)
if "database" in postgres_config: if "database" not in hs_config:
postgres_config = postgres_config["database"] sys.stderr.write("The configuration file must have a 'database' section.\n")
sys.exit(4)
postgres_config = hs_config["database"]
if "name" not in postgres_config: if "name" not in postgres_config:
sys.stderr.write("Malformed database config: no 'name'") sys.stderr.write("Malformed database config: no 'name'\n")
sys.exit(2) sys.exit(2)
if postgres_config["name"] != "psycopg2": if postgres_config["name"] != "psycopg2":
sys.stderr.write("Database must use 'psycopg2' connector.") sys.stderr.write("Database must use the 'psycopg2' connector.\n")
sys.exit(3) sys.exit(3)
config = HomeServerConfig()
config.parse_config_dict(hs_config, "", "")
def start(stdscr=None): def start(stdscr=None):
if stdscr: if stdscr:
progress = CursesProgress(stdscr) progress = CursesProgress(stdscr)
@ -946,9 +1035,9 @@ if __name__ == "__main__":
porter = Porter( porter = Porter(
sqlite_config=sqlite_config, sqlite_config=sqlite_config,
postgres_config=postgres_config,
progress=progress, progress=progress,
batch_size=args.batch_size, batch_size=args.batch_size,
hs_config=config,
) )
reactor.callWhenRunning(porter.run) reactor.callWhenRunning(porter.run)

View file

@ -36,7 +36,7 @@ try:
except ImportError: except ImportError:
pass pass
__version__ = "1.4.1" __version__ = "1.5.0rc1"
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)): if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
# We import here so that we don't have to install a bunch of deps when # We import here so that we don't have to install a bunch of deps when

View file

@ -25,7 +25,13 @@ from twisted.internet import defer
import synapse.logging.opentracing as opentracing import synapse.logging.opentracing as opentracing
import synapse.types import synapse.types
from synapse import event_auth from synapse import event_auth
from synapse.api.constants import EventTypes, JoinRules, Membership, UserTypes from synapse.api.constants import (
EventTypes,
JoinRules,
LimitBlockingTypes,
Membership,
UserTypes,
)
from synapse.api.errors import ( from synapse.api.errors import (
AuthError, AuthError,
Codes, Codes,
@ -726,7 +732,7 @@ class Auth(object):
self.hs.config.hs_disabled_message, self.hs.config.hs_disabled_message,
errcode=Codes.RESOURCE_LIMIT_EXCEEDED, errcode=Codes.RESOURCE_LIMIT_EXCEEDED,
admin_contact=self.hs.config.admin_contact, admin_contact=self.hs.config.admin_contact,
limit_type=self.hs.config.hs_disabled_limit_type, limit_type=LimitBlockingTypes.HS_DISABLED,
) )
if self.hs.config.limit_usage_by_mau is True: if self.hs.config.limit_usage_by_mau is True:
assert not (user_id and threepid) assert not (user_id and threepid)
@ -759,5 +765,5 @@ class Auth(object):
"Monthly Active User Limit Exceeded", "Monthly Active User Limit Exceeded",
admin_contact=self.hs.config.admin_contact, admin_contact=self.hs.config.admin_contact,
errcode=Codes.RESOURCE_LIMIT_EXCEEDED, errcode=Codes.RESOURCE_LIMIT_EXCEEDED,
limit_type="monthly_active_user", limit_type=LimitBlockingTypes.MONTHLY_ACTIVE_USER,
) )

View file

@ -131,3 +131,10 @@ class RelationTypes(object):
ANNOTATION = "m.annotation" ANNOTATION = "m.annotation"
REPLACE = "m.replace" REPLACE = "m.replace"
REFERENCE = "m.reference" REFERENCE = "m.reference"
class LimitBlockingTypes(object):
"""Reasons that a server may be blocked"""
MONTHLY_ACTIVE_USER = "monthly_active_user"
HS_DISABLED = "hs_disabled"

View file

@ -56,8 +56,8 @@ from synapse.rest.client.v1.room import (
RoomStateEventRestServlet, RoomStateEventRestServlet,
) )
from synapse.server import HomeServer from synapse.server import HomeServer
from synapse.storage.data_stores.main.user_directory import UserDirectoryStore
from synapse.storage.engines import create_engine from synapse.storage.engines import create_engine
from synapse.storage.user_directory import UserDirectoryStore
from synapse.util.httpresourcetree import create_resource_tree from synapse.util.httpresourcetree import create_resource_tree
from synapse.util.manhole import manhole from synapse.util.manhole import manhole
from synapse.util.versionstring import get_version_string from synapse.util.versionstring import get_version_string

View file

@ -39,8 +39,8 @@ from synapse.replication.tcp.client import ReplicationClientHandler
from synapse.rest.admin import register_servlets_for_media_repo from synapse.rest.admin import register_servlets_for_media_repo
from synapse.rest.media.v0.content_repository import ContentRepoResource from synapse.rest.media.v0.content_repository import ContentRepoResource
from synapse.server import HomeServer from synapse.server import HomeServer
from synapse.storage.data_stores.main.media_repository import MediaRepositoryStore
from synapse.storage.engines import create_engine from synapse.storage.engines import create_engine
from synapse.storage.media_repository import MediaRepositoryStore
from synapse.util.httpresourcetree import create_resource_tree from synapse.util.httpresourcetree import create_resource_tree
from synapse.util.manhole import manhole from synapse.util.manhole import manhole
from synapse.util.versionstring import get_version_string from synapse.util.versionstring import get_version_string

View file

@ -54,8 +54,8 @@ from synapse.rest.client.v1.initial_sync import InitialSyncRestServlet
from synapse.rest.client.v1.room import RoomInitialSyncRestServlet from synapse.rest.client.v1.room import RoomInitialSyncRestServlet
from synapse.rest.client.v2_alpha import sync from synapse.rest.client.v2_alpha import sync
from synapse.server import HomeServer from synapse.server import HomeServer
from synapse.storage.data_stores.main.presence import UserPresenceState
from synapse.storage.engines import create_engine from synapse.storage.engines import create_engine
from synapse.storage.presence import UserPresenceState
from synapse.util.httpresourcetree import create_resource_tree from synapse.util.httpresourcetree import create_resource_tree
from synapse.util.manhole import manhole from synapse.util.manhole import manhole
from synapse.util.stringutils import random_string from synapse.util.stringutils import random_string

View file

@ -42,8 +42,8 @@ from synapse.replication.tcp.streams.events import (
) )
from synapse.rest.client.v2_alpha import user_directory from synapse.rest.client.v2_alpha import user_directory
from synapse.server import HomeServer from synapse.server import HomeServer
from synapse.storage.data_stores.main.user_directory import UserDirectoryStore
from synapse.storage.engines import create_engine from synapse.storage.engines import create_engine
from synapse.storage.user_directory import UserDirectoryStore
from synapse.util.caches.stream_change_cache import StreamChangeCache from synapse.util.caches.stream_change_cache import StreamChangeCache
from synapse.util.httpresourcetree import create_resource_tree from synapse.util.httpresourcetree import create_resource_tree
from synapse.util.manhole import manhole from synapse.util.manhole import manhole

View file

@ -48,7 +48,7 @@ class AppServiceConfig(Config):
# Uncomment to enable tracking of application service IP addresses. Implicitly # Uncomment to enable tracking of application service IP addresses. Implicitly
# enables MAU tracking for application service users. # enables MAU tracking for application service users.
# #
#track_appservice_user_ips: True #track_appservice_user_ips: true
""" """

View file

@ -62,11 +62,11 @@ DEFAULT_CONFIG = """\
# body: >- # body: >-
# To continue using this homeserver you must review and agree to the # To continue using this homeserver you must review and agree to the
# terms and conditions at %(consent_uri)s # terms and conditions at %(consent_uri)s
# send_server_notice_to_guests: True # send_server_notice_to_guests: true
# block_events_error: >- # block_events_error: >-
# To continue using this homeserver you must review and agree to the # To continue using this homeserver you must review and agree to the
# terms and conditions at %(consent_uri)s # terms and conditions at %(consent_uri)s
# require_at_registration: False # require_at_registration: false
# policy_name: Privacy Policy # policy_name: Privacy Policy
# #
""" """

View file

@ -304,13 +304,13 @@ class EmailConfig(Config):
# smtp_port: 25 # SSL: 465, STARTTLS: 587 # smtp_port: 25 # SSL: 465, STARTTLS: 587
# smtp_user: "exampleusername" # smtp_user: "exampleusername"
# smtp_pass: "examplepassword" # smtp_pass: "examplepassword"
# require_transport_security: False # require_transport_security: false
# notif_from: "Your Friendly %(app)s Home Server <noreply@example.com>" # notif_from: "Your Friendly %(app)s Home Server <noreply@example.com>"
# app_name: Matrix # app_name: Matrix
# #
# # Enable email notifications by default # # Enable email notifications by default
# # # #
# notif_for_new_users: True # notif_for_new_users: true
# #
# # Defining a custom URL for Riot is only needed if email notifications # # Defining a custom URL for Riot is only needed if email notifications
# # should contain links to a self-hosted installation of Riot; when set # # should contain links to a self-hosted installation of Riot; when set

View file

@ -70,7 +70,7 @@ class MetricsConfig(Config):
# Enable collection and rendering of performance metrics # Enable collection and rendering of performance metrics
# #
#enable_metrics: False #enable_metrics: false
# Enable sentry integration # Enable sentry integration
# NOTE: While attempts are made to ensure that the logs don't contain # NOTE: While attempts are made to ensure that the logs don't contain

View file

@ -180,7 +180,7 @@ class RegistrationConfig(Config):
# where d is equal to 10%% of the validity period. # where d is equal to 10%% of the validity period.
# #
#account_validity: #account_validity:
# enabled: True # enabled: true
# period: 6w # period: 6w
# renew_at: 1w # renew_at: 1w
# renew_email_subject: "Renew your %%(app)s account" # renew_email_subject: "Renew your %%(app)s account"

View file

@ -176,7 +176,7 @@ class SAML2Config(Config):
# - url: https://our_idp/metadata.xml # - url: https://our_idp/metadata.xml
# #
# # By default, the user has to go to our login page first. If you'd like # # By default, the user has to go to our login page first. If you'd like
# # to allow IdP-initiated login, set 'allow_unsolicited: True' in a # # to allow IdP-initiated login, set 'allow_unsolicited: true' in a
# # 'service.sp' section: # # 'service.sp' section:
# # # #
# #service: # #service:

View file

@ -171,6 +171,7 @@ class ServerConfig(Config):
) )
self.mau_trial_days = config.get("mau_trial_days", 0) self.mau_trial_days = config.get("mau_trial_days", 0)
self.mau_limit_alerting = config.get("mau_limit_alerting", True)
# How long to keep redacted events in the database in unredacted form # How long to keep redacted events in the database in unredacted form
# before redacting them. # before redacting them.
@ -192,7 +193,6 @@ class ServerConfig(Config):
# Options to disable HS # Options to disable HS
self.hs_disabled = config.get("hs_disabled", False) self.hs_disabled = config.get("hs_disabled", False)
self.hs_disabled_message = config.get("hs_disabled_message", "") self.hs_disabled_message = config.get("hs_disabled_message", "")
self.hs_disabled_limit_type = config.get("hs_disabled_limit_type", "")
# Admin uri to direct users at should their instance become blocked # Admin uri to direct users at should their instance become blocked
# due to resource constraints # due to resource constraints
@ -532,7 +532,7 @@ class ServerConfig(Config):
# Whether room invites to users on this server should be blocked # Whether room invites to users on this server should be blocked
# (except those sent by local server admins). The default is False. # (except those sent by local server admins). The default is False.
# #
#block_non_admin_invites: True #block_non_admin_invites: true
# Room searching # Room searching
# #
@ -673,9 +673,8 @@ class ServerConfig(Config):
# Global blocking # Global blocking
# #
#hs_disabled: False #hs_disabled: false
#hs_disabled_message: 'Human readable reason for why the HS is blocked' #hs_disabled_message: 'Human readable reason for why the HS is blocked'
#hs_disabled_limit_type: 'error code(str), to help clients decode reason'
# Monthly Active User Blocking # Monthly Active User Blocking
# #
@ -695,15 +694,22 @@ class ServerConfig(Config):
# sign up in a short space of time never to return after their initial # sign up in a short space of time never to return after their initial
# session. # session.
# #
#limit_usage_by_mau: False # 'mau_limit_alerting' is a means of limiting client side alerting
# should the mau limit be reached. This is useful for small instances
# where the admin has 5 mau seats (say) for 5 specific people and no
# interest increasing the mau limit further. Defaults to True, which
# means that alerting is enabled
#
#limit_usage_by_mau: false
#max_mau_value: 50 #max_mau_value: 50
#mau_trial_days: 2 #mau_trial_days: 2
#mau_limit_alerting: false
# If enabled, the metrics for the number of monthly active users will # If enabled, the metrics for the number of monthly active users will
# be populated, however no one will be limited. If limit_usage_by_mau # be populated, however no one will be limited. If limit_usage_by_mau
# is true, this is implied to be true. # is true, this is implied to be true.
# #
#mau_stats_only: False #mau_stats_only: false
# Sometimes the server admin will want to ensure certain accounts are # Sometimes the server admin will want to ensure certain accounts are
# never blocked by mau checking. These accounts are specified here. # never blocked by mau checking. These accounts are specified here.
@ -728,7 +734,7 @@ class ServerConfig(Config):
# #
# Uncomment the below lines to enable: # Uncomment the below lines to enable:
#limit_remote_rooms: #limit_remote_rooms:
# enabled: True # enabled: true
# complexity: 1.0 # complexity: 1.0
# complexity_error: "This room is too complex." # complexity_error: "This room is too complex."

View file

@ -289,6 +289,9 @@ class TlsConfig(Config):
"http://localhost:8009/.well-known/acme-challenge" "http://localhost:8009/.well-known/acme-challenge"
) )
# flake8 doesn't recognise that variables are used in the below string
_ = tls_enabled, proxypassline, acme_enabled, default_acme_account_file
return ( return (
"""\ """\
## TLS ## ## TLS ##
@ -451,7 +454,11 @@ class TlsConfig(Config):
#tls_fingerprints: [{"sha256": "<base64_encoded_sha256_fingerprint>"}] #tls_fingerprints: [{"sha256": "<base64_encoded_sha256_fingerprint>"}]
""" """
% locals() # Lowercase the string representation of boolean values
% {
x[0]: str(x[1]).lower() if isinstance(x[1], bool) else x[1]
for x in locals().items()
}
) )
def read_tls_certificate(self): def read_tls_certificate(self):

View file

@ -56,5 +56,5 @@ class VoipConfig(Config):
# connect to arbitrary endpoints without having first signed up for a # connect to arbitrary endpoints without having first signed up for a
# valid account (e.g. by passing a CAPTCHA). # valid account (e.g. by passing a CAPTCHA).
# #
#turn_allow_guests: True #turn_allow_guests: true
""" """

View file

@ -493,8 +493,7 @@ def _check_power_levels(event, auth_events):
new_level_too_big = new_level is not None and new_level > user_level new_level_too_big = new_level is not None and new_level > user_level
if old_level_too_big or new_level_too_big: if old_level_too_big or new_level_too_big:
raise AuthError( raise AuthError(
403, 403, "You don't have permission to add ops level greater than your own"
"You don't have permission to add ops level greater " "than your own",
) )

View file

@ -30,7 +30,7 @@ from synapse.federation.units import Edu
from synapse.handlers.presence import format_user_presence_state from synapse.handlers.presence import format_user_presence_state
from synapse.metrics import sent_transactions_counter from synapse.metrics import sent_transactions_counter
from synapse.metrics.background_process_metrics import run_as_background_process from synapse.metrics.background_process_metrics import run_as_background_process
from synapse.storage import UserPresenceState from synapse.storage.presence import UserPresenceState
from synapse.util.retryutils import NotRetryingDestination, get_retry_limiter from synapse.util.retryutils import NotRetryingDestination, get_retry_limiter
# This is defined in the Matrix spec and enforced by the receiver. # This is defined in the Matrix spec and enforced by the receiver.

View file

@ -16,8 +16,8 @@
from synapse.replication.slave.storage._base import BaseSlavedStore from synapse.replication.slave.storage._base import BaseSlavedStore
from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker
from synapse.storage.account_data import AccountDataWorkerStore from synapse.storage.data_stores.main.account_data import AccountDataWorkerStore
from synapse.storage.tags import TagsWorkerStore from synapse.storage.data_stores.main.tags import TagsWorkerStore
class SlavedAccountDataStore(TagsWorkerStore, AccountDataWorkerStore, BaseSlavedStore): class SlavedAccountDataStore(TagsWorkerStore, AccountDataWorkerStore, BaseSlavedStore):

View file

@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.appservice import ( from synapse.storage.data_stores.main.appservice import (
ApplicationServiceTransactionWorkerStore, ApplicationServiceTransactionWorkerStore,
ApplicationServiceWorkerStore, ApplicationServiceWorkerStore,
) )

View file

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.client_ips import LAST_SEEN_GRANULARITY from synapse.storage.data_stores.main.client_ips import LAST_SEEN_GRANULARITY
from synapse.util.caches import CACHE_SIZE_FACTOR from synapse.util.caches import CACHE_SIZE_FACTOR
from synapse.util.caches.descriptors import Cache from synapse.util.caches.descriptors import Cache

View file

@ -15,7 +15,7 @@
from synapse.replication.slave.storage._base import BaseSlavedStore from synapse.replication.slave.storage._base import BaseSlavedStore
from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker
from synapse.storage.deviceinbox import DeviceInboxWorkerStore from synapse.storage.data_stores.main.deviceinbox import DeviceInboxWorkerStore
from synapse.util.caches.expiringcache import ExpiringCache from synapse.util.caches.expiringcache import ExpiringCache
from synapse.util.caches.stream_change_cache import StreamChangeCache from synapse.util.caches.stream_change_cache import StreamChangeCache

View file

@ -15,8 +15,8 @@
from synapse.replication.slave.storage._base import BaseSlavedStore from synapse.replication.slave.storage._base import BaseSlavedStore
from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker
from synapse.storage.devices import DeviceWorkerStore from synapse.storage.data_stores.main.devices import DeviceWorkerStore
from synapse.storage.end_to_end_keys import EndToEndKeyWorkerStore from synapse.storage.data_stores.main.end_to_end_keys import EndToEndKeyWorkerStore
from synapse.util.caches.stream_change_cache import StreamChangeCache from synapse.util.caches.stream_change_cache import StreamChangeCache

View file

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.directory import DirectoryWorkerStore from synapse.storage.data_stores.main.directory import DirectoryWorkerStore
from ._base import BaseSlavedStore from ._base import BaseSlavedStore

View file

@ -20,15 +20,17 @@ from synapse.replication.tcp.streams.events import (
EventsStreamCurrentStateRow, EventsStreamCurrentStateRow,
EventsStreamEventRow, EventsStreamEventRow,
) )
from synapse.storage.event_federation import EventFederationWorkerStore from synapse.storage.data_stores.main.event_federation import EventFederationWorkerStore
from synapse.storage.event_push_actions import EventPushActionsWorkerStore from synapse.storage.data_stores.main.event_push_actions import (
from synapse.storage.events_worker import EventsWorkerStore EventPushActionsWorkerStore,
from synapse.storage.relations import RelationsWorkerStore )
from synapse.storage.roommember import RoomMemberWorkerStore from synapse.storage.data_stores.main.events_worker import EventsWorkerStore
from synapse.storage.signatures import SignatureWorkerStore from synapse.storage.data_stores.main.relations import RelationsWorkerStore
from synapse.storage.state import StateGroupWorkerStore from synapse.storage.data_stores.main.roommember import RoomMemberWorkerStore
from synapse.storage.stream import StreamWorkerStore from synapse.storage.data_stores.main.signatures import SignatureWorkerStore
from synapse.storage.user_erasure_store import UserErasureWorkerStore from synapse.storage.data_stores.main.state import StateGroupWorkerStore
from synapse.storage.data_stores.main.stream import StreamWorkerStore
from synapse.storage.data_stores.main.user_erasure_store import UserErasureWorkerStore
from ._base import BaseSlavedStore from ._base import BaseSlavedStore
from ._slaved_id_tracker import SlavedIdTracker from ._slaved_id_tracker import SlavedIdTracker

View file

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.filtering import FilteringStore from synapse.storage.data_stores.main.filtering import FilteringStore
from ._base import BaseSlavedStore from ._base import BaseSlavedStore

View file

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage import KeyStore from synapse.storage.data_stores.main.keys import KeyStore
# KeyStore isn't really safe to use from a worker, but for now we do so and hope that # KeyStore isn't really safe to use from a worker, but for now we do so and hope that
# the races it creates aren't too bad. # the races it creates aren't too bad.

View file

@ -14,7 +14,7 @@
# limitations under the License. # limitations under the License.
from synapse.storage import DataStore from synapse.storage import DataStore
from synapse.storage.presence import PresenceStore from synapse.storage.data_stores.main.presence import PresenceStore
from synapse.util.caches.stream_change_cache import StreamChangeCache from synapse.util.caches.stream_change_cache import StreamChangeCache
from ._base import BaseSlavedStore, __func__ from ._base import BaseSlavedStore, __func__

View file

@ -14,7 +14,7 @@
# limitations under the License. # limitations under the License.
from synapse.replication.slave.storage._base import BaseSlavedStore from synapse.replication.slave.storage._base import BaseSlavedStore
from synapse.storage.profile import ProfileWorkerStore from synapse.storage.data_stores.main.profile import ProfileWorkerStore
class SlavedProfileStore(ProfileWorkerStore, BaseSlavedStore): class SlavedProfileStore(ProfileWorkerStore, BaseSlavedStore):

View file

@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.push_rule import PushRulesWorkerStore from synapse.storage.data_stores.main.push_rule import PushRulesWorkerStore
from ._slaved_id_tracker import SlavedIdTracker from ._slaved_id_tracker import SlavedIdTracker
from .events import SlavedEventStore from .events import SlavedEventStore

View file

@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.pusher import PusherWorkerStore from synapse.storage.data_stores.main.pusher import PusherWorkerStore
from ._base import BaseSlavedStore from ._base import BaseSlavedStore
from ._slaved_id_tracker import SlavedIdTracker from ._slaved_id_tracker import SlavedIdTracker

View file

@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.receipts import ReceiptsWorkerStore from synapse.storage.data_stores.main.receipts import ReceiptsWorkerStore
from ._base import BaseSlavedStore from ._base import BaseSlavedStore
from ._slaved_id_tracker import SlavedIdTracker from ._slaved_id_tracker import SlavedIdTracker

View file

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.registration import RegistrationWorkerStore from synapse.storage.data_stores.main.registration import RegistrationWorkerStore
from ._base import BaseSlavedStore from ._base import BaseSlavedStore

View file

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.room import RoomWorkerStore from synapse.storage.data_stores.main.room import RoomWorkerStore
from ._base import BaseSlavedStore from ._base import BaseSlavedStore
from ._slaved_id_tracker import SlavedIdTracker from ._slaved_id_tracker import SlavedIdTracker

View file

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
from synapse.storage.transactions import TransactionStore from synapse.storage.data_stores.main.transactions import TransactionStore
from ._base import BaseSlavedStore from ._base import BaseSlavedStore

Some files were not shown because too many files have changed in this diff Show more