MirrorHub/synapse
0
0
Fork 1
mirror of https://mau.dev/maunium/synapse.git synced 2024-06-26 22:48:20 +02:00
Code Issues Projects Releases Wiki Activity
17486 commits 5 branches 702 tags 78 MiB
Commit graph

72 commits

Author SHA1 Message Date
Richard van der Hoff 8d6f97f932 Merge remote-tracking branch 'origin/release-v1.20.0' into develop 2020-09-08 09:58:07 +01:00
Richard van der Hoff 0dae7d80bf
Add more logging to debug slow startup (#8264) 
I'm hoping this will provide some pointers for debugging
https://github.com/matrix-org/synapse/issues/7968.
 2020-09-07 13:36:02 +01:00
Richard van der Hoff 96312536f2
Refuse to upgrade database on worker processes (#8266) 2020-09-07 13:04:10 +01:00
Richard van der Hoff 5b452df23b
Run database updates in a transaction (#8265) 
Fixes: #6467
 2020-09-07 11:41:50 +01:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Erik Johnston a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Erik Johnston 664409b169
Fix bug in account data replication stream. (#7656) 
* Ensure account data stream IDs are unique.

The account data stream is shared between three tables, and the maximum
allocated ID was tracked in a dedicated table. Updating the max ID
happened outside the transaction that allocated the ID, leading to a
race where if the server was restarted then the same ID could be
allocated but the max ID failed to be updated, leading it to be reused.

The ID generators have support for tracking across multiple tables, so
we may as well use that instead of a dedicated table.

* Fix bug in account data replication stream.

If the same stream ID was used in both global and room account data then
the getting updates for the replication stream would fail due to
`heapq.merge(..)` trying to compare a `str` with a `None`. (This is
because you'd have two rows like `(534, '!room')` and `(534, None)` from
the room and global account data tables).

Fix is just to order by stream ID, since we don't rely on the ordering
beyond that. The bug where stream IDs can be reused should be fixed now,
so this case shouldn't happen going forward.

Fixes #7617
 2020-06-09 16:28:57 +01:00
Andrew Morgan 0a6e837aaa
Fix incorrect placeholder syntax in database prepartion code (#7575) 
We were using `logger` syntax which isn't supported by `Exception`s.
 2020-05-27 16:26:59 +01:00
Richard van der Hoff edd9a7214c
Replace device_27_unique_idx bg update with a fg one (#7562) 
The bg update never managed to complete, because it kept being interrupted by
transactions which want to take a lock.

Just doing it in the foreground isn't that bad, and is a good deal simpler.
 2020-05-26 11:43:17 +01:00
Erik Johnston d7983b63a6
Support any process writing to cache invalidation stream. (#7436) 2020-05-07 13:51:08 +01:00
Richard van der Hoff 7b608cf468 Only run one background update at a time 2020-03-31 17:43:58 +01:00
Richard van der Hoff e66f099ca9
Sanity-check database before running upgrades (#6982) 
Some of the database deltas rely on `config.server_name` being set correctly,
so we should check that it is before running the deltas.

Fixes #6870.
 2020-02-25 17:46:00 +00:00
Patrick Cloke 509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) 
Ensure good comprehension hygiene using flake8-comprehensions.
 2020-02-21 07:15:07 -05:00
Erik Johnston 02b44db922
Warn if postgres database has non-C locale. (#6734) 
As using non-C locale can cause issues on upgrading OS.
 2020-01-28 13:44:21 +00:00
Erik Johnston 28c98e51ff
Add local_current_membership table (#6655) 
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
 2020-01-15 14:59:33 +00:00
Erik Johnston 75d8f26ac8
Split state groups into a separate data store (#6296) 2019-12-20 10:48:24 +00:00
Erik Johnston 0b5dbadd96
Explode on duplicate delta file names. (#6565) 2019-12-19 15:07:37 +00:00
Richard van der Hoff bca30cefee
Improve diagnostics on database upgrade failure (#6570) 
`Failed to upgrade database` is not helpful, and it's unlikely that UPGRADE.rst
has anything useful.
 2019-12-19 14:53:15 +00:00
Erik Johnston 2284eb3a53
Add database config class (#6513) 
This encapsulates config for a given database and is the way to get new
connections.
 2019-12-18 10:45:12 +00:00
Andrew Morgan 3916e1b97a
Clean up newline quote marks around the codebase (#6362) 2019-11-21 12:00:14 +00:00
Erik Johnston 23d62eded2 Clean up prepare_database.py a bit and add comments 2019-10-22 18:43:31 +01:00
Erik Johnston ffd24545bb Fix schema management to work with multiple data stores. 2019-10-21 16:08:40 +01:00
Olivier Wilkinson (reivilibre) d1e0b91083 Code style (Black) 
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
 2019-08-27 09:39:11 +01:00
Olivier Wilkinson (reivilibre) aefa76f5cd Allow schema deltas to be engine-specific 
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
 2019-08-27 09:14:00 +01:00
Erik Johnston 6de09e07a6 Add membership column to current_state_events table. 
It turns out that doing a join is surprisingly expensive for the DB to
do when room_membership table is larger than the disk cache.
 2019-07-18 14:15:57 +01:00
Richard van der Hoff 0e8b35f7b0
Fix "Unexpected entry in 'full_schemas'" log warning (#5509) 
There is a README.txt which always sets off this warning, which is a bit
alarming when you first start synapse. I don't think we need to warn about
this.
 2019-06-21 15:11:57 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Erik Johnston 1b8cb64393
Merge pull request #5320 from matrix-org/hawkowl/full-schema-v1 
Make a full SQL schema
 2019-06-06 18:35:38 +01:00
Andrew Morgan 3719680ee4
Add ability to perform password reset via email without trusting the identity server (#5377) 
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.

This PR is a culmination of 3 smaller PRs which have each been separately reviewed:

* #5308
* #5345
* #5368
 2019-06-06 17:34:07 +01:00
Amber Brown 7f81b967ca fix schemas 2019-06-03 22:23:40 +10:00
Amber Brown 37057d5d60 prepare 2019-06-03 22:02:47 +10:00
Neil Johnson e8419554ff
Remove presence lists (#4989) 
Remove presence list support as per MSC 1819
 2019-04-03 11:11:15 +01:00
Amber Brown 7efd1d87c2 Run black on the rest of the storage module (#4996) 2019-04-03 10:07:29 +01:00
Aaron Raimist 704c5298f0
Drop sent_transactions 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-12-01 23:07:35 -06:00
Richard van der Hoff 6c18cc4b50 Ignore __pycache__ directories in schema delta dir 
Now that we use py3, compiled python ends up in __pycache__ rather than *.pyc.
 2018-11-20 22:52:34 +00:00
Erik Johnston 4917ff5523 Add state_group index to event_to_state_groups 
This is needed to efficiently check for unreferenced state groups during
purge.
 2018-10-04 16:03:08 +01:00
Neil Johnson 6ef983ce5c api into monthly_active_users table 2018-07-31 16:36:24 +01:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Neil Johnson c379acd4fd bump version 2018-05-29 17:47:28 +01:00
Neil Johnson 617bf40924 Generate user daily stats 2018-04-25 17:37:29 +01:00
David Baker 79452edeee Add joinability for groups 
Adds API to set the 'joinable' flag, and corresponding flag in the
table.
 2018-03-28 14:03:37 +01:00
Erik Johnston 05f98a2224 Keep track of last access time for local media 2018-01-17 10:24:43 +00:00
Richard van der Hoff 1650eb5847 DB schema interface for password auth providers 
Provide an interface by which password auth providers can register db schema
files to be run at startup
 2017-10-31 14:01:53 +00:00
Luke Barnard cfa4e658e0 Bump schema version to 46 2017-10-26 17:23:49 +01:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:' 
what could possibly go wrong
 2017-10-23 15:52:32 +01:00
Erik Johnston c2c47550f9 Fix schema delta versions 2017-10-11 13:23:15 +01:00
Erik Johnston 9ccb4226ba Delete expired url cache data 2017-09-28 12:18:06 +01:00
Erik Johnston ebcd55d641 Add DB schema for tracking users who share rooms 2017-06-15 09:45:48 +01:00
Erik Johnston 6e614e9e10 Add background task to clear out old event_auth 2017-05-24 15:23:34 +01:00
Erik Johnston 00957d1aa4 User Cursor.__iter__ instead of fetchall 
This prevents unnecessary construction of lists
 2017-03-23 17:53:49 +00:00