MirrorHub/synapse
0
0
Fork 1
mirror of https://mau.dev/maunium/synapse.git synced 2024-05-17 02:53:46 +02:00
Code Issues Projects Releases Wiki Activity
23784 commits 5 branches 695 tags 74 MiB
Commit graph

290 commits

Author SHA1 Message Date
Dirk Klimpel 8d5325ec0c
Drop unused table presence (#14825) 2023-01-13 14:17:03 +00:00
Andrew Morgan f5ea9f2b1d
Add rust linting commands to scripts-dev/lint.sh (#14822) 2023-01-12 16:20:34 +00:00
Andrew Morgan c4456114e1
Add experimental support for MSC3391: deleting account data (#14714) 2023-01-01 03:40:46 +00:00
reivilibre 46993770e5
Suppress the update check in the ruff linter. (#14741) 
* Suppress update check in ruff

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
 2022-12-28 17:23:19 +00:00
Patrick Cloke 7010a3d015
Switch to ruff instead of flake8. (#14633) 
ruff is a flake8-compatible Python linter written in Rust.
It supports the flake8 plugins that we use and is significantly
faster in testing.
 2022-12-21 13:05:21 -05:00
David Robertson 3d87847ecc
Enable --warn-redundant-casts option in mypy (#14671) 
* Enable `--warn-redundant-casts` option in mypy

Doesn't do much but helps me sleep better at night.

* Changelog

* Fix name of the ignore

* Fix one more missed cast

Not sure why I didn't see this one locally, maybe I needed a poetry update

* Remove old comment

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2022-12-12 21:25:07 +00:00
reivilibre be3a8a85e3
Add --editable flag to complement.sh which uses an editable install of Synapse for faster turn-around times whilst developing iteratively. (#14548) 
Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
 2022-12-07 15:45:31 +00:00
Eric Eastwood 8f10c8b054
Move MSC3030 /timestamp_to_event endpoint to stable v1 location (#14471) 
Fix https://github.com/matrix-org/synapse/issues/14390

 - Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>`
 - Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>`

Complement test changes: https://github.com/matrix-org/complement/pull/559
 2022-11-28 15:54:18 -06:00
Richard van der Hoff 8d133a8464
Fixes to federation_client dev script (#14479) 
* Attempt to fix federation-client devscript handling of .well-known

The script was setting the wrong value in the Host header

* Fix TLS verification

Turns out that actually doing TLS verification isn't that hard. Let's enable
it.
 2022-11-20 17:41:17 +00:00
David Robertson 2cc592584a
Remove unused type-ignores (#14433) 
* Remove unused type-ignores

Oversights in #14427 and #14429.

* Changelog
 2022-11-14 13:46:29 +00:00
Sean Quah 0cf48f2d5f
Build Debian packages for Ubuntu 22.10 Kinetic Kudu (#14396) 
Signed-off-by: Sean Quah <seanq@matrix.org>
 2022-11-09 10:33:13 +00:00
realtyem 69814eb282
Allow override for requesting specific worker types for Complement on command line. (#14324) 
* Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested.

* Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint.

* Update docs to reflect this new ability.

* Changelog

* Don't rely on soft wrapping to format long strings

Good idea dklimpel. Thanks for catching that.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>

* Small nits just noticed in docs.

* Fixup new line in docs.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
 2022-11-08 12:34:09 +00:00
Patrick Cloke 59ca73006c
Enable testing MSC3874 in complement. (#14339) 2022-11-01 13:26:28 -04:00
Patrick Cloke c604d2c218
Mark /relations endpoint as usable on workers. (#14028) 
Co-authored-by: Eric Eastwood <erice@element.io>
 2022-10-12 06:46:13 -04:00
David Robertson 285d72556b
Update mypy and mypy-zope, attempt 3 (#13993) 
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2022-09-30 17:36:28 +01:00
David Robertson 8e52cb0bce
Revert "Update mypy and mypy-zope (#13925)" 
This reverts commit 6d543d6d9f.
 2022-09-30 16:37:48 +01:00
David Robertson 6d543d6d9f
Update mypy and mypy-zope (#13925) 
* Update mypy and mypy-zope

* Unignore assigning to LogRecord attributes

Presumably https://github.com/python/typeshed/pull/8064 makes this ok

Cherry-picked from #13521

* Remove unused ignores due to mypy ParamSpec fixes

https://github.com/python/mypy/pull/12668

Cherry-picked from #13521

* Remove additional unused ignores

* Fix new mypy complaints related to `assertGreater`

Presumably due to https://github.com/python/typeshed/pull/8077

* Changelog

* Reword changelog

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2022-09-30 16:34:47 +01:00
David Robertson 0a38c7ec6d
Snapshot schema 72 (#13873) 
Including another batch of fixes to the schema dump script
 2022-09-26 18:28:32 +01:00
Mathieu Velten 6bd8763804
Add cache invalidation across workers to module API (#13667) 
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
 2022-09-21 15:32:01 +02:00
reivilibre a35842caec
Fix the release script not publishing binary wheels. (#13850) 2022-09-21 09:43:08 +00:00
David Robertson fff9b955fa
Generate separate snapshots for logical databases (#13792) 
* Generate separate snapshots for sqlite, postgres and common
* Cleanup postgres dbs in the TRAP
* Say which logical DB we're applying updates to
* Run background updates on the state DB
* Add new option for accepting a SCHEMA_NUMBER
 2022-09-20 14:14:12 +01:00
David Robertson b60d47ab2c
Updates to the schema dump script (#13770) 2022-09-13 10:53:11 +01:00
reivilibre c7b18d9d44
Extend the release script to wait for GitHub Actions to finish and to be usable as a guide for the whole process. (#13483) 2022-09-05 12:16:59 +01:00
David Robertson ba8938b090
Reject non-strict types in Pydantic models (#13502) 2022-08-17 11:17:04 +01:00
reivilibre 1c910e2216
Add a merge-back command to the release script, which automates merging the correct branches after a release. (#13393) 2022-08-02 15:56:28 +00:00
reivilibre 05aeeb3a80
Enable Complement CI tests in the 'latest deps' test run. (#13213) 
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
 2022-08-01 10:55:31 +00:00
reivilibre 543dc9c93e
Extend the release script to automatically push a new SyTest branch, rather than having that be a manual process. (#12978) 
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2022-07-26 18:08:14 +01:00
Patrick Cloke 1efe6b8c41
Stop building Ubuntu 21.10 (Impish Indri) which is end of life. (#13326) 2022-07-19 09:08:46 -04:00
reivilibre 0c95313a44
Add --build-only option to complement.sh to prevent actually running Complement. (#13158) 2022-07-07 14:18:38 +00:00
reivilibre 65e675504f
Add the ability to set the log level using the SYNAPSE_TEST_LOG_LEVEL environment when using complement.sh. (#13152) 2022-07-05 09:46:20 +00:00
reivilibre c04e25789e
Enable Complement testing in the 'Twisted Trunk' CI runs. (#13079) 2022-07-01 15:42:49 +00:00
Richard van der Hoff fe910fb10e
complement.sh: Permit skipping docker build (#13143) 
Add a `-f` argument to `complement.sh` to skip the docker build
 2022-07-01 12:33:59 +00:00
Erik Johnston de334ac183
Add a CI job to check that schema deltas are in the correct folder. (#13063) 2022-06-15 16:27:18 +01:00
reivilibre 538044ac01
Collapse Docker build commands in Complement CI runs to make the logs easier to read. (#13058) 2022-06-15 14:42:27 +00:00
Erik Johnston aef398457f
Up complement time outs (#13048) 2022-06-14 17:59:06 +01:00
Patrick Cloke 1df22e16df
Fix example of running complement.sh. (#12990) 2022-06-09 07:21:15 -04:00
reivilibre 67f51c84f8
Merge the Complement testing Docker images into a single, multi-purpose image. (#12881) 
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2022-06-08 09:57:05 +00:00
David Robertson 81d9f2a8e9
Fixes to MSC3787 implementation (#12858) 2022-05-24 16:50:50 +00:00
reivilibre 7a68203cde
Disable 'faster room join' Complement tests when testing against Synapse with workers. (#12842) 2022-05-23 17:27:05 +01:00
David Robertson 5331fb5b47
allow on_invalidate=None in @cached methods (#12769) 2022-05-17 16:06:45 +00:00
Richard van der Hoff 34e84fee68
Tweaks to workers-under-complement (#12637) 
* Bump the HS startup timeout
* Log prefixes for more processes
* Bump the overall timeout
 2022-05-09 22:41:06 +01:00
Andrew Morgan c4514b97db
Add missing space before 'docker' link in release announcement script (#12612) 2022-05-03 14:46:42 +00:00
David Robertson bf2fea8f7d
Add sanity checks to the release script (#12556) 
Check we're on the right branch before tagging, and on the right tag before uploading

* Abort if we're on the wrong branch
* Check we have the right tag checked out
* Clarify that `publish` only releases to GitHub
 2022-05-03 10:50:03 +00:00
David Robertson 30c8e7e408
Make scripts-dev pass mypy --disallow-untyped-defs (#12356) 
Not enforced in config yet. One day.
 2022-04-27 13:10:31 +00:00
David Robertson 6b9e95015b
Lint the release script 2022-04-26 11:53:37 +01:00
David Robertson 730fcda546
Update release script to be poetry-aware 
Poetry now manages the project version in pyproject.toml.
 2022-04-26 11:14:27 +01:00
Shay 99ab45423a
build debian package for jammy jellyfish (#12543) 2022-04-26 10:34:59 +01:00
Richard van der Hoff f5668f0b4a
Await un-partial-stating after a partial-state join (#12399) 
When we join a room via the faster-joins mechanism, we end up with "partial
state" at some points on the event DAG. Many parts of the codebase need to
wait for the full state to load. So, we implement a mechanism to keep track of
which events have partial state, and wait for them to be fully-populated.
 2022-04-21 07:42:03 +01:00
David Robertson 09b4f6e46d
Remove leftover references to setup.py (#12514) 
* Remove leftover references to setup.py

Missed in #12478.

* Changelog
 2022-04-20 18:16:49 +00:00
Richard van der Hoff 798deb3a10
Fix typo in deb changelogs from release script (#12497) 
The release script used to incorrectly write `New synapse release 1.57.0~rc1.`
instead of `New synapse release 1.57.0rc1.`
 2022-04-19 16:41:52 +01:00
Jan Christian Grünhage a1f87f57ff
Implement MSC3383: include destination in X-Matrix auth header (#11398) 
Co-authored-by: Jan Christian Grünhage <jan.christian@gruenhage.xyz>
Co-authored-by: Marcus Hoffmann <bubu@bubu1.eu>
 2022-04-19 16:23:53 +01:00
David Robertson 535a689cfc
Reintroduce the lint targets in the linter script (#12455) 2022-04-14 11:33:06 +01:00
Erik Johnston ba1588461b
Fix typos in release script docs (#12450) 2022-04-12 15:12:57 +00:00
Richard van der Hoff 3cdf5a1386
Fix up healthcheck generation for workers docker image (#12405) 
This wasn't quite generating the right thing.
 2022-04-11 13:38:58 +00:00
Richard van der Hoff 5f72ea1bde
Move complement setup stuff into the Synapse repo (#12404) 
Fixes matrix-org/complement#330 (or it will, once we remove the old files).

It's not quite a lift-and-shift: I've also taken the opportunity to get rid of the custom CA that we used to use to sign the TLS certs, which has been superceded by the CA exposed by Complement.
 2022-04-11 11:39:28 +01:00
David Robertson dd5cc37aa4
Stop maintaining a list of lint targets in lint.sh (#12420) 2022-04-08 14:36:09 +01:00
David Robertson 80839a44f1
Remove more dead/broken dev scripts (#12355) 2022-04-03 21:21:08 +01:00
David Robertson 336bff1104
Burn check_signature dev script. (#12351) 2022-04-01 13:41:42 +00:00
David Robertson 21351820e0
Remove list_url_patterns dev script (#12349) 2022-04-01 13:05:21 +00:00
David Robertson 15cdcf8f30
Remove dockerfile-pgtests (#12336) 2022-03-31 11:30:07 +01:00
David Robertson a7fb66e800
Update dead links in check-newsfragment.sh (#12331) 2022-03-30 11:44:28 +00:00
Andrew Morgan 19a1d6a42a
Remove references to "msc2403" (#12165) 2022-03-30 11:15:47 +00:00
David Robertson 12d1f82db2
Generate announcement links in release script (#12242) 2022-03-17 13:46:05 +00:00
Richard van der Hoff 00a67f831a Merge remote-tracking branch 'origin/release-v1.54' into develop 2022-03-04 22:40:51 +00:00
David Robertson 4aeb00ca20
Move synctl into synapse._scripts and expose as an entrypoint (#12140) 2022-03-04 11:58:49 +00:00
Eric Eastwood a511a890d7
Enable MSC2716 Complement tests in Synapse (#12145) 
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2022-03-03 11:19:20 +00:00
Eric Eastwood 31b125ccec
Enable MSC3030 Complement tests in Synapse (#12144) 
The Complement tests for MSC3030 are now merged, https://github.com/matrix-org/complement/pull/178

Synapse implmentation: https://github.com/matrix-org/synapse/pull/9445
 2022-03-03 11:45:23 +01:00
David Robertson 1fbe0316a9
Add suffices to scripts in scripts-dev (#12137) 
* Rename scripts-dev to have suffices

* Update references to `scripts-dev`

* Changelog

* These scripts don't pass mypy
 2022-03-02 18:00:26 +00:00
Erik Johnston 6d282a9c89
Make release script write correct no-op changelog (#12127) 
As we want to include the previous version in the "No new changes..."
string.
 2022-03-02 14:28:18 +00:00
David Robertson f3f0ab10fe
Move scripts directory inside synapse, exposing as setuptools entry_points (#12118) 
* Two scripts are basically entry_points already
* Move and rename scripts/* to synapse/_scripts/*.py
* Delete sync_room_to_group.pl
* Expose entry points in setup.py
* Update linter script and config
* Fixup scripts & docs mentioning scripts that moved

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2022-03-02 13:00:16 +00:00
David Robertson 5f62a094de
Detox, part 1 of N (#12119) 
* Don't use `tox` for `check-sampleconfig`
* Don't use `tox` for check-newsfragment
 2022-03-01 19:47:02 +00:00
Nicolas Werner c1ac2a8135
Rename default branch of complement.sh to main (#12063) 
The complement.sh script relies on the name of the ref matching the name
of the unpacked folder. The branch redirect from renaming the default
branch breaks that assumption.

Signed-off-by: Nicolas Werner <n.werner@famedly.com>
 2022-02-23 10:06:18 +00:00
Shay 1e12efa1b2
Drop support for EOL Ubuntu 21.04 (#11961) 2022-02-10 13:59:42 -08:00
Nicolas Werner fa583c2198
Allow overriding the complement ref. (#11766) 
Updates complement.sh to read the ref from an environment
variable (defaulting to master) when downloading a complement
bundle for testing.
 2022-01-20 13:04:58 +00:00
Richard van der Hoff d41c4654db
Use buildkit's cache feature to speed up docker builds (#11691) 
Having spent much of the last week attempting to run complement tests from somewhere with damp string instead of internet... something had to be done.
 2022-01-12 10:37:57 +00:00
Michael Kaye 338e70c617
Complement environment variable name change and update .gitignore. (#11718) 2022-01-10 23:18:56 +00:00
Richard van der Hoff b38bdae3a2
Fix AssertionErrors after purging events (#11642) 
* Fix AssertionErrors after purging events

If you purged a bunch of events from your database, and then restarted synapse
without receiving more events, then you would get a bunch of AssertionErrors on
restart.

This fixes the situation by rewinding the stream processors.

* `check-newsfragment`: ignore deleted newsfiles
 2022-01-04 16:36:33 +00:00
Shay 13c974ed35
Drop Bionic from Debian builds (#11633) 
* update Trove classifiers to remove py36

* stop building bionic

* update dh-virtualenv

* newsfragment

* fix newsfragment

* update version refs

* another try at correct tag

* Update changelog
 2022-01-03 11:17:16 -08:00
Richard van der Hoff f61462e1be
scripts-dev/sign_json: support for signing events (#11486) 2021-12-02 15:18:40 +00:00
Patrick Cloke a4521ce0a8
Support the stable /hierarchy endpoint from MSC2946 (#11329) 
This also makes additional updates where the implementation
had drifted from the approved MSC.

Unstable endpoints will be removed at a later data.
 2021-11-29 14:32:20 -05:00
Patrick Cloke 9d1971a5c4
Return the stable event field from /send_join per MSC3083. (#11413) 
This does not remove the unstable field and still parses both.
Handling of the unstable field will need to be removed in the
future.
 2021-11-29 15:43:20 +00:00
Patrick Cloke 5505da2109
Remove msc2716 from the list of tests for complement. (#11389) 
As the tests are currently failing and not run in CI.
 2021-11-19 07:06:16 -05:00
Patrick Cloke 60ecb6b4d4
Fix running complement.sh script. (#11368) 
By reverting changes from #11166 in this script. Specifically commit
13f084eb58.
 2021-11-17 09:04:50 -05:00
Dan Callahan d7141e0b8b
Fix Shellcheck SC2006: Use $(...) notation 
Use $(...) notation instead of legacy backticked `...`.

https://github.com/koalaman/shellcheck/wiki/SC2006

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:55 +01:00
Dan Callahan 13f084eb58
Fix Shellcheck SC2086: Quote to prevent splitting 
Double quote to prevent globbing and word splitting.

https://github.com/koalaman/shellcheck/wiki/SC2086

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:54 +01:00
Dan Callahan 6a9d84a676
Fix Shellcheck SC2166: test -a is not well defined 
Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.

https://github.com/koalaman/shellcheck/wiki/SC2166

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:54 +01:00
Dan Callahan 64adbb7b54
Fix Shellcheck SC2046: Quote to prevent word split 
Quote this to prevent word splitting

https://www.shellcheck.net/wiki/SC2046

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:53 +01:00
Dan Callahan 12d79ff1b6
Fix Shellcheck SC2164: exit in case cd fails. 
Use `cd ... || exit` in case cd fails.

https://github.com/koalaman/shellcheck/wiki/SC2164

Signed-off-by: Dan Callahan <danc@element.io>
 2021-10-22 23:08:53 +01:00
Richard van der Hoff 57501d9194
Update sign_json to support inline key config (#11139) 
It's been possible to configure a key inline in the homeserver.yaml since
13bc1e0746. Update `sign_json` to work with this.
 2021-10-20 18:49:20 +01:00
Hillery Shay 5dcacdf6d1
Add support for ubuntu 21.10 "Impish Indri" (#11024) 
* support ubuntu 21.10 indri

* add changelog

* update to correct codename

Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>

Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
 2021-10-12 11:29:39 -07:00
reivilibre b742cb2e4a
Release script improvements (#10966) 
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2021-10-11 14:48:38 +01:00
Nick Barrett c80878d22a
Add --run-background-updates option to update_database script. (#10954) 
Signed-off-by: Nick Barrett <nick@beeper.com>
 2021-10-06 11:26:18 +01:00
Erik Johnston d37841787a
Sign the git tag in release script (#10925) 2021-09-27 15:39:49 +01:00
David Robertson ea01d4c2de
Update postgresql testing script (#10906) 
- Use sytest:bionic. Sytest:latest is two years old (do we want
  CI to push out latest at all?) and comes with Python 3.5, which we
  explictly no longer support. The script now runs under PostgreSQL 10
  as a result.
- Advertise script in the docs
- Move pg testing script to scripts-dev directory
- Write to host as the script's exector, not root

A few changes to make it speedier to re-run the tests:

- Create blank DB in the container, not the script, so we don't have to
  `initdb` each time
- Use a named volume to persist the tox environment, so we don't have to
  fetch and install a bunch of packages from PyPI each time

Co-authored-by: reivilibre <olivier@librepush.net>
 2021-09-24 14:27:09 +00:00
Brendan Abolivier 5154afc00d
Add a script to update the debian changelog for non-Debian systems (#10778) 
When releasing 1.42.0 with @Azrenbeth and talking with @clokep yesterday I realised doing the dch incantations related to releasing Synapse wasn't trivial on eg a macOS system, so this is a script to run in a Debian container to make things a bit easier.
 2021-09-08 17:03:25 +02:00
Patrick Cloke 882539e423
Ensure the base Docker image is rebuilt when running complement with workers. (#10686) 
We now always rebuild the matrixdotorg/synapse image, then
build the matrixdotorg/synapse-workers image on top of it.
 2021-08-25 10:18:23 -04:00
Richard van der Hoff c8132f4a31
Build debs for bookworm (#10612) 2021-08-17 13:48:59 +01:00
David Robertson d2ad397d3c
Stop building a debian package for Groovy Gorilla (#10588) 2021-08-12 16:50:18 +01:00
David Robertson 314a739160 Also rename in lint.sh 2021-08-12 10:40:44 +01:00
David Robertson c0ebdfc77e Kill off the .buildkite dir completely 2021-08-10 16:38:13 +01:00