1298 commits

Author	SHA1	Message	Date
Dirk Klimpel	a36a38b1ca	Add some example configurations for worker (#12492) ... Signed-off-by: Dirk Klimpel <dirk@klimpel.org>	2022-04-25 13:17:03 +01:00
Richard van der Hoff	f46b223354	turn-howto: fix some links	2022-04-22 14:23:40 +01:00
David Robertson	ecef741add	Recommend poetry in docs (#12475) ... * Recommend poetry in docs - readme - contributor guide - upgrade notes - new dev cheat sheet for poetry Co-authored-by: Shay <hillerys@element.io> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2022-04-20 15:18:21 +01:00
Olivier Wilkinson (reivilibre)	0921d93dcd	Merge branch 'master' into develop	2022-04-19 13:28:28 +01:00
Richard van der Hoff	9f512ff537	fix typo in upgrade notes	2022-04-19 11:25:12 +01:00
Shay	3c758d9808	Add a manual documenting config file options (#12368)	2022-04-18 11:32:30 -07:00
reivilibre	7efddbebef	Update documentation to reflect that run_background_tasks_on is no longer experimental. (#12451) ... * Background workers aren't experimental anymore * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Stream writers aren't experimental either	2022-04-14 15:25:22 +01:00
Dirk Klimpel	a743f7d33e	Replace federation_reader with generic_worker in docs (#12457)	2022-04-14 13:09:07 +01:00
David Robertson	535a689cfc	Reintroduce the lint targets in the linter script (#12455)	2022-04-14 11:33:06 +01:00
Andrew Morgan	0922462fc7	docs: Don't render the table of contents on the print page (#12340)	2022-04-13 08:27:51 -07:00
Erik Johnston	a468768104	Merge branch 'release-v1.57' into develop	2022-04-12 15:28:41 +01:00
Patrick Cloke	9535fd0f9c	Disable groups/communities by default. (#12344) ... This disables the endpoints (and sync response fields) for groups/communities by default.	2022-04-12 10:20:46 -04:00
Richard van der Hoff	643c0c50c1	Update UPGRADE notes to include more details about v1.57 upgrade failure mode (#12448)	2022-04-12 14:56:05 +01:00
Erik Johnston	e31d06f6f0	Split changelogs by year	2022-04-12 13:54:36 +01:00
David Robertson	dd5cc37aa4	Stop maintaining a list of lint targets in lint.sh (#12420)	2022-04-08 14:36:09 +01:00
Eric Eastwood	350062661c	Clarify that we mark as outliers because we don't have any state for them (#12345) ... As discussed at https://github.com/matrix-org/synapse/pull/12179#discussion_r837263852	2022-04-06 16:34:33 -05:00
Patrick Cloke	6902e9ff2b	Remove docs for converting a legacy structured logging config. (#12392) ... And update the upgrade notes to point at old versions of the documentation.	2022-04-06 13:41:14 -04:00
Sean Quah	ab3fdcf960	Merge branch 'master' into develop	2022-04-05 15:14:53 +01:00
reivilibre	708d88b1a2	Allow specifying the Postgres database's port when running unit tests with Postgres. (#12376)	2022-04-05 12:44:48 +01:00
Shay	f608e6c8cf	Update broken link in workers.md (#12369)	2022-04-04 17:40:31 -07:00
Nick Mills-Barrett	993d90f82b	Use a sequence to generate AS transaction IDs, drop last_txn AS state (#12209) ... Switching to a sequence means there's no need to track `last_txn` on the AS state table to generate new TXN IDs. This also means that there is no longer contention between the AS scheduler and AS handler on updates to the `application_services_state` table, which will prevent serialization errors during the complete AS txn transaction.	2022-04-01 13:33:25 +00:00
Brendan Abolivier	e4409301ba	Add a module callback to react to account data changes (#12327) ... Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2022-04-01 11:22:48 +02:00
Brendan Abolivier	5e88143dff	Add a callback to react to 3PID associations (#12302)	2022-03-31 18:27:21 +02:00
David Robertson	15cdcf8f30	Remove dockerfile-pgtests (#12336)	2022-03-31 11:30:07 +01:00
reivilibre	2fc15ac718	Flesh out documentation for running SyTest against Synapse, including use of Postgres and worker mode. (#12271) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2022-03-30 12:55:05 +00:00
Brendan Abolivier	437a8ed9ef	Add a configuration to exclude rooms from sync response (#12310)	2022-03-30 09:43:04 +00:00
Andrew Morgan	013f3f5e44	Move warning of open registration to v1.56 upgrade notes (#12322)	2022-03-29 14:12:41 +01:00
villepeh	a4643a685c	HAProxy guide update (#12279) ... HAproxy reverse proxy guide update to stop sending IPv4-mapped address to HS.	2022-03-28 10:48:18 +00:00
Shay	3c41d87b67	Add restrictions by default to open registration in Synapse (#12091)	2022-03-25 10:11:01 -07:00
IronTooch	61aae18d45	Authentik OpenID minor doc update (#12275)	2022-03-25 12:40:10 +00:00
Brendan Abolivier	5859e2fe0c	Mention the new behaviour on unsafe database locale in the upgrade notes (#12288) ... Co-authored-by: Shay <hillerys@element.io>	2022-03-25 10:56:18 +01:00
Shay	e78d4f61fc	Refuse to start if DB has an unsafe locale (#12262)	2022-03-23 10:23:05 -07:00
Jonathan de Jong	c5776780f0	Remove mutual_rooms update_user_directory check, and add extra documentation (#12038) ... Resolves #10339	2022-03-23 12:47:07 +00:00
Patrick Cloke	2177e356bc	Sync more worker regexes in the documentation. (#12243)	2022-03-18 12:51:27 -04:00
David Robertson	872dbb0181	Correct check_username_for_spam annotations and docs (#12246) ... * Formally type the UserProfile in user searches * export UserProfile in synapse.module_api * Update docs Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>	2022-03-18 13:51:41 +00:00
Patrick Cloke	54f674f7a9	Deprecate the groups/communities endpoints and add an experimental configuration flag. (#12200)	2022-03-12 13:23:37 -05:00
Shay	ef3619e61d	Add config settings for background update parameters (#11980)	2022-03-11 10:46:45 -08:00
Brendan Abolivier	e6a106fd5e	Implement a Jinja2 filter to extract localparts from email addresses (#12212)	2022-03-11 15:15:11 +00:00
Brendan Abolivier	003cc6910a	Update the SSO username picker template to comply with SIWA guidelines (#12210) ... Fixes https://github.com/matrix-org/synapse/issues/12205	2022-03-11 13:20:00 +00:00
Andrew Morgan	3b12f6d61b	Note that contributors can sign off privately (#12204) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2022-03-11 11:10:20 +00:00
~creme	7577894bec	Document that most streams can only have a single writer. (#12196) ... This includes the `typing`, `to_device`, `account_data`, `receipts`, and `presence` streams (really anything except the `events` stream).	2022-03-10 18:15:19 +00:00
reivilibre	72e7f1c420	Remove workaround introduced in Synapse v1.50.0rc1 for Mjolnir compatibility. Breaks compatibility with Mjolnir v1.3.1 and earlier. (#11700)	2022-03-10 15:53:23 +00:00
Richard van der Hoff	52a947dc46	Updates to the Room DAG concepts development document (#12179) ... Some stuff that came up while we were talking about #12173.	2022-03-10 15:18:31 +00:00
Will Hunt	15382b1afa	Add third_party module callbacks to check if a user can delete a room and deactivate a user (#12028) ... * Add check_can_deactivate_user * Add check_can_shutdown_rooms * Documentation * callbacks, not functions * Various suggested tweaks * Add tests for test_check_can_shutdown_room and test_check_can_deactivate_user * Update check_can_deactivate_user to not take a Requester * Fix check_can_shutdown_room docs * Renegade and use `by_admin` instead of `admin_user_id` * fix lint * Update docs/modules/third_party_rules_callbacks.md Co-authored-by: Brendan Abolivier <babolivier@matrix.org> * Update docs/modules/third_party_rules_callbacks.md Co-authored-by: Brendan Abolivier <babolivier@matrix.org> * Update docs/modules/third_party_rules_callbacks.md Co-authored-by: Brendan Abolivier <babolivier@matrix.org> * Update docs/modules/third_party_rules_callbacks.md Co-authored-by: Brendan Abolivier <babolivier@matrix.org> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>	2022-03-09 18:23:57 +00:00
Patrick Cloke	9a0172d49f	Clean-up demo scripts & documentation (#12143) ... * Rewrites the demo documentation to be clearer, accurate, and moves it to our documentation tree. * Improvements to the demo scripts: * `clean.sh` now runs `stop.sh` first to avoid zombie processes. * Uses more modern Synapse configuration (and removes some obsolete configuration). * Consistently use the HTTP ports for server name, etc. * Remove the `demo/etc` directory and place everything into the `demo/808x` directories.	2022-03-08 15:02:59 -05:00
David Robertson	4aeb00ca20	Move synctl into synapse._scripts and expose as an entrypoint (#12140)	2022-03-04 11:58:49 +00:00
David Robertson	1fbe0316a9	Add suffices to scripts in scripts-dev (#12137) ... * Rename scripts-dev to have suffices * Update references to `scripts-dev` * Changelog * These scripts don't pass mypy	2022-03-02 18:00:26 +00:00
Andrew Morgan	b4461e7d8a	Enable complexity checking in complexity checking docs example (#11998)	2022-03-02 16:11:16 +00:00
David Robertson	f3f0ab10fe	Move scripts directory inside synapse, exposing as setuptools entry_points (#12118) ... * Two scripts are basically entry_points already * Move and rename scripts/* to synapse/_scripts/*.py * Delete sync_room_to_group.pl * Expose entry points in setup.py * Update linter script and config * Fixup scripts & docs mentioning scripts that moved Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2022-03-02 13:00:16 +00:00
Brendan Abolivier	300ed0b8a6	Add module callbacks called for reacting to deactivation status change and profile update (#12062)	2022-03-01 15:00:03 +00:00
Patrick Cloke	7754af24ab	Remove the unstable /spaces endpoint. (#12073) ... ...and various code supporting it. The /spaces endpoint was from an old version of MSC2946 and included both a Client-Server and Server-Server API. Note that the unstable /hierarchy endpoint (from the final version of MSC2946) is not yet removed.	2022-02-28 18:33:00 +00:00
Patrick Cloke	c56bfb08bc	Add documentation for missing worker types. (#11599) ... And clean-up the endpoints which should be routed to workers.	2022-02-23 22:49:04 +00:00
Richard van der Hoff	e24ff8ebe3	Remove HomeServer.get_datastore() (#12031) ... The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733	2022-02-23 11:04:02 +00:00
Patrick Cloke	7bcc28f82f	Use room version 9 as the default room version (per MSC3589). (#12058)	2022-02-22 14:09:40 +00:00
Erik Johnston	79c18e0709	Merge remote-tracking branch 'origin/master' into develop	2022-02-22 12:05:48 +00:00
Patrick Cloke	444b04058b	Document why auth providers aren't validated in the admin API. (#12004) ... Since it is reasonable to give a future or past auth provider, which might not be in the current configuration.	2022-02-18 17:24:25 +00:00
Brendan Abolivier	707049c6ff	Allow modules to set a display name on registration (#12009) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2022-02-17 16:54:16 +00:00
Patrick Cloke	e69f8f0a8e	Remove support for the legacy structured logging configuration. (#12008)	2022-02-17 13:32:18 +00:00
David Robertson	73fc488783	Explain the meaning of spam checker callbacks' return values (#12003) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2022-02-16 12:25:43 +00:00
Andrew Morgan	7812fe9edd	Note when unstable MSC3283 prefixes will be removed (#11989)	2022-02-14 19:07:59 +00:00
H. Shay	9c4563c5cd	remove empty file, reword/rename newsfragment, and add line to upgrade doc	2022-02-14 10:21:00 -08:00
Shay	b2b971f28a	Enable cache time-based expiry by default (#11849)	2022-02-11 11:05:20 -08:00
Brendan Abolivier	0171fa5226	Remove deprecated user_may_create_room_with_invites callback (#11950) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2022-02-11 13:58:11 +00:00
Dirk Klimpel	d36943c4df	Support the stable API endpoint for MSC3283: new settings in /capabilities endpoint (#11933)	2022-02-11 09:32:11 +00:00
Patrick Cloke	e4fdf459e2	Basic documentation for the release cycle. (#11954)	2022-02-10 13:15:10 +00:00
Brad Jones	3914576b2b	Fix example for structured logging. (#11946) ... The StreamHandler takes a stream parameter, not location.	2022-02-09 20:56:33 +00:00
Brendan Abolivier	6f440fd859	Recommend upgrading treq alongside twisted (#11943)	2022-02-09 15:06:51 +00:00
Brendan Abolivier	6b91315ddf	Merge branch 'master' into develop	2022-02-08 13:26:09 +00:00
Brendan Abolivier	1aa2231e27	Fix wording	2022-02-08 12:03:49 +00:00
Brendan Abolivier	5cdd491310	Add words about the Twisted security fix	2022-02-08 11:47:35 +00:00
Brendan Abolivier	0640f8ebaa	Add a callback to allow modules to deny 3PID (#11854) ... Part of the Tchap Synapse mainlining. This allows modules to implement extra logic to figure out whether a given 3PID can be added to the local homeserver. In the Tchap use case, this will allow a Synapse module to interface with the custom endpoint /internal_info.	2022-02-08 11:20:32 +01:00
Dirk Klimpel	65ef21b1c7	Clarify that users' media are also preview images (#11862)	2022-02-04 14:39:14 +00:00
Jonathan de Jong	02632b3504	Stabilise MSC3231 (Token Based Registration) (#11867)	2022-02-04 12:15:13 +00:00
Richard van der Hoff	b3d155a749	Delete MSC1711_certificates_FAQ.md (#11907) ... This document isn't really relevant any more, and its existence is more confusing than helpful.	2022-02-04 11:27:00 +00:00
Patrick Cloke	119edf51eb	Remove support for the webclient listener. (#11895) ... Also remove support for non-HTTP(S) web_client_location.	2022-02-03 18:36:49 +00:00
Christian Paul	6b1c265c21	Fix typo: unpind -> unbind (#11859) ... Co-authored-by: reivilibre <olivier@librepush.net>	2022-02-03 17:20:44 +00:00
Brendan Abolivier	d80d39b035	Add a ratelimiter for 3pid invite (#11892)	2022-02-03 14:28:15 +01:00
Dirk Klimpel	0da2301b21	Consolidate the access_token information in the admin api (#11861) ... Co-authored-by: reivilibre <olivier@librepush.net>	2022-01-31 16:24:29 +00:00
Dirk Klimpel	7eb198ddc8	Remove not needed old table of contents in documentation (#11860)	2022-01-31 14:40:20 +00:00
Brendan Abolivier	bf60da1a60	Configurable limits on avatars (#11846) ... Only allow files which file size and content types match configured limits to be set as avatar. Most of the inspiration from the non-test code comes from matrix-org/synapse-dinsic#19	2022-01-28 15:41:33 +01:00
Dirk Klimpel	fd65139714	Fix some indentation inconsistencies in the sample config (modules) (#11838)	2022-01-27 10:06:29 +00:00
Shay	ec07062e31	Update installation docs to indicate that we support Python 3.10 (#11820)	2022-01-26 16:05:29 -08:00
Brendan Abolivier	2d3bd9aa67	Add a module callback to set username at registration (#11790) ... This is in the context of mainlining the Tchap fork of Synapse. Currently in Tchap usernames are derived from the user's email address (extracted from the UIA results, more specifically the m.login.email.identity step). This change also exports the check_username method from the registration handler as part of the module API, so that a module can check if the username it's trying to generate is correct and doesn't conflict with an existing one, and fallback gracefully if not. Co-authored-by: David Robertson <davidr@element.io>	2022-01-26 14:21:13 +00:00
Brendan Abolivier	95b3f952fa	Add a config flag to inhibit M_USER_IN_USE during registration (#11743) ... This is mostly motivated by the tchap use case, where usernames are automatically generated from the user's email address (in a way that allows figuring out the email address from the username). Therefore, it's an issue if we respond to requests on /register and /register/available with M_USER_IN_USE, because it can potentially leak email addresses (which include the user's real name and place of work). This commit adds a flag to inhibit the M_USER_IN_USE errors that are raised both by /register/available, and when providing a username early into the registration process. This error will still be raised if the user completes the registration process but the username conflicts. This is particularly useful when using modules (https://github.com/matrix-org/synapse/pull/11790 adds a module callback to set the username of users at registration) or SSO, since they can ensure the username is unique. More context is available in the PR that introduced this behaviour to synapse-dinsic: matrix-org/synapse-dinsic#48 - as well as the issue in the matrix-dinsic repo: matrix-org/matrix-dinsic#476	2022-01-26 13:02:54 +01:00
Dirk Klimpel	6a72c910f1	Add admin API to get a list of federated rooms (#11658)	2022-01-25 16:11:40 +00:00
Richard van der Hoff	fc8598bc87	Minor updates, and docs, for schema delta files (#11823) ... * Make functions in python deltas optional It's annoying to always have to write stubs for these. * Documentation for delta files * changelog	2022-01-25 14:11:13 +00:00
Forest Johnson	4210143f53	Docs: add missing PR submission process how-tos (#11821) ... * Docs: add missing PR submission process how-tos The documentation says that in order to submit a pull request you have to run the linter and links to [Run the linters](https://matrix-org.github.io/synapse/latest/development/contributing_guide.html#run-the-linters). IMO "Run the linters" should explain that development dependencies are a pre-requisite. I also included `pip install wheel` which I had to run inside my virtual environment on ubuntu before I `pip install -e ".[all,dev]"` would succeed.	2022-01-25 14:09:56 +00:00
Dirk Klimpel	0d6cfea9b8	Add admin API to reset connection timeouts for remote server (#11639) ... * Fix get federation status of destination if no error occured	2022-01-25 12:06:29 +00:00
reivilibre	df54c8485a	Remove account data (including client config, push rules and ignored users) upon user deactivation. (#11621) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2022-01-24 13:37:00 +00:00
Shay	9006ee36d1	Drop support for and remove references to EOL Python 3.6 (#11683) ... * remove reference in comments to python3.6 * upgrade tox python env in script * bump python version in example for completeness * upgrade python version requirement in setup doc * upgrade necessary python version in __init__.py * upgrade python version in setup.py * newsfragment * drops refs to bionic and replace with focal * bump refs to postgres 9.6 to 10 * fix hanging ci * try installing tzdata first * revert change made in b979f336 * ignore new random mypy error while debugging other error * fix lint error for temporary workaround * revert change to install list * try passing env var * export debian frontend var? * move line and add comment * bump pillow dependency * bump lxml depenency * install libjpeg-dev for pillow * bump automat version to one compatible with py3.8 * add libwebp for pillow * bump twisted trunk python version * change suffix of newsfragment * remove redundant python 3.7 checks * lint	2022-01-21 14:23:26 -08:00
Richard van der Hoff	56834ab779	installation.md: drop python 3.6 support (#11781) ... #11595 dropped support for python 3.6, but forgot to update this doc.	2022-01-20 14:37:11 +00:00
Patrick Cloke	91221b6961	Add deprecation warnings for webclient listener and non-HTTP(S) web_client_location. (#11774) ... This changes the behaviour of the root endpoint to redirect directly to the configuration of `web_client_location` if it is given an HTTP(S) URL.	2022-01-20 14:21:06 +00:00
Philippe Daouadi	15ffc4143c	Fix preview of imgur and Tenor URLs. (#11669) ... By scraping Open Graph information from the HTML even when an autodiscovery endpoint is found. The results are then combined to capture as much information as possible from the page.	2022-01-18 13:20:24 -05:00
AndrewFerr	cefd4b87a3	Warn against using Let's Encrypt certs for encrypted TURN (#11686) ... * Warn against using Let's Encrypt certs for encrypted TURN This helps to avoid client-side issues: * https://github.com/vector-im/element-android/issues/1533 * https://github.com/vector-im/element-ios/issues/2712 Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>	2022-01-17 15:13:09 +00:00
Richard van der Hoff	86615aa965	Fix up docs for track_puppeted_user_ips (again) (#11757) ... Fixes #11741	2022-01-17 14:55:30 +00:00
Richard van der Hoff	b0352f9c08	Update documentation for configuring facebook login (#11755) ... ... and a minor thinko fix in the sample config.	2022-01-17 12:35:00 +00:00
Andrew Morgan	18862f20b5	Remove the 'password_hash' from the Users Admin API endpoint response dictionary (#11576)	2022-01-14 14:53:33 +00:00
Jason Robinson	904bb04409	Fix sample_config.yaml in regards track_puppeted_user_ips (#11749) ... * Fix sample_config.yaml in regards track_puppeted_user_ips Closes #11741 Signed-off-by: Jason Robinson <jasonr@matrix.org>	2022-01-14 14:11:55 +00:00
qwertyforce	b9632046fb	update room spec url in config files (#11739) ... * change spec url in config files * Create 11739.txt * .txt -> .doc	2022-01-13 17:09:15 +00:00
Andy Balaam	5ff5f17377	Mention python3-venv and libpq-dev dependencies in contribution guide (#11740)	2022-01-13 16:33:37 +00:00