1307 commits

Author	SHA1	Message	Date
Patrick Cloke	d19bccdbec	Update SSO mapping providers documentation about unique IDs. (#9980)	2021-05-13 14:37:20 -04:00
Patrick Cloke	28c6841102	Send the m.room.create stripped event with invites (support MSC1772). (#9966) ... MSC1772 specifies the m.room.create event should be sent as part of the invite_state. This was done optionally behind an experimental flag, but is now done by default due to MSC1772 being approved.	2021-05-11 10:58:58 -04:00
Aaron Raimist	dc6366a9bd	Add config option to hide device names over federation (#9945) ... Now that cross signing exists there is much less of a need for other people to look at devices and verify them individually. This PR adds a config option to allow you to prevent device display names from being shared with other servers. Signed-off-by: Aaron Raimist <aaron@raim.ist>	2021-05-11 14:03:23 +01:00
Erik Johnston	1fb9a2d0bf	Limit how often GC happens by time. (#9902) ... Synapse can be quite memory intensive, and unless care is taken to tune the GC thresholds it can end up thrashing, causing noticable performance problems for large servers. We fix this by limiting how often we GC a given generation, regardless of current counts/thresholds. This does not help with the reverse problem where the thresholds are set too high, but that should only happen in situations where they've been manually configured. Adds a `gc_min_seconds_between` config option to override the defaults. Fixes #9890.	2021-05-05 16:53:45 +01:00
Andrew Morgan	b85821aca2	Add port parameter to the sample config for psycopg2 args (#9911) ... Adds the `port` option with the default value to the sample config file.	2021-05-04 13:28:59 +01:00
ThibF	0085dc5abc	Delete room endpoint (#9889) ... Support the delete of a room through DELETE request and mark previous request as deprecated through documentation. Signed-off-by: Thibault Ferrante <thibault.ferrante@pm.me>	2021-04-29 10:31:45 +01:00
Patrick Cloke	683d6f75af	Rename handler and config modules which end in handler/config. (#9816)	2021-04-20 14:55:20 -04:00
Andrew Morgan	71f0623de9	Port "Allow users to click account renewal links multiple times without hitting an 'Invalid Token' page #74" from synapse-dinsic (#9832) ... This attempts to be a direct port of https://github.com/matrix-org/synapse-dinsic/pull/74 to mainline. There was some fiddling required to deal with the changes that have been made to mainline since (mainly dealing with the split of `RegistrationWorkerStore` from `RegistrationStore`, and the changes made to `self.make_request` in test code).	2021-04-19 19:16:34 +01:00
Dirk Klimpel	1fc97ee876	Add an admin API to manage ratelimit for a specific user (#9648)	2021-04-13 10:26:37 +01:00
Andrew Morgan	04819239ba	Add a Synapse Module for configuring presence update routing (#9491) ... At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though. This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around. A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being: * Sending state for a specific set or all known users to a defined set of local and remote users. * The ability to trigger an initial sync for specific users, so they receive all current state.	2021-04-06 14:38:30 +01:00
Erik Johnston	5688a74cf3	Merge branch 'master' into develop	2021-04-06 13:29:29 +01:00
Erik Johnston	a888cbdd31	Add deprecation policy doc (#9723) ... Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2021-04-06 13:07:36 +01:00
Dirk Klimpel	bb0fe02a52	Add order_by to list user admin API (#9691)	2021-04-01 11:28:53 +01:00
Denis Kasak	5ff8eb97c6	Make sample config allowed_local_3pids regex stricter. (#9719) ... The regex should be terminated so that subdomain matches of another domain are not accepted. Just ensuring that someone doesn't shoot themselves in the foot by copying our example. Signed-off-by: Denis Kasak <dkasak@termina.org.uk>	2021-03-31 12:27:20 +00:00
Richard van der Hoff	f02663c4dd	Replace room_invite_state_types with room_prejoin_state (#9700) ... `room_invite_state_types` was inconvenient as a configuration setting, because anyone that ever set it would not receive any new types that were added to the defaults. Here, we deprecate the old setting, and replace it with a couple of new settings under `room_prejoin_state`.	2021-03-30 12:12:44 +01:00
Richard van der Hoff	4bbd535450	Update the OIDC sample config (#9695) ... I've reiterated the advice about using `oidc` to migrate, since I've seen a few people caught by this. I've also removed a couple of the examples as they are duplicating the OIDC documentation, and I think they might be leading people astray.	2021-03-29 15:40:11 +01:00
Paul Tötterman	d7d4232a2d	Preserve host in example apache config (#9696) ... Fixes redirect loop Signed-off-by: Paul Tötterman <paul.totterman@iki.fi>	2021-03-26 10:38:31 +00:00
Ben Banfield-Zanin	4655d2221e	docs: fallback/web endpoint does not appear to be mounted on workers (#9679)	2021-03-24 11:43:04 +00:00
Erik Johnston	e550ab17ad	Increase default join burst ratelimiting (#9674) ... It's legitimate behaviour to try and join a bunch of rooms at once.	2021-03-23 14:52:20 +00:00
Hubbe	dd5e5dc1d6	Add SSO attribute requirements for OIDC providers (#9609) ... Allows limiting who can login using OIDC via the claims made from the IdP.	2021-03-16 11:46:07 -04:00
Andrew Morgan	847ecdd8fa	Pass SSO IdP information to spam checker's registration function (#9626) ... Fixes https://github.com/matrix-org/synapse/issues/9572 When a SSO user logs in for the first time, we create a local Matrix user for them. This goes through the register_user flow, which ends up triggering the spam checker. Spam checker modules don't currently have any way to differentiate between a user trying to sign up initially, versus an SSO user (whom has presumably already been approved elsewhere) trying to log in for the first time. This PR passes `auth_provider_id` as an argument to the `check_registration_for_spam` function. This argument will contain an ID of an SSO provider (`"saml"`, `"cas"`, etc.) if one was used, else `None`.	2021-03-16 12:41:41 +00:00
Richard van der Hoff	dd69110d95	Add support for stable MSC2858 API (#9617) ... The stable format uses different brand identifiers, so we need to support two identifiers for each IdP.	2021-03-16 11:21:26 +00:00
Richard van der Hoff	5b5bc188cf	Clean up config settings for stats (#9604) ... ... and complain if people try to turn it off.	2021-03-16 10:57:54 +00:00
Jason Robinson	17cd48fe51	Fix spam checker modules documentation example (#9580) ... Mention that parse_config must exist and note the check_media_file_for_spam method.	2021-03-10 10:42:51 -05:00
Andrew Morgan	e9df3f496b	Link to the List user's media admin API from media Admin API docs (#9571) ... Earlier [I was convinced](https://github.com/matrix-org/synapse/issues/9565) that we didn't have an Admin API for listing media uploaded by a user. Foolishly I was looking under the Media Admin API documentation, instead of the User Admin API documentation. I thought it'd be helpful to link to the latter so others don't hit the same dead end :)	2021-03-09 15:15:52 +00:00
Richard van der Hoff	eaada74075	JWT OIDC secrets for Sign in with Apple (#9549) ... Apple had to be special. They want a client secret which is generated from an EC key. Fixes #9220. Also fixes #9212 while I'm here.	2021-03-09 15:03:37 +00:00
Erik Johnston	576c91c7c1	Fixup sample config ... After 0764d0c6e5	2021-03-09 11:40:45 +00:00
Leo Bärring	0fc4eb103a	Update reverse proxy to add OpenBSD relayd example configuration. (#9508) ... Update reverse proxy to add OpenBSD relayd example configuration. Signed-off-by: Leo Bärring <leo.barring@protonmail.com>	2021-03-06 11:49:19 +00:00
Richard van der Hoff	a5daae2a5f	Update nginx reverse-proxy docs (#9512) ... Turns out nginx overwrites the Host header by default.	2021-03-03 11:08:11 +00:00
Richard van der Hoff	d8e95e5452	Add support for X-Forwarded-Proto (#9472) ... rewrite XForwardedForRequest to set `isSecure()` based on `X-Forwarded-Proto`. Also implement `getClientAddress()` while we're here.	2021-02-24 18:11:33 +00:00
Andrew Morgan	00bf80cb8e	Fix typo in spam checker documentation	2021-02-24 17:51:52 +00:00
Erik Johnston	66f4949e7f	Fix deleting pushers when using sharded pushers. (#9465)	2021-02-22 21:14:42 +00:00
Dirk Klimpel	71c9f8de6d	Add an order_by field to list users' media admin API. (#8978)	2021-02-22 14:38:51 -05:00
Richard van der Hoff	70ea9593ff	example systemd config: propagate reloads to units (#9463) ... It should be possible to reload `synapse.target` to have the reload propagate to all the synapse units.	2021-02-22 18:43:17 +00:00
Andrew Morgan	e22b71810e	Clean up the user directory sample config section (#9385) ... The user directory sample config section was a little messy, and didn't adhere to our [recommended config format guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format). This PR cleans that up a bit.	2021-02-22 11:44:31 +00:00
Andrew Morgan	13e9029f44	Add a config option to prioritise local users in user directory search results (#9383) ... This PR adds a homeserver config option, `user_directory.prefer_local_users`, that when enabled will show local users higher in user directory search results than remote users. This option is off by default. Note that turning this on doesn't necessarily mean that remote users will always be put below local users, but they should be assuming all other ranking factors (search query match, profile information present etc) are identical. This is useful for, say, University networks that are openly federating, but want to prioritise local students and staff in the user directory over other random users.	2021-02-19 11:02:03 +00:00
AndrewFerr	9bc74743d5	Add configs to make profile data more private (#9203) ... Add off-by-default configuration settings to: - disable putting an invitee's profile info in invite events - disable profile lookup via federation Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>	2021-02-19 09:50:41 +00:00
Rishabh Arya	e17553e185	Parse ui_auth.session_timeout as a duration (instead of treating it as ms) (#9426)	2021-02-18 09:18:14 -05:00
David Vo	bb2577f6b7	Add http2 to the nginx example config (#9390)	2021-02-18 08:46:16 -05:00
Richard van der Hoff	626afd7e89	Revert "Update workers.md" ... This reverts commit a8878960c0.	2021-02-18 11:56:25 +00:00
Dirk Klimpel	c8d9383cfb	Add the shadow-banning status to the display user admin API. (#9400)	2021-02-17 15:19:23 -05:00
Richard van der Hoff	53f1c4da81	Update workers.md	2021-02-17 17:14:23 +00:00
Richard van der Hoff	a8878960c0	Update workers.md ... tiny typo in sso paths	2021-02-17 17:11:24 +00:00
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381) ... - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](80d6dc9783/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version	2021-02-16 22:32:34 +00:00
Marcus	5636e597c3	Fix OIDC gitiea redirect URL. (#9404) ... Fixes a "conflict" from 846b9d3df0 and d1f13c7485.	2021-02-16 14:06:55 -05:00
Erik Johnston	ddfdf94506	Document that pusher instances are shardable (#9407)	2021-02-16 13:27:49 +00:00
Andrew Morgan	ff40c8099d	Fix sample config ... Just a small change missed in 7950aa8a27.	2021-02-12 22:18:40 +00:00
Patrick Cloke	2c9b4a5f16	Synapse 1.27.0rc2 (2021-02-11) ... ============================== Features -------- - Further improvements to the user experience of registration via single sign-on. ([\#9297](https://github.com/matrix-org/synapse/issues/9297)) Bugfixes -------- - Fix ratelimiting introduced in v1.27.0rc1 for invites to respect the `ratelimit` flag on application services. ([\#9302](https://github.com/matrix-org/synapse/issues/9302)) - Do not automatically calculate `public_baseurl` since it can be wrong in some situations. Reverts behaviour introduced in v1.26.0. ([\#9313](https://github.com/matrix-org/synapse/issues/9313)) Improved Documentation ---------------------- - Clarify the sample configuration for changes made to the template loading code. ([\#9310](https://github.com/matrix-org/synapse/issues/9310)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAmAlX4AACgkQM/xY9qcR MEjxhg/+JzOGDmgxy4Vm9oU84htkkJsflU755ykxhwbIjVy2j+07pMgtWMmebXIS /tWjinwRB9OgKIK+j5RWdYzz/GRTkLCbafTLhhNOWsnaCO7nnAfZFozLOMd9g2qN 309cS8efZkxGCISGSRm1QQKjQDx6HHx7nZVAwpqb778Q9TMrBYRAQeCk9yws7FfL GsP5YrBI42n84YhvflWA8J8QRYkeJJaggVWzE0XvwdI6raY1BmxMKcDvyl9iKpGb /QXsdketD3eJxdOlU92O5ZxRXmL00bdyAFdw2+J3Y+wjoGBnC5njKSFG3j8Z2UcP xIdB6w/zyGoPLKd4s7tkDI3axE7qrwFetA4NytannWGEHZ9q5tgOd5PA1kTeKYvn ao2os4pKMjWQOHiWLskqZVXgmoW5Kb2zgyZU/vhFAz8ImuidFULOjpEytAWqtt1B iuL/fRlM3z0BpwQNd3NBsblYTpZri8gdTp7ULJYtaKpT2MeG0sg3swJAptUjw9L4 awbQuBFZ8o/E/0xwiGieSxbR6b6Nz5WiCyLaeMz6b3Y4YdomttOAZ6pL6E9y8ygU 2wKYleSmg+tbaBKKZR5CSCwXOduSYPhWw7oqsfEfvT0NCQLuItfGNFZqxRzsf73H lCV0wwxNZBv8kRyMs6KivMvSI6SKinAjXS8AjzHE402ozdwOCv0= =NSyp -----END PGP SIGNATURE----- Merge tag 'v1.27.0rc2' into develop Synapse 1.27.0rc2 (2021-02-11) ============================== Features -------- - Further improvements to the user experience of registration via single sign-on. ([\#9297](https://github.com/matrix-org/synapse/issues/9297)) Bugfixes -------- - Fix ratelimiting introduced in v1.27.0rc1 for invites to respect the `ratelimit` flag on application services. ([\#9302](https://github.com/matrix-org/synapse/issues/9302)) - Do not automatically calculate `public_baseurl` since it can be wrong in some situations. Reverts behaviour introduced in v1.26.0. ([\#9313](https://github.com/matrix-org/synapse/issues/9313)) Improved Documentation ---------------------- - Clarify the sample configuration for changes made to the template loading code. ([\#9310](https://github.com/matrix-org/synapse/issues/9310))	2021-02-11 11:56:03 -05:00
Patrick Cloke	e40d88cff3	Backout changes for automatically calculating the public baseurl. (#9313) ... This breaks some people's configurations (if their Client-Server API is not accessed via port 443).	2021-02-11 11:16:54 -05:00
Thomas Mortagne	5f716fa777	Add XWiki OIDC provider example. (#9324)	2021-02-09 11:54:52 -05:00
David Teller	b0b2cac057	Merge pull request #9150 from Yoric/develop-context ... New API /_synapse/admin/rooms/{roomId}/context/{eventId}	2021-02-08 15:53:44 +01:00
Erik Johnston	adc96d4236	Merge branch 'erikj/media_spam_checker' into develop	2021-02-04 17:01:59 +00:00
Erik Johnston	7e8083eb48	Add check_media_file_for_spam spam checker hook	2021-02-04 17:01:30 +00:00
dykstranet	982d9eb211	Correct matrix-synapse.service reference in TURN howto docs. (#9308)	2021-02-04 11:22:44 -05:00
Patrick Cloke	2ab6e67ab7	Fix escaping of braces in OIDC sample config. (#9317) ... This fixes the Jinja2 templates for the mapping provider.	2021-02-04 09:06:20 -05:00
Richard van der Hoff	3f534d3fdf	Merge branch 'social_login_hotfixes' into develop	2021-02-03 20:34:27 +00:00
Patrick Cloke	afa18f1baa	Clarify documentation about escaping URLs in templates. (#9310)	2021-02-03 14:51:38 -05:00
dykstranet	e4cdecb310	config: Add detail to auto_join_rooms comment (#9291) ... config: Add detail to auto_join_rooms comment Signed-off-by: Gary Dykstra <gary@dykstranet.com>	2021-02-03 15:21:30 +00:00
Patrick Cloke	4ca054a4ea	Convert blacklisted IPv4 addresses to compatible IPv6 addresses. (#9240) ... Also add a few more IP ranges to the default blacklist.	2021-02-03 07:13:46 -05:00
Travis Ralston	b60bb28bbc	Add an admin API to get the current room state (#9168) ... This could arguably replace the existing admin API for `/members`, however that is out of scope of this change. This sort of endpoint is ideal for moderation use cases as well as other applications, such as needing to retrieve various bits of information about a room to perform a task (like syncing power levels between two places). This endpoint exposes nothing more than an admin would be able to access with a `select *` query on their database.	2021-02-02 11:16:29 +00:00
Richard van der Hoff	8f75bf1df7	Put SAML callback URI under /_synapse/client. (#9289)	2021-02-02 09:43:50 +00:00
Richard van der Hoff	846b9d3df0	Put OIDC callback URI under /_synapse/client. (#9288)	2021-02-01 22:56:01 +00:00
Oliver Hanikel	d1f13c7485	Add an OpenID example config for Gitea. (#9134)	2021-02-01 16:21:09 -05:00
Richard van der Hoff	5963426b95	Merge branch 'social_login' into develop	2021-02-01 18:46:12 +00:00
Richard van der Hoff	c543bf87ec	Collect terms consent from the user during SSO registration (#9276)	2021-02-01 18:37:41 +00:00
Richard van der Hoff	e5d70c8a82	Improve styling and wording of SSO UIA templates (#9286) ... fixes #9171	2021-02-01 18:36:04 +00:00
Richard van der Hoff	18ab35284a	Merge branch 'social_login' into develop	2021-02-01 17:28:37 +00:00
Richard van der Hoff	4167494c90	Replace username picker with a template (#9275) ... There's some prelimiary work here to pull out the construction of a jinja environment to a separate function. I wanted to load the template at display time rather than load time, so that it's easy to update on the fly. Honestly, I think we should do this with all our templates: the risk of ending up with malformed templates is far outweighed by the improved turnaround time for an admin trying to update them.	2021-02-01 15:52:50 +00:00
Richard van der Hoff	8aed29dc61	Improve styling and wording of SSO redirect confirm template (#9272)	2021-02-01 15:50:56 +00:00
Richard van der Hoff	9c715a5f19	Fix SSO on workers (#9271) ... Fixes #8966. * Factor out build_synapse_client_resource_tree Start a function which will mount resources common to all workers. * Move sso init into build_synapse_client_resource_tree ... so that we don't have to do it for each worker * Fix SSO-login-via-a-worker Expose the SSO login endpoints on workers, like the documentation says. * Update workers config for new endpoints Add documentation for endpoints recently added (#8942, #9017, #9262) * remove submit_token from workers endpoints list this *doesn't* work on workers (yet). * changelog * Add a comment about the odd path for SAML2Resource	2021-02-01 15:47:59 +00:00
Erik Johnston	f2c1560eca	Ratelimit invites by room and target user (#9258)	2021-01-29 16:38:29 +00:00
Richard van der Hoff	0d81a6fa3e	Merge branch 'social_login' into develop	2021-01-28 22:08:11 +00:00
Erik Johnston	4b73488e81	Ratelimit 3PID /requestToken API (#9238)	2021-01-28 17:39:21 +00:00
David Teller	fe52dae6bd	FIXUP: Documenting /_synapse/admin/v1/rooms/<room_id>/context/<event_id>	2021-01-28 12:30:21 +01:00
Richard van der Hoff	34efb4c604	Add notes on integrating with Facebook for SSO login. (#9244)	2021-01-27 22:57:16 +00:00
Richard van der Hoff	a083aea396	Add 'brand' field to MSC2858 response (#9242) ... We've decided to add a 'brand' field to help clients decide how to style the buttons. Also, fix up the allowed characters for idp_id, while I'm in the area.	2021-01-27 21:31:45 +00:00
Richard van der Hoff	869667760f	Support for scraping email addresses from OIDC providers (#9245)	2021-01-27 21:28:59 +00:00
Erik Johnston	93b61589b0	Add a note to changelog about redis usage (#9227)	2021-01-27 14:06:27 +00:00
Patrick Cloke	26837d5dbe	Do not require the CAS service URL setting (use public_baseurl instead). (#9199) ... The current configuration is handled for backwards compatibility, but is considered deprecated.	2021-01-26 10:49:25 -05:00
Jason Robinson	cee4010f94	Merge branch 'develop' into jaywink/admin-forward-extremities ... # Conflicts: # synapse/rest/admin/__init__.py	2021-01-26 10:15:32 +02:00
Patrick Cloke	4a55d267ee	Add an admin API for shadow-banning users. (#9209) ... This expands the current shadow-banning feature to be usable via the admin API and adds documentation for it. A shadow-banned users receives successful responses to their client-server API requests, but the events are not propagated into rooms. Shadow-banning a user should be used as a tool of last resort and may lead to confusing or broken behaviour for the client.	2021-01-25 14:49:39 -05:00
Jason Robinson	8965b6cfec	Merge branch 'develop' into jaywink/admin-forward-extremities	2021-01-23 21:41:35 +02:00
Jason Robinson	930ba00971	Add depth and received_ts to forward_extremities admin API response ... Also add a warning on the admin API documentation. Signed-off-by: Jason Robinson <jasonr@matrix.org>	2021-01-23 21:34:32 +02:00
Richard van der Hoff	b5120f09f1	Merge remote-tracking branch 'origin/release-v1.26.0' into develop	2021-01-21 13:17:07 +00:00
Richard van der Hoff	7447f19702	Prefix idp_id with "oidc-" (#9189) ... ... to avoid clashes with other SSO mechanisms	2021-01-21 12:25:02 +00:00
Patrick Cloke	f81d02d75b	Synapse 1.26.0rc1 (2021-01-20) ... ============================== This release brings a new schema version for Synapse and rolling back to a previous verious is not trivial. Please review [UPGRADE.rst](UPGRADE.rst) for more details on these changes and for general upgrade guidance. Features -------- - Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015), [\#9017](https://github.com/matrix-org/synapse/issues/9017), [\#9036](https://github.com/matrix-org/synapse/issues/9036), [\#9067](https://github.com/matrix-org/synapse/issues/9067), [\#9081](https://github.com/matrix-org/synapse/issues/9081), [\#9082](https://github.com/matrix-org/synapse/issues/9082), [\#9105](https://github.com/matrix-org/synapse/issues/9105), [\#9107](https://github.com/matrix-org/synapse/issues/9107), [\#9109](https://github.com/matrix-org/synapse/issues/9109), [\#9110](https://github.com/matrix-org/synapse/issues/9110), [\#9127](https://github.com/matrix-org/synapse/issues/9127), [\#9153](https://github.com/matrix-org/synapse/issues/9153), [\#9154](https://github.com/matrix-org/synapse/issues/9154), [\#9177](https://github.com/matrix-org/synapse/issues/9177)) - During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091)) - Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159)) - Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024)) - Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984)) - Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086)) - Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932)) - Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948)) - Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042), [\#9043](https://github.com/matrix-org/synapse/issues/9043), [\#9044](https://github.com/matrix-org/synapse/issues/9044), [\#9130](https://github.com/matrix-org/synapse/issues/9130)) - Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068)) - Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092)) - Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104), [\#9166](https://github.com/matrix-org/synapse/issues/9166)) Bugfixes -------- - Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023)) - Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028)) - Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051)) - Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053)) - Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054)) - Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059)) - Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070)) - Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071)) - Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114), [\#9116](https://github.com/matrix-org/synapse/issues/9116)) - Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117)) - Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128)) - Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108)) - Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145)) - Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161)) Improved Documentation ---------------------- - Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997)) - Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035)) - Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040)) - Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057)) - Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151)) Deprecations and Removals ------------------------- - Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039)) Internal Changes ---------------- - Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868), [\#9029](https://github.com/matrix-org/synapse/issues/9029), [\#9115](https://github.com/matrix-org/synapse/issues/9115), [\#9118](https://github.com/matrix-org/synapse/issues/9118), [\#9124](https://github.com/matrix-org/synapse/issues/9124)) - Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939)) - Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016)) - Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018)) - Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025)) - Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030)) - Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031)) - Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033)) - Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038)) - Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041)) - Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055)) - Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058)) - Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063)) - Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069)) - Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080)) - Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093)) - Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098)) - Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106)) - Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112)) - Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125)) - Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144)) - Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146)) - Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAmAIVFkACgkQM/xY9qcR MEhLfQ//Z4xnP5Icx732k2xpdR+JaXX6heNVFm/UuLNd6wq+uXjO/Dgc9IKQ5LXD kfG+/OnNGDr/eUk4mQ5s/ccBvaTrOid2IIemJd4lUBy/Q5LvPvfKIp08QFr72WJT 7Y4Ma54nNSSMUAdESgj/aFAHjATbfHhxPOZ5OdQHGLAuJ/OUwR2ksasVNCuINbgh 8vrxrzjeYn1Zl3UTBrsdCcat7AFqYjxK/Y0i6JxfSgRwDZjUbW4M2C4OGWZJPUrU 2YTPHangxd+22+HObE2KDEzaHDdCR/Kj4dhCiJBynZ5iucuHSS+mgf39RtbjqWML 01cmXXUV+FH7rnCilmF9rYdmHZ6L9BJxtrQDQU4dB106BX9G7hKR6rYHc9Z6tjOh 3tg0cKWYMXOsTNnvRqiFNh2q0yBKS228HRAwyGosW/6NiYj43ArtYP6knX7dQyBX 0mYNkfry2dyX6Kj5el0i9MOTHYQxfc8apsBm2M3OTNYukgQKA0NhbAquibaAxEEW 2qIqbSp4CUlGPvM7+u+ZGTu0hEbVKKNjqMfXhuY8A4/BWiV2mSqKcEirR5cE/LqY mq5Ac7vbO4Uh+1xiRw/G9ITi1dqAjYIzhBawlhdUPOh+aINWTUikYYzjUSmm4PbN H9BTPLA9iAdVRfuWJ9um2G0DdS8Qpx/aIRh3MScXVly6cGCC6Do= =kgqn -----END PGP SIGNATURE----- Merge tag 'v1.26.0rc1' into develop Synapse 1.26.0rc1 (2021-01-20) ============================== This release brings a new schema version for Synapse and rolling back to a previous verious is not trivial. Please review [UPGRADE.rst](UPGRADE.rst) for more details on these changes and for general upgrade guidance. Features -------- - Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015), [\#9017](https://github.com/matrix-org/synapse/issues/9017), [\#9036](https://github.com/matrix-org/synapse/issues/9036), [\#9067](https://github.com/matrix-org/synapse/issues/9067), [\#9081](https://github.com/matrix-org/synapse/issues/9081), [\#9082](https://github.com/matrix-org/synapse/issues/9082), [\#9105](https://github.com/matrix-org/synapse/issues/9105), [\#9107](https://github.com/matrix-org/synapse/issues/9107), [\#9109](https://github.com/matrix-org/synapse/issues/9109), [\#9110](https://github.com/matrix-org/synapse/issues/9110), [\#9127](https://github.com/matrix-org/synapse/issues/9127), [\#9153](https://github.com/matrix-org/synapse/issues/9153), [\#9154](https://github.com/matrix-org/synapse/issues/9154), [\#9177](https://github.com/matrix-org/synapse/issues/9177)) - During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091)) - Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159)) - Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024)) - Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984)) - Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086)) - Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932)) - Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948)) - Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042), [\#9043](https://github.com/matrix-org/synapse/issues/9043), [\#9044](https://github.com/matrix-org/synapse/issues/9044), [\#9130](https://github.com/matrix-org/synapse/issues/9130)) - Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068)) - Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092)) - Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104), [\#9166](https://github.com/matrix-org/synapse/issues/9166)) Bugfixes -------- - Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023)) - Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028)) - Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051)) - Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053)) - Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054)) - Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059)) - Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070)) - Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071)) - Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114), [\#9116](https://github.com/matrix-org/synapse/issues/9116)) - Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117)) - Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128)) - Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108)) - Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145)) - Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161)) Improved Documentation ---------------------- - Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997)) - Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035)) - Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040)) - Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057)) - Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151)) Deprecations and Removals ------------------------- - Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039)) Internal Changes ---------------- - Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868), [\#9029](https://github.com/matrix-org/synapse/issues/9029), [\#9115](https://github.com/matrix-org/synapse/issues/9115), [\#9118](https://github.com/matrix-org/synapse/issues/9118), [\#9124](https://github.com/matrix-org/synapse/issues/9124)) - Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939)) - Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016)) - Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018)) - Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025)) - Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030)) - Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031)) - Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033)) - Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038)) - Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041)) - Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055)) - Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058)) - Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063)) - Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069)) - Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080)) - Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093)) - Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098)) - Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106)) - Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112)) - Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125)) - Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144)) - Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146)) - Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157))	2021-01-20 11:27:39 -05:00
Richard van der Hoff	0cd2938bc8	Support icons for Identity Providers (#9154)	2021-01-20 08:15:14 -05:00
rht	a5b9c87ac6	docs: Add link to Matrix VoIP tester for turn-howto (#9135) ... Signed-off-by: rht <rhtbot@protonmail.com>	2021-01-20 12:41:57 +00:00
Richard van der Hoff	fa50e4bf4d	Give public_baseurl a default value (#9159)	2021-01-20 12:30:41 +00:00
Patrick Cloke	de45bf5b5b	Quote pip install with brackets to avoid shell interpretation. (#9151)	2021-01-18 11:12:20 -05:00
Matthew Hodgson	883d4e6f2b	link to the scalability blog post from workers.md	2021-01-18 00:27:27 +00:00
Richard van der Hoff	9de6b94117	Land support for multiple OIDC providers (#9110) ... This is the final step for supporting multiple OIDC providers concurrently. First of all, we reorganise the config so that you can specify a list of OIDC providers, instead of a single one. Before: oidc_config: enabled: true issuer: "https://oidc_provider" # etc After: oidc_providers: - idp_id: prov1 issuer: "https://oidc_provider" - idp_id: prov2 issuer: "https://another_oidc_provider" The old format is still grandfathered in. With that done, it's then simply a matter of having OidcHandler instantiate a new OidcProvider for each configured provider.	2021-01-15 16:55:29 +00:00
Patrick Cloke	3e4cdfe5d9	Add an admin API endpoint to protect media. (#9086) ... Protecting media stops it from being quarantined when e.g. all media in a room is quarantined. This is useful for sticker packs and other media that is uploaded by server administrators, but used by many people.	2021-01-15 11:18:09 -05:00
Richard van der Hoff	5310808d3b	Give the user a better error when they present bad SSO creds ... If a user tries to do UI Auth via SSO, but uses the wrong account on the SSO IdP, try to give them a better error. Previously, the UIA would claim to be successful, but then the operation in question would simply fail with "auth fail". Instead, serve up an error page which explains the failure.	2021-01-13 20:22:41 +00:00
Patrick Cloke	d1eb1b96e8	Register the /devices endpoint on workers. (#9092)	2021-01-13 12:35:40 -05:00
Dirk Klimpel	7a2e9b549d	Remove user's avatar URL and displayname when deactivated. (#8932) ... This only applies if the user's data is to be erased.	2021-01-12 16:30:15 -05:00
Jason Robinson	da16d06301	Address pr feedback ... * docs updates * prettify SQL * add missing copyright * cursor_to_dict * update touched files copyright years Signed-off-by: Jason Robinson <jasonr@matrix.org>	2021-01-11 23:43:58 +02:00
Jason Robinson	0b77329fe2	Clarify rooms.md ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2021-01-11 23:05:36 +02:00
David Teller	b161528fcc	Also support remote users on the joined_rooms admin API. (#8948) ... For remote users, only the rooms which the server knows about are returned. Local users have all of their joined rooms returned.	2021-01-11 14:32:17 -05:00
Erik Johnston	1315a2e8be	Use a chain cover index to efficiently calculate auth chain difference (#8868)	2021-01-11 16:09:22 +00:00
Jason Robinson	e2c16edc78	Add changelog and admin API docs ... Signed-off-by: Jason Robinson <jasonr@matrix.org>	2021-01-09 22:58:29 +02:00
Christopher Rücker	bce0c91d9a	Keycloak mapping_provider example (#9037) (#9057) ... This PR adds the missing user_mapping_provider section in oidc.md Signed-off-by: Christopher Rücker chris-ruecker@protonmail.com	2021-01-08 18:29:30 +00:00
Emelie	9066c2fd7f	Fix typo in docs/systemd-with-workers/README.md (#9035) ... Signed-off-by: Emelie em@nao.sh	2021-01-07 15:31:01 +00:00
Richard van der Hoff	111b673fc1	Add initial support for a "pick your IdP" page (#9017) ... During login, if there are multiple IdPs enabled, offer the user a choice of IdPs.	2021-01-05 11:25:28 +00:00
Jerin J Titus	cfcf5541b4	Update the value of group_creation_prefix in sample config. (#8992) ... Removes the trailing slash with causes issues with matrix.to/Element.	2020-12-29 09:30:48 -05:00
Patrick Cloke	68bb26da69	Allow redacting events on workers (#8994) ... Adds the redacts endpoint to workers that have the client listener.	2020-12-29 07:40:12 -05:00
Patrick Cloke	4218473f9e	Refactor the CAS handler in prep for using the abstracted SSO code. (#8958) ... This makes the CAS handler look more like the SAML/OIDC handlers: * Render errors to users instead of throwing JSON errors. * Internal reorganization.	2020-12-18 13:09:45 -05:00
Patrick Cloke	56e00ca85e	Send the location of the web client to the IS when inviting via 3PIDs. (#8930) ... Adds a new setting `email.invite_client_location` which, if defined, is passed to the identity server during invites.	2020-12-18 11:01:57 -05:00
Erik Johnston	d781a81e69	Allow server admin to get admin bit in rooms where local user is an admin (#8756) ... This adds an admin API that allows a server admin to get power in a room if a local user has power in a room. Will also invite the user if they're not in the room and its a private room. Can specify another user (rather than the admin user) to be granted power. Co-authored-by: Matthew Hodgson <matthew@matrix.org>	2020-12-18 15:37:19 +00:00
Richard van der Hoff	28877fade9	Implement a username picker for synapse (#8942) ... The final part (for now) of my work to implement a username picker in synapse itself. The idea is that we allow `UsernameMappingProvider`s to return `localpart=None`, in which case, rather than redirecting the browser back to the client, we redirect to a username-picker resource, which allows the user to enter a username. We *then* complete the SSO flow (including doing the client permission checks). The static resources for the username picker itself (in https://github.com/matrix-org/synapse/tree/rav/username_picker/synapse/res/username_picker) are essentially lifted wholesale from https://github.com/matrix-org/matrix-synapse-saml-mozilla/tree/master/matrix_synapse_saml_mozilla/res. As the comment says, we might want to think about making them customisable, but that can be a follow-up. Fixes #8876.	2020-12-18 14:19:46 +00:00
Patrick Cloke	5d4c330ed9	Allow re-using a UI auth validation for a period of time (#8970)	2020-12-18 07:33:57 -05:00
Dirk Klimpel	06006058d7	Make search statement in List Room and User Admin API case-insensitive (#8931)	2020-12-17 10:43:37 +00:00
Patrick Cloke	44b7d4c6d6	Fix the sample config location for the ip_range_whitelist setting. (#8954) ... Move it from the federation section to the server section to match ip_range_blacklist.	2020-12-16 14:40:47 -05:00
David Teller	f14428b25c	Allow spam-checker modules to be provide async methods. (#8890) ... Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.	2020-12-11 14:05:15 -05:00
Dirk Klimpel	0a34cdfc66	Add number of local devices to Room Details Admin API (#8886)	2020-12-11 10:42:47 +00:00
Dirk Klimpel	a5f7aff5e5	Deprecate Shutdown Room and Purge Room Admin API (#8829) ... Deprecate both APIs in favour of the Delete Room API. Related: #8663 and #8810	2020-12-10 11:42:48 +00:00
Patrick Cloke	344ab0b53a	Default to blacklisting reserved IP ranges and add a whitelist. (#8870) ... This defaults `ip_range_blacklist` to reserved IP ranges and also adds an `ip_range_whitelist` setting to override it.	2020-12-09 13:56:06 -05:00
Dirk Klimpel	43bf3c5178	Combine related media admin API docs (#8839) ... Related: #8810 Also a few small improvements. Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-12-09 16:19:57 +00:00
Richard van der Hoff	025fa06fc7	Clarify config template comments (#8891)	2020-12-08 14:03:08 +00:00
Patrick Cloke	96358cb424	Add authentication to replication endpoints. (#8853) ... Authentication is done by checking a shared secret provided in the Synapse configuration file.	2020-12-04 10:56:28 -05:00
Patrick Cloke	112f6bd49e	Synapse 1.24.0rc2 (2020-12-04) ... ============================== Bugfixes -------- - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878)) Internal Changes ---------------- - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl/KQ6MACgkQM/xY9qcR MEjqOg/9Fd4Ow0QdnK3O6J0bmXP6+g2GfjJCAKlYlF/nwX1ez9jZVed00D2h9/hD YCUJCEv+XZpbv5F4usiYwoLGithbueo7AKEOQJQMEuQBHU1/E6TJ9iiKR3lTBNNw ccuyMMqqEatYxoa4DIO/lrwm2mhwRNdMPt0r1DPlML13bb/TVbeXXykbOCyZSKm1 YuXpxYDngRPL9o46I77l7/KNTSpYYeZi2qwp2orl6GBfa0KSvR2Oi6uohKYzuoHC +XLvHSFMpHAZkaZsIMxU5hRVH6jtkb/D37CABuKQsdlvHcooGK4Sdt6aoUyhPtoh 6YwajSazWKEJ/9YO5JV22qkWFPi9T6s/lPZfjOSu/euwMw7DDQJ/8t8Sm6N7sJDS A/5FctOAd1jd9BiAaqkHB2zLHZSomIHQpJJfDyE3UgAGQQ6fm2Wg7SkLByDx/MYC 4D74dBuVBPENM2VxNoyjHbpVSeQi8t0RgWb44/PUAOtpwm3f3eVRuS8zY6uR2bZr YZUbJylqKhRW6dBdZH+EzVolRGvBgLkc27IzRCQ3kGUEo2Wem05s6MuKMvLxoIvN NHLWNClUWITT8FgkfFP3c6CK40dwIGJzYryI+71qPW/R7snf2fazuoHlH1pAmFh0 E0/nBMTw1deixOYXbG1ah63AhJ1NjdTfimDsKmWZLqVc6g7g4d4= =is1Y -----END PGP SIGNATURE----- Merge tag 'v1.24.0rc2' into develop Synapse 1.24.0rc2 (2020-12-04) ============================== Bugfixes -------- - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878)) Internal Changes ---------------- - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))	2020-12-04 09:14:31 -05:00
Patrick Cloke	22c6c19f91	Fix a regression that mapping providers should be able to redirect users. (#8878) ... This was broken in #8801.	2020-12-04 08:25:15 -05:00
Richard van der Hoff	6e4f71c057	Fix a buglet in the SAML username mapping provider doc (#8873) ... the constructor is called with a `module_api`.	2020-12-04 10:14:15 +00:00
Patrick Cloke	30fba62108	Apply an IP range blacklist to push and key revocation requests. (#8821) ... Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).	2020-12-02 11:09:24 -05:00
Andrew Morgan	d1be293f00	Fix typo in password_auth_providers doc ... A word got removed accidentally in 83434df381.	2020-12-01 10:34:52 +00:00
Andrew Morgan	17fa58bdd1	Add a config option to change whether unread push notification counts are per-message or per-room (#8820) ... This PR adds a new config option to the `push` section of the homeserver config, `group_unread_count_by_room`. By default Synapse will group push notifications by room (so if you have 1000 unread messages, if they lie in 55 rooms, you'll see an unread count on your phone of 55). However, it is also useful to be able to send out the true count of unread messages if desired. If `group_unread_count_by_room` is set to `false`, then with the above example, one would see an unread count of 1000 (email anyone?).	2020-11-30 18:43:54 +00:00
Richard van der Hoff	a090b86209	Add force_purge option to delete-room admin api. (#8843)	2020-11-30 16:48:12 +00:00
Dirk Klimpel	14f81a6d24	Improve documentation how to configure prometheus for workers (#8822)	2020-11-26 10:42:55 +00:00
Dirk Klimpel	3f0ff53158	Remove deprecated /_matrix/client/*/admin endpoints (#8785) ... These are now only available via `/_synapse/admin/v1`.	2020-11-25 16:26:11 -05:00
Andrew Morgan	2b110dda2a	Fix the formatting of push config section (#8818) ... This PR updates the push config's formatting to better align with our [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format).	2020-11-25 21:02:53 +00:00
Patrick Cloke	4fd222ad70	Support trying multiple localparts for OpenID Connect. (#8801) ... Abstracts the SAML and OpenID Connect code which attempts to regenerate the localpart of a matrix ID if it is already in use.	2020-11-25 10:04:22 -05:00
Dirk Klimpel	b08dc7effe	Clarify documentation of the admin list media API (#8795) ... Clarify that the list media API only shows media from unencrypted events.	2020-11-24 09:04:51 -05:00
Richard van der Hoff	e3d7806704	Update turn-howto (#8779) ... Some hopefully-useful notes on setting up a turnserver.	2020-11-24 12:52:22 +00:00
Patrick Cloke	79bfe966e0	Improve error checking for OIDC/SAML mapping providers (#8774) ... Checks that the localpart returned by mapping providers for SAML and OIDC are valid before registering new users. Extends the OIDC tests for existing users and invalid data.	2020-11-19 14:25:17 -05:00
Ben Banfield-Zanin	53a6f5ddf0	SAML: Allow specifying the IdP entityid to use. (#8630) ... If the SAML metadata includes multiple IdPs it is necessary to specify which IdP to redirect users to for authentication.	2020-11-19 09:57:13 -05:00
Marcus Schopen	d356588339	SAML: Document allowing a clock/time difference from IdP (#8731) ... Updates the sample configuration with the pysaml2 configuration for accepting clock skew/drift between the homeserver and IdP.	2020-11-18 07:36:28 -05:00
chagai95	e487d9fabc	a comma too much (#8771) ... Signed-off-by: Chagai Friedlander chagai95@gmail.com	2020-11-17 14:13:56 +00:00
Erik Johnston	f737368a26	Add admin API for logging in as a user (#8617)	2020-11-17 10:51:25 +00:00
Adrian Wannenmacher	f1de4bb58b	Clarify the usecase for an msisdn delegate (#8734) ... Signed-off-by: Adrian Wannenmacher <tfld@tfld.dev>	2020-11-14 23:09:36 +00:00
Dirk Klimpel	023f791143	Migrate documentation docs/admin_api/event_reports to markdown (#8742) ... Related to #8714. `event_reports.rst` was introduced in Synapse 1.21.0.	2020-11-13 13:57:55 +00:00
Marcus Schopen	68fc0dcb5a	SAML: add <mdui:UIInfo> element examples (#8718) ... add some mdui:UIInfo element examples for saml2_config in homeserver.yaml	2020-11-13 12:07:50 +00:00
Marcus Schopen	c059413001	Notes on SSO logins and media_repository worker (#8701) ... If SSO login is used (e.g. SAML) in a multi worker setup, it should be mentioned that currently all SAML logins must run on the same worker, see https://github.com/matrix-org/synapse/issues/7530 Also, if you are using different ports (for example 443 and 8448) in a reverse proxy for client and federation, the path `/_matrix/media` on the client and federation port must point to the listener of the `media_repository` worker, otherwise you'll get a 404 on the federation port for the path `/_matrix/media`, if a remote server is trying to get the media object on federation port, see https://github.com/matrix-org/synapse/issues/8695	2020-11-06 14:33:07 +00:00
Dirk Klimpel	c3119d1536	Add an admin API for users' media statistics (#8700) ... Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users. Related to #6094 It is the first API for statistics. Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats) Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-11-05 18:59:12 +00:00
Dirk Klimpel	e4676bd877	Add displayname to Shared-Secret Registration for admins (#8722) ... Add `displayname` to Shared-Secret Registration for admins to `POST /_synapse/admin/v1/register`	2020-11-05 13:55:45 +00:00
Dirk Klimpel	4fda58ddd2	Remove the "draft" status of the Room Details Admin API (#8702) ... Fixes #8550	2020-11-03 12:48:25 +00:00
Erik Johnston	4b09b7438e	Document how to set up multiple event persisters (#8706)	2020-11-03 10:27:11 +00:00
Matthew Hodgson	d04c2d19b3	grammar	2020-11-02 21:22:36 +00:00
Matthew Hodgson	11fd90a2b7	typo	2020-11-02 13:33:56 +00:00
Andrew Morgan	26b46796ea	Fix typos in systemd-with-workers doc	2020-11-02 12:56:16 +00:00
Andrew Morgan	305545682d	Fix typo in workers doc	2020-11-02 12:36:18 +00:00
Patrick Cloke	8f1aefa694	Improve the sample config for SSO (OIDC, SAML, and CAS). (#8635)	2020-10-30 10:01:59 -04:00
Patrick Cloke	00b24aa545	Support generating structured logs in addition to standard logs. (#8607) ... This modifies the configuration of structured logging to be usable from the standard Python logging configuration. This also separates the formatting of logs from the transport allowing JSON logs to files or standard logs to sockets.	2020-10-29 07:27:37 -04:00
Dirk Klimpel	2239813278	Add an admin APIs to allow server admins to list users' pushers (#8610) ... Add an admin API `GET /_synapse/admin/v1/users/<user_id>/pushers` like https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-pushers	2020-10-28 15:02:42 +00:00
Michael Kaye	f49c2093b5	Cross-link documentation to the prometheus recording rules. (#8667)	2020-10-27 15:29:50 -04:00
Dirk Klimpel	9b7c28283a	Add admin API to list users' local media (#8647) ... Add admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information of users' uploaded files.	2020-10-27 14:12:31 +00:00
Dirk Klimpel	66e6801c3e	Split admin API for reported events into a detail and a list view (#8539) ... Split admin API for reported events in detail und list view. API was introduced with #8217 in synapse v.1.21.0. It makes the list (`GET /_synapse/admin/v1/event_reports`) less complex and provides a better overview. The details can be queried with: `GET /_synapse/admin/v1/event_reports/<report_id>`. It is similar to room and users API. It is a kind of regression in `GET /_synapse/admin/v1/event_reports`. `event_json` was removed. But the api was introduced one version before and it is an admin API (not under spec). Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-10-26 18:16:37 +00:00
Peter Krantz	6c9ab61df5	Added basic instructions for Azure AD to OpenId documentation (#8582) ... Signed-off-by: Peter Krantz peter.krantz@gmail.com	2020-10-26 17:49:55 +00:00
Dirk Klimpel	49d72dea2a	Add an admin api to delete local media. (#8519) ... Related to: #6459, #3479 Add `DELETE /_synapse/admin/v1/media/<server_name>/<media_id>` to delete a single file from server.	2020-10-26 17:02:28 +00:00
Andrew Morgan	f6a3859a73	Fix filepath of Dex example config (#8657)	2020-10-26 16:53:11 +00:00
Andrew Morgan	73d8209694	Correct the package name in OpenID Connect install instructions (#8634) ... The OpenID Connect install instructions suggested installing `synapse[oidc]`, but our PyPI package is called `matrix-synapse`.	2020-10-26 14:45:33 +00:00
Dirk Klimpel	913f8a06e4	Add field total to device list in admin API (#8644)	2020-10-26 14:07:51 +00:00
Patrick Cloke	34a5696f93	Fix typos and spelling errors. (#8639)	2020-10-23 12:38:40 -04:00
Andrew Morgan	4fb7a68a65	Correct the package name in authlib install instructions	2020-10-22 18:25:58 +01:00
Christopher May-Townsend	1cf4a68108	Add note to manhole.md about bind_address when using with docker (#8526) ... Signed-off-by: Christopher May-Townsend <chris@maytownsend.co.uk>	2020-10-14 15:28:59 +01:00
Brendan Abolivier	9e66f3761c	Update documentation on retention policies limits (#8529) ... * Update documentation on retention policies limits Document the changes from https://github.com/matrix-org/synapse/pull/8104	2020-10-14 15:00:49 +01:00
Erik Johnston	8de3703d21	Make event persisters periodically announce position over replication. (#8499) ... Currently background proccesses stream the events stream use the "minimum persisted position" (i.e. `get_current_token()`) rather than the vector clock style tokens. This is broadly fine as it doesn't matter if the background processes lag a small amount. However, in extreme cases (i.e. SyTests) where we only write to one event persister the background processes will never make progress. This PR changes it so that the `MultiWriterIDGenerator` keeps the current position of a given instance as up to date as possible (i.e using the latest token it sees if its not in the process of persisting anything), and then periodically announces that over replication. This then allows the "minimum persisted position" to advance, albeit with a small lag.	2020-10-12 15:51:41 +01:00
Mateusz Przybyłowicz	ca2db5dd0c	Increase default max_upload_size from 10M to 50M (#8502) ... Signed-off-by: Mateusz Przybyłowicz <uamfhq@gmail.com>	2020-10-09 16:58:23 +01:00
Richard van der Hoff	4f0637346a	Combine SpamCheckerApi with the more generic ModuleApi. (#8464) ... Lots of different module apis is not easy to maintain. Rather than adding yet another ModuleApi(hs, hs.get_auth_handler()) incantation, first add an hs.get_module_api() method and use it where possible.	2020-10-07 12:03:26 +01:00
Andrew Morgan	01f82bfe32	Remove docs/sphinx and related references (#8480) ... https://github.com/matrix-org/synapse/tree/develop/docs/sphinx doesn't seem to really be utilised or changed recently since the initial commit. I like the idea of exportable documentation of the codebase, but at the moment after running through the build instructions the generated website wasn't very useful...	2020-10-07 11:45:31 +01:00
Richard van der Hoff	785437dc0d	Update default room version to 6 (#8461) ... Per https://github.com/matrix-org/matrix-doc/pull/2788	2020-10-05 21:40:51 +01:00
Patrick Cloke	f64c6aae68	Update manhole documentation for async/await. (#8462)	2020-10-05 09:40:19 -04:00
Patrick Cloke	62894673e6	Allow background tasks to be run on a separate worker. (#8369)	2020-10-02 08:23:15 -04:00
BBBSnowball	05ee048f2c	Add config option for always using "userinfo endpoint" for OIDC (#7658) ... This allows for connecting to certain IdPs, e.g. GitLab.	2020-10-01 13:54:35 -04:00
Richard van der Hoff	c1ef579b63	Add prometheus metrics to track federation delays (#8430) ... Add a pair of federation metrics to track the delays in sending PDUs to/from particular servers.	2020-10-01 11:09:12 +01:00
Patrick Cloke	8b40843392	Allow additional SSO properties to be passed to the client (#8413)	2020-09-30 13:02:43 -04:00
Aaron Raimist	8238b55e08	Update description of server_name config option (#8415)	2020-09-29 13:50:25 -04:00
Erik Johnston	bd380d942f	Add checks for postgres sequence consistency (#8402)	2020-09-28 18:00:30 +01:00
Tdxdxoz	abd04b6af0	Allow existing users to login via OpenID Connect. (#8345) ... Co-authored-by: Benjamin Koch <bbbsnowball@gmail.com> This adds configuration flags that will match a user to pre-existing users when logging in via OpenID Connect. This is useful when switching to an existing SSO system.	2020-09-25 07:01:45 -04:00
Julian Fietkau	a4e63e5a47	Add note to reverse_proxy.md about disabling Apache's mod_security2 (#8375) ... This change adds a note and a few lines of configuration settings for Apache users to disable ModSecurity for Synapse's virtual hosts. With ModSecurity enabled and running with its default settings, Matrix clients are unable to send chat messages through the Synapse installation. With this change, ModSecurity can be disabled only for the Synapse virtual hosts.	2020-09-23 11:14:08 +01:00
Dirk Klimpel	4da01f9c61	Admin API for reported events (#8217) ... Add an admin API to read entries of table `event_reports`. API: `GET /_synapse/admin/v1/event_reports`	2020-09-22 18:15:04 +01:00
Dirk Klimpel	d688b4bafc	Admin API for querying rooms where a user is a member (#8306) ... Add a new admin API `GET /_synapse/admin/v1/users/<user_id>/joined_rooms` to list all rooms where a user is a member.	2020-09-18 15:26:36 +01:00
Tulir Asokan	b82d68c0bd	Add the topic and avatar to the room details admin API (#8305)	2020-09-14 10:07:04 -04:00
Patrick Cloke	6605470bfb	Improve SAML error messages (#8248)	2020-09-14 09:05:36 -04:00
Patrick Cloke	a9dbe98ef9	Synapse 1.20.0rc3 (2020-09-11) ... ============================== Bugfixes -------- - Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl9bbA8ACgkQM/xY9qcR MEjjJg//ZGcvPLr8y3F5JXptcjWA3AFja3DgztBA3uNFoDrwVhR9m3+F4rL/Fbat gnaccWsKIXOjZw5WOThQEUIfYMKmHEsaDl7xtlUPC4ylmSV+31ypu1KiRSxND2kT TmOupvqw+4/E4RajpYX6WT3e2oQUpIcBAKsUyZL3ZXECR/yNYrA6w2w7i4wrv3cU QDzBFrCcr5aJql7VUc88BlaZUG6xom2/kkPtjmO6imPnPGBHLN22uPU7zQ4n7Vgi Nrg45v5WHVCx57WoXqAEap1zdKgoRCna1x0NkqYh0OAXq1l6aVwlf/Pdynt91f2x yyaMYjMskjyatHlzONMV4kz0w03dUrGJXiAx2ldEDd32SKBUxLJ/CYtNEdoXZ4Mm o0OJfDbaSJwlqK7FhdZJE3kCSkUvlmVasDErXfQRDVHy5sx9Ufr4P0kkaZyXDys1 UAWFcCJwy9dgFR2679PoXk1s/gHf10wuk6FOs8ESbcJJOvCljxWrOIqKNzSUL3HQ Hj5V1zHYXB6HtgiI7tbKxSn/d2uajHI9b8LVOkyV9RKsb1DGGRgNQQ5Kz4Zud7dC vuQ2jVw7JIo41W3QIXvcL1KdQeje2nwYwuwNREZaEDZJsGLjxaJ6dNyBVgkmYYFP gxhplSOSsFjQRGEsBrc2utWxDnLVGPHEAt7iHVSqjwVUyARvxjo= =e2T1 -----END PGP SIGNATURE----- Merge tag 'v1.20.0rc3' into develop Synapse 1.20.0rc3 (2020-09-11) ============================== Bugfixes -------- - Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))	2020-09-11 08:30:36 -04:00
Andrew Morgan	95d869c357	Add /_synapse/client to the reverse proxy docs (#8227) ... This PR adds a information about forwarding `/_synapse/client` endpoints through your reverse proxy. The first of these endpoints are introduced in https://github.com/matrix-org/synapse/pull/8004.	2020-09-10 13:26:34 +01:00
Andrew Morgan	192e98111d	Remove shared rooms info from upgrade/workers doc as it's still experimental (#8290)	2020-09-10 13:08:08 +01:00
Andrew Morgan	a3a90ee031	Show a confirmation page during user password reset (#8004) ... This PR adds a confirmation step to resetting your user password between clicking the link in your email and your password actually being reset. This is to better align our password reset flow with the industry standard of requiring a confirmation from the user after email validation.	2020-09-10 11:45:12 +01:00
Andrew Morgan	094896a69d	Add a config option for validating 'next_link' parameters against a domain whitelist (#8275) ... This is a config option ported over from DINUM's Sydent: https://github.com/matrix-org/sydent/pull/285 They've switched to validating 3PIDs via Synapse rather than Sydent, and would like to retain this functionality. This original purpose for this change is phishing prevention. This solution could also potentially be replaced by a similar one to https://github.com/matrix-org/synapse/pull/8004, but across all `*/submit_token` endpoint. This option may still be useful to enterprise even with that safeguard in place though, if they want to be absolutely sure that their employees don't follow links to other domains.	2020-09-08 16:03:09 +01:00
Richard van der Hoff	ad28030c12	Systemd docs: configure workers to start after main process. (#8276)	2020-09-08 10:57:43 +01:00
Will Hunt	b257c788c0	Add /user/{user_id}/shared_rooms/ api (#7785) ... * Add shared_rooms api * Add changelog * Add . * Wrap response in {"rooms": } * linting * Add unstable_features key * Remove options from isort that aren't part of 5.x `-y` and `-rc` are now default behaviour and no longer exist. `dont-skip` is no longer required https://timothycrosley.github.io/isort/CHANGELOG/#500-penny-july-4-2020 * Update imports to make isort happy * Add changelog * Update tox.ini file with correct invocation * fix linting again for isort * Vendor prefix unstable API * Fix to match spec * import Codes * import Codes * Use FORBIDDEN * Update changelog.d/7785.feature Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Implement get_shared_rooms_for_users * a comma * trailing whitespace * Handle the easy feedback * Switch to using runInteraction * Add tests * Feedback * Seperate unstable endpoint from v2 * Add upgrade node * a line * Fix style by adding a blank line at EOF. * Update synapse/storage/databases/main/user_directory.py Co-authored-by: Tulir Asokan <tulir@maunium.net> * Update synapse/storage/databases/main/user_directory.py Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Update UPGRADE.rst Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix UPGRADE/CHANGELOG unstable paths unstable unstable unstable Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-authored-by: Tulir Asokan <tulir@maunium.net> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Tulir Asokan <tulir@maunium.net>	2020-09-02 13:18:40 +01:00
Brendan Abolivier	b5133dd97f	Explain better what GDPR-erased means (#8189) ... Fixes https://github.com/matrix-org/synapse/issues/8185	2020-09-01 16:31:59 +01:00
Andrew Morgan	74bf8d4d06	Wording fixes to 'name' user admin api filter (#8163) ... Some fixes to wording I noticed after merging #7377.	2020-08-25 15:03:24 +01:00
Andrew Morgan	79ac619403	Fix missing double-backtick in RST document	2020-08-25 14:24:06 +01:00
Manuel Stahl	97962ad17b	Search in columns 'name' and 'displayname' in the admin users endpoint (#7377) ... * Search in columns 'name' and 'displayname' in the admin users endpoint Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	2020-08-25 14:18:14 +01:00
Brendan Abolivier	420484a334	Allow capping a room's retention policy (#8104)	2020-08-24 18:21:04 +01:00
Ryan Cole	cfeb37f039	Updated docs: Added note about missing 308 redirect support. (#8120) ... * Updated docs: Added note about missing 308 redirect support. * Added changelog	2020-08-19 12:26:50 +01:00
Patrick Cloke	acfb7c3b5d	Add a link to the matrix-synapse-rest-password-provider. (#8111)	2020-08-18 09:54:35 -04:00
Andrew Morgan	e04e465b4d	Use the default templates when a custom template file cannot be found (#8037) ... Fixes https://github.com/matrix-org/synapse/issues/6583	2020-08-17 17:05:00 +01:00
Erik Johnston	0304ad0c3d	Move setting of Filter into code. ... We do this to prevent foot guns. The default config uses a MemoryFilter, but users are free to change to logging to files directly. If they do then they have to ensure to set the `filters: [context]` on the right handler, otherwise records get written with the wrong context. Instead we move the logic to happen when we generate a record, which is when we *log* rather than *handle*. (It's possible to add filters to loggers in the config, however they don't apply to descendant loggers and so they have to be manually set on *every* logger used in the code base)	2020-08-11 18:10:46 +01:00
Erik Johnston	db131b6b22	Change the default log config to reduce disk I/O and storage (#8040) ... * Change default log config to buffer by default. This batches up writes to the filesystem, which is more efficient for disk I/O. This means that it can take some time for logs to get written to disk. Note that ERROR logs (and above) immediately flush the buffer. This only effects new installs, as we only write the log config if started with `--generate-config` (in the same way we do for generating signing keys). * Default to keeping last 4 days of logs. This hopefully reduces the amount of logs kept for new servers. Keeping the last 1GB of logs is likely overkill for new servers, but equally may not be enough for busy ones. Instead, we keep the last four days worth of logs, enough so that admins can investigate any problems that happened over e.g. a long weekend.	2020-08-11 18:09:46 +01:00
Richard van der Hoff	0cb169900e	Implement login blocking based on SAML attributes (#8052) ... Hopefully this mostly speaks for itself. I also did a bit of cleaning up of the error handling. Fixes #8047	2020-08-11 16:08:10 +01:00
Travis Ralston	1048ed2afa	Clarify that undoing a shutdown might not be possible (#8010)	2020-08-07 17:16:24 +01:00
Erik Johnston	7620912d84	Add health check endpoint (#8048)	2020-08-07 14:21:24 +01:00
Erik Johnston	079bc3c8e3	Fixup worker doc (again) (#8000)	2020-08-06 10:35:59 +01:00
Erik Johnston	a7bdf98d01	Rename database classes to make some sense (#8033)	2020-08-05 21:38:57 +01:00
Erik Johnston	faba873d4b	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/add_rate_limiting_to_joins	2020-07-31 15:07:01 +01:00
Erik Johnston	18de00adb4	Add ratelimiting on joins	2020-07-31 15:06:56 +01:00
Travis Ralston	e2a4ba6f9b	Add docs for undoing room shutdowns (#7998) ... These docs were tested successfully in production by a customer, so it's probably fine.	2020-07-31 04:41:44 +01:00
Stuart Mumford	6d4b790021	Update workers docs (#7990)	2020-07-30 17:30:11 +01:00
Erik Johnston	606805bf06	Fix typo in docs/workers.md (#7992)	2020-07-30 16:28:36 +01:00
Olivier Wilkinson (reivilibre)	3aa36b782c	Merge branch 'master' into develop	2020-07-30 15:18:36 +01:00
Erik Johnston	2c1b9d6763	Update worker docs with recent enhancements (#7969)	2020-07-29 23:22:13 +01:00
Aaron Raimist	2184f61fae	Various improvements to the docs (#7899)	2020-07-29 10:35:44 -04:00
Dirk Klimpel	e866e3b896	Add an option to disable purge in delete room admin API (#7964) ... Add option ```purge``` to ```POST /_synapse/admin/v1/rooms/<room_id>/delete``` Fixes: #3761 Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-07-28 20:08:23 +01:00
lugino-emeritus	3857de2194	Option to allow server admins to join complex rooms (#7902) ... Fixes #7901. Signed-off-by: Niklas Tittjung <nik_t.01@web.de>	2020-07-28 13:41:44 +01:00
Erik Johnston	aaf9ce72a0	Fix typo in metrics docs (#7966)	2020-07-28 10:03:18 +01:00
Patrick Cloke	83434df381	Update the auth providers to be async. (#7935)	2020-07-23 15:45:39 -04:00
Brendan Abolivier	55f2617f8c	Update the dates for ACME v1 EOL ... As per https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430	2020-07-22 16:18:40 +01:00
Adrian	64d2280299	Fix a typo in the sample config. (#7890)	2020-07-20 13:42:52 -04:00
Andrew Morgan	5ecf98f59e	Change sample config's postgres user to synapse_user (#7889) ... The [postgres setup docs](https://github.com/matrix-org/synapse/blob/develop/docs/postgres.md#set-up-database) recommend setting up your database with user `synapse_user`. However, uncommenting the postgres defaults in the sample config leave you with user `synapse`. This PR switches the sample config to recommend `synapse_user`. Took a me a second to figure this out, so assume this will beneficial to others.	2020-07-20 18:29:25 +01:00
Patrick Cloke	852930add7	Add a default limit (of 100) to get/sync operations. (#7858)	2020-07-17 07:59:23 -04:00
Michael Albert	fff483ea96	Add admin endpoint to get members in a room. (#7842)	2020-07-16 16:43:23 -04:00
Luke Faraone	b0f031f92a	Combine nginx federation server blocks (#7823) ... I'm pretty sure there's no technical reason these have to be distinct server blocks, so collapse into one and go with the more terse location block. Signed-off-by: Luke W Faraone <luke@faraone.cc>	2020-07-16 16:01:45 +01:00
Patrick Cloke	8c7d0f163d	Allow accounts to be re-activated from the admin APIs. (#7847)	2020-07-15 11:00:21 -04:00
Patrick Cloke	111e70d75c	Return the proper 403 Forbidden error during errors with JWT logins. (#7844)	2020-07-15 07:10:21 -04:00
Brendan Abolivier	85223106f3	Allow email subjects to be customised through Synapse's configuration (#7846)	2020-07-14 19:10:42 +01:00
Dirk Klimpel	491f0dab1b	Add delete room admin endpoint (#7613) ... The Delete Room admin API allows server admins to remove rooms from server and block these rooms. `DELETE /_synapse/admin/v1/rooms/<room_id>` It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md)" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md)" API. Fixes: #6425 It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`. It should return `None` if the room is unknown. But it returns an `IndexError`. 901b1fa561/synapse/storage/data_stores/main/room.py (L99-L105) Related to: - #5575 - https://github.com/Awesome-Technologies/synapse-admin/issues/17 Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-07-14 12:36:23 +01:00
Patrick Cloke	77d2c05410	Add the option to validate the iss and aud claims for JWT logins. (#7827)	2020-07-14 07:16:43 -04:00
Erik Johnston	f299441cc6	Add ability to shard the federation sender (#7798)	2020-07-10 18:26:36 +01:00
Nicolai Søborg	96bb01d8ec	Change Caddy links (old is deprecated) (#7789) ... * Change Caddy links Current links points to Caddy v1 which is deprecated. Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>	2020-07-08 10:09:16 +01:00
Patrick Cloke	2a266f4511	Add documentation for JWT login type and improve sample config. (#7776)	2020-07-06 08:31:51 -04:00
Patrick Cloke	71cccf1593	Additional configuration options for auto-join rooms (#7763)	2020-06-30 15:41:36 -04:00
Erik Johnston	b44bdd7f7b	Support running multiple media repos. (#7706) ... This requires a new config option to specify which media repo should be responsible for running background jobs to e.g. clear out expired URL preview caches.	2020-06-17 14:13:30 +01:00
Richard van der Hoff	e452973fd2	fix broken link in sample config (#7712)	2020-06-16 19:50:16 +01:00
hungrymonkey	5c5516f80e	Add instructions for authing with Keycloak via OpenID (#7659)	2020-06-16 11:28:21 -04:00
Patrick Cloke	b9df7f70bb	Increase the default SAML session expirary time to 15 minutes. (#7664)	2020-06-11 07:55:45 -04:00
wondratsch	c746889bb0	fix typo in sample_config.yaml (#7652) ... Just a simple typo fix. Signed-off-by: wondratsch 28294257+wondratsch@users.noreply.github.com	2020-06-11 11:51:10 +01:00
Andrew Morgan	fcd6961441	Add option to enable encryption by default for new rooms (#7639) ... Fixes https://github.com/matrix-org/synapse/issues/2431 Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used. Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637 Based on #7637	2020-06-10 17:44:34 +01:00
Travis Ralston	09099313e6	Add an option to disable autojoin for guest accounts (#6637) ... Fixes https://github.com/matrix-org/synapse/issues/3177	2020-06-05 18:18:15 +01:00
Richard van der Hoff	1bc00fd76d	Clarifications to the admin api documentation (#7647) ... * Clarify how to authenticate * path params are not the same thing as query params * Fix documentation for `/_synapse/admin/v2/users/<user_id>`	2020-06-05 17:31:05 +01:00
Dirk Klimpel	2970ce8367	Add device management to admin API (#7481) ... - Admin is able to - change displaynames - delete devices - list devices - get device informations Fixes #7330	2020-06-05 13:07:22 +01:00
Richard van der Hoff	11de843626	Cleanups to the OpenID Connect integration (#7628) ... docs, default configs, comments. Nothing very significant.	2020-06-03 21:13:17 +01:00
Richard van der Hoff	1bbc9e2df6	Clean up exception handling in SAML2ResponseResource (#7614) ... * Expose `return_html_error`, and allow it to take a Jinja2 template instead of a raw string * Clean up exception handling in SAML2ResponseResource * use the existing code in `return_html_error` instead of re-implementing it (giving it a jinja2 template rather than inventing a new form of template) * do the exception-catching in the REST layer rather than in the handler layer, to make sure we catch all exceptions.	2020-06-03 10:41:12 +01:00
Christopher Cooper	c4a820b32a	allow emails to be passed through SAML (#7385) ... Signed-off-by: Christopher Cooper <cooperc@ocf.berkeley.edu>	2020-05-27 17:40:08 +01:00
Jason Robinson	4be968d05d	Fix sample config docs error (#7581) ... 'client_auth_method' commented out value was erronously 'client_auth_basic', when code and docstring says it should be 'client_secret_basic'. Signed-off-by: Jason Robinson <jasonr@matrix.org>	2020-05-27 13:52:18 +01:00
Erik Johnston	d7d8a2e7ee	Fix up comments	2020-05-27 13:34:46 +01:00
Erik Johnston	4ba55559ac	Fix specifying cache factors via env vars with * in name. (#7580) ... This mostly applise to `*stateGroupCache*` and co. Broke in #6391.	2020-05-27 13:17:01 +01:00
Richard van der Hoff	66a564c859	Fix some DETECTED VIOLATIONS in the config file (#7550) ... consistency ftw	2020-05-22 10:11:50 +01:00
David Vo	d74cdc1a42	Ensure worker config exists in systemd service (#7528)	2020-05-21 13:47:23 +01:00
Richard van der Hoff	4fa74c7606	Minor clarifications to the TURN docs (#7533)	2020-05-20 11:04:34 +01:00
Paul Tötterman	ab3e19d814	Improve API doc readability (#7527)	2020-05-19 11:20:23 +01:00
Richard van der Hoff	24d9151a08	Formatting for reverse-proxy docs (#7514) ... also a small clarification to nginx	2020-05-15 15:13:39 +01:00
Jeff Peeler	572b444dab	Add Caddy 2 example (#7463) ... The specific headers that are passed using this new configuration format are Host and X-Forwarded-For, which should be all that's required. Note that for production another matcher should be added in the first section to properly handle the base_url lookup: reverse_proxy /.well-known/matrix/* http://localhost:8008 Signed-off-by: Jeff Peeler <jpeeler@gmail.com>	2020-05-15 14:36:01 +01:00
Richard van der Hoff	ec0b72bc4e	Merge branch 'master' into develop	2020-05-14 18:12:00 +01:00
Richard van der Hoff	66d03639dc	Notes on using git (#7496) ... * general updates to CONTRIBUTING.md * notes on updating your PR * Notes on squash-merging or otherwise * document git branching model	2020-05-14 18:03:10 +01:00
Patrick Cloke	51fb0fc2e5	Update documentation about SSO mapping providers (#7458)	2020-05-12 10:51:07 -04:00
Amber Brown	7cb8b4bc67	Allow configuration of Synapse's cache without using synctl or environment variables (#6391)	2020-05-11 18:45:23 +01:00
Andrew Morgan	5cf758cdd6	Merge branch 'release-v1.13.0' into develop ... * release-v1.13.0: Don't UPGRADE database rows RST indenting Put rollback instructions in upgrade notes Fix changelog typo Oh yeah, RST Absolute URL it is then Fix upgrade notes link Provide summary of upgrade issues in changelog. Fix ) Move next version notes from changelog to upgrade notes Changelog fixes 1.13.0rc1 Documentation on setting up redis (#7446) Rework UI Auth session validation for registration (#7455) Fix errors from malformed log line (#7454) Drop support for redis.dbid (#7450)	2020-05-11 16:46:33 +01:00
Neil Johnson	85155654c5	Documentation on setting up redis (#7446)	2020-05-11 13:21:15 +01:00
Andrew Morgan	67feea8044	Extend spam checker to allow for multiple modules (#7435)	2020-05-08 19:25:48 +01:00
Quentin Gliech	616af44137	Implement OpenID Connect-based login (#7256)	2020-05-08 08:30:40 -04:00
Manuel Stahl	a4a5ec4096	Add room details admin endpoint (#7317)	2020-05-07 15:33:07 -04:00
Brendan Abolivier	5bb26b7c4f	Merge branch 'release-v1.13.0' into develop	2020-05-07 17:31:19 +02:00
Erik Johnston	d7983b63a6	Support any process writing to cache invalidation stream. (#7436)	2020-05-07 13:51:08 +01:00
Brendan Abolivier	d9b8d27494	Add a configuration setting for the dummy event threshold (#7422) ... Add dummy_events_threshold which allows configuring the number of forward extremities a room needs for Synapse to send forward extremities in it.	2020-05-07 10:35:23 +01:00
Richard van der Hoff	207b1737ee	Update reverse_proxy.md ... a couple of cleanups	2020-05-05 11:29:29 +01:00
Erik Johnston	37f6823f5b	Add instance name to RDATA/POSITION commands (#7364) ... This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.	2020-04-29 16:23:08 +01:00
Andrew Morgan	c58ae367d8	Clean up admin api docs (#7361)	2020-04-28 20:06:03 +01:00
Manuel Stahl	04dd7d182d	Return total number of users and profile attributes in admin users endpoint (#6881) ... Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	2020-04-28 18:19:36 +01:00
Brendan Abolivier	036fab5d8a	Document monitoring workers (#7357) ... It doesn't seem to be documented anywhere and means that you suddenly start losing metrics without any obvious reason when you go from monolith to workers (e.g. #7312).	2020-04-27 21:36:47 +02:00
lub	aa2492907f	Add some explanation to application_services.md (#7091) ... Signed-off-by: Simon Körner <git@lubiland.de>	2020-04-27 15:03:09 +01:00
Patrick Cloke	7bfe0902ce	Add documentation to the sample config about the templates for SSO. (#7343)	2020-04-24 15:03:49 -04:00
Patrick Cloke	204664d1ad	Synapse v1.12.4 ... Features: * Always send users their own device updates. (#7160) * Add support for handling GET requests for account_data on a worker. (#7311) Bugfixes: * Fix a bug that prevented cross-signing with users on worker-mode synapses. (#7255) * Do not treat display names as globs in push rules. (#7271) * Fix a bug with cross-signing devices belonging to remote users who did not share a room with any user on the local homeserver. (#7289) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl6hs9EACgkQM/xY9qcR MEjq2w/9EC6m4Bx/2bDlohUy3GKfe9BUpZnMDGSgQR1v+J+OIA4yZ1s16hmY95fr rGjNaZlcUbLUFj9GfaLOOZjvuXQ0KslO2ojP1lcSaKCfx9WK1CH0DtCNVB0lAYQv sfu2C2I91XetkiMHo5GqYBP1d6IK3OTI7YRz72sHOQTVPFcSB1/wCUPuan7P90HE gBdfxtl7Xl1TU0tfvJoeRLVarktVnE4LiXREfm1eGpRECPclewu7sbx5p1ESN5mg /RBIh0FP4rOL4cP4Qt5qiCo68woJmBc4CwiV6pvcOyUMtMGqmwO9K/wJxqY+pxnh JKBNqv90BCSxD55Nywslx9yA52kshuRdcdqHuOVYOHo4Dcwu+ecyySpmzuZ2EJPc hDEkf3dnpfaVoMn3uesbwGNvHEr69QXxR8yE1XZvlAdKzvy4Po/0qRAZuD+NG9qf YjN94bDbkASlFQgPn3wF7R+4RA/HdOf/1Ns6YKt2dFRikHb18WCBPWxHcSwscrgE W8xma8rVTlqfRBuAcZ6y4K3KSHFX6dxLUNxqUngyAhjTInSPuqt2EzlAdqRpnJBS t+hhj+zOh2ixxbz4ZrGdMx0jPF2XARr38x3F/LIjhHLcgOlHdCl4WWkxi9BSXOTd cscvQ8GJBH9c0ANy7PGBntXlohQiW3eVVqblWRbV95TpHVoimME= =Hc/7 -----END PGP SIGNATURE----- Merge tag 'v1.12.4' Synapse v1.12.4 Features: * Always send users their own device updates. (#7160) * Add support for handling GET requests for account_data on a worker. (#7311) Bugfixes: * Fix a bug that prevented cross-signing with users on worker-mode synapses. (#7255) * Do not treat display names as globs in push rules. (#7271) * Fix a bug with cross-signing devices belonging to remote users who did not share a room with any user on the local homeserver. (#7289)	2020-04-23 12:03:33 -04:00
Brendan Abolivier	2e3b9a0fcb	Revert "Revert "Merge pull request #7315 from matrix-org/babolivier/request_token"" ... This reverts commit 1adf6a5587.	2020-04-23 11:23:53 +02:00
Brendan Abolivier	fb825759e3	Merge branch 'master' into develop	2020-04-23 11:23:33 +02:00
Brendan Abolivier	1adf6a5587	Revert "Merge pull request #7315 from matrix-org/babolivier/request_token" ... This reverts commit 6f4319368b, reversing changes made to 0d775fcc2d.	2020-04-23 11:23:10 +02:00
Brendan Abolivier	6f4319368b	Merge pull request #7315 from matrix-org/babolivier/request_token ... Config option to inhibit 3PID errors on /requestToken	2020-04-23 10:38:57 +02:00
Richard van der Hoff	71a1abb8a1	Stop the master relaying USER_SYNC for other workers (#7318) ... Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication. In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits. Fixes (I hope) #7257.	2020-04-22 22:39:04 +01:00
Brendan Abolivier	69ad7cc13b	Config option to inhibit 3PID errors on /requestToken ... Adds a request_token_inhibit_errors configuration flag (disabled by default) which, if enabled, change the behaviour of all /requestToken endpoints so that they return a 200 and a fake sid if the 3PID was/was not found associated with an account (depending on the endpoint), instead of an error. Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2020-04-22 23:38:42 +02:00
Dirk Klimpel	6b6685db9f	Extend room admin api with additional attributes (#7225)	2020-04-22 13:38:41 +01:00
Richard van der Hoff	2aa5bf13c8	Merge branch 'release-v1.12.4' into develop	2020-04-22 13:09:23 +01:00
Richard van der Hoff	974c0d726a	Support GET account_data requests on a worker (#7311)	2020-04-21 10:46:30 +01:00
Lars Franke	13917232d5	Fix indention in generated config file (#7300) ... Also adjust sample_config.yaml Signed-off-by: Lars Franke <frcl@mailbox.org>	2020-04-20 16:51:27 +01:00
nataraj-hates-MS-for-stealing-github	0d775fcc2d	Improve example TURN configuration in documentation (#7284)	2020-04-17 08:04:23 -04:00
Tristan Lins	c07fca9e2f	Clarify the comments for media_storage_providers options (#7272)	2020-04-17 07:09:33 -04:00
Andrew Morgan	a48138784e	Allow specifying the value of Accept-Language header for URL previews (#7265)	2020-04-15 13:35:29 +01:00
Ryan Hovland	4a0dadafbe	Add setting to nginx configuration to allow larger file uploads (#7251)	2020-04-13 17:23:36 +01:00
Andrew Morgan	a026bdaab7	Add matrix-synapse-shared-secret-auth as an example password provider (#7248)	2020-04-09 12:49:05 +01:00
Richard van der Hoff	cae4121484	Make systemd-with-workers doc official (#7234) ... Simplify and update this documentation, and make it part of the core dist.	2020-04-08 11:59:26 +01:00
Andrew Morgan	29b7e22b93	Add documentation to password_providers config option (#7238)	2020-04-08 00:46:50 +01:00
Martin Milata	b0db928c63	Extend web_client_location to handle absolute URLs (#7006) ... Log warning when filesystem path is used. Signed-off-by: Martin Milata <martin@martinmilata.cz>	2020-04-03 11:57:34 -04:00
Richard van der Hoff	0122ef1037	Revert "Merge pull request #7153 from matrix-org/babolivier/sso_whitelist_login_fallback" ... This was incorrectly merged to master. This reverts commit 319c41f573, reversing changes made to 229eb81498.	2020-04-03 11:17:39 +01:00
siroccal	250f87d0de	Update postgres.md (#7119)	2020-04-01 12:44:51 +01:00
Jostein Kjønigsen	2e826cd80c	Improve TURN documentation. (#7167)	2020-03-31 15:50:48 +01:00
Andrew Morgan	d9f29f8dae	Fix a small typo in the metrics_flags config option. (#7171)	2020-03-30 17:38:21 +01:00
Erik Johnston	4f21c33be3	Remove usage of "conn_id" for presence. (#7128) ... * Remove `conn_id` usage for UserSyncCommand. Each tcp replication connection is assigned a "conn_id", which is used to give an ID to a remotely connected worker. In a redis world, there will no longer be a one to one mapping between connection and instance, so instead we need to replace such usages with an ID generated by the remote instances and included in the replicaiton commands. This really only effects UserSyncCommand. * Add CLEAR_USER_SYNCS command that is sent on shutdown. This should help with the case where a synchrotron gets restarted gracefully, rather than rely on 5 minute timeout.	2020-03-30 16:37:24 +01:00
Patrick Cloke	c5f89fba55	Add developer documentation for running a local CAS server (#7147)	2020-03-30 07:28:42 -04:00
Richard van der Hoff	b7da598a61	Always whitelist the login fallback for SSO (#7153) ... That fallback sets the redirect URL to itself (so it can process the login token then return gracefully to the client). This would make it pointless to ask the user for confirmation, since the URL the confirmation page would be showing wouldn't be the client's.	2020-03-27 20:24:52 +00:00
Dirk Klimpel	fb69690761	Admin API to join users to a room. (#7051)	2020-03-27 19:16:43 +00:00
Dirk Klimpel	8327eb9280	Add options to prevent users from changing their profile. (#7096)	2020-03-27 19:15:23 +00:00
Brendan Abolivier	319c41f573	Merge pull request #7153 from matrix-org/babolivier/sso_whitelist_login_fallback ... Always whitelist the login fallback for SSO	2020-03-27 15:34:41 +01:00
Brendan Abolivier	63aea691a7	Update the wording of the config comment	2020-03-27 15:09:12 +01:00
Brendan Abolivier	7083147961	Regenerate sample config	2020-03-26 19:01:54 +01:00
Dirk Klimpel	e8e2ddb60a	Allow server admins to define and enforce a password policy (MSC2000). (#7118)	2020-03-26 16:51:13 +00:00
Aaron Raimist	6ca5e56fd1	Remove unused captcha_bypass_secret option (#7137) ... Signed-off-by: Aaron Raimist <aaron@raim.ist>	2020-03-25 17:49:34 +00:00
Erik Johnston	4cff617df1	Move catchup of replication streams to worker. (#7024) ... This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.	2020-03-25 14:54:01 +00:00
Richard van der Hoff	39230d2171	Clean up some LoggingContext stuff (#7120) ... * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.	2020-03-24 14:45:33 +00:00
Richard van der Hoff	5126cb1253	Merge branch 'master' into develop	2020-03-23 13:54:29 +00:00
Richard van der Hoff	229eb81498	Synapse 1.12.0 (2020-03-23) ... =========================== No significant changes since 1.12.0rc1. Debian packages and Docker images are rebuilt using the latest versions of dependency libraries, including Twisted 20.3.0. **Please see security advisory below**. Security advisory ----------------- Synapse may be vulnerable to request-smuggling attacks when it is used with a reverse-proxy. The vulnerabilties are fixed in Twisted 20.3.0, and are described in [CVE-2020-10108](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10108) and [CVE-2020-10109](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10109). For a good introduction to this class of request-smuggling attacks, see https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn. We are not aware of these vulnerabilities being exploited in the wild, and do not believe that they are exploitable with current versions of any reverse proxies. Nevertheless, we recommend that all Synapse administrators ensure that they have the latest versions of the Twisted library to ensure that their installation remains secure. * Administrators using the [`matrix.org` Docker image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu packages from `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages) should ensure that they have version 1.12.0 installed: these images include Twisted 20.3.0. * Administrators who have [installed Synapse from source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source) should upgrade Twisted within their virtualenv by running: ```sh <path_to_virtualenv>/bin/pip install 'Twisted>=20.3.0' ``` * Administrators who have installed Synapse from distribution packages should consult the information from their distributions. The `matrix.org` Synapse instance was not vulnerable to these vulnerabilities. Advance notice of change to the default `git` branch for Synapse ---------------------------------------------------------------- Currently, the default `git` branch for Synapse is `master`, which tracks the latest release. After the release of Synapse 1.13.0, we intend to change this default to `develop`, which is the development tip. This is more consistent with common practice and modern `git` usage. Although we try to keep `develop` in a stable state, there may be occasions where regressions creep in. Developers and distributors who have scripts which run builds using the default branch of `Synapse` should therefore consider pinning their scripts to `master`. Synapse 1.12.0rc1 (2020-03-19) ============================== Features -------- - Changes related to room alias management ([MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432)): - Publishing/removing a room from the room directory now requires the user to have a power level capable of modifying the canonical alias, instead of the room aliases. ([\#6965](https://github.com/matrix-org/synapse/issues/6965)) - Validate the `alt_aliases` property of canonical alias events. ([\#6971](https://github.com/matrix-org/synapse/issues/6971)) - Users with a power level sufficient to modify the canonical alias of a room can now delete room aliases. ([\#6986](https://github.com/matrix-org/synapse/issues/6986)) - Implement updated authorization rules and redaction rules for aliases events, from [MSC2261](https://github.com/matrix-org/matrix-doc/pull/2261) and [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432). ([\#7037](https://github.com/matrix-org/synapse/issues/7037)) - Stop sending m.room.aliases events during room creation and upgrade. ([\#6941](https://github.com/matrix-org/synapse/issues/6941)) - Synapse no longer uses room alias events to calculate room names for push notifications. ([\#6966](https://github.com/matrix-org/synapse/issues/6966)) - The room list endpoint no longer returns a list of aliases. ([\#6970](https://github.com/matrix-org/synapse/issues/6970)) - Remove special handling of aliases events from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260) added in v1.10.0rc1. ([\#7034](https://github.com/matrix-org/synapse/issues/7034)) - Expose the `synctl`, `hash_password` and `generate_config` commands in the snapcraft package. Contributed by @devec0. ([\#6315](https://github.com/matrix-org/synapse/issues/6315)) - Check that server_name is correctly set before running database updates. ([\#6982](https://github.com/matrix-org/synapse/issues/6982)) - Break down monthly active users by `appservice_id` and emit via Prometheus. ([\#7030](https://github.com/matrix-org/synapse/issues/7030)) - Render a configurable and comprehensible error page if something goes wrong during the SAML2 authentication process. ([\#7058](https://github.com/matrix-org/synapse/issues/7058), [\#7067](https://github.com/matrix-org/synapse/issues/7067)) - Add an optional parameter to control whether other sessions are logged out when a user's password is modified. ([\#7085](https://github.com/matrix-org/synapse/issues/7085)) - Add prometheus metrics for the number of active pushers. ([\#7103](https://github.com/matrix-org/synapse/issues/7103), [\#7106](https://github.com/matrix-org/synapse/issues/7106)) - Improve performance when making HTTPS requests to sygnal, sydent, etc, by sharing the SSL context object between connections. ([\#7094](https://github.com/matrix-org/synapse/issues/7094)) Bugfixes -------- - When a user's profile is updated via the admin API, also generate a displayname/avatar update for that user in each room. ([\#6572](https://github.com/matrix-org/synapse/issues/6572)) - Fix a couple of bugs in email configuration handling. ([\#6962](https://github.com/matrix-org/synapse/issues/6962)) - Fix an issue affecting worker-based deployments where replication would stop working, necessitating a full restart, after joining a large room. ([\#6967](https://github.com/matrix-org/synapse/issues/6967)) - Fix `duplicate key` error which was logged when rejoining a room over federation. ([\#6968](https://github.com/matrix-org/synapse/issues/6968)) - Prevent user from setting 'deactivated' to anything other than a bool on the v2 PUT /users Admin API. ([\#6990](https://github.com/matrix-org/synapse/issues/6990)) - Fix py35-old CI by using native tox package. ([\#7018](https://github.com/matrix-org/synapse/issues/7018)) - Fix a bug causing `org.matrix.dummy_event` to be included in responses from `/sync`. ([\#7035](https://github.com/matrix-org/synapse/issues/7035)) - Fix a bug that renders UTF-8 text files incorrectly when loaded from media. Contributed by @TheStranjer. ([\#7044](https://github.com/matrix-org/synapse/issues/7044)) - Fix a bug that would cause Synapse to respond with an error about event visibility if a client tried to request the state of a room at a given token. ([\#7066](https://github.com/matrix-org/synapse/issues/7066)) - Repair a data-corruption issue which was introduced in Synapse 1.10, and fixed in Synapse 1.11, and which could cause `/sync` to return with 404 errors about missing events and unknown rooms. ([\#7070](https://github.com/matrix-org/synapse/issues/7070)) - Fix a bug causing account validity renewal emails to be sent even if the feature is turned off in some cases. ([\#7074](https://github.com/matrix-org/synapse/issues/7074)) Improved Documentation ---------------------- - Updated CentOS8 install instructions. Contributed by Richard Kellner. ([\#6925](https://github.com/matrix-org/synapse/issues/6925)) - Fix `POSTGRES_INITDB_ARGS` in the `contrib/docker/docker-compose.yml` example docker-compose configuration. ([\#6984](https://github.com/matrix-org/synapse/issues/6984)) - Change date in [INSTALL.md](./INSTALL.md#tls-certificates) for last date of getting TLS certificates to November 2019. ([\#7015](https://github.com/matrix-org/synapse/issues/7015)) - Document that the fallback auth endpoints must be routed to the same worker node as the register endpoints. ([\#7048](https://github.com/matrix-org/synapse/issues/7048)) Deprecations and Removals ------------------------- - Remove the unused query_auth federation endpoint per [MSC2451](https://github.com/matrix-org/matrix-doc/pull/2451). ([\#7026](https://github.com/matrix-org/synapse/issues/7026)) Internal Changes ---------------- - Add type hints to `logging/context.py`. ([\#6309](https://github.com/matrix-org/synapse/issues/6309)) - Add some clarifications to `README.md` in the database schema directory. ([\#6615](https://github.com/matrix-org/synapse/issues/6615)) - Refactoring work in preparation for changing the event redaction algorithm. ([\#6874](https://github.com/matrix-org/synapse/issues/6874), [\#6875](https://github.com/matrix-org/synapse/issues/6875), [\#6983](https://github.com/matrix-org/synapse/issues/6983), [\#7003](https://github.com/matrix-org/synapse/issues/7003)) - Improve performance of v2 state resolution for large rooms. ([\#6952](https://github.com/matrix-org/synapse/issues/6952), [\#7095](https://github.com/matrix-org/synapse/issues/7095)) - Reduce time spent doing GC, by freezing objects on startup. ([\#6953](https://github.com/matrix-org/synapse/issues/6953)) - Minor perfermance fixes to `get_auth_chain_ids`. ([\#6954](https://github.com/matrix-org/synapse/issues/6954)) - Don't record remote cross-signing keys in the `devices` table. ([\#6956](https://github.com/matrix-org/synapse/issues/6956)) - Use flake8-comprehensions to enforce good hygiene of list/set/dict comprehensions. ([\#6957](https://github.com/matrix-org/synapse/issues/6957)) - Merge worker apps together. ([\#6964](https://github.com/matrix-org/synapse/issues/6964), [\#7002](https://github.com/matrix-org/synapse/issues/7002), [\#7055](https://github.com/matrix-org/synapse/issues/7055), [\#7104](https://github.com/matrix-org/synapse/issues/7104)) - Remove redundant `store_room` call from `FederationHandler._process_received_pdu`. ([\#6979](https://github.com/matrix-org/synapse/issues/6979)) - Update warning for incorrect database collation/ctype to include link to documentation. ([\#6985](https://github.com/matrix-org/synapse/issues/6985)) - Add some type annotations to the database storage classes. ([\#6987](https://github.com/matrix-org/synapse/issues/6987)) - Port `synapse.handlers.presence` to async/await. ([\#6991](https://github.com/matrix-org/synapse/issues/6991), [\#7019](https://github.com/matrix-org/synapse/issues/7019)) - Add some type annotations to the federation base & client classes. ([\#6995](https://github.com/matrix-org/synapse/issues/6995)) - Port `synapse.rest.keys` to async/await. ([\#7020](https://github.com/matrix-org/synapse/issues/7020)) - Add a type check to `is_verified` when processing room keys. ([\#7045](https://github.com/matrix-org/synapse/issues/7045)) - Add type annotations and comments to the auth handler. ([\#7063](https://github.com/matrix-org/synapse/issues/7063)) -----BEGIN PGP SIGNATURE----- iQEzBAABCAAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAl54vN0ACgkQOSor00I9 eP/LOgf/U22VypPu2Cl4vofxigeeUL+ph0gEQDCsF7i3EPi9ObeTi4aUggC95dBs MY4JQENmVrk1NhVWQpC08mjehsr4xUzJuwnPPIeGQ6X5U/2BS4YUasxOt9A+iLUz 8IxTzUgx4T+CTfibZvn6xdE/tZK/b2N3BoOikesutD2aQGC5Fm6w2HReoY4Qrdgw AwHlsIV22PYgEn0RL5y6DJ2NUU9SdeSmPjKNe+R1rHDlTpvH7LSyINhOFkYDgRPY xmlH+Ek5+7vLi3AlWg6pA001mMWGADlC4T84URcf1fQv6hXT1iM+A9CqC57jGlfT nHDphCtz0Uk9kmgT4To+hclLDWecYw== =iSEk -----END PGP SIGNATURE----- Merge tag 'v1.12.0' Synapse 1.12.0 (2020-03-23) =========================== No significant changes since 1.12.0rc1. Debian packages and Docker images are rebuilt using the latest versions of dependency libraries, including Twisted 20.3.0. **Please see security advisory below**. Security advisory ----------------- Synapse may be vulnerable to request-smuggling attacks when it is used with a reverse-proxy. The vulnerabilties are fixed in Twisted 20.3.0, and are described in [CVE-2020-10108](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10108) and [CVE-2020-10109](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10109). For a good introduction to this class of request-smuggling attacks, see https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn. We are not aware of these vulnerabilities being exploited in the wild, and do not believe that they are exploitable with current versions of any reverse proxies. Nevertheless, we recommend that all Synapse administrators ensure that they have the latest versions of the Twisted library to ensure that their installation remains secure. * Administrators using the [`matrix.org` Docker image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu packages from `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages) should ensure that they have version 1.12.0 installed: these images include Twisted 20.3.0. * Administrators who have [installed Synapse from source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source) should upgrade Twisted within their virtualenv by running: ```sh <path_to_virtualenv>/bin/pip install 'Twisted>=20.3.0' ``` * Administrators who have installed Synapse from distribution packages should consult the information from their distributions. The `matrix.org` Synapse instance was not vulnerable to these vulnerabilities. Advance notice of change to the default `git` branch for Synapse ---------------------------------------------------------------- Currently, the default `git` branch for Synapse is `master`, which tracks the latest release. After the release of Synapse 1.13.0, we intend to change this default to `develop`, which is the development tip. This is more consistent with common practice and modern `git` usage. Although we try to keep `develop` in a stable state, there may be occasions where regressions creep in. Developers and distributors who have scripts which run builds using the default branch of `Synapse` should therefore consider pinning their scripts to `master`. Synapse 1.12.0rc1 (2020-03-19) ============================== Features -------- - Changes related to room alias management ([MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432)): - Publishing/removing a room from the room directory now requires the user to have a power level capable of modifying the canonical alias, instead of the room aliases. ([\#6965](https://github.com/matrix-org/synapse/issues/6965)) - Validate the `alt_aliases` property of canonical alias events. ([\#6971](https://github.com/matrix-org/synapse/issues/6971)) - Users with a power level sufficient to modify the canonical alias of a room can now delete room aliases. ([\#6986](https://github.com/matrix-org/synapse/issues/6986)) - Implement updated authorization rules and redaction rules for aliases events, from [MSC2261](https://github.com/matrix-org/matrix-doc/pull/2261) and [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432). ([\#7037](https://github.com/matrix-org/synapse/issues/7037)) - Stop sending m.room.aliases events during room creation and upgrade. ([\#6941](https://github.com/matrix-org/synapse/issues/6941)) - Synapse no longer uses room alias events to calculate room names for push notifications. ([\#6966](https://github.com/matrix-org/synapse/issues/6966)) - The room list endpoint no longer returns a list of aliases. ([\#6970](https://github.com/matrix-org/synapse/issues/6970)) - Remove special handling of aliases events from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260) added in v1.10.0rc1. ([\#7034](https://github.com/matrix-org/synapse/issues/7034)) - Expose the `synctl`, `hash_password` and `generate_config` commands in the snapcraft package. Contributed by @devec0. ([\#6315](https://github.com/matrix-org/synapse/issues/6315)) - Check that server_name is correctly set before running database updates. ([\#6982](https://github.com/matrix-org/synapse/issues/6982)) - Break down monthly active users by `appservice_id` and emit via Prometheus. ([\#7030](https://github.com/matrix-org/synapse/issues/7030)) - Render a configurable and comprehensible error page if something goes wrong during the SAML2 authentication process. ([\#7058](https://github.com/matrix-org/synapse/issues/7058), [\#7067](https://github.com/matrix-org/synapse/issues/7067)) - Add an optional parameter to control whether other sessions are logged out when a user's password is modified. ([\#7085](https://github.com/matrix-org/synapse/issues/7085)) - Add prometheus metrics for the number of active pushers. ([\#7103](https://github.com/matrix-org/synapse/issues/7103), [\#7106](https://github.com/matrix-org/synapse/issues/7106)) - Improve performance when making HTTPS requests to sygnal, sydent, etc, by sharing the SSL context object between connections. ([\#7094](https://github.com/matrix-org/synapse/issues/7094)) Bugfixes -------- - When a user's profile is updated via the admin API, also generate a displayname/avatar update for that user in each room. ([\#6572](https://github.com/matrix-org/synapse/issues/6572)) - Fix a couple of bugs in email configuration handling. ([\#6962](https://github.com/matrix-org/synapse/issues/6962)) - Fix an issue affecting worker-based deployments where replication would stop working, necessitating a full restart, after joining a large room. ([\#6967](https://github.com/matrix-org/synapse/issues/6967)) - Fix `duplicate key` error which was logged when rejoining a room over federation. ([\#6968](https://github.com/matrix-org/synapse/issues/6968)) - Prevent user from setting 'deactivated' to anything other than a bool on the v2 PUT /users Admin API. ([\#6990](https://github.com/matrix-org/synapse/issues/6990)) - Fix py35-old CI by using native tox package. ([\#7018](https://github.com/matrix-org/synapse/issues/7018)) - Fix a bug causing `org.matrix.dummy_event` to be included in responses from `/sync`. ([\#7035](https://github.com/matrix-org/synapse/issues/7035)) - Fix a bug that renders UTF-8 text files incorrectly when loaded from media. Contributed by @TheStranjer. ([\#7044](https://github.com/matrix-org/synapse/issues/7044)) - Fix a bug that would cause Synapse to respond with an error about event visibility if a client tried to request the state of a room at a given token. ([\#7066](https://github.com/matrix-org/synapse/issues/7066)) - Repair a data-corruption issue which was introduced in Synapse 1.10, and fixed in Synapse 1.11, and which could cause `/sync` to return with 404 errors about missing events and unknown rooms. ([\#7070](https://github.com/matrix-org/synapse/issues/7070)) - Fix a bug causing account validity renewal emails to be sent even if the feature is turned off in some cases. ([\#7074](https://github.com/matrix-org/synapse/issues/7074)) Improved Documentation ---------------------- - Updated CentOS8 install instructions. Contributed by Richard Kellner. ([\#6925](https://github.com/matrix-org/synapse/issues/6925)) - Fix `POSTGRES_INITDB_ARGS` in the `contrib/docker/docker-compose.yml` example docker-compose configuration. ([\#6984](https://github.com/matrix-org/synapse/issues/6984)) - Change date in [INSTALL.md](./INSTALL.md#tls-certificates) for last date of getting TLS certificates to November 2019. ([\#7015](https://github.com/matrix-org/synapse/issues/7015)) - Document that the fallback auth endpoints must be routed to the same worker node as the register endpoints. ([\#7048](https://github.com/matrix-org/synapse/issues/7048)) Deprecations and Removals ------------------------- - Remove the unused query_auth federation endpoint per [MSC2451](https://github.com/matrix-org/matrix-doc/pull/2451). ([\#7026](https://github.com/matrix-org/synapse/issues/7026)) Internal Changes ---------------- - Add type hints to `logging/context.py`. ([\#6309](https://github.com/matrix-org/synapse/issues/6309)) - Add some clarifications to `README.md` in the database schema directory. ([\#6615](https://github.com/matrix-org/synapse/issues/6615)) - Refactoring work in preparation for changing the event redaction algorithm. ([\#6874](https://github.com/matrix-org/synapse/issues/6874), [\#6875](https://github.com/matrix-org/synapse/issues/6875), [\#6983](https://github.com/matrix-org/synapse/issues/6983), [\#7003](https://github.com/matrix-org/synapse/issues/7003)) - Improve performance of v2 state resolution for large rooms. ([\#6952](https://github.com/matrix-org/synapse/issues/6952), [\#7095](https://github.com/matrix-org/synapse/issues/7095)) - Reduce time spent doing GC, by freezing objects on startup. ([\#6953](https://github.com/matrix-org/synapse/issues/6953)) - Minor perfermance fixes to `get_auth_chain_ids`. ([\#6954](https://github.com/matrix-org/synapse/issues/6954)) - Don't record remote cross-signing keys in the `devices` table. ([\#6956](https://github.com/matrix-org/synapse/issues/6956)) - Use flake8-comprehensions to enforce good hygiene of list/set/dict comprehensions. ([\#6957](https://github.com/matrix-org/synapse/issues/6957)) - Merge worker apps together. ([\#6964](https://github.com/matrix-org/synapse/issues/6964), [\#7002](https://github.com/matrix-org/synapse/issues/7002), [\#7055](https://github.com/matrix-org/synapse/issues/7055), [\#7104](https://github.com/matrix-org/synapse/issues/7104)) - Remove redundant `store_room` call from `FederationHandler._process_received_pdu`. ([\#6979](https://github.com/matrix-org/synapse/issues/6979)) - Update warning for incorrect database collation/ctype to include link to documentation. ([\#6985](https://github.com/matrix-org/synapse/issues/6985)) - Add some type annotations to the database storage classes. ([\#6987](https://github.com/matrix-org/synapse/issues/6987)) - Port `synapse.handlers.presence` to async/await. ([\#6991](https://github.com/matrix-org/synapse/issues/6991), [\#7019](https://github.com/matrix-org/synapse/issues/7019)) - Add some type annotations to the federation base & client classes. ([\#6995](https://github.com/matrix-org/synapse/issues/6995)) - Port `synapse.rest.keys` to async/await. ([\#7020](https://github.com/matrix-org/synapse/issues/7020)) - Add a type check to `is_verified` when processing room keys. ([\#7045](https://github.com/matrix-org/synapse/issues/7045)) - Add type annotations and comments to the auth handler. ([\#7063](https://github.com/matrix-org/synapse/issues/7063))	2020-03-23 13:54:17 +00:00
Richard van der Hoff	c165c1233b	Improve database configuration docs (#6988) ... Attempts to clarify the sample config for databases, and add some stuff about tcp keepalives to `postgres.md`.	2020-03-20 15:24:22 +00:00
Patrick Cloke	88b41986db	Add an option to the set password API to choose whether to logout other devices. (#7085)	2020-03-18 07:50:00 -04:00
Richard van der Hoff	4ce50519cd	Update postgres.md ... fix broken link	2020-03-17 18:08:43 +00:00
Richard van der Hoff	6a35046363	Revert "Add options to disable setting profile info for prevent changes. (#7053)" ... This reverts commit 54dd28621b, reversing changes made to 6640460d05.	2020-03-17 11:25:01 +00:00
Brendan Abolivier	f9e98176bf	Put the file in the templates directory	2020-03-11 20:31:42 +00:00
Brendan Abolivier	900bca9707	Update wording and config	2020-03-11 19:40:30 +00:00
Brendan Abolivier	54dd28621b	Add options to disable setting profile info for prevent changes. (#7053)	2020-03-10 22:23:01 +00:00
Dirk Klimpel	751d51dd12	Update sample_config.yaml	2020-03-10 21:41:25 +01:00
Brendan Abolivier	51c094c4ac	Update sample config	2020-03-10 14:00:29 +00:00
dklimpel	885134529f	updates after review	2020-03-09 22:09:29 +01:00
Patrick Cloke	66315d862f	Update routing of fallback auth in the worker docs. (#7048)	2020-03-09 07:19:24 -04:00
dklimpel	99bbe177b6	add disable_3pid_changes	2020-03-08 21:58:12 +01:00
dklimpel	20545a2199	lint2	2020-03-08 15:28:00 +01:00
dklimpel	fb078f921b	changelog	2020-03-08 15:19:07 +01:00
Brendan Abolivier	43f874055d	Merge branch 'master' into develop	2020-03-03 15:20:49 +00:00
Richard van der Hoff	b68041df3d	Add a whitelist for the SSO confirmation step.	2020-03-02 17:05:09 +00:00
Brendan Abolivier	b2bd54a2e3	Add a confirmation step to the SSO login flow	2020-03-02 16:36:32 +00:00
Richard van der Hoff	4c2ed3f20e	Fix minor issues with email config (#6962) ... * Give `notif_template_html`, `notif_template_text` default values (fixes #6960) * Don't complain if `smtp_host` and `smtp_port` are unset, since they have sensible defaults (fixes #6961) * Set the example for `enable_notifs` to `True`, for consistency and because it's more useful * Raise errors as ConfigError rather than RuntimeError for nicer formatting	2020-02-24 15:18:38 +00:00
Patrick Cloke	509e381afa	Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) ... Ensure good comprehension hygiene using flake8-comprehensions.	2020-02-21 07:15:07 -05:00
Brendan Abolivier	bbe39f808c	Merge pull request #6940 from matrix-org/babolivier/federate.md ... Clean up and update federation docs	2020-02-19 10:58:59 +00:00
Brendan Abolivier	71d65407e7	Incorporate review	2020-02-19 10:03:19 +00:00
Brendan Abolivier	61b457e3ec	Incorporate review	2020-02-18 17:20:03 +00:00
Brendan Abolivier	a0d2f9d089	Phrasing	2020-02-18 16:16:49 +00:00
Brendan Abolivier	d484126bf7	Merge pull request #6907 from matrix-org/babolivier/acme-config ... Add mention and warning about ACME v1 deprecation to the TLS config	2020-02-18 16:11:31 +00:00
Erik Johnston	818def8248	Fix worker docs to point /publicised_groups API correctly. (#6938)	2020-02-18 15:27:45 +00:00
Brendan Abolivier	d009535639	Add mention of SRV records as an advanced topic	2020-02-18 14:07:41 +00:00
Brendan Abolivier	ba7a523854	Argh trailing spaces	2020-02-18 13:57:15 +00:00
Brendan Abolivier	e837be5b5c	Fix links in the reverse proxy doc	2020-02-18 13:53:58 +00:00
Brendan Abolivier	3c67eee6dc	Make federate.md more of a sumary of the steps to follow to set up replication	2020-02-18 13:51:03 +00:00
Brendan Abolivier	8ee0d74516	Split the delegating documentation out of federate.md and trim it down	2020-02-18 12:05:45 +00:00
Richard van der Hoff	97a42bbc3a	Add a warning about indentation to generated config (#6920) ... Fixes #6916.	2020-02-14 16:22:30 +00:00
Patrick Cloke	49f877d32e	Filter the results of user directory searching via the spam checker (#6888) ... Add a method to the spam checker to filter the user directory results.	2020-02-14 07:17:54 -05:00
Brendan Abolivier	5820ed905f	Add mention and warning about ACME v1 deprecation to the Synapse config	2020-02-13 14:20:08 +00:00
Patrick Cloke	361de49c90	Add documentation for the spam checker module (#6906) ... Add documentation for the spam checker.	2020-02-13 07:40:57 -05:00
Brendan Abolivier	862669d6cc	Update docs/ACME.md	2020-02-13 11:29:08 +00:00
Brendan Abolivier	459d089af7	Mention that using Synapse to serve certificates requires restarts	2020-02-12 21:05:30 +00:00
Brendan Abolivier	e45a7c0939	Remove duplicated info about certbot et al	2020-02-12 20:14:59 +00:00
Brendan Abolivier	f092029d2d	Update ACME.md to mention ACME v1 deprecation	2020-02-12 20:14:16 +00:00
Brendan Abolivier	6cd34da8b1	Merge pull request #6891 from matrix-org/babolivier/retention-doc-amend ... Spell out that the last event sent to a room won't be deleted by a purge	2020-02-12 20:12:20 +00:00
Brendan Abolivier	08e050c3fd	Rephrase	2020-02-12 15:39:40 +00:00
Brendan Abolivier	47acbc519f	Merge branch 'master' into develop	2020-02-12 13:24:09 +00:00
Brendan Abolivier	d9239b5257	Synapse 1.10.0 (2020-02-12) ... =========================== **WARNING to client developers**: As of this release Synapse validates `client_secret` parameters in the Client-Server API as per the spec. See [\#6766](https://github.com/matrix-org/synapse/issues/6766) for details. Updates to the Docker image --------------------------- - Update the docker images to Alpine Linux 3.11. ([\#6897](https://github.com/matrix-org/synapse/issues/6897)) Synapse 1.10.0rc5 (2020-02-11) ============================== Bugfixes -------- - Fix the filtering introduced in 1.10.0rc3 to also apply to the state blocks returned by `/sync`. ([\#6884](https://github.com/matrix-org/synapse/issues/6884)) Synapse 1.10.0rc4 (2020-02-11) ============================== This release candidate was built incorrectly and is superceded by 1.10.0rc5. Synapse 1.10.0rc3 (2020-02-10) ============================== Features -------- - Filter out `m.room.aliases` from the CS API to mitigate abuse while a better solution is specced. ([\#6878](https://github.com/matrix-org/synapse/issues/6878)) Internal Changes ---------------- - Fix continuous integration failures with old versions of `pip`, which were introduced by a release of the `zipp` library. ([\#6880](https://github.com/matrix-org/synapse/issues/6880)) Synapse 1.10.0rc2 (2020-02-06) ============================== Bugfixes -------- - Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844)) - Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848)) Internal Changes ---------------- - Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850)) Synapse 1.10.0rc1 (2020-01-31) ============================== Features -------- - Add experimental support for updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260). ([\#6787](https://github.com/matrix-org/synapse/issues/6787), [\#6790](https://github.com/matrix-org/synapse/issues/6790), [\#6794](https://github.com/matrix-org/synapse/issues/6794)) Bugfixes -------- - Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS). ([\#6734](https://github.com/matrix-org/synapse/issues/6734)) - Minor fixes to `PUT /_synapse/admin/v2/users` admin api. ([\#6761](https://github.com/matrix-org/synapse/issues/6761)) - Validate `client_secret` parameter using the regex provided by the Client-Server API, temporarily allowing `:` characters for older clients. The `:` character will be removed in a future release. ([\#6767](https://github.com/matrix-org/synapse/issues/6767)) - Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key). ([\#6771](https://github.com/matrix-org/synapse/issues/6771)) - Fix outbound federation request metrics. ([\#6795](https://github.com/matrix-org/synapse/issues/6795)) - Fix bug where querying a remote user's device keys that weren't cached resulted in only returning a single device. ([\#6796](https://github.com/matrix-org/synapse/issues/6796)) - Fix race in federation sender worker that delayed sending of device updates. ([\#6799](https://github.com/matrix-org/synapse/issues/6799), [\#6800](https://github.com/matrix-org/synapse/issues/6800)) - Fix bug where Synapse didn't invalidate cache of remote users' devices when Synapse left a room. ([\#6801](https://github.com/matrix-org/synapse/issues/6801)) - Fix waking up other workers when remote server is detected to have come back online. ([\#6811](https://github.com/matrix-org/synapse/issues/6811)) Improved Documentation ---------------------- - Clarify documentation related to `user_dir` and `federation_reader` workers. ([\#6775](https://github.com/matrix-org/synapse/issues/6775)) Internal Changes ---------------- - Record room versions in the `rooms` table. ([\#6729](https://github.com/matrix-org/synapse/issues/6729), [\#6788](https://github.com/matrix-org/synapse/issues/6788), [\#6810](https://github.com/matrix-org/synapse/issues/6810)) - Propagate cache invalidates from workers to other workers. ([\#6748](https://github.com/matrix-org/synapse/issues/6748)) - Remove some unnecessary admin handler abstraction methods. ([\#6751](https://github.com/matrix-org/synapse/issues/6751)) - Add some debugging for media storage providers. ([\#6757](https://github.com/matrix-org/synapse/issues/6757)) - Detect unknown remote devices and mark cache as stale. ([\#6776](https://github.com/matrix-org/synapse/issues/6776), [\#6819](https://github.com/matrix-org/synapse/issues/6819)) - Attempt to resync remote users' devices when detected as stale. ([\#6786](https://github.com/matrix-org/synapse/issues/6786)) - Delete current state from the database when server leaves a room. ([\#6792](https://github.com/matrix-org/synapse/issues/6792)) - When a client asks for a remote user's device keys check if the local cache for that user has been marked as potentially stale. ([\#6797](https://github.com/matrix-org/synapse/issues/6797)) - Add background update to clean out left rooms from current state. ([\#6802](https://github.com/matrix-org/synapse/issues/6802), [\#6816](https://github.com/matrix-org/synapse/issues/6816)) - Refactoring work in preparation for changing the event redaction algorithm. ([\#6803](https://github.com/matrix-org/synapse/issues/6803), [\#6805](https://github.com/matrix-org/synapse/issues/6805), [\#6806](https://github.com/matrix-org/synapse/issues/6806), [\#6807](https://github.com/matrix-org/synapse/issues/6807), [\#6820](https://github.com/matrix-org/synapse/issues/6820)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEdVkXOgzrGzds0jtrHgFcFF8ZFs0FAl5D780ACgkQHgFcFF8Z Fs32MxAApvBbTGMhC5E2SLt8Yh4DSfDmg8yCPLMRVTdJuUQyUVgKYFSl9LqzqCGe oLpzwbXXDpN1FYGuO3daFBSN3RaU1XCQPMayekTaF6Wvq9N9rNW1tAi6cQTjUf20 bTAxZljXAIkfXOMbahlFIqAtL+ItTbECKPW4zbndIbC4TiWw/bJPFHeSuoQbHbsI TOgPjM4q2aydHFeTDL2Wqqm99pobPvnEji5sBzHOK+IyIaN0HZI7BZAsm38SDSEA HQTW342RNl7Vm0t2KPdhBq/43r4ENKSUbZjCctFZA+iqbV2mmE96zd2pLOIh9OLP 7KoCDKlplQjInJjN3MQvPzn/cxHnZOmt3AwhQjfhOMcJWcwz00ouWp38oszgZi1c 8y1htf1/Tr26veYxF7umlPHT0h8LNj4Wi7ceQjoIDPu4X70X6vXOk5sJmeJ+KqI5 PJ4yUgw0erWX0I7jybavIvruyA9ZqCwk5FIIyG54IBIZLfughi/fcce49gfvJTa6 Rdjvp0g28qxynkRKq9hVOBSX/TRXTedHe8tHBvSzoDRScvI2aDWQYjJkmGzzLF6L +eWiZ2ICJ2fs7XYIxPhEr/vSpmc/1R+k6vZIPeBTAs0muH8WoK2en6KZPmJ7C25L 2FHdA+1S1Ybqsvyue5S5Sl6yGcvpzGzc1R4mHoB0jZki2v3Cwcs= =j25Z -----END PGP SIGNATURE----- Merge tag 'v1.10.0' Synapse 1.10.0 (2020-02-12) =========================== **WARNING to client developers**: As of this release Synapse validates `client_secret` parameters in the Client-Server API as per the spec. See [\#6766](https://github.com/matrix-org/synapse/issues/6766) for details. Updates to the Docker image --------------------------- - Update the docker images to Alpine Linux 3.11. ([\#6897](https://github.com/matrix-org/synapse/issues/6897)) Synapse 1.10.0rc5 (2020-02-11) ============================== Bugfixes -------- - Fix the filtering introduced in 1.10.0rc3 to also apply to the state blocks returned by `/sync`. ([\#6884](https://github.com/matrix-org/synapse/issues/6884)) Synapse 1.10.0rc4 (2020-02-11) ============================== This release candidate was built incorrectly and is superceded by 1.10.0rc5. Synapse 1.10.0rc3 (2020-02-10) ============================== Features -------- - Filter out `m.room.aliases` from the CS API to mitigate abuse while a better solution is specced. ([\#6878](https://github.com/matrix-org/synapse/issues/6878)) Internal Changes ---------------- - Fix continuous integration failures with old versions of `pip`, which were introduced by a release of the `zipp` library. ([\#6880](https://github.com/matrix-org/synapse/issues/6880)) Synapse 1.10.0rc2 (2020-02-06) ============================== Bugfixes -------- - Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844)) - Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848)) Internal Changes ---------------- - Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850)) Synapse 1.10.0rc1 (2020-01-31) ============================== Features -------- - Add experimental support for updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260). ([\#6787](https://github.com/matrix-org/synapse/issues/6787), [\#6790](https://github.com/matrix-org/synapse/issues/6790), [\#6794](https://github.com/matrix-org/synapse/issues/6794)) Bugfixes -------- - Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS). ([\#6734](https://github.com/matrix-org/synapse/issues/6734)) - Minor fixes to `PUT /_synapse/admin/v2/users` admin api. ([\#6761](https://github.com/matrix-org/synapse/issues/6761)) - Validate `client_secret` parameter using the regex provided by the Client-Server API, temporarily allowing `:` characters for older clients. The `:` character will be removed in a future release. ([\#6767](https://github.com/matrix-org/synapse/issues/6767)) - Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key). ([\#6771](https://github.com/matrix-org/synapse/issues/6771)) - Fix outbound federation request metrics. ([\#6795](https://github.com/matrix-org/synapse/issues/6795)) - Fix bug where querying a remote user's device keys that weren't cached resulted in only returning a single device. ([\#6796](https://github.com/matrix-org/synapse/issues/6796)) - Fix race in federation sender worker that delayed sending of device updates. ([\#6799](https://github.com/matrix-org/synapse/issues/6799), [\#6800](https://github.com/matrix-org/synapse/issues/6800)) - Fix bug where Synapse didn't invalidate cache of remote users' devices when Synapse left a room. ([\#6801](https://github.com/matrix-org/synapse/issues/6801)) - Fix waking up other workers when remote server is detected to have come back online. ([\#6811](https://github.com/matrix-org/synapse/issues/6811)) Improved Documentation ---------------------- - Clarify documentation related to `user_dir` and `federation_reader` workers. ([\#6775](https://github.com/matrix-org/synapse/issues/6775)) Internal Changes ---------------- - Record room versions in the `rooms` table. ([\#6729](https://github.com/matrix-org/synapse/issues/6729), [\#6788](https://github.com/matrix-org/synapse/issues/6788), [\#6810](https://github.com/matrix-org/synapse/issues/6810)) - Propagate cache invalidates from workers to other workers. ([\#6748](https://github.com/matrix-org/synapse/issues/6748)) - Remove some unnecessary admin handler abstraction methods. ([\#6751](https://github.com/matrix-org/synapse/issues/6751)) - Add some debugging for media storage providers. ([\#6757](https://github.com/matrix-org/synapse/issues/6757)) - Detect unknown remote devices and mark cache as stale. ([\#6776](https://github.com/matrix-org/synapse/issues/6776), [\#6819](https://github.com/matrix-org/synapse/issues/6819)) - Attempt to resync remote users' devices when detected as stale. ([\#6786](https://github.com/matrix-org/synapse/issues/6786)) - Delete current state from the database when server leaves a room. ([\#6792](https://github.com/matrix-org/synapse/issues/6792)) - When a client asks for a remote user's device keys check if the local cache for that user has been marked as potentially stale. ([\#6797](https://github.com/matrix-org/synapse/issues/6797)) - Add background update to clean out left rooms from current state. ([\#6802](https://github.com/matrix-org/synapse/issues/6802), [\#6816](https://github.com/matrix-org/synapse/issues/6816)) - Refactoring work in preparation for changing the event redaction algorithm. ([\#6803](https://github.com/matrix-org/synapse/issues/6803), [\#6805](https://github.com/matrix-org/synapse/issues/6805), [\#6806](https://github.com/matrix-org/synapse/issues/6806), [\#6807](https://github.com/matrix-org/synapse/issues/6807), [\#6820](https://github.com/matrix-org/synapse/issues/6820))	2020-02-12 13:23:22 +00:00
Brendan Abolivier	6b21986e4e	Also spell it out in the purge history API doc	2020-02-11 17:56:04 +00:00
Brendan Abolivier	a443d2a25d	Spell out that Synapse never purges the last event sent in a room	2020-02-11 17:37:09 +00:00
Erik Johnston	21db35f77e	Add support for putting fed user query API on workers (#6873)	2020-02-07 15:45:39 +00:00
Erik Johnston	de2d267375	Allow moving group read APIs to workers (#6866)	2020-02-07 11:14:19 +00:00
Dirk Klimpel	56ca93ef59	Admin api to add an email address (#6789)	2020-02-07 10:29:36 +00:00
Robin Vleij	f0561fcffd	Update documentation (#6859) ... Update documentation to reflect the correct format of user_id (fully qualified).	2020-02-05 21:27:38 +00:00
Erik Johnston	02b44db922	Warn if postgres database has non-C locale. (#6734) ... As using non-C locale can cause issues on upgrading OS.	2020-01-28 13:44:21 +00:00
Jason Robinson	1fe5001369	Fix federation_reader listeners doc as per PR review ... Signed-off-by: Jason Robinson <jasonr@matrix.org>	2020-01-27 10:21:25 +02:00
Jason Robinson	aa6ad288f1	Clarifications to the workers documentation ... * Add note that user_dir requires disabling user dir updates from the main synapse process. * Add note that federation_reader should have the federation listener resource. Signed-off-by: Jason Robinson <jasonr@matrix.org>	2020-01-24 11:08:50 +02:00
Andrew Morgan	d31f5f4d89	Update admin room docs with correct endpoints (#6770)	2020-01-23 11:37:26 +00:00
Andrew Morgan	90a28fb475	Admin API to list, filter and sort rooms (#6720)	2020-01-22 13:36:43 +00:00
Erik Johnston	5d7a6ad223	Allow streaming cache invalidate all to workers. (#6749)	2020-01-22 10:37:00 +00:00
Erik Johnston	a8a50f5b57	Wake up transaction queue when remote server comes back online (#6706) ... This will be used to retry outbound transactions to a remote server if we think it might have come back up.	2020-01-17 10:27:19 +00:00
Richard van der Hoff	5ce0b17e38	Clarify the account_validity and email sections of the sample configuration. (#6685) ... Generally try to make this more comprehensible, and make it match the conventions. I've removed the documentation for all the settings which allow you to change the names of the template files, because I can't really see why they are useful.	2020-01-17 10:04:15 +00:00
Andrew Morgan	1177d3f3a3	Quarantine media by ID or user ID (#6681)	2020-01-13 18:10:43 +00:00
Richard van der Hoff	47f4f493f0	Document more supported endpoints for workers (#6698)	2020-01-13 15:32:02 +00:00
Manuel Stahl	d2906fe666	Allow admin users to create or modify users without a shared secret (#6495) ... Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	2020-01-09 13:31:00 +00:00
Brendan Abolivier	3889fcd9d7	Fix typo in message retention policies doc	2020-01-08 13:27:29 +00:00
Brendan Abolivier	cff1cb8685	Merge pull request #6624 from matrix-org/babolivier/retention_doc ... Add complete documentation of the message retention policies support	2020-01-08 11:24:47 +00:00
Brendan Abolivier	2b6b7f482a	Merge pull request #6621 from matrix-org/babolivier/purge_job_config_typo ... Fix a typo in the purge jobs configuration example	2020-01-07 16:17:40 +01:00
Brendan Abolivier	3675fb9bc6	Fix reference	2020-01-07 15:15:16 +00:00
Brendan Abolivier	7ba98a2874	Incorporate review	2020-01-07 15:14:33 +00:00
Brendan Abolivier	4be582d7c8	Merge branch 'develop' into babolivier/retention_doc	2020-01-07 15:07:19 +00:00
Brendan Abolivier	01fbd95736	Apply suggestions from code review ... Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2020-01-07 15:59:38 +01:00
Brendan Abolivier	391fb47791	Reword	2020-01-07 14:54:32 +00:00
Brendan Abolivier	3a86477162	Change the example from 5min to 12h ... Have a purge job running every 5min is probably not something we want to advise admins to do as a sort-of default.	2020-01-07 14:53:07 +00:00
Richard van der Hoff	08815566bc	Automate generation of the sample and debian log configs (#6627)	2020-01-03 17:14:00 +00:00
Richard van der Hoff	98247c4a0e	Remove unused, undocumented "content repo" resource (#6628) ... This looks like it got half-killed back in #888. Fixes #6567.	2020-01-03 17:10:52 +00:00
Brendan Abolivier	b7dec300b7	Fix vacuum instructions for sqlite	2020-01-03 13:51:59 +01:00
Brendan Abolivier	9279a2c4e4	Add a complete documentation of the message retention policies support	2020-01-03 13:45:03 +01:00
Brendan Abolivier	dd2954f78d	Update sample config	2020-01-03 12:58:12 +01:00
ewaf1	0495097a7f	Added the section 'Configuration' in /docs/turn-howto.md (#6614) ... put the 2nd part of the "source installation"-section into a new section, because it also applies to Debian packages	2020-01-02 10:41:30 +00:00
Aaron Raimist	32779b59fa	Reword sections of federate.md that explained delegation at time of Synapse 1.0 transition (#6601) ... * Remove sections of federate.md explaining delegation at time of Synapse 1.0 transition Signed-off-by: Aaron Raimist <aaron@raim.ist> * Add changelog Signed-off-by: Aaron Raimist <aaron@raim.ist>	2020-01-02 10:28:20 +00:00
Richard van der Hoff	dc96943d51	Merge branch 'master' into develop	2019-12-31 11:01:06 +00:00
Richard van der Hoff	f03c877b32	sample log config ... TODO: automate generation of this	2019-12-24 17:39:58 +00:00
Richard van der Hoff	b95b762560	Add an export_signing_key script (#6546) ... I want to do some key rotation, and it is silly that we don't have a way to do this.	2019-12-19 11:11:14 +00:00
Will Hunt	bfb95654c9	Add option to allow profile queries without sharing a room (#6523)	2019-12-16 16:11:55 +00:00
Andrew Morgan	0b90fc6ed2	Document Shutdown Room admin API (#6541)	2019-12-13 15:28:48 +00:00
Erik Johnston	f46e05d053	Merge branch 'master' into develop	2019-12-13 10:55:47 +00:00
Erik Johnston	bee1982d17	Synapse 1.7.0 (2019-12-13) ... ========================== This release changes the default settings so that only local authenticated users can query the server's room directory. See the [upgrade notes](UPGRADE.rst#upgrading-to-v170) for details. Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11. Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available [here](docs/postgres.md). A future release of synapse will, by default, disable federation for servers using SQLite. No significant changes since 1.7.0rc2. Synapse 1.7.0rc2 (2019-12-11) ============================= Bugfixes -------- - Fix incorrect error message for invalid requests when setting user's avatar URL. ([\#6497](https://github.com/matrix-org/synapse/issues/6497)) - Fix support for SQLite 3.7. ([\#6499](https://github.com/matrix-org/synapse/issues/6499)) - Fix regression where sending email push would not work when using a pusher worker. ([\#6507](https://github.com/matrix-org/synapse/issues/6507), [\#6509](https://github.com/matrix-org/synapse/issues/6509)) Synapse 1.7.0rc1 (2019-12-09) ============================= Features -------- - Implement per-room message retention policies. ([\#5815](https://github.com/matrix-org/synapse/issues/5815), [\#6436](https://github.com/matrix-org/synapse/issues/6436)) - Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](https://github.com/matrix-org/synapse/issues/5858)) - Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](https://github.com/matrix-org/synapse/issues/6119)) - Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](https://github.com/matrix-org/synapse/issues/6176)) - Configure privacy-preserving settings by default for the room directory. ([\#6355](https://github.com/matrix-org/synapse/issues/6355)) - Add ephemeral messages support by partially implementing [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). ([\#6409](https://github.com/matrix-org/synapse/issues/6409)) - Add support for [MSC 2367](https://github.com/matrix-org/matrix-doc/pull/2367), which allows specifying a reason on all membership events. ([\#6434](https://github.com/matrix-org/synapse/issues/6434)) Bugfixes -------- - Transfer non-standard power levels on room upgrade. ([\#6237](https://github.com/matrix-org/synapse/issues/6237)) - Fix error from the Pillow library when uploading RGBA images. ([\#6241](https://github.com/matrix-org/synapse/issues/6241)) - Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](https://github.com/matrix-org/synapse/issues/6329)) - Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](https://github.com/matrix-org/synapse/issues/6332)) - Prevent account data syncs getting lost across TCP replication. ([\#6333](https://github.com/matrix-org/synapse/issues/6333)) - Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](https://github.com/matrix-org/synapse/issues/6406)) - Fix an intermittent exception when handling read-receipts. ([\#6408](https://github.com/matrix-org/synapse/issues/6408)) - Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](https://github.com/matrix-org/synapse/issues/6420)) - Fix startup error when http proxy is defined. ([\#6421](https://github.com/matrix-org/synapse/issues/6421)) - Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](https://github.com/matrix-org/synapse/issues/6449)) - Fix uploading multiple cross signing signatures for the same user. ([\#6451](https://github.com/matrix-org/synapse/issues/6451)) - Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](https://github.com/matrix-org/synapse/issues/6462)) - Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](https://github.com/matrix-org/synapse/issues/6470)) - Improve sanity-checking when receiving events over federation. ([\#6472](https://github.com/matrix-org/synapse/issues/6472)) - Fix inaccurate per-block Prometheus metrics. ([\#6491](https://github.com/matrix-org/synapse/issues/6491)) - Fix small performance regression for sending invites. ([\#6493](https://github.com/matrix-org/synapse/issues/6493)) - Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](https://github.com/matrix-org/synapse/issues/6494)) Improved Documentation ---------------------- - Update documentation and variables in user contributed systemd reference file. ([\#6369](https://github.com/matrix-org/synapse/issues/6369), [\#6490](https://github.com/matrix-org/synapse/issues/6490)) - Fix link in the user directory documentation. ([\#6388](https://github.com/matrix-org/synapse/issues/6388)) - Add build instructions to the docker readme. ([\#6390](https://github.com/matrix-org/synapse/issues/6390)) - Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](https://github.com/matrix-org/synapse/issues/6443)) - Write some docs for the quarantine_media api. ([\#6458](https://github.com/matrix-org/synapse/issues/6458)) - Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](https://github.com/matrix-org/synapse/issues/6461)) Deprecations and Removals ------------------------- - Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](https://github.com/matrix-org/synapse/issues/6488)) Internal Changes ---------------- - Add benchmarks for structured logging and improve output performance. ([\#6266](https://github.com/matrix-org/synapse/issues/6266)) - Improve the performance of outputting structured logging. ([\#6322](https://github.com/matrix-org/synapse/issues/6322)) - Refactor some code in the event authentication path for clarity. ([\#6343](https://github.com/matrix-org/synapse/issues/6343), [\#6468](https://github.com/matrix-org/synapse/issues/6468), [\#6480](https://github.com/matrix-org/synapse/issues/6480)) - Clean up some unnecessary quotation marks around the codebase. ([\#6362](https://github.com/matrix-org/synapse/issues/6362)) - Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](https://github.com/matrix-org/synapse/issues/6379)) - Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](https://github.com/matrix-org/synapse/issues/6392)) - Clarifications for the email configuration settings. ([\#6423](https://github.com/matrix-org/synapse/issues/6423)) - Add more tests to the blacklist when running in worker mode. ([\#6429](https://github.com/matrix-org/synapse/issues/6429)) - Refactor data store layer to support multiple databases in the future. ([\#6454](https://github.com/matrix-org/synapse/issues/6454), [\#6464](https://github.com/matrix-org/synapse/issues/6464), [\#6469](https://github.com/matrix-org/synapse/issues/6469), [\#6487](https://github.com/matrix-org/synapse/issues/6487)) - Port synapse.rest.client.v1 to async/await. ([\#6482](https://github.com/matrix-org/synapse/issues/6482)) - Port synapse.rest.client.v2_alpha to async/await. ([\#6483](https://github.com/matrix-org/synapse/issues/6483)) - Port SyncHandler to async/await. ([\#6484](https://github.com/matrix-org/synapse/issues/6484)) -----BEGIN PGP SIGNATURE----- iQJEBAABCgAuFiEEumuwyPtYLL2OMhYdOtoG7cdT0R4FAl3zbaUQHGVyaWtAbWF0 cml4Lm9yZwAKCRA62gbtx1PRHqxED/9Ldjx5pxRm99Rkfhpv6qaw79GaQsAERW+i rPzdC0u59PRDLyfB2MRGU+q4oTTmBN3APPuW79mhY4CRisKfUvw6tuaJM5avvRsF vlY6m4ObewPXvK9dumlKD4ORqks45Sv0VGLHoZH4MLvWu/3Z+ycUn0qf+KNy0JrV TNNnNfsEeo3FXrkLU/12TiEBqNTVCU2vi/nZaHoGlHtZGVoykyFk4z2i9hLKENH4 ScwaN4zeFRTIWYMiEb+kI/6QQLTASvar5FstwXPpM2F+aZhrMc5M7YsEe16jx+bQ jOM1L67hMetcWWmE9QrgP030clIDg30xhNbVxFSD4WoGgEPFkf/zE91v0CF5ZMRJ u83hlhe8tHoO8DOfVl2DNw3x6Evx4Vb5P8bvszNwKAyVDanvkSe0GvYm84B0Tat6 2UCyPR5mbU3fyiwpouaNYun8MppyMRQ/fCnNElvZwwoj9EQQcJ8QOnMK0pFv1AXV n4wwXqRb+WvvbhnKPxTsiTK0DPBbIiGqo1TIONSbhPF+wySR3YV80HCWofd+qlRT V+GbZihk4SCcmamgqYLeQYneOrIu6UWGP348t3scCz/RCUp7hmxaJhJ3GAC3hOmU alN6tDJn7Lo0CwY2frX8xPS6S31sq9IZyDhHkaL0chsEa1rgWJMOK3FRp/b+c8yU b3CEJObIjw== =RQVG -----END PGP SIGNATURE----- Merge tag 'v1.7.0' Synapse 1.7.0 (2019-12-13) ========================== This release changes the default settings so that only local authenticated users can query the server's room directory. See the [upgrade notes](UPGRADE.rst#upgrading-to-v170) for details. Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11. Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available [here](docs/postgres.md). A future release of synapse will, by default, disable federation for servers using SQLite. No significant changes since 1.7.0rc2. Synapse 1.7.0rc2 (2019-12-11) ============================= Bugfixes -------- - Fix incorrect error message for invalid requests when setting user's avatar URL. ([\#6497](https://github.com/matrix-org/synapse/issues/6497)) - Fix support for SQLite 3.7. ([\#6499](https://github.com/matrix-org/synapse/issues/6499)) - Fix regression where sending email push would not work when using a pusher worker. ([\#6507](https://github.com/matrix-org/synapse/issues/6507), [\#6509](https://github.com/matrix-org/synapse/issues/6509)) Synapse 1.7.0rc1 (2019-12-09) ============================= Features -------- - Implement per-room message retention policies. ([\#5815](https://github.com/matrix-org/synapse/issues/5815), [\#6436](https://github.com/matrix-org/synapse/issues/6436)) - Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](https://github.com/matrix-org/synapse/issues/5858)) - Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](https://github.com/matrix-org/synapse/issues/6119)) - Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](https://github.com/matrix-org/synapse/issues/6176)) - Configure privacy-preserving settings by default for the room directory. ([\#6355](https://github.com/matrix-org/synapse/issues/6355)) - Add ephemeral messages support by partially implementing [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). ([\#6409](https://github.com/matrix-org/synapse/issues/6409)) - Add support for [MSC 2367](https://github.com/matrix-org/matrix-doc/pull/2367), which allows specifying a reason on all membership events. ([\#6434](https://github.com/matrix-org/synapse/issues/6434)) Bugfixes -------- - Transfer non-standard power levels on room upgrade. ([\#6237](https://github.com/matrix-org/synapse/issues/6237)) - Fix error from the Pillow library when uploading RGBA images. ([\#6241](https://github.com/matrix-org/synapse/issues/6241)) - Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](https://github.com/matrix-org/synapse/issues/6329)) - Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](https://github.com/matrix-org/synapse/issues/6332)) - Prevent account data syncs getting lost across TCP replication. ([\#6333](https://github.com/matrix-org/synapse/issues/6333)) - Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](https://github.com/matrix-org/synapse/issues/6406)) - Fix an intermittent exception when handling read-receipts. ([\#6408](https://github.com/matrix-org/synapse/issues/6408)) - Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](https://github.com/matrix-org/synapse/issues/6420)) - Fix startup error when http proxy is defined. ([\#6421](https://github.com/matrix-org/synapse/issues/6421)) - Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](https://github.com/matrix-org/synapse/issues/6449)) - Fix uploading multiple cross signing signatures for the same user. ([\#6451](https://github.com/matrix-org/synapse/issues/6451)) - Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](https://github.com/matrix-org/synapse/issues/6462)) - Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](https://github.com/matrix-org/synapse/issues/6470)) - Improve sanity-checking when receiving events over federation. ([\#6472](https://github.com/matrix-org/synapse/issues/6472)) - Fix inaccurate per-block Prometheus metrics. ([\#6491](https://github.com/matrix-org/synapse/issues/6491)) - Fix small performance regression for sending invites. ([\#6493](https://github.com/matrix-org/synapse/issues/6493)) - Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](https://github.com/matrix-org/synapse/issues/6494)) Improved Documentation ---------------------- - Update documentation and variables in user contributed systemd reference file. ([\#6369](https://github.com/matrix-org/synapse/issues/6369), [\#6490](https://github.com/matrix-org/synapse/issues/6490)) - Fix link in the user directory documentation. ([\#6388](https://github.com/matrix-org/synapse/issues/6388)) - Add build instructions to the docker readme. ([\#6390](https://github.com/matrix-org/synapse/issues/6390)) - Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](https://github.com/matrix-org/synapse/issues/6443)) - Write some docs for the quarantine_media api. ([\#6458](https://github.com/matrix-org/synapse/issues/6458)) - Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](https://github.com/matrix-org/synapse/issues/6461)) Deprecations and Removals ------------------------- - Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](https://github.com/matrix-org/synapse/issues/6488)) Internal Changes ---------------- - Add benchmarks for structured logging and improve output performance. ([\#6266](https://github.com/matrix-org/synapse/issues/6266)) - Improve the performance of outputting structured logging. ([\#6322](https://github.com/matrix-org/synapse/issues/6322)) - Refactor some code in the event authentication path for clarity. ([\#6343](https://github.com/matrix-org/synapse/issues/6343), [\#6468](https://github.com/matrix-org/synapse/issues/6468), [\#6480](https://github.com/matrix-org/synapse/issues/6480)) - Clean up some unnecessary quotation marks around the codebase. ([\#6362](https://github.com/matrix-org/synapse/issues/6362)) - Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](https://github.com/matrix-org/synapse/issues/6379)) - Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](https://github.com/matrix-org/synapse/issues/6392)) - Clarifications for the email configuration settings. ([\#6423](https://github.com/matrix-org/synapse/issues/6423)) - Add more tests to the blacklist when running in worker mode. ([\#6429](https://github.com/matrix-org/synapse/issues/6429)) - Refactor data store layer to support multiple databases in the future. ([\#6454](https://github.com/matrix-org/synapse/issues/6454), [\#6464](https://github.com/matrix-org/synapse/issues/6464), [\#6469](https://github.com/matrix-org/synapse/issues/6469), [\#6487](https://github.com/matrix-org/synapse/issues/6487)) - Port synapse.rest.client.v1 to async/await. ([\#6482](https://github.com/matrix-org/synapse/issues/6482)) - Port synapse.rest.client.v2_alpha to async/await. ([\#6483](https://github.com/matrix-org/synapse/issues/6483)) - Port SyncHandler to async/await. ([\#6484](https://github.com/matrix-org/synapse/issues/6484))	2019-12-13 10:55:33 +00:00
Mark Nowiasz	58fdcbdfe7	Update workers.md to make media_repository work (again) (#6519)	2019-12-11 16:23:38 +00:00
Andrew Morgan	4947de5a14	Allow SAML username provider plugins (#6411)	2019-12-10 17:30:16 +00:00
Manuel Stahl	649b6bc088	Replace /admin/v1/users_paginate endpoint with /admin/v2/users (#5925)	2019-12-05 18:12:23 +00:00
Neil Johnson	cb0aeb147e	privacy by default for room dir (#6355) ... Ensure that the the default settings for the room directory are that the it is hidden from public view by default.	2019-12-04 09:46:16 +00:00
Richard van der Hoff	620f98b65b	write some docs for the quarantine_media api (#6458)	2019-12-03 18:20:39 +00:00
Richard van der Hoff	c48ea98007	Clarifications for the email configuration settings. (#6423) ... Cf #6422	2019-11-28 09:29:18 +00:00
Brendan Abolivier	9e937c28ee	Merge branch 'develop' into babolivier/message_retention	2019-11-26 17:53:57 +00:00
Aaron Raimist	24cc31ee96	Fix link to user_dir_populate.sql in the user directory docs (#6388)	2019-11-21 17:38:14 +00:00
Andrew Morgan	473acedcdd	Merge branch 'develop' of github.com:matrix-org/synapse into anoa/homeserver_copy ... * 'develop' of github.com:matrix-org/synapse: Blacklist PurgeRoomTestCase (#6361) Set room version default to 5	2019-11-14 10:26:27 +00:00