Commit graph

12943 commits

Author SHA1 Message Date
Andrew Morgan 77dfe51aba Name metric consistently 2019-01-23 15:04:05 +00:00
Andrew Morgan ef7865e2f2 Track user_dir current event stream position 2019-01-23 15:03:54 +00:00
Matthew Hodgson 5cb15c0443 warn if we ignore device lists 2019-01-15 22:11:46 +00:00
Matthew Hodgson b43172ffbc
Merge pull request #4396 from matrix-org/matthew/bodge_device_update_dos
limit remote device lists to 10000 entries per user
2019-01-15 21:47:00 +00:00
Matthew Hodgson b4796d1814 drop the limit to 1K as e2e will be hosed beyond that point anyway 2019-01-15 21:46:29 +00:00
Matthew Hodgson 482d06774a don't store remote device lists if they have more than 10K devices 2019-01-15 21:38:07 +00:00
Matthew Hodgson 046d731fbd limit remote device lists to 1000 entries per user 2019-01-15 21:07:12 +00:00
Richard van der Hoff 892f6c98ec Synapse 0.34.1.1 (2019-01-11)
=============================
 
 This release fixes CVE-2019-5885 and is recommended for all users of Synapse 0.34.1.
 
 This release is compatible with Python 2.7 and 3.5+. Python 3.7 is fully supported.
 
 Bugfixes
 --------
 
 - Fix spontaneous logout on upgrade
   ([\#4374](https://github.com/matrix-org/synapse/issues/4374))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEIQBQJ4l+yK4dlKkFIwi0edOSShEFAlw3XecACgkQIwi0edOS
 ShGS5gf+OQ7JajiAKnNP7EcWgl2yXislV8ho5366sIgpfcmsMduOQm7VF6qUrHvz
 vge/uip4Xghk0Qs5MPW38RqWw+dC4UziaEyJ99jVitPQNrvJziWvpayIvzfmr8XQ
 SUD+vpp6iZeHhFoVontVSOCCldD1Sn/KRK3anBARH4Lw1ajOVUT566nQBoJFMgGq
 PdSg5MgwsvqS4mPp/YutEvWHiB3OZdfOzmUwrq9TZXUK93zHbR1zyq4EEcTT2HoE
 2Lv4YP7lbINqa47KwTo8x3lsPY1PGP5KRMGF+DcNG1i8FCD5HpVXnx83mFH6bQx9
 Znm6pR6215R+Q8IHQgNA5MJ35UxtdQ==
 =N5tR
 -----END PGP SIGNATURE-----

Merge tag 'v0.34.1.1' into matrix-org-hotfixes

Synapse 0.34.1.1 (2019-01-11)
=============================

This release fixes CVE-2019-5885 and is recommended for all users of Synapse 0.34.1.

This release is compatible with Python 2.7 and 3.5+. Python 3.7 is fully supported.

Bugfixes
--------

- Fix spontaneous logout on upgrade
  ([\#4374](https://github.com/matrix-org/synapse/issues/4374))
2019-01-11 10:21:18 +00:00
Amber Brown 5c792ee5c3 changelog 2019-01-11 01:59:10 +11:00
Amber Brown 6dc06c3775 version 2019-01-11 01:56:37 +11:00
Amber Brown 3933ce9f13
Merge pull request #4374 from matrix-org/rav/macaroon_key_fix_0.34.1
Fix spontaneous logout
2019-01-11 01:48:53 +11:00
Richard van der Hoff de80e979c9 changelog 2019-01-10 14:26:01 +00:00
Richard van der Hoff e0910d0145 Merge branch rav/macaroon_key_fix_0.34 into rav/macaroon_key_fix_0.34.1
Fixes #4371
2019-01-10 14:12:50 +00:00
Richard van der Hoff ba41aeed6a Revert "Fix macaroon_secret_key fallback logic"
This is already fixed in 0.34.1, by 59f93bb

This reverts commit efc522c55e.
2019-01-10 14:09:26 +00:00
Richard van der Hoff 4f24452ead changelog 2019-01-10 14:00:23 +00:00
Richard van der Hoff aa70d24125 Merge branch 'rav/macaroon_key_fix' into rav/macaroon_key_fix_0.34 2019-01-10 12:58:33 +00:00
Richard van der Hoff efc522c55e Fix macaroon_secret_key fallback logic 2019-01-10 12:57:27 +00:00
Richard van der Hoff 566947ff34 Skip macaroon check for access tokens in the db 2019-01-10 12:57:21 +00:00
Richard van der Hoff 353f2407b7 Fix fallback to signing key for macaroon-secret-key 2019-01-10 12:42:56 +00:00
Richard van der Hoff 95fca1c7e9 fix docker build to install optional deps 2019-01-09 16:37:51 +00:00
Richard van der Hoff 2394e832a8 debian: Remove Breaks: matrix-synapse-ldap3 2019-01-09 15:35:11 +00:00
Richard van der Hoff 998f5225c1 0.34.1 2019-01-09 14:53:54 +00:00
Erik Johnston 484867d35d Merge branch 'develop' into release-v0.34.1 2019-01-09 11:38:13 +00:00
Erik Johnston 7fafa2d954 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-01-09 09:13:16 +00:00
Erik Johnston d1d81d0651
Merge pull request #4362 from matrix-org/erikj/better_errors
Use RequestSendFailed when fail to parse content type headers
2019-01-09 09:08:35 +00:00
Erik Johnston d91b99abe0 Newsfile 2019-01-08 14:24:05 +00:00
Erik Johnston d422570e9b Use RequestSendFailed when fail to parse content type headers 2019-01-08 14:22:18 +00:00
Erik Johnston c4530b97b2
Merge pull request #4361 from matrix-org/erikj/better_errors
Don't log stack traces for HTTP error responses
2019-01-08 14:19:00 +00:00
Erik Johnston 83c50bf752 Newsfile 2019-01-08 12:28:35 +00:00
Erik Johnston 1371d5b798 Don't log stack traces for HTTP error responses 2019-01-08 12:28:30 +00:00
Richard van der Hoff 1d63046542 Synapse 0.34.1rc1 (2019-01-08)
==============================
 
 Features
 --------
 
 - Special-case a support user for use in verifying behaviour of a given server. The support user does not appear in user directory or monthly active user counts. ([\#4141](https://github.com/matrix-org/synapse/issues/4141), [\#4344](https://github.com/matrix-org/synapse/issues/4344))
 - Support for serving .well-known files ([\#4262](https://github.com/matrix-org/synapse/issues/4262))
 - Rework SAML2 authentication ([\#4265](https://github.com/matrix-org/synapse/issues/4265), [\#4267](https://github.com/matrix-org/synapse/issues/4267))
 - SAML2 authentication: Initialise user display name from SAML2 data ([\#4272](https://github.com/matrix-org/synapse/issues/4272))
 - Synapse can now have its conditional/extra dependencies installed by pip. This functionality can be used by using `pip install matrix-synapse[feature]`, where feature is a comma separated list with the possible values `email.enable_notifs`, `matrix-synapse-ldap3`, `postgres`, `resources.consent`, `saml2`, `url_preview`, and `test`. If you want to install all optional dependencies, you can use "all" instead. ([\#4298](https://github.com/matrix-org/synapse/issues/4298), [\#4325](https://github.com/matrix-org/synapse/issues/4325), [\#4327](https://github.com/matrix-org/synapse/issues/4327))
 - Add routes for reading account data. ([\#4303](https://github.com/matrix-org/synapse/issues/4303))
 - Add opt-in support for v2 rooms ([\#4307](https://github.com/matrix-org/synapse/issues/4307))
 - Add a script to generate a clean config file ([\#4315](https://github.com/matrix-org/synapse/issues/4315))
 - Return server data in /login response ([\#4319](https://github.com/matrix-org/synapse/issues/4319))
 
 Bugfixes
 --------
 
 - Fix contains_url check to be consistent with other instances in code-base and check that value is an instance of string. ([\#3405](https://github.com/matrix-org/synapse/issues/3405))
 - Fix CAS login when username is not valid in an MXID ([\#4264](https://github.com/matrix-org/synapse/issues/4264))
 - Send CORS headers for /media/config ([\#4279](https://github.com/matrix-org/synapse/issues/4279))
 - Add 'sandbox' to CSP for media reprository ([\#4284](https://github.com/matrix-org/synapse/issues/4284))
 - Make the new landing page prettier. ([\#4294](https://github.com/matrix-org/synapse/issues/4294))
 - Fix deleting E2E room keys when using old SQLite versions. ([\#4295](https://github.com/matrix-org/synapse/issues/4295))
 - The metric synapse_admin_mau:current previously did not update when config.mau_stats_only was set to True ([\#4305](https://github.com/matrix-org/synapse/issues/4305))
 - Fixed per-room account data filters ([\#4309](https://github.com/matrix-org/synapse/issues/4309))
 - Fix indentation in default config ([\#4313](https://github.com/matrix-org/synapse/issues/4313))
 - Fix synapse:latest docker upload ([\#4316](https://github.com/matrix-org/synapse/issues/4316))
 - Fix test_metric.py compatibility with prometheus_client 0.5. Contributed by Maarten de Vries <maarten@de-vri.es>. ([\#4317](https://github.com/matrix-org/synapse/issues/4317))
 - Avoid packaging _trial_temp directory in -py3 debian packages ([\#4326](https://github.com/matrix-org/synapse/issues/4326))
 - Check jinja version for consent resource ([\#4327](https://github.com/matrix-org/synapse/issues/4327))
 - fix NPE in /messages by checking if all events were filtered out ([\#4330](https://github.com/matrix-org/synapse/issues/4330))
 - Fix `python -m synapse.config` on Python 3. ([\#4356](https://github.com/matrix-org/synapse/issues/4356))
 
 Deprecations and Removals
 -------------------------
 
 - Remove the deprecated v1/register API on Python 2. It was never ported to Python 3. ([\#4334](https://github.com/matrix-org/synapse/issues/4334))
 
 Internal Changes
 ----------------
 
 - Getting URL previews of IP addresses no longer fails on Python 3. ([\#4215](https://github.com/matrix-org/synapse/issues/4215))
 - drop undocumented dependency on dateutil ([\#4266](https://github.com/matrix-org/synapse/issues/4266))
 - Update the example systemd config to use a virtualenv ([\#4273](https://github.com/matrix-org/synapse/issues/4273))
 - Update link to kernel DCO guide ([\#4274](https://github.com/matrix-org/synapse/issues/4274))
 - Make isort tox check print diff when it fails ([\#4283](https://github.com/matrix-org/synapse/issues/4283))
 - Log room_id in Unknown room errors ([\#4297](https://github.com/matrix-org/synapse/issues/4297))
 - Documentation improvements for coturn setup. Contributed by Krithin Sitaram. ([\#4333](https://github.com/matrix-org/synapse/issues/4333))
 - Update pull request template to use absolute links ([\#4341](https://github.com/matrix-org/synapse/issues/4341))
 - Update README to not lie about required restart when updating TLS certificates ([\#4343](https://github.com/matrix-org/synapse/issues/4343))
 - Update debian packaging for compatibility with transitional package ([\#4349](https://github.com/matrix-org/synapse/issues/4349))
 - Fix command hint to generate a config file when trying to start without a config file ([\#4353](https://github.com/matrix-org/synapse/issues/4353))
 - Add better logging for unexpected errors while sending transactions ([\#4358](https://github.com/matrix-org/synapse/issues/4358))
 -----BEGIN PGP SIGNATURE-----
 
 iQFHBAABCgAxFiEEQlNDQm4FMsm53u1sih+T1XW16NUFAlw0iaYTHHJpY2hhcmRA
 bWF0cml4Lm9yZwAKCRCKH5PVdbXo1V1HB/0b/AB8QQ/9oIcwMpKI+SDY9RL9Ltzk
 IUv/5RoyNZtHlTWYXRlMQoev95tM4ej2l5EkHUpfCdUMbRif9M6Q4EgAJQDvxKYf
 eSqqv9Uk2FMHgv/zwsQO59b0cPfVUMdsUFZNrIjgnszuGVtjmUe2w3Bka+DYtA2O
 vYRx88bjcqTra6UnII6yVt66Dgk/kxeP6TmboMJGuVWxkvjQdRCvcdEbLAVHW47j
 Dc050mL/5oTpUBNa+yks28QcNbXLZV+4UDoBMja78izXZnaLH8nbJT7cF8mwt0oT
 6q5DisF8MLc2fEAUtjFeLkyDkfjzSlSSH6hfoqp51KKN8wn78/Zl662Z
 =5ID1
 -----END PGP SIGNATURE-----

Merge tag 'v0.34.1rc1' into matrix-org-hotfixes

Synapse 0.34.1rc1 (2019-01-08)
==============================

Features
--------

- Special-case a support user for use in verifying behaviour of a given server. The support user does not appear in user directory or monthly active user counts. ([\#4141](https://github.com/matrix-org/synapse/issues/4141), [\#4344](https://github.com/matrix-org/synapse/issues/4344))
- Support for serving .well-known files ([\#4262](https://github.com/matrix-org/synapse/issues/4262))
- Rework SAML2 authentication ([\#4265](https://github.com/matrix-org/synapse/issues/4265), [\#4267](https://github.com/matrix-org/synapse/issues/4267))
- SAML2 authentication: Initialise user display name from SAML2 data ([\#4272](https://github.com/matrix-org/synapse/issues/4272))
- Synapse can now have its conditional/extra dependencies installed by pip. This functionality can be used by using `pip install matrix-synapse[feature]`, where feature is a comma separated list with the possible values `email.enable_notifs`, `matrix-synapse-ldap3`, `postgres`, `resources.consent`, `saml2`, `url_preview`, and `test`. If you want to install all optional dependencies, you can use "all" instead. ([\#4298](https://github.com/matrix-org/synapse/issues/4298), [\#4325](https://github.com/matrix-org/synapse/issues/4325), [\#4327](https://github.com/matrix-org/synapse/issues/4327))
- Add routes for reading account data. ([\#4303](https://github.com/matrix-org/synapse/issues/4303))
- Add opt-in support for v2 rooms ([\#4307](https://github.com/matrix-org/synapse/issues/4307))
- Add a script to generate a clean config file ([\#4315](https://github.com/matrix-org/synapse/issues/4315))
- Return server data in /login response ([\#4319](https://github.com/matrix-org/synapse/issues/4319))

Bugfixes
--------

- Fix contains_url check to be consistent with other instances in code-base and check that value is an instance of string. ([\#3405](https://github.com/matrix-org/synapse/issues/3405))
- Fix CAS login when username is not valid in an MXID ([\#4264](https://github.com/matrix-org/synapse/issues/4264))
- Send CORS headers for /media/config ([\#4279](https://github.com/matrix-org/synapse/issues/4279))
- Add 'sandbox' to CSP for media reprository ([\#4284](https://github.com/matrix-org/synapse/issues/4284))
- Make the new landing page prettier. ([\#4294](https://github.com/matrix-org/synapse/issues/4294))
- Fix deleting E2E room keys when using old SQLite versions. ([\#4295](https://github.com/matrix-org/synapse/issues/4295))
- The metric synapse_admin_mau:current previously did not update when config.mau_stats_only was set to True ([\#4305](https://github.com/matrix-org/synapse/issues/4305))
- Fixed per-room account data filters ([\#4309](https://github.com/matrix-org/synapse/issues/4309))
- Fix indentation in default config ([\#4313](https://github.com/matrix-org/synapse/issues/4313))
- Fix synapse:latest docker upload ([\#4316](https://github.com/matrix-org/synapse/issues/4316))
- Fix test_metric.py compatibility with prometheus_client 0.5. Contributed by Maarten de Vries <maarten@de-vri.es>. ([\#4317](https://github.com/matrix-org/synapse/issues/4317))
- Avoid packaging _trial_temp directory in -py3 debian packages ([\#4326](https://github.com/matrix-org/synapse/issues/4326))
- Check jinja version for consent resource ([\#4327](https://github.com/matrix-org/synapse/issues/4327))
- fix NPE in /messages by checking if all events were filtered out ([\#4330](https://github.com/matrix-org/synapse/issues/4330))
- Fix `python -m synapse.config` on Python 3. ([\#4356](https://github.com/matrix-org/synapse/issues/4356))

Deprecations and Removals
-------------------------

- Remove the deprecated v1/register API on Python 2. It was never ported to Python 3. ([\#4334](https://github.com/matrix-org/synapse/issues/4334))

Internal Changes
----------------

- Getting URL previews of IP addresses no longer fails on Python 3. ([\#4215](https://github.com/matrix-org/synapse/issues/4215))
- drop undocumented dependency on dateutil ([\#4266](https://github.com/matrix-org/synapse/issues/4266))
- Update the example systemd config to use a virtualenv ([\#4273](https://github.com/matrix-org/synapse/issues/4273))
- Update link to kernel DCO guide ([\#4274](https://github.com/matrix-org/synapse/issues/4274))
- Make isort tox check print diff when it fails ([\#4283](https://github.com/matrix-org/synapse/issues/4283))
- Log room_id in Unknown room errors ([\#4297](https://github.com/matrix-org/synapse/issues/4297))
- Documentation improvements for coturn setup. Contributed by Krithin Sitaram. ([\#4333](https://github.com/matrix-org/synapse/issues/4333))
- Update pull request template to use absolute links ([\#4341](https://github.com/matrix-org/synapse/issues/4341))
- Update README to not lie about required restart when updating TLS certificates ([\#4343](https://github.com/matrix-org/synapse/issues/4343))
- Update debian packaging for compatibility with transitional package ([\#4349](https://github.com/matrix-org/synapse/issues/4349))
- Fix command hint to generate a config file when trying to start without a config file ([\#4353](https://github.com/matrix-org/synapse/issues/4353))
- Add better logging for unexpected errors while sending transactions ([\#4358](https://github.com/matrix-org/synapse/issues/4358))
2019-01-08 11:37:25 +00:00
Richard van der Hoff 32172f2297 clean up changelog 2019-01-08 11:26:11 +00:00
Richard van der Hoff ba85786d71 0.34.1rc1 2019-01-08 11:15:34 +00:00
Richard van der Hoff bc1a4b5576 changelog 2019-01-08 11:14:59 +00:00
Erik Johnston b970cb0e96 Refactor request sending to have better excpetions (#4358)
* Correctly retry and back off if we get a HTTPerror response

* Refactor request sending to have better excpetions

MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).

This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
2019-01-08 11:04:28 +00:00
Amber Brown 1dcb086f33
Fix synapse.config.__main__ on python 3 (#4356) 2019-01-08 10:03:09 +11:00
Jason Robinson 26e5abf20d Fix command hint to generate a config file (#4353)
* Fix command hint to generate a config file

When trying to start Synapse without a config file, it will complain
and give a hint towards what command to run. This hinted command
is missing the "report_stats" parameter, which is required with either
yes or no value. Add this to the command.

Not an ideal situation but makes the given command work without the
user getting another error, even though it might be unclear what
"report_stats" represents.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-07 16:28:40 +00:00
Travis Ralston bc1fa8cd01 Add GET account data routes (#4303)
As per https://github.com/matrix-org/matrix-doc/issues/1339
2019-01-07 10:27:54 +00:00
Richard van der Hoff b7c0218812
Check jinja version for consent resource (#4327)
* Raise a ConfigError if an invalid resource is specified

* Require Jinja 2.9 for the consent resource

* changelog
2019-01-07 10:14:31 +00:00
Michael Telatynski a27e501b09 fix the check for whether is_url to match all the other ones in codebase (#3405)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2019-01-06 21:43:25 +00:00
Richard van der Hoff 29f20a8a1a
Update debian Conflicts specifications (#4349)
...  to allow installation alongside our matrix-synapse transitional package.
2019-01-04 17:24:13 +00:00
Matthew Hodgson 27128145e6 fix NPE in /messages by checking if all events were filtered out (#4330) 2019-01-02 23:37:39 +00:00
Aaron Raimist 37f8bdc1d5 Update README to not lie about required restart (#4343)
* Update README to not lie about required restart

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-02 16:32:37 +00:00
Neil Johnson f95581332f
Merge pull request #4344 from matrix-org/neilj/fix_synchrotron_fix_4312
Ensure synchrotrons can access is_support_user in the storage layer
2019-01-02 10:42:01 +00:00
Neil Johnson 84b6fae1f5 Ensure synchrotrons can access is_support_user in the storage layer 2019-01-02 10:19:59 +00:00
jribal 265513e499 Update docker-compose.yml (#4282)
Hi, the original docker-compose file did not work by default.
You get federation port working but no client port.

My proposal is to let federation port work as it is by default (8448) and let traefik handle client http/https traffic.
2019-01-02 07:38:18 +00:00
Aaron Raimist 252c0c81fa Update PR template to use absolute links (#4341)
* Update PR template to use absolute links

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-02 07:33:13 +00:00
Richard van der Hoff e9cdfedff3
Avoid packaging _trial_temp directory (#4326)
Make sure we don't put the _trial_temp directory in the package target
directory.

Fixes https://github.com/matrix-org/synapse/issues/4322
2019-01-02 07:30:31 +00:00
Richard van der Hoff 7134832c01
Install the optional dependencies into the debian package (#4325)
since #4298, the optional dependencies are no longer installed with a simple
`pip install .`, which meant that they were not being included in the debian
package.

The easy fix to that is dh_virtualenv --extras, but that needs dh_virtualenv
1.1...
2019-01-02 07:17:39 +00:00
Amber Brown d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console (#4334) 2018-12-29 23:12:30 +11:00