Commit graph

13294 commits

Author SHA1 Message Date
Brendan Abolivier
1895d14e12
Support .well-known delegation when issuing certificates through ACME 2019-02-15 12:05:08 +00:00
Richard van der Hoff
bf4fd14806 Merge branch 'master' into develop 2019-02-14 17:34:09 +00:00
Richard van der Hoff
f830a3be2a Merge branch 'release-v0.99.1' 2019-02-14 17:31:45 +00:00
Richard van der Hoff
649fe1c2be Fix debian build dockerfile
Make sure it refreshes the apt cache before trying to install stuff
2019-02-14 17:29:40 +00:00
Richard van der Hoff
f595d6ac57 0.99.1.1 2019-02-14 17:20:02 +00:00
Richard van der Hoff
f311018823
Fix errors in acme provisioning (#4648)
* Better logging for errors on startup

* Fix "TypeError: '>' not supported" when starting without an existing
  certificate
* Fix a bug where an existing certificate would be reprovisoned every day
2019-02-14 17:10:36 +00:00
Richard van der Hoff
b02465b9db Merge branch 'master' into develop 2019-02-14 14:42:03 +00:00
Richard van der Hoff
00cf679bf2 Synapse 0.99.1 (2019-02-14)
===========================
 
 Features
 --------
 
 - Include m.room.encryption on invites by default ([\#3902](https://github.com/matrix-org/synapse/issues/3902))
 - Federation OpenID listener resource can now be activated even if federation is disabled ([\#4420](https://github.com/matrix-org/synapse/issues/4420))
 - Synapse's ACME support will now correctly reprovision a certificate that approaches its expiry while Synapse is running. ([\#4522](https://github.com/matrix-org/synapse/issues/4522))
 - Add ability to update backup versions ([\#4580](https://github.com/matrix-org/synapse/issues/4580))
 - Allow the "unavailable" presence status for /sync.
   This change makes Synapse compliant with r0.4.0 of the Client-Server specification. ([\#4592](https://github.com/matrix-org/synapse/issues/4592))
 - There is no longer any need to specify `no_tls`: it is inferred from the absence of TLS listeners ([\#4613](https://github.com/matrix-org/synapse/issues/4613), [\#4615](https://github.com/matrix-org/synapse/issues/4615), [\#4617](https://github.com/matrix-org/synapse/issues/4617), [\#4636](https://github.com/matrix-org/synapse/issues/4636))
 - The default configuration no longer requires TLS certificates. ([\#4614](https://github.com/matrix-org/synapse/issues/4614))
 
 Bugfixes
 --------
 
 - Copy over room federation ability on room upgrade. ([\#4530](https://github.com/matrix-org/synapse/issues/4530))
 - Fix noisy "twisted.internet.task.TaskStopped" errors in logs ([\#4546](https://github.com/matrix-org/synapse/issues/4546))
 - Synapse is now tolerant of the `tls_fingerprints` option being None or not specified. ([\#4589](https://github.com/matrix-org/synapse/issues/4589))
 - Fix 'no unique or exclusion constraint' error ([\#4591](https://github.com/matrix-org/synapse/issues/4591))
 - Transfer Server ACLs on room upgrade. ([\#4608](https://github.com/matrix-org/synapse/issues/4608))
 - Fix failure to start when not TLS certificate was given even if TLS was disabled. ([\#4618](https://github.com/matrix-org/synapse/issues/4618))
 - Fix self-signed cert notice from generate-config. ([\#4625](https://github.com/matrix-org/synapse/issues/4625))
 - Fix performance of `user_ips` table deduplication background update ([\#4626](https://github.com/matrix-org/synapse/issues/4626), [\#4627](https://github.com/matrix-org/synapse/issues/4627))
 
 Internal Changes
 ----------------
 
 - Change the user directory state query to use a filtered call to the db instead of a generic one. ([\#4462](https://github.com/matrix-org/synapse/issues/4462))
 - Reject federation transactions if they include more than 50 PDUs or 100 EDUs. ([\#4513](https://github.com/matrix-org/synapse/issues/4513))
 - Reduce duplication of ``synapse.app`` code. ([\#4567](https://github.com/matrix-org/synapse/issues/4567))
 - Fix docker upload job to push -py2 images. ([\#4576](https://github.com/matrix-org/synapse/issues/4576))
 - Add port configuration information to ACME instructions. ([\#4578](https://github.com/matrix-org/synapse/issues/4578))
 - Update MSC1711 FAQ to calrify .well-known usage ([\#4584](https://github.com/matrix-org/synapse/issues/4584))
 - Clean up default listener configuration ([\#4586](https://github.com/matrix-org/synapse/issues/4586))
 - Clarifications for reverse proxy docs ([\#4607](https://github.com/matrix-org/synapse/issues/4607))
 - Move ClientTLSOptionsFactory init out of `refresh_certificates` ([\#4611](https://github.com/matrix-org/synapse/issues/4611))
 - Fail cleanly if listener config lacks a 'port' ([\#4616](https://github.com/matrix-org/synapse/issues/4616))
 - Remove redundant entries from docker config ([\#4619](https://github.com/matrix-org/synapse/issues/4619))
 - README updates ([\#4621](https://github.com/matrix-org/synapse/issues/4621))
 -----BEGIN PGP SIGNATURE-----
 
 iQFHBAABCgAxFiEEQlNDQm4FMsm53u1sih+T1XW16NUFAlxlemgTHHJpY2hhcmRA
 bWF0cml4Lm9yZwAKCRCKH5PVdbXo1eKYCACR9TcOvMver/YyD2qP+dY6Lt24f8zG
 zYYzHGAHin+p204q8Pp6o0XLe4UuLDuhAyNVPZyj1wzwHYdubRvdah1uFwPdxmCY
 tGbJG5p37ykSEfEwcxdXEjYfPqflOwQL5aCeXyCCLWSdVVFKkWCXGgw8F6WPkgrI
 QwWKTfsM3wCnfa8ryKAXHxcmX2G1JncZ0ouUZTVNz5vokBsA19IaLvfJ5Rv3Kk59
 eXsBB/yE+9Dat4A439AGfVDQDKiGYvuhppJmUdYRMqxulzakd8diyZqBDAHZafqt
 QdjxnDx2e0OtSxI3RSevABnDnNyJ4NsUEtrny1Lh/MV72T9K3yEbHuwH
 =UCD1
 -----END PGP SIGNATURE-----

Merge tag 'v0.99.1'

Synapse 0.99.1 (2019-02-14)
===========================

Features
--------

- Include m.room.encryption on invites by default ([\#3902](https://github.com/matrix-org/synapse/issues/3902))
- Federation OpenID listener resource can now be activated even if federation is disabled ([\#4420](https://github.com/matrix-org/synapse/issues/4420))
- Synapse's ACME support will now correctly reprovision a certificate that approaches its expiry while Synapse is running. ([\#4522](https://github.com/matrix-org/synapse/issues/4522))
- Add ability to update backup versions ([\#4580](https://github.com/matrix-org/synapse/issues/4580))
- Allow the "unavailable" presence status for /sync.
  This change makes Synapse compliant with r0.4.0 of the Client-Server specification. ([\#4592](https://github.com/matrix-org/synapse/issues/4592))
- There is no longer any need to specify `no_tls`: it is inferred from the absence of TLS listeners ([\#4613](https://github.com/matrix-org/synapse/issues/4613), [\#4615](https://github.com/matrix-org/synapse/issues/4615), [\#4617](https://github.com/matrix-org/synapse/issues/4617), [\#4636](https://github.com/matrix-org/synapse/issues/4636))
- The default configuration no longer requires TLS certificates. ([\#4614](https://github.com/matrix-org/synapse/issues/4614))

Bugfixes
--------

- Copy over room federation ability on room upgrade. ([\#4530](https://github.com/matrix-org/synapse/issues/4530))
- Fix noisy "twisted.internet.task.TaskStopped" errors in logs ([\#4546](https://github.com/matrix-org/synapse/issues/4546))
- Synapse is now tolerant of the `tls_fingerprints` option being None or not specified. ([\#4589](https://github.com/matrix-org/synapse/issues/4589))
- Fix 'no unique or exclusion constraint' error ([\#4591](https://github.com/matrix-org/synapse/issues/4591))
- Transfer Server ACLs on room upgrade. ([\#4608](https://github.com/matrix-org/synapse/issues/4608))
- Fix failure to start when not TLS certificate was given even if TLS was disabled. ([\#4618](https://github.com/matrix-org/synapse/issues/4618))
- Fix self-signed cert notice from generate-config. ([\#4625](https://github.com/matrix-org/synapse/issues/4625))
- Fix performance of `user_ips` table deduplication background update ([\#4626](https://github.com/matrix-org/synapse/issues/4626), [\#4627](https://github.com/matrix-org/synapse/issues/4627))

Internal Changes
----------------

- Change the user directory state query to use a filtered call to the db instead of a generic one. ([\#4462](https://github.com/matrix-org/synapse/issues/4462))
- Reject federation transactions if they include more than 50 PDUs or 100 EDUs. ([\#4513](https://github.com/matrix-org/synapse/issues/4513))
- Reduce duplication of ``synapse.app`` code. ([\#4567](https://github.com/matrix-org/synapse/issues/4567))
- Fix docker upload job to push -py2 images. ([\#4576](https://github.com/matrix-org/synapse/issues/4576))
- Add port configuration information to ACME instructions. ([\#4578](https://github.com/matrix-org/synapse/issues/4578))
- Update MSC1711 FAQ to calrify .well-known usage ([\#4584](https://github.com/matrix-org/synapse/issues/4584))
- Clean up default listener configuration ([\#4586](https://github.com/matrix-org/synapse/issues/4586))
- Clarifications for reverse proxy docs ([\#4607](https://github.com/matrix-org/synapse/issues/4607))
- Move ClientTLSOptionsFactory init out of `refresh_certificates` ([\#4611](https://github.com/matrix-org/synapse/issues/4611))
- Fail cleanly if listener config lacks a 'port' ([\#4616](https://github.com/matrix-org/synapse/issues/4616))
- Remove redundant entries from docker config ([\#4619](https://github.com/matrix-org/synapse/issues/4619))
- README updates ([\#4621](https://github.com/matrix-org/synapse/issues/4621))
2019-02-14 14:41:40 +00:00
Richard van der Hoff
06cd757ae7 0.99.1 2019-02-14 14:24:24 +00:00
Дамјан Георгиевски
a214ba93e0 implement reload by sending the HUP signal (#4622)
* implement `reload` by sending the HUP signal

According to the 0.99 release info* synapse now uses the HUP signal to reload certificates:

> Synapse will now reload TLS certificates from disk upon SIGHUP. (#4495, #4524)

So the matrix-synapse.service unit file should include a reload directive.

Signed-off-by: Дамјан Георгиевски <gdamjan@gmail.com>
2019-02-14 13:44:22 +00:00
Richard van der Hoff
c6e75c9f2d
Merge pull request #4450 from 14mRh4X0r/fix-dependency-message
Fix error message for optional dependencies
2019-02-13 16:12:49 +00:00
Richard van der Hoff
3bc238629e 0.99.1rc2 2019-02-13 14:46:18 +00:00
Richard van der Hoff
c1dfd6a18a Merge remote-tracking branch 'origin/release-v0.99.1' into develop 2019-02-13 14:27:45 +00:00
Richard van der Hoff
464c301584
Merge pull request #4636 from matrix-org/rav/bind_address_fixes
Fix errors when using default bind_addresses with replication/metrics listeners
2019-02-13 14:16:02 +00:00
Erik Johnston
309f3bb322
Update synapse/app/_base.py
Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com>
2019-02-13 13:24:27 +00:00
Amber Brown
bb4fd8f927
Run black on user directory code (#4635) 2019-02-13 23:05:32 +11:00
Richard van der Hoff
2d0e0a4044 changelog 2019-02-13 12:00:34 +00:00
Richard van der Hoff
767686af48 Use listen_tcp for the replication listener
Fixes the "can't listen on 0.0.0.0" error. Also makes it more consistent with
what we do elsewhere.
2019-02-13 11:59:04 +00:00
Richard van der Hoff
2a5a15aff8 Improve logging around listening services
I wanted to bring listen_tcp into line with listen_ssl in terms of returning a
list of ports, and wanted to check that was a safe thing to do - hence the
logging in `refresh_certificate`.

Also, pull the 'Synapse now listening' message up to homeserver.py, because it
was being duplicated everywhere else.
2019-02-13 11:58:54 +00:00
Richard van der Hoff
e3a0300431 Special-case the default bind_addresses for metrics listener
turns out it doesn't really support ipv6, so let's hack around that by only
listening on ipv4 by default.
2019-02-13 11:48:56 +00:00
Erik Johnston
19818d66af Fixup changelog 2019-02-12 13:25:05 +00:00
Erik Johnston
d2fa7b7e99 Update changelog and version 2019-02-12 13:22:25 +00:00
Erik Johnston
ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade
Transfer Server ACLs on room upgrade
2019-02-12 13:20:06 +00:00
Erik Johnston
b18cd25e42 Fixup changelog entries 2019-02-12 13:05:31 +00:00
Erik Johnston
cf82338930
Merge pull request #4627 from matrix-org/erikj/user_ips_analyze
Analyze user_ips before running deduplication
2019-02-12 13:05:09 +00:00
Erik Johnston
3df8fcca25
Merge pull request #4626 from matrix-org/erikj/fixup_user_ips_dedupe
Reduce user_ips bloat during dedupe background update
2019-02-12 13:02:58 +00:00
Erik Johnston
495ea92350 Fix pep8 2019-02-12 12:40:42 +00:00
Erik Johnston
b2327eb9cb Newsfile 2019-02-12 11:58:36 +00:00
Erik Johnston
483ba85c7a Analyze user_ips before running deduplication
Due to the table locks taken out by the naive upsert, the table
statistics may be out of date. During deduplication it is important that
the correct index is used as otherwise a full table scan may be
incorrectly used, which can end up thrashing the database badly.
2019-02-12 11:55:27 +00:00
Erik Johnston
218cc071c5 Newsfile 2019-02-12 11:39:36 +00:00
Erik Johnston
362d80b770 Reduce user_ips bloat during dedupe background update
The background update to remove duplicate rows naively deleted and
reinserted the duplicates. For large tables with a large number of
duplicates this causes a lot of bloat (with postgres), as the inserted
rows are appended to the table, since deleted rows will not be
overwritten until a VACUUM has happened.

This should hopefully also help ensure that the query in the last batch
uses the correct index, as inserting a large number of new rows without
analyzing will upset the query planner.
2019-02-12 11:39:34 +00:00
Erik Johnston
3c03c37883
Merge pull request #4625 from matrix-org/rav/fix_generate_config_warnings
fix self-signed cert notice from generate-config
2019-02-12 11:24:45 +00:00
Richard van der Hoff
2418b91bb7
README updates (#4621)
Lots of updates to the README/INSTALL.md.

Fixes #4601.
2019-02-12 10:53:28 +00:00
Richard van der Hoff
a4ce91396b
Disable TLS by default (#4614) 2019-02-12 10:52:08 +00:00
Richard van der Hoff
32b781bfe2
Fix error when loading cert if tls is disabled (#4618)
If TLS is disabled, it should not be an error if no cert is given.

Fixes #4554.
2019-02-12 10:51:31 +00:00
Richard van der Hoff
dfc846a316 fix self-signed cert notice from generate-config
fixes #4620
2019-02-12 10:37:59 +00:00
Erik Johnston
46b8a79b3a
Merge pull request #4619 from matrix-org/rav/remove_docker_no_tls_hacks
Remove redundant entries from docker config
2019-02-12 10:00:38 +00:00
Erik Johnston
8a2e316413
Merge pull request #4613 from matrix-org/rav/deprecate_no_tls
Infer no_tls from presence of TLS listeners
2019-02-12 09:59:53 +00:00
Richard van der Hoff
91f8cd3307 Remove redundant entries from docker config
* no_tls is now redundant (#4613)
* we don't need a dummy cert any more (#4618)
2019-02-11 22:16:44 +00:00
Richard van der Hoff
0ca2908653 fix tests 2019-02-11 22:01:27 +00:00
Richard van der Hoff
4fddf8fc77 Infer no_tls from presence of TLS listeners
Rather than have to specify `no_tls` explicitly, infer whether we need to load
the TLS keys etc from whether we have any TLS-enabled listeners.
2019-02-11 21:39:14 +00:00
Richard van der Hoff
15272f837c Merge branch 'rav/no_create_server_contexts_if_no_tls' into rav/tls_cert/work 2019-02-11 21:34:19 +00:00
Richard van der Hoff
9645728619 Don't create server contexts when TLS is disabled
we aren't going to use them anyway.
2019-02-11 21:32:01 +00:00
Richard van der Hoff
be794c7cf7 Merge branch 'rav/tls_config_logging_fixes' into rav/tls_cert/work 2019-02-11 21:16:00 +00:00
Richard van der Hoff
2129dd1a02 Fail cleanly if listener config lacks a 'port'
... otherwise we would fail with a mysterious KeyError or something later.
2019-02-11 21:15:01 +00:00
Richard van der Hoff
086f6f27d4 Logging improvements around TLS certs
Log which file we're reading keys and certs from, and refactor the code a bit
in preparation for other work
2019-02-11 21:02:06 +00:00
Richard van der Hoff
5d27730a73
Move ClientTLSOptionsFactory init out of refresh_certificates (#4611)
It's nothing to do with refreshing the certificates. No idea why it was here.
2019-02-11 18:03:30 +00:00
Erik Johnston
719e073f00
Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating
add updating of backup versions
2019-02-11 13:45:49 +00:00
Richard van der Hoff
24b7f3916d
Clean up default listener configuration (#4586)
Rearrange the comments to try to clarify them, and expand on what some of it
means.

Use a sensible default 'bind_addresses' setting.

For the insecure port, only bind to localhost, and enable x_forwarded, since
apparently it's for use behind a load-balancer.
2019-02-11 12:50:30 +00:00
Richard van der Hoff
c475275926
Clarifications for reverse proxy docs (#4607)
Factor out the reverse proxy info to a separate file, add some more info on
reverse-proxying the federation port.
2019-02-11 11:44:28 +00:00