MirrorHub/synapse
0
0
Fork 1
mirror of https://mau.dev/maunium/synapse.git synced 2024-11-04 21:58:54 +01:00
Code Issues Projects Releases Wiki Activity
14363 commits 5 branches 728 tags 90 MiB
Commit graph

1478 commits

Author SHA1 Message Date
Richard van der Hoff
b2200a8690
Logging improvements for the pusher (#4691) 2019-02-20 11:36:50 +00:00
Richard van der Hoff
c88bc53903
Fix TaskStopped exceptions when outbound requests time out (#4690) 2019-02-20 11:35:52 +00:00
Richard van der Hoff
8d98dc8ffe
Clean up gitignores (#4688) 
I just got bitten by a file being caught by the .gitignore, which shouldn't
have been, and am now pissed off with the .gitignore. I have basically declared
bankruptcy on it and started again.
 2019-02-20 11:10:50 +00:00
Richard van der Hoff
86920ac266
Merge pull request #4678 from matrix-org/rav/tls_install_instructions 
Attempt to clarify installation/config instructions
 2019-02-20 11:10:33 +00:00
Erik Johnston
dbdc565dfd Fix registration on workers (#4682) 
* Move RegistrationHandler init to HomeServer

* Move post registration actions to RegistrationHandler

* Add post regisration replication endpoint

* Newsfile
 2019-02-20 18:47:31 +11:00
Richard van der Hoff
c594cc8076 Run unit tests against python 3.7 (#4677) 
* Run unit tests against python 3.7

... so that we span the full range of our supported python versions

* Switch to xenial

* fix psql fail

* pep8 etc want python 3.6
 2019-02-20 18:42:24 +11:00
Richard van der Hoff
ae753fed8c changelog 2019-02-19 13:56:44 +00:00
Erik Johnston
c003450057
Merge pull request #4671 from matrix-org/erikj/state_cache_invalidation 
Batch cache invalidation over replication
 2019-02-19 13:14:30 +00:00
Richard van der Hoff
1c0eb8bbb2
Merge pull request #4676 from matrix-org/rav/pg95 
Test against Postgres 9.5 as well as 9.4
 2019-02-19 11:19:21 +00:00
Brendan Abolivier
a288bdf0b1
Merge pull request #4652 from matrix-org/babolivier/acme-delegated 
Support .well-known delegation when issuing certificates through ACME
 2019-02-19 11:15:38 +00:00
Erik Johnston
32590b7139
Merge pull request #4670 from matrix-org/erikj/register_login_split 
Split /login into client_reader
 2019-02-19 10:46:22 +00:00
Richard van der Hoff
7c70b8f8a6
Try and make TLS federation client code faster (#4674) 
* fix to use makeContext so that we don't need to rebuild the certificateoptions each time
 2019-02-19 10:19:16 +00:00
Richard van der Hoff
107aeb6915 misc->feature 2019-02-19 10:18:48 +00:00
Andrew Morgan
968a30a75c
Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade 
Transfer bans on room upgrade
 2019-02-19 10:16:42 +00:00
Richard van der Hoff
0869f01e74 Test against Postgres 9.5 as well as 9.4 
Postgres 9.5 is the first to support UPSERTs, so we should really run against
it as well as 9.4.
 2019-02-19 09:54:31 +00:00
Amber Brown
2b2466f78b changelog 2019-02-19 16:18:48 +11:00
Erik Johnston
92e6fb5c89 Newsfile 2019-02-18 17:58:17 +00:00
Andrew Morgan
f8b9ca53ce Move member event processing and changelog fix 2019-02-18 17:28:52 +00:00
Erik Johnston
d154f5a055
Merge pull request #4632 from matrix-org/erikj/basic_sentry 
Add basic optional sentry.io integration
 2019-02-18 17:22:45 +00:00
Erik Johnston
f3ab0b2390 Newsfile 2019-02-18 17:22:01 +00:00
Erik Johnston
128902d60a Update worker docs 2019-02-18 17:21:51 +00:00
Erik Johnston
fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split 
Split out registration to worker
 2019-02-18 17:18:06 +00:00
Erik Johnston
459d3d5046
Merge pull request #4668 from matrix-org/erikj/catch_exceptions 
Correctly handle HttpResponseException when handling device updates
 2019-02-18 16:55:43 +00:00
Erik Johnston
028267acd2
Merge pull request #4669 from matrix-org/erikj/log_exception 
Cleanup top level request exception logging
 2019-02-18 16:41:41 +00:00
Erik Johnston
d08bac4136
Merge pull request #4651 from matrix-org/matthew/well-known-cors 
set CORS on .well-known URI to unbreak modular
 2019-02-18 16:38:11 +00:00
Erik Johnston
c30f73c86a
Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers 
Fix kicking guest users in worker mode
 2019-02-18 15:55:48 +00:00
Erik Johnston
092b541401 Fixup 2019-02-18 15:52:26 +00:00
Erik Johnston
12ae64ce0d Newsfile 2019-02-18 15:23:10 +00:00
Erik Johnston
e85aabb030 Newsfile 2019-02-18 15:06:22 +00:00
Matthew Hodgson
d9713e916e changelog 2019-02-18 16:00:22 +01:00
Erik Johnston
2f16857ca9 Newsfile 2019-02-18 14:55:11 +00:00
Brendan Abolivier
68a53f825f Merge branch 'develop' into babolivier/acme-delegated 2019-02-18 14:52:23 +00:00
Erik Johnston
d1b060b492
Merge pull request #4643 from matrix-org/erikj/catch_exceptions 
Correctly handle RequestSendFailed exceptions
 2019-02-18 13:53:24 +00:00
Andrew Morgan
7033b05cad Add changelog 2019-02-18 13:52:56 +00:00
Will Hunt
e83a190643
Update changelog.d/4666.feature 
Co-Authored-By: erikjohnston <erikj@jki.re>
 2019-02-18 13:46:13 +00:00
Erik Johnston
91c8a7f9f4 Newsfile 2019-02-18 12:15:27 +00:00
Andrew Morgan
5b68e12fd8
Typo in changelog 
Co-Authored-By: babolivier <contact@brendanabolivier.com>
 2019-02-18 11:36:44 +00:00
Travis Ralston
da95867d30 Changelog 2019-02-15 22:24:39 -07:00
Erik Johnston
bd4505f765
Merge pull request #4647 from matrix-org/erikj/add_room_publishing_rules 
Add configurable room list publishing rules
 2019-02-15 22:11:01 +00:00
Brendan Abolivier
1895d14e12
Support .well-known delegation when issuing certificates through ACME 2019-02-15 12:05:08 +00:00
Erik Johnston
4074c8b968 Newsfile 2019-02-14 16:04:12 +00:00
Richard van der Hoff
b02465b9db Merge branch 'master' into develop 2019-02-14 14:42:03 +00:00
Erik Johnston
0927adb012 Newsfile 2019-02-14 14:02:04 +00:00
Erik Johnston
6cb415b63f Fixup comments and add warning 2019-02-13 16:15:11 +00:00
Richard van der Hoff
c6e75c9f2d
Merge pull request #4450 from 14mRh4X0r/fix-dependency-message 
Fix error message for optional dependencies
 2019-02-13 16:12:49 +00:00
Richard van der Hoff
3bc238629e 0.99.1rc2 2019-02-13 14:46:18 +00:00
Richard van der Hoff
c1dfd6a18a Merge remote-tracking branch 'origin/release-v0.99.1' into develop 2019-02-13 14:27:45 +00:00
Amber Brown
bb4fd8f927
Run black on user directory code (#4635) 2019-02-13 23:05:32 +11:00
Richard van der Hoff
2d0e0a4044 changelog 2019-02-13 12:00:34 +00:00
Erik Johnston
dc70789056 Newsfile 2019-02-12 16:07:43 +00:00
Erik Johnston
d2fa7b7e99 Update changelog and version 2019-02-12 13:22:25 +00:00
Erik Johnston
ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade 
Transfer Server ACLs on room upgrade
 2019-02-12 13:20:06 +00:00
Erik Johnston
b18cd25e42 Fixup changelog entries 2019-02-12 13:05:31 +00:00
Erik Johnston
cf82338930
Merge pull request #4627 from matrix-org/erikj/user_ips_analyze 
Analyze user_ips before running deduplication
 2019-02-12 13:05:09 +00:00
Erik Johnston
3df8fcca25
Merge pull request #4626 from matrix-org/erikj/fixup_user_ips_dedupe 
Reduce user_ips bloat during dedupe background update
 2019-02-12 13:02:58 +00:00
Erik Johnston
b2327eb9cb Newsfile 2019-02-12 11:58:36 +00:00
Erik Johnston
218cc071c5 Newsfile 2019-02-12 11:39:36 +00:00
Erik Johnston
3c03c37883
Merge pull request #4625 from matrix-org/rav/fix_generate_config_warnings 
fix self-signed cert notice from generate-config
 2019-02-12 11:24:45 +00:00
Richard van der Hoff
2418b91bb7
README updates (#4621) 
Lots of updates to the README/INSTALL.md.

Fixes #4601.
 2019-02-12 10:53:28 +00:00
Richard van der Hoff
a4ce91396b
Disable TLS by default (#4614) 2019-02-12 10:52:08 +00:00
Richard van der Hoff
32b781bfe2
Fix error when loading cert if tls is disabled (#4618) 
If TLS is disabled, it should not be an error if no cert is given.

Fixes #4554.
 2019-02-12 10:51:31 +00:00
Richard van der Hoff
dfc846a316 fix self-signed cert notice from generate-config 
fixes #4620
 2019-02-12 10:37:59 +00:00
Erik Johnston
46b8a79b3a
Merge pull request #4619 from matrix-org/rav/remove_docker_no_tls_hacks 
Remove redundant entries from docker config
 2019-02-12 10:00:38 +00:00
Richard van der Hoff
91f8cd3307 Remove redundant entries from docker config 
* no_tls is now redundant (#4613)
* we don't need a dummy cert any more (#4618)
 2019-02-11 22:16:44 +00:00
Richard van der Hoff
4fddf8fc77 Infer no_tls from presence of TLS listeners 
Rather than have to specify `no_tls` explicitly, infer whether we need to load
the TLS keys etc from whether we have any TLS-enabled listeners.
 2019-02-11 21:39:14 +00:00
Richard van der Hoff
15272f837c Merge branch 'rav/no_create_server_contexts_if_no_tls' into rav/tls_cert/work 2019-02-11 21:34:19 +00:00
Richard van der Hoff
9645728619 Don't create server contexts when TLS is disabled 
we aren't going to use them anyway.
 2019-02-11 21:32:01 +00:00
Richard van der Hoff
be794c7cf7 Merge branch 'rav/tls_config_logging_fixes' into rav/tls_cert/work 2019-02-11 21:16:00 +00:00
Richard van der Hoff
2129dd1a02 Fail cleanly if listener config lacks a 'port' 
... otherwise we would fail with a mysterious KeyError or something later.
 2019-02-11 21:15:01 +00:00
Richard van der Hoff
086f6f27d4 Logging improvements around TLS certs 
Log which file we're reading keys and certs from, and refactor the code a bit
in preparation for other work
 2019-02-11 21:02:06 +00:00
Richard van der Hoff
5d27730a73
Move ClientTLSOptionsFactory init out of refresh_certificates (#4611) 
It's nothing to do with refreshing the certificates. No idea why it was here.
 2019-02-11 18:03:30 +00:00
Erik Johnston
719e073f00
Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating 
add updating of backup versions
 2019-02-11 13:45:49 +00:00
Richard van der Hoff
24b7f3916d
Clean up default listener configuration (#4586) 
Rearrange the comments to try to clarify them, and expand on what some of it
means.

Use a sensible default 'bind_addresses' setting.

For the insecure port, only bind to localhost, and enable x_forwarded, since
apparently it's for use behind a load-balancer.
 2019-02-11 12:50:30 +00:00
Richard van der Hoff
c475275926
Clarifications for reverse proxy docs (#4607) 
Factor out the reverse proxy info to a separate file, add some more info on
reverse-proxying the federation port.
 2019-02-11 11:44:28 +00:00
Andrew Morgan
eff2042217 Changelog 2019-02-11 11:41:57 +00:00
Amber Brown
6e2a5aa050 ACME Reprovisioning (#4522) 2019-02-11 10:36:26 +00:00
Amber Brown
4ffd10f46d Be tolerant of blank TLS fingerprints config (#4589) 2019-02-11 10:04:27 +00:00
Erik Johnston
b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener 
New listener resource for the federation API "openid/userinfo" endpoint
 2019-02-11 09:44:00 +00:00
Valentin Anger
2dc2b6e9f1 Allow "unavailable" presence status for /sync (#4592) 
* Allow "unavailable" presence status for /sync

Closes #3772, closes #3779

Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com>

* Add changelog for PR 4592
 2019-02-08 21:09:56 +00:00
Richard van der Hoff
56710c7df5
Fix 'no unique or exclusion constraint' error (#4591) 
Add more tables to the list of tables which need a background update to
complete before we can upsert into them, which fixes a race against the
background updates.
 2019-02-08 18:30:46 +00:00
Amber Brown
9cd33d2f4b
Deduplicate some code in synapse.app (#4567) 2019-02-08 17:25:57 +00:00
Erik Johnston
acb2ac5863 Update MSC1711 FAQ to be explicit about well-known (#4584) 
A surprising number of people are using the well-known method, and are
simply copying the example configuration. This is problematic as the
example includes an explicit port, which causes inbound federation
requests to have the HTTP Host header include the port, upsetting some
reverse proxies.

Given that, we update the well-known example to be more explicit about
the various ways you can set it up, and the consequence of using an
explict port.
 2019-02-07 19:30:32 +00:00
Richard van der Hoff
7a22a645b5 Merge branch 'master' into develop 2019-02-07 19:18:48 +00:00
Richard van der Hoff
624b172e08 Merge remote-tracking branch 'origin/release-v0.99.0' 2019-02-07 19:18:26 +00:00
Andrew Morgan
c17b128b83 Update ACME docs to include port instructions (#4578) 2019-02-07 19:18:08 +00:00
Hubert Chathi
51b73be63b add changelog entry 2019-02-06 21:39:56 -05:00
Richard van der Hoff
7fe407a87a Merge branch 'master' into develop 2019-02-06 09:50:54 +00:00
Richard van der Hoff
d8e63846e2
Fix docker upload job to push -py2 images (#4576) 2019-02-06 09:41:54 +00:00
Richard van der Hoff
2475434080 Merge branch 'master' into develop 2019-02-05 18:44:49 +00:00
Richard van der Hoff
3bd9daf4b8 v0.99.0 2019-02-05 18:33:02 +00:00
Neil Johnson
6585ef4799
Neilj/1711faq (#4572) 
MSC1711 certificates FAQ
 2019-02-05 17:19:28 +00:00
Richard van der Hoff
40b35fb875
Enable ACME support in the docker image (#4566) 
Also:

* Fix wrapping in docker readme
* Clean up some docs on the docker image
* a workaround for #4554
 2019-02-05 13:42:21 +00:00
Andrew Morgan
627ecd358e Filter user directory state query to a subset of state events (#4462) 
* Filter user directory state query to a subset of state events

* Add changelog
 2019-02-05 12:16:28 +00:00
Richard van der Hoff
3ef71a6ea0
Docker: only copy what we need to the build image (#4562) 
There are two reasons this is a good thing:

 * first, it means that you don't end up with stuff kicking around your working
   copy ending up in the build image by mistake (which can upset the pip
   install process)

 * second: it means that the docker image cache is more effective, and we can
   reuse docker images when iterating on the docker stuff.
 2019-02-05 11:44:40 +00:00
Richard van der Hoff
bf1e4d96ad
Fix default ACME config for py2 (#4564) 
Fixes #4559
 2019-02-05 11:37:33 +00:00
Richard van der Hoff
9a75c0b52e
switch docker image to py3 by default (#4558) 
Switch the matrixdotorg/synapse:latest Docker image to use python 3
 2019-02-05 11:33:26 +00:00
Richard van der Hoff
d7e27a1f08
fix typo in config comments (#4557) 2019-02-05 11:32:45 +00:00
Andrew Morgan
bcc78bb0b8 Merge branch 'release-v0.99.0' of github.com:matrix-org/synapse into anoa/acme_docs 2019-02-01 15:57:32 +00:00
Richard van der Hoff
30fd2f89db 0.99.0rc4 2019-02-01 15:52:28 +00:00
Andrew Morgan
c5fc09322c Add changelog 2019-02-01 15:05:10 +00:00
Richard van der Hoff
ef43a03fc5
Merge pull request #4546 from matrix-org/rav/silence_critical_error_from_federation 
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
 2019-02-01 14:37:22 +00:00
Richard van der Hoff
f8db967d5a Merge remote-tracking branch 'origin/release-v0.99.0' into develop 2019-02-01 13:20:15 +00:00
Richard van der Hoff
9763a73af0 Merge branch 'release-v0.99.0' into develop 2019-02-01 12:30:22 +00:00
Richard van der Hoff
f0ba34f581 Fix noisy "twisted.internet.task.TaskStopped" errors in logs 
Fixes #4003
 2019-02-01 12:22:57 +00:00
Richard van der Hoff
8a21b03fba Treat an invalid .well-known the same as an absent one 
... basically, carry on and fall back to SRV etc.
 2019-02-01 11:37:31 +00:00
Richard van der Hoff
0390c961ac changelog 2019-02-01 09:40:58 +00:00
Richard van der Hoff
d428b46346 Update federation routing logic to check .well-known before SRV 2019-01-31 23:14:18 +00:00
Richard van der Hoff
85129d7068 v0.99.0rc3 2019-01-31 18:35:38 +00:00
Richard van der Hoff
07dfe148de
Add some debug for membership syncing issues (#4538) 
I can't figure out what's going on with #4422 and #4436; perhaps this will help.
 2019-01-31 18:30:40 +00:00
Andrew Morgan
bbb97a35fd Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable 2019-01-31 18:11:00 +00:00
Richard van der Hoff
e707e7b38d
Fix infinite loop when an event is redacted in a v3 room (#4535) 2019-01-31 15:34:17 +00:00
Andrew Morgan
563f6a832b Reject large transactions on federation (#4513) 
* Reject large transactions on federation

* Add changelog

* lint

* Simplify large transaction handling
 2019-01-31 11:44:04 +00:00
Richard van der Hoff
35f544410a update debian installation instructions (#4526) 
* update debian installation instructions

* docs PR is docs
 2019-01-31 10:29:15 +00:00
Andrew Morgan
cf9a2676d0 Add changelog 2019-01-30 19:04:48 +00:00
Richard van der Hoff
6fba9fd20c Merge remote-tracking branch 'origin/release-v0.99.0' into develop 2019-01-30 17:02:32 +00:00
Richard van der Hoff
b8b898666e v0.99.0rc2 2019-01-30 16:31:07 +00:00
Matthew Hodgson
ad7ac8853c by default include m.room.encryption on invites (#3902) 
* by default include m.room.encryption on invites

* fix constant

* changelog
 2019-01-30 16:26:13 +00:00
Richard van der Hoff
c74b96755c Merge remote-tracking branch 'origin/develop' into release-v0.99.0 2019-01-30 16:23:28 +00:00
Amber Brown
fbaee26c68 ACME Upgrade Docs (#4528) 2019-01-30 16:22:37 +00:00
Erik Johnston
e25ab58c5e Newsfile 2019-01-30 15:50:28 +00:00
Richard van der Hoff
a5d0c771a3 0.99.0rc1 2019-01-30 15:11:18 +00:00
Erik Johnston
6587b0b89b
Merge pull request #4472 from matrix-org/neilj/room_capabilities 
Server capabilities support
 2019-01-30 14:26:56 +00:00
Erik Johnston
a4f52a33fe Fix replication for room v3 (#4523) 
* Fix replication for room v3

We were not correctly quoting the path fragments over http replication,
which meant that it exploded when the event IDs had a slash in them

* Newsfile
 2019-01-30 14:19:52 +00:00
Richard van der Hoff
7615a8ced1 ACME config cleanups (#4525) 
* Handle listening for ACME requests on IPv6 addresses

the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses
without extra quoting. Building a string which you are about to parse again
seems like a weird choice. Let's just use listenTCP, which is consistent with
what we do elsewhere.

* Clean up the default ACME config

make it look a bit more consistent with everything else, and tweak the defaults
to listen on port 80.

* newsfile
 2019-01-30 14:17:55 +00:00
Erik Johnston
e6a7a15f93 Newsfile 2019-01-30 12:17:38 +00:00
Richard van der Hoff
a79034aedf
Merge pull request #4521 from matrix-org/rav/fed_routing/cleanups 
Tiny .well-known fixes
 2019-01-30 11:47:24 +00:00
Richard van der Hoff
c7b24ac3d0
Follow redirects on .well-known (#4520) 2019-01-30 11:43:33 +00:00
Amber Brown
f6813919e8
SIGHUP for TLS cert reloading (#4495) 2019-01-30 11:00:02 +00:00
Richard van der Hoff
283753c33a newsfile 2019-01-30 10:59:21 +00:00
Richard van der Hoff
bc5f6e1797
Add a caching layer to .well-known responses (#4516) 2019-01-30 10:55:25 +00:00
Amber Brown
3f189c902e
Fix flake8 (#4519) 2019-01-30 10:53:17 +00:00
Neil Johnson
ee4df7fd7a
Merge branch 'develop' into neilj/room_capabilities 2019-01-30 10:28:08 +00:00
Richard van der Hoff
457fbfaf22
Merge pull request #4486 from xperimental/workaround-4216 
Implement workaround for login error.
 2019-01-30 07:06:11 +00:00
Robert Jacob
2a7f0b8953 Implement workaround for login error. 
Signed-off-by: Robert Jacob <xperimental@solidproject.de>
 2019-01-30 01:06:39 +01:00
Erik Johnston
ebcffbc3eb Newsfile 2019-01-29 23:09:10 +00:00
Erik Johnston
67b82f1336 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:58:38 +00:00
Erik Johnston
7740eddd04
Merge pull request #4514 from matrix-org/erikj/remove_event_id 
Remove usages of event ID's domain
 2019-01-29 22:54:25 +00:00
Erik Johnston
a696c48133 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:00:33 +00:00
Erik Johnston
6f9cdc2d47
Merge pull request #4483 from matrix-org/erikj/event_v2 
Implement event format V2
 2019-01-29 21:40:00 +00:00
Andrew Morgan
03b086647f
Merge pull request #4512 from matrix-org/anoa/consent_dir 
Check consent dir path on startup
 2019-01-29 20:08:18 +00:00
Erik Johnston
3680bc18e9 Newsfile 2019-01-29 18:06:11 +00:00
Erik Johnston
b40abe0724 Newsfile 2019-01-29 18:02:26 +00:00
Richard van der Hoff
cc2d650ef7
Relax requirement for a content-type on .well-known (#4511) 2019-01-29 16:49:17 +00:00
Travis Ralston
d02c5ccb11
Merge pull request #4498 from matrix-org/travis/fix-docs-public_baseurl 
Don't recommend :8448 to people on public_baseurl
 2019-01-29 09:06:16 -07:00
Neil Johnson
19259d903c update to reflect broadening scope 2019-01-29 16:01:46 +00:00
Andrew Morgan
9adbc912b3 Add changelog 2019-01-29 15:34:06 +00:00
Amber Brown
6bd4374636
Do not generate self-signed TLS certificates by default. (#4509) 2019-01-29 14:09:10 +00:00
Erik Johnston
b8d75ef53e
Merge pull request #4481 from matrix-org/erikj/event_builder 
Refactor event building into EventBuilder
 2019-01-29 14:07:23 +00:00
Richard van der Hoff
99e36d5e24
Implement MSC1708 (.well-known lookups for server routing) (#4489) 2019-01-29 13:53:02 +00:00
Erik Johnston
2562319821
Merge pull request #4510 from matrix-org/erikj/fixup_compute_event_signature 
Fixup calls to `comput_event_signature`
 2019-01-29 13:35:19 +00:00
Richard van der Hoff
5488cadaae
Enable configuring test log level via env var (#4506) 
I got fed up with always adding '@unittest.DEBUG' every time I needed to debug a test.
 2019-01-29 12:07:00 +00:00
Erik Johnston
7d1024d574 Newsfile 2019-01-29 11:58:16 +00:00
Erik Johnston
0c55b7701c Newsfile 2019-01-29 11:42:33 +00:00
Erik Johnston
ff37acb8ce
Merge pull request #4496 from matrix-org/erikj/invite_fallback 
Implement fallback for V2 invite API
 2019-01-29 11:28:23 +00:00
Erik Johnston
a388d59d44 Newsfile 2019-01-29 11:13:08 +00:00
Amber Brown
f815bd7feb
Make linearizer more quiet (#4507) 2019-01-29 11:05:31 +00:00
Erik Johnston
073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator 
Split up event validation between event and builder
 2019-01-29 10:55:07 +00:00
Amber Brown
94fb63e44f
Fix typo in upserts code (#4505) 
* fix obvious problem :|

* changelog
 2019-01-29 10:04:23 +00:00
Erik Johnston
17709f8f9c
Merge pull request #4493 from matrix-org/erikj/refactor_event_signing 
Refactor event signing to work on dicts
 2019-01-29 09:52:54 +00:00
Richard van der Hoff
f2b553d656 Use SimpleResolverComplexifier in tests (#4497) 
two reasons for this. One, it saves a bunch of boilerplate. Two, it squashes
unicode to IDNA-in-a-`str` (even on python 3) in a way that it turns out we
rely on to give consistent behaviour between python 2 and 3.
 2019-01-29 09:38:29 +00:00
Erik Johnston
f1a04462eb
Merge pull request #4482 from matrix-org/erikj/event_auth_room_version 
Pass through room version to event auth
 2019-01-28 20:09:38 +00:00
Travis Ralston
c4045647eb
Create 4498.misc 2019-01-28 12:16:39 -07:00
Erik Johnston
8cbc99cc19 Newsfile 2019-01-28 17:36:06 +00:00
Amber Brown
5d976c0c7c
Fix worker TLS (#4492) 
* load cert

* changelog

* fix
 2019-01-28 17:18:33 +00:00
Erik Johnston
1977a9b006 Newsfile 2019-01-28 17:05:04 +00:00
Erik Johnston
b8bea3424f Newsfile 2019-01-28 16:47:12 +00:00
Amber Brown
7072fe3084
Fix UPSERTs on SQLite 3.24+ (#4477) 2019-01-28 15:43:32 +00:00
Andrew Morgan
4026d555fa Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-28 14:08:24 +00:00
Aaron Raimist
57a3e96e8e Remove --process-dependency-links from UPGRADE.rst (#4485) 
* Remove --process-dependency-links from UPGRADE.rst

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-01-28 13:54:25 +00:00
Richard van der Hoff
ff05ad147a changelog 2019-01-28 10:34:30 +00:00
Richard van der Hoff
d840019192
Fix idna and ipv6 literal handling in MatrixFederationAgent (#4487) 
Turns out that the library does a better job of parsing URIs than our
reinvented wheel. Who knew.

There are two things going on here. The first is that, unlike
parse_server_name, URI.fromBytes will strip off square brackets from IPv6
literals, which means that it is valid input to ClientTLSOptionsFactory and
HostnameEndpoint.

The second is that we stay in `bytes` throughout (except for the argument to
ClientTLSOptionsFactory), which avoids the weirdness of (sometimes) ending up
with idna-encoded values being held in `unicode` variables. TBH it probably
would have been ok but it made the tests fragile.
 2019-01-28 09:56:59 +00:00
Erik Johnston
f01c7488ab Newsfile 2019-01-25 18:32:50 +00:00
Erik Johnston
b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version 
Require event format version to parse or create events
 2019-01-25 15:59:36 +00:00
Erik Johnston
431e485914 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sqlite_native_upsert 2019-01-25 14:11:17 +00:00
Richard van der Hoff
4a3f138832
Fix quoting for allowed_local_3pids example config (#4476) 
If you use double-quotes here, you have to escape your backslashes. It's much
easier with single-quotes.

(Note that the existing double-backslashes are already interpreted by python's
""" parsing.)
 2019-01-25 13:57:52 +00:00
Richard van der Hoff
8520bc3109
Fix Host header sent by MatrixFederationAgent (#4468) 
Move the Host header logic down here so that (a) it is used if we reuse the
agent elsewhere, and (b) we can mess about with it with .well-known.
 2019-01-25 12:38:16 +00:00
Andrew Morgan
0b3fd1401f Don't require sqlite3 when using postgres (#4466) 2019-01-25 22:25:02 +11:00
Neil Johnson
a3f0556bea towncrier 2019-01-25 11:15:41 +00:00
Andrew Morgan
821b65aeb5 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-25 11:09:53 +00:00
Andrew Morgan
b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms 
Ability to search entire room history after upgrading room
 2019-01-25 10:49:58 +00:00
Erik Johnston
8dcfa6e75c Newsfile 2019-01-25 10:48:40 +00:00
Erik Johnston
5d881cbcb8 Newsfile 2019-01-25 10:37:13 +00:00
Erik Johnston
62514bb81b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813 2019-01-25 10:07:08 +00:00
Erik Johnston
edc1e21dbe
Merge pull request #4448 from matrix-org/erikj/get_pdu_versions 
Add room_version param to get_pdu
 2019-01-24 18:47:15 +00:00
Erik Johnston
80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites 
Store rejected remote invite events as outliers
 2019-01-24 18:33:27 +00:00
Erik Johnston
664b7a2920
Merge pull request #4437 from matrix-org/erikj/event_format_version_v2 
Add support for persisting event format versions
 2019-01-24 18:32:30 +00:00
Andrew Morgan
50c396a7ee
Merge pull request #4461 from matrix-org/anoa/room_dir_quick_fix 
Add a 60s timeout to filtered room directory queries
 2019-01-24 15:24:35 +00:00
Richard van der Hoff
4a6e863843
Merge pull request #4464 from matrix-org/rav/fix_srv_lookup 
MatrixFederationAgent: Look up the right SRV record
 2019-01-24 13:57:51 +00:00
Richard van der Hoff
8c58c10697
Generate the debian config during build (#4444) 
Rather than hardcoding a config which we always forget to update, generate it
from the default config.
 2019-01-24 13:39:01 +00:00
Richard van der Hoff
f4697b5ec1
Fix UnboundLocalError in post_urlencoded_get_json (#4460) 
This could cause exceptions if the id server returned 4xx responses.
 2019-01-24 13:38:29 +00:00
Richard van der Hoff
afd69a0920 Look up the right SRV record 2019-01-24 13:31:43 +00:00
Neil Johnson
10b89d5c2e
Merge pull request #4435 from matrix-org/neilj/fix_threepid_auth_check 
Neilj/fix threepid auth check
 2019-01-24 13:02:50 +00:00
Andrew Morgan
2a360e834f Add changelog 2019-01-24 12:47:35 +00:00
Neil Johnson
5c41b22359
Merge pull request #4458 from matrix-org/dbkr/public_baseurl_doc 
Clarify docs for public_baseurl
 2019-01-24 12:36:09 +00:00
Amber Brown
0e27501ee5
Fix UPSERT check (#4459) 2019-01-24 22:57:41 +11:00
David Baker
1f2058fca5 Changelog 2019-01-24 10:57:12 +00:00
Amber Brown
58f6c48183
Use native UPSERTs where possible (#4306) 2019-01-24 21:31:54 +11:00
Richard van der Hoff
97fd29c019
Don't send IP addresses as SNI (#4452) 
The problem here is that we have cut-and-pasted an impl from Twisted, and then
failed to maintain it. It was fixed in Twisted in
https://github.com/twisted/twisted/pull/1047/files; let's do the same here.
 2019-01-24 09:34:44 +00:00
Erik Johnston
183738f469 Newsfile 2019-01-23 20:04:57 +00:00
Willem Mulder
d528406cb8 Fix error message for optional dependencies 
Signed-off-by: Willem Mulder <willemmaster@hotmail.com>
 2019-01-23 18:44:57 +01:00
Erik Johnston
c5a125b24f Update newsfile 2019-01-23 17:26:06 +00:00
Erik Johnston
4a8b715679 Newsfile 2019-01-23 17:25:27 +00:00
Erik Johnston
a4ef8d8dd5 Newsfile 2019-01-23 17:24:44 +00:00
Andrew Morgan
6b90ae6efc
Merge pull request #4445 from matrix-org/anoa/user_dir_develop_backport 
Add metric for user dir current event stream position
 2019-01-23 15:58:40 +00:00
Richard van der Hoff
a0ae475219
Merge pull request #4428 from matrix-org/rav/matrix_federation_agent 
Move SRV magic into an Agent-like thing
 2019-01-23 15:50:25 +00:00
Andrew Morgan
cb0e637a94 Add changelog 2019-01-23 15:38:27 +00:00
Richard van der Hoff
2f88881c93
debian package: symlink to python-3.X (#4433) 
In the debian package, make the virtualenv symlink python to /usr/bin/python3.X
rather than /usr/bin/python3. Also make sure we depend on the right python3.x
package.

This might help a bit with subtle failures when people install a package from
the wrong distro (https://github.com/matrix-org/synapse/issues/4431).
 2019-01-23 11:43:04 +00:00
Erik Johnston
4cd50d983d Newsfile 2019-01-23 11:30:24 +00:00
Erik Johnston
90743c9d89 Fixup removal of duplicate user_ips rows (#4432) 
* Remove unnecessary ORDER BY clause

* Add logging

* Newsfile
 2019-01-23 19:45:18 +11:00
Amber Brown
6129e52f43
Support ACME for certificate provisioning (#4384) 2019-01-23 19:39:06 +11:00
Jason Robinson
db33634b1d Collapse changelog to one line 
Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2019-01-23 10:32:41 +02:00
Jason Robinson
1d2c69fee8 Add changelog for openid resource addition 
Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2019-01-23 10:32:41 +02:00
Richard van der Hoff
c66f4bf7f1 changelog 2019-01-22 20:34:35 +00:00
Neil Johnson
c99c2d58d7 move guard out of is_threepid_reserved and into register.py 2019-01-22 17:47:00 +00:00
Neil Johnson
d619b113ed Fix None guard in config.server.is_threepid_reserved 2019-01-22 16:52:29 +00:00
Erik Johnston
12699a701f
Merge pull request #4434 from matrix-org/erikj/fix_user_ips_dedup 
Fix bug when removing duplicate rows from user_ips
 2019-01-22 16:51:57 +00:00
Neil Johnson
388c164aea
Merge pull request #4423 from matrix-org/neilj/disable_msisdn_on_registration 
Config option to disable requesting MSISDN on registration
 2019-01-22 16:23:08 +00:00
Erik Johnston
c658425e6f Newsfile 2019-01-22 13:34:10 +00:00
Andrew Morgan
117bc94cd2 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-22 13:16:51 +00:00
Richard van der Hoff
6bfa735a69
Make key fetches use regular federation client (#4426) 
All this magic is redundant.
 2019-01-22 11:04:20 +00:00
Andrew Morgan
25d64a846a Fix typos 2019-01-22 11:00:04 +00:00
Andrew Morgan
1f18c7cfc9 Add changelog 2019-01-22 11:00:04 +00:00
Richard van der Hoff
33a55289cb
Refactor and bugfix for resove_service (#4427) 2019-01-22 10:59:27 +00:00
Amber Brown
23b0813599
Require ECDH key exchange & remove dh_params (#4429) 
* remove dh_params and set better cipher string
 2019-01-22 21:58:50 +11:00
Neil Johnson
5349262302 Config option to disable requesting MSISDN on registration 2019-01-21 14:59:37 +00:00
Erik Johnston
35e1d67b4e Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_v2_invite_server 2019-01-21 14:04:19 +00:00
Erik Johnston
5f54765587
Merge pull request #4390 from matrix-org/erikj/versioned_fed_apis 
Add groundwork for new versions of federation APIs
 2019-01-21 11:44:05 +00:00
Andrew Morgan
702c4b750c Migrate encryption state on room upgrade (#4411) 
* Migrate encryption state on room upgrade

Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>

* Add changelog file
 2019-01-21 20:42:58 +11:00
Erik Johnston
25dd56ace3 Fix race when persisting create event (#4404) 
* Fix race when persisting create event

When persisting a chunk of DAG it is sometimes requried to do a state
resolution, which requires knowledge of the room version. If this
happens while we're persisting the create event then we need to use that
event rather than attempting to look it up in the database.
 2019-01-18 23:17:04 +11:00
Erik Johnston
71b94eac46 Tweak code coverage settings (#4400) 
* Tweak code coverage settings

* Fix manifest

* Newsfile

* Fix commit status?
 2019-01-18 23:13:14 +11:00
Richard van der Hoff
de6888e7ce Remove redundant WrappedConnection (#4409) 
* Remove redundant WrappedConnection

The matrix federation client uses an HTTP connection pool, which times out its
idle HTTP connections, so there is no need for any of this business.
 2019-01-18 23:07:38 +11:00
Andrew Morgan
cb80db8941 Add changelog 2019-01-18 11:22:00 +00:00
Richard van der Hoff
676cf2ee26
Fix incorrect logcontexts after a Deferred was cancelled (#4407) 2019-01-17 14:00:23 +00:00
Richard van der Hoff
9feb5d0b71
sign_request -> build_auth_headers (#4408) 
Just got very confused about the fact that the headers are only an output, not
an input.
 2019-01-17 12:40:09 +00:00
Richard van der Hoff
3982a6ee07
Changing macaroon_secret_key no longer logs you out (#4387) 2019-01-16 23:14:41 +00:00
Richard van der Hoff
05e1296649
don't store more remote device lists if they have more than 1K devices (#4397) 2019-01-16 23:14:11 +00:00
Erik Johnston
f788c9eb70 Newsfile 2019-01-16 13:46:36 +00:00
Andrej Shadura
3b31a54a6e
Add a changelog entry 
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
 2019-01-16 10:54:43 +01:00
Neil Johnson
9ec56d6935 ALL_USER_TYPES should be a tuple 2019-01-15 14:38:15 +00:00
Erik Johnston
1a8f4139a5 Newsfile 2019-01-15 11:21:52 +00:00
Richard van der Hoff
34b25dcc8e Silence travis-ci build warnings by removing non-functional python3.6 (#4377) 
* Remove non-functional python3.6 in travis env

* changelog
 2019-01-12 06:22:56 +11:00
Amber Brown
a35c66a00b
Remove duplicates in the user_ips table and add an index (#4370) 2019-01-12 06:21:50 +11:00
Richard van der Hoff
8c818af38e
Merge pull request #4342 from aaronraimist/new-virtualenv 
Update README to use new virtualenv (#4328)
 2019-01-10 14:59:33 +00:00
Richard van der Hoff
d566e6b17a Merge branch 'master' into develop 2019-01-09 15:00:46 +00:00
Richard van der Hoff
998f5225c1 0.34.1 2019-01-09 14:53:54 +00:00
Erik Johnston
55c3e853c5
Merge pull request #4368 from matrix-org/erikj/better_errors 
Fixup docstrings for matrixfederationclient
 2019-01-09 11:39:28 +00:00
Erik Johnston
484867d35d Merge branch 'develop' into release-v0.34.1 2019-01-09 11:38:13 +00:00
Amber Brown
7960c26fda
Fix adding new rows instead of updating them if one of the key values is a NULL in upserts. (#4369) 2019-01-09 22:26:25 +11:00
Erik Johnston
0dce21ba77 Newsfile 2019-01-09 09:27:03 +00:00
Erik Johnston
d91b99abe0 Newsfile 2019-01-08 14:24:05 +00:00
Erik Johnston
83c50bf752 Newsfile 2019-01-08 12:28:35 +00:00
Richard van der Hoff
bc1a4b5576 changelog 2019-01-08 11:14:59 +00:00
Erik Johnston
b970cb0e96 Refactor request sending to have better excpetions (#4358) 
* Correctly retry and back off if we get a HTTPerror response

* Refactor request sending to have better excpetions

MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).

This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
 2019-01-08 11:04:28 +00:00
Amber Brown
1dcb086f33
Fix synapse.config.__main__ on python 3 (#4356) 2019-01-08 10:03:09 +11:00
Jason Robinson
26e5abf20d Fix command hint to generate a config file (#4353) 
* Fix command hint to generate a config file

When trying to start Synapse without a config file, it will complain
and give a hint towards what command to run. This hinted command
is missing the "report_stats" parameter, which is required with either
yes or no value. Add this to the command.

Not an ideal situation but makes the given command work without the
user getting another error, even though it might be unclear what
"report_stats" represents.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2019-01-07 16:28:40 +00:00
Travis Ralston
bc1fa8cd01 Add GET account data routes (#4303) 
As per https://github.com/matrix-org/matrix-doc/issues/1339
 2019-01-07 10:27:54 +00:00
Richard van der Hoff
b7c0218812
Check jinja version for consent resource (#4327) 
* Raise a ConfigError if an invalid resource is specified

* Require Jinja 2.9 for the consent resource

* changelog
 2019-01-07 10:14:31 +00:00
Michael Telatynski
a27e501b09 fix the check for whether is_url to match all the other ones in codebase (#3405) 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
 2019-01-06 21:43:25 +00:00
Richard van der Hoff
29f20a8a1a
Update debian Conflicts specifications (#4349) 
...  to allow installation alongside our matrix-synapse transitional package.
 2019-01-04 17:24:13 +00:00
Matthew Hodgson
27128145e6 fix NPE in /messages by checking if all events were filtered out (#4330) 2019-01-02 23:37:39 +00:00
Aaron Raimist
37f8bdc1d5 Update README to not lie about required restart (#4343) 
* Update README to not lie about required restart

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-01-02 16:32:37 +00:00
Neil Johnson
84b6fae1f5 Ensure synchrotrons can access is_support_user in the storage layer 2019-01-02 10:19:59 +00:00
Aaron Raimist
252c0c81fa Update PR template to use absolute links (#4341) 
* Update PR template to use absolute links

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-01-02 07:33:13 +00:00
Richard van der Hoff
e9cdfedff3
Avoid packaging _trial_temp directory (#4326) 
Make sure we don't put the _trial_temp directory in the package target
directory.

Fixes https://github.com/matrix-org/synapse/issues/4322
 2019-01-02 07:30:31 +00:00
Richard van der Hoff
7134832c01
Install the optional dependencies into the debian package (#4325) 
since #4298, the optional dependencies are no longer installed with a simple
`pip install .`, which meant that they were not being included in the debian
package.

The easy fix to that is dh_virtualenv --extras, but that needs dh_virtualenv
1.1...
 2019-01-02 07:17:39 +00:00
Aaron Raimist
7975d39cbd Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-01-01 15:40:55 -06:00
Amber Brown
d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console (#4334) 2018-12-29 23:12:30 +11:00
Krithin Sitaram
98df67a8de Remove mention of lt-cred-mech in the sample coturn config. (#4333) 
* Remove mention of lt-cred-mech in the sample coturn config.

See https://github.com/coturn/coturn/pull/262 for more context.
Also clean up some minor formatting issues while I'm here.

* Add changelog.

Signed-off-by: Krithin Sitaram <krithin@gmail.com>
 2018-12-28 23:31:49 +00:00
Will Hunt
0708f437cc Log roomid along with Unknown room (#4297) 2018-12-24 10:49:42 +01:00
Richard van der Hoff
cd018e3543
Merge pull request #4307 from matrix-org/erikj/v2_rooms 
Add v2 room version
 2018-12-24 10:48:59 +01:00
Richard van der Hoff
f85676cc93
Return well_known in /login response (#4319) 
... as per MSC1730.
 2018-12-24 10:44:33 +01:00
Richard van der Hoff
3355c1a9ec
Merge pull request #4317 from de-vri-es/test-metric-prometheus-0.5 
Fix test_metrics.py compatibility prometheus_client 0.5
 2018-12-23 00:14:47 +01:00
Richard van der Hoff
597dafb5e8
Merge pull request #4316 from matrix-org/rav/fix_docker_upload 
Fix circleci config for synapse:latest docker upload
 2018-12-21 17:33:34 +01:00
Richard van der Hoff
9c2af7b2c5 Add a script to generate a clean config file (#4315) 2018-12-22 02:04:57 +11:00
Richard van der Hoff
f3561f8d86 Fix indentation in default config (#4313) 
These settings are not supposed to be under 'listeners'.
 2018-12-22 02:01:43 +11:00
Amber Brown
ea6abf6724
Fix IP URL previews on Python 3 (#4215) 2018-12-22 01:56:13 +11:00
Amber Brown
c26f49a664
Make the dependencies more like a standard Python project and hook up the optional dependencies to setuptools (#4298) 2018-12-22 01:37:26 +11:00
Maarten de Vries
48b7ff7a35 Fix test_metrics.py compatibility prometheus_client 0.5 
prometheus_client 0.5 has a named-tuple Sample type with more member
than the old plain tuple had. This commit makes sure the unit test
detects this and changes the way it reads the sample.

Signed-off-by: Maarten de Vries <maarten@de-vri.es>
 2018-12-21 01:53:57 +01:00
Richard van der Hoff
d9aaf26539 Fix circleci config for synapse:latest docker upload 
Give the image the right tag, so that we can push it.
 2018-12-21 00:22:31 +00:00
Nathan Pennie
da8628ba2c
Create 4309.bugfix 2018-12-19 14:42:49 -05:00
Erik Johnston
1dc7492ce5 Newsfile 2018-12-18 18:11:56 +00:00
Neil Johnson
7e22cd90f5
ensure can report mau stats when hs.config.mau_stats_only is set (#4305) 
* ensure can report mau stats when hs.config.mau_stats_only is set
 2018-12-18 14:36:11 +00:00
Neil Johnson
d2f7c4e6b1
create support user (#4141) 
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
 2018-12-14 18:20:59 +00:00
Amber Brown
e93a0ebf50
Settings Fix deleting e2e room keys on xenial (#4295) 2018-12-14 05:10:31 +11:00
Nad Chishtie
cef8ae272a Improved welcome page (#4294) 
* Improved static/index.html look, feel and content.

* Added accessibility tags to in-line SVG.

* Grammar.
 2018-12-14 02:42:51 +11:00
Richard van der Hoff
a83826ae99
Merge pull request #4274 from matrix-org/michaelkaye/update_kernel_dco_link 
Update link to kernel.org DCO usage
 2018-12-11 20:55:43 +01:00
Richard van der Hoff
f208f608cb Merge branch 'release-v0.34.0' into develop 2018-12-11 15:43:20 +00:00
Richard van der Hoff
eb835bf65b 0.34.0rc2 2018-12-11 15:35:47 +00:00
Richard van der Hoff
03116da984 remove changelog files 2018-12-11 15:35:10 +00:00
Richard van der Hoff
188945713e
Merge pull request #4290 from matrix-org/rav/remove_webclient 
Stop installing Matrix Console by default
 2018-12-11 16:24:15 +01:00
Richard van der Hoff
bd52978fd7 Merge branch 'rav/disable_pager_in_ci' into rav/welcome_page 2018-12-11 14:13:02 +00:00
Richard van der Hoff
2755a0d48a Disable pager for ci script 
... otherwise it hangs
 2018-12-11 14:12:22 +00:00
Richard van der Hoff
f537432ef9 Add a welcome page to the static resources 
This is largely a precursor for the removal of the bundled webclient. The idea
is to present a page at / which reassures people that something is working, and
to give them some links for next steps.

The welcome page lives at `/_matrix/static/`, so is enabled alongside the other
`static` resources (which, in practice, means the client API is enabled). We'll
redirect to it from `/` if we have nothing better to display there.

It would be nice to have a way to disable it (in the same way that you might
disable the nginx welcome page), but I can't really think of a good way to do
that without a load of ickiness.

It's based on the work done by @krombel for #2601.
 2018-12-11 13:26:22 +00:00
Richard van der Hoff
df96177ca7 Stop installing Matrix Console by default 
This is based on the work done by @krombel in #2601.
 2018-12-11 13:20:33 +00:00
David Baker
89ac2a5bdb Add 'sandbox' to CSP for media repo (#4284) 
* Add 'sandbox' to the CSP for media repo

* Changelog
 2018-12-11 04:05:02 +11:00
Will Hunt
989f1167af
Merge pull request #4279 from matrix-org/hs/fix-config-cors 
Make /config more CORS-y
 2018-12-10 15:24:03 +00:00
Erik Johnston
74c3606c53 Newsfile 2018-12-10 14:15:43 +00:00
Will Hunt
91206e09f2 changelog & isort 2018-12-09 17:39:44 +00:00
Michael Kaye
912a843294
changelog 2018-12-07 17:50:52 +00:00
Richard van der Hoff
30da50a5b8
Initialise user displayname from SAML2 data (#4272) 
When we register a new user from SAML2 data, initialise their displayname
correctly.
 2018-12-07 14:44:46 +01:00
Richard van der Hoff
35e13477cf
Update the example systemd config to use a virtualenv (#4273) 
If you're installing as a system package, the system package should have set up
the systemd config, so it's more useful to give an example of running in a
virtualenv here.
 2018-12-07 14:43:41 +01:00
Richard van der Hoff
c7401a697f
Implement SAML2 authentication (#4267) 
This implements both a SAML2 metadata endpoint (at
`/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at
`/_matrix/saml2/authn_response`). If the SAML2 response matches what's been
configured, we complete the SSO login flow by redirecting to the client url
(aka `RelayState` in SAML2 jargon) with a login token.

What we don't yet have is anything to build a SAML2 request and redirect the
user to the identity provider. That is left as an exercise for the reader.
 2018-12-07 13:11:11 +01:00
Richard van der Hoff
c588b9b9e4
Factor SSO success handling out of CAS login (#4264) 
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
 2018-12-07 13:10:07 +01:00
rkfg
ae19a7db8c Prevent crash on pagination. 2018-12-06 13:32:05 +03:00
Richard van der Hoff
b0c24a66ec Rip out half-implemented m.login.saml2 support (#4265) 
* Rip out half-implemented m.login.saml2 support

This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.

* drop undocumented dependency on dateutil

It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
 2018-12-06 19:44:38 +11:00
Richard van der Hoff
9a3e24a13d drop undocumented dependency on dateutil (#4266) 
It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
 2018-12-06 04:52:42 +11:00
Richard van der Hoff
e8d98466b0
Implement .well-known handling (#4262) 
Sometimes it's useful for synapse to generate its own .well-known file.
 2018-12-05 14:38:58 +01:00
Richard van der Hoff
a2ed0f287e
Merge pull request #4260 from matrix-org/rav/python3 
Notes on upgrading to python3, and README updates.
 2018-12-04 14:46:31 +01:00
Richard van der Hoff
75937e9033 Remove obsolete settings from docker homeserver.yaml 
These aren't used, because we have a `log_config` setting.
 2018-12-04 12:31:00 +00:00
Richard van der Hoff
4acd1a3549 Notes on upgrading to python3, and README updates. 2018-12-04 12:28:24 +00:00
Travis Ralston
1737753a62 Add an option to enable recording IPs for appservice users (#3831) 2018-12-04 12:44:41 +01:00
Amber Brown
fd96dd75a3 Fix non-ASCII pushrules (#4248) 2018-12-04 12:44:02 +01:00
Travis Ralston
158ffb92f1 Add an option to disable search for homeservers which may not be interested in it (#4230) 
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
 2018-12-04 12:01:02 +01:00
Aaron Raimist
512e94d230 Add note to UPGRADE.rst about removing riot.im from list of trusted identity servers (#4224) 
* Add note to UPGRADE.rst about removing riot.im from list of trusted identity servers

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-12-04 11:59:09 +01:00
Ben Parsons
b5ac0ffa0a add more detail to logging regarding "More than one row matched" error (#4234) 2018-12-04 11:57:39 +01:00
Richard van der Hoff
ecc23188f4
Fix UnicodeDecodeError when postgres is not configured in english (#4253) 
This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252. Fundamentally the right answer is to drop support for Python 2.
 2018-12-04 11:55:52 +01:00
Richard van der Hoff
f144c0a210
Merge pull request #4244 from aaronraimist/drop-sent-txt 
Drop sent_transactions
 2018-12-04 11:41:28 +01:00
Richard van der Hoff
48972ce9d1
Patch defer.inlineCallbacks to check logcontexts in tests (#4205) 2018-12-04 11:30:32 +01:00
Richard van der Hoff
a077e710a3
Merge pull request #4250 from matrix-org/hawkowl/pusher-remove-py3 
Fix removing pushers on python 3
 2018-12-04 11:22:46 +01:00
Richard van der Hoff
a484735bb0
Merge pull request #4257 from aaronraimist/add-editorconfig 
Add a basic .editorconfig
 2018-12-04 11:10:02 +01:00
Richard van der Hoff
52e87fbfbe
Run the AS senders as background processes (#4189) 
This should fix some "Starting db connection from sentinel context" warnings,
and will mean we get metrics for these processes.
 2018-12-04 10:53:49 +01:00
Aaron Raimist
3518c28aa8
Add a basic .editorconfig 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-12-03 22:38:47 -06:00
Amber Brown
998ba41493 changelog 2018-12-03 22:28:12 +11:00
Richard van der Hoff
c03324294d Workaround for non-ascii event ids (#4241) 
It turns out that we accept events with non-ascii IDs, which would later cause
an explosion during state res.

Fixes #4226
 2018-12-03 21:47:48 +11:00
Aaron Raimist
44dc4c365b
Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-12-01 23:10:21 -06:00
Neil Johnson
7039ece8fb Neilj/fix autojoin (#4223) 
* Fix auto join failures for servers that require user consent

* Fix auto join failures for servers that require user consent
 2018-11-28 22:24:57 +11:00
Amber Brown
8ca53fb53e
Report combined coverage to codecov (#4225) 2018-11-28 20:59:31 +11:00
Neil Johnson
f9b136a886 Neilj/fix mau initial reserved users (#4211) 
* fix transaction wrapping bug that caused get_user_id_by_threepid_txn to fail

* towncrier

* white space
 2018-11-28 20:33:41 +11:00
Richard van der Hoff
944d524f18 Support m.login.sso (#4220) 
* Clean up the CSS for the fallback login form

I was finding this hard to work with, so simplify a bunch of things. Each
flow is now a form inside a div of class login_flow.

The login_flow class now has a fixed width, as that looks much better than each
flow having a differnt width.

* Support m.login.sso

MSC1721 renames m.login.cas to m.login.sso. This implements the change
(retaining support for m.login.cas for older clients).

* changelog
 2018-11-27 18:51:52 +11:00
Richard van der Hoff
a44c0a096f Check logcontexts before and after each test (#4190) 
* Add better diagnostics to flakey keyring test

* fix interpolation fail

* Check logcontexts before and after each test

* update changelog

* update changelog
 2018-11-27 13:47:18 +11:00
Richard van der Hoff
80527b568d Fix more logcontext leaks in tests (#4209) 2018-11-27 13:01:04 +11:00
Richard van der Hoff
de8772a655 Do a GC after each test to fix logcontext leaks (#4227) 
* Some words about garbage collections and logcontexts

* Do a GC after each test to fix logcontext leaks

This feels like an awful hack, but...

* changelog
 2018-11-27 13:00:33 +11:00
Richard van der Hoff
6c18cc4b50 Ignore __pycache__ directories in schema delta dir 
Now that we use py3, compiled python ends up in __pycache__ rather than *.pyc.
 2018-11-20 22:52:34 +00:00
Neil Johnson
78ba0e7ab8 Remove riot.im from the list of trusted Identity Servers in the default configuration (#4207) 2018-11-20 12:29:25 +01:00
Richard van der Hoff
416c671474
Merge pull request #4204 from matrix-org/rav/logcontext_leak_fixes 
Fix some logcontext leaks
 2018-11-20 12:19:19 +01:00
Amber Brown
31425d82a3 Merge remote-tracking branch 'origin/master' into develop 2018-11-19 12:55:25 -06:00
Amber Brown
80cac86b2c
Fix fallback auth on Python 3 (#4197) 2018-11-19 12:27:33 -06:00
Richard van der Hoff
0c05da2e2e changelog 2018-11-19 17:07:42 +00:00
Richard van der Hoff
10cdf519aa
Merge pull request #4182 from aaronraimist/update-issue-template 
Add a pull request template and add multiple issue templates
 2018-11-19 14:24:30 +01:00
Aaron Raimist
cc2cf2da97
Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-11-18 12:42:08 -06:00
Amber Brown
4285c818ec
Merge pull request #4193 from kivikakk/add-openbsd-prereq 
add jpeg to OpenBSD prereq list
 2018-11-17 14:27:53 -06:00
Ashe Connor
ceca3b2f30
add changelog.d entry 2018-11-17 15:01:02 +11:00
Travis Ralston
d75db3df59 Changelog 2018-11-15 20:44:57 -07:00
Amber Brown
8b1affe7d5
Fix Content-Disposition in media repository (#4176) 2018-11-15 15:55:58 -06:00
Travis Ralston
835779f7fb Add option to track MAU stats (but not limit people) (#3830) 2018-11-15 18:08:27 +00:00
Amber Brown
df758e155d
Use <meta> tags to discover the per-page encoding of html previews (#4183) 2018-11-15 11:05:08 -06:00
Amber Brown
a51288e5d6
Add a coveragerc (#4180) 2018-11-15 10:50:08 -06:00
Richard van der Hoff
4f8bb633c7 Update README for #1491 fix 2018-11-15 10:03:36 +00:00
Richard van der Hoff
0c4dc6fd76 changelog 2018-11-14 10:48:08 +00:00
David Baker
0869566ad3
Merge pull request #4113 from matrix-org/dbkr/e2e_backup_versions_are_numbers 
Make e2e backup versions numeric in the DB
 2018-11-14 07:55:48 +00:00
Aaron Raimist
9ca1215582
Add changelog 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-11-13 21:46:48 -06:00
David Baker
bca3b91c2d Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers 2018-11-09 18:35:02 +00:00
Erik Johnston
db5a1c059a
Merge pull request #4166 from matrix-org/erikj/drop_unknown_events 
Drop incoming events from federation for unknown rooms
 2018-11-09 17:59:34 +00:00
Erik Johnston
7b22421a7b
Merge pull request #4164 from matrix-org/erikj/fix_device_comparison 
Fix noop checks when updating device keys
 2018-11-08 14:37:20 +00:00
Erik Johnston
94896d7ffe Newsfile 2018-11-08 12:30:25 +00:00
Erik Johnston
06c3d8050f Newsfile 2018-11-08 12:18:41 +00:00
Richard van der Hoff
de6223836e changelog 2018-11-08 11:06:28 +00:00
Amber Brown
264cb14402
Port hash_password to Python 3 (#4161) 
* port hash_password

* changelog
 2018-11-08 04:57:28 +11:00
Amber Brown
b3708830b8
Fix URL preview bugs (type error when loading cache from db, content-type including quotes) (#4157) 2018-11-08 01:37:43 +11:00
Richard van der Hoff
c8ba79327b
Merge pull request #4155 from rubo77/purge-api 
add purge_history.sh and purge_remote_media.sh scripts
 2018-11-07 14:06:41 +00:00
rubo77
2904d133f3 add purge_history.sh and purge_remote_media.sh scripts to contrib/purge_api/ 
Signed-off-by: Ruben Barkow <github@r.z11.de>
 2018-11-07 14:02:41 +01:00
Amber Brown
e62f7f17b3
Remove some boilerplate in tests (#4156) 2018-11-07 03:00:00 +11:00
Travis Ralston
0f5e51f726 Add config variables for enabling terms auth and the policy name (#4142) 
So people can still collect consent the old way if they want to.
 2018-11-06 10:32:34 +00:00
Hubert Chathi
f1087106cf
handle empty backups according to latest spec proposal (#4123) 
fixes #4056
 2018-11-05 17:59:29 -05:00
Amber Brown
efdcbbe46b
Tests for user consent resource (#4140) 2018-11-06 05:53:44 +11:00
Amber Brown
5a63589e80
Add some tests for the HTTP pusher (#4149) 2018-11-06 05:53:24 +11:00
Erik Johnston
bc80b3f454 Add helpers for getting prev and auth events (#4139) 
* Add helpers for getting prev and auth events

This is in preparation for allowing the event format to change between
room versions.
 2018-11-06 00:35:15 +11:00
Amber Brown
0467384d2f
Set the encoding to UTF8 in the default logconfig (#4138) 2018-11-03 02:28:07 +11:00
Erik Johnston
90d713b8c6
Merge pull request #4137 from matrix-org/erikj/clean_up_events 
Clean up event accesses and tests
 2018-11-02 14:12:49 +00:00
Erik Johnston
76cd7de108 Newsfile 2018-11-02 13:45:56 +00:00
Amber Brown
cb7a6b2379
Fix typing being reset causing infinite syncs (#4127) 2018-11-03 00:19:23 +11:00
Richard van der Hoff
efb9343c8c
Merge pull request #4132 from matrix-org/rav/fix_device_list_locking 
Fix locked upsert on device_lists_remote_cache
 2018-11-02 10:50:53 +00:00
Richard van der Hoff
00f12e00f8
Merge pull request #4133 from matrix-org/travis/fix-terms-auth 
Fix logic error that prevented guests from seeing the privacy policy
 2018-11-02 10:50:43 +00:00
Erik Johnston
b199534518
Merge pull request #4135 from matrix-org/erikj/fix_state_res_none 
Fix None exception in state res v2
 2018-11-02 10:45:57 +00:00
Richard van der Hoff
1cc6671ec4 changelog 2018-11-02 10:36:13 +00:00
Erik Johnston
f05d97e283 Newsfile 2018-11-02 10:32:06 +00:00
Travis Ralston
552f090f62 Changelog 2018-11-01 16:51:11 -06:00
Richard van der Hoff
3149d55b7d
Merge pull request #3778 from z3ntu/patch-1 
Fix build of Docker image with docker-compose
 2018-11-01 17:34:56 +00:00
Travis Ralston
c68aab1536
Merge pull request #4004 from matrix-org/travis/login-terms 
Add m.login.terms to the registration flow
 2018-11-01 11:03:38 -06:00
Erik Johnston
1b21e771d0
Merge pull request #4128 from matrix-org/erikj/state_res_v2_version 
Add STATE_V2_TEST room version
 2018-11-01 13:17:57 +00:00
Erik Johnston
62d683161e Newsfile 2018-11-01 11:44:44 +00:00
Amber Brown
073d400b84 Merge branch 'master' into develop 2018-11-01 21:32:12 +11:00
Travis Ralston
d1e7b9c44c Merge branch 'develop' into travis/login-terms 2018-10-31 13:15:14 -06:00
Richard van der Hoff
1729ba1650
Merge pull request #4101 from matrix-org/rav/aliases_for_upgrades 
Attempt to move room aliases on room upgrades
 2018-10-31 17:52:18 +00:00
Richard van der Hoff
94c7fadc98 Attempt to move room aliases on room upgrades 2018-10-31 15:43:57 +00:00
Richard van der Hoff
9b827c40ca
Log some bits about event creation (#4121) 
I found these helpful in debugging my room upgrade tests.
 2018-10-31 15:42:23 +00:00
Richard van der Hoff
60f128a401
Merge pull request #4124 from matrix-org/rav/fix_tox 
Attempt to fix tox installs
 2018-10-31 15:41:55 +00:00
Amber Brown
916efc8249
Remove fetching keys via the deprecated v1 kex method (#4120) 2018-10-31 23:14:39 +11:00
Amber Brown
f79f454485
Remove deprecated v1 key exchange endpoint (#4119) 2018-10-31 22:29:02 +11:00
Richard van der Hoff
a2d8bff0dc changelog 2018-10-30 21:21:05 +00:00
Amber Brown
e615e95590 changelog 2018-10-31 06:28:11 +11:00
Amber Brown
086e1a8f3e Fix search 500ing (#4122) 2018-10-31 06:26:46 +11:00
Amber Brown
3bade14ec0
Fix search 500ing (#4122) 2018-10-31 04:33:41 +11:00
Amber Brown
2e223a8c22
Remove the unused /pull federation API (#4118) 2018-10-31 04:24:59 +11:00
Erik Johnston
0794504bce
Merge pull request #4006 from matrix-org/erikj/purge_state_groups 
Delete unreferenced state groups during purge
 2018-10-30 16:58:22 +00:00
Amber Brown
0dce9e1379
Write some tests for the email pusher (#4095) 2018-10-30 23:55:43 +11:00
David Baker
4eacf0f200 news fragment 2018-10-30 10:05:51 +00:00
Erik Johnston
169851b412
Merge pull request #4109 from matrix-org/erikj/repl_devices 
A couple of replication fixes for device lists
 2018-10-29 18:16:48 +00:00
Erik Johnston
4f0fa7a120 Newsfile 2018-10-29 18:15:42 +00:00
Erik Johnston
39f419868f Newsfile 2018-10-29 17:38:09 +00:00
Amber Brown
4cd1c9f2ff
Delete the disused & unspecced identicon functionality (#4106) 2018-10-29 23:57:24 +11:00
Richard van der Hoff
7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar 
Remember to copy the avatar on room upgrades
 2018-10-29 12:49:21 +00:00
Richard van der Hoff
56ca578f77
Merge pull request #4099 from matrix-org/rav/upgrade_odd_pls 
Better handling of odd PLs during room upgrades
 2018-10-29 12:48:51 +00:00
Richard van der Hoff
bf33eed609
Merge pull request #4091 from matrix-org/rav/room_version_upgrades 
Room version upgrade support
 2018-10-29 12:47:20 +00:00
Amber Brown
c4b3698a80
Make the replication logger quieter (#4108) 2018-10-29 22:59:44 +11:00
Amber Brown
3b0a85fc8e changelog 2018-10-29 21:54:01 +11:00
Richard van der Hoff
db24d7f15e Better handling of odd PLs during room upgrades 
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
 2018-10-27 00:54:26 +01:00
Richard van der Hoff
5caf79b312 Remember to copy the avatar on room upgrades 2018-10-26 23:56:40 +01:00
Erik Johnston
03e634dad4 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-26 16:22:45 +01:00
Amber Brown
77d70a7646
Port register_new_matrix_user to Python 3 and add tests (#4085) 2018-10-26 22:05:22 +11:00
Richard van der Hoff
6cb2e2448a
Merge pull request #4089 from dekonnection/master 
Make Docker image listen on ipv6 as well as ipv4
 2018-10-25 19:23:37 +01:00
Richard van der Hoff
68c0ce62d8 changelog 2018-10-25 19:18:25 +01:00
Cédric Laudrel
379376e5e6 Make Docker image listening on ipv6 as well as ipv4 
Signed-off-by: Cédric Laudrel <dek@iono.me>
 2018-10-25 20:03:47 +02:00
Erik Johnston
cb53ce9d64
Refactor state group lookup to reduce DB hits (#4011) 
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
 2018-10-25 17:49:55 +01:00
Richard van der Hoff
e5da60d75d Merge remote-tracking branch 'origin/master' into develop 2018-10-25 17:08:09 +01:00
Erik Johnston
c85e063302
Merge pull request #4051 from matrix-org/erikj/alias_disallow_list 
Add config option to control alias creation
 2018-10-25 17:04:59 +01:00
Neil Johnson
95ad128851
Merge pull request #4081 from matrix-org/neilj/fix_mau_init 
fix race condiftion in calling initialise_reserved_users
 2018-10-25 16:33:40 +01:00
Erik Johnston
b94a43d5b5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list 2018-10-25 15:25:31 +01:00
Neil Johnson
f7f487e14c Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin 2018-10-25 14:40:06 +01:00
Travis Ralston
54def42c19 Merge branch 'develop' into travis/login-terms 2018-10-24 13:22:59 -06:00
Neil Johnson
ea69a84bbb fix style inconsistencies 2018-10-24 17:18:08 +01:00
Neil Johnson
07126e43a4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_init 2018-10-24 16:25:39 +01:00
Richard van der Hoff
9f72c209ee
Update changelog.d/3975.feature 
Co-Authored-By: neilisfragile <neil@matrix.org>
 2018-10-24 14:37:36 +01:00
Richard van der Hoff
3ad359e5be Merge remote-tracking branch 'origin/develop' into rav/fix_event_filter_validation 2018-10-24 11:23:49 +01:00
Richard van der Hoff
7328039117
Merge pull request #4082 from matrix-org/rav/fix_pep8 
Fix a number of flake8 errors
 2018-10-24 11:23:35 +01:00
Erik Johnston
3904cbf307
Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase 
Add v2 state resolution algorithm
 2018-10-24 11:12:12 +01:00
Richard van der Hoff
7e07d25ed6 Allow backslashes in event field filters 
Fixes a bug introduced in https://github.com/matrix-org/synapse/pull/1783 which
meant that single backslashes were not allowed in event field filters.

The intention here is to allow single-backslashes, but disallow
double-backslashes.
 2018-10-24 11:11:24 +01:00
Richard van der Hoff
ef771cc4c2 Fix a number of flake8 errors 
Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
 2018-10-24 10:39:03 +01:00
Richard van der Hoff
e0b9d5f0af
Merge pull request #4075 from matrix-org/rav/fix_pusher_logcontexts 
Clean up the way logcontexts and threads work in the pushers
 2018-10-24 09:53:57 +01:00
Erik Johnston
cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation 
Clean up room alias creation
 2018-10-24 09:40:27 +01:00
Neil Johnson
a67d8ace9b remove errant exception and style 2018-10-23 17:44:39 +01:00
Travis Ralston
43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests 
Handle HttpResponseException more safely for federated groups
 2018-10-23 10:41:04 -06:00
Neil Johnson
329d18b39c remove white space 2018-10-23 15:27:20 +01:00
Neil Johnson
6105c6101f fix race condiftion in calling initialise_reserved_users 2018-10-23 15:24:58 +01:00
Richard van der Hoff
b3f6dddad2
Give some more things logcontexts (#4077) 2018-10-23 14:29:17 +01:00
Richard van der Hoff
5c445114d3
Correctly account for cpu usage by background threads (#4074) 
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.

While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.

Fixes #4064
 2018-10-23 13:12:32 +01:00
Richard van der Hoff
1fe6bbb555
Merge pull request #3698 from spantaleev/add-matrix-docker-ansible-deploy 
Add information about the matrix-docker-ansible-deploy playbook
 2018-10-23 10:11:58 +01:00
Richard van der Hoff
abd9914683 Changelog 2018-10-22 16:12:11 +01:00
Richard van der Hoff
7aea00069c
Merge pull request #4076 from matrix-org/rav/fix_init_logcontexts 
Run MAU queries as background processes
 2018-10-22 14:46:59 +01:00
Richard van der Hoff
911db96658
Merge pull request #4073 from matrix-org/rav/require_psutil 
Make psutil an explicit dependency
 2018-10-22 12:33:21 +01:00
Richard van der Hoff
a6f421e812 Run MAU queries as background processes 
Fixes #3820
 2018-10-20 02:14:35 +01:00
Amber Brown
e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) (#4068) 2018-10-20 11:16:55 +11:00
steamport
b85fe45f46
Add CL 2018-10-19 21:55:38 +00:00
Richard van der Hoff
e5b52d0f94 Make psutil an explicit dependency 
As of #4027, we require psutil to be installed, so it should be in our
dependency list. We can also remove some of the conditional import code
introduced by #992.

Fixes #4062.
 2018-10-19 21:51:15 +01:00
Richard van der Hoff
f62c597d14
Merge pull request #4069 from matrix-org/rav/fix_email_templates_4065 
Calculate absolute path for email templates
 2018-10-19 16:33:54 +01:00
Erik Johnston
056f099126 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-19 15:48:59 +01:00
Richard van der Hoff
cc325c7069 Calculate absolute path for email templates 2018-10-19 14:01:59 +01:00
Amber Brown
e404ba9aac
Fix manhole on py3 (pt 2) (#4067) 2018-10-19 22:26:00 +11:00
Amber Brown
b69216f768
Make the metrics less racy (#4061) 2018-10-19 21:45:45 +11:00
Erik Johnston
f9d6c677ea Newsfile 2018-10-19 10:22:45 +01:00
Erik Johnston
0d31109ed5 Newsfile 2018-10-19 10:14:29 +01:00
Amber Brown
1d17fc52ae changelog 2018-10-19 09:27:10 +11:00
Richard van der Hoff
c69026a758 Use the right python when starting workers 
We should use the same python to start the workers as we do for the main
synapse (ie, the same one used to run synctl.)
 2018-10-18 21:06:30 +01:00
Travis Ralston
49a044aa5f Merge branch 'develop' into travis/login-terms 2018-10-18 09:57:58 -06:00
Richard van der Hoff
03287c350e remove redundant changelog file 
this change has been released
 2018-10-18 15:09:34 +01:00
Richard van der Hoff
c632bc8654 Merge branch 'master' into develop 2018-10-18 15:07:03 +01:00
Erik Johnston
e77f24d80a
Merge pull request #4049 from matrix-org/erikj/synctl_colour 
Only colourise synctl output when attached to tty
 2018-10-18 10:51:14 +01:00
Richard van der Hoff
52e3d3813b prep changelog 2018-10-17 17:40:01 +01:00
Richard van der Hoff
0fd2321629 Fix incorrect truncation in get_missing_events 
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
 2018-10-17 17:35:26 +01:00
Richard van der Hoff
1519572961 Ship the email templates as package_data 
move the example email templates into the synapse package so that they can be
used as package data, which should mean that all of the packaging mechanisms
(pip, docker, debian, arch, etc) should now come with the example templates.

In order to grandfather in people who relied on the templates being in the old
place, check for that situation and fall back to using the defaults if the
templates directory does not exist.
 2018-10-17 16:46:02 +01:00
Erik Johnston
3a5d8d5891 Newsfile 2018-10-17 16:12:45 +01:00
Erik Johnston
1af16acd4c Newsfile 2018-10-17 13:48:09 +01:00
Will Hunt
d6a7797dd1 Fix roomlist since tokens on Python 3 (#4046) 
Thanks @Half-Shot !!!
 2018-10-17 23:04:55 +11:00
Richard van der Hoff
6ec9d8ba0a
Merge pull request #4045 from matrix-org/rav/fix_get_missing_events 
Fix incorrect truncation in get_missing_events
 2018-10-17 11:40:03 +01:00
Neil Johnson
c6584f4b5f clean up config error logic and imports 2018-10-17 11:36:41 +01:00
Richard van der Hoff
fc0f13dd03 Fix incorrect truncation in get_missing_events 
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
 2018-10-16 21:10:04 +01:00
Richard van der Hoff
017eb9d17a changelog 2018-10-16 16:31:47 +01:00
Erik Johnston
fc954960e9 Newsfile 2018-10-16 16:28:42 +01:00
Richard van der Hoff
b8a5b0097c
Various cleanups in the federation client code (#4031) 
- Improve logging: log things in the right order, include destination and txids
  in all log lines, don't log successful responses twice

- Fix the docstring on TransportLayerClient.send_transaction

- Don't use treq.request, which is overcomplicated for our purposes: just use a
  twisted.web.client.Agent.

- simplify the logic for setting up the bodyProducer

- fix bytes/str confusions
 2018-10-16 10:44:49 +01:00
Travis Ralston
f293d124b6 Merge branch 'develop' into travis/login-terms 2018-10-15 14:44:32 -06:00
Amber Brown
24bc15eab4 update changelog 2018-10-15 22:23:13 +11:00
Amber Brown
4e50fe3edb update changelog 2018-10-15 22:22:49 +11:00
David Baker
03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups 
E2E backups
 2018-10-15 10:19:43 +01:00
Slavi Pantaleev
c187638ee9 Add information about the matrix-docker-ansible-deploy playbook 
Signed-off-by: Slavi Pantaleev <slavi@devture.com>
 2018-10-14 21:50:18 +03:00
Ivan Shapovalov
06bc8d2fe5 synapse/app: frontend_proxy.py: actually make workers work on py3 2018-10-14 20:08:39 +03:00
Ivan Shapovalov
fb216a22db synapse/visibility.py: fix SyntaxError on py3.7 2018-10-14 20:08:17 +03:00
Travis Ralston
7ede650956 Merge branch 'develop' into travis/login-terms 2018-10-12 16:24:07 -06:00
Travis Ralston
e3586f7c06 Merge branch 'develop' into travis/fix-federated-group-requests 2018-10-12 14:49:58 -06:00
Neil Johnson
a2bfb778c8 improve auto room join logic, comments and tests 2018-10-12 18:17:36 +01:00
Amber Brown
381d2cfdf0
Make workers work on Py3 (#4027) 2018-10-13 00:14:08 +11:00
Erik Johnston
49840f5ab2 Update newsfile 2018-10-10 11:31:02 +01:00
Erik Johnston
20733857ab Newsfile 2018-10-09 14:18:16 +01:00
David Baker
d34657e1f2 Add changelog 2018-10-09 11:15:54 +01:00
Erik Johnston
9eb1a79100
Merge pull request #4008 from matrix-org/erikj/log_looping_exceptions 
Log looping call exceptions
 2018-10-09 10:12:10 +01:00
Richard van der Hoff
495975e231 Optimisation for filter_events_for_server 
We're better off hashing just the event_id than the whole ((type, state_key),
event_id) tuple - so use a dict instead of a set.

Also, iteritems > items.
 2018-10-08 13:46:52 +01:00
Erik Johnston
8164f6daf3 Newsfile 2018-10-05 11:25:09 +01:00
Neil Johnson
ed82043efb
Merge branch 'develop' into matthew/autocreate_autojoin 2018-10-04 17:26:59 +01:00
Richard van der Hoff
c6dbd216e6
Merge pull request #3995 from matrix-org/rav/no_deextrem_outliers 
Fix bug in forward_extremity update logic
 2018-10-04 16:27:05 +01:00
Erik Johnston
d9f3db5081 Newsfile 2018-10-04 16:03:08 +01:00
Amber Brown
d86794325f Merge branch 'master' into develop 2018-10-04 22:41:52 +10:00
Amber Brown
92faeb2a3f changelog 2018-10-04 22:38:10 +10:00
Travis Ralston
158d6c75b6 Changelog 2018-10-03 17:54:08 -06:00
Richard van der Hoff
a59d899668 Pin to prometheus_client<0.4 to avoid renaming all of our metrics 2018-10-03 17:20:15 +01:00
Erik Johnston
8935ec5a93
Merge pull request #3999 from matrix-org/erikj/fix_3pid_invite_rejetion 
Fix handling of rejected threepid invites
 2018-10-03 14:47:41 +01:00
Erik Johnston
81e2813948
Merge pull request #3996 from matrix-org/erikj/fix_bg_iteration 
Fix exception in background metrics collection
 2018-10-03 14:14:38 +01:00
Erik Johnston
93a8603904 Newsfile 2018-10-03 11:59:05 +01:00
Erik Johnston
6e0c66f651 Newsfile 2018-10-03 11:40:02 +01:00
Erik Johnston
c69faf8c4a Newsfile 2018-10-03 11:29:44 +01:00
Richard van der Hoff
2a4ea3baa8 fix newsfile name 2018-10-03 07:09:25 +01:00
Richard van der Hoff
ae61ade891 Fix bug in forward_extremity update logic 
An event does not stop being a forward_extremity just because an outlier or
rejected event refers to it.
 2018-10-02 23:36:08 +01:00
Amber Brown
ccce6f26a6 changelog 2018-10-03 02:16:21 +10:00
Erik Johnston
25baf3b2ac
Merge pull request #3991 from matrix-org/erikj/fix_pop_retry_cache 
Fix bug when invalidating destination retry timings
 2018-10-02 16:42:28 +01:00
Amber Brown
058a4c665e
Remove Jenkins & other old dev junk (#3988) 2018-10-03 00:59:11 +10:00
Erik Johnston
8f614f842d Newsfile 2018-10-02 15:54:31 +01:00
Richard van der Hoff
2b8d28b095
Merge pull request #3960 from matrix-org/rav/fix_missing_create_event_error 
Fix error handling for missing auth_event
 2018-10-02 13:57:54 +01:00
Amber Brown
7232917f12
Disable frozen dicts by default (#3987) 2018-10-02 22:53:47 +10:00
Erik Johnston
eeb755d17b Newsfile 2018-10-02 12:06:25 +01:00
Richard van der Hoff
8c41b0ca66
Merge pull request #3989 from matrix-org/rav/better_stacktraces 
Avoid reraise, to improve stacktraces
 2018-10-02 10:47:38 +01:00
Erik Johnston
bc29946809
Merge pull request #3986 from matrix-org/erikj/fix_sync_with_redacted_state 
Fix lazy loaded sync with rejected state events
 2018-10-02 11:38:35 +02:00
Richard van der Hoff
8174c6725b Avoid reraise, to improve stacktraces 2018-10-01 18:50:34 +01:00
Richard van der Hoff
5908a8f6f5 Merge branch 'develop' into rav/fix_missing_create_event_error 2018-10-01 17:16:20 +01:00
Amber Brown
abdc141c2b
Update instructions to point to pip install (#3985) 2018-10-02 01:08:38 +10:00
Richard van der Hoff
b5b93f45d5
Merge pull request #3968 from matrix-org/rav/fix_federation_errors 
Fix exceptions when handling incoming transactions
 2018-10-01 15:54:24 +01:00
Amber Brown
6e05fd032c
Fix userconsent on Python 3 (#3938) 2018-10-02 00:11:58 +10:00
Erik Johnston
5fa27eac78 Newsfile 2018-10-01 14:24:41 +01:00
Erik Johnston
8f5c23d0cd
Merge pull request #3933 from matrix-org/erikj/destination_retry_cache 
Add a five minute cache to get_destination_retry_timings
 2018-10-01 14:20:30 +02:00
Richard van der Hoff
53c5fa4e6c
Further reduce the size of the docker image (#3972) 
Rewrite the dockerfile as a multistage build: this means we can get rid of a whole load of cruft which we don't need.
 2018-10-01 12:29:17 +01:00
Matthew Hodgson
faa462ef79 changelog 2018-09-29 02:21:01 +01:00
Bruno Windels
9a8bbc9a59 add --no-admin flag to registration script (#3836) 2018-09-28 13:36:56 +01:00
Richard van der Hoff
3deaad2fb4
Merge pull request #3964 from matrix-org/rav/remove_localhost_checks 
remove spurious federation checks on localhost
 2018-09-28 13:35:47 +01:00
Richard van der Hoff
26557c9db4
Merge pull request #3980 from matrix-org/rav/remove_broken_cache_call 
Remove redundant call to start_get_pdu_cache
 2018-09-28 13:13:19 +01:00
Jan Christian Grünhage
5304449738 build python 3 docker images on circle CI (#3976) 2018-09-28 12:36:35 +01:00
Richard van der Hoff
19475cf337 Remove redundant call to start_get_pdu_cache 
I think this got forgotten in #3932. We were getting away with it because it
was the last call in this function.
 2018-09-28 12:01:23 +01:00
Richard van der Hoff
9c8cec5dab Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache 2018-09-28 10:51:09 +01:00
Richard van der Hoff
f094f715cf Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errors 2018-09-27 15:18:21 +01:00
Richard van der Hoff
36c62a67c4
Merge pull request #3794 from matrix-org/erikj/faster_typing 
Improve performance of getting typing updates for replication
 2018-09-27 15:14:26 +01:00
Richard van der Hoff
b5c976347b
Merge pull request #3946 from matrix-org/michaelkaye/automate_docker_hub_upload 
Build and push docker image to hub automatically
 2018-09-27 15:07:41 +01:00
Richard van der Hoff
e1e3e77bfd
Merge pull request #3967 from matrix-org/rav/federation_handler_cleanups 
Clarifications in FederationHandler
 2018-09-27 15:06:59 +01:00
Amber Brown
6de3884e5e
Merge pull request #3965 from matrix-org/rav/notify_app_services_bg_process 
Run notify_app_services as a bg process
 2018-09-27 23:40:30 +10:00
Amber Brown
a512e637ac
Merge pull request #3970 from schnuffle/develop-py3 
Replaced all occurences of e.message with str(e)
 2018-09-27 23:39:45 +10:00
Amber Brown
b3064532d0
Run our oldest supported configuration in CI (#3952) 2018-09-27 23:21:54 +10:00
Amber Brown
861c063ebc
Update 3970.bugfix 2018-09-27 22:47:01 +10:00
Schnuffle
a873896096 Added changelog fragment 2018-09-27 14:01:44 +02:00
Richard van der Hoff
51d33d5178
Merge pull request #3961 from matrix-org/neilj/lock_mau_upserts 
fix #3854 MAU transaction errors
 2018-09-27 12:31:37 +01:00
Michael Kaye
74bbdd0412 Do the changelog.d dance 2018-09-27 12:01:43 +01:00
Richard van der Hoff
948a6d8776 changelog 2018-09-27 11:37:39 +01:00
Richard van der Hoff
ad8e137062 changelog 2018-09-27 11:31:51 +01:00
Richard van der Hoff
92abd3d6d6
Merge pull request #3966 from matrix-org/rav/rx_txn_logging_2 
Logging improvements
 2018-09-27 11:27:46 +01:00
Richard van der Hoff
4a15a3e4d5
Include eventid in log lines when processing incoming federation transactions (#3959) 
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.

This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
 2018-09-27 11:25:34 +01:00
Richard van der Hoff
ae6ad4cf41
docstrings and unittests for storage.state (#3958) 
I spent ages trying to figure out how I was going mad...
 2018-09-27 11:22:25 +01:00
Travis Ralston
2a7b3439de Changelog 2018-09-26 13:51:34 -06:00
Richard van der Hoff
e70b4ce069 Logging improvements 
Some logging tweaks to help with debugging incoming federation transactions
 2018-09-26 17:36:14 +01:00
Richard van der Hoff
3c37c7e45b changelog 2018-09-26 17:01:30 +01:00
Richard van der Hoff
0c4a99ea2d changelog 2018-09-26 16:54:54 +01:00
Richard van der Hoff
ab59f3d8da changelog 2018-09-26 16:52:24 +01:00
Neil Johnson
d514608b5c towncrier 2018-09-26 16:19:53 +01:00
Amber Brown
d4e0861ff9
Reduce the load on our CI (#3957) 
* changelog

* reduce circleci config

* plus a handy script

* fix regex
 2018-09-27 00:23:21 +10:00
Richard van der Hoff
a5e70b31a1 changelog 2018-09-26 14:41:14 +01:00
Richard van der Hoff
4e8276a34a
Merge pull request #3956 from matrix-org/rav/fix_expiring_cache_len 
Fix ExpiringCache.__len__ to be accurate
 2018-09-26 13:10:13 +01:00
Richard van der Hoff
5b4028fa78 Merge branch 'rav/fix_expiring_cache_len' into erikj/destination_retry_cache 2018-09-26 12:55:53 +01:00