tileradotorg/synapse
1
0
Fork 0
forked from MirrorHub/synapse
Code Pull requests Activity
13742 commits 4 branches 629 tags 119 MiB
Commit graph

1131 commits

Author SHA1 Message Date
Erik Johnston
4eb8408ed2 Newsfile 2018-08-23 10:46:13 +01:00
Erik Johnston
fd2dbf1836 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-22 17:06:10 +01:00
Richard van der Hoff
c7181dcc6c Merge branch 'master' into develop 2018-08-22 14:37:11 +01:00
Erik Johnston
db10f553ba
Merge pull request #3724 from Half-Shot/hs/guest-fetch-event 
Allow guests to use /rooms/:roomId/event/:eventId
 2018-08-22 13:41:08 +01:00
Erik Johnston
764030cf63
Merge pull request #3659 from matrix-org/erikj/split_profiles 
Allow profile updates to happen on workers
 2018-08-22 11:35:55 +01:00
Erik Johnston
3bf8bab8f9
Merge pull request #3673 from matrix-org/erikj/refactor_state_handler 
Refactor state module to support multiple room versions
 2018-08-22 10:04:55 +01:00
Richard van der Hoff
a4cf660a32
Merge pull request #3735 from matrix-org/travis/federation-spelling 
limt -> limit
 2018-08-22 09:34:21 +01:00
Richard van der Hoff
0d568ff403 Merge remote-tracking branch 'origin/release-v0.33.3' into develop 2018-08-22 09:15:44 +01:00
Travis Ralston
365f588d98
Create 3734.misc 2018-08-21 23:38:38 -06:00
Travis Ralston
eb7be75a10
Create 3735.misc 2018-08-21 23:38:06 -06:00
Matthew Hodgson
bb81e78ec6
Split the state_group_cache in two (#3726) 
Splits the state_group_cache in two.

One half contains normal state events; the other contains member events.

The idea is that the lazyloading common case of: "I want a subset of member events plus all of the other state" can be accomplished efficiently by splitting the cache into two, and asking for "all events" from the non-members cache, and "just these keys" from the members cache.  This means we can avoid having to make DictionaryCache aware of these sort of complicated queries, whilst letting LL requests benefit from the caching.

Previously we were unable to sensibly use the caching and had to pull all state from the DB irrespective of the filtering, which made things slow.  Hopefully fixes https://github.com/matrix-org/synapse/issues/3720.
 2018-08-22 00:56:37 +02:00
Richard van der Hoff
afb4b490a4 changelog 2018-08-21 23:19:14 +01:00
Richard van der Hoff
a52f276990 Bugfixes 
--------
 
 - Fix bug in v0.33.3rc1 which caused infinite loops and OOMs
 ([\#3723](https://github.com/matrix-org/synapse/issues/3723))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEIQBQJ4l+yK4dlKkFIwi0edOSShEFAlt6/yMACgkQIwi0edOS
 ShFrzAgAhLA74fClydGLfhPNCXIw12Amv2ASA6+r0ukqjYQ/9MZ0CRsL+PMZJTk6
 Z75PK3TkX38zSpPaUx9HdMQO/yE+rpI99ZxlBM+8jzsf1jyF8ljqxSfixXUlaXX9
 698PQQ3f+IDoSs59NKrHvoWhUThaBpf/1GwRxG5mUcm2oKJItNMHk/uLurNCaIIZ
 YSwWcxvZdluOMX0WrlF/S524DqtgULcRf3E9x3mumobkVGxI3UH73hXgGUcPi7XV
 ttLuy7fVbV2Cun/wvjBqD7kc8LNbKwSaT9y+/xS0D5QYjph4rpQiTG7S5rtPLPMU
 Z2lL1C5ggBCkD43g3orDvehbNWCL5g==
 =0vEe
 -----END PGP SIGNATURE-----

Merge tag 'v0.33.3rc2' into develop

Bugfixes
--------

- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs
([\#3723](https://github.com/matrix-org/synapse/issues/3723))
 2018-08-21 20:30:09 +01:00
Erik Johnston
46c832eaac
Merge pull request #3727 from matrix-org/erikj/dont_error_on_missing_keys 
Don't log exceptions when failing to fetch server keys
 2018-08-21 17:07:20 +01:00
Erik Johnston
79d3b4689e Newsfile 2018-08-21 11:21:48 +01:00
Amber Brown
3b5b64ac99 changelog 2018-08-21 03:48:55 +10:00
Will Hunt
8bd585b09b 3724.feature 2018-08-20 18:27:42 +01:00
Richard van der Hoff
012d612f9d changelog 2018-08-20 18:26:27 +01:00
Erik Johnston
e2c0aa2c26 Newsfile 2018-08-20 17:40:59 +01:00
Amber Brown
80bf7d3580 changelog 2018-08-21 00:01:14 +10:00
Amber Brown
324525f40c
Port over enough to get some sytests running on Python 3 (#3668) 2018-08-20 23:54:49 +10:00
Erik Johnston
4d664278af Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_state_handler 2018-08-20 14:49:43 +01:00
Erik Johnston
48a910e128 Newsfile 2018-08-20 13:33:20 +01:00
Erik Johnston
2aa7cc6a46
Merge pull request #3713 from matrix-org/erikj/fixup_fed_logging 
Fix logging bug in EDU handling over replication
 2018-08-20 10:51:45 +01:00
Richard van der Hoff
3cef867cc1
Merge pull request #3709 from matrix-org/rav/logcontext_for_replication_commands 
Logcontexts for replication command handlers
 2018-08-17 16:22:07 +01:00
Richard van der Hoff
c144252a8c
Merge pull request #3710 from matrix-org/rav/logcontext_for_pusher_updates 
Fix logcontexts for running pushers
 2018-08-17 16:21:49 +01:00
Amber Brown
c334ca67bb
Integrate presence from hotfixes (#3694) 2018-08-18 01:08:45 +10:00
Amber Brown
04f5d2db62
Remove v1/register's broken shared secret functionality (#3703) 2018-08-18 00:55:01 +10:00
Richard van der Hoff
63260397c6
Merge pull request #3701 from matrix-org/rav/use_producer_for_responses 
Use a producer to stream back responses
 2018-08-17 14:58:45 +01:00
Richard van der Hoff
3f8709ffe4
Merge pull request #3700 from matrix-org/rav/wait_for_producers 
Refactor request logging code
 2018-08-17 14:57:45 +01:00
Neil Johnson
3ee57bdcbb Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-17 14:34:10 +01:00
Erik Johnston
782689bd40 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles 2018-08-17 14:15:48 +01:00
Neil Johnson
b5f638f1f4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-17 14:04:15 +01:00
Neil Johnson
69c49d3fa3
Merge branch 'develop' into neilj/limit_exceeded_error 2018-08-17 12:44:26 +00:00
Neil Johnson
a2d872e7b3 Merge pull request #3708 from matrix-org/neilj/resource_Limit_block_event_creation 
Neilj/resource limit block event creation
 2018-08-17 12:42:59 +00:00
Erik Johnston
73737bd0f9 Newsfile 2018-08-17 11:14:05 +01:00
Travis Ralston
b99a0f3941
Create 3712.misc 2018-08-17 02:47:31 -06:00
Richard van der Hoff
4c9da1440f changelog 2018-08-17 00:50:36 +01:00
Richard van der Hoff
d9efd87d55 changelog 2018-08-17 00:49:22 +01:00
Neil Johnson
7e51342196 For resource limit blocked users, prevent writing into rooms 2018-08-16 23:05:20 +01:00
Neil Johnson
7edd11623d add new error type ResourceLimit 2018-08-16 21:04:30 +01:00
Erik Johnston
aae86a81ef Newsfile 2018-08-16 15:41:48 +01:00
Neil Johnson
bfb6c58624 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-16 15:10:16 +01:00
Will Hunt
c151b32b1d Add GET media/v1/config (#3184) 2018-08-16 14:23:38 +01:00
Matthew Hodgson
762a758fea lazyload aware /messages (#3589) 2018-08-16 14:22:47 +01:00
Neil Johnson
25d2b5d55f Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-16 11:37:17 +01:00
Matthew Hodgson
3f543dc021 initial cut at a room summary API (#3574) 2018-08-16 09:46:50 +01:00
Neil Johnson
8cfad2e686 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-15 17:20:38 +01:00
Neil Johnson
81d727efa9
Merge pull request #3689 from matrix-org/neilj/fix_off_by_1+maus 
Fix Mau off by one errors
 2018-08-15 16:19:41 +00:00
Neil Johnson
da7fe43899 Fix mau blocking calulation bug on login 2018-08-15 17:03:30 +01:00
Richard van der Hoff
e2e846628d Remove incorrectly re-added changelog files 
These were removed for the 0.33.2 release, but were incorrectly re-added in
 2018-08-15 16:38:46 +01:00
Matthew Hodgson
2f78f432c4 speed up /members and add at= and membership params (#3568) 2018-08-15 16:35:22 +01:00
Neil Johnson
fc5d937550 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking 2018-08-15 16:31:40 +01:00
Richard van der Hoff
d82fa0e9a6 changelog 2018-08-15 15:12:35 +01:00
Erik Johnston
dc56c47dc0
Merge pull request #3653 from matrix-org/erikj/split_federation 
Move more federation APIs to workers
 2018-08-15 14:59:02 +01:00
Neil Johnson
4601129c44
Merge pull request #3687 from matrix-org/neilj/admin_email 
support admin_email config and pass through into blocking errors,
 2018-08-15 13:52:25 +00:00
Erik Johnston
ef184caf30 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation 2018-08-15 14:25:46 +01:00
Neil Johnson
39176f27f7 remove changelog referencing another PR 2018-08-15 13:53:51 +01:00
Richard van der Hoff
2de813a611 changelog 2018-08-15 13:49:03 +01:00
Neil Johnson
c4eb97518f Merge branch 'neilj/update_limits_error_codes' of github.com:matrix-org/synapse into neilj/admin_email 2018-08-15 11:53:01 +01:00
Neil Johnson
55afba0fc5 update admin email to uri 2018-08-15 11:41:18 +01:00
Neil Johnson
1c5e690a6b
Merge pull request #3690 from matrix-org/neilj/change_prometheus_mau_metric_name 
combine mau metrics into one group
 2018-08-15 09:53:59 +00:00
Erik Johnston
fef2e65d12
Merge pull request #3667 from matrix-org/erikj/fixup_unbind 
Don't fail requests to unbind 3pids for non supporting ID servers
 2018-08-15 10:32:12 +01:00
Neil Johnson
596ce63576 update resource limit error codes 2018-08-15 10:23:28 +01:00
Neil Johnson
19b433e3f4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_email 2018-08-14 17:44:46 +01:00
Neil Johnson
70e48cbbb1 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/change_prometheus_mau_metric_name 2018-08-14 17:40:59 +01:00
Neil Johnson
cd0c749c4f Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users 2018-08-14 17:06:51 +01:00
Neil Johnson
629f390035 Rename MAU prometheus metric 2018-08-14 16:38:05 +01:00
Neil Johnson
414d54b61a
Merge pull request #3670 from matrix-org/neilj/mau_sync_block 
Block ability to read via sync if mau limit exceeded
 2018-08-14 15:21:31 +00:00
Amber Brown
614e6d517d
Dockerised sytest (#3660) 2018-08-14 21:30:34 +10:00
Amber Brown
591bf87c6a Merge remote-tracking branch 'origin/develop' into neilj/fix_reap_users_in_postgres 2018-08-14 20:56:23 +10:00
Amber Brown
bdfbd934d6
Implement a new test baseclass to cut down on boilerplate (#3684) 2018-08-14 20:53:43 +10:00
Jan Christian Grünhage
8f0c430ca4
Merge pull request #3669 from matrix-org/jcgruenhage/update-docker-base 
update docker base-image to alpine 3.8
 2018-08-14 11:43:59 +02:00
Neil Johnson
1f24d8681b set admin email via config 2018-08-13 21:26:43 +01:00
Richard van der Hoff
8ba2dac6ea Name changelog after PR, not bug 2018-08-13 18:54:01 +01:00
Richard van der Hoff
50bcaf1a27 Merge pull request #3677 from andrewshadura/master 
Use recaptcha_ajax.js directly from Google
 2018-08-13 18:52:56 +01:00
Richard van der Hoff
34f51babc5
Revert "Use recaptcha_ajax.js directly from Google" 2018-08-13 18:48:03 +01:00
Amber Brown
99dd975dae
Run tests under PostgreSQL (#3423) 2018-08-13 16:47:46 +10:00
Neil Johnson
ac205a54b2 Fixes test_reap_monthly_active_users so it passes under postgres 2018-08-11 22:50:09 +01:00
Neil Johnson
c79c4c0a7d server notices for resource limit blocking 2018-08-10 15:25:47 +01:00
Amber Brown
a001038b92
Merge pull request #3679 from matrix-org/hawkowl/blackify-tests 
Blackify the tests
 2018-08-11 00:12:56 +10:00
Richard van der Hoff
c31793a784 Merge branch 'rav/fix_linearizer_cancellation' into develop 2018-08-10 14:57:27 +01:00
Amber Brown
dd16e7dfcc changelog 2018-08-10 23:54:46 +10:00
Amber Brown
b37c472419
Rename async to async_helpers because async is a keyword on Python 3.7 (#3678) 2018-08-10 23:50:21 +10:00
Andrej Shadura
c75b71a397
Use recaptcha_ajax.js directly from Google 
The script recaptcha_ajax.js contains minified non-free code which
we probably cannot redistribute. Since it talks to Google servers
anyway, it is better to just download it from Google directly instead
of shipping it.

This fixes #1932.
 2018-08-10 13:36:14 +02:00
Richard van der Hoff
3c0213a217
Merge pull request #3439 from vojeroen/send_sni_for_federation_requests 
send SNI for federation requests
 2018-08-10 12:23:54 +01:00
Richard van der Hoff
178ab76ac0 changelog 2018-08-10 11:05:23 +01:00
Erik Johnston
5075e444f4 Newsfile 2018-08-09 14:58:49 +01:00
Neil Johnson
c6b28fb479 Where server is disabled, block ability for locked out users to read new messages 2018-08-09 12:45:58 +01:00
Jan Christian Grünhage
d967653705 update docker base-image to alpine 3.8 2018-08-09 13:31:10 +02:00
Neil Johnson
0ad98e38d0
Merge pull request #3655 from matrix-org/neilj/disable_hs 
Flag to disable HS without disabling federation
 2018-08-09 10:41:43 +00:00
Erik Johnston
54a9bea88c Newsfile 2018-08-09 10:39:29 +01:00
Erik Johnston
8876ce7f77 Newsfile 2018-08-09 10:37:42 +01:00
Amber Brown
984376745b Merge branch 'master' into develop 2018-08-09 19:23:47 +10:00
Erik Johnston
2bdafaf3c1
Merge pull request #3632 from matrix-org/erikj/refactor_repl_servlet 
Add helper base class for generating new replication endpoints
 2018-08-09 10:06:23 +01:00
Erik Johnston
62564797f5 Fixup wording and remove dead code 2018-08-09 09:56:10 +01:00
Amber Brown
2511f3f8a0
Test fixes for Python 3 (#3647) 2018-08-09 12:22:01 +10:00
Richard van der Hoff
bb89c84614
Merge pull request #3664 from matrix-org/rav/federation_metrics 
more metrics for the federation and appservice senders
 2018-08-08 23:16:58 +01:00
Neil Johnson
5298d79fb5
Merge branch 'develop' into neilj/disable_hs 2018-08-08 16:13:03 +00:00
Richard van der Hoff
8521ae13e3
Merge pull request #3654 from matrix-org/rav/room_versions 
Support for room versioning
 2018-08-08 17:10:53 +01:00
Neil Johnson
d2f3ef98ac
Merge branch 'develop' into neilj/disable_hs 2018-08-08 15:55:47 +00:00
Neil Johnson
54685d294d Ability to disable client/server Synapse via conf toggle 2018-08-08 15:38:54 +01:00
Neil Johnson
be59910b93 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/reserved_users 2018-08-08 13:45:21 +01:00
Neil Johnson
990fe9fc23
Merge pull request #3633 from matrix-org/neilj/mau_tracker 
API for monthly_active_users table
 2018-08-08 12:44:15 +00:00
Neil Johnson
d92675a486 Ability to whitelist specific threepids against monthly active user limiting 2018-08-08 12:06:42 +01:00
Erik Johnston
360ba89c50 Don't fail requests to unbind 3pids for non supporting ID servers 
Older identity servers may not support the unbind 3pid request, so we
shouldn't fail the requests if we received one of 400/404/501. The
request still fails if we receive e.g. 500 responses, allowing clients
to retry requests on transient identity server errors that otherwise do
support the API.

Fixes #3661
 2018-08-08 12:06:18 +01:00
Richard van der Hoff
e6d73b8582 changelog 2018-08-07 22:14:35 +01:00
Amber Brown
cf8ef11f35 changelog 2018-08-07 21:11:24 +10:00
Richard van der Hoff
865d07cd38 changelog 2018-08-07 10:02:01 +01:00
Richard van der Hoff
7f3f108561 changelog 2018-08-06 16:30:52 +01:00
Will Hunt
16d9701892 Return M_NOT_FOUND when a profile could not be found. (#3596) 2018-08-03 19:08:05 +01:00
Neil Johnson
da90337d89 Add ability to limit number of monthly active users on the server 2018-08-03 14:05:20 +01:00
Michael Kaye
df7f9871c1
Merge pull request #3644 from matrix-org/michaelkaye/refactor_docker_locations_v2 
Refactor Dockerfile location
 2018-08-03 13:44:35 +01:00
Erik Johnston
cb298ff623 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_repl_servlet 2018-08-03 09:25:15 +01:00
Michael Kaye
70baa61e95 newsfragment 2018-08-02 18:58:43 +01:00
Michael Kaye
489949879e Add news entry 2018-08-02 18:49:50 +01:00
Richard van der Hoff
1fa98495d0
Merge pull request #3639 from matrix-org/rav/refactor_error_handling 
Clean up handling of errors from outbound requests
 2018-08-02 17:38:24 +01:00
Richard van der Hoff
bdae8f2e68
Merge pull request #3638 from matrix-org/rav/refactor_federation_client_exception_handling 
Factor out exception handling in federation_client
 2018-08-02 17:37:46 +01:00
Erik Johnston
40c1c59cf4
Merge pull request #3621 from matrix-org/erikj/split_fed_store 
Split out DB writes in federation handler
 2018-08-02 10:41:42 +01:00
Neil Johnson
085435e13a
Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits 
Initial impl of capping MAU
 2018-08-01 15:58:45 +00:00
Richard van der Hoff
b8d7d3996b
Merge pull request #3620 from fuzzmz/return-404-room-not-found 
return 404 if room not found
 2018-08-01 16:34:32 +01:00
Richard van der Hoff
908be65e64 changelog 2018-08-01 16:23:31 +01:00
Neil Johnson
7ff44d9215 improve clarity 2018-08-01 16:17:00 +01:00
Richard van der Hoff
38b98e5a98 changelog 2018-08-01 16:07:49 +01:00
Amber Brown
da7785147d
Python 3: Convert some unicode/bytes uses (#3569) 2018-08-02 00:54:06 +10:00
Neil Johnson
303f1c851f Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits 2018-08-01 13:42:50 +01:00
Jan Christian Grünhage
c4842e16cb
Merge pull request #3543 from bebehei/docker 
Improvements for Docker usage
 2018-08-01 11:32:45 +02:00
Richard van der Hoff
6e63d6868c
Update 2952.bugfix 2018-08-01 10:31:22 +01:00
Richard van der Hoff
f49147d14f
Merge pull request #3634 from matrix-org/rav/wtf_is_a_replication_layer 
rename replication_layer to federation_client
 2018-08-01 10:29:29 +01:00
Richard van der Hoff
cab782c17e
Merge pull request #3384 from matrix-org/rav/rewrite_cachedlist_decorator 
Rewrite cache list decorator
 2018-08-01 10:28:56 +01:00
Serban Constantin
70af98e361
return NotFoundError if room not found 
Per the Client-Server API[0] we should return
`M_NOT_FOUND` if the room isn't found instead
of generic SynapseError.

This ensures that /directory/list API returns
404 for room not found instead of 400.

[0]: https://matrix.org/docs/spec/client_server/unstable.html#get-matrix-client-r0-directory-list-room-roomid

Signed-off-by: Serban Constantin <serban.constantin@gmail.com>
 2018-07-31 21:47:23 +03:00
Travis Ralston
5e2ee64660
Merge pull request #3628 from turt2live/travis/goodby-pdu-failures 
Remove pdu_failures from transactions
 2018-07-31 12:13:09 -06:00
Richard van der Hoff
1841672c85 changelog 2018-07-31 18:26:54 +01:00
Erik Johnston
16bd63f32f Newsfile 2018-07-31 14:48:43 +01:00
Richard van der Hoff
5de936caa1
Merge pull request #3612 from matrix-org/rav/store_heirarchy 
Make EventStore inherit from EventFederationStore
 2018-07-31 13:44:04 +01:00
Neil Johnson
5bb39b1e0c mau limts 2018-07-31 13:22:14 +01:00
Travis Ralston
e908b86832 Remove pdu_failures from transactions 
The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.
 2018-07-30 16:28:47 -06:00
Krombel
254e8267e2 Only import secrets when available 
secrets got introduced in python 3.6 so this class is not available
in 3.5 and before.

This now checks for the current running version and only tries using
secrets if the version is 3.6 or above

Signed-Off-By: Matthias Kesler <krombel@krombel.de>
 2018-07-30 23:59:02 +02:00
Erik Johnston
143f1a2532 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_fed_store 2018-07-30 09:56:18 +01:00
Matthew Hodgson
e9b2d047f6
make /context lazyload & filter aware (#3567) 
make /context lazyload & filter aware.
 2018-07-27 15:12:50 +01:00
Erik Johnston
ad7cd95a78 Newsfile 2018-07-27 15:02:57 +01:00
Richard van der Hoff
f102c05856 Rewrite cache list decorator 
Because it was complicated and annoyed me. I suspect this will be more
efficient too.
 2018-07-27 13:47:04 +01:00
Richard van der Hoff
b0b5566f36
Merge pull request #3391 from t3chguy/t3chguy/default_inviter_display_name_3pid 
if inviter_display_name == ""||None then default to inviter MXID
 2018-07-27 10:08:39 +01:00
Richard van der Hoff
7041cd872b
Merge branch 'develop' into send_sni_for_federation_requests 2018-07-27 09:17:11 +01:00
Richard van der Hoff
d42455cdc9
Merge pull request #3616 from matrix-org/travis/event_id_send_leave 
Update the send_leave path to be an event_id
 2018-07-26 23:25:17 +01:00
Richard van der Hoff
65c8dee900
Merge pull request #3614 from matrix-org/rav/stop_populating_event_content 
Stop populating events.content
 2018-07-26 22:54:08 +01:00
Matthew Hodgson
a75231b507
Deduplicate redundant lazy-loaded members (#3331) 
* attempt at deduplicating lazy-loaded members

as per the proposal; we can deduplicate redundant lazy-loaded members
which are sent in the same sync sequence. we do this heuristically
rather than requiring the client to somehow tell us which members it
has chosen to cache, by instead caching the last N members sent to
a client, and not sending them again.  For now we hardcode N to 100.
Each cache for a given (user,device) tuple is in turn cached for up to
X minutes (to avoid the caches building up).  For now we hardcode X to 30.

* add include_redundant_members filter option & make it work

* remove stale todo

* add tests for _get_some_state_from_cache

* incorporate review
 2018-07-26 22:51:30 +01:00
Richard van der Hoff
9e68b1bd2d
Merge pull request #3613 from matrix-org/rav/stop_using_event_edges_room_id 
Remove some redundant joins on event_edges.room_id
 2018-07-26 22:31:01 +01:00
Travis Ralston
49254d43a6
Create 3616.misc 2018-07-26 14:45:48 -06:00
Richard van der Hoff
85531a06a2 changelog 2018-07-26 14:55:29 +01:00
Richard van der Hoff
cf78eaebad changelog 2018-07-26 13:22:40 +01:00
Richard van der Hoff
a15ed52267 changelog 2018-07-26 12:53:18 +01:00
Richard van der Hoff
03751a6420 Fix some looping_call calls which were broken in #3604 
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604.

Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
 2018-07-26 11:48:08 +01:00
Matthew Hodgson
1bcd0490c2
Merge pull request #2970 from matrix-org/matthew/filter_members 
Implement the lazy_load_members room state filter parameter
 2018-07-26 00:03:01 +01:00
Travis Ralston
6185650f9c
Create 3609.misc 2018-07-25 15:46:56 -06:00
Matthew Hodgson
2565804030 Merge branch 'develop' into matthew/filter_members 2018-07-25 17:27:49 +01:00
Erik Johnston
3849f7f69f
Merge pull request #3603 from matrix-org/erikj/handle_outliers 
Correctly handle outliers during persist events
 2018-07-25 13:24:04 +01:00
Richard van der Hoff
cee1ae1b72
Merge pull request #3606 from matrix-org/rav/logcontext_fixes_once_more 
Fix another logcontext leak in _persist_events
 2018-07-25 11:56:00 +01:00
Richard van der Hoff
32b30e15f2
Merge pull request #3607 from matrix-org/rav/fix_persist_events_integrity_error 
Fix occasional 'tuple index out of range' error
 2018-07-25 11:55:45 +01:00
Richard van der Hoff
4081bd1560
Merge pull request #3605 from matrix-org/rav/fix_update_remote_profile_cache 
Fix updating of cached remote profiles
 2018-07-25 11:54:46 +01:00
Richard van der Hoff
1be94440d3 Fix occasional 'tuple index out of range' error 
This fixes a bug in _delete_existing_rows_txn which was introduced in #3435
(though it's been on matrix-org-hotfixes for *years*). This code is only called
when there is some sort of conflict the first time we try to persist an event,
so it only happens rarely. Still, the exceptions are annoying.
 2018-07-25 11:05:58 +01:00
Richard van der Hoff
07defd5fe6 Fix another logcontext leak in _persist_events 
We need to run the errback in the sentinel context to avoid losing our own
context.

Also: add logging to runInteraction to help identify where "Starting db
connection from sentinel context" warnings are coming from
 2018-07-25 10:53:23 +01:00
Richard van der Hoff
9c237a7ab5 changelog 2018-07-25 10:46:01 +01:00
Richard van der Hoff
3f11d84534 Changelog 2018-07-25 09:43:25 +01:00
Erik Johnston
0b300d323a Newsfile 2018-07-25 09:39:45 +01:00
Michael Telatynski
38eaa5280d
add changelog entry for PR#3391 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
 2018-07-24 17:25:42 +01:00
Erik Johnston
1674a85238 Move newsfile 2018-07-24 17:20:27 +01:00
Erik Johnston
f14c866e37 Newsfile 2018-07-24 16:51:19 +01:00
Erik Johnston
60a1d147a7
Merge pull request #3595 from matrix-org/erikj/use_deltas 
Use deltas to calculate current state deltas
 2018-07-24 15:41:18 +01:00
Erik Johnston
f33c596533 Newsfile 2018-07-24 14:28:04 +01:00
Richard van der Hoff
81946db9cf
Merge pull request #3587 from matrix-org/rav/better_exception_logging 
Improve logging for exceptions when handling PDUs
 2018-07-24 14:12:12 +01:00
Richard van der Hoff
a321f78991
Merge pull request #3586 from matrix-org/rav/optimise_resolve_state_groups 
Fixes and optimisations for resolve_state_groups
 2018-07-24 14:11:45 +01:00
Richard van der Hoff
93b0722c50
Merge pull request #3583 from matrix-org/rav/remove_who_forgot_in_room 
Remove redundant checks on room forgottenness
 2018-07-24 14:11:11 +01:00
Matthew Hodgson
454f59b7ad Merge branch 'develop' into matthew/filter_members 2018-07-24 14:03:37 +01:00
Matthew Hodgson
d19fba3655 Merge branch 'develop' into matthew/filter_members 2018-07-24 12:39:54 +01:00
Richard van der Hoff
30bfed5aa5 Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_room 2018-07-24 11:46:09 +01:00
Erik Johnston
97acd385a3 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/speed_up_calculate_state_delta 2018-07-24 11:32:13 +01:00
Erik Johnston
2581eb3e1d Newsfile 2018-07-24 11:14:28 +01:00
Richard van der Hoff
a678145010
Merge branch 'develop' into rav/logcontext_fixes 2018-07-24 10:43:30 +01:00
Erik Johnston
d436ad332c
Merge pull request #3555 from matrix-org/erikj/client_apis_move 
Make client_reader support some more read only APIs
 2018-07-24 10:42:28 +01:00
Erik Johnston
536bc63a4e
Merge branch 'develop' into erikj/client_apis_move 2018-07-24 09:57:05 +01:00
Richard van der Hoff
c6e66821a9 Changelog 2018-07-24 00:38:39 +01:00
Richard van der Hoff
30957a941a changelog 2018-07-24 00:24:38 +01:00
Matthew Hodgson
004a83b43a changelog 2018-07-23 22:32:35 +01:00
Richard van der Hoff
d8709df739 changelog 2018-07-23 22:16:22 +01:00
Richard van der Hoff
cc99256e90 newsfile 2018-07-23 19:10:50 +01:00
Erik Johnston
2d5bba151b Newsfile 2018-07-23 17:29:32 +01:00
Richard van der Hoff
4f5cc8e4e7 Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_room 2018-07-23 17:15:12 +01:00
Richard van der Hoff
dae6dc1e77 Remove redundant checks on room forgottenness 
Fixes #3550
 2018-07-23 17:13:34 +01:00
Erik Johnston
9f41ad491d Newsfile 2018-07-23 16:31:46 +01:00
Erik Johnston
a4d24781bf Newsfile 2018-07-23 15:28:51 +01:00
Erik Johnston
0b0b24cb82 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move 2018-07-23 13:21:15 +01:00
Erik Johnston
f3182bb1d0 Newsfile 2018-07-23 13:19:24 +01:00
Erik Johnston
959f4b9074 Newsfile 2018-07-23 12:22:59 +01:00
Matthew Hodgson
9b34f3ea3a
Merge branch 'develop' into matthew/sync_deleted_devices 2018-07-23 10:03:28 +01:00
Amber Brown
3132b89f12
Make the rest of the .iterwhatever go away (#3562) 2018-07-21 15:47:18 +10:00
Richard van der Hoff
5c30cb709a Changelog 2018-07-20 14:01:36 +01:00
Richard van der Hoff
4f67623674
Merge pull request #3571 from matrix-org/rav/limiter_fixes 
A set of improvements to the Limiter
 2018-07-20 13:53:03 +01:00
Amber Brown
e1a237eaab
Admin API for creating new users (#3415) 2018-07-20 22:41:13 +10:00
Richard van der Hoff
683f4058c1 changelogs 2018-07-20 13:16:39 +01:00
Amber Brown
7cf76c9a09 changelog 2018-07-19 21:26:30 +10:00
Amber Brown
9a8acdc720 Merge branch 'master' into develop 2018-07-19 21:16:44 +10:00
Amber Brown
d69decd5c7 0.33.0 final changelog 2018-07-19 21:12:15 +10:00
Amber Brown
13d501c773 update changelogs 2018-07-19 21:11:24 +10:00
Amber Brown
ce0545eca1 Revert "0.33.0rc1 changelog" 
This reverts commit 21d3b87943.
 2018-07-19 21:03:15 +10:00
Amber Brown
95ccb6e2ec
Don't spew errors because we can't save metrics (#3563) 2018-07-19 20:58:18 +10:00
Richard van der Hoff
1ffb7bec20 Merge remote-tracking branch 'origin/release-v0.33.0' into develop 2018-07-19 11:12:33 +01:00
Richard van der Hoff
c754e006f4
Merge pull request #3556 from matrix-org/rav/background_processes 
Run things as background processes
 2018-07-19 11:04:18 +01:00
Amber Brown
a97c845271
Move v1-only APIs into their own module & isolate deprecated ones (#3460) 2018-07-19 20:03:33 +10:00
Richard van der Hoff
18a2b2c0b4 changelog 2018-07-19 10:54:39 +01:00
Benedikt Heine
f1dd89fe86 [Docker] Use sorted multiline package lists 
This matches docker best practices.

Signed-off-by: Benedikt Heine <bebe@bebehei.de>
 2018-07-19 11:38:58 +02:00
Erik Johnston
6f62a6ef21
Merge pull request #3554 from matrix-org/erikj/response_metrics_code 
Add response code to response timer metrics
 2018-07-19 10:25:18 +01:00
Richard van der Hoff
eed24893fa changelog 2018-07-18 22:12:19 +01:00
Richard van der Hoff
08436c556a changelog 2018-07-18 20:55:05 +01:00
Erik Johnston
3a993a660d Newsfile 2018-07-18 15:39:49 +01:00
Richard van der Hoff
65d6a0e477
Merge pull request #3553 from matrix-org/rav/background_process_tracking 
Resource tracking for background processes
 2018-07-18 14:27:50 +01:00
Erik Johnston
00845c49d2 Newsfile 2018-07-18 14:03:16 +01:00
Richard van der Hoff
dab00faa83
Merge pull request #3367 from matrix-org/rav/drop_re_signing_hacks 
Remove event re-signing hacks
 2018-07-18 12:46:27 +01:00
David Baker
c91a44572e
Merge pull request #3514 from matrix-org/dbkr/turn_dont_add_defaults 
Comment dummy TURN parameters in default config
 2018-07-18 11:56:18 +01:00
Richard van der Hoff
92aecd557b changelog 2018-07-18 11:29:48 +01:00
Amber Brown
21d3b87943 0.33.0rc1 changelog 2018-07-18 12:53:32 +10:00
Richard van der Hoff
4a11df5b64 changelog 2018-07-17 14:24:29 +01:00
Richard van der Hoff
9c04b4abf9
Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server 
Refactor and optimze filter_events_for_server
 2018-07-17 14:01:39 +01:00
Amber Brown
bc006b3c9d
Refactor REST API tests to use explicit reactors (#3351) 2018-07-17 20:43:18 +10:00
Erik Johnston
9952d18e4d Newsfile 2018-07-17 10:31:51 +01:00
Amber Brown
3fe0938b76
Merge pull request #3530 from matrix-org/erikj/stream_cache 
Don't return unknown entities in get_entities_changed
 2018-07-17 13:44:46 +10:00
Richard van der Hoff
2731bf7ac3 Changelog 2018-07-16 14:12:25 +01:00
Krombel
4a27000548 check isort by travis 2018-07-16 13:57:33 +02:00
Jeroen
505530f36a Merge remote-tracking branch 'upstream/develop' into send_sni_for_federation_requests 
# Conflicts:
#	synapse/crypto/context_factory.py
 2018-07-14 20:24:46 +02:00
Amber Brown
8a4f05fefb
Fix develop because I broke it :( (#3535) 2018-07-14 09:51:00 +10:00
Amber Brown
8532953c04
Merge pull request #3534 from krombel/use_parse_and_asserts_from_servlet 
Use parse and asserts from http.servlet
 2018-07-14 09:09:19 +10:00
Amber Brown
33b60c01b5
Make auth & transactions more testable (#3499) 2018-07-14 07:34:49 +10:00
Krombel
516f960ad8 add changelog 2018-07-13 22:19:19 +02:00
Richard van der Hoff
08546be40f better changelog 2018-07-13 16:47:13 +01:00
Richard van der Hoff
b03a0e14db changelog 2018-07-13 16:28:28 +01:00
Erik Johnston
5f263b607e Newsfile 2018-07-13 15:47:25 +01:00
Richard van der Hoff
6dff49b8a9
Merge pull request #3521 from matrix-org/rav/optimise_stream_change_cache 
Reduce set building in get_entities_changed
 2018-07-12 12:08:49 +01:00
Matthew Hodgson
37c4fba0ac changelog 2018-07-12 11:45:33 +01:00
Richard van der Hoff
38c5fa7ee4 changelog 2018-07-12 11:45:28 +01:00
Richard van der Hoff
c969754a91 changelog 2018-07-12 10:00:57 +01:00
Erik Johnston
0456e05977
Merge pull request #3505 from matrix-org/erikj/receipts_cahce 
Use stream cache in get_linearized_receipts_for_room
 2018-07-12 09:46:29 +01:00
David Baker
1b5425527c I failed to correctly guess the PR number 2018-07-11 16:02:29 +01:00
David Baker
36f4fd3e1e Comment dummy TURN parameters in default config 
This default config is parsed and used a base before the actual
config is overlaid, so with these values not commented out, the
code to detect when no turn params were set and refuse to generate
credentials was never firing because the dummy default was always set.
 2018-07-11 15:49:29 +01:00
Amber Brown
129ffd7b88
Merge pull request #3498 from OlegGirko/fix_attrs_syntax 
* Use more portable syntax using attrs package.

Newer syntax

    attr.ib(factory=dict)

is just a syntactic sugar for

    attr.ib(default=attr.Factory(dict))

It was introduced in newest version of attrs package (18.1.0)
and doesn't work with older versions.

We should either require minimum version of attrs to be 18.1.0,
or use older (slightly more verbose) syntax.
Requiring newest version is not a good solution because
Linux distributions may have older version of attrs (17.4.0 in Fedora 28),
and requiring to build (and package)
newer version just to use newer syntactic sugar in only one test
is just too much.
It's much better to fix that test to use older syntax.

Signed-off-by: Oleg Girko <ol@infoserver.lv>
 2018-07-11 04:22:46 +10:00
Amber Brown
85354bb18e changelog entry 2018-07-11 03:27:03 +10:00
Erik Johnston
bb3d536087 Newsfile 2018-07-10 17:28:31 +01:00
Richard van der Hoff
c3c29aa196
Attempt to include db threads in cpu usage stats (#3496) 
Let's try to include time spent in the DB threads in the per-request/block cpu
usage metrics.
 2018-07-10 16:12:36 +01:00
Richard van der Hoff
55370331da
Refactor logcontext resource usage tracking (#3501) 
Factor out the resource usage tracking out to a separate object, which can be
passed around and copied independently of the logcontext itself.
 2018-07-10 13:56:07 +01:00
Richard van der Hoff
f3b3b9dd8f changelog 2018-07-09 18:16:52 +01:00
Jeroen
b5e157d895 Merge branch 'develop' into send_sni_for_federation_requests 
# Conflicts:
#	synapse/http/endpoint.py
 2018-07-09 08:51:11 +02:00
Amber Brown
09477bd884 changelog 2018-07-09 16:09:37 +10:00
Amber Brown
2ee9f1bd1a
Add an isort configuration (#3463) 2018-07-09 16:05:21 +10:00
Amber Brown
1241156c82 changelog 2018-07-07 10:48:30 +10:00
Amber Brown
e845fd41c2
Correct attrs package name in requirements (#3492) 2018-07-07 10:46:59 +10:00
Amber Brown
89690aaaeb changelog 2018-07-05 20:46:40 +10:00
Amber Brown
be8b32dbc2 ACL changelog 2018-07-05 20:45:12 +10:00
Erik Johnston
1a88640677
Merge pull request #3483 from matrix-org/rav/more_server_name_validation 
More server_name validation
 2018-07-05 10:04:20 +01:00
Richard van der Hoff
546bc9e28b More server_name validation 
We need to do a bit more validation when we get a server name, but don't want
to be re-doing it all over the shop, so factor out a separate
parse_and_validate_server_name, and do the extra validation.

Also, use it to verify the server name in the config file.
 2018-07-04 18:59:51 +01:00
Richard van der Hoff
f192a93875
Merge pull request #3481 from matrix-org/rav/fix_cachedescriptor_test 
Reinstate lost run_on_reactor in unit test
 2018-07-04 18:55:33 +01:00
Erik Johnston
13f7adf84b
Merge pull request #3473 from matrix-org/erikj/thread_cache 
Invalidate cache on correct thread
 2018-07-04 10:11:38 +01:00
Erik Johnston
40252d13d1
Merge pull request #3474 from matrix-org/erikj/py3_auth 
Fix up auth check
 2018-07-04 09:41:33 +01:00
Richard van der Hoff
ea555d5633 Reinstate lost run_on_reactor in unit test 
a61738b removed a call to run_on_reactor from a unit test, but that call was
doing something useful, in making the function in question asynchronous.

Reinstate the call and add a check that we are testing what we wanted to be
testing.
 2018-07-04 09:40:01 +01:00
Richard van der Hoff
c4e7ad0e0f Add changelog 2018-07-04 09:15:45 +01:00
Richard van der Hoff
508196e08a
Reject invalid server names (#3480) 
Make sure that server_names used in auth headers are sane, and reject them with
a sensible error code, before they disappear off into the depths of the system.
 2018-07-03 14:36:14 +01:00
Matthew Hodgson
6ec3aa2f72 news snippet 2018-07-02 13:43:34 +01:00
Erik Johnston
abb183438c Correct newsfile 2018-07-02 13:12:38 +01:00
Erik Johnston
f88dea577d Newsfile 2018-07-02 11:46:32 +01:00
Erik Johnston
cea4662b13 Newsfile 2018-07-02 11:46:20 +01:00
Matthew Hodgson
75d4986a8c
Merge pull request #3467 from matrix-org/hawkowl/contributor-requirements 
Clarify "real name" in contributor requirements
 2018-06-30 00:21:10 +01:00
Amber Brown
7c0cdd330f topfile 2018-06-29 14:13:15 +01:00
Erik Johnston
e3b4043800
Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking 
Check the state of prev_events a bit more thoroughly when coming over federation
 2018-06-29 13:55:02 +01:00
Matthew Hodgson
0d7eabeada add towncrier snippet 2018-06-28 20:59:01 +01:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy (#3462) 2018-06-28 14:49:57 +01:00
Amber Brown
cfda61e9cd topfile update 2018-06-28 11:13:08 +01:00
Amber Brown
72d2143ea8
Revert "Revert "Try to not use as much CPU in the StreamChangeCache"" (#3454) 2018-06-28 11:04:18 +01:00
Amber Brown
caf07f770a topfile 2018-06-27 15:07:16 +01:00
Amber Brown
cd6bcdaf87
Better testing framework for homeserver-using things (#3446) 2018-06-27 10:37:24 +01:00
Jeroen
26651b0d6a towncrier changelog 2018-06-26 21:10:45 +02:00
Matthew Hodgson
8057489b26
Revert "Try to not use as much CPU in the StreamChangeCache" 2018-06-26 18:09:01 +01:00
Amber Brown
bd3d329c88 fixes 2018-06-26 17:28:12 +01:00
Amber Brown
36cb570641
Use towncrier to build the changelog (#3425) 2018-06-25 14:42:27 +01:00