Richard van der Hoff
d897be6a98
Fix visibility of events from erased users over federation
2018-07-17 14:02:07 +01:00
Richard van der Hoff
9c04b4abf9
Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server
...
Refactor and optimze filter_events_for_server
2018-07-17 14:01:39 +01:00
Richard van der Hoff
94440ae994
fix imports
2018-07-17 11:51:26 +01:00
Amber Brown
bc006b3c9d
Refactor REST API tests to use explicit reactors ( #3351 )
2018-07-17 20:43:18 +10:00
Erik Johnston
c7320a5564
Merge pull request #3544 from matrix-org/erikj/fixup_stream_cache
...
Fix perf regression in PR #3530
2018-07-17 11:16:59 +01:00
Richard van der Hoff
2172a3d8cb
add a comment
2018-07-17 11:13:57 +01:00
Erik Johnston
b2aa05a8d6
Use efficient .intersection
2018-07-17 11:07:04 +01:00
Erik Johnston
850238b4ef
Add unit test
2018-07-17 10:59:02 +01:00
Erik Johnston
9952d18e4d
Newsfile
2018-07-17 10:31:51 +01:00
Erik Johnston
547b1355d3
Fix perf regression in PR #3530
...
The get_entities_changed function was changed to return all changed
entities since the given stream position, rather than only those changed
from a given list of entities. This resulted in the function incorrectly
returning large numbers of entities that, for example, caused large
increases in database usage.
2018-07-17 10:27:51 +01:00
Amber Brown
3fe0938b76
Merge pull request #3530 from matrix-org/erikj/stream_cache
...
Don't return unknown entities in get_entities_changed
2018-07-17 13:44:46 +10:00
Amber Brown
fe10dd9fb2
Merge pull request #3540 from krombel/enforce_isort
...
check isort by travis
2018-07-17 13:41:59 +10:00
Krombel
9677b1d1c0
rename 'isort' to 'check_isort' as requested
2018-07-16 16:03:41 +02:00
Richard van der Hoff
2731bf7ac3
Changelog
2018-07-16 14:12:25 +01:00
Richard van der Hoff
09e29fb58b
Attempt to make _filter_events_for_server more efficient
2018-07-16 14:06:09 +01:00
Richard van der Hoff
15b13b537f
Add a test which profiles filter_events_for_server in a large room
2018-07-16 14:06:09 +01:00
Krombel
78a9ddcf9a
rerun isort with latest version
2018-07-16 14:23:25 +02:00
Richard van der Hoff
ea69d35651
Move filter_events_for_server out of FederationHandler
...
for easier unit testing.
2018-07-16 13:06:24 +01:00
Krombel
4a27000548
check isort by travis
2018-07-16 13:57:33 +02:00
Amber Brown
8a4f05fefb
Fix develop because I broke it :( ( #3535 )
2018-07-14 09:51:00 +10:00
Amber Brown
8532953c04
Merge pull request #3534 from krombel/use_parse_and_asserts_from_servlet
...
Use parse and asserts from http.servlet
2018-07-14 09:09:19 +10:00
Amber Brown
a2374b2c7f
fix sytests
2018-07-14 07:52:58 +10:00
Amber Brown
33b60c01b5
Make auth & transactions more testable ( #3499 )
2018-07-14 07:34:49 +10:00
Krombel
516f960ad8
add changelog
2018-07-13 22:19:19 +02:00
Krombel
3366b9c534
rename assert_params_in_request to assert_params_in_dict
...
the method "assert_params_in_request" does handle dicts and not
requests. A request body has to be parsed to json before this method
can be used
2018-07-13 21:53:01 +02:00
Krombel
32fd6910d0
Use parse_{int,str} and assert from http.servlet
...
parse_integer and parse_string can take a request and raise errors
in case we have wrong or missing params.
This PR tries to use them more to deduplicate some code and make it
better readable
2018-07-13 21:40:14 +02:00
Erik Johnston
bc832f822f
Fixup unit test
2018-07-13 17:03:04 +01:00
Richard van der Hoff
2aba1f549c
Merge pull request #3533 from matrix-org/rav/fix_federation_ratelimite_queue
...
Make FederationRateLimiter queue requests properly
2018-07-13 16:59:18 +01:00
Richard van der Hoff
08546be40f
better changelog
2018-07-13 16:47:13 +01:00
Richard van der Hoff
b03a0e14db
changelog
2018-07-13 16:28:28 +01:00
Richard van der Hoff
3b391d9c45
Fix unit tests
2018-07-13 16:28:04 +01:00
Richard van der Hoff
33b40d0a25
Make FederationRateLimiter queue requests properly
...
popitem removes the *most recent* item by default [1]. We want the oldest.
Fixes #3524
[1]: https://docs.python.org/2/library/collections.html#collections.OrderedDict.popitem
2018-07-13 16:19:40 +01:00
Erik Johnston
5f263b607e
Newsfile
2018-07-13 15:47:25 +01:00
Erik Johnston
77b692e65d
Don't return unknown entities in get_entities_changed
...
The stream cache keeps track of all entities that have changed since
a particular stream position, so get_entities_changed does not need to
return unknown entites when given a larger stream position.
This makes it consistent with the behaviour of has_entity_changed.
2018-07-13 15:26:10 +01:00
Matthew Hodgson
ba22b6a456
typo
2018-07-13 12:03:39 +01:00
Richard van der Hoff
6dff49b8a9
Merge pull request #3521 from matrix-org/rav/optimise_stream_change_cache
...
Reduce set building in get_entities_changed
2018-07-12 12:08:49 +01:00
Richard van der Hoff
38c5fa7ee4
changelog
2018-07-12 11:45:28 +01:00
Richard van der Hoff
fa5c2bc082
Reduce set building in get_entities_changed
...
This line shows up as about 5% of cpu time on a synchrotron:
not_known_entities = set(entities) - set(self._entity_to_key)
Presumably the problem here is that _entity_to_key can be largeish, and
building a set for its keys every time this function is called is slow.
Here we rewrite the logic to avoid building so many sets.
2018-07-12 11:37:44 +01:00
Richard van der Hoff
4c4dd6299d
Merge pull request #3316 from matrix-org/rav/enforce_report_api
...
Enforce the specified API for report_event
2018-07-12 10:50:05 +01:00
Richard van der Hoff
c969754a91
changelog
2018-07-12 10:00:57 +01:00
Richard van der Hoff
482d17b58b
Merge branch 'develop' into rav/enforce_report_api
2018-07-12 09:56:28 +01:00
Erik Johnston
0456e05977
Merge pull request #3505 from matrix-org/erikj/receipts_cahce
...
Use stream cache in get_linearized_receipts_for_room
2018-07-12 09:46:29 +01:00
Erik Johnston
aff1dfdf3d
Update return value docstring
2018-07-12 09:45:37 +01:00
Amber Brown
129ffd7b88
Merge pull request #3498 from OlegGirko/fix_attrs_syntax
...
* Use more portable syntax using attrs package.
Newer syntax
attr.ib(factory=dict)
is just a syntactic sugar for
attr.ib(default=attr.Factory(dict))
It was introduced in newest version of attrs package (18.1.0)
and doesn't work with older versions.
We should either require minimum version of attrs to be 18.1.0,
or use older (slightly more verbose) syntax.
Requiring newest version is not a good solution because
Linux distributions may have older version of attrs (17.4.0 in Fedora 28),
and requiring to build (and package)
newer version just to use newer syntactic sugar in only one test
is just too much.
It's much better to fix that test to use older syntax.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
2018-07-11 04:22:46 +10:00
Amber Brown
85354bb18e
changelog entry
2018-07-11 03:27:03 +10:00
Erik Johnston
6ccefef07a
Use 'is not None' and add comments
2018-07-10 18:12:39 +01:00
Matthew Hodgson
ea752bdd99
s/becuase/because/g
2018-07-10 17:58:18 +01:00
Erik Johnston
bb3d536087
Newsfile
2018-07-10 17:28:31 +01:00
Erik Johnston
05f5dabc10
Use stream cache in get_linearized_receipts_for_room
...
This avoids us from uncessarily hitting the database when there has been
no change for the room
2018-07-10 17:22:42 +01:00
Richard van der Hoff
c3c29aa196
Attempt to include db threads in cpu usage stats ( #3496 )
...
Let's try to include time spent in the DB threads in the per-request/block cpu
usage metrics.
2018-07-10 16:12:36 +01:00
