MirrorHub/synapse
0
0
Fork 1
mirror of https://mau.dev/maunium/synapse.git synced 2024-11-18 16:02:15 +01:00
Code Issues Projects Releases Wiki Activity
10837 commits 5 branches 731 tags 91 MiB
Commit graph

837 commits

Author SHA1 Message Date
Erik Johnston
310197bab5 Fix AS retries 2016-06-07 09:34:50 +01:00
Erik Johnston
05e01f21d7 Remove event fetching from DB threads 2016-06-03 17:22:13 +01:00
Erik Johnston
73c7112433 Change CacheMetrics to be quicker 
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
 2016-06-03 11:26:52 +01:00
Mark Haines
56d15a0530 Store the typing users as user_id strings. (#819) 
Rather than storing them as UserID objects.
 2016-06-02 16:28:54 +01:00
Mark Haines
70599ce925 Allow external processes to mark a user as syncing. (#812) 
* Add infrastructure to the presence handler to track sync requests in external processes

* Expire stale entries for dead external processes

* Add an http endpoint for making users as syncing

Add some docstrings and comments.

* Fixes
 2016-06-02 15:20:15 +01:00
David Baker
4a10510cd5 Split out the auth handler 2016-06-02 13:31:45 +01:00
David Baker
195254cae8 Inject fake room list handler in tests 
Otherwise it tries to start the remote public room list updating looping call which breaks.
 2016-06-01 11:14:16 +01:00
Mark Haines
c626fc576a Move the AS handler out of the Handlers object. 
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
 2016-05-31 13:53:48 +01:00
Negi Fazeli
6fe04ffef2 Fix set profile error with Requester. 
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
 2016-05-23 19:50:28 +02:00
Mark Haines
0cb441fedd Move typing handler out of the Handlers object 2016-05-17 15:58:46 +01:00
Mark Haines
425e6b4983 Merge branch 'develop' into markjh/member_cleanup 2016-05-17 10:13:16 +01:00
Mark Haines
816df9f267 get_room_members is unused now 2016-05-16 19:51:43 +01:00
Mark Haines
821306120a Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room 2016-05-16 19:48:07 +01:00
Mark Haines
3b86ecfa79 Move the presence handler out of the Handlers object 2016-05-16 18:56:37 +01:00
Mark Haines
21018c2c13 Merge pull request #783 from matrix-org/markjh/slave_account_data 
Add a slaved datastore for account data
 2016-05-13 15:56:04 +01:00
Mark Haines
3abab26458 Add a slaved datastore for account data 2016-05-13 15:34:06 +01:00
Negi Fazeli
40aa6e8349 Create user with expiry 
- Add unittests for client, api and handler

Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
 2016-05-13 15:34:15 +02:00
Mark Haines
c0d8e0eb63 Replicate push actions 2016-04-21 15:25:58 +01:00
Mark Haines
f52dd35ac3 Merge pull request #738 from matrix-org/markjh/slaved_receipts 
Add a slaved receipts store
 2016-04-19 17:31:59 +01:00
Mark Haines
5bbd424ee0 Add a slaved receipts store 2016-04-19 17:14:08 +01:00
Mark Haines
e99365f601 Replicate get_invited_rooms_for_user 2016-04-19 15:22:14 +01:00
Erik Johnston
eb8619e256 Create log context in Measure if one doesn't exist 2016-04-18 16:08:32 +01:00
Erik Johnston
79fc4ff6f9 Merge pull request #677 from matrix-org/erikj/dns_cache 
Read from DNS cache if within TTL
 2016-04-08 14:09:56 +01:00
Mark Haines
7e2f971c08 Remove some unused functions (#711) 
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
 2016-04-08 14:01:56 +01:00
Mark Haines
ceb599e789 Add tests for redactions 2016-04-07 16:52:07 +01:00
Mark Haines
8c82b06904 Merge pull request #704 from matrix-org/markh/slaveIII 
Add tests for get_latest_event_ids_in_room and get_current_state
 2016-04-07 16:49:34 +01:00
Mark Haines
57fa1801c3 Add sensible __eq__ operators inside the tests. 
Rather than adding them globally. This limits the changes to only
affect the tests.
 2016-04-07 16:41:37 +01:00
Erik Johnston
af03ecf352 Deduplicate joins 2016-04-07 14:19:02 +01:00
Mark Haines
60ec9793fb Add tests for get_latest_event_ids_in_room and get_current_state 2016-04-07 13:17:56 +01:00
Erik Johnston
a28d066732 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache 2016-04-07 11:11:17 +01:00
Mark Haines
6bfec56796 Test that room membership is replicated 2016-04-06 16:20:13 +01:00
Mark Haines
e815763b7f Merge pull request #697 from matrix-org/markjh/slaveI 
Add a slaved events store class
 2016-04-06 16:19:25 +01:00
Mark Haines
75fb9ac1be Add a slaved events store class 
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
 2016-04-06 14:18:35 +01:00
Erik Johnston
8aab9d87fa Don't require config to create database 2016-04-06 14:15:45 +01:00
Erik Johnston
11860637e1 Tests 2016-04-06 10:12:30 +01:00
Erik Johnston
c906f30661 Do checks for memberships before creating events 2016-04-01 16:17:32 +01:00
Erik Johnston
f699b8f997 Read from DNS cache if within TTL 2016-03-31 10:04:28 +01:00
Mark Haines
31a9eceda5 Add a replication stream for state groups 2016-03-30 16:01:58 +01:00
Mark Haines
673c96ce97 Remove dead code left over from presence changes 2016-03-17 11:03:47 +00:00
David Baker
ff7d3dc3a0 Fix tests 2016-03-16 14:25:14 +00:00
Mark Haines
b6e8420aee Add replication stream for pushers 2016-03-15 17:33:10 +00:00
Mark Haines
e9c1cabac2 Use parse_json_object_from_request to parse JSON out of request bodies 2016-03-11 16:41:03 +00:00
blide
1be438f2a6 Flake8 fix 2016-03-10 12:13:35 +03:00
blide
40160e24ab Register endpoint returns refresh_token 
Guest registration still doesn't return refresh_token
 2016-03-10 10:29:19 +03:00
David Baker
316c00936f Fix tests 2016-03-07 17:32:36 +00:00
Mark Haines
3110c37d02 Fix unit tests 2016-03-04 14:48:17 +00:00
Mark Haines
ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Patrik Oldsberg
bb0e82fff1 tests/utils: added room_invite_state_types to test config 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-03-04 10:43:17 +01:00
Daniel Wagner-Hall
b4022cc487 Pass whole requester to ratelimiting 
This will enable more detailed decisions
 2016-03-03 16:43:42 +00:00
Daniel Wagner-Hall
95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids 
Mark AS users with their AS's ID
 2016-03-03 13:56:28 +00:00
Mark Haines
2223204eba Hook push rules up to the replication API 2016-03-02 17:26:20 +00:00
Erik Johnston
2a78dac60d Merge pull request #612 from matrix-org/erikj/cache_size 
Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
 2016-03-02 14:36:10 +00:00
Mark Haines
60a0f81c7a Add a /replication API for extracting the updates that happened on 
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
 2016-03-01 14:49:41 +00:00
Erik Johnston
ce2cdced61 Move cache size fiddling to descriptors only. Fix tests 2016-03-01 13:21:46 +00:00
Mark Haines
5e2890bd49 Check that the disable_registration config key is handled correctly 2016-02-22 16:01:29 +00:00
Mark Haines
7641a90c34 Add a test for TreeCache.__contains__ 2016-02-22 15:22:38 +00:00
Mark Haines
5c79ef9396 Test Filter.filter_rooms 
Also check that the __repr__ method for FilterCollection does something
sensible.
 2016-02-19 17:55:28 +00:00
Mark Haines
700487a7c7 Fix flake8 warnings for tests 2016-02-19 15:34:38 +00:00
Erik Johnston
929cb0ed7d Don't set currently_active for remote presence 2016-02-19 10:58:27 +00:00
Erik Johnston
b4796a62ee Add unit test 2016-02-18 11:52:33 +00:00
Erik Johnston
9da9826b85 Remove old tests. 2016-02-18 10:46:16 +00:00
Erik Johnston
e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Erik Johnston
9e7900da1e Add wheeltimer impl 2016-02-17 14:29:28 +00:00
Daniel Wagner-Hall
763360594d Mark AS users with their AS's ID 2016-02-11 17:26:42 +00:00
Erik Johnston
e557dc80b8 Merge pull request #566 from matrix-org/erikj/logcontext 
Don't bother copying records on parent context
 2016-02-10 11:41:45 +00:00
Erik Johnston
78a5482267 Typo 2016-02-09 16:23:11 +00:00
Erik Johnston
e664e9737c Fix test 2016-02-09 14:57:43 +00:00
Erik Johnston
13ba8d878c Fix test 2016-02-09 14:55:21 +00:00
Daniel Wagner-Hall
6a9f1209df Error if macaroon key is missing from config 
Currently we store all access tokens in the DB, and fall back to that
check if we can't validate the macaroon, so our fallback works here, but
for guests, their macaroons don't get persisted, so we don't get to
find them in the database. Each restart, we generate a new ephemeral
key, so guests lose access after each server restart.

I tried to fix up the config stuff to be less insane, but gave up, so
instead I bolt on yet another piece of custom one-off insanity.

Also, add some basic tests for config generation and loading.
 2016-02-05 01:58:23 +00:00
Daniel Wagner-Hall
66bb255fcd Merge pull request #556 from matrix-org/daniel/config 
Rename config field to reflect yaml name
 2016-02-03 14:55:54 +00:00
Daniel Wagner-Hall
5054806ec1 Rename config field to reflect yaml name 2016-02-03 14:42:01 +00:00
Erik Johnston
f8aae79a72 Simplify get_rooms 2016-02-03 13:24:35 +00:00
Mark Haines
f2d5ff5bf2 Fix the mock homserver used in the tests 2016-01-29 14:53:14 +00:00
Erik Johnston
b18114e19e Merge pull request #536 from matrix-org/erikj/sync 
Make /sync "better".
 2016-01-29 13:04:51 +00:00
Erik Johnston
02a9c3be6c Merge pull request #538 from matrix-org/erikj/fix_lru_cache 
Fix LruCache. Make TreeCache track its own size.
 2016-01-29 11:53:55 +00:00
Erik Johnston
4fce59f274 Add tests 2016-01-29 11:33:11 +00:00
Erik Johnston
35981c8b71 Fix test 2016-01-28 17:20:05 +00:00
Erik Johnston
167d1df699 Merge pull request #534 from matrix-org/erikj/setup 
Add a Homeserver.setup method
 2016-01-28 14:45:36 +00:00
Erik Johnston
4e7948b47a Allow paginating backwards from stream token 2016-01-28 11:52:34 +00:00
Erik Johnston
61eaa6ec64 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync 2016-01-28 10:50:57 +00:00
Richard van der Hoff
e26390ca46 Merge pull request #535 from matrix-org/rav/paginate_from_stream_token 
Make it possible to paginate forwards from stream tokens
 2016-01-28 09:49:46 +00:00
Richard van der Hoff
5cba88ea7c Make it possible to paginate forwards from stream tokens 
In order that we can fill the gap after a /sync, make it possible to paginate
forwards from a stream token.
 2016-01-27 17:42:45 +00:00
Erik Johnston
0487c9441f Fix tests 2016-01-27 17:33:13 +00:00
Erik Johnston
9959d9ece8 Remove redundated BaseHomeServer 2016-01-26 13:52:29 +00:00
Erik Johnston
8c6012a4af Fix tests 2016-01-25 13:12:35 +00:00
Erik Johnston
88baa3865e Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync 2016-01-22 15:57:12 +00:00
David Baker
d552861346 Revert all the bits changing keys of eeverything that used LRUCaches to tuples 2016-01-22 12:18:14 +00:00
David Baker
10f76dc5da Make LRU cache not default to treecache & add options to use it 2016-01-22 12:10:33 +00:00
David Baker
31a051b677 Test treecache directly 2016-01-22 11:22:00 +00:00
Erik Johnston
8f9c74e9f1 Fix tests 2016-01-22 10:48:27 +00:00
David Baker
4efcaa43c8 Add tests for treecache directly and test del_multi at the LruCache level too. 2016-01-22 10:37:37 +00:00
David Baker
f1f8122120 Change LRUCache to be tree-based so we can delete subtrees. 2016-01-21 19:16:25 +00:00
Erik Johnston
191070123d Cache dns lookups, and use the cache if we fail to lookup servers later 2016-01-20 11:34:09 +00:00
Erik Johnston
2c176e02ae Make unit tests work 2016-01-18 14:48:50 +00:00
Daniel Wagner-Hall
ac5a4477ad Require unbanning before other membership changes 2016-01-15 16:27:26 +00:00
Daniel Wagner-Hall
2680043bc6 Require ID and as_token be unique for ASs 
Defaults ID to as_token if not specified. This will change
when IDs are fully supported.
 2016-01-14 14:34:01 +00:00
Mark Haines
c0a279e808 Delete the table objects from TransactionStore 2016-01-13 11:15:20 +00:00
Daniel Wagner-Hall
42aa1f3f33 Merge pull request #478 from matrix-org/daniel/userobject 
Introduce a User object

I'm sick of passing around more and more things as tuple items around
the whole world, and needing to edit every call site every time there is
more information about a user. So pass them around together as an
object.

This object has incredibly poorly named fields because we have a
convention that `user` indicates a UserID object, and `user_id`
indicates a string. I tried to clean up the whole repo to fix this, but
gave up. So instead, I introduce a second convention. A user_object is a
User, and a user_id_object is a UserId. I may have cried a little bit.
 2016-01-11 17:50:22 +00:00
Daniel Wagner-Hall
2110e35fd6 Introduce a Requester object 
This tracks data about the entity which made the request. This is
instead of passing around a tuple, which requires call-site
modifications every time a new piece of optional context is passed
around.

I tried to introduce a User object. I gave up.
 2016-01-11 17:48:45 +00:00
David Baker
c232780081 Merge pull request #456 from matrix-org/store_event_actions 
Send unread notification counts
 2016-01-08 14:47:15 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
David Baker
0e48f7f245 fix tests 2016-01-06 16:46:41 +00:00
David Baker
eb03625626 Merge remote-tracking branch 'origin/develop' into store_event_actions 2016-01-05 18:39:50 +00:00
Daniel Wagner-Hall
cfd07aafff Allow guests to upgrade their accounts 2016-01-05 18:01:18 +00:00
David Baker
92a1e74b20 fix tests 2016-01-04 14:17:35 +00:00
David Baker
928c575c6f Merge remote-tracking branch 'origin/develop' into store_event_actions 2016-01-04 13:39:51 +00:00
Mark Haines
7fa71e3267 Add a unit test for the snapshot cache 2015-12-23 11:48:03 +00:00
David Baker
d79e90f078 Add mocks to make tests work again 2015-12-22 17:56:56 +00:00
Mark Haines
d547afeae0 Merge remote-tracking branch 'origin/master' into develop 2015-12-07 13:13:43 +00:00
Oleg Girko
4013216fcc Fix mock import in tests. 
For some reason, one test imports Mock class from mock.mock
rather than from mock.
This change fixes this error.

Signed-off-by: Oleg Girko <ol@infoserver.lv>
 2015-12-06 20:50:11 +00:00
Daniel Wagner-Hall
14d7acfad4 Host /unstable and /r0 versions of r0 APIs 2015-12-01 17:34:32 +00:00
Mark Haines
7b593af7e1 rename the method in the tests as well 2015-12-01 16:06:17 +00:00
Erik Johnston
bd3de8f39a Update tests 2015-11-17 18:38:48 +00:00
Daniel Wagner-Hall
468a2ed4ec Return non-room events from guest /events calls 2015-11-12 16:45:28 +00:00
Richard van der Hoff
78f6010207 Fix an issue with ignoring power_level changes on divergent graphs 
Changes to m.room.power_levels events are supposed to be handled at a high
priority; however a typo meant that the relevant bit of code was never
executed, so they were handled just like any other state change - which meant
that a bad person could cause room state changes by forking the graph from a
point in history when they were allowed to do so.
 2015-11-12 15:24:59 +00:00
Erik Johnston
e21cef9bb5 Merge pull request #359 from matrix-org/markjh/incremental_indexing 
Incremental background updates for db indexes
 2015-11-11 17:19:51 +00:00
Daniel Wagner-Hall
cf437900e0 Return world_readable and guest_can_join in /publicRooms 2015-11-10 17:10:27 +00:00
Mark Haines
36c58b18a3 Test for background updates 2015-11-10 15:51:40 +00:00
Richard van der Hoff
9107ed23b7 Add a couple of unit tests for room/<x>/messages 
... merely because I was trying to figure out how it worked, and couldn't.
 2015-11-09 16:16:43 +00:00
Daniel Wagner-Hall
ca2f90742d Open up /events to anonymous users for room events only 
Squash-merge of PR #345 from daniel/anonymousevents
 2015-11-05 14:32:26 +00:00
Daniel Wagner-Hall
4e62ffdb21 Merge branch 'develop' of github.com:matrix-org/synapse into develop 2015-11-04 17:31:01 +00:00
Daniel Wagner-Hall
f522f50a08 Allow guests to register and call /events?room_id= 
This follows the same flows-based flow as regular registration, but as
the only implemented flow has no requirements, it auto-succeeds. In the
future, other flows (e.g. captcha) may be required, so clients should
treat this like the regular registration flow choices.
 2015-11-04 17:29:07 +00:00
Mark Haines
33b3e04049 Merge branch 'develop' into daniel/removesomelies 
Conflicts:
	synapse/notifier.py
 2015-11-04 16:01:00 +00:00
Mark Haines
c452dabc3d Remove the LockManager class because it wasn't being used 2015-11-04 14:08:15 +00:00
Daniel Wagner-Hall
771ca56c88 Remove more unused parameters 2015-11-02 15:31:57 +00:00
Mark Haines
0e36756383 Merge branch 'develop' into markjh/room_tags 2015-11-02 10:57:00 +00:00
Mark Haines
fb46937413 Support clients supplying older tokens, fix short poll test 2015-10-30 16:38:35 +00:00
Richard van der Hoff
f69a5c9134 Fix a 500 error resulting from empty room_ids 
POST /_matrix/client/api/v1/rooms//send/a.b.c gave a 500 error, because we
assumed that rooms always had at least one character.
 2015-10-26 18:44:03 +00:00
Erik Johnston
b0ac0a9438 Merge pull request #319 from matrix-org/erikj/filter_refactor 
Refactor api.filtering to have a Filter API
 2015-10-22 13:17:10 +01:00
Mark Haines
87da71bace Merge pull request #314 from matrix-org/paul/event-redaction 
Add some unit tests of prune_events()
 2015-10-22 11:07:20 +01:00
Erik Johnston
45cd2b0233 Refactor api.filtering to have a Filter API 2015-10-20 15:33:25 +01:00
Daniel Wagner-Hall
f510586372 Merge branch 'develop' of github.com:matrix-org/synapse into develop 2015-10-20 12:00:22 +01:00
Daniel Wagner-Hall
137fafce4e Allow rejecting invites 
This is done by using the same /leave flow as you would use if you had
already accepted the invite and wanted to leave.
 2015-10-20 11:58:58 +01:00
Paul "LeoNerd" Evans
9ed784098a Invoke EventBuilder directly instead of going via the EventBuilderFactory 2015-10-19 17:42:34 +01:00
Paul "LeoNerd" Evans
531e3aa75e Capture __init__.py 2015-10-19 17:37:35 +01:00
Paul "LeoNerd" Evans
a8795c9644 Use assertIn() instead of assertTrue on the 'in' operator 2015-10-19 15:24:49 +01:00
Paul "LeoNerd" Evans
07b58a431f Another signing test vector using an 'm.room.message' with content, so that the implementation will have to redact it 2015-10-19 15:00:52 +01:00
Paul "LeoNerd" Evans
0aab34004b Initial minimial hack at a test of event hashing and signing 2015-10-19 14:40:15 +01:00
Paul "LeoNerd" Evans
aff4d850bd Add some unit tests of prune_events() 2015-10-16 19:56:46 +01:00
Mark Haines
8897781558 update filtering tests 2015-10-13 14:13:51 +01:00
Erik Johnston
17c80c8a3d rename schema_prepare to prepare_database 2015-10-13 13:56:22 +01:00
Erik Johnston
ec398af41c Expose error more nicely 2015-10-13 11:43:43 +01:00
Erik Johnston
d5e081c7ae Merge branch 'develop' of github.com:matrix-org/synapse into erikj/unfederatable 2015-10-02 10:33:49 +01:00
Mark Haines
1ee3d26432 synapse/storage/_base.py:_simple_selectupdate_one was unused 2015-09-23 10:30:03 +01:00
Mark Haines
bb4dddd6c4 Move NullSource out of synapse and into tests since it is only used by the tests 2015-09-22 18:33:34 +01:00
Daniel Wagner-Hall
7213588083 Implement configurable stats reporting 
SYN-287

This requires that HS owners either opt in or out of stats reporting.

When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
 2015-09-22 12:57:40 +01:00
Mark Haines
ee2d722f0f Merge pull request #276 from matrix-org/markjh/history_for_rooms_that_have_been_left 
SPEC-216: Allow users to view the history of rooms that they have left.
 2015-09-21 14:38:13 +01:00
Daniel Wagner-Hall
728d07c8c1 Merge pull request #256 from matrix-org/auth 
Attempt to validate macaroons
 2015-09-14 18:09:33 +01:00
Mark Haines
e2054ce21a Allow users to GET individual state events for rooms that they have left 2015-09-10 15:06:47 +01:00
Mark Haines
bc8b25eb56 Allow users that have left the room to view the member list from the point they left 2015-09-09 15:42:16 +01:00
Erik Johnston
187320b019 Merge branch 'erikj/check_room_exists' into erikj/unfederatable 2015-09-01 15:58:10 +01:00
Erik Johnston
b345853918 Check against sender rather than event_id 2015-09-01 15:57:35 +01:00
Erik Johnston
00149c063b Fix tests 2015-09-01 15:42:03 +01:00
Daniel Wagner-Hall
e255c2c32f s/user_id/user/g for consistency 2015-09-01 12:41:16 +01:00
Daniel Wagner-Hall
3063383547 Swap out bcrypt for md5 in tests 
This reduces our ~8 second sequential test time down to ~7 seconds
 2015-08-26 15:59:32 +01:00
Daniel Wagner-Hall
81450fded8 Turn TODO into thing which actually will fail 2015-08-26 13:56:01 +01:00
Daniel Wagner-Hall
6a4b650d8a Attempt to validate macaroons 
A couple of weird caveats:
 * If we can't validate your macaroon, we fall back to checking that
   your access token is in the DB, and ignoring the failure
 * Even if we can validate your macaroon, we still have to hit the DB to
   get the access token ID, which we pretend is a device ID all over the
   codebase.

This mostly adds the interesting code, and points out the two pieces we
need to delete (and necessary conditions) in order to fix the above
caveats.
 2015-08-26 13:22:23 +01:00
Daniel Wagner-Hall
a9d8bd95e7 Stop looking up "admin", which we never read 2015-08-25 16:29:39 +01:00
Daniel Wagner-Hall
a0b181bd17 Remove completely unused concepts from codebase 
Removes device_id and ClientInfo

device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.

This doesn't remove the columns from the database, because that's
fiddly.
 2015-08-25 16:23:06 +01:00
Daniel Wagner-Hall
e8cf77fa49 Merge branch 'develop' into refresh 
Conflicts:
	synapse/rest/client/v1/login.py
 2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall
cecbd636e9 /tokenrefresh POST endpoint 
This allows refresh tokens to be exchanged for (access_token,
refresh_token).

It also starts issuing them on login, though no clients currently
interpret them.
 2015-08-20 16:21:35 +01:00
Daniel Wagner-Hall
13a6517d89 s/by_token/by_access_token/g 
We're about to have two kinds of token, access and refresh
 2015-08-20 16:01:29 +01:00
Daniel Wagner-Hall
617501dd2a Move token generation to auth handler 
I prefer the auth handler to worry about all auth, and register to call
into it as needed, than to smatter auth logic between the two.
 2015-08-20 11:35:56 +01:00
Mark Haines
a0b8e5f2fe Merge pull request #211 from matrix-org/email_in_use 
Changes for unique emails
 2015-08-20 10:04:04 +01:00
Daniel Wagner-Hall
70e265e695 Re-add whitespace around caveat operators 2015-08-19 14:30:31 +01:00
Daniel Wagner-Hall
ce832c38d4 Remove padding space around caveat operators 2015-08-18 17:39:26 +01:00
Daniel Wagner-Hall
42e858daeb Fix units in test 
I made the non-test seconds instead of ms, but not the test
 2015-08-18 17:38:37 +01:00
Daniel Wagner-Hall
3e6fdfda00 Fix some formatting to use tuples 2015-08-18 15:18:50 +01:00
Daniel Wagner-Hall
1469141023 Merge branch 'develop' into auth 2015-08-18 14:43:44 +01:00
Daniel Wagner-Hall
2d3462714e Issue macaroons as opaque auth tokens 
This just replaces random bytes with macaroons. The macaroons are not
inspected by the client or server.

In particular, they claim to have an expiry time, but nothing verifies
that they have not expired.

Follow-up commits will actually enforce the expiration, and allow for
token refresh.

See https://bit.ly/matrix-auth for more information
 2015-08-18 14:22:02 +01:00
Erik Johnston
d7451e0f22 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-12 10:30:30 +01:00
Erik Johnston
2df8dd9b37 Move all the caches into their own package, synapse.util.caches 2015-08-11 18:00:59 +01:00
Daniel Wagner-Hall
4ff0228c25 Remove call to recently removed function in mock 2015-08-11 16:56:30 +01:00
Daniel Wagner-Hall
dcd5983fe4 Remove call to recently removed function in mock 2015-08-11 16:54:06 +01:00
Erik Johnston
20addfa358 Change Cache to not use *args in its interface 2015-08-07 18:32:47 +01:00
Erik Johnston
b8e386db59 Change Cache to not use *args in its interface 2015-08-07 11:52:21 +01:00
Erik Johnston
f596ff402e Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache 2015-08-06 13:37:56 +01:00
Erik Johnston
2efb93af52 Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds 2015-08-06 13:35:28 +01:00
Erik Johnston
7eea3e356f Make @cached cache deferreds rather than the deferreds' values 2015-08-06 13:33:34 +01:00
Erik Johnston
07507643cb Use dictionary cache to do group -> state fetching 2015-08-05 15:11:42 +01:00
Erik Johnston
c67ba143fa Move DictionaryCache 2015-08-04 15:58:28 +01:00
Erik Johnston
e7768e77f5 Add basic dictionary cache 2015-08-04 15:56:56 +01:00
David Baker
07ad03d5df Fix tests 2015-08-04 15:18:40 +01:00
Kegan Dougal
11b0a34074 Use the same reg paths as register v1 for ASes. 
Namely this means using registration_handler.appservice_register.
 2015-07-29 10:00:54 +01:00
Kegan Dougal
a4d62ba36a Fix v2_alpha registration. Add unit tests. 
V2 Registration forced everyone (including ASes) to create a password for a
user, when ASes should be able to omit passwords. Also unbreak AS registration
in general which checked too early if the given username was claimed by an AS;
it was checked before knowing if the AS was the one doing the registration! Add
unit tests for AS reg, user reg and disabled_registration flag.
 2015-07-28 17:34:12 +01:00
Erik Johnston
f0dd6d4cbd Fix test. 2015-07-07 16:18:36 +01:00
Erik Johnston
bd1236c0ee Consolidate duplicate code in notifier 2015-07-02 11:46:05 +01:00
Erik Johnston
ddf7979531 Add receipts_key to StreamToken 2015-07-02 11:45:44 +01:00
Kegan Dougal
d88e20cdb9 Fix bug where synapse was sending AS user queries incorrectly. 
Bug introduced in 92b20713d7
which reversed the comparison when checking if a user existed
in the users table. Added UTs to prevent this happening again.
 2015-06-17 17:26:03 +01:00
Erik Johnston
261ccd7f5f Fix tests 2015-06-12 17:17:29 +01:00
Erik Johnston
3bbd0d0e09 Merge pull request #180 from matrix-org/erikj/prev_state_context 
Don't needlessly compute prev_state
 2015-06-03 17:20:56 +01:00
Erik Johnston
9dda396baa Merge pull request #179 from matrix-org/erikj/state_group_outliers 
Don't compute EventContext for outliers.
 2015-06-03 17:20:40 +01:00
Erik Johnston
55bf90b9e4 Don't needlessly compute prev_state 2015-06-03 16:44:24 +01:00
Erik Johnston
1c3d844e73 Don't needlessly compute context 2015-06-03 16:41:51 +01:00
Erik Johnston
d8866d7277 Caches should be bound to instances. 
Before, caches were global and so different instances of the stores
would share caches. This caused problems in the unit tests.
 2015-06-03 14:45:17 +01:00
Mark Haines
17167898c8 Fix the presence tests 2015-05-22 16:22:54 +01:00
Mark Haines
49a2c10279 Merge pull request #157 from matrix-org/markjh/presence_performance 
Improve presence performance in loadtest
 2015-05-22 16:04:40 +01:00
Mark Haines
f43544eecc Make the appservice use 'users_in_room' rather than get_room_members since it is cached 2015-05-22 11:01:28 +01:00
Mark Haines
88f1ea36ce Oops, get_rooms_for_user returns a namedtuple, not a room_id 2015-05-21 15:23:58 +01:00
Erik Johnston
a551c5dad7 Merge pull request #155 from matrix-org/erikj/perf 
Bulk and batch retrieval of events.
 2015-05-21 14:54:40 +01:00
Mark Haines
8eca5bd50a Fix the presence tests 2015-05-20 13:22:18 +01:00
Mark Haines
0b0033c40b Merge branch 'develop' into notifier_performance 2015-05-18 13:50:01 +01:00
Erik Johnston
131bdf9bb1 Merge branch 'erikj/events_move' of github.com:matrix-org/synapse into erikj/perf 2015-05-18 10:23:37 +01:00
Erik Johnston
a2c4f3f150 Fix daedlock 2015-05-15 10:54:04 +01:00
Mark Haines
c5d1b4986b Remove unused arguments and doc PresenceHandler.push_update_to_clients 2015-05-14 14:59:31 +01:00
Mark Haines
f1b83d88a3 Discard unused NotifierUserStreams 2015-05-13 16:54:02 +01:00
Mark Haines
63878c0379 Don't bother checking for updates if the stream token hasn't advanced for a user 2015-05-13 13:42:21 +01:00
Erik Johnston
43c2e8deae Add support for using executemany 2015-05-05 15:13:25 +01:00
Erik Johnston
327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston
6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
David Baker
6532b6e607 Merge branch 'develop' into csauth 
Conflicts:
	synapse/http/server.py
 2015-04-24 09:37:54 +01:00
Paul "LeoNerd" Evans
e1e5e53127 Remove users from the remote_offline_serials list (and clean up empty elements) when they go online again 2015-04-23 19:01:37 +01:00
Paul "LeoNerd" Evans
8a785c3006 Store a list of the presence serial number at which remote users went offline, so that when we delete them from the cachemap, we can still synthesize OFFLINE events for them (SYN-261) 2015-04-23 18:40:19 +01:00
David Baker
cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
Erik Johnston
6ba2e3df4e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-17 11:22:31 +01:00
Paul "LeoNerd" Evans
04c7f3576e Various minor fixes to unit-test structure around typing notifications 2015-04-15 23:32:11 +01:00
Paul "LeoNerd" Evans
0268d40281 Have TypingNotificationEventSource.get_new_events_for_user() return a deferred, for consistency and extensibility 2015-04-15 23:09:35 +01:00
Erik Johnston
22d7a59306 Fix tests after commit 9a0579 2015-04-08 16:57:14 +01:00
Erik Johnston
4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Erik Johnston
9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal
9fa8bda099 Merge branch 'develop' into application-services-registration-script 2015-04-01 10:19:17 +01:00
Kegan Dougal
09cbff174a Fix thinko whereby events *for the AS specifically* were not passed on. 
This was caused by not explicitly checking the service.sender field. This
has now been fixed and a regression test has been added.
 2015-03-31 16:44:45 +01:00
Kegan Dougal
cf1fa59f4b Use a sender localpart instead of a user ID. 
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
 2015-03-31 13:48:03 +01:00
Kegan Dougal
c217504949 Edit SQL schema to use string IDs not ints. Use token as ID. Update tests. 2015-03-31 12:07:56 +01:00
Kegan Dougal
b59aa74556 Fix tests and missing returns on deferreds. 2015-03-31 11:35:45 +01:00
Kegan Dougal
d33ae65efc Remove more reg/unreg methods. Read config not database for cache. 2015-03-31 11:00:00 +01:00
Kegan Dougal
4edcbcee3b Merge branch 'develop' into application-services-txn-reliability 
Conflicts:
	synapse/storage/__init__.py
 2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans
9ba6487b3f Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict() 2015-03-25 19:05:34 +00:00
Paul "LeoNerd" Evans
7ab9f91a60 Unit-test that Cache() key eviction is ordered 2015-03-25 18:50:43 +00:00
David Baker
ce2766d19c Fix tests 2015-03-24 18:56:51 +00:00
David Baker
438a21c87b Don't test exact equality of the list: as long as it has the fields we expect, that's just fine. I added the user_id (as in database pkey) and it broke: no point testing what that comes out as: it's determined by the db. 2015-03-24 18:21:54 +00:00
Paul "LeoNerd" Evans
0f86312c4c Pull out the cache logic from the @cached wrapper into its own class we can reuse 2015-03-20 18:25:42 +00:00
Erik Johnston
7e282a53a5 Tidy up _simple_... methods 2015-03-20 15:05:10 +00:00
Kegan Dougal
d04fa1f712 Implement ServiceQueuer with tests. 2015-03-16 14:03:16 +00:00
Kegan Dougal
6279285b2a Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer. 2015-03-16 13:15:40 +00:00
Kegan Dougal
835e01fc70 Minor PR comment tweaks. 2015-03-16 10:16:59 +00:00
Kegan Dougal
f9232c7917 Merge branch 'develop' into application-services-txn-reliability 
Conflicts:
	synapse/storage/appservice.py
 2015-03-16 10:09:15 +00:00
Paul "LeoNerd" Evans
c1cdd7954d Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f1fbe3e09f Rename TimerMetric to DistributionMetric; as it could count more than just time 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
cbc0406be8 Export CacheMetric as hits+total, rather than hits+misses, as it's easier to derive hit ratio from that 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
0e847540c3 Prometheus needs "escaped" label values 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b3a0179d64 Bugfix to rendering output of vectored TimerMetrics 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f9478e475b Rename Metrics' "keys" to "labels" 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
72625f2f4d Initial hack at a TimerMetric; for storing counts + duration accumulators 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
23ab0c68c2 Implement vector CallbackMetrics 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
8664599af7 Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
d8caa5454d Initial attempt at a scalar callback-based metric to give instantaneous snapshot gauges 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
ce8b5769f7 Create the concept of a cachecounter metric; generating two counters specific to caches 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e7420a3bef Initial tiny attempt at (vectorable) counter metrics 2015-03-12 16:24:50 +00:00
Kegan Dougal
b98cd03193 Use event IDs instead of dumping event content in the txns table. 2015-03-09 17:25:20 +00:00
Kegan Dougal
21fd84dcb8 Use seconds; start gluing in the AS scheduler into the AS handler. 2015-03-09 17:01:19 +00:00
Erik Johnston
6d74e46621 Fix tests 2015-03-09 17:01:11 +00:00
Kegan Dougal
0a60bbf4fa Finish appservice txn storage impl and tests. 2015-03-09 15:53:03 +00:00
Kegan Dougal
1ead1caa18 Implement create_appservice_txn with tests. 2015-03-09 13:54:20 +00:00
Kegan Dougal
1c2dcf762a Partially implement txn store methods with tests. 2015-03-09 13:10:31 +00:00
Kegan Dougal
10766f1e93 Update UTs 2015-03-06 16:17:01 +00:00
Kegan Dougal
2602ddc379 Apply clarity and docstrings 2015-03-06 16:16:14 +00:00
Kegan Dougal
0354659f9d Finish synapse.appservice.scheduler implementation. 
With tests to assert behaviour. Not hooked up yet. Stub datastore methods
not implemented yet.
 2015-03-06 16:09:05 +00:00
Kegan Dougal
f260cb72cd Flesh out more stub functions. 2015-03-06 15:12:24 +00:00
Kegan Dougal
0fbfe1b08a Add more tests; fix bugs. 2015-03-06 14:36:52 +00:00
Kegan Dougal
192e228a98 Start adding some tests 2015-03-06 11:50:27 +00:00
Kegsay
8ad024ea80 Merge pull request #93 from matrix-org/application-services-exclusive 
Application services exclusive flag support
 2015-03-02 14:56:32 +00:00
Erik Johnston
0a036944bd Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu 2015-03-02 13:53:30 +00:00
Kegan Dougal
de190e49d5 Add more unit tests for exclusive namespaces. 2015-02-27 11:51:06 +00:00
Kegan Dougal
127efeeb68 Update unit tests to use new format. 2015-02-27 11:10:48 +00:00
Kegan Dougal
0ebd632d39 Fix unit tests 2015-02-27 09:46:38 +00:00
Paul "LeoNerd" Evans
f53fcbce97 Use cache.pop() instead of a separate membership test + del [] 2015-02-23 18:30:45 +00:00
Paul "LeoNerd" Evans
bd84755e64 Merge remote-tracking branch 'origin/develop' into performance-cache-improvements 2015-02-23 17:16:03 +00:00
Paul "LeoNerd" Evans
e76d485e29 Allow @cached-wrapped functions to have a prefill method for setting entries 2015-02-23 15:41:54 +00:00
Erik Johnston
852816befe Fix presence tests 2015-02-23 15:14:09 +00:00
Paul "LeoNerd" Evans
ebc3db295b Take named arguments to @cached() decorator, add a 'max_entries' limit 2015-02-19 18:36:02 +00:00
Erik Johnston
5f4c28d313 Update tests 2015-02-19 14:34:32 +00:00
Erik Johnston
5358966a87 Use git aware version string in User-Agent and Server headers 2015-02-18 16:52:04 +00:00
Mark Haines
0d872f5aa6 Merge pull request #50 from matrix-org/application-services 
Application Services
 2015-02-13 15:06:14 +00:00
Erik Johnston
021d93db11 Merge pull request #62 from matrix-org/state-chache 
State chache
 2015-02-11 17:10:28 +00:00
Kegan Dougal
cb43fbeeb4 Fix tests which broke when event caching was introduced. 2015-02-11 16:46:01 +00:00
Kegan Dougal
f2fdcb7c4b Merge branch 'develop' into application-services 2015-02-11 16:43:26 +00:00
Mark Haines
f5a70e0d2e Add a cache for get_event 2015-02-11 15:01:15 +00:00
Mark Haines
d8324d5a2b Add a lru cache class 2015-02-11 14:52:23 +00:00
Mark Haines
896253e085 Factor out some of the common homeserver setup code into a 
setup_test_homeserver function in utils.
 2015-02-11 11:37:30 +00:00
Kegan Dougal
c7783d6fee Notify ASes for events sent by other users in a room which an AS user is a part of. 2015-02-11 10:36:08 +00:00
Erik Johnston
164f6b9256 Fix tests 2015-02-09 14:23:57 +00:00
Kegan Dougal
ab3c897ce1 Remove unused imports. 2015-02-09 14:16:36 +00:00
Kegan Dougal
5a7dd05818 Modify auth.get_user_by_req for authing appservices directly. 
Add logic to map the appservice token to the autogenned appservice user ID.
Add unit tests for all forms of get_user_by_req (user/appservice,
valid/bad/missing tokens)
 2015-02-09 14:14:15 +00:00
Kegan Dougal
11e6b3d18b Dependency inject ApplicationServiceApi when creating ApplicationServicesHandler. 2015-02-05 17:04:59 +00:00
Kegan Dougal
951690e54d Merge branch 'develop' into application-services 2015-02-05 14:28:03 +00:00
Kegan Dougal
131e036402 Fix unit tests. 2015-02-05 13:22:20 +00:00
Kegan Dougal
bc658907f0 Add unit test for appservice_handler.query_room_alias_exists 2015-02-05 11:54:36 +00:00
Kegan Dougal
f0c730252f Add unknown user ID check. Use store.get_aliases_for_room(room_id) when searching for services by alias. 2015-02-05 11:25:32 +00:00
Kegan Dougal
27091f146a Add hs_token column and generate a different token f.e application service. 2015-02-05 10:08:12 +00:00
Kegan Dougal
89f2e8fbdf Fix bug in store defer. Add more unit tests. 2015-02-04 15:21:03 +00:00
Kegan Dougal
525a218b2b Begin to add unit tests for appservice glue and regex testing. 2015-02-04 12:24:20 +00:00
Erik Johnston
e7ca813dd4 Try to ensure we don't persist an event we have already persisted. In persist_event check if we already have the event, if so then update instead of replacing so that we don't cause a bump of the stream_ordering. 2015-02-03 10:39:41 +00:00
Erik Johnston
1bd540ef79 Merge branch 'develop' of github.com:matrix-org/synapse into rejections 
Conflicts:
	synapse/storage/schema/im.sql
 2015-01-30 15:16:38 +00:00
Erik Johnston
471c47441d Merge pull request #37 from matrix-org/client_v2_filter 
Client v2 filter
 2015-01-30 14:56:08 +00:00
Erik Johnston
2aaedab203 Merge branch 'develop' of github.com:matrix-org/synapse into new_state_resolution 2015-01-30 14:09:32 +00:00
Erik Johnston
2ebf795c0a Merge branch 'develop' of github.com:matrix-org/synapse into rejections 
Conflicts:
	synapse/storage/__init__.py
	synapse/storage/schema/delta/v12.sql
 2015-01-30 11:10:37 +00:00
Mark Haines
93ed31dda2 Create a separate filter object to do the actual filtering, so that we can 
split the storage and management of filters from the actual filter code
and don't have to load a filter from the db each time we filter an event
 2015-01-29 17:45:07 +00:00
Erik Johnston
78015948a7 Initial implementation of auth conflict resolution 2015-01-29 16:52:33 +00:00
Mark Haines
33391db5f8 Merge in auth changes from develop 2015-01-29 15:54:54 +00:00