Travis Ralston
923d9300ed
Add a blurb explaining the main synapse worker
...
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-02-17 21:53:46 -07:00
Richard van der Hoff
a71a080cd2
Merge pull request #2882 from matrix-org/rav/fix_purge_tablescan
...
(Really) fix tablescan of event_push_actions on purge
2018-02-16 15:18:47 +00:00
Richard van der Hoff
d1a3325f99
(Really) fix tablescan of event_push_actions on purge
...
commit 278d21b5
2018-02-16 14:02:31 +00:00
Erik Johnston
bf5ef10a93
Merge pull request #2874 from matrix-org/erikj/creator_push_actions
...
Store push actions in DB staging area instead of context
2018-02-16 11:52:51 +00:00
Erik Johnston
6af025d3c4
Fix typo of double is_highlight
2018-02-16 11:35:31 +00:00
Erik Johnston
012e8e142a
Comments
2018-02-16 11:35:01 +00:00
Erik Johnston
3a061cae26
Fix unit test
2018-02-15 16:31:59 +00:00
Erik Johnston
b96278d6fe
Ensure that we delete staging push actions on errors
2018-02-15 15:47:06 +00:00
Erik Johnston
4810f7effd
Remove context.push_actions
2018-02-15 15:47:06 +00:00
Erik Johnston
c714c61853
Update event_push_actions table from staging table
2018-02-15 15:47:06 +00:00
Erik Johnston
acac21248c
Store push actions in staging area
2018-02-15 15:47:04 +00:00
Erik Johnston
6ed9ff69c2
Merge pull request #2873 from matrix-org/erikj/event_creator_no_state
...
Don't serialize current state over replication
2018-02-15 14:09:42 +00:00
Erik Johnston
106906a65e
Don't serialize current state over replication
2018-02-15 13:53:18 +00:00
Erik Johnston
5fb347fc41
Merge pull request #2872 from matrix-org/erikj/event_worker_dont_log
...
Don't log errors propogated from send_event
2018-02-15 12:31:49 +00:00
Erik Johnston
cd94728e93
Merge pull request #2871 from matrix-org/erikj/event_creator_state
...
Fix state group storage bug in workers
2018-02-15 11:34:01 +00:00
Erik Johnston
fd1601c596
Fix state group storage bug in workers
...
We needed to move `_count_state_group_hops_txn` to the
StateGroupWorkerStore.
2018-02-15 11:04:32 +00:00
Erik Johnston
ef344b10e5
Don't log errors propogated from send_event
2018-02-15 11:03:49 +00:00
Richard van der Hoff
b8d821aa68
Merge pull request #2867 from matrix-org/rav/rework_purge
...
purge_history cleanups
2018-02-15 09:49:07 +00:00
Richard van der Hoff
d28ec43e15
Merge pull request #2769 from matrix-org/matthew/hit_the_gin
...
switch back from GIST to GIN indexes
2018-02-14 16:59:03 +00:00
Richard van der Hoff
39bf47319f
purge_history: fix sqlite syntax error
...
apparently sqlite insists on indexes being named
2018-02-14 16:42:19 +00:00
Richard van der Hoff
ac27f6a35e
purge_history: handle sqlite asshattery
...
apparently creating a temporary table commits the transaction. because that's a
useful thing.
2018-02-14 16:41:12 +00:00
Richard van der Hoff
5978dccff0
remove overzealous exception handling
2018-02-14 15:54:09 +00:00
Richard van der Hoff
278d21b5e4
purge_history: fix index use
...
event_push_actions doesn't have an index on event_id, so we need to specify
room_id.
2018-02-14 15:44:51 +00:00
Richard van der Hoff
5fcbf1e07c
Rework event purge to use a temporary table
...
... which should speed things up by reducing the amount of data being shuffled
across the connection
2018-02-14 11:02:22 +00:00
Erik Johnston
c0c9327fe0
Merge pull request #2854 from matrix-org/erikj/event_create_worker
...
Create a worker for event creation
2018-02-13 18:07:10 +00:00
Erik Johnston
059d3a6c8e
Update docs
2018-02-13 17:53:56 +00:00
Richard van der Hoff
d627174da2
Fix log message in purge_history
...
(we don't just remove remote events)
2018-02-13 16:51:21 +00:00
Richard van der Hoff
ddb6a79b68
Merge branch 'matthew/gin_work_mem' into matthew/hit_the_gin
2018-02-13 16:45:36 +00:00
Richard van der Hoff
0b27ae8dc3
move search reindex to schema 47
...
We're up to schema v47 on develop now, so this will have to go in there to have
an effect.
This might cause an error if somebody has already run it in the v46 guise, and
runs it again in the v47 guise, because it will cause a duplicate entry in the
bbackground_updates table. On the other hand, the entry is removed once it is
complete, and it is unlikely that anyone other than matrix.org has run it on
v46. The update itself is harmless to re-run because it deliberately copes with
the index already existing.
2018-02-13 16:44:46 +00:00
Richard van der Hoff
4a6d551704
GIN reindex: Fix syntax errors, improve exception handling
2018-02-13 16:44:46 +00:00
Richard van der Hoff
bfdf7b9237
Merge pull request #2864 from matrix-org/rav/persist_event_caching
...
Use StateResolutionHandler to resolve state in persist_events
2018-02-13 14:45:57 +00:00
Richard van der Hoff
630caf8a70
style nit
2018-02-13 14:29:22 +00:00
Richard van der Hoff
8fd1a32456
Fix typos in purge api & doc
...
* It's supposed to be purge_local_events, not ..._history
* Fix the doc to have valid json
2018-02-13 13:09:39 +00:00
Richard van der Hoff
4d09366656
Merge pull request #2695 from okurz/feature/allow_recent_pysaml
...
Allow use of higher versions of saml2
2018-02-13 12:24:08 +00:00
Richard van der Hoff
a9b712e9dc
Merge branch 'develop' into matthew/gin_work_mem
2018-02-13 12:16:01 +00:00
Erik Johnston
32c7b8e48b
Update workers docs to include http port
2018-02-12 17:21:23 +00:00
Erik Johnston
1026690cd2
Merge pull request #2857 from matrix-org/erikj/upload_store
...
Tell storage providers about new file so they can upload
2018-02-12 13:52:58 +00:00
Richard van der Hoff
10b34dbb9a
Merge pull request #2858 from matrix-org/rav/purge_updates
...
delete_local_events for purge_room_history
2018-02-09 14:11:00 +00:00
Richard van der Hoff
39a6b35496
purge: move room_depth update to end
...
... to avoid locking the table for too long
2018-02-09 13:07:41 +00:00
Richard van der Hoff
74fcbf741b
delete_local_events for purge_history
...
Add a flag which makes the purger delete local events
2018-02-09 13:07:41 +00:00
Richard van der Hoff
e571aef06d
purge: Move cache invalidation to more appropriate place
...
it was a bit of a non-sequitur there
2018-02-09 13:07:41 +00:00
Richard van der Hoff
61ffaa8137
bump purge logging to info
...
this thing takes ages and the only sign of any progress is the logs, so having
some logs is useful.
2018-02-09 13:07:41 +00:00
Richard van der Hoff
671540dccf
rename delete_old_state -> purge_history
...
(beacause it deletes more than state)
2018-02-09 13:07:41 +00:00
Erik Johnston
5fa571a91b
Tell storage providers about new file so they can upload
2018-02-07 13:35:08 +00:00
Erik Johnston
053255f36c
Merge pull request #2856 from matrix-org/erikj/remove_ratelimit
...
Remove pointless ratelimit check
2018-02-07 11:12:14 +00:00
Erik Johnston
f133228cb3
Add note in docs/workers.rst
2018-02-07 10:34:31 +00:00
Erik Johnston
50fe92cd26
Move presence handling into handle_new_client_event
...
As we want to have it run on the main synapse instance
2018-02-07 10:34:09 +00:00
Erik Johnston
8ec2e638be
Add event_creator worker
2018-02-07 10:32:32 +00:00
Erik Johnston
24dd73028a
Add replication http endpoint for event sending
2018-02-07 10:32:32 +00:00
Erik Johnston
e3624fad5f
Remove pointless ratelimit check
...
The intention was for the check to be called as early as possible in the
request, but actually was called just before the main ratelimit check,
so was fairly pointless.
2018-02-07 10:30:25 +00:00
