98 commits

Author	SHA1	Message	Date
Sumner Evans	d898aefff1	backfill loops: combine immediate and deferred loops	2022-04-28 15:48:38 -06:00
Sumner Evans	ac7a437ad8	backfill: add more logging to the queue	2022-04-28 15:48:04 -06:00
Sumner Evans	f32c5f2e9a	backfill: fix priority of backfills again ... I think this finally has the correct logic for doing each stage for each portal before going on to the next stage.	2022-04-23 21:50:47 -06:00
Sumner Evans	a1e2e673dc	backfill: fix bridging of ephemeral and view-once messages	2022-04-22 14:07:59 -06:00
Sumner Evans	e74bb62fc1	backfill logging: fix logging of Backfill structs	2022-04-19 11:19:42 -06:00
Sumner Evans	f2e762680c	media backfill: add ability to automatically request media not on WA server ... This adds a new backfill type for media that sends a request to the phone for every media that is not available on the WA servers. WA deletes media from their servers after about two weeks, so you have to ask the phone to re-upload it. In order to use this, you need to enable bridge.history_sync.backfill_media and configure the requests that will be made per portal using bridge.history_sync.media (which is similar to the deferred backfill config). If you already have backfilled portals, but want to do a one-off media backfill for all existing portals, you can set bridge.history_sync.enqueue_backfill_media_next_start to true.	2022-04-19 08:12:51 -06:00
Tulir Asokan	c98b7f32c9	Fix handling errors from batch sending	2022-04-19 11:39:17 +03:00
Tulir Asokan	2c68ccc0a3	Clean up some backfill things	2022-04-18 20:12:24 +03:00
Tulir Asokan	1250e2d605	More backfill fixes	2022-04-17 00:46:15 +03:00
Tulir Asokan	3933b743e7	Maybe fix backfilling multiple batches within deferred backfill ... Might fix #471	2022-04-17 00:09:31 +03:00
Tulir Asokan	e6da32518e	Don't create rooms with no messages when backfilling	2022-04-16 23:58:15 +03:00
Tulir Asokan	1a5d5f5adf	Adjust logs and other things	2022-04-16 22:58:47 +03:00
Sumner Evans	daed67420c	historysync: fix backfill logic again ... * INITIAL_BOOTSTRAP doesn't have 100% as the progress * Enqueue deferred backfills after both RECENT and FULL progress gets to the necessary levels	2022-04-13 10:15:51 -06:00
Sumner Evans	22cc21a438	historysync: refactor backfill enqueue logic	2022-04-12 08:58:47 -06:00
Sumner Evans	eedbecd5ab	historysync: add logging of progress	2022-04-12 08:32:00 -06:00
Sumner Evans	78c6d57299	historysync: only save when backfill is enabled	2022-04-06 09:00:48 -06:00
Sumner Evans	eb0a13a753	historysync: use userID, conversationID, messageID as PK	2022-04-05 13:16:31 -06:00
Sumner Evans	173edfaaf1	batch send: fix plaintext content ... Co-authored-by: Tulir Asokan <tulir@maunium.net>	2022-04-05 13:16:29 -06:00
Sumner Evans	a8be4b11a8	historysync: start deferred backfill on non-full syncs ... Co-authored-by: Tulir Asokan <tulir@maunium.net>	2022-04-05 13:16:01 -06:00
Sumner Evans	748c9509a1	historysync: lock earlier to prevent races and duplicate messages ... Closes BRI-2709	2022-04-04 16:07:25 -06:00
Sumner Evans	54534f6b42	historysync: enqueue immediate backfill on INITIAL_BOOTSTRAP	2022-03-30 17:59:55 -06:00
Sumner Evans	830c294b91	historysync: delete history sync messages once backfilled	2022-03-30 17:40:23 -06:00
Sumner Evans	8a49fea812	backfill: backfill conversation when started ... This applies to when the conversation is started via the provisioning API (start new chat) or when a new message comes in to that portal.	2022-03-29 23:40:22 -06:00
Sumner Evans	12a23e2ca5	historysync: refactor to utilize backfill queue ... Also sends the `m.room.marker` event when a backfill stage is complete.	2022-03-29 19:26:11 -06:00
Tulir Asokan	44f232f37b	Remove special-cased fast backfill ... It's not particularly useful and causes the portals to jump around more	2022-03-15 00:04:40 +02:00
Tulir Asokan	df64bb0e10	Handle disappearing messages in history sync properly	2022-03-15 00:03:37 +02:00
Tulir Asokan	1eb210c249	Add support for bridging reactions	2022-03-05 21:22:31 +02:00
Tulir Asokan	4bba398cfe	Remove unused backfill dummy state event type	2022-03-01 00:14:36 +02:00
Tulir Asokan	a3ba1e3826	Remove dummy state event in history sync ... It's not needed anymore as the relevant PR was merged, and Synapse 1.53 seems to include a regression which breaks sending non-member state events (which seems to be fixed by matrix-org/synapse#12083).	2022-02-28 23:59:05 +02:00
Tulir Asokan	4d19d27dad	Fix storing own message sender metadata from history syncs	2022-02-10 19:41:34 +02:00
Tulir Asokan	528fbda53f	Add initial support for requesting media retries from phone	2022-02-10 19:18:49 +02:00
Tulir Asokan	aacb1d57df	Set portal disappearing message timer from history sync	2022-01-19 16:23:25 +02:00
Tulir Asokan	e2b0879f16	Add support for multi-contact messages	2022-01-03 16:11:39 +02:00
Tulir Asokan	7e5c2769c6	Clean up personal filtering space support	2021-12-29 21:40:08 +02:00
Tulir Asokan	5e2b54f97e	Add constant for double puppet value	2021-12-15 13:51:20 +02:00
Sumner Evans	c15b963748	double puppeting: use fi.mau.double_puppet_source	2021-12-15 02:51:26 -07:00
Tulir Asokan	5c1b57d7b8	Don't override current pushnames in history sync	2021-11-09 23:59:28 +02:00
Tulir Asokan	00088c12e5	Read pushnames from historical messages	2021-11-09 23:44:35 +02:00
Tulir Asokan	53eac40711	Send backfilling bridge state again	2021-11-08 22:56:34 +02:00
Tulir Asokan	19104d857b	Remove unused variable and fix formatting	2021-11-06 21:30:27 +02:00
Tulir Asokan	95453a4748	Maybe fix backfilling when double puppeting isn't enabled at all	2021-11-06 21:28:42 +02:00
Tulir Asokan	58e2a3a511	Disable backfilling with double puppets on other servers	2021-11-06 15:33:59 +02:00
Tulir Asokan	ea045160a2	Never return nil in IntentFor(). Fixes #379	2021-11-06 14:20:56 +02:00
Tulir Asokan	7e20452783	Move all SetReply calls to one place	2021-11-05 11:47:51 +02:00
Tulir Asokan	bc1bf6079a	Fix INITIAL_BOOTSTRAP history syncs not being handled	2021-11-04 01:35:17 +02:00
Tulir Asokan	a7d770ea62	Add some more debug logs for backfilling	2021-11-04 00:09:23 +02:00
Tulir Asokan	f426b87dce	Fix panic in backfill if there are no messages to backfill	2021-11-03 22:04:03 +02:00
Tulir Asokan	752aff48f1	Prioritize backfilling recent messages and fix some bugs	2021-11-03 21:34:06 +02:00