Sumner Evans
b5551ee16a
database: add media requests buffer table
2022-05-11 17:10:55 -06:00
Sumner Evans
e362743f18
config: add settings for automatic media requests
2022-05-11 17:10:55 -06:00
Tulir Asokan
317b4dbffb
Fix panic in requesting retries
2022-05-02 15:47:35 +03:00
Tulir Asokan
4bc4e92877
Remove media backfill queue and just request immediately
2022-05-02 15:00:57 +03:00
Sumner Evans
89131bd20c
backfill: add notification of disappearing messages at correct timestamp
...
This makes it so that the timestamp of the chat in Matrix looks correct,
even though the message is not there to be bridged since it has
disappeared.
2022-04-29 16:59:37 -06:00
Tulir Asokan
331138bc6b
Split forward backfilling and do it in one batch
...
Multiple batches is complicated, as it would require figuring out where
the batch ended when handling the next batch of new-old messages.
2022-04-29 11:07:05 +03:00
Sumner Evans
76c6d0bf87
backfill: fixed bug where the media backfill loop would sleep too often
...
If the number of requested message was a multiple of the number of batch
events, then it would sleep on every single other message (even if not
an errored media message).
2022-04-28 15:48:40 -06:00
Sumner Evans
d898aefff1
backfill loops: combine immediate and deferred loops
2022-04-28 15:48:38 -06:00
Sumner Evans
ac7a437ad8
backfill: add more logging to the queue
2022-04-28 15:48:04 -06:00
Sumner Evans
f32c5f2e9a
backfill: fix priority of backfills again
...
I think this finally has the correct logic for doing each stage for each
portal before going on to the next stage.
2022-04-23 21:50:47 -06:00
Sumner Evans
a1e2e673dc
backfill: fix bridging of ephemeral and view-once messages
2022-04-22 14:07:59 -06:00
Sumner Evans
e74bb62fc1
backfill logging: fix logging of Backfill structs
2022-04-19 11:19:42 -06:00
Sumner Evans
f2e762680c
media backfill: add ability to automatically request media not on WA server
...
This adds a new backfill type for media that sends a request to the
phone for every media that is not available on the WA servers. WA
deletes media from their servers after about two weeks, so you have to
ask the phone to re-upload it.
In order to use this, you need to enable
bridge.history_sync.backfill_media and configure the requests that will
be made per portal using bridge.history_sync.media (which is similar to
the deferred backfill config).
If you already have backfilled portals, but want to do a one-off media
backfill for all existing portals, you can set
bridge.history_sync.enqueue_backfill_media_next_start to true.
2022-04-19 08:12:51 -06:00
Tulir Asokan
c98b7f32c9
Fix handling errors from batch sending
2022-04-19 11:39:17 +03:00
Tulir Asokan
2c68ccc0a3
Clean up some backfill things
2022-04-18 20:12:24 +03:00
Tulir Asokan
1250e2d605
More backfill fixes
2022-04-17 00:46:15 +03:00
Tulir Asokan
3933b743e7
Maybe fix backfilling multiple batches within deferred backfill
...
Might fix #471
2022-04-17 00:09:31 +03:00
Tulir Asokan
e6da32518e
Don't create rooms with no messages when backfilling
2022-04-16 23:58:15 +03:00
Tulir Asokan
1a5d5f5adf
Adjust logs and other things
2022-04-16 22:58:47 +03:00
Sumner Evans
daed67420c
historysync: fix backfill logic again
...
* INITIAL_BOOTSTRAP doesn't have 100% as the progress
* Enqueue deferred backfills after both RECENT and FULL progress gets to
the necessary levels
2022-04-13 10:15:51 -06:00
Sumner Evans
22cc21a438
historysync: refactor backfill enqueue logic
2022-04-12 08:58:47 -06:00
Sumner Evans
eedbecd5ab
historysync: add logging of progress
2022-04-12 08:32:00 -06:00
Sumner Evans
78c6d57299
historysync: only save when backfill is enabled
2022-04-06 09:00:48 -06:00
Sumner Evans
eb0a13a753
historysync: use userID, conversationID, messageID as PK
2022-04-05 13:16:31 -06:00
Sumner Evans
173edfaaf1
batch send: fix plaintext content
...
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-05 13:16:29 -06:00
Sumner Evans
a8be4b11a8
historysync: start deferred backfill on non-full syncs
...
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-05 13:16:01 -06:00
Sumner Evans
748c9509a1
historysync: lock earlier to prevent races and duplicate messages
...
Closes BRI-2709
2022-04-04 16:07:25 -06:00
Sumner Evans
54534f6b42
historysync: enqueue immediate backfill on INITIAL_BOOTSTRAP
2022-03-30 17:59:55 -06:00
Sumner Evans
830c294b91
historysync: delete history sync messages once backfilled
2022-03-30 17:40:23 -06:00
Sumner Evans
8a49fea812
backfill: backfill conversation when started
...
This applies to when the conversation is started via the provisioning
API (start new chat) or when a new message comes in to that portal.
2022-03-29 23:40:22 -06:00
Sumner Evans
12a23e2ca5
historysync: refactor to utilize backfill queue
...
Also sends the `m.room.marker` event when a backfill stage is complete.
2022-03-29 19:26:11 -06:00
Tulir Asokan
44f232f37b
Remove special-cased fast backfill
...
It's not particularly useful and causes the portals to jump around more
2022-03-15 00:04:40 +02:00
Tulir Asokan
df64bb0e10
Handle disappearing messages in history sync properly
2022-03-15 00:03:37 +02:00
Tulir Asokan
1eb210c249
Add support for bridging reactions
2022-03-05 21:22:31 +02:00
Tulir Asokan
4bba398cfe
Remove unused backfill dummy state event type
2022-03-01 00:14:36 +02:00
Tulir Asokan
a3ba1e3826
Remove dummy state event in history sync
...
It's not needed anymore as the relevant PR was merged, and Synapse 1.53
seems to include a regression which breaks sending non-member state events
(which seems to be fixed by matrix-org/synapse#12083 ).
2022-02-28 23:59:05 +02:00
Tulir Asokan
4d19d27dad
Fix storing own message sender metadata from history syncs
2022-02-10 19:41:34 +02:00
Tulir Asokan
528fbda53f
Add initial support for requesting media retries from phone
2022-02-10 19:18:49 +02:00
Tulir Asokan
aacb1d57df
Set portal disappearing message timer from history sync
2022-01-19 16:23:25 +02:00
Tulir Asokan
e2b0879f16
Add support for multi-contact messages
2022-01-03 16:11:39 +02:00
Tulir Asokan
7e5c2769c6
Clean up personal filtering space support
2021-12-29 21:40:08 +02:00
Tulir Asokan
5e2b54f97e
Add constant for double puppet value
2021-12-15 13:51:20 +02:00
Sumner Evans
c15b963748
double puppeting: use fi.mau.double_puppet_source
2021-12-15 02:51:26 -07:00
Tulir Asokan
5c1b57d7b8
Don't override current pushnames in history sync
2021-11-09 23:59:28 +02:00
Tulir Asokan
00088c12e5
Read pushnames from historical messages
2021-11-09 23:44:35 +02:00
Tulir Asokan
53eac40711
Send backfilling bridge state again
2021-11-08 22:56:34 +02:00
Tulir Asokan
19104d857b
Remove unused variable and fix formatting
2021-11-06 21:30:27 +02:00
Tulir Asokan
95453a4748
Maybe fix backfilling when double puppeting isn't enabled at all
2021-11-06 21:28:42 +02:00
Tulir Asokan
58e2a3a511
Disable backfilling with double puppets on other servers
2021-11-06 15:33:59 +02:00
Tulir Asokan
ea045160a2
Never return nil in IntentFor(). Fixes #379
2021-11-06 14:20:56 +02:00
Tulir Asokan
7e20452783
Move all SetReply calls to one place
2021-11-05 11:47:51 +02:00
Tulir Asokan
bc1bf6079a
Fix INITIAL_BOOTSTRAP history syncs not being handled
2021-11-04 01:35:17 +02:00
Tulir Asokan
a7d770ea62
Add some more debug logs for backfilling
2021-11-04 00:09:23 +02:00
Tulir Asokan
f426b87dce
Fix panic in backfill if there are no messages to backfill
2021-11-03 22:04:03 +02:00
Tulir Asokan
752aff48f1
Prioritize backfilling recent messages and fix some bugs
2021-11-03 21:34:06 +02:00
