Commit graph

91 commits

Author SHA1 Message Date
Tulir Asokan 4bfd3bd644 Fix marking messages as disappearing while backfilling on SQLite 2022-11-10 23:12:08 +02:00
Sumner Evans f51fb5c5c2
historysync: fix typo
Signed-off-by: Sumner Evans <sumner@beeper.com>
2022-11-10 10:07:08 -07:00
Sumner Evans 433d901658 backfill: add option to only sync unread state if chat is younger than threshold
Signed-off-by: Sumner Evans <sumner@beeper.com>
2022-10-12 13:48:38 -06:00
Tulir Asokan ea8c58b4f5 Fix merged captions in backfill 2022-09-28 16:43:27 +03:00
Tulir Asokan 7a933124fa Use transaction for saving reactions in backfill 2022-09-28 15:54:08 +03:00
Tulir Asokan 7a6c4d53d3 Backfill reactions when using hungryserv 2022-09-28 15:07:10 +03:00
Tulir Asokan f0401ee81e Generate different deterministic event IDs for captions 2022-09-20 17:31:12 +03:00
Tulir Asokan 2e934abe43 Disable state_events_at_start on hungryserv 2022-09-12 22:14:41 +03:00
Tulir Asokan 985b33fd7c Backfill with deterministic event IDs on hungryserv 2022-08-25 17:05:45 +03:00
Tulir Asokan f79efa9037 Update mautrix-go 2022-08-14 19:37:11 +03:00
Tulir Asokan 2bc5c32d48 Add double puppet value to backfilled event wire content 2022-07-05 16:41:28 +03:00
Tulir Asokan bf4c01648f Move most double puppet source key adding to mautrix-go 2022-06-30 20:56:26 +03:00
Tulir Asokan aa0daceb85 Add config option to bundle caption with media message 2022-06-17 11:34:24 +03:00
Sumner Evans 1f292dc1c5
Merge pull request #502 from mautrix/backfill-queue-fixes
backfill: fixes and minor refactors
2022-05-25 08:40:44 -06:00
Sumner Evans 0939bb3bc3
backfill status event: send first expected timestamp in ms after epoch instead of seconds 2022-05-25 08:39:08 -06:00
Sumner Evans 03d46630c2
backfill status event: send after room exists 2022-05-24 17:34:43 -06:00
Sumner Evans b850995888
backfill: block deferred requests on immediate and forward backfills 2022-05-24 17:34:43 -06:00
Tulir Asokan 372929f454 Fix small things 2022-05-24 14:02:06 +03:00
Tulir Asokan 1185b4bb71 Add mutex to block new messages in specific backfill cases
Specifically, when sending events using the `com.beeper.new_messages`
query param, normal new messages shouldn't be processed, as ordering
might get messed up.

3a264b77ed
2022-05-24 13:39:29 +03:00
Sumner Evans cdf97a2df3
backfill: set backfill status state event 2022-05-20 10:22:33 -06:00
Sumner Evans 4eac1ea719
backfill state: update during backfill 2022-05-20 10:22:30 -06:00
Tulir Asokan 962179fb75 Handle edge cases with missing senders in historical messages 2022-05-19 12:08:31 +03:00
Tulir Asokan f39fff3285
Send first batch of events without historical flag (#498) 2022-05-19 11:36:42 +03:00
Sumner Evans ec8f75e50e
backfill: add option to mark unread using account data 2022-05-18 10:40:29 -06:00
Sumner Evans df46ca99f9
backfill: remove intermediate buffer for backfill 2022-05-16 14:04:22 -06:00
Sumner Evans f3f6d88e55
backfill: perform batch finish in transaction 2022-05-15 21:43:12 -06:00
Sumner Evans c1bf0e6555
backfill queue: set the dispatch time instead of completed time 2022-05-15 21:42:42 -06:00
Sumner Evans 1d70cbff48
backfill queue: add dispatch time, remove time end to table
* the dispatch time is going to be what the completed time used to be
* the time end column was always nil, so I got rid of it
2022-05-15 21:42:38 -06:00
Tulir Asokan f74af8eb30 Update whatsmeow and improve history sync logging 2022-05-14 14:17:03 +03:00
Tulir Asokan 6f14475667 Switch to ParseWebMessage from whatsmeow 2022-05-13 16:08:33 +03:00
Tulir Asokan 1f6cedf948 Reduce unnecessary invite calls during history sync 2022-05-13 11:19:52 +03:00
Sumner Evans 8cb41b8949
Merge pull request #490 from mautrix/sumner/bri-3238
media backfill: allow media requests to be performed at a specific local time for the user
2022-05-12 11:31:54 -06:00
Sumner Evans b42aa16684
media backfill: store media key in request
This way, we don't have to fetch the event from Matrix in these situations
2022-05-12 11:30:42 -06:00
Tulir Asokan 4a5fbcb13a Fix disappearing message notice spam when backfilling 2022-05-12 20:00:31 +03:00
Sumner Evans 08e77fab29
media backfill: send retry requests at the configured time
Only does the batch send of requests if the request method is 'local_time'
2022-05-11 17:10:55 -06:00
Sumner Evans 7de7fedc02
media backfill: enable enqueue to media backfill buffer instead of immediately requesting retry receipt 2022-05-11 17:10:55 -06:00
Sumner Evans b5551ee16a
database: add media requests buffer table 2022-05-11 17:10:55 -06:00
Sumner Evans e362743f18
config: add settings for automatic media requests 2022-05-11 17:10:55 -06:00
Tulir Asokan 317b4dbffb Fix panic in requesting retries 2022-05-02 15:47:35 +03:00
Tulir Asokan 4bc4e92877 Remove media backfill queue and just request immediately 2022-05-02 15:00:57 +03:00
Sumner Evans 89131bd20c
backfill: add notification of disappearing messages at correct timestamp
This makes it so that the timestamp of the chat in Matrix looks correct,
even though the message is not there to be bridged since it has
disappeared.
2022-04-29 16:59:37 -06:00
Tulir Asokan 331138bc6b Split forward backfilling and do it in one batch
Multiple batches is complicated, as it would require figuring out where
the batch ended when handling the next batch of new-old messages.
2022-04-29 11:07:05 +03:00
Sumner Evans 76c6d0bf87
backfill: fixed bug where the media backfill loop would sleep too often
If the number of requested message was a multiple of the number of batch
events, then it would sleep on every single other message (even if not
an errored media message).
2022-04-28 15:48:40 -06:00
Sumner Evans d898aefff1
backfill loops: combine immediate and deferred loops 2022-04-28 15:48:38 -06:00
Sumner Evans ac7a437ad8
backfill: add more logging to the queue 2022-04-28 15:48:04 -06:00
Sumner Evans f32c5f2e9a
backfill: fix priority of backfills again
I think this finally has the correct logic for doing each stage for each
portal before going on to the next stage.
2022-04-23 21:50:47 -06:00
Sumner Evans a1e2e673dc
backfill: fix bridging of ephemeral and view-once messages 2022-04-22 14:07:59 -06:00
Sumner Evans e74bb62fc1
backfill logging: fix logging of Backfill structs 2022-04-19 11:19:42 -06:00
Sumner Evans f2e762680c
media backfill: add ability to automatically request media not on WA server
This adds a new backfill type for media that sends a request to the
phone for every media that is not available on the WA servers. WA
deletes media from their servers after about two weeks, so you have to
ask the phone to re-upload it.

In order to use this, you need to enable
bridge.history_sync.backfill_media and configure the requests that will
be made per portal using bridge.history_sync.media (which is similar to
the deferred backfill config).

If you already have backfilled portals, but want to do a one-off media
backfill for all existing portals, you can set
bridge.history_sync.enqueue_backfill_media_next_start to true.
2022-04-19 08:12:51 -06:00
Tulir Asokan c98b7f32c9 Fix handling errors from batch sending 2022-04-19 11:39:17 +03:00