Tulir Asokan
8012368de5
Fix backfill queue query. Fixes #517
2022-06-17 10:33:47 +03:00
Tulir Asokan
2463f8e60f
Port some more old db schema upgrades
2022-05-26 11:23:15 +03:00
Tulir Asokan
52748374b7
Actually fix first_expected_ts upgrade. Fixes #504
2022-05-25 23:48:39 +03:00
Tulir Asokan
241de2f18f
Fix changing first_expected_ts column type
2022-05-25 18:51:51 +03:00
Sumner Evans
1f292dc1c5
Merge pull request #502 from mautrix/backfill-queue-fixes
...
backfill: fixes and minor refactors
2022-05-25 08:40:44 -06:00
Sumner Evans
b850995888
backfill: block deferred requests on immediate and forward backfills
2022-05-24 17:34:43 -06:00
Sumner Evans
a0ffed43dd
database/backfill_state: change type of first_expected_ts to BIGINT
2022-05-24 17:34:43 -06:00
Tulir Asokan
66cd7cd080
Fix mixed indent in .sql files
2022-05-23 20:00:08 +03:00
Tulir Asokan
a42c7bca17
Remove unused crypto store and fix comment
2022-05-22 22:03:08 +03:00
Tulir Asokan
a948ea0146
Move a bunch of stuff to mautrix-go
...
See d578d1a610
2022-05-22 01:06:30 +03:00
Sumner Evans
4eac1ea719
backfill state: update during backfill
2022-05-20 10:22:30 -06:00
Sumner Evans
89663c2745
backfill state: add table for tracking state
2022-05-20 10:18:24 -06:00
Sumner Evans
feb9f71cb5
backfill: re-dispatch if 15 minutes have elapsed
...
If 15 minutes have passed since the backfill request was dispatched and
backfill hasn't completed, then re-dispatch.
2022-05-20 10:18:17 -06:00
Tulir Asokan
d33f5aa3d2
Stop ignoring errors deleting backfill queue stuff
2022-05-19 11:45:08 +03:00
Tulir Asokan
68b2d663ac
Fix deleting historical messages of chats that won't be bridged
2022-05-19 11:41:18 +03:00
Tulir Asokan
60496e5976
Fix last message timestamp in history_sync_conversation table
2022-05-19 11:41:02 +03:00
Sumner Evans
ec8f75e50e
backfill: add option to mark unread using account data
2022-05-18 10:40:29 -06:00
Tulir Asokan
8b7abe6f7d
Don't send read receipts for errored messages
2022-05-18 01:00:23 +03:00
Sumner Evans
5b73ba7efd
Merge pull request #494 from mautrix/sumner/bri-3309
...
backfill: remove intermediate queues and unblock re-checking DB
2022-05-16 17:13:29 -06:00
Sumner Evans
54f3e8a439
database: combine backfill queue upgrades together
2022-05-16 17:09:47 -06:00
Sumner Evans
df46ca99f9
backfill: remove intermediate buffer for backfill
2022-05-16 14:04:22 -06:00
Tulir Asokan
3ee7c8527a
Send a valid request when doing hacky phone pings
2022-05-16 11:22:41 +03:00
Sumner Evans
f3f6d88e55
backfill: perform batch finish in transaction
2022-05-15 21:43:12 -06:00
Sumner Evans
c1bf0e6555
backfill queue: set the dispatch time instead of completed time
2022-05-15 21:42:42 -06:00
Sumner Evans
17c697445d
history sync: store timestamp when the message got put into the DB
...
This will be useful for debugging issues with missing messages
2022-05-15 21:42:42 -06:00
Sumner Evans
1d70cbff48
backfill queue: add dispatch time, remove time end to table
...
* the dispatch time is going to be what the completed time used to be
* the time end column was always nil, so I got rid of it
2022-05-15 21:42:38 -06:00
Tulir Asokan
c7b0f98f97
Fix scanning null values in timezone column
2022-05-12 20:51:37 +03:00
Sumner Evans
b42aa16684
media backfill: store media key in request
...
This way, we don't have to fetch the event from Matrix in these situations
2022-05-12 11:30:42 -06:00
Sumner Evans
08e77fab29
media backfill: send retry requests at the configured time
...
Only does the batch send of requests if the request method is 'local_time'
2022-05-11 17:10:55 -06:00
Sumner Evans
b5551ee16a
database: add media requests buffer table
2022-05-11 17:10:55 -06:00
Tulir Asokan
4bc4e92877
Remove media backfill queue and just request immediately
2022-05-02 15:00:57 +03:00
Tulir Asokan
4b22e786f5
Make signal store retry if postgres dies
2022-04-29 19:38:44 +03:00
Tulir Asokan
573eec44c8
Fix indentation in SQL query
2022-04-29 17:03:27 +03:00
Tulir Asokan
6b69ea707b
Use single SQL query for fetching backfill queue items
2022-04-29 10:44:22 +03:00
Sumner Evans
caf0a363c9
backfill: delete messages for rooms the user is not in
...
If the chat turns out to be not bridgable due to the user not being in
the room, delete the messages, and stop trying to backfill it.
2022-04-27 08:29:20 -06:00
Sumner Evans
e74bb62fc1
backfill logging: fix logging of Backfill structs
2022-04-19 11:19:42 -06:00
Sumner Evans
f2e762680c
media backfill: add ability to automatically request media not on WA server
...
This adds a new backfill type for media that sends a request to the
phone for every media that is not available on the WA servers. WA
deletes media from their servers after about two weeks, so you have to
ask the phone to re-upload it.
In order to use this, you need to enable
bridge.history_sync.backfill_media and configure the requests that will
be made per portal using bridge.history_sync.media (which is similar to
the deferred backfill config).
If you already have backfilled portals, but want to do a one-off media
backfill for all existing portals, you can set
bridge.history_sync.enqueue_backfill_media_next_start to true.
2022-04-19 08:12:51 -06:00
Tulir Asokan
2c68ccc0a3
Clean up some backfill things
2022-04-18 20:12:24 +03:00
Tulir Asokan
5b9fe8e08a
Delete messages by timestamp range instead of message ID
2022-04-18 20:12:01 +03:00
Tulir Asokan
1a5d5f5adf
Adjust logs and other things
2022-04-16 22:58:47 +03:00
Tulir Asokan
6e976fb3fc
Default to backfilling all chats in history sync
2022-04-16 22:42:17 +03:00
Sumner Evans
bd26fc4af9
Minor cleanups from CR
...
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-07 09:20:43 -06:00
Sumner Evans
c664e5f107
Updates from CR
...
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-06 08:56:58 -06:00
Sumner Evans
eb0a13a753
historysync: use userID, conversationID, messageID as PK
2022-04-05 13:16:31 -06:00
Sumner Evans
005fbb09f8
backfill queue: don't delete, just mark as complete
2022-04-05 13:16:31 -06:00
Sumner Evans
830c294b91
historysync: delete history sync messages once backfilled
2022-03-30 17:40:23 -06:00
Sumner Evans
12a23e2ca5
historysync: refactor to utilize backfill queue
...
Also sends the `m.room.marker` event when a backfill stage is complete.
2022-03-29 19:26:11 -06:00
Sumner Evans
1ecdb71ac3
database: add backfill queue and history sync tables
2022-03-29 12:16:30 -06:00
Tulir Asokan
caacac15c7
Check database owner and foreign tables before starting
2022-03-16 01:15:13 +02:00
Tulir Asokan
c82a96dccc
Handle incoming reaction removals
2022-03-12 19:47:47 +02:00
