Sumner Evans
e74bb62fc1
backfill logging: fix logging of Backfill structs
2022-04-19 11:19:42 -06:00
Sumner Evans
a3d56704a7
Merge pull request #474 from mautrix/sumner/bri-2926
...
media backfill: add ability to automatically request media not on WA server
2022-04-19 11:04:29 -06:00
Sumner Evans
91098fa005
Merge pull request #473 from mautrix/dev-update-stable-and-nightly
...
ci: automatically update both STABLE and NIGHTLY on dev environment
2022-04-19 08:57:12 -06:00
Sumner Evans
f2e762680c
media backfill: add ability to automatically request media not on WA server
...
This adds a new backfill type for media that sends a request to the
phone for every media that is not available on the WA servers. WA
deletes media from their servers after about two weeks, so you have to
ask the phone to re-upload it.
In order to use this, you need to enable
bridge.history_sync.backfill_media and configure the requests that will
be made per portal using bridge.history_sync.media (which is similar to
the deferred backfill config).
If you already have backfilled portals, but want to do a one-off media
backfill for all existing portals, you can set
bridge.history_sync.enqueue_backfill_media_next_start to true.
2022-04-19 08:12:51 -06:00
Tulir Asokan
c98b7f32c9
Fix handling errors from batch sending
2022-04-19 11:39:17 +03:00
Sumner Evans
8e2077b9ae
ci: automatically update both STABLE and NIGHTLY on dev environment
2022-04-18 19:26:05 -06:00
Tulir Asokan
2c68ccc0a3
Clean up some backfill things
2022-04-18 20:12:24 +03:00
Tulir Asokan
5b9fe8e08a
Delete messages by timestamp range instead of message ID
2022-04-18 20:12:01 +03:00
Sumner Evans
7a77b1976c
reactions: send checkpoints for matrix reactions ( #472 )
2022-04-18 11:14:43 +03:00
Tulir Asokan
7f2d12aa5f
Update mautrix-go to switch to /v3 paths
2022-04-17 13:09:54 +03:00
Tulir Asokan
1250e2d605
More backfill fixes
2022-04-17 00:46:15 +03:00
Tulir Asokan
3933b743e7
Maybe fix backfilling multiple batches within deferred backfill
...
Might fix #471
2022-04-17 00:09:31 +03:00
Tulir Asokan
e6da32518e
Don't create rooms with no messages when backfilling
2022-04-16 23:58:15 +03:00
Tulir Asokan
8711519c2b
Merge branch 'release-0.3.1'
2022-04-16 23:25:51 +03:00
Tulir Asokan
72556a3767
Bump version to v0.3.1
2022-04-16 23:21:15 +03:00
Tulir Asokan
8e9fd57491
Update dependencies
2022-04-16 23:17:52 +03:00
Tulir Asokan
a58cae16cd
Backport fixing custom fields leaking in history sync
2022-04-16 23:16:05 +03:00
Tulir Asokan
2142f92089
Don't print generated registration message if config is invalid
2022-04-16 23:14:55 +03:00
Tulir Asokan
1a5d5f5adf
Adjust logs and other things
2022-04-16 22:58:47 +03:00
Tulir Asokan
6e976fb3fc
Default to backfilling all chats in history sync
2022-04-16 22:42:17 +03:00
Tulir Asokan
8e4995cd70
Update dependencies
2022-04-16 22:36:17 +03:00
Sumner Evans
d2d9be7a68
Merge pull request #469 from mautrix/sumner/bri-2852-3
...
historysync: fix backfill logic again
2022-04-15 15:04:27 -06:00
Sumner Evans
daed67420c
historysync: fix backfill logic again
...
* INITIAL_BOOTSTRAP doesn't have 100% as the progress
* Enqueue deferred backfills after both RECENT and FULL progress gets to
the necessary levels
2022-04-13 10:15:51 -06:00
Sumner Evans
8f4eedb75e
Merge pull request #468 from mautrix/sumner/bri-2852-2
...
historysync: refactor backfill enqueue logic
2022-04-12 09:21:15 -06:00
Sumner Evans
22cc21a438
historysync: refactor backfill enqueue logic
2022-04-12 08:58:47 -06:00
Sumner Evans
7742ccb8ec
Merge pull request #467 from mautrix/sumner/bri-2852
...
historysync: add logging of progress
2022-04-12 08:35:34 -06:00
Sumner Evans
eedbecd5ab
historysync: add logging of progress
2022-04-12 08:32:00 -06:00
Tulir Asokan
34f83237b9
Update whatsmeow
2022-04-12 10:59:58 +03:00
Tulir Asokan
f085b4d11c
Don't print generated registration message if config is invalid
2022-04-09 17:04:36 +03:00
Tulir Asokan
0dc5c820c4
Update dependencies
2022-04-09 16:56:33 +03:00
Sumner Evans
37b8065db5
Merge pull request #461 from mautrix/sumner/bri-2227
...
history sync: implement prioritized backfill
2022-04-07 09:28:25 -06:00
Sumner Evans
bd26fc4af9
Minor cleanups from CR
...
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-07 09:20:43 -06:00
Tulir Asokan
d4318d54a4
Add missing return
2022-04-07 10:48:36 +03:00
Sumner Evans
78c6d57299
historysync: only save when backfill is enabled
2022-04-06 09:00:48 -06:00
Sumner Evans
c664e5f107
Updates from CR
...
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-06 08:56:58 -06:00
Sumner Evans
83d397900f
media: add better error when media couldn't be retrieved due to WA deleting it from their servers
2022-04-05 15:35:02 -06:00
Sumner Evans
eb0a13a753
historysync: use userID, conversationID, messageID as PK
2022-04-05 13:16:31 -06:00
Sumner Evans
005fbb09f8
backfill queue: don't delete, just mark as complete
2022-04-05 13:16:31 -06:00
Sumner Evans
173edfaaf1
batch send: fix plaintext content
...
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-05 13:16:29 -06:00
Sumner Evans
a8be4b11a8
historysync: start deferred backfill on non-full syncs
...
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-05 13:16:01 -06:00
Sumner Evans
748c9509a1
historysync: lock earlier to prevent races and duplicate messages
...
Closes BRI-2709
2022-04-04 16:07:25 -06:00
Sumner Evans
54534f6b42
historysync: enqueue immediate backfill on INITIAL_BOOTSTRAP
2022-03-30 17:59:55 -06:00
Sumner Evans
830c294b91
historysync: delete history sync messages once backfilled
2022-03-30 17:40:23 -06:00
Sumner Evans
8a49fea812
backfill: backfill conversation when started
...
This applies to when the conversation is started via the provisioning
API (start new chat) or when a new message comes in to that portal.
2022-03-29 23:40:22 -06:00
Sumner Evans
e702bf39d2
commands/backfill: add command to manually backfill a portal
2022-03-29 23:40:22 -06:00
Sumner Evans
12a23e2ca5
historysync: refactor to utilize backfill queue
...
Also sends the `m.room.marker` event when a backfill stage is complete.
2022-03-29 19:26:11 -06:00
Sumner Evans
536d340f20
config: add settings for prioritized backfill
...
bridge.history_sync.max_initial_conversations:
This setting determines the maximum number of initial conversations
that should be backfilled. The data for all the other conversations
will be stored in the database for backfill at a later time.
bridge.history_sync.immediate:
These settings are for the initial backfill that should be performed
to populate each of the initial chats with a few messages so that
users can continue their conversations without loosing context.
bridge.history_sync.deferred:
These settings are for backfilling the rest of the chat history that
was not covered by the immediate backfills. These should generally be
done at a slower pace to avoid overloading the homeserver.
2022-03-29 12:16:31 -06:00
Sumner Evans
1ecdb71ac3
database: add backfill queue and history sync tables
2022-03-29 12:16:30 -06:00
Tulir Asokan
25c77b0fb5
Update mautrix-go and whatsmeow
2022-03-29 21:14:16 +03:00
Tulir Asokan
0a61d21373
Update whatsmeow
2022-03-29 16:10:28 +03:00
