Commit graph

1197 commits

Author SHA1 Message Date
Sumner Evans
8e2077b9ae
ci: automatically update both STABLE and NIGHTLY on dev environment 2022-04-18 19:26:05 -06:00
Tulir Asokan
2c68ccc0a3 Clean up some backfill things 2022-04-18 20:12:24 +03:00
Tulir Asokan
5b9fe8e08a Delete messages by timestamp range instead of message ID 2022-04-18 20:12:01 +03:00
Sumner Evans
7a77b1976c
reactions: send checkpoints for matrix reactions (#472) 2022-04-18 11:14:43 +03:00
Tulir Asokan
7f2d12aa5f Update mautrix-go to switch to /v3 paths 2022-04-17 13:09:54 +03:00
Tulir Asokan
1250e2d605 More backfill fixes 2022-04-17 00:46:15 +03:00
Tulir Asokan
3933b743e7 Maybe fix backfilling multiple batches within deferred backfill
Might fix #471
2022-04-17 00:09:31 +03:00
Tulir Asokan
e6da32518e Don't create rooms with no messages when backfilling 2022-04-16 23:58:15 +03:00
Tulir Asokan
8711519c2b Merge branch 'release-0.3.1' 2022-04-16 23:25:51 +03:00
Tulir Asokan
72556a3767 Bump version to v0.3.1 2022-04-16 23:21:15 +03:00
Tulir Asokan
8e9fd57491 Update dependencies 2022-04-16 23:17:52 +03:00
Tulir Asokan
a58cae16cd Backport fixing custom fields leaking in history sync 2022-04-16 23:16:05 +03:00
Tulir Asokan
2142f92089 Don't print generated registration message if config is invalid 2022-04-16 23:14:55 +03:00
Tulir Asokan
1a5d5f5adf Adjust logs and other things 2022-04-16 22:58:47 +03:00
Tulir Asokan
6e976fb3fc Default to backfilling all chats in history sync 2022-04-16 22:42:17 +03:00
Tulir Asokan
8e4995cd70 Update dependencies 2022-04-16 22:36:17 +03:00
Sumner Evans
d2d9be7a68
Merge pull request #469 from mautrix/sumner/bri-2852-3
historysync: fix backfill logic again
2022-04-15 15:04:27 -06:00
Sumner Evans
daed67420c
historysync: fix backfill logic again
* INITIAL_BOOTSTRAP doesn't have 100% as the progress
* Enqueue deferred backfills after both RECENT and FULL progress gets to
  the necessary levels
2022-04-13 10:15:51 -06:00
Sumner Evans
8f4eedb75e
Merge pull request #468 from mautrix/sumner/bri-2852-2
historysync: refactor backfill enqueue logic
2022-04-12 09:21:15 -06:00
Sumner Evans
22cc21a438
historysync: refactor backfill enqueue logic 2022-04-12 08:58:47 -06:00
Sumner Evans
7742ccb8ec
Merge pull request #467 from mautrix/sumner/bri-2852
historysync: add logging of progress
2022-04-12 08:35:34 -06:00
Sumner Evans
eedbecd5ab
historysync: add logging of progress 2022-04-12 08:32:00 -06:00
Tulir Asokan
34f83237b9 Update whatsmeow 2022-04-12 10:59:58 +03:00
Tulir Asokan
f085b4d11c Don't print generated registration message if config is invalid 2022-04-09 17:04:36 +03:00
Tulir Asokan
0dc5c820c4 Update dependencies 2022-04-09 16:56:33 +03:00
Sumner Evans
37b8065db5
Merge pull request #461 from mautrix/sumner/bri-2227
history sync: implement prioritized backfill
2022-04-07 09:28:25 -06:00
Sumner Evans
bd26fc4af9
Minor cleanups from CR
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-07 09:20:43 -06:00
Tulir Asokan
d4318d54a4 Add missing return 2022-04-07 10:48:36 +03:00
Sumner Evans
78c6d57299
historysync: only save when backfill is enabled 2022-04-06 09:00:48 -06:00
Sumner Evans
c664e5f107
Updates from CR
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-06 08:56:58 -06:00
Sumner Evans
83d397900f
media: add better error when media couldn't be retrieved due to WA deleting it from their servers 2022-04-05 15:35:02 -06:00
Sumner Evans
eb0a13a753
historysync: use userID, conversationID, messageID as PK 2022-04-05 13:16:31 -06:00
Sumner Evans
005fbb09f8
backfill queue: don't delete, just mark as complete 2022-04-05 13:16:31 -06:00
Sumner Evans
173edfaaf1
batch send: fix plaintext content
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-05 13:16:29 -06:00
Sumner Evans
a8be4b11a8
historysync: start deferred backfill on non-full syncs
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2022-04-05 13:16:01 -06:00
Sumner Evans
748c9509a1
historysync: lock earlier to prevent races and duplicate messages
Closes BRI-2709
2022-04-04 16:07:25 -06:00
Sumner Evans
54534f6b42
historysync: enqueue immediate backfill on INITIAL_BOOTSTRAP 2022-03-30 17:59:55 -06:00
Sumner Evans
830c294b91
historysync: delete history sync messages once backfilled 2022-03-30 17:40:23 -06:00
Sumner Evans
8a49fea812
backfill: backfill conversation when started
This applies to when the conversation is started via the provisioning
API (start new chat) or when a new message comes in to that portal.
2022-03-29 23:40:22 -06:00
Sumner Evans
e702bf39d2
commands/backfill: add command to manually backfill a portal 2022-03-29 23:40:22 -06:00
Sumner Evans
12a23e2ca5
historysync: refactor to utilize backfill queue
Also sends the `m.room.marker` event when a backfill stage is complete.
2022-03-29 19:26:11 -06:00
Sumner Evans
536d340f20
config: add settings for prioritized backfill
bridge.history_sync.max_initial_conversations:

  This setting determines the maximum number of initial conversations
  that should be backfilled. The data for all the other conversations
  will be stored in the database for backfill at a later time.

bridge.history_sync.immediate:

  These settings are for the initial backfill that should be performed
  to populate each of the initial chats with a few messages so that
  users can continue their conversations without loosing context.

bridge.history_sync.deferred:

  These settings are for backfilling the rest of the chat history that
  was not covered by the immediate backfills. These should generally be
  done at a slower pace to avoid overloading the homeserver.
2022-03-29 12:16:31 -06:00
Sumner Evans
1ecdb71ac3
database: add backfill queue and history sync tables 2022-03-29 12:16:30 -06:00
Tulir Asokan
25c77b0fb5 Update mautrix-go and whatsmeow 2022-03-29 21:14:16 +03:00
Tulir Asokan
0a61d21373 Update whatsmeow 2022-03-29 16:10:28 +03:00
Tulir Asokan
5ef66496d6 Acquire roomCreateLock when updating puppet info to prevent race conditions 2022-03-25 00:02:38 +02:00
Tulir Asokan
44eaf285e6 Adjust outdated version logs 2022-03-23 14:46:30 +02:00
Tulir Asokan
f88abb99a8 Use Go 1.18 for lint 2022-03-23 13:14:36 +02:00
Tulir Asokan
d1c70fbda1 Update changelog 2022-03-22 13:49:28 +02:00
Tulir Asokan
f79ca422e8 Add option to use async media uploads
https://github.com/matrix-org/matrix-spec-proposals/pull/2246
2022-03-21 21:08:48 +02:00