0
0
Fork 0
mirror of https://github.com/matrix-org/dendrite synced 2024-06-30 18:28:21 +02:00
Commit graph

1356 commits

Author SHA1 Message Date
Neil Alexander 53112a62cc
Rely on positions from notifier rather than manually advancing them 2020-12-17 14:49:53 +00:00
Neil Alexander c35e46521f
Tweaks, restore StreamingToken.MarshalText which somehow went missing? 2020-12-17 13:28:45 +00:00
Neil Alexander 7200691610
Fix typo 2020-12-17 13:06:08 +00:00
Neil Alexander aad3e573e9
Merge branch 'master' into neilalexander/synctokens2 2020-12-17 13:04:54 +00:00
Neil Alexander 9a1ef81fa2
Add CS sendevent metrics
Squashed commit of the following:

commit c38c39107b6dda0c8e6320d61da2365c47eea4e9
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Dec 17 10:13:09 2020 +0000

    Update buckets

commit 5a3bcc8bd4167150374827b0e42a0dea0366beff
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Dec 17 09:41:43 2020 +0000

    Update buckets

commit 78dff8165ddf596e28af04faf56466752ebc17af
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Dec 17 09:37:27 2020 +0000

    Register sendEventDuration

commit 612b0f3d84f1547ff30131f7b084baf609edab52
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Dec 17 09:35:59 2020 +0000

    sendevent metrics
2020-12-17 10:52:31 +00:00
Neil Alexander e1ace7e44a
Add event ID index on current state table (helps performance) (#1649) 2020-12-16 18:16:39 +00:00
Kegsay 76becac003
Add start_stream to /messages (#1648) 2020-12-16 18:10:39 +00:00
Kegsay 42e9cbf342
NOTSPEC: Make ?from= optional in /messages (#1647) 2020-12-16 17:31:03 +00:00
Neil Alexander 56b5847c74
Add prometheus metrics for destination queues, sync requests
Squashed commit of the following:

commit 7ed1c6cfe67429dbe378a763d832c150eb0f781d
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Wed Dec 16 14:53:27 2020 +0000

    Updates

commit 8442099d08760b8d086e6d58f9f30284e378a2cd
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Wed Dec 16 14:43:18 2020 +0000

    Add some sync statistics

commit ffe2a11644ed3d5297d1775a680886c574143fdb
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Wed Dec 16 14:37:00 2020 +0000

    Fix backing off display

commit 27443a93855aa60a49806ecabbf9b09f818301bd
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Wed Dec 16 14:28:43 2020 +0000

    Add some destination queue metrics
2020-12-16 15:02:39 +00:00
Neil Alexander b891c00b09
Add RoomInfo cache, remove RoomServerRoomNIDsCache (#1646)
* Add RoomInfo cache, remove RoomServerRoomNID cache, ensure caches are thread-safe

* Don't panic if the roomInfo isn't known yet

* LRU package is already threadsafe

* Use RoomInfo cache to find room version if possible in Events()

* Adding comments about RoomInfoCache safety
2020-12-16 12:15:12 +00:00
Neil Alexander 9057143033
Hit the database far less in Events to find room NIDs and room versions (#1643)
* Hit the database far less to find room NIDs for event NIDs

* Close the rows

* Fix SQLite selectRoomNIDsForEventNIDsSQL

* Give same treatment to room version lookups
2020-12-16 10:33:28 +00:00
Neil Alexander d2bcc5f746
Fix main.wasm output path (fixes #1645) 2020-12-16 09:33:48 +00:00
Neil Alexander 38318b0f16
De-map device list positions in streaming tokens (#1642)
* De-map device list positions in streaming tokens

* Fix lint error

* Tweak toOffset
2020-12-15 15:09:10 +00:00
Andrew Morgan 98ebbd01e5
Remove incorrect comment in find-lint.sh (#1619)
The referenced issue is about `go build`, not `go get`.
2020-12-15 09:45:36 +00:00
Shubham Upreti 502051421c
Added govet to golanci-lint linters (#1639) 2020-12-15 09:44:37 +00:00
Neil Alexander 9a5e0a81f9
Tweaks 2020-12-14 17:45:43 +00:00
Neil Alexander c7a9cee5cc
Fix typing next position in all cases 2020-12-14 16:59:27 +00:00
Neil Alexander ebfa8d36c2
Include keyserver position tweak 2020-12-14 16:57:10 +00:00
Neil Alexander e8d485d881
Tweak typing 2020-12-14 16:55:55 +00:00
Neil Alexander 1bc1347ce4
Merge branch 'master' into neilalexander/devicelist 2020-12-14 16:12:49 +00:00
Neil Alexander f4134ea7c6
Merge branch 'master' into neilalexander/synctokens2 2020-12-14 15:27:49 +00:00
Neil Alexander 2885eb0422
Don't use request context for input room event queued tasks (#1640) 2020-12-14 14:40:57 +00:00
Neil Alexander f5869daaab
Don't start more goroutines than needed on RS input, increase input worker buffer size (#1638) 2020-12-14 10:42:21 +00:00
Neil Alexander d7824ed5b1
Update INSTALL.md (#1637) 2020-12-14 09:42:27 +00:00
Neil Alexander b794873fa0
Update INSTALL.md 2020-12-12 20:55:34 +00:00
Neil Alexander db9c5ef07f
Increment by number of read receipts 2020-12-11 14:15:45 +00:00
Neil Alexander 0eb3bece77
Merge branch 'master' into neilalexander/synctokens2 2020-12-11 14:02:34 +00:00
Neil Alexander ebcacd1bb5
Give receipts their own stream ID in the database (#1631)
* Give read recipts their own database sequence

* Give receipts their own stream ID

* Change migration names

* Reset sequences

* Add max receipt queries, missing stream_id table entry for SQLite
2020-12-11 14:02:17 +00:00
Neil Alexander 7ed3f2676a
Fix MarshalText as non-pointer, PrevBatch is optional 2020-12-11 11:10:46 +00:00
Neil Alexander c55361c1b8
Fix max_open_conns in Dendrite sample config 2020-12-11 10:48:24 +00:00
Neil Alexander 0634d9fed1
Fix positions, add ApplyUpdates 2020-12-11 10:42:55 +00:00
Neil Alexander 9eff386e21
Update sync responses 2020-12-11 09:58:13 +00:00
Neil Alexander 9c03b0a4fa
Refactor sync tokens (#1628)
* Refactor sync tokens

* Comment out broken notifier test

* Update types, sytest-whitelist

* More robust token checking

* Remove New functions for streaming tokens

* Export Logs in StreamingToken

* Fix tests
2020-12-10 18:57:10 +00:00
Neil Alexander bad81c028f
Don't recalculate event ID so often in sync (#1624)
* Don't bail so quickly in fetchMissingStateEvents

* Don't recalculate event IDs so often in sync API

* Add comments

* Fix comments

* Update to matrix-org/gomatrixserverlib@eb6a890
2020-12-09 18:07:17 +00:00
Neil Alexander 851c02659a
Version 0.3.3 2020-12-09 14:13:02 +00:00
Neil Alexander d9b3035342
Adjust latest events updater (#1623)
* Adjust forward elatest events updater

* Populate newLatest in all cases

* Re-add existingPrevs loop
2020-12-09 13:34:37 +00:00
Neil Alexander f64c8822bc
Federation sender refactor (#1621)
* Refactor federation sender, again

* Clean up better

* Missing operators

* Try to get overflowed events from database

* Fix queries

* Log less

* Comments

* nil PDUs/EDUs shouldn't happen but guard against them for safety

* Tweak logging

* Fix transaction coalescing

* Update comments

* Check nils more

* Remove channels as they add extra complexity and possibly will deadlock

* Don't hold lock while sending transaction

* Less spam about sleeping queues

* Comments

* Bug-fixing

* Don't try to rehydrate twice

* Don't queue in memory for blacklisted destinations

* Don't queue in memory for blacklisted destinations

* Fix a couple of bugs

* Check for duplicates when pulling things out of the database

* Durable transactions, some more refactoring

* Revert "Durable transactions, some more refactoring"

This reverts commit 5daf924eaa.

* Fix deadlock
2020-12-09 10:03:22 +00:00
Neil Alexander e1d32e2ff1
Update to matrix-org/gomatrixserverlib@232bace 2020-12-07 16:21:55 +00:00
bodqhrohro 45dec20593
Update links to golangci-lint documentation (#1617)
Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-12-07 11:23:01 +00:00
Neil Alexander 8f5393db5f
Update FAQ 2020-12-07 10:20:19 +00:00
Neil Alexander 5d65a879a5
Federation sender event cache (#1614)
* Cache federation sender events

* Store in the correct cache

* Update federation event cache

* Fix Unset

* Give EDUs same caching treatment as PDUs

* Make federationsender_cache_size configurable

* Default caches configuration

* Fix unit tests

* Revert "Fix unit tests"

This reverts commit 24eb5d2252.

* Revert "Default caches configuration"

This reverts commit 464ecd1e64.

* Revert "Make federationsender_cache_size configurable"

This reverts commit 4631f53241.
2020-12-04 14:52:10 +00:00
Kegsay b507312d4c
MSC2836 threading: part 2 (#1596)
* Update GMSL

* Add MSC2836EventRelationships to fedsender

* Call MSC2836EventRelationships in reqCtx

* auth remote servers

* Extract room ID and servers from previous events; refactor a bit

* initial cut of federated threading

* Use the right client/fed struct in the response

* Add QueryAuthChain for use with MSC2836

* Add auth chain to federated response

* Fix pointers

* under CI: more logging and enable mscs, nil fix

* Handle direction: up

* Actually send message events to the roomserver..

* Add children and children_hash to unsigned, with tests

* Add logic for exploring threads and tracking children; missing storage functions

* Implement storage functions for children

* Add fetchUnknownEvent

* Do federated hits for include_children if we have unexplored children

* Use /ev_rel rather than /event as the former includes child metadata

* Remove cross-room threading impl

* Enable MSC2836 in the p2p demo

* Namespace mscs db

* Enable msc2836 for ygg

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-12-04 14:11:01 +00:00
TR_SLimey c052edafdd
Fix broken image in CONTRIBUTING.md (#1612) 2020-12-04 14:10:01 +00:00
Neil Alexander 68d92fd3b4
Update sytest-whitelist/blacklist 2020-12-04 14:09:21 +00:00
Ronnie Ebrin a677a288bd
federationsender/roomserver: don't panic while federation is disabled (#1615) 2020-12-04 14:08:17 +00:00
Neil Alexander 1ce9c52442
Don't recalculate event IDs so often (#1610)
* Don't recalculate event IDs so often

* Revert invite change

* Make sure we're using the right NIDs

* Update gomatrixserverlib

* Update to NewEventFromTrustedJSONWithEventID

* Fix go.mod

* Update gomatrixserverlib to matrix-org/gomatrixserverlib#243

* Use BulkSelectEventID
2020-12-04 10:41:07 +00:00
Petter Jakub Økland 19b1d40d64
Hiawatha reverse proxy sample for monolith and fixes to polylith sample (#1609)
* Created polylith reverse proxy sample for Hiawatha

* Create monolith-sample.conf

* Added timeout to proxies and binding explanation.

* Fixed typo.

* Consistency with regards to polylith-sample.conf.
2020-12-03 13:52:00 +00:00
Neil Alexander 52905ffb82
Update PROFILING.md 2020-12-03 13:49:34 +00:00
Neil Alexander 253b05ccde
Update PROFILING.md 2020-12-03 13:48:02 +00:00
Neil Alexander 246866a131
Add PROFILING.md 2020-12-03 13:46:56 +00:00