* Add madvdontneed=1 to decrease RAM consumption for systemd service
* Update monolith-example.service
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Add stub functions for MSC2946
* Implement core space walking algorithm
* Flesh out stub functions; add test stubs
* Implement storage bits and add sanity check test
* Implement world_readable auth with test
* Linting
Squashed commit of the following:
commit e5e2d793119733ecbcf9b85f966e018ab0318741
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Jan 13 17:28:12 2021 +0000
Add dendrite_roomserver_processroomevent_duration_millis to prometheus
* Simplify send-to-device messaging
* Don't return error if there's no work to do
* Remove SQLite migrations for now
* Tweak Postgres migrations
* Tweaks
* Fixes
* Cleanup separately
* Fix SQLite migration
* Tweak pagination tokens
* start should be the specified from
* Don't reverse start and end
* Tweak getStartEnd again
* Update sytest-whitelist
* NOTSPEC: Re-add iOS end of topology
* It's half-alive
* Wakeups largely working
* Other tweaks, typing works
* Fix bugs, add receipt stream
* Delete notifier, other tweaks
* Dedupe a bit, add a template for the invite stream
* Clean up, add templates for other streams
* Don't leak channels
* Bring forward some more PDU logic, clean up other places
* Add some more wakeups
* Use addRoomDeltaToResponse
* Log tweaks, typing fixed?
* Fix timed out syncs
* Don't reset next batch position on timeout
* Add account data stream/position
* End of day
* Fix complete sync for receipt, typing
* Streams package
* Clean up a bit
* Complete sync send-to-device
* Don't drop errors
* More lightweight notifications
* Fix typing positions
* Don't advance position on remove again unless needed
* Device list updates
* Advance account data position
* Use limit for incremental sync
* Limit fixes, amongst other things
* Remove some fmt.Println
* Tweaks
* Re-add notifier
* Fix invite position
* Fixes
* Notify account data without advancing PDU position in notifier
* Apply account data position
* Get initial position for account data
* Fix position update
* Fix complete sync positions
* Review comments @Kegsay
* Room consumer parameters
The config section for MSC's was missing from the example config. Even though these are experimental features, it might be good to add the section into the example config for visibility.
Signed-off-by: Jason Robinson <mail@jasonrobinson.me>
* Ensure appservices can auth as users in their namespaces
Currently in Dendrite appservices can only auth as a user if the user was created by said appservice. This does not align with the appservices spec which specifically says:
> The application service may specify the virtual user to act as through use of a user_id query string parameter on the request. The user specified in the query string must be covered by one of the application service’s user namespaces.
https://matrix.org/docs/spec/application_service/r0.1.2#identity-assertion
In the case that a user has been created for example via manual registration but belongs to an appservice namespace, the current functionality does not allow appservices to auth as them. This PR fixes that by replacing the appservice ID check with a check against the appservice namespace.
This also matches Synapse functionality, which I confirmed to allow appservices to auth as a user in their namespace, irregardless of how the user was registered.
* Also allow appservice itself to auth with user_id
Appservice user_id + access token check needs to work both when user_id is the appservice and when appservice has the user in their user namespace.
Signed-off-by: Jason Robinson <mail@jasonrobinson.me>
* Do not check if a username is exclusive if the request is for an appservice in /register
* remove useless comment
* Move statements
* fix broken test
* Also fix the senderLocalpart problem
* Check domain name is ours
* Handle accessTokenErr
* Return unauthorised instead of forbidden
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Update sync responses
* Fix positions, add ApplyUpdates
* Fix MarshalText as non-pointer, PrevBatch is optional
* Increment by number of read receipts
* Merge branch 'master' into neilalexander/devicelist
* Tweak typing
* Include keyserver position tweak
* Fix typing next position in all cases
* Tweaks
* Fix typo
* Tweaks, restore StreamingToken.MarshalText which somehow went missing?
* Rely on positions from notifier rather than manually advancing them
* Revert "Rely on positions from notifier rather than manually advancing them"
This reverts commit 53112a62cc.
* Give invites their own position, fix other things
* Fix test
* Fix invites maybe
* Un-whitelist tests that look to be genuinely wrong
* Use real receipt positions
* Ensure send-to-device uses real positions too
Squashed commit of the following:
commit c38c39107b6dda0c8e6320d61da2365c47eea4e9
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Thu Dec 17 10:13:09 2020 +0000
Update buckets
commit 5a3bcc8bd4167150374827b0e42a0dea0366beff
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Thu Dec 17 09:41:43 2020 +0000
Update buckets
commit 78dff8165ddf596e28af04faf56466752ebc17af
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Thu Dec 17 09:37:27 2020 +0000
Register sendEventDuration
commit 612b0f3d84f1547ff30131f7b084baf609edab52
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Thu Dec 17 09:35:59 2020 +0000
sendevent metrics
Squashed commit of the following:
commit 7ed1c6cfe67429dbe378a763d832c150eb0f781d
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Dec 16 14:53:27 2020 +0000
Updates
commit 8442099d08760b8d086e6d58f9f30284e378a2cd
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Dec 16 14:43:18 2020 +0000
Add some sync statistics
commit ffe2a11644ed3d5297d1775a680886c574143fdb
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Dec 16 14:37:00 2020 +0000
Fix backing off display
commit 27443a93855aa60a49806ecabbf9b09f818301bd
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Dec 16 14:28:43 2020 +0000
Add some destination queue metrics
* Add RoomInfo cache, remove RoomServerRoomNID cache, ensure caches are thread-safe
* Don't panic if the roomInfo isn't known yet
* LRU package is already threadsafe
* Use RoomInfo cache to find room version if possible in Events()
* Adding comments about RoomInfoCache safety
* Hit the database far less to find room NIDs for event NIDs
* Close the rows
* Fix SQLite selectRoomNIDsForEventNIDsSQL
* Give same treatment to room version lookups
* Give read recipts their own database sequence
* Give receipts their own stream ID
* Change migration names
* Reset sequences
* Add max receipt queries, missing stream_id table entry for SQLite
* Don't bail so quickly in fetchMissingStateEvents
* Don't recalculate event IDs so often in sync API
* Add comments
* Fix comments
* Update to matrix-org/gomatrixserverlib@eb6a890
* Refactor federation sender, again
* Clean up better
* Missing operators
* Try to get overflowed events from database
* Fix queries
* Log less
* Comments
* nil PDUs/EDUs shouldn't happen but guard against them for safety
* Tweak logging
* Fix transaction coalescing
* Update comments
* Check nils more
* Remove channels as they add extra complexity and possibly will deadlock
* Don't hold lock while sending transaction
* Less spam about sleeping queues
* Comments
* Bug-fixing
* Don't try to rehydrate twice
* Don't queue in memory for blacklisted destinations
* Don't queue in memory for blacklisted destinations
* Fix a couple of bugs
* Check for duplicates when pulling things out of the database
* Durable transactions, some more refactoring
* Revert "Durable transactions, some more refactoring"
This reverts commit 5daf924eaa.
* Fix deadlock
