0
0
Fork 0
mirror of https://github.com/matrix-org/dendrite synced 2024-12-14 21:23:49 +01:00
Commit graph

219 commits

Author SHA1 Message Date
Neil Alexander
a050503d8d
Version 0.10.1 2022-09-30 17:15:43 +01:00
Neil Alexander
0e2fb63b4f
Version 0.10.0 (#2753)
Changelog and version bump.
2022-09-30 13:31:05 +01:00
Till
87be32ca26
Fulltext implementation using Bleve (#2675)
Based on #2480

This actually indexes events based on their event type. They are removed
from the index if we receive a `m.room.redaction` event on the
`OutputRoomEvent` stream.
An admin endpoint is added to reindex all existing events.


Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-09-27 18:06:49 +02:00
Neil Alexander
f40e280327
Version 0.9.9 (#2732)
Changelog and version bump.
2022-09-22 14:54:25 +01:00
Till
0ddfb0cad4
Tweak InsertMigration to avoid logging (#2720)
We'd still produce logs in Postgres when trying to insert a migration we
already ran. This should stop us from creating those log entries.
2022-09-22 15:53:15 +02:00
Neil Alexander
bd39748b5c
Update dependencies (#2729)
This updates Dendrite dependencies.
2022-09-20 15:01:19 +01:00
Till
100fa9b235
Check unique constraint errors when manually inserting migrations (#2712)
This should avoid unnecessary logging on startup if the migration (were
we need `InsertMigration`) was already executed.
This now checks for "unique constraint errors" for SQLite and Postgres
and fails the startup process if the migration couldn't be manually
inserted for some other reason.
2022-09-13 08:07:43 +02:00
Neil Alexander
47b2a5d6b8
Version 0.9.8 2022-09-12 13:41:07 +01:00
Neil Alexander
e8687f6f82
Version 0.9.7 (#2707)
Changelog and version bump.
2022-09-09 15:01:13 +01:00
Till
8196b29657
Change detection of already executed migrations (#2665)
This changes the detection of already executed migrations for the
roomserver state block and keychange refactor. It now uses schema tables
provided by the database engine to check if the column was already
removed. We now also store the migration in the migrations table.

This should stop e.g. Postgres from logging errors like `ERROR: column
"event_nid" does not exist at character 8`.
2022-09-09 13:14:52 +01:00
Till
64472d9aab
Update getting pushrules, add tests, tweak pushrules (#2705)
This PR
- adds tests for `evaluatePushrules`
- removes the need for the UserAPI on the `OutputStreamEventConsumer`
(for easier testing)
- adds a method to get the pushrules from the database
- adds a new default pushrule for `m.reaction` events (and some other
tweaks)
2022-09-09 13:56:33 +02:00
Till
d5876abbe9
Fulltext implementation incl. config (#2480)
This adds the main component of the fulltext search.
This PR doesn't do anything yet, besides creating an empty fulltextindex
folder if enabled. Indexing events is done in a separate PR.
2022-09-07 18:15:54 +02:00
Till
440eb0f3a2
Handle errors differently in the DeviceListUpdater (#2695)
`If a device list update goes missing, the server resyncs on the next
one` was failing because a previous test would receive a `waitTime` of
1h, resulting in the test timing out.
This now tries to handle the returned errors differently, e.g. by using
the default `waitTime` of 2s. Also doesn't try further users in the
list, if one of the errors would cause a longer `waitTime`.
2022-09-07 11:44:27 +02:00
Neil Alexander
489ccc1c60
Stronger default power levels 2022-09-06 17:25:17 +01:00
Neil Alexander
fd7661f69c
Tweak Sentry again 2022-09-05 17:37:17 +01:00
Neil Alexander
5992b4c7ed
Set Sentry user where known 2022-09-05 17:35:43 +01:00
Neil Alexander
cd22ba22b0
Improve Sentry reporting 2022-09-05 17:25:11 +01:00
Neil Alexander
304acd7adc
Version 0.9.6 (#2689)
Changelog and version bump.
2022-09-01 14:49:42 +01:00
Neil Alexander
ed79e8626a
Version 0.9.5 (#2673)
Changelog and version bump.
2022-08-25 14:14:10 +01:00
Neil Alexander
56b55a28f5
Version 0.9.4 (#2658)
### Fixes

* A bug in the roomserver around handling rejected outliers has been fixed
* Backfilled events will now use the correct history visibility where possible
* The device list updater backoff has been fixed, which should reduce the number of outbound HTTP requests and `Failed to query device keys for some users` log entries for dead servers
* The `/sync` endpoint will no longer incorrectly return room entries for retired invites which could cause some rooms to show up in the client "Historical" section
* The `/createRoom` endpoint will now correctly populate `is_direct` in invite membership events, which may help clients to classify direct messages correctly
* The `create-account` tool will now log an error if the shared secret is not set in the Dendrite config
* A couple of minor bugs have been fixed in the membership lazy-loading
* Queued EDUs in the federation API are now cached properly
2022-08-19 12:46:14 +01:00
Neil Alexander
bcdbd5c00a
Version 0.9.3 2022-08-15 10:56:54 +01:00
Neil Alexander
ef7262a7a2
Version 0.9.2 (#2638) 2022-08-12 15:18:31 +01:00
Till
48600d5540
Use /admin/v1/register in create-account (#2484)
* Get all account data on CompleteSync

* Revert "Get all account data on CompleteSync"

This reverts commit 44a3e566d8.

* Use /_synapse/admin/v1/register to create account

* Linting

* Linter again :)

* Update docs

* Use HTTP API to reset password, add option to User API `PerformPasswordUpdate` to invalidate sessions

* Fix routing name

* Tell me more about what went wrong

* Deprecate the `-reset-password` flag, document the new API

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-08-12 12:00:07 +01:00
Tak Wai Wong
fad3ac8e78
Protect user_interactive reads and writes with locks (#2635)
* Protect user_interactive reads and writes with locks

* Ignore golangci-lint false positive

* fix lint

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
2022-08-12 09:12:05 +01:00
Neil Alexander
c45d0936b5
Generic-based internal HTTP API (#2626)
* Generic-based internal HTTP API (tested out on a few endpoints in the federation API)

* Add `PerformInvite`

* More tweaks

* Fix metric name

* Fix LookupStateIDs

* Lots of changes to clients

* Some serverside stuff

* Some error handling

* Use paths as metric names

* Revert "Use paths as metric names"

This reverts commit a9323a6a34.

* Namespace metric names

* Remove duplicate entry

* Remove another duplicate entry

* Tweak error handling

* Some more tweaks

* Update error behaviour

* Some more error tweaking

* Fix API path for `PerformDeleteKeys`

* Fix another path

* Tweak federation client proxying

* Fix another path

* Don't return typed nils

* Some more tweaks, not that it makes any difference

* Tweak federation client proxying

* Maybe fix the key backup test
2022-08-11 15:29:33 +01:00
Till
3a156a434a
Invalidate lazyLoadCache if we're doing an initial sync (#2623)
* Bypass lazyLoadCache if we're doing an initial sync

* Make the linter happy again?

* Revert "Make the linter happy again?"

This reverts commit 52a5691ba3.

* Try that again

* Invalidate LazyLoadCache on initial syncs

* Remove unneeded check

* Add TODO

* Rename Invalite -> InvalidateLazyLoadedUser

* Thanks IDE
2022-08-05 14:27:27 +02:00
Neil Alexander
c8935fb53f
Do not use ioutil as it is deprecated (#2625) 2022-08-05 10:26:59 +01:00
Till
1b7f84250a
Fix linter issues (#2624)
* Try that again

* All hail the mighty linter?

* And once again

* goimport all the things
2022-08-05 11:12:41 +02:00
Neil Alexander
a2bed259dd
Version 0.9.1 (#2616)
* Version 0.9.1

* Update CHANGES.md
2022-08-03 17:42:13 +01:00
Neil Alexander
2250768be1
Remove roominfo cache (#2615)
* Remove roominfo cache

It's the source of a number of race conditions which are seemingly causing bugs and CI failures.

* Make the linter less sad
2022-08-03 17:14:21 +01:00
Neil Alexander
3d51624fef
Version 0.9.0 (#2602) 2022-08-01 17:34:29 +01:00
Till
081f5e7226
Update database migrations, remove goose (#2264)
* Add new db migration

* Update migrations
Remove goose

* Add possibility to test direct upgrades

* Try to fix WASM test

* Add checks for specific migrations

* Remove AddMigration
Use WithTransaction
Add Dendrite version to table

* Fix linter issues

* Update tests

* Update comments, outdent if

* Namespace migrations

* Add direct upgrade tests, skipping over one version

* Split migrations

* Update go version in CI

* Fix copy&paste mistake

* Use contexts in migrations

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-25 10:39:22 +01:00
Neil Alexander
5c01306bb5
Add event state key cache (#2576) 2022-07-19 12:15:48 +01:00
Neil Alexander
99b696e775
Tweak cache counters (#2575)
* Tweak cache counters

This makes the number of counters relative to the
maximum cache size. Since the counters
effectively manage the size of the bloom filter,
larger caches need more counters and smaller
caches need less.

10 counters per 1KB data means that the default
cache size of 1GB should result in a bloom filter
and TinyLRU admission set of about 16MB
estimated.

* Remove line left by accident
2022-07-18 14:49:41 +01:00
Neil Alexander
a1f9b02edf
Pointerise types.RoomInfo in the cache so we can update it in-place in the latest events updater 2022-07-13 10:13:34 +01:00
Neil Alexander
3ea21273bc
Ristretto cache (#2563)
* Try Ristretto cache

* Tweak

* It's beautiful

* Update GMSL

* More strict keyable interface

* Fix that some more

* Make less panicky

* Don't enforce mutability checks for now

* Determine mutability using deep equality

* Tweaks

* Namespace keys

* Make federation caches mutable

* Update cost estimation, add metric

* Update GMSL

* Estimate cost for metrics better

* Reduce counters a bit

* Try caching events

* Some guards

* Try again

* Try this

* Use separate caches for hopefully better hash distribution

* Fix bug with admitting events into cache

* Try to fix bugs

* Check nil

* Try that again

* Preserve order jeezo this is messy

* thanks VS Code for doing exactly the wrong thing

* Try this again

* Be more specific

* aaaaargh

* One more time

* That might be better

* Stronger sorting

* Cache expiries, async publishing of EDUs

* Put it back

* Use a shared cache again

* Cost estimation fixes

* Update ristretto

* Reduce counters a bit

* Clean up a bit

* Update GMSL

* 1GB

* Configurable cache sizees

* Tweaks

* Add `config.DataUnit` for specifying friendly cache sizes

* Various tweaks

* Update GMSL

* Add back some lazy loading caching

* Include key in cost

* Include key in cost

* Tweak max age handling, config key name

* Only register prometheus metrics if requested

* Review comments @S7evinK

* Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere)

* Review comments

* Update sample configs

* Update GHA Workflow

* Update Complement images to Go 1.18

* Remove the cache test from the federation API as we no longer guarantee immediate cache admission

* Don't check the caches in the renewal test

* Possibly fix the upgrade tests

* Update to matrix-org/gomatrixserverlib#322

* Update documentation to refer to Go 1.18
2022-07-11 14:31:31 +01:00
Neil Alexander
b5c55faf98
Version 0.8.9 (#2549)
* Version 0.8.9

* Update changelog
2022-07-01 12:00:32 +01:00
Neil Alexander
83797573be
Version 0.8.8 (#2525) 2022-06-09 12:18:50 +01:00
Neil Alexander
6d4bd5d890
Rate limiting changes (#2519)
* Rate limiting changes

This makes the following changes:

* For logged in users, the rate limiting now applies to the device session rather than the remote IP address;
* For non-logged in users, the rate limiting continues to apply to remote address as it does today;
* It is now possible to add user IDs to the `exempt_user_ids` option under `rate_limiting` to exclude bots from rate limiting;
* Admin and appservice users are now exempt from rate limiting by default.

* Fix build with media API
2022-06-07 14:24:04 +01:00
Neil Alexander
f41931b566
Version 0.8.7 (#2509) 2022-06-01 11:40:37 +01:00
Neil Alexander
45c4c00672
Version 0.8.6 (#2497) 2022-05-26 13:01:04 +01:00
Neil Alexander
9e46d5025a
Upgrade various dependencies (#2496)
* Upgrade various dependencies

* Yaml dependency too
2022-05-26 12:34:43 +01:00
kegsay
6de29c1cd2
bugfix: E2EE device keys could sometimes not be sent to remote servers (#2466)
* Fix flakey sytest 'Local device key changes get to remote servers'

* Debug logs

* Remove internal/test and use /test only

Remove a lot of ancient code too.

* Use FederationRoomserverAPI in more places

* Use more interfaces in federationapi; begin adding regression test

* Linting

* Add regression test

* Unbreak tests

* ALL THE LOGS

* Fix a race condition which could cause events to not be sent to servers

If a new room event which rewrites state arrives, we remove all joined hosts
then re-calculate them. This wasn't done in a transaction so for a brief period
we would have no joined hosts. During this interim, key change events which arrive
would not be sent to destination servers. This would sporadically fail on sytest.

* Unbreak new tests

* Linting
2022-05-17 13:23:35 +01:00
Neil Alexander
6af35385ba
Version 0.8.5 (#2461)
* Version 0.8.5

* Update changelog

* Update changelog
2022-05-13 13:17:15 +01:00
Neil Alexander
1897e2f1c0
Version 0.8.4 2022-05-10 12:44:29 +01:00
Neil Alexander
77722c5a4f
Back out matrix-org/dendrite#2421 by restoring http.Clients
This creates problems with non-HTTPS endpoints and should fix #2444.
2022-05-10 11:08:10 +01:00
Neil Alexander
6b3c183396
Version 0.8.3 (#2431)
* Version 0.8.3

* Update changelog
2022-05-09 17:31:14 +01:00
Neil Alexander
633ca06eb9
Version 0.8.3rc1 2022-05-06 16:34:52 +01:00
Till
6493c0c0f2
Move LL cache (#2429) 2022-05-06 15:33:34 +02:00
kegsay
85704eff20
Clean up interface definitions (#2427)
* tidy up interfaces

* remove unused GetCreatorIDForAlias

* Add RoomserverUserAPI interface

* Define more interfaces

* Use AppServiceInternalAPI for consistent naming

* clean up federationapi constructor a bit

* Fix monolith in -http mode
2022-05-06 12:39:26 +01:00