This removes most of the code used for polylith/API mode.
This removes the `/api` internal endpoints entirely.
Binary size change roughly 5%:
```
51437560 Feb 13 10:15 dendrite-monolith-server # old
48759008 Feb 13 10:15 dendrite-monolith-server # new
```
This adds Sytest and Complement coverage reporting to the nightly
scheduled CI runs.
Fixes a few API mode related issues as well, since we seemingly never
really ran them with Complement.
Also fixes a bug related to device list changes: When we pass in an
empty `newlyLeftRooms` slice, we got a list of all currently joined
rooms with the corresponding members. When we then got the
`newlyJoinedRooms`, we wouldn't update the `changed` slice, because we
already got the user from the `newlyLeftRooms` query. This is fixed by
simply ignoring empty `newlyLeftRooms`.
As discussed yesterday, a simplified version of [my
helm](https://github.com/S7evinK/dendrite-helm) which deploys a monolith
with internal NATS and an optionally enabled PostgreSQL server. If the
PostgreSQL dependency is not enabled, a user specified connection string
is constructed.
Co-authored-by: kegsay <kegan@matrix.org>
This PR
- adds several tests for the clientapi, mostly around `/register` and
auth fallback.
- removes the now deprecated `homeserver` field from responses to
`/register` and `/login`
- slightly refactors auth fallback handling
In an attempt to:
- make on-boarding a bit easier (`go test ./...` should now not need
additional postgres setup)
- get code coverage faster, not only scheduled at night
- test the `create-account` binary
This builds on @S7evinK's work to make multi-stage Docker builds. Now
that we can build SQLite without Cgo this should be much simpler and
should make Docker builds in CI significantly faster.
Co-authored-by: Till Faelligen <tfaelligen@gmail.com>
Co-authored-by: Till Faelligen <davidf@element.io>
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
This is #2819 but rebased on latest `main`. This PR is against main too
as opposed to the `moderncsqlite` branch.
The main change here is simply:
```go
// add query parameters to the dsn
if strings.Contains(dsn, "?") {
dsn += "&"
} else {
dsn += "?"
}
// wait some time before erroring if the db is locked
// https://gitlab.com/cznic/sqlite/-/issues/106#note_1058094993
dsn += "_pragma=busy_timeout%3d10000"
```
### Pull Request Checklist
<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->
* [x] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately
Signed off privately.
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Updates/adds a new multistage (build-kit) Dockerfile. (if accepted,
could make `Dockerfile.monolith` and `Dockerfile.polylith` in
`build/docker` obsolete)
There's no huge difference between the dockerfiles, except this uses a
non-root user when running the container, also doesn't copy the working
directory to the image when building.
Also adds vulnerabilities scans using
[Trivy](https://github.com/aquasecurity/trivy) for the created docker
images. (untested)
Building images is done using
```
docker build . --target image-monolith -t dendrite-monolith
docker build . --target image-polylith -t dendrite-polylith
```
As noted in the comments, only adds `dendrite-polylith-multi` to the
polylith image and all required binaries to the monolith image.
Probably needs some docs updating, if this is accepted.
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
This pull request adds the configuration and CI steps to build and
publish a container wrapping the `dendrite-demo-pinecone` command as
well as fixes a sentence structure issue in the pull request template.
As this does not touch any go source code no tests have been added
### Pull Request Checklist
<!-- Please read docs/CONTRIBUTING.md before submitting your pull
request -->
* [x] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign
off](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md#sign-off)
Signed-off-by: networkException <git@nwex.de> (by private sign-off)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
This PR does the following:
- adds a `keysize` parameter to `generate-keys`, so we can use lower sized keys when running in CI
- updates the Complement docker files to use BuildKit (requires Docker >18.09)
- uses `exec` when executing `dendrite-monotlith-server`, making it PID 1 inside docker, which results in Dendrite actually receiving the `SIGTERM` signal send by Docker. (Making it faster when running tests with Complement, as we don't take 10 seconds to timeout)
* Add new db migration
* Update migrations
Remove goose
* Add possibility to test direct upgrades
* Try to fix WASM test
* Add checks for specific migrations
* Remove AddMigration
Use WithTransaction
Add Dendrite version to table
* Fix linter issues
* Update tests
* Update comments, outdent if
* Namespace migrations
* Add direct upgrade tests, skipping over one version
* Split migrations
* Update go version in CI
* Fix copy&paste mistake
* Use contexts in migrations
Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Try Ristretto cache
* Tweak
* It's beautiful
* Update GMSL
* More strict keyable interface
* Fix that some more
* Make less panicky
* Don't enforce mutability checks for now
* Determine mutability using deep equality
* Tweaks
* Namespace keys
* Make federation caches mutable
* Update cost estimation, add metric
* Update GMSL
* Estimate cost for metrics better
* Reduce counters a bit
* Try caching events
* Some guards
* Try again
* Try this
* Use separate caches for hopefully better hash distribution
* Fix bug with admitting events into cache
* Try to fix bugs
* Check nil
* Try that again
* Preserve order jeezo this is messy
* thanks VS Code for doing exactly the wrong thing
* Try this again
* Be more specific
* aaaaargh
* One more time
* That might be better
* Stronger sorting
* Cache expiries, async publishing of EDUs
* Put it back
* Use a shared cache again
* Cost estimation fixes
* Update ristretto
* Reduce counters a bit
* Clean up a bit
* Update GMSL
* 1GB
* Configurable cache sizees
* Tweaks
* Add `config.DataUnit` for specifying friendly cache sizes
* Various tweaks
* Update GMSL
* Add back some lazy loading caching
* Include key in cost
* Include key in cost
* Tweak max age handling, config key name
* Only register prometheus metrics if requested
* Review comments @S7evinK
* Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere)
* Review comments
* Update sample configs
* Update GHA Workflow
* Update Complement images to Go 1.18
* Remove the cache test from the federation API as we no longer guarantee immediate cache admission
* Don't check the caches in the renewal test
* Possibly fix the upgrade tests
* Update to matrix-org/gomatrixserverlib#322
* Update documentation to refer to Go 1.18
