Kegsay
6b8791b868
Always call overridden form of GetLatestEventsForUpdate ( #1554 )
...
This ensures we don't make txns on sqlite still, which can cause
'database is locked' errors.
2020-10-20 19:32:33 +01:00
Neil Alexander
7612f64e3c
Update gomatrixserverlib to matrix-org/gomatrixserverlib#220
2020-10-20 17:23:04 +01:00
Neil Alexander
24e38c4135
Internal HTTP APIs over H2C ( #1541 )
...
* H2C on internal HTTP because SCIENCE
* Update comments
2020-10-20 17:13:12 +01:00
Neil Alexander
7ca89ef511
Update gomatrixserverlib
2020-10-20 17:10:37 +01:00
Neil Alexander
6a16d46fba
Version 0.2.0 ( #1551 )
...
* v0.2.0-rc2
* Update CHANGES.md (also with some markdown lint suggestions)
* Update version number
* Update CHANGES.md
* Update CHANGES.md
* Update CHANGES.md
* Add known issue
* Update CHANGES.md
2020-10-20 16:46:53 +01:00
Neil Alexander
39c7a8915c
Multi-personality polylith binary ( #1552 )
...
* Initial work oon multipersonality binary
* Remove old binaries
* Monolith and polylith binaries
* Better logging
* dendrite-poly-multi
* Fix path
* Copyright notices etc
* Tweaks
* Update Docker, INSTALL.md
* Take first argument if flags package doesn't find any args
* Postgres 9.6 or later, fix some more Docker stuff
* Don't create unnecessary e2ekey DB
* Run go mod tidy
2020-10-20 16:11:24 +01:00
Neil Alexander
6c3c621de0
Remove invalid state delta check ( #1550 )
2020-10-20 12:36:16 +01:00
Kegan Dougal
837c295c26
Linting
2020-10-20 12:29:53 +01:00
Pika
53a745f333
fix create-account ( #1546 )
2020-10-20 11:47:37 +01:00
Kegsay
eb86e2b336
Fix sqlite locking bugs present on sytest ( #1543 )
...
* Fix sqite locking bugs present on sytest
Comments do the explaining.
* Fix deadlock in sqlite mode
Caused by starting a writer whilst within a writer
* Only complain about invalid state deltas for non-overwrite events
* Do not re-process outlier unnecessarily
2020-10-20 11:42:54 +01:00
Neil Alexander
92982a402f
Update Docker ( #1542 )
...
* Separate Docker images, rather than tags
* Allow specifying tag to build/push/pull
* Include goose in Docker builds
2020-10-20 11:34:22 +01:00
Neil Alexander
a71360d099
Update INSTALL.md ( #1549 )
2020-10-20 11:22:46 +01:00
Neil Alexander
c7bf122a26
Update sytest lists
2020-10-19 15:38:42 +01:00
Neil Alexander
5d80ff11a0
Update sytest-whitelist
2020-10-19 15:17:28 +01:00
Devon Johnson
45abdcaeb9
Send state after event, not current
...
Signed-off-by: Devon Johnson <djohnson1865@gmail.com>
2020-10-19 15:12:37 +01:00
Neil Alexander
6e63df1d9a
KindOld ( #1531 )
...
* Add KindOld
* Don't process latest events/memberships for old events
* Allow federationsender to ignore duplicate key entries when LatestEventIDs is duplicated by RS output events
* Signal to downstream components if an event has become a forward extremity
* Don't exclude from sync
* Soft-fail checks on KindNew
* Don't run the latest events updater at all for KindOld
* Don't make federation sender change after all
* Kind in federation sender join
* Don't send isForwardExtremity
* Fix syncapi
* Update comments
* Fix SendEventWithState
* Update sytest-whitelist
* Generate old output events
* Sync API consumes old room events
* Update comments
2020-10-19 14:59:13 +01:00
S7evinK
0974f6e2c0
Fix internal http api ( #1535 )
...
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-10-19 09:38:10 +01:00
Neil Alexander
640e8c50ec
Take write lock for rate limit map ( #1532 )
...
* Take write lock for rate limit map
* Fix potential race condition
2020-10-16 15:44:39 +01:00
Kegsay
4a7fb9c045
Automatically upgrade databases on startup ( #1529 )
...
* Support auto-upgrading accounts DB
* Auto-upgrade device DB deltas
* Support up/downgrading from cmd/goose
* Linting
* Create tables then do migrations then prepare statements
To avoid failing due to some things not existing
* Linting
2020-10-15 18:09:41 +01:00
Neil Alexander
3e5d38e284
Improve state resolution v2 performance ( matrix-org/gomatrixserverlib#237 )
2020-10-15 16:20:19 +01:00
Neil Alexander
e154c45b51
Better logging around db.StoreEvent
2020-10-15 14:14:17 +01:00
Neil Alexander
49abe359e6
Start Kafka connections for each component that needs them ( #1527 )
...
* Start Kafka connection for each component that needs one
* Fix roomserver unit tests
* Rename to naffkaInstance (@Kegsay review comment)
* Fix import cycle
2020-10-15 13:27:13 +01:00
Neil Alexander
10f1beb0de
Don't re-run state resolution on a single trusted state snapshot ( #1526 )
...
* Don't re-run state resolution on a single trusted state snapshot
* Lint
* Check if backward extremity is create event before checking missing state
2020-10-15 12:08:49 +01:00
Kegsay
e3c2b081c7
txn nil guard when rolling back LatestEventsUpdater ( #1524 )
...
* txn nil guard when rolling back LatestEventsUpdater
* Spell lint correctly
2020-10-14 17:05:09 +01:00
Neil Alexander
6f12b8f85c
Ignore typing events where sender doesn't match origin ( #1523 )
...
* Ignore typing notifications where the sender doesn't match the origin
* Update sytest-whitelist
* Fix formatting directives
2020-10-14 16:49:25 +01:00
Neil Alexander
e3a3908654
Update sytest-whitelist ( closes #1315 )
2020-10-14 16:07:15 +01:00
Neil Alexander
8d9ecb3996
Ignore duplicate redaction entries ( #1522 )
2020-10-14 15:24:43 +01:00
Neil Alexander
286dd408ae
Better semver compliance for version reporting
2020-10-14 12:51:06 +01:00
Neil Alexander
7a1fd123de
Improved state handling in /send ( #1521 )
...
* Capture errors
* Don't request only state key tuples needed for auth (we end up discarding room state this way)
* QueryStateAfterEvent returns all state when no tuples supplied
* Resolve state
* Comments
2020-10-14 12:39:37 +01:00
Neil Alexander
20aec70ead
Send cumulative state when creating room ( #1519 )
...
* Send state with new room events
* lookupEvent sends outliers
* Revert "lookupEvent sends outliers"
This reverts commit 3e16556441
2020-10-14 09:57:27 +01:00
Neil Alexander
9d6b77c58a
Try to retrieve missing auth events from multiple servers ( #1516 )
...
* Recursively fetch auth events if needed
* Fix processEvent call
* Ask more servers in lookupEvent
* Don't panic!
* Panic at the Disco
* Find servers more aggressively
* Add getServers
* Fix number of servers to 5, don't bail making RespState if auth events missing
* Fix panic
* Ignore missing state events too
* Report number of servers correctly
* Don't reuse request context for /send_join
* Update federation API tests
* Don't recurse processEvents
* Implement getEvents differently
2020-10-13 11:53:20 +01:00
Kegsay
d7ea814fa8
Wrap NewMembershipUpdater in a db writer ( #1515 )
2020-10-13 10:20:27 +01:00
Neil Alexander
73bc28b11f
Update docker-compose.deps.yml
2020-10-13 09:54:07 +01:00
Neil Alexander
2f578531ba
Update Docker samples
2020-10-13 09:50:11 +01:00
Neil Alexander
8001627cfc
Get missing event tweaks ( #1514 )
...
* Adjust backfill to send backward extremity with state before other backfilled events, include prev_events with no state amongst missing events
* Not finished refactor
* Fix test
* Remove isInboundTxn
* Remove debug logging
2020-10-12 15:56:15 +01:00
Pieter
0804594a61
Add systemd example service to docs ( #1512 )
...
Signed-off-by: Pieter Hollander <git@pieterhollander.nl>
2020-10-12 10:06:56 +01:00
Neil Alexander
6b579e6aba
Don't ignore local keys from fetchers ( #1513 )
2020-10-12 10:02:20 +01:00
Kegsay
9096bfcee8
Validate m.room.create events in send_join responses ( #1505 )
...
* Validate m.room.create events in send_join responses
For sytest compliance, refs #1315 and #1317
Fixes #1317
* Linting
2020-10-10 00:21:15 +01:00
Neil Alexander
fe5d1400bf
Update federation timeouts ( #1504 )
...
* Update to matrix-org/gomatrixserverlib#234
* Update gomatrixserverlib
* Update federation timeouts
* Fix dendritejs
* Increase /send context time in destination queue
2020-10-09 17:08:32 +01:00
Neil Alexander
4df7e345bb
Only return 500 on /send if a database error occurs ( #1503 )
2020-10-09 15:06:43 +01:00
S7evinK
2bd0449c5b
Use default value when adding column which is not null ( #1501 )
...
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-10-09 12:41:53 +01:00
S7evinK
1cd525ef0d
Extend device_devices table ( #1471 )
...
* Add last_used_ts and IP to database
* Add migrations
* Rename column
Prepare statements
* Add interface method and implement it
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Rename struct fields
* Add user_agent to database
* Add userAgent to registration calls
* Add missing "IF NOT EXISTS"
* use txn writer
* Add UserAgent to Device
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:17:23 +01:00
Pestdoktor
c4c8bfd027
reject invalid UTF-8 ( #1472 )
...
* reject invalid UTF-8
Signed-off-by: Jonas Fentker <jonas@fentker.eu>
* update sytest-whitelist
Signed-off-by: Jonas Fentker <jonas@fentker.eu>
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:15:51 +01:00
Loïck Bonniot
f3e8ae01ef
Implement fully read markers ( #1475 )
...
See #653
Signed-off-by: Loïck Bonniot <git@lesterpig.com>
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:15:35 +01:00
Kegsay
009401ad4d
Version 0.1.0
...
Beta README (#1466 )
* Beta docs
* More tweaks
* More docs
* Update README.md (#1497 )
* Call out missing features
* Add CHANGES
* Call out CHANGES
* Update INSTALL.md
* Update README.md
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-10-08 17:47:43 +01:00
Kegan Dougal
b12b7abcc0
v0.1.0
...
Squashed commit of the following:
commit 570528e5f19d76fe2fd7ea646246eea9650b1b60
Author: Kegan Dougal <kegan@matrix.org>
Date: Thu Oct 8 14:52:10 2020 +0100
v0.1.0
commit 02c020bced3be6d1a63e1001ab9c0ef6231ba087
Merge: db840f02 8035c50c78f6e1a3c8fc68553e12f6e9c8fc68555c15bf385141e7d9eea48bca7a83e7d9eea4a03fa76370f2f7cecdd952ddded7f7cecdd9a8
2020-10-08 14:53:46 +01:00
Neil Alexander
8035c50c06
Don't get into situations where we have no forward extremities
2020-10-08 14:46:25 +01:00
Neil Alexander
78f6e1a31e
Don't set new state NID if state regression
2020-10-08 13:30:30 +01:00
Kegsay
a846dad0e1
Return what we have when we encounter missing events when servicing backfill/gme ( #1499 )
...
We expect to have missing events as we walk back in the DAG over federation
as we didn't always create the room. When checking if the server is allowed
to see those events, just give up and stop rather than fail the request.
2020-10-08 12:22:00 +01:00
Neil Alexander
429bd48129
Return a non-fatal error to the federation API on a state regression ( #1498 )
...
* Return a non-fatal error to the federation API on a state regression
* Return no error but don't morph state
2020-10-08 12:13:50 +01:00
