https://github.com/matrix-org/synapse/issues/9962 uncovered that we accidentally removed all but one of the presence updates that we store in the database when persisting multiple updates. This could cause users' presence state to be stale.
The bug was fixed in #10014, and this PR just adds a test that failed on the old code, and was used to initially verify the bug.
The test attempts to insert some presence into the database in a batch using `PresenceStore.update_presence`, and then simply pulls it out again.
Fixes: https://github.com/matrix-org/synapse/issues/9962
This is a fix for above problem.
I fixed it by swaping the order of insertion of new records and deletion of old ones. This ensures that we don't delete fresh database records as we do deletes before inserts.
Signed-off-by: Marek Matys <themarcq@gmail.com>
Also add support for giving a callback to generate the JSON object to
verify. This should reduce memory usage, as we no longer have the event
in memory in dict form (which has a large memory footprint) for extend
periods of time.
Instead of parsing the full response to `/send_join` into Python objects (which can be huge for large rooms) and *then* parsing that into events, we instead use ijson to stream parse the response directly into `EventBase` objects.
To be more consistent with similar code. The check now automatically
raises an AuthError instead of passing back a boolean. It also absorbs
some shared logic between callers.
- use a tuple rather than a list for the iterable that is passed into the
wrapped function, for performance
- test that we can pass an iterable and that keys are correctly deduped.
It's not obvious that instances of SQLBaseStore each need their own
instances of random.SystemRandom(); let's just use random directly.
Introduced by 52839886d6
Signed-off-by: Dan Callahan <danc@element.io>
We can get away with just catching UnicodeError here.
⋮
+-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
⋮
https://docs.python.org/3/library/exceptions.html#exception-hierarchy
Signed-off-by: Dan Callahan <danc@element.io>
Functionally identical, but more obviously cryptographically secure.
...Explicit is better than implicit?
Avoids needing to know that SystemRandom() implies a CSPRNG, and
complies with the big scary red box on the documentation for random:
> Warning:
> The pseudo-random generators of this module should not be used for
> security purposes. For security or cryptographic uses, see the
> secrets module.
https://docs.python.org/3/library/random.html
Signed-off-by: Dan Callahan <danc@element.io>
This should help ensure that equivalent results are achieved between
homeservers querying for the summary of a space.
This implements modified MSC1772 rules, according to MSC2946.
The different is that the origin_server_ts of the m.room.create event
is not used as a tie-breaker since this might not be known if the
homeserver is not part of the room.
Per changes in MSC2946, the C-S and S-S APIs for spaces summary
should use GET requests.
Until this is stable, the POST endpoints still exist.
This does not switch federation requests to use the GET version yet
since it is newly added and already deployed servers might not support
it. When switching to the stable endpoint we should switch to GET
requests.
MSC1772 specifies the m.room.create event should be sent as part
of the invite_state. This was done optionally behind an experimental
flag, but is now done by default due to MSC1772 being approved.
Now that cross signing exists there is much less of a need for other people to look at devices and verify them individually. This PR adds a config option to allow you to prevent device display names from being shared with other servers.
Signed-off-by: Aaron Raimist <aaron@raim.ist>
* tests for push rule pattern matching
* tests for acl pattern matching
* factor out common `re.escape`
* Factor out common re.compile
* Factor out common anchoring code
* add word_boundary support to `glob_to_regex`
* Use `glob_to_regex` in push rule evaluator
NB that this drops support for character classes. I don't think anyone ever
used them.
* Improve efficiency of globs with multiple wildcards
The idea here is that we compress multiple `*` globs into a single `.*`. We
also need to consider `?`, since `*?*` is as hard to implement efficiently as
`**`.
* add assertion on regex pattern
* Fix mypy
* Simplify glob_to_regex
* Inline the glob_to_regex helper function
Signed-off-by: Dan Callahan <danc@element.io>
* Moar comments
Signed-off-by: Dan Callahan <danc@element.io>
Co-authored-by: Dan Callahan <danc@element.io>
We were pulling the full auth chain for the room out of the DB each time
we backfilled, which can be *huge* for large rooms and is totally
unnecessary.
