Erik Johnston
d025b5ab50
Correctly changes to required state config in sliding sync ( #17785 )
...
Fixes https://github.com/element-hq/synapse/issues/17698
This handles `required_state` changes by checking if new state has been
added to the config, and if so fetching and returning that from the
current state.
This also takes care to ensure that given a state entry S that is added,
removed and then re-added that we do *not* send S down a second time if
there have been no changes to S in the current state. This is fine for
Rust SDK (as it just remembers all state), but we might decide not to do
this behaviour in the MSC. If we decide to always send down S then its
easy enough to rip out all the code.
---------
Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
2024-10-14 13:31:22 +01:00
dependabot[bot]
ae6179b382
Bump mypy-zope from 1.0.5 to 1.0.7 ( #17827 )
2024-10-14 13:26:40 +01:00
dependabot[bot]
5dd6157972
Bump types-setuptools from 75.1.0.20240917 to 75.1.0.20241014 ( #17828 )
2024-10-14 13:26:23 +01:00
dependabot[bot]
1266138b66
Bump sentry-sdk from 2.15.0 to 2.16.0 ( #17829 )
2024-10-14 13:26:12 +01:00
Erik Johnston
24975eca4d
Build debian packages for new Ubuntu versions ( #17824 )
...
c.f. https://wiki.ubuntu.com/Releases for the currently supported Ubuntu
releases.
Note: this removes support for 23.04 and 23.10, which are EOL.
Fixes #17811
2024-10-14 11:34:33 +01:00
Andrew Morgan
451a9dc7b9
Clarify when 3PID invite module callbacks are called ( #17627 )
...
Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
2024-10-14 11:31:49 +01:00
Erik Johnston
f6a3e5e1c2
Fix release script to check GH token ( #17803 )
...
The current logic didn't work.
2024-10-10 08:59:01 +00:00
Nathan
05576f0b4b
Added display_name_claim in jwt_config which sets the user's display name upon registration ( #17708 )
2024-10-09 12:21:08 +00:00
Martin Weinelt
60aebdb27e
Fix saving of non-RGB thumbnails as PNG ( #17736 )
2024-10-08 18:32:25 +01:00
Erik Johnston
b1b4b2944d
Merge branch 'release-v1.117' into develop
2024-10-08 16:35:35 +01:00
Andrew Ferrazzutti
bdcc9fa388
Fix incorrectly documented config path argument ( #17802 )
2024-10-08 15:05:36 +01:00
Erik Johnston
6a0c21fabd
Fixup changlog
2024-10-08 15:04:20 +01:00
dependabot[bot]
f40641c29b
Bump sigstore/cosign-installer from 3.6.0 to 3.7.0 ( #17798 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:56:15 +01:00
dependabot[bot]
1bb528ee44
Bump phonenumbers from 8.13.46 to 8.13.47 ( #17797 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:41:27 +01:00
dependabot[bot]
165f4ca776
Bump sentry-sdk from 2.14.0 to 2.15.0 ( #17795 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:41:03 +01:00
dependabot[bot]
475e192cbe
Bump tomli from 2.0.1 to 2.0.2 ( #17796 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:40:12 +01:00
dependabot[bot]
43040a4051
Bump ruff from 0.6.8 to 0.6.9 ( #17794 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:39:19 +01:00
Erik Johnston
b3e2d10f39
1.117.0rc1
2024-10-08 14:37:36 +01:00
Shay
a5986ac229
Improvements to admin redact api ( #17792 )
...
- better validation on user input
- fix an early task completion
- when checking membership in rooms, check for rooms user has been
banned from as well
2024-10-08 14:23:21 +01:00
Andrew Ferrazzutti
006251a5d0
Add missing license header ( #17799 )
...
Co-authored-by: Erik Johnston <erik@matrix.org>
2024-10-08 12:01:44 +01:00
Erik Johnston
422f3ecec1
Sliding sync: omit bump stamp when it is unchanged ( #17788 )
...
This saves some DB lookups in rooms
2024-10-08 11:17:23 +01:00
Erik Johnston
4e90221d87
Sliding sync minor performance speed up using new table ( #17787 )
...
Use the new tables to work out which rooms have changed.
---------
Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
2024-10-08 11:06:31 +01:00
Erik Johnston
e2610de208
Speed up sliding sync when there are many active subscriptions ( #17789 )
...
Two changes: a) use a batch lookup function instead of a loop, b) check
existing data to see if we already have what we need and only fetch what
we don't.
2024-10-08 10:35:15 +01:00
Andrew Morgan
e8c8924b81
Clarify test_forget_when_not_left docstring ( #17628 )
2024-10-07 16:34:32 +01:00
V02460
e8e0f0fad7
Add config option redis.password_path ( #17717 )
...
Adds the option to load the Redis password from a file, instead of
giving it in the config directly. The code is similar to how it’s done
for `registration_shared_secret_path`. I changed the example in the
documentation to represent the best practice regarding the handling of
secrets.
Reading secrets from files has the security advantage of separating the
secrets from the config. It also simplifies secrets management in
Kubernetes.
2024-10-07 09:46:51 +01:00
Henrique
beb7a951f4
docs: add note about PYTHONMALLOC for accurate jemalloc memory tracking ( #17709 )
...
Added a note in the documentation suggesting that users may set
`PYTHONMALLOC=malloc` when using `jemalloc`. This allows jemalloc to
track memory usage more accurately by bypassing Python's internal
small-object allocator (`pymalloc`), helping to ensure that
`cache_autotuning` functions as expected.
This doc change aims to provide more clarity for users configuring
jemalloc with Synapse.
Based on:
4ac783549c/synapse/metrics/jemalloc.py (L198-L201)
2024-10-07 08:37:39 +00:00
dependabot[bot]
d34f827ed8
Bump python-multipart from 0.0.10 to 0.0.12 ( #17772 )
2024-10-07 09:14:30 +01:00
Andrew Ferrazzutti
9920417723
Don't say MSC4140 is supported when it's disabled ( #17780 )
2024-10-04 13:42:34 +01:00
Andrew Morgan
316d635906
Fix NAME attribute of ReplicationRemovePusherRestServlet ( #17779 )
2024-10-04 09:53:35 +01:00
Dirk Klimpel
8bbe66a9b9
explain load balancing for federation_sender_instances ( #17776 )
...
Adding information on how the load is distributed for
`federation_sender_instances`.
Thx to @devonh for the information.
causal source:
c2e5e9e67c/synapse/config/_base.py (L946-L989)https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: Devon Hudson <devon.dmytro@gmail.com>
2024-10-03 22:01:33 +00:00
Andrew Morgan
d4e3ad04cd
Merge branch 'master' into develop
2024-10-01 12:18:22 +01:00
Andrew Morgan
55c0391cc8
1.116.0
2024-10-01 11:14:13 +01:00
Erik Johnston
81e0f57800
Fix perf when streams don't change often ( #17767 )
...
There is a bug with the `StreamChangeCache` where it would incorrectly
return that all entities had changed if asked for entities changed
*since* the earliest stream position.
Note that for streams we use the inequalities: `$min_stream_id <
stream_id <= $max_stream_id`, i.e. when we ask the stream change cache
for all things that have changed since `$stream_id` we don't care for
events that happened *at* `$stream_id`.
Specifically: `_earliest_known_stream_pos` is the position at which we
know that we'll have entries for all changes since that point, we can
use the cache for any stream IDs that equal
`_earliest_known_stream_pos`.
`_earliest_known_stream_pos` is set in three places:
- On startup we set it either to:
- the current maximum stream ID, with not prefilled values; or
- the minimum of the latest N values we pulled from the DB
- When we evict items from the bottom, we set it to the stream ID of the
evicted items.
This was changed in https://github.com/matrix-org/synapse/pull/14435 ,
but I think we were overly conservative there.
---------
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-09-30 13:52:33 +01:00
Erik Johnston
ae4862c38f
Optimise notifier mk2 ( #17766 )
...
Based on #17765 .
Basically the idea is to reduce the overhead of calling
`ObservableDeferred` in a loop. The two gains are: a) just using a list
of deferreds rather than the machinery of `ObservableDeferred`, and b)
only calling `PreseverLoggingContext` once.
`PreseverLoggingContext` in particular is expensive to call a lot as
each time it needs to call `get_thread_resource_usage` twice, so that it
an update the CPU metrics of the log context.
2024-09-30 13:32:31 +01:00
dependabot[bot]
602956ef64
Bump ruff from 0.6.7 to 0.6.8 ( #17774 )
2024-09-30 13:08:56 +01:00
dependabot[bot]
444b565c76
Bump phonenumbers from 8.13.45 to 8.13.46 ( #17773 )
2024-09-30 13:07:57 +01:00
dependabot[bot]
8068f31146
Bump regex from 1.10.6 to 1.11.0 ( #17770 )
2024-09-30 13:06:43 +01:00
Erik Johnston
5210565c12
Reduce overhead of sliding sync E2EE loops ( #17771 )
...
Mainly toning down logging and only calling
`get_membership_from_event_ids` if something has changed.
2024-09-30 13:00:14 +01:00
Erik Johnston
de955293cf
Add fast path for sliding sync streams that only ask for extensions ( #17768 )
...
Principally useful for EX e2ee sliding sync connections.
2024-09-30 12:59:50 +01:00
Erik Johnston
93889eb2e7
Optimise notifier ( #17765 )
...
The notifier is quite inefficient when it has to wake up many user
streams all at once
From a silly benchmark this takes the time to notify 1M user streams
from ~30s to ~5s
2024-09-30 12:58:13 +01:00
Erik Johnston
ece66ba61c
Minor perf speed up for large accounts on SSS ( #17751 )
...
This works as instead of passing *all* rooms to `record_sent_rooms` we
only need to pass rooms that were previously not in the LIVE state.
This came from a py-spy where we were spending ~10% CPU calling these
functions. Note that `record_sent_rooms` is a no-op for rooms that are
already in the `LIVE` state, so we only need to call them for
`PREVIOUSLY` or `INITIAL` rooms.
2024-09-30 12:58:02 +01:00
Quentin Gliech
ef9ef99f59
Merge branch 'release-v1.116' into develop
2024-09-26 16:19:32 +02:00
Quentin Gliech
cfbddc258f
1.116.0rc2
2024-09-26 15:29:13 +02:00
Andrew Ferrazzutti
302534c348
Support MSC3757: Restricting who can overwrite a state event ( #17513 )
...
Link to the
MSC: https://github.com/matrix-org/matrix-spec-proposals/pull/3757
---------
Co-authored-by: Quentin Gliech <quenting@element.io>
2024-09-26 15:25:05 +02:00
Erik Johnston
f144b4c7e9
Remove spurious TODO in debian install step ( #17749 )
...
This was a note added in the PR to move to AGPL, which we failed to
remove before landing.
(The context for this was that we needed to decide if we were going to
change which debian repository we published too, but decided not to in
the end)
2024-09-26 13:18:28 +01:00
Quentin Gliech
13dea6949b
Changelog fixes
2024-09-25 12:07:51 +02:00
Quentin Gliech
386cabda83
1.116.0rc1
2024-09-25 11:34:36 +02:00
dependabot[bot]
f53a3a56e2
Bump treq from 23.11.0 to 24.9.1 ( #17744 )
...
Bumps [treq](https://github.com/twisted/treq ) from 23.11.0 to 24.9.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/twisted/treq/releases ">treq's
releases</a>.</em></p>
<blockquote>
<h2>Treq 24.9.0</h2>
<h2>Features</h2>
<ul>
<li>treq now ships type annotations. (<a
href="https://redirect.github.com/twisted/treq/issues/366 ">#366</a>)</li>
<li>The new <code>treq.cookies</code> module provides helper functions
for working with <code>http.cookiejar.Cookie</code> and
<code>CookieJar</code> objects. (<a
href="https://redirect.github.com/twisted/treq/issues/384 ">#384</a>)</li>
<li>Python 3.13 is now supported. (<a
href="https://redirect.github.com/twisted/treq/issues/391 ">#391</a>)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li><code>treq.content.text_content()</code> no longer generates
deprecation warnings due to use of the <code>cgi</code> module. (<a
href="https://redirect.github.com/twisted/treq/issues/355 ">#355</a>)</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Mixing the <em>json</em> argument with <em>files</em> or
<em>data</em> now raises <code>TypeError</code>. (<a
href="https://redirect.github.com/twisted/treq/issues/297 ">#297</a>)</li>
<li>Passing non-string (<code>str</code> or <code>bytes</code>) values
as part of a dict to the <em>headers</em> argument now results in a
<code>TypeError</code>, as does passing any collection other than a
<code>dict</code> or <code>Headers</code> instance. (<a
href="https://redirect.github.com/twisted/treq/issues/302 ">#302</a>)</li>
<li>Support for Python 3.7 and PyPy 3.8, which have reached end of
support, has been dropped. (<a
href="https://redirect.github.com/twisted/treq/issues/378 ">#378</a>)</li>
</ul>
<h2>Misc</h2>
<ul>
<li><a
href="https://redirect.github.com/twisted/treq/issues/336 ">#336</a>, <a
href="https://redirect.github.com/twisted/treq/issues/382 ">#382</a>, <a
href="https://redirect.github.com/twisted/treq/issues/395 ">#395</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/twisted/treq/blob/trunk/CHANGELOG.rst ">treq's
changelog</a>.</em></p>
<blockquote>
<h1>24.9.1 (2024-09-19)</h1>
<h2>Bugfixes</h2>
<ul>
<li>treq has vendored its dependency on the <code>multipart</code>
library to avoid import
conflicts with <code>python-multipart</code>; it should now be
installable alongside
that library. (<code>[#399 ](https://github.com/twisted/treq/issues/399 )
<https://github.com/twisted/treq/issues/399> ;</code>__)</li>
</ul>
<h1>24.9.0 (2024-09-17)</h1>
<h2>Features</h2>
<ul>
<li>treq now ships type annotations.
(<code>[#366 ](https://github.com/twisted/treq/issues/366 )
<https://github.com/twisted/treq/issues/366> ;</code>__)</li>
<li>The new :mod:<code>treq.cookies</code> module provides helper
functions for working with <code>http.cookiejar.Cookie</code> and
<code>CookieJar</code> objects.
(<code>[#384 ](https://github.com/twisted/treq/issues/384 )
<https://github.com/twisted/treq/issues/384> ;</code>__)</li>
<li>Python 3.13 is now supported.
(<code>[#391 ](https://github.com/twisted/treq/issues/391 )
<https://github.com/twisted/treq/issues/391> ;</code>__)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>:mod:<code>treq.content.text_content()</code> no longer generates
deprecation warnings due to use of the <code>cgi</code> module.
(<code>[#355 ](https://github.com/twisted/treq/issues/355 )
<https://github.com/twisted/treq/issues/355> ;</code>__)</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Mixing the <em>json</em> argument with <em>files</em> or
<em>data</em> now raises <code>TypeError</code>.
(<code>[#297 ](https://github.com/twisted/treq/issues/297 )
<https://github.com/twisted/treq/issues/297> ;</code>__)</li>
<li>Passing non-string (<code>str</code> or <code>bytes</code>) values
as part of a dict to the <em>headers</em> argument now results in a
<code>TypeError</code>, as does passing any collection other than a
<code>dict</code> or <code>Headers</code> instance.
(<code>[#302 ](https://github.com/twisted/treq/issues/302 )
<https://github.com/twisted/treq/issues/302> ;</code>__)</li>
<li>Support for Python 3.7 and PyPy 3.8, which have reached end of
support, has been dropped.
(<code>[#378 ](https://github.com/twisted/treq/issues/378 )
<https://github.com/twisted/treq/issues/378> ;</code>__)</li>
</ul>
<h2>Misc</h2>
<ul>
<li><code>[#336 ](https://github.com/twisted/treq/issues/336 )
<https://github.com/twisted/treq/issues/336> ;</code><strong>,
<code>[#382 ](https://github.com/twisted/treq/issues/382 )
<https://github.com/twisted/treq/issues/382> ;</code></strong>,
<code>[#395 ](https://github.com/twisted/treq/issues/395 )
<https://github.com/twisted/treq/issues/395> ;</code>__</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="caaf9fcb629cedb088b4https://redirect.github.com/twisted/treq/issues/400 ">#400</a> from
twisted/vendor-multipart-for-now</li>
<li><a
href="4aa1ee8a3cd7c16de8f54fd3c842c2f0a5148cba7f16b87f0a1526431a37https://github.com/defnull/multipa ">https://github.com/defnull/multipa </a>...</li>
<li><a
href="7c52d4917fca3966f57ahttps://redirect.github.com/twisted/treq/issues/398 ">#398</a> from
twisted/397-release-24.9.0</li>
<li>Additional commits viewable in <a
href="https://github.com/twisted/treq/compare/release-23.11.0...treq-24.9.1 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Quentin Gliech <quenting@element.io>
2024-09-25 11:19:03 +02:00
V02460
2fc43e4219
Remove the deprecated cgi module ( #17741 )
...
Removes all uses of the `cgi` module from Synapse. It was deprecated in
Python version 3.11 and removed in version 3.13 ([“dead
battery”](https://docs.python.org/3.13/whatsnew/3.13.html#pep-594-remove-dead-batteries-from-the-standard-library )).
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: Quentin Gliech <quenting@element.io>
2024-09-25 11:15:34 +02:00
dependabot[bot]
b0d2aca164
Bump phonenumbers from 8.13.44 to 8.13.45 ( #17762 )
2024-09-25 06:38:37 +00:00
