0
0
Fork 1
mirror of https://mau.dev/maunium/synapse.git synced 2024-12-14 21:33:53 +01:00
Commit graph

24347 commits

Author SHA1 Message Date
dependabot[bot]
6ececb8f2a
Bump psycopg2 from 2.9.9 to 2.9.10 (#17843)
Bumps [psycopg2](https://github.com/psycopg/psycopg2) from 2.9.9 to
2.9.10.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/psycopg/psycopg2/blob/master/NEWS">psycopg2's
changelog</a>.</em></p>
<blockquote>
<h2>Current release</h2>
<p>What's new in psycopg 2.9.10
^^^^^^^^^^^^^^^^^^^^^^^^^^^^</p>
<ul>
<li>Add support for Python 3.13.</li>
<li>Receive notifications on commit
(🎫<code>[#1728](https://github.com/psycopg/psycopg2/issues/1728)</code>).</li>
<li><code>~psycopg2.errorcodes</code> map and
<code>~psycopg2.errors</code> classes updated to
PostgreSQL 17.</li>
<li>Drop support for Python 3.7.</li>
</ul>
<p>What's new in psycopg 2.9.9
^^^^^^^^^^^^^^^^^^^^^^^^^^^</p>
<ul>
<li>Add support for Python 3.12.</li>
<li>Drop support for Python 3.6.</li>
</ul>
<p>What's new in psycopg 2.9.8
^^^^^^^^^^^^^^^^^^^^^^^^^^^</p>
<ul>
<li>Wheel package bundled with PostgreSQL 16 libpq in order to add
support for
recent features, such as <code>sslcertmode</code>.</li>
</ul>
<p>What's new in psycopg 2.9.7
^^^^^^^^^^^^^^^^^^^^^^^^^^^</p>
<ul>
<li>Fix propagation of exceptions raised during module initialization

(🎫<code>[#1598](https://github.com/psycopg/psycopg2/issues/1598)</code>).</li>
<li>Fix building when pg_config returns an empty string
(🎫<code>[#1599](https://github.com/psycopg/psycopg2/issues/1599)</code>).</li>
<li>Wheel package bundled with OpenSSL 1.1.1v.</li>
</ul>
<p>What's new in psycopg 2.9.6
^^^^^^^^^^^^^^^^^^^^^^^^^^^</p>
<ul>
<li>Package manylinux 2014 for aarch64 and ppc64le platforms, in order
to
include libpq 15 in the binary package
(🎫<code>[#1396](https://github.com/psycopg/psycopg2/issues/1396)</code>).</li>
<li>Wheel package bundled with OpenSSL 1.1.1t.</li>
</ul>
<p>What's new in psycopg 2.9.5
^^^^^^^^^^^^^^^^^^^^^^^^^^^</p>
<ul>
<li>Add support for Python 3.11.</li>
<li>Add support for rowcount in MERGE statements in binary packages

(🎫<code>[#1497](https://github.com/psycopg/psycopg2/issues/1497)</code>).</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/psycopg/psycopg2/commits">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=psycopg2&package-manager=pip&previous-version=2.9.9&new-version=2.9.10)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-17 14:29:05 +00:00
Erik Johnston
2ce7a1edf7 Merge branch 'master' into develop 2024-10-15 15:01:48 +01:00
Erik Johnston
ec885ffd33 1.117.0 2024-10-15 10:46:33 +01:00
Tulir Asokan
11bc9a1b3a
Implement MSC4210: Remove legacy mentions (#17783) 2024-10-14 14:24:28 +01:00
Andrew Morgan
c5b379de66
Enable the .org.matrix.msc4028.encrypted_event push rule by default (#17826)
Clients will still only see this rule if the corresponding experimental
feature, `msc4028_push_encrypted_events`, is also enabled.

This aligns the implementation with MSC4028, specifically [this
section](https://github.com/matrix-org/matrix-spec-proposals/blob/giomfo/push_encrypted_events/proposals/4028-push-all-encrypted-events-except-for-muted-rooms.md#unstable-prefix).
See https://github.com/element-hq/synapse/issues/16846 for context.
2024-10-14 13:49:43 +01:00
Eric Eastwood
adda2a4613
Sliding Sync: Slight optimization when fetching state for the room (get_events_as_list(...)) (#17718)
Spawning from @kegsay [pointing
out](https://matrix.to/#/!cnVVNLKqgUzNTOFQkz:matrix.org/$ExOO7J8uPUQSyH-9Uxc_QCa8jlXX9uK4VRtkSC0EI3o?via=element.io&via=matrix.org&via=jki.re)
that the Sliding Sync endpoint doesn't handle a large room with a lot of
state well on initial sync (requesting all state via `required_state: [
["*","*"] ]`) (it just takes forever).

After investigating further, the slow part is just
`get_events_as_list(...)` fetching all of the current state ID's out for
the room (which can be 100k+ events for rooms with a lot of membership).
This is just a slow thing in Synapse in general and the same thing
happens in Sync v2 or the `/state` endpoint.


---

The only idea I had to improve things was to use `batch_iter` to only
try fetching a fixed amount at a time instead of working with large
maps, lists, and sets. This doesn't seem to have much effect though.

There is already a `batch_iter(event_ids, 200)` in
`_fetch_event_rows(...)` for when we actually have to touch the database
and that's inside a queue to deduplicate work.

I did notice one slight optimization to use `get_events_as_list(...)`
directly instead of `get_events(...)`. `get_events(...)` just turns the
result from `get_events_as_list(...)` into a dict and since we're just
iterating over the events, we don't need the dict/map.
2024-10-14 13:47:35 +01:00
Andrew Morgan
5d47138b46
Fix typo in target_cache_memory_usage docs (#17825) 2024-10-14 13:34:55 +01:00
Erik Johnston
d025b5ab50
Correctly changes to required state config in sliding sync (#17785)
Fixes https://github.com/element-hq/synapse/issues/17698

This handles `required_state` changes by checking if new state has been
added to the config, and if so fetching and returning that from the
current state.

This also takes care to ensure that given a state entry S that is added,
removed and then re-added that we do *not* send S down a second time if
there have been no changes to S in the current state. This is fine for
Rust SDK (as it just remembers all state), but we might decide not to do
this behaviour in the MSC. If we decide to always send down S then its
easy enough to rip out all the code.

---------

Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
2024-10-14 13:31:22 +01:00
dependabot[bot]
ae6179b382
Bump mypy-zope from 1.0.5 to 1.0.7 (#17827) 2024-10-14 13:26:40 +01:00
dependabot[bot]
5dd6157972
Bump types-setuptools from 75.1.0.20240917 to 75.1.0.20241014 (#17828) 2024-10-14 13:26:23 +01:00
dependabot[bot]
1266138b66
Bump sentry-sdk from 2.15.0 to 2.16.0 (#17829) 2024-10-14 13:26:12 +01:00
Erik Johnston
24975eca4d
Build debian packages for new Ubuntu versions (#17824)
c.f. https://wiki.ubuntu.com/Releases for the currently supported Ubuntu
releases.

Note: this removes support for 23.04 and 23.10, which are EOL.

Fixes #17811
2024-10-14 11:34:33 +01:00
Andrew Morgan
451a9dc7b9
Clarify when 3PID invite module callbacks are called (#17627)
Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
2024-10-14 11:31:49 +01:00
Erik Johnston
f6a3e5e1c2
Fix release script to check GH token (#17803)
The current logic didn't work.
2024-10-10 08:59:01 +00:00
Nathan
05576f0b4b
Added display_name_claim in jwt_config which sets the user's display name upon registration (#17708) 2024-10-09 12:21:08 +00:00
Martin Weinelt
60aebdb27e
Fix saving of non-RGB thumbnails as PNG (#17736) 2024-10-08 18:32:25 +01:00
Erik Johnston
b1b4b2944d Merge branch 'release-v1.117' into develop 2024-10-08 16:35:35 +01:00
Andrew Ferrazzutti
bdcc9fa388
Fix incorrectly documented config path argument (#17802) 2024-10-08 15:05:36 +01:00
Erik Johnston
6a0c21fabd Fixup changlog 2024-10-08 15:04:20 +01:00
dependabot[bot]
f40641c29b
Bump sigstore/cosign-installer from 3.6.0 to 3.7.0 (#17798)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:56:15 +01:00
dependabot[bot]
1bb528ee44
Bump phonenumbers from 8.13.46 to 8.13.47 (#17797)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:41:27 +01:00
dependabot[bot]
165f4ca776
Bump sentry-sdk from 2.14.0 to 2.15.0 (#17795)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:41:03 +01:00
dependabot[bot]
475e192cbe
Bump tomli from 2.0.1 to 2.0.2 (#17796)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:40:12 +01:00
dependabot[bot]
43040a4051
Bump ruff from 0.6.8 to 0.6.9 (#17794)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 14:39:19 +01:00
Erik Johnston
b3e2d10f39 1.117.0rc1 2024-10-08 14:37:36 +01:00
Shay
a5986ac229
Improvements to admin redact api (#17792)
- better validation on user input
- fix an early task completion
- when checking membership in rooms, check for rooms user has been
banned from as well
2024-10-08 14:23:21 +01:00
Andrew Ferrazzutti
006251a5d0
Add missing license header (#17799)
Co-authored-by: Erik Johnston <erik@matrix.org>
2024-10-08 12:01:44 +01:00
Erik Johnston
422f3ecec1
Sliding sync: omit bump stamp when it is unchanged (#17788)
This saves some DB lookups in rooms
2024-10-08 11:17:23 +01:00
Erik Johnston
4e90221d87
Sliding sync minor performance speed up using new table (#17787)
Use the new tables to work out which rooms have changed.

---------

Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
2024-10-08 11:06:31 +01:00
Erik Johnston
e2610de208
Speed up sliding sync when there are many active subscriptions (#17789)
Two changes: a) use a batch lookup function instead of a loop, b) check
existing data to see if we already have what we need and only fetch what
we don't.
2024-10-08 10:35:15 +01:00
Andrew Morgan
e8c8924b81
Clarify test_forget_when_not_left docstring (#17628) 2024-10-07 16:34:32 +01:00
V02460
e8e0f0fad7
Add config option redis.password_path (#17717)
Adds the option to load the Redis password from a file, instead of
giving it in the config directly. The code is similar to how it’s done
for `registration_shared_secret_path`. I changed the example in the
documentation to represent the best practice regarding the handling of
secrets.

Reading secrets from files has the security advantage of separating the
secrets from the config. It also simplifies secrets management in
Kubernetes.
2024-10-07 09:46:51 +01:00
Henrique
beb7a951f4
docs: add note about PYTHONMALLOC for accurate jemalloc memory tracking (#17709)
Added a note in the documentation suggesting that users may set
`PYTHONMALLOC=malloc` when using `jemalloc`. This allows jemalloc to
track memory usage more accurately by bypassing Python's internal
small-object allocator (`pymalloc`), helping to ensure that
`cache_autotuning` functions as expected.

This doc change aims to provide more clarity for users configuring
jemalloc with Synapse.


Based on:
4ac783549c/synapse/metrics/jemalloc.py (L198-L201)
2024-10-07 08:37:39 +00:00
dependabot[bot]
d34f827ed8
Bump python-multipart from 0.0.10 to 0.0.12 (#17772) 2024-10-07 09:14:30 +01:00
Andrew Ferrazzutti
9920417723
Don't say MSC4140 is supported when it's disabled (#17780) 2024-10-04 13:42:34 +01:00
Andrew Morgan
316d635906
Fix NAME attribute of ReplicationRemovePusherRestServlet (#17779) 2024-10-04 09:53:35 +01:00
Dirk Klimpel
8bbe66a9b9
explain load balancing for federation_sender_instances (#17776)
Adding information on how the load is distributed for
`federation_sender_instances`.

Thx to @devonh for the information.

causal source:
c2e5e9e67c/synapse/config/_base.py (L946-L989)

### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))

---------

Co-authored-by: Devon Hudson <devon.dmytro@gmail.com>
2024-10-03 22:01:33 +00:00
Andrew Morgan
d4e3ad04cd Merge branch 'master' into develop 2024-10-01 12:18:22 +01:00
Andrew Morgan
55c0391cc8 1.116.0 2024-10-01 11:14:13 +01:00
Erik Johnston
81e0f57800
Fix perf when streams don't change often (#17767)
There is a bug with the `StreamChangeCache` where it would incorrectly
return that all entities had changed if asked for entities changed
*since* the earliest stream position.

Note that for streams we use the inequalities: `$min_stream_id <
stream_id <= $max_stream_id`, i.e. when we ask the stream change cache
for all things that have changed since `$stream_id` we don't care for
events that happened *at* `$stream_id`.

Specifically: `_earliest_known_stream_pos` is the position at which we
know that we'll have entries for all changes since that point, we can
use the cache for any stream IDs that equal
`_earliest_known_stream_pos`.

`_earliest_known_stream_pos` is set in three places:
- On startup we set it either to:
  - the current maximum stream ID, with not prefilled values; or
  - the minimum of the latest N values we pulled from the DB
- When we evict items from the bottom, we set it to the stream ID of the
evicted items.

This was changed in https://github.com/matrix-org/synapse/pull/14435,
but I think we were overly conservative there.

---------

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-09-30 13:52:33 +01:00
Erik Johnston
ae4862c38f
Optimise notifier mk2 (#17766)
Based on #17765.

Basically the idea is to reduce the overhead of calling
`ObservableDeferred` in a loop. The two gains are: a) just using a list
of deferreds rather than the machinery of `ObservableDeferred`, and b)
only calling `PreseverLoggingContext` once.

`PreseverLoggingContext` in particular is expensive to call a lot as
each time it needs to call `get_thread_resource_usage` twice, so that it
an update the CPU metrics of the log context.
2024-09-30 13:32:31 +01:00
dependabot[bot]
602956ef64
Bump ruff from 0.6.7 to 0.6.8 (#17774) 2024-09-30 13:08:56 +01:00
dependabot[bot]
444b565c76
Bump phonenumbers from 8.13.45 to 8.13.46 (#17773) 2024-09-30 13:07:57 +01:00
dependabot[bot]
8068f31146
Bump regex from 1.10.6 to 1.11.0 (#17770) 2024-09-30 13:06:43 +01:00
Erik Johnston
5210565c12
Reduce overhead of sliding sync E2EE loops (#17771)
Mainly toning down logging and only calling
`get_membership_from_event_ids` if something has changed.
2024-09-30 13:00:14 +01:00
Erik Johnston
de955293cf
Add fast path for sliding sync streams that only ask for extensions (#17768)
Principally useful for EX e2ee sliding sync connections.
2024-09-30 12:59:50 +01:00
Erik Johnston
93889eb2e7
Optimise notifier (#17765)
The notifier is quite inefficient when it has to wake up many user
streams all at once

From a silly benchmark this takes the time to notify 1M user streams
from ~30s to ~5s
2024-09-30 12:58:13 +01:00
Erik Johnston
ece66ba61c
Minor perf speed up for large accounts on SSS (#17751)
This works as instead of passing *all* rooms to `record_sent_rooms` we
only need to pass rooms that were previously not in the LIVE state.

This came from a py-spy where we were spending ~10% CPU calling these
functions. Note that `record_sent_rooms` is a no-op for rooms that are
already in the `LIVE` state, so we only need to call them for
`PREVIOUSLY` or `INITIAL` rooms.
2024-09-30 12:58:02 +01:00
Quentin Gliech
ef9ef99f59
Merge branch 'release-v1.116' into develop 2024-09-26 16:19:32 +02:00
Quentin Gliech
cfbddc258f
1.116.0rc2 2024-09-26 15:29:13 +02:00