From e7598b666ba6c91f69f312d367017caca8f44a8f Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 23 May 2018 11:14:23 +0100 Subject: [PATCH 01/10] Some docs about server notices --- docs/server_notices.md | 53 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 docs/server_notices.md diff --git a/docs/server_notices.md b/docs/server_notices.md new file mode 100644 index 000000000..a37aed679 --- /dev/null +++ b/docs/server_notices.md @@ -0,0 +1,53 @@ +Server Notices +============== + +'Server Notices' are a new feature introduced in Synapse 0.30. They provide a +channel whereby server administrators can send messages to users on the server. + +They are used as part of the communication of Privacy Polices (see +[privacy_policy.md]), however the intention is that they may also find a use +for features such as "Message of the day". + +This is a feature specific to Synapse, but it uses standard Matrix +communication mechanisms, so should work with any Matrix client. + +User experience +--------------- + +When the user is first sent a server notice, they will get an invitation to a +room (typically called 'Server Notices', though this is configurable in +`homeserver.yaml`). They will be **unable to reject** this invitation - +attempts to do so will receive an error. + +Once they accept the invitation, they will see the notice message in the room +history; it will appear to have come from the 'server notices user' (see +below). + +The user is prevented from sending any messages in this room by the power +levels. They also cannot leave it. + +Synapse configuration +--------------------- + +Server notices come from a specific user id on the server. Server +administrators are free to choose the user id - something like `server` is +suggested, meaning the notices will come from +`@server:`. Once the server notices user is configured, that +user id becomes a special, privileged user, so administrators should ensure +that **it is not already allocated**. + +In order to support server notices, it is necessary to add some configuration +to the `homeserver.yaml` file. In particular, you should add a `server_notices` +section, which should look like this: + +```yaml +server_notices: + system_mxid_localpart: server + system_mxid_display_name: "Server Notices" + room_name: "Server Notices" +``` + +The only compulsory setting is `system_mxid_localpart`, which defines the user +id of the server notices user, as above. `system_mxid_display_name` and +`room_name` define the displayname of the system notices user, and of +the notices room, respectively. From 833db2d92245a0b7cab9e16f9badc135346f0750 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 23 May 2018 12:32:38 +0100 Subject: [PATCH 02/10] consent tracking docs --- docs/consent_tracking.md | 152 ++++++++++++++++++++++++ docs/privacy_policy_templates/README.md | 23 ---- docs/server_notices.md | 4 +- 3 files changed, 154 insertions(+), 25 deletions(-) create mode 100644 docs/consent_tracking.md delete mode 100644 docs/privacy_policy_templates/README.md diff --git a/docs/consent_tracking.md b/docs/consent_tracking.md new file mode 100644 index 000000000..eb34749b5 --- /dev/null +++ b/docs/consent_tracking.md @@ -0,0 +1,152 @@ +Support in Synapse for tracking agreement to server terms and conditions +======================================================================== + +Synapse 0.30 introduces support for tracking whether users have agreed to the +terms and conditions set by the administrator of a server - and blocking access +to the server until they have. + +There are several parts to this functionality; each requires some specific +configuration in `homeserver.yaml` to be enabled. + +Note that various parts of the configuation and this document refer to the +"privacy policy": agreement with a privacy policy is one particular use of this +feature, but of course adminstrators can specify other terms and conditions +unrelated to "privacy" per se. + +Collecting policy agreement from a user +--------------------------------------- + +Synapse can be configured to serve the user a simple policy form with an +"accept" button. Clicking "Accept" records the user's acceptance in the +database and shows a success page. + +To enable this, first create templates for the policy and success pages. +These should be stored on the local filesystem. + +These templates use the [Jinja2](http://jinja.pocoo.org) templating language, +and the `privacy_policy_templates` subdirectory of this `docs` directory gives +examples of the sort of thing that can be done. + +Note that the templates must be stored under a name giving the language of the +template - currently this must always be `en` (for "English"); +internationalisation support is intended for the future. + +The template for the policy itself should be versioned - for example +`1.0.html`. The version of the policy which the user has agreed to is stored in +the database. + +Once the templates are in place, make the following changes to `homeserver.yaml`: + + 1. Add a `user_consent` section, which should look like: + + ```yaml + user_consent: + template_dir: privacy_policy_templates + version: 1.0 + ``` + + `template_dir` points to the directory containing the policy + templates. `version` defines the version of the policy which will be served + to the user. In the example above, Synapse will serve + `privacy_policy_templates/en/1.0.html`. + + + 2. Add a `form_secret` setting at the top level: + + + ```yaml + form_secret: "" + ``` + + This should be set to an arbitrary secret string (try `pwgen -y 30` to + generate suitable secrets). + + More on what this is used for below. + + 3. Add `consent` wherever the `client` resource is currently enabled in the + `listeners` configuration. For example: + + ``` + listeners: + - port: 8008 + resources: + - names: + - client + - consent + ``` + + +Finally, ensure that `jinja2` is installed. If you are using a virtualenv, this +should be a matter of `pip install Jinja2`. On debian, try `apt-get install +python-jinja2`. + +Once this is complete, and the server has been restarted, try visiting +`https:///_matrix/consent`. If configuration has been done correctly, +this should give an error "Missing string query parameter 'u'". It is now +possible to manually construct URIs where users can give their consent. + +Constructing the consent URI +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It may be useful to manually construct the "consent URI" for a given user - for +instance, in order to send them an email asking them to consent. To do this, +take the base `https:///_matrix/consent` URL and add the following +query parameters: + + * `u`: the user id of the user. This can either be a full MXID + (`@user:server.com`) or just the localpart (`user`). + + * `h`: hex-encoded HMAC-SHA256 of `u` using the `form_secret` as a key. It is + possible to calculate this on the commandline with something like: + + ```bash + echo -n '' | openssl sha256 -hmac '' + ``` + + This should result in a URI which looks something like: + `https:///_matrix/consent?u=&h=68a152465a4d...`. + + +Sending users a server notice asking them to agree to the policy +---------------------------------------------------------------- + +It is possible to configure Synapse to send a [server +notice](server_notices.md) to anybody who has not yet agreed to the current +version of the policy. To do so: + + * ensure that the consent resource is configured, as in the previous section + + * ensure that server notices are configured, as in [server_notices.md]. + + * Add `server_notice_content` under `user_consent` in `homeserver.yaml`. For + example: + + ```yaml + user_consent: + server_notice_content: + msgtype: m.text + body: >- + Please give your consent to the privacy policy at %(consent_uri)s. + ``` + + Synapse automatically replaces the placeholder `%(consent_uri)s` with the + consent uri for that user. + +Blocking users from using the server until they agree to the policy +------------------------------------------------------------------- + +Synapse can be configured to block any attempts to join rooms or send messages +until the user has given their agreement to the policy. (Joining the server +notices room is exempted from this). + +To enable this, add `block_events_error` under `user_consent`. For example: + +``` +user_consent: + block_events_error: >- + You can't send any messages until you consent to the privacy policy at + %(consent_uri)s. +``` + +Synapse automatically replaces the placeholder `%(consent_uri)s` with the +consent uri for that user. diff --git a/docs/privacy_policy_templates/README.md b/docs/privacy_policy_templates/README.md deleted file mode 100644 index a3e6fc098..000000000 --- a/docs/privacy_policy_templates/README.md +++ /dev/null @@ -1,23 +0,0 @@ -If enabling the 'consent' resource in synapse, you will need some templates -for the HTML to be served to the user. This directory contains very simple -examples of the sort of thing that can be done. - -You'll need to add this sort of thing to your homeserver.yaml: - -``` -form_secret: - -user_consent: - template_dir: docs/privacy_policy_templates - version: 1.0 -``` - -You should then be able to enable the `consent` resource under a `listener` -entry. For example: - -``` -listeners: - - port: 8008 - resources: - - names: [client, consent] -``` diff --git a/docs/server_notices.md b/docs/server_notices.md index a37aed679..5d7857d15 100644 --- a/docs/server_notices.md +++ b/docs/server_notices.md @@ -4,8 +4,8 @@ Server Notices 'Server Notices' are a new feature introduced in Synapse 0.30. They provide a channel whereby server administrators can send messages to users on the server. -They are used as part of the communication of Privacy Polices (see -[privacy_policy.md]), however the intention is that they may also find a use +They are used as part of the communication of the server polices (see +[consent_tracking.md]), however the intention is that they may also find a use for features such as "Message of the day". This is a feature specific to Synapse, but it uses standard Matrix From 2574ea3dc83c58d1cb596606ac760d91022f336f Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Wed, 23 May 2018 12:34:34 +0100 Subject: [PATCH 03/10] server_notices.md: fix link --- docs/server_notices.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/server_notices.md b/docs/server_notices.md index 5d7857d15..22bca332d 100644 --- a/docs/server_notices.md +++ b/docs/server_notices.md @@ -5,8 +5,8 @@ Server Notices channel whereby server administrators can send messages to users on the server. They are used as part of the communication of the server polices (see -[consent_tracking.md]), however the intention is that they may also find a use -for features such as "Message of the day". +[consent_tracking.md](consent_tracking.md)), however the intention is that +they may also find a use for features such as "Message of the day". This is a feature specific to Synapse, but it uses standard Matrix communication mechanisms, so should work with any Matrix client. From 563606b8f2216f6a8ba64c4b55bf389f67c99e9e Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Wed, 23 May 2018 12:37:39 +0100 Subject: [PATCH 04/10] consent_tracking: formatting etc --- docs/consent_tracking.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/docs/consent_tracking.md b/docs/consent_tracking.md index eb34749b5..61e41468d 100644 --- a/docs/consent_tracking.md +++ b/docs/consent_tracking.md @@ -31,9 +31,9 @@ Note that the templates must be stored under a name giving the language of the template - currently this must always be `en` (for "English"); internationalisation support is intended for the future. -The template for the policy itself should be versioned - for example -`1.0.html`. The version of the policy which the user has agreed to is stored in -the database. +The template for the policy itself should be versioned and named according to +the version: for example `1.0.html`. The version of the policy which the user +has agreed to is stored in the database. Once the templates are in place, make the following changes to `homeserver.yaml`: @@ -66,7 +66,7 @@ Once the templates are in place, make the following changes to `homeserver.yaml` 3. Add `consent` wherever the `client` resource is currently enabled in the `listeners` configuration. For example: - ``` + ```yaml listeners: - port: 8008 resources: @@ -85,8 +85,7 @@ Once this is complete, and the server has been restarted, try visiting this should give an error "Missing string query parameter 'u'". It is now possible to manually construct URIs where users can give their consent. -Constructing the consent URI -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +### Constructing the consent URI It may be useful to manually construct the "consent URI" for a given user - for instance, in order to send them an email asking them to consent. To do this, @@ -116,7 +115,7 @@ version of the policy. To do so: * ensure that the consent resource is configured, as in the previous section - * ensure that server notices are configured, as in [server_notices.md]. + * ensure that server notices are configured, as in [server_notices.md](server_notices.md). * Add `server_notice_content` under `user_consent` in `homeserver.yaml`. For example: @@ -141,7 +140,7 @@ notices room is exempted from this). To enable this, add `block_events_error` under `user_consent`. For example: -``` +```yaml user_consent: block_events_error: >- You can't send any messages until you consent to the privacy policy at From 5ad1149f384e1a826ad73a791896d63c910d874c Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 23 May 2018 13:45:17 +0100 Subject: [PATCH 05/10] Notes on the manhole --- docs/manhole.md | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 docs/manhole.md diff --git a/docs/manhole.md b/docs/manhole.md new file mode 100644 index 000000000..7375f5ad4 --- /dev/null +++ b/docs/manhole.md @@ -0,0 +1,43 @@ +Using the synapse manhole +========================= + +The "manhole" allows server administrators to access a Python shell on a running +Synapse installation. This is a very powerful mechanism for administration and +debugging. + +To enable it, first uncomment the `manhole` listener configuration in +`homeserver.yaml`: + +```yaml +listeners: + - port: 9000 + bind_addresses: ['::1', '127.0.0.1'] + type: manhole +``` + +(`bind_addresses` in the above is important: it ensures that access to the +manhole is only possible for local users). + +Note that this will give administrative access to synapse to **all users** with +shell access to the server. It should therefore **not** be enabled in +environments where untrusted users have shell access. + +Then restart synapse, and point an ssh client at port 9000 on localhost, using +the username `matrix`: + +```bash +ssh -p9000 matrix@localhost +``` + +The password is `rabbithole`. + +This gives a Python REPL in which `hs` gives access to the +`synapse.server.HomeServer` object - which in turn gives access to many other +parts of the process. + +As a simple example, retrieving an event from the database: + +``` +>>> hs.get_datastore().get_event('$1416420717069yeQaw:matrix.org') +> +``` From 052d08a6a579f4f9a98a76a3cb39c05d8d5aaa90 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 23 May 2018 13:55:39 +0100 Subject: [PATCH 06/10] Using the manhole to send server notices --- docs/server_notices.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/server_notices.md b/docs/server_notices.md index 22bca332d..a96a5b88a 100644 --- a/docs/server_notices.md +++ b/docs/server_notices.md @@ -51,3 +51,16 @@ The only compulsory setting is `system_mxid_localpart`, which defines the user id of the server notices user, as above. `system_mxid_display_name` and `room_name` define the displayname of the system notices user, and of the notices room, respectively. + +Sending notices +--------------- + +As of the current version of synapse, there is no convenient interface for +sending notices (other than the automated ones sent as part of consent +tracking). + +In the meantime, it is possible to test this feature using the manhole. Having gone into the manhole as described in [manhole.md](manhole.md), a notice can be sent with something like: + +``` +>>> hs.get_server_notices_manager().send_notice('@user:server.com', {'msgtype':'m.text', 'body':'foo'}) +``` From 1cbb8e5a33326441ec311ce212bb62fcd6704669 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 23 May 2018 13:58:28 +0100 Subject: [PATCH 07/10] fix wrapping --- docs/server_notices.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/server_notices.md b/docs/server_notices.md index a96a5b88a..989688138 100644 --- a/docs/server_notices.md +++ b/docs/server_notices.md @@ -59,7 +59,9 @@ As of the current version of synapse, there is no convenient interface for sending notices (other than the automated ones sent as part of consent tracking). -In the meantime, it is possible to test this feature using the manhole. Having gone into the manhole as described in [manhole.md](manhole.md), a notice can be sent with something like: +In the meantime, it is possible to test this feature using the manhole. Having +gone into the manhole as described in [manhole.md](manhole.md), a notice can be +sent with something like: ``` >>> hs.get_server_notices_manager().send_notice('@user:server.com', {'msgtype':'m.text', 'body':'foo'}) From cd8ab9a0d8d3e0ee305ab56788f76459596075f9 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 23 May 2018 14:43:09 +0100 Subject: [PATCH 08/10] mention public_baseurl --- docs/consent_tracking.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/consent_tracking.md b/docs/consent_tracking.md index 61e41468d..66beb9263 100644 --- a/docs/consent_tracking.md +++ b/docs/consent_tracking.md @@ -131,6 +131,11 @@ version of the policy. To do so: Synapse automatically replaces the placeholder `%(consent_uri)s` with the consent uri for that user. + * ensure that `public_baseurl` is set in `homeserver.yaml`, and gives the base + URI that clients use to connect to the server. (It is used to construct + `consent_uri` in the server notice.) + + Blocking users from using the server until they agree to the policy ------------------------------------------------------------------- @@ -149,3 +154,7 @@ user_consent: Synapse automatically replaces the placeholder `%(consent_uri)s` with the consent uri for that user. + +ensure that `public_baseurl` is set in `homeserver.yaml`, and gives the base +URI that clients use to connect to the server. (It is used to construct +`consent_uri` in the error.) From 2df8c3139ab7be10af4aaa750bbdc4ad39209583 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 23 May 2018 15:39:52 +0100 Subject: [PATCH 09/10] minor post-review tweaks --- docs/consent_tracking.md | 6 +++--- docs/server_notices.md | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/consent_tracking.md b/docs/consent_tracking.md index 66beb9263..9dc009388 100644 --- a/docs/consent_tracking.md +++ b/docs/consent_tracking.md @@ -81,9 +81,9 @@ should be a matter of `pip install Jinja2`. On debian, try `apt-get install python-jinja2`. Once this is complete, and the server has been restarted, try visiting -`https:///_matrix/consent`. If configuration has been done correctly, -this should give an error "Missing string query parameter 'u'". It is now -possible to manually construct URIs where users can give their consent. +`https:///_matrix/consent`. If correctly configured, this should give +an error "Missing string query parameter 'u'". It is now possible to manually +construct URIs where users can give their consent. ### Constructing the consent URI diff --git a/docs/server_notices.md b/docs/server_notices.md index 989688138..8e18e3d95 100644 --- a/docs/server_notices.md +++ b/docs/server_notices.md @@ -4,7 +4,7 @@ Server Notices 'Server Notices' are a new feature introduced in Synapse 0.30. They provide a channel whereby server administrators can send messages to users on the server. -They are used as part of the communication of the server polices (see +They are used as part of communication of the server polices(see [consent_tracking.md](consent_tracking.md)), however the intention is that they may also find a use for features such as "Message of the day". @@ -32,7 +32,7 @@ Synapse configuration Server notices come from a specific user id on the server. Server administrators are free to choose the user id - something like `server` is suggested, meaning the notices will come from -`@server:`. Once the server notices user is configured, that +`@server:`. Once the Server Notices user is configured, that user id becomes a special, privileged user, so administrators should ensure that **it is not already allocated**. From e206b2c9ace8a202bf47ec419581e19f0baa39fb Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Wed, 23 May 2018 15:57:10 +0100 Subject: [PATCH 10/10] consent_tracking.md: clarify link --- docs/consent_tracking.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/consent_tracking.md b/docs/consent_tracking.md index 9dc009388..064eae82f 100644 --- a/docs/consent_tracking.md +++ b/docs/consent_tracking.md @@ -24,7 +24,7 @@ To enable this, first create templates for the policy and success pages. These should be stored on the local filesystem. These templates use the [Jinja2](http://jinja.pocoo.org) templating language, -and the `privacy_policy_templates` subdirectory of this `docs` directory gives +and [docs/privacy_policy_templates](privacy_policy_templates) gives examples of the sort of thing that can be done. Note that the templates must be stored under a name giving the language of the