From 9d720223f25982500ba4076caef9b70b608e0292 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 20 Aug 2015 14:12:01 +0100 Subject: [PATCH 1/6] Bump version and changelog --- CHANGES.rst | 73 +++++++++++++++++++++++++++++++++++++++++++++ synapse/__init__.py | 2 +- 2 files changed, 74 insertions(+), 1 deletion(-) diff --git a/CHANGES.rst b/CHANGES.rst index 6a5fce899..e2c6afb6a 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -1,3 +1,76 @@ +Changes in synapse v0.10.0-rc1 (2015-08-20) +=========================================== + +Also see v0.9.4-rc1 changelog, which has been amalgamated into this release. + +General: + +* Upgrade to Twisted 15 (PR #173) +* Add support for serving and fetching encryption keys over federation. + (PR #208) +* Add support for logging in with email address (PR #234) +* Add support for new ``m.room.canonical_alias`` event. (PR #233) +* Error if a user tries to register with an email already in use. (PR #211) +* Add extra and improve existing caches (PR #212, #219, #226, #228) +* Batch various storage request (PR #226, #228) +* Fix bug where we didn't correctly log the entity that triggered the request + if the request came in via an application service (PR #230) +* Fix bug where we needlessly regenerated the full list of rooms an AS is + interested in. (PR #232) + + +Configuration: + +* Add ``--generate-keys`` that will generate any missing cert and key files in + the configuration files. This is equivalent to running ``--generate-config`` + on an existing configuration file. (PR #220) +* ``--generate-config`` now no longer requires a ``--server-name`` parameter + when used on existing configuration files. (PR #220) +* Add ``--print-pidfile`` flag that controls the printing of the pid to stdout + of the demonised process. (PR #213) + +Media Repository: + +* Fix bug where we picked a lower resolution image than requested. (PR #205) +* Add support for specifying if a the media repository should dynamically + thumbnail images or not. (PR #206) + +Metrics: + +* Add statistics from the reactor to the metrics API. (PR #224, #225) + +Demo Homeservers: + +* Fix starting the demo homeservers without rate-limiting enabled. (PR #182) +* Fix enabling registration on demo homeservers (PR #223) + + +Changes in synapse v0.9.4-rc1 (2015-07-21) +========================================== + +General: + +* Add basic implementation of receipts. (SPEC-99) +* Add support for configuration presets in room creation API. (PR #203) +* Add auth event that limits the visibility of history for new users. + (SPEC-134) +* Add SAML2 login/registration support. (PR #201. Thanks Muthu Subramanian!) +* Add client side key management APIs for end to end encryption. (PR #198) +* Change power level semantics so that you cannot kick, ban or change power + levels of users that have equal or greater power level than you. (SYN-192) +* Improve performance by bulk inserting events where possible. (PR #193) +* Improve performance by bulk verifying signatures where possible. (PR #194) + + +Configuration: + +* Add support for including TLS certificate chains. + +Media Repository: + +* Add Content-Disposition headers to content repository responses. (SYN-150) + + Changes in synapse v0.9.3 (2015-07-01) ====================================== diff --git a/synapse/__init__.py b/synapse/__init__.py index 96e37308d..5853165a2 100644 --- a/synapse/__init__.py +++ b/synapse/__init__.py @@ -16,4 +16,4 @@ """ This is a reference implementation of a Matrix home server. """ -__version__ = "0.9.3" +__version__ = "0.10.0-rc1" From 23b21e5215d3a4d88925aa28e5bcaeda169d81cc Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 20 Aug 2015 14:25:57 +0100 Subject: [PATCH 2/6] Update changelog --- CHANGES.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGES.rst b/CHANGES.rst index e2c6afb6a..e6d1a3730 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -17,6 +17,7 @@ General: if the request came in via an application service (PR #230) * Fix bug where we needlessly regenerated the full list of rooms an AS is interested in. (PR #232) +* Add support for AS's to use v2_alpha registration API (PR #210) Configuration: From bb9611bd46c77252e7ecf3ef90251cf4cd2f446e Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Thu, 20 Aug 2015 15:08:18 +0100 Subject: [PATCH 3/6] Add generic update instructions to UPGRADE.rst and add link to them from the README.rst --- README.rst | 11 ++++------- UPGRADE.rst | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 7 deletions(-) diff --git a/README.rst b/README.rst index 5ff53f2df..b01d693a4 100644 --- a/README.rst +++ b/README.rst @@ -362,14 +362,11 @@ This should end with a 'PASSED' result:: Upgrading an existing Synapse ============================= -IMPORTANT: Before upgrading an existing synapse to a new version, please -refer to UPGRADE.rst for any additional instructions. - -Otherwise, simply re-install the new codebase over the current one - e.g. -by ``pip install --process-dependency-links -https://github.com/matrix-org/synapse/tarball/master`` -if using pip, or by ``git pull`` if running off a git working copy. +The instructions for upgrading synapse are in `UPGRADE.rst`_. +Please check these instructions as upgrading may require extra steps for some +versions of synapse. +.. _UPGRADE.rst: UPGRADE.rst Setting up Federation ===================== diff --git a/UPGRADE.rst b/UPGRADE.rst index d98460f64..a82dabbd3 100644 --- a/UPGRADE.rst +++ b/UPGRADE.rst @@ -1,3 +1,36 @@ +Upgrading Synapse +================= + +Before upgrading check if any special steps are required to upgrade from the +what you currently have installed to current version of synapse. The extra +instructions that may be required are listed later in this document. + +If you installed synapse in a virtualenv then active that virtualenv before +upgrading. If synapse is installed in a virtualenv in ``~/.synapse/`` then run: + +.. code:: bash + + source ~/.synapse/bin/activate + +If you installed synapse using pip then upgrade to the latest version by +running: + +.. code:: bash + + pip install --upgrade --process-dependency-links https://github.com/matrix-org/synapse/tarball/master + +If you installed synapse using git then upgrade to the latest version by +running: + +.. code:: bash + + # Pull the latest version of the master branch. + git pull + # Update the versions of synapses python dependencies. + python synapse/python_dependencies.py | xargs -n1 pip install + + + Upgrading to v0.9.0 =================== From fd88ea19c0702d970af3a737a094234be1809b9e Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Thu, 20 Aug 2015 15:12:44 +0100 Subject: [PATCH 4/6] Tweak the wording a bit --- UPGRADE.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/UPGRADE.rst b/UPGRADE.rst index a82dabbd3..35a0333a7 100644 --- a/UPGRADE.rst +++ b/UPGRADE.rst @@ -5,28 +5,28 @@ Before upgrading check if any special steps are required to upgrade from the what you currently have installed to current version of synapse. The extra instructions that may be required are listed later in this document. -If you installed synapse in a virtualenv then active that virtualenv before +If synapse was installed in a virtualenv then active that virtualenv before upgrading. If synapse is installed in a virtualenv in ``~/.synapse/`` then run: .. code:: bash source ~/.synapse/bin/activate -If you installed synapse using pip then upgrade to the latest version by +If synapse was installed using pip then upgrade to the latest version by running: .. code:: bash pip install --upgrade --process-dependency-links https://github.com/matrix-org/synapse/tarball/master -If you installed synapse using git then upgrade to the latest version by +If synapse was installed using git then upgrade to the latest version by running: .. code:: bash # Pull the latest version of the master branch. git pull - # Update the versions of synapses python dependencies. + # Update the versions of synapse's python dependencies. python synapse/python_dependencies.py | xargs -n1 pip install From 482648123f608fb6e77dc47960ddda501c8f5c70 Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Thu, 20 Aug 2015 15:20:07 +0100 Subject: [PATCH 5/6] Clean up some of restructured text formatting in the README.rst --- README.rst | 62 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 32 insertions(+), 30 deletions(-) diff --git a/README.rst b/README.rst index 5ff53f2df..01d8cb1a7 100644 --- a/README.rst +++ b/README.rst @@ -7,7 +7,7 @@ Matrix is an ambitious new ecosystem for open federated Instant Messaging and VoIP. The basics you need to know to get up and running are: - Everything in Matrix happens in a room. Rooms are distributed and do not - exist on any single server. Rooms can be located using convenience aliases + exist on any single server. Rooms can be located using convenience aliases like ``#matrix:matrix.org`` or ``#test:localhost:8448``. - Matrix user IDs look like ``@matthew:matrix.org`` (although in the future @@ -23,7 +23,7 @@ The overall architecture is:: accessed by the web client at http://matrix.org/beta or via an IRC bridge at irc://irc.freenode.net/matrix. -Synapse is currently in rapid development, but as of version 0.5 we believe it +Synapse is currently in rapid development, but as of version 0.5 we believe it is sufficiently stable to be run as an internet-facing service for real usage! About Matrix @@ -104,7 +104,7 @@ Installing prerequisites on Ubuntu or Debian:: sudo apt-get install build-essential python2.7-dev libffi-dev \ python-pip python-setuptools sqlite3 \ libssl-dev python-virtualenv libjpeg-dev - + Installing prerequisites on ArchLinux:: sudo pacman -S base-devel python2 python-pip \ @@ -115,7 +115,7 @@ Installing prerequisites on Mac OS X:: xcode-select --install sudo easy_install pip sudo pip install virtualenv - + To install the synapse homeserver run:: virtualenv -p python2.7 ~/.synapse @@ -180,7 +180,7 @@ The advantages of Postgres include: * allowing basic active/backup high-availability with a "hot spare" synapse pointing at the same DB master, as well as enabling DB replication in synapse itself. - + The only disadvantage is that the code is relatively new as of April 2015 and may have a few regressions relative to SQLite. @@ -190,8 +190,8 @@ For information on how to install and use PostgreSQL, please see Running Synapse =============== -To actually run your new homeserver, pick a working directory for Synapse to run -(e.g. ``~/.synapse``), and:: +To actually run your new homeserver, pick a working directory for Synapse to +run (e.g. ``~/.synapse``), and:: cd ~/.synapse source ./bin/activate @@ -214,13 +214,13 @@ defaults to python 3, but synapse currently assumes python 2.7 by default: pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 ):: sudo pip2.7 install --upgrade pip - + You also may need to explicitly specify python 2.7 again during the install request:: pip2.7 install --process-dependency-links \ https://github.com/matrix-org/synapse/tarball/master - + If you encounter an error with lib bcrypt causing an Wrong ELF Class: ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly compile it under the right architecture. (This should not be needed if @@ -228,7 +228,7 @@ installing under virtualenv):: sudo pip2.7 uninstall py-bcrypt sudo pip2.7 install py-bcrypt - + During setup of Synapse you need to call python2.7 directly again:: cd ~/.synapse @@ -236,25 +236,27 @@ During setup of Synapse you need to call python2.7 directly again:: --server-name machine.my.domain.name \ --config-path homeserver.yaml \ --generate-config - + ...substituting your host and domain name as appropriate. Windows Install --------------- Synapse can be installed on Cygwin. It requires the following Cygwin packages: - - gcc - - git - - libffi-devel - - openssl (and openssl-devel, python-openssl) - - python - - python-setuptools +- gcc +- git +- libffi-devel +- openssl (and openssl-devel, python-openssl) +- python +- python-setuptools The content repository requires additional packages and will be unable to process uploads without them: - - libjpeg8 - - libjpeg8-devel - - zlib + +- libjpeg8 +- libjpeg8-devel +- zlib + If you choose to install Synapse without these packages, you will need to reinstall ``pillow`` for changes to be applied, e.g. ``pip uninstall pillow`` ``pip install pillow --user`` @@ -276,8 +278,8 @@ Troubleshooting Troubleshooting Installation ---------------------------- -Synapse requires pip 1.7 or later, so if your OS provides too old a version and -you get errors about ``error: no such option: --process-dependency-links`` you +Synapse requires pip 1.7 or later, so if your OS provides too old a version and +you get errors about ``error: no such option: --process-dependency-links`` you may need to manually upgrade it:: sudo pip install --upgrade pip @@ -288,9 +290,9 @@ created. To reset the installation:: rm -rf /tmp/pip_install_matrix -pip seems to leak *lots* of memory during installation. For instance, a Linux -host with 512MB of RAM may run out of memory whilst installing Twisted. If this -happens, you will have to individually install the dependencies which are +pip seems to leak *lots* of memory during installation. For instance, a Linux +host with 512MB of RAM may run out of memory whilst installing Twisted. If this +happens, you will have to individually install the dependencies which are failing, e.g.:: pip install twisted @@ -301,8 +303,8 @@ will need to export CFLAGS=-Qunused-arguments. Troubleshooting Running ----------------------- -If synapse fails with ``missing "sodium.h"`` crypto errors, you may need -to manually upgrade PyNaCL, as synapse uses NaCl (http://nacl.cr.yp.to/) for +If synapse fails with ``missing "sodium.h"`` crypto errors, you may need +to manually upgrade PyNaCL, as synapse uses NaCl (http://nacl.cr.yp.to/) for encryption and digital signatures. Unfortunately PyNACL currently has a few issues (https://github.com/pyca/pynacl/issues/53) and @@ -313,7 +315,7 @@ fix try re-installing from PyPI or directly from # Install from PyPI pip install --user --upgrade --force pynacl - + # Install from github pip install --user https://github.com/pyca/pynacl/tarball/master @@ -431,7 +433,7 @@ private federation (``localhost:8080``, ``localhost:8081`` and http://localhost:8080. Simply run:: demo/start.sh - + This is mainly useful just for development purposes. Running The Demo Web Client @@ -494,7 +496,7 @@ time. Where's the spec?! ================== -The source of the matrix spec lives at https://github.com/matrix-org/matrix-doc. +The source of the matrix spec lives at https://github.com/matrix-org/matrix-doc. A recent HTML snapshot of this lives at http://matrix.org/docs/spec From 8a951540f6b47c8cc33dc41f3178295c55350c74 Mon Sep 17 00:00:00 2001 From: Mark Haines Date: Thu, 20 Aug 2015 15:22:26 +0100 Subject: [PATCH 6/6] Further formatting clean ups --- README.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.rst b/README.rst index 01d8cb1a7..e63f86b60 100644 --- a/README.rst +++ b/README.rst @@ -174,12 +174,12 @@ traditionally used for convenience and simplicity. The advantages of Postgres include: - * significant performance improvements due to the superior threading and - caching model, smarter query optimiser - * allowing the DB to be run on separate hardware - * allowing basic active/backup high-availability with a "hot spare" synapse - pointing at the same DB master, as well as enabling DB replication in - synapse itself. +* significant performance improvements due to the superior threading and + caching model, smarter query optimiser +* allowing the DB to be run on separate hardware +* allowing basic active/backup high-availability with a "hot spare" synapse + pointing at the same DB master, as well as enabling DB replication in + synapse itself. The only disadvantage is that the code is relatively new as of April 2015 and may have a few regressions relative to SQLite. @@ -326,7 +326,7 @@ If running `$ synctl start` fails with 'returned non-zero exit status 1', you will need to explicitly call Python2.7 - either running as:: python2.7 -m synapse.app.homeserver --daemonize -c homeserver.yaml - + ...or by editing synctl with the correct python executable. Synapse Development