Today we release Synapse 1.2.1 as a critical security update. It contains patches relating to redactions and event federation. The patches address long standing bugs, and are not regressions specific to the previous version (1.2). All admins, regardless of current version, should upgrade asap.
This release includes four security fixes:
Prevent an attack where a federated server could send redactions for arbitrary events in v1 and v2 rooms. (#5767)
Prevent a denial-of-service attack where cycles of redaction events would make Synapse spin infinitely. Thanks to @lrizika:matrix.org for identifying and responsibly disclosing this issue. (0f2ecb961)
Prevent an attack where users could be joined or parted from public rooms without their consent. Thanks to Dylanger for identifying and responsibly disclosing this issue. (#5744)
Fix a vulnerability where a federated server could spoof read-receipts from
users on other servers. Thanks to Dylanger for identifying this issue too. (#5743)
Additionally, the following fix was in Synapse 1.2.0, but was not correctly
identified during the original release:
It was possible for a room moderator to send a redaction for an m.room.create event, which would downgrade the room to version 1. Thanks to @/dev/ponies:ponies.im for identifying and responsibly disclosing this issue! (#5701)
Hey hey, Synapse 1.2.0 is here. It contains aggregations support, better error handling for deactivated accounts and some important bug fixes for redacting messages. Special thanks to community members skalarproduktraum and Lrizika for submissions to improve our documentation.
Move logging code out of synapse.util and into synapse.logging. (#5606, #5617)
Add a blacklist file to the repo to blacklist certain sytests from failing CI. (#5611)
Make runtime errors surrounding password reset emails much clearer. (#5616)
Remove dead code for persiting outgoing federation transactions. (#5622)
Add lint.sh to the scripts-dev folder which will run all linting steps required by CI. (#5627)
Move RegistrationHandler.get_or_create_user to test code. (#5628)
Add some more common python virtual-environment paths to the black exclusion list. (#5630)
Some counter metrics exposed over Prometheus have been renamed, with the old names preserved for backwards compatibility and deprecated. See docs/metrics-howto.rst for details. (#5636)
Right folks, this is our first post 1.0 release, which means that we have now
officially dropped support for Python 2 and Postgres
9.4.
This means that we can start making use of Python 3 specific features and you
should expect lots of associated performance wins over the coming months. See the upgrade notes for more.
Synapse 1.1.0 also contains a reworked approach to the Docker image, as well lots
of performance improvements with special focus on DB IO - expect more to come
in this area.
Special thanks to community member Alexander
Trost for rounding out our SAML support and also
to Daniel Hoffend for contributing the ability
to disable local password authentication.
Fix regression in 1.1rc1 where OPTIONS requests to the media repo would fail. (#5593)
Removed the SYNAPSE_SMTP_* docker container environment variables. Using these environment variables prevented the docker container from starting in Synapse v1.0, even though they didn't actually allow any functionality anyway. (#5596)
Fix a number of "Starting txn from sentinel context" warnings. (#5605)
Added possibility to disable local password authentication. Contributed by Daniel Hoffend. (#5092)
Add monthly active users to phonehome stats. (#5252)
Allow expired user to trigger renewal email sending manually. (#5363)
Statistics on forward extremities per room are now exposed via Prometheus. (#5384, #5458, #5461)
Add --no-daemonize option to run synapse in the foreground, per issue #4130. Contributed by Soham Gumaste. (#5412, #5587)
Fully support SAML2 authentication. Contributed by Alexander Trost - thank you! (#5422)
Allow server admins to define implementations of extra rules for allowing or denying incoming events. (#5440, #5474, #5477)
Add support for handling pagination APIs on client reader worker. (#5505, #5513, #5531)
Improve help and cmdline option names for --generate-config options. (#5512)
Allow configuration of the path used for ACME account keys. (#5516, #5521, #5522)
Add --data-dir and --open-private-ports options. (#5524)
Split public rooms directory auth config in two settings, in order to manage client auth independently from the federation part of it. Obsoletes the "restrict_public_rooms_to_local_users" configuration setting. If "restrict_public_rooms_to_local_users" is set in the config, Synapse will act as if both new options are enabled, i.e. require authentication through the client API and deny federation requests. (#5534)
The minimum TLS version used for outgoing federation requests can now be set with federation_client_minimum_tls_version. (#5550)
Optimise devices changed query to not pull unnecessary rows from the database, reducing database load. (#5559)
Add new metrics for number of forward extremities being persisted and number of state groups involved in resolution. (#5476)
Fix bug processing incoming events over federation if call to /get_missing_events fails. (#5042)
Prevent more than one room upgrade happening simultaneously on the same room. (#5051)
Fix a bug where running synapse_port_db would cause the account validity feature to fail because it didn't set the type of the email_sent column to boolean. (#5325)
Warn about disabling email-based password resets when a reset occurs, and remove warning when someone attempts a phone-based reset. (#5387)
Fix email notifications for unnamed rooms with multiple people. (#5388)
Fix exceptions in federation reader worker caused by attempting to renew attestations, which should only happen on master worker. (#5389)
Fix handling of failures fetching remote content to not log failures as exceptions. (#5390)
Fix a bug where deactivated users could receive renewal emails if the account validity feature is on. (#5394)
Fix missing invite state after exchanging 3PID invites over federaton. (#5464)
Fix intermittent exceptions on Apple hardware. Also fix bug that caused database activity times to be under-reported in log lines. (#5498)
Fix logging error when a tampered event is detected. (#5500)
Fix bug where clients could tight loop calling /sync for a period. (#5507)
Fix bug with jinja2 preventing Synapse from starting. Users who had this problem should now simply need to run pip install matrix-synapse. (#5514)
Fix a regression where homeservers on private IP addresses were incorrectly blacklisted. (#5523)
Fixed m.login.jwt using unregistered user_id and added pyjwt>=1.6.4 as jwt conditional dependencies. Contributed by Pau Rodriguez-Estivill. (#5555, #5586)
Fix a bug that would cause invited users to receive several emails for a single 3PID invite in case the inviter is rate limited. (#5576)
Fix "Unexpected entry in 'full_schemas'" log warning. (#5509)
Improve logging when generating config files. (#5510)
Refactor and clean up Config parser for maintainability. (#5511)
Make the config clearer in that email.template_dir is relative to the Synapse's root directory, not the synapse/ folder within it. (#5543)
Update v1.0.0 release changelog to include more information about changes to password resets. (#5545)
Remove non-functioning check_event_hash.py dev script. (#5548)
Synapse will now only allow TLS v1.2 connections when serving federation, if it terminates TLS. As Synapse's allowed ciphers were only able to be used in TLSv1.2 before, this does not change behaviour. (#5550)
Logging when running GC collection on generation 0 is now at the DEBUG level, not INFO. (#5557)
Reduce the amount of stuff we send in the docker context. (#5564)
Point the reverse links in the Purge History contrib scripts at the intended location. (#5570)
The big news this week is that we declared ourselves to be out of beta. You'll want the full post for all details, but here's a taste:
We are very excited to announce the first fully stable release of the Matrix protocol and specification across all APIs - as well as the Synapse 1.0 reference implementation which implements the full Matrix 1.0 API surface.
This means that after just over 5 years since the initial work on Matrix began, we are proud to have finally exited beta!! This is the conclusion of the work which we announced at FOSDEM 2019 when we cut the first stable release of the Server-Server API and began the Synapse 0.99 release series in anticipation of releasing a 1.0.
As part of the 1.0 we also announced the Matrix.org Foundation.
For the full update on the Foundation, please check out the new website content at https://matrix.org/foundation which should tell you everything you could possibly want to know about the Foundation, the Guardians, the Foundation’s legal Articles of Association, and the day-to-day Rules which define the Open Governance process.
Thanks to everyone who has stuck with us along the way to make this possible.
We released Synapse 1.0.0 this week to coincide with Matrix 1.0. As a release Synapse 1.0.0 focuses on security and stability which in turn builds a firm foundation for the performance improvements that you can expect to see over the Summer. Read all about it here.
We have not one but two working end to end demos of device cross signing! This is going to be huge and is key step towards being able to make Matrix e2ee by default.
Cross Signing Demo
Additional Treats
But wait there's more! Pantalaimon not only provides a way for clients and bots to participate in e2ee rooms, it now provides the ability to search. Checkout this (religious themed) demo.
Pattle is going great guns at the moment. Here's Wilko:-
A new version of Pattle has been pushed to F-droid!
Add ability to create group chats!
Show chat creation events ('Wilko has created this group')!
Show emote messages correctly!
Handle display name changes! Display names of messages will now be as they were at time of sending.
Don't show invite and join events in direct chats This is only happens for the two initial users in the direct chat. If someone invites someone else to the direct chat (trough another client), the invitation will show up in the timeline.
Use the timeout parameter while syncing. This means that receiving new messages should be way quicker! (Thanks Mathieu!)
Store messages retrieved remotely (thanks Mathieu!) This means that scrolling up in a chat will be faster now, because the messages are cached.
Always show a date header above the oldest event
Show replies correctly in chat overview
Show sent state icon next to own message in chat overview
Show newly joined rooms at the top in the chat overview
Use a bit bolder font for chat names in overview
To install this release, add the following repo in F-droid:
Crypto has been merged to develop \o/. We are still working on the feature, for the remaining actions: delete device, export and import keys, keys backup / SAS UI polishing, cleanup keys when signing out, and also fixing bugs…
A new screen has been added to create Rooms.
Animation of the Floating Action Button on the catchup screen has been improved.
Valere has started to work on notifications.
François is working on migration to Dagger2 (instead of Koin). It should improve performance and will allow us to implement multi-account support(!)
Copyright, Term and conditions, privacy policy and third party license screens are coming soon.
Progress indicator on Home for initial/catchup sync is coming soon as well.
Editing now supports editing unsent messages(!!), editing emotes, and lots of polish
Reactions now instantly cancel when you redact them
Redactions now instantly redact when you send them
Released v1.2.2-rc1 (up for testing at https://riot.im/staging/) - this release includes room breadcrumbs being out of labs, some fixes for inviting by email, fixes uploads in chrome canary / firefox nightly, config file validation, some fixes for the new emoji font and lots of other bug fixes.
New versions of the ruma-api and ruma-api-macros libraries were released, and work is underway to bring ruma-events up to date with client-server spec r0.5.0
I wrote a bot to control ansible playbooks https://github.com/Half-Shot/matrix-ansible-bot. I'm informed such a thing already exists as an ansible module, but I failed to realise that and wrote a independent one in TypeScript.
I made a simple reminder maubot: https://github.com/maubot/reminder It's available at https://matrix.to/#/@reminder:maunium.net
For room admins wondering if they can upgrade their rooms to v5, I made a bash script that checks all the servers in a room and prints a nice summary of the number of servers and members on each version: https://gist.github.com/tulir/aa2df287a0d192b86e5b675687791d16
Currently grin is building a backend for matrixservers.net that will collect data and push statistics every hour to our website. Next to that I wanted to push forward that me and several others are looking into building a Code of Conduct that should help every new and existing home server. This code of conduct will be build and curated from scratch in-order to help and support the network or any general project.
Synapse 1.0 is the reference implementation of the Matrix 1.0 spec. The goal of the release overall has been to focus on security and stability, such that we can officially declare Synapse (and Matrix) out of beta and recommended for production use. This means changing the default room protocol version used for new rooms to be v4, which includes the new state resolution algorithm, as well as collision-resistant event IDs, which are now formatted to be URL safe.
Synapse 1.0 also ships with support for the upcoming v5 room protocol (which enforces honouring server key validity periods), but this will not be used as the default for new rooms until a sufficient number of servers support it.
Please note that Synapse 1.0 does not include significant performance work or new features - our focus has been almost exclusively on providing a reference implementation of the Matrix 1.0 protocol. But having cleared our backlog on security/stability issues we will finally be now unblocked to pursue work around reducing RAM footprint, eliminating forward-extremity build up, and shipping new features like Edits, Reactions & E2E cross-signing support.
As part of the security work, Synapse 1.0 contains a breaking change that requires a valid TLS certificate on the federation API endpoint. Servers that do not configure their certificate will no longer be able to federate post 1.0.
It is also worth noting that Synapse 1.0.0 is the last release that will support Python 2.x and Postgres 9.4. For more information see here but the TL;DR is that you should upgrade asap.
This release has been a long time coming. Many thanks indeed to everyone who helped test the release candidates and provided feedback along the way.
Synapse 1.0 is just one component of a larger Matrix 1.0 release, which you can read all about here.
Synapse now more efficiently collates room statistics. (#4338, #5260, #5324)
Add experimental support for relations (aka reactions and edits). (#5220)
Ability to configure default room version. (#5223, #5249)
Allow configuring a range for the account validity startup job. (#5276)
CAS login will now hit the r0 API, not the deprecated v1 one. (#5286)
Validate federation server TLS certificates by default (implements MSC1711). (#5359)
Update /_matrix/client/versions to reference support for r0.5.0. (#5360)
Add a script to generate new signing-key files. (#5361)
Update upgrade and installation guides ahead of 1.0. (#5371)
Replace the perspectives configuration section with trusted_key_servers, and make validating the signatures on responses optional (since TLS will do this job for us). (#5374)
Add ability to perform password reset via email without trusting the identity server. (#5377)
Fixes client-server API not sending "m.heroes" to lazy-load /sync requests when a rooms name or its canonical alias are empty. Thanks to @dnaf for this work! (#5089)
Prevent federation device list updates breaking when processing multiple updates at once. (#5156)
Fix worker registration bug caused by ClientReaderSlavedStore being unable to see get_profileinfo. (#5200)
Fix race when backfilling in rooms with worker mode. (#5221)
0.99.5.2 contains a critical performance fix following a regression that was introduced in 0.99.5. Affected servers will have experienced increased CPU and RAM usage with a knock on effect of generally sluggish performance.
Separately, we are also looking into reports relating to further performance degradations that may have been introduced as part of 0.99.5, though consider the 0.99.5.2 fix to be a significant improvement on previous 0.99.5.x releases.
Okay folks, this is an important one. v0.99.5.1 will be the last release before we ship Synapse v1.0. It is really important that you upgrade to v0.99.5.1 because it implements rooms version 4 - which is the room version that Synapse 1.0 will default to.
This means that Synapse 1.0 servers will create new rooms as version 4 by default and servers that have not upgraded to at least v0.99.5.1 will not be able to join those rooms.
Over the coming days we will announce a release day for Synapse v1.0, the idea is to give admins 2 weeks notice so that anyone yet to configure their federation SSL certificate has time to do so. This is important, failure to configure your certs will mean not being able to federate with v1.0 servers. If you are not sure if you certs are valid, you can test here and read here for more info on what to do.
Aside from room v4, this release also includes the ability to blacklist specific IPs from federating as well as experimental support for edits and reactions. We are not quite ready to mark the feature 'done done', but it is very close. Watch out for news as the feature lands properly.
We're really close to v1.0 now, give us a few more days and we'll announce an official release date.
Add ability to blacklist IP ranges for the federation client. (#5043)
Ratelimiting configuration for clients sending messages and the federation server has been altered to match login ratelimiting. The old configuration names will continue working. Check the sample config for details of the new names. (#5181)
Drop support for the undocumented /_matrix/client/v2_alpha API prefix. (#5190)
Add an option to disable per-room profiles. (#5196)
Stick an expiration date to any registered user missing one at startup if account validity is enabled. (#5204)
Add experimental support for relations (aka reactions and edits). (#5209, #5211, #5203, #5212)
Add a room version 4 which uses a new event ID format, as per MSC2002. (#5210, #5217)
Fix a bug where the register endpoint would fail with M_THREEPID_IN_USE instead of returning an account previously registered in the same session. (#5187)
Prevent registration for user ids that are too long to fit into a state key. Contributed by Reid Anderson. (#5198)
Fix incompatibility between ACME support and Python 3.5.2. (#5218)
Fix error handling for rooms whose versions are unknown. (#5219)
Make /sync attempt to return device updates for both joined and invited users. Note that this doesn't currently work correctly due to other bugs. (#3484)
Update tests to consistently be configured via the same code that is used when loading from configuration files. (#5171, #5185)
Allow client event serialization to be async. (#5183)
Expose DataStore._get_events as get_events_as_list. (#5184)
Make generating SQL bounds for pagination generic. (#5191)
Stop telling people to install the optional dependencies by default. (#5197)
0.99.4 is a maintenance release collecting together all of the bug fixes and performance improvements over the past few weeks, additionally there is further support for the upcoming 1.0 release (more info coming soon). One thing worth calling out is how many community contributions have made their way into 0.99.4, take a look at the change log for details, but many thanks to everyone submitting PRs, keep them coming!
Add systemd-python to the optional dependencies to enable logging to the systemd journal. Install with pip install matrix-synapse[systemd]. (#4339)
Add a default .m.rule.tombstone push rule. (#4867)
Add ability for password provider modules to bind email addresses to users upon registration. (#4947)
Implementation of MSC1711 including config options for requiring valid TLS certificates for federation traffic, the ability to disable TLS validation for specific domains, and the ability to specify your own list of CA certificates. (#4967)
Remove presence list support as per MSC 1819. (#4989)
Reduce CPU usage starting pushers during start up. (#4991)
Configure the example systemd units to have a log identifier of matrix-synapse
instead of the executable name, python.
Contributed by Christoph Müller. (#5023)
Avoid redundant URL encoding of redirect URL for SSO login in the fallback login page. Fixes a regression introduced in #4220. Contributed by Marcel Fabian Krüger ("zaugin"). (#4555)
Fix bug where presence updates were sent to all servers in a room when a new server joined, rather than to just the new server. (#4942, #5103)
Fix sync bug which made accepting invites unreliable in worker-mode synapses. (#4955, #4956)
start.sh: Fix the --no-rate-limit option for messages and make it bypass rate limit on registration and login too. (#4981)
Prevent the ability to kick users from a room they aren't in. (#4999)
Fix issue #4596 so synapse_port_db script works with --curses option on Python 3. Contributed by Anders Jensen-Waud anders@jensenwaud.com. (#5003)
Clients timing out/disappearing while downloading from the media repository will now no longer log a spurious "Producer was not unregistered" message. (#5009)
Fix "cannot import name execute_batch" error with postgres. (#5032)
Run black to clean up formatting on synapse/storage/roommember.py and synapse/storage/events.py. (#4959)
Remove log line for password via the admin API. (#4965)
Fix typo in TLS filenames in docker/README.md. Also add the '-p' commandline option to the 'docker run' example. Contributed by Jurrie Overgoor. (#4968)
New Riot.im application has been delivered to the PlayStore: https://play.google.com/store/apps/details?id=im.vector.app. It replaces the previous app. More details here: https://medium.com/@RiotChat/riot-im-android-security-update-2b3f655ad739
François and Benoit were at AndroidMakers Paris on Tuesday and Wednesday. We’ve seen plenty of interesting conferences and come back with many ideas to improve Riot UX/UI/Implementation/testing/etc.
SAS device verification review is over, will be merged once we have the tagged OLM library.
olm 3.1.0 has been released. This release adds new functions to help with SAS-based key verification (a.k.a. emoji-based verification) and with cross-signing. The Python bindings are also now available on pypi, so you can install it using "pip install python-olm", though you need the olm library and development files installed first.
Spectral's redesign continues, featuring a beautiful responsive UI(not kirigami yet, sorry) and more functionalities. Legacy UIs such as the room detail panel are changed to fit into redesigned UI better. Basic room upgrade support is added, allowing you to switch between the old room and the new room. Room settings and user detail dialogs are added. You can also ignore users in the user detail dialog.
matrix-nsfw has been ported from Golang to Rust. The backend machine learning framework is also switched to Tensorflow, giving a major performance boost. For anyone that doesn't know what matrix-nsfw is, it is a bot-like utility that detects NSFW images in a room. The new repo is at https://gitlab.com/b0/matrix-nsfw-rust
If a room member is not visible on screen, updating their name doesn't require switching to the main UI thread
Apply formatting when viewing the json source of an event
Reuse GUI components to improve performance, update content of views instead of creating new ones
Use a hash set to avoid going through the list of room members in some cases * Move more of local storage into the database: names and avatars of users and rooms, room membership, recently used accounts, etc.
Placeholder avatars are made with GUI components and instead of generated bitmap images
Switch to gradle multi-project build to modularize
After switching from plaintext files to an embedded database, some components are still in the process of being rewritten, coming next week: load messages from server on demand when scrolling, if they are not yet stored in database; add support for invitations;
matrix-appservice-discord work has finally resumed! The PRs for both migrating the room and user store to SQL have been merged, and many awesome new things should follow up soon!
The reactions and edits API is taking shape, we’re making progress on our small homeserver setup, and we’re hunting a new set of device key management bugs that came to light in the absence of matrix.org.
We’ve been a bit disrupted these past few weeks, but work towards Synapse 1.0 continues and we’ll soon be ready to offer a release candidate.
From the dates described below, we will no longer guarantee support for deprecated versions. This means that Synapse may continue to work with these versions but we will not make any attempt to ensure compatibility and will remove old library versions from our CI.
Synapse 1.0 will continue to support both technologies, but subsequent releases may not:-
For Python, we shared that we would discontinue to Python 2.x support from April 1st 2019, so for the first release that follows 1.0 we do not guarantee Python 2.x support.
For Postgres, will give server admins 6 weeks to upgrade to a newer version, and will guarantee support up until 20th May 2019.
We want to make use of new language features not supported in old versions. This will enable us to continue to improve the performance and maintainability of Synapse.
You will be able to upgrade to Synapse 1.0, but will likely experience incompatibilities that prevent you upgrading further. Seriously, you really need to upgrade.
