tl;dr

After some emails with the Matrix security response team, the Matrix Synapse servers have been found affected by a security issue about the lack of a validation system for "Server-to-server" API leading to SSRF and Cache poisoning subsequently marked by the team as “feature” or “intended”.
The scope of this article is to clarify this point; a malicious user, if not specifically denied by configuration files, could effectively load malicious content using what is called the “Server-To-Server” API and, since the caching mechanism allows a lifetime of 24h, it could allow to host arbitrary files for that duration.
After a discussion with the team, as I stated before, they came to conclusion that this is an intended behavior and the best solution is to use an offload antivirus to scan for common malicious patterns on the uploaded files.

In conclusion, I want to thank the security response team for the response and for having this discussion.

Reponsible disclosure timeline

2020-04-29 Vendor is notified
2020-05-31 First contact with the security response team
2020-06-08 Discussion about the issue, marked as "intended"
2020-06-16 Disclosure

 

Introduction

Matrix is an open standard for a decentralized, interoperable and real time communication over IP.
This standard is widely documented and it defines a set of open APIs for communications. Its main point of strength is the ability to have a decentralized, securely encrypted and segregated federation of servers, giving the opportunity to every user that uses this standard to take advantage of the following capabilities:

  • Instant Messaging (IM)
  • Voice over IP (VoIP)
  • Internet of Things (IoT) communication
  • A good amount of integrations (or bridges) with external IM/VoIP protocols like:
    • IRC
    • Skype
    • Telegram
    • WhatsApp
    • E-Mail
    • Hangouts
    • WeChat
    • Twitter
    • Keybase
  • A good and simple set of JSON Rest APIs that allows developers to create their own clients or bots
  • Fully opensource
  • KISS principle
  • End-To-End encryption feature by design

 

The federation concept

The idea behind a federation server is simple and yet effective; an isolated, on-premise Matrix server will be capable to interact with other servers using a common API.
Following a schema taken from the matrix.org website:

 

undefined

Figure 1 The matrix federation architecture

Every matrix client connected to a federated server can, if not specifically denied by configuration, communicate with other people from other servers or channels. Cryptographic’ private keys will not be shared among the federated servers.
The communication between servers is documented in the Federation API (or Server-Server API) docs.

The federation API

Matrix home-servers use the Federation API in order to communicate with each other. Home-servers interact with these APIs in order to push messages to each user, join channels, retrieve history and query the information about users on each server using JSON format.
One part of this is called the “Server discovery” API. And it will be the main subject of this research.


The bug - Server-Side Request Forgery and Cache Posioning

If not specified, matrix home-servers allows to integrate with other federated servers by using an exposed API. The following screenshot is an example of a file download that is located in the matrix.org federated server:

 

undefined


The highlighted part defines which federated server should be used to download the file. By giving a different URL, the following response is received:

 

undefined

Figure 2 Original request

 

undefined

Figure 3 Callback

 

So, the federated server is looking for a “server” file inside the “.well-known/matrix” path. As specified in the documentation, the server is expecting a JSON formatted file that specifies the information about the federated server like the following:

 

undefined

Figure 4 Expected configuration file


With this in mind, some aspects should be considered:

  • Loopback or localhost address, even specified as numeric IPs (like 127.0.0.1) are blacklisted by default (the “federation_ip_range_blacklist” parameter on “homeserver.yaml” file)
  • Using protocols different from HTTP or HTTPS will result in a timeout
  • 30x redirects are accepted
  • The cache lifetime is statically set to 86400 seconds (1 day). When a subsequent request to the same host is made, the result will be the same as the last one (this will open a Cache Poisoning scenario described later).
  • If not specified, the 8448 port will be used
  • The TLS certificate must be valid, so no unencrypted protocols can be used (if not allowed from configuration file)
     

 

The next step is to create a configuration file that points to the attacker’s-controlled endpoint:

undefined

Figure 6 Configuration file

 

And, after issuing another request, a callback is received:

 

undefined

 

undefined

Figure 8 Callback

 

In order to obtain a full exploitation scenario, a “malicious file” is created.

undefined

Figure 9 Creating a "malicious" file

 

And a new host is configured on .well-known/matrix/server file:

undefined

Figure 10 New matrix server configuration file

 

Then, a new request for the “malicious_file” is made:

 

undefined

Figure 11 The malicious file is cached for 24h

 

So, from now on, this file will be available in matrix.org servers for 24 hours; even if the external federated server is not available.

 

undefined

Figure 12 File download

 

Impact of this vulnerability

An attacker could exploit these two vulnerabilities in order to:

  • Host malwares or, generally speaking, infected files
  • Use it as a C2 server in order to broadcast files or messages
  • Use it to launch denial of service attacks
  • Hide the attacker IP while sending HTTP requests (using the server as a reverse proxy).
  • If an XSS vulnerable application is hosted on the same host, this could bypass the Same-Origin Policy and allows an attacker to embed
  • JavaScript files directly into the application, as long as share application cookies.

Affected versions

At the time of writing, the affected version is “1.12.3” as reported from the matrix.org API:

Remediation

The remediation is simple and it’s just a matter of specifying a list of whitelisted federated servers on the “/etc/matrix-synapse/homeserver.yaml” file. Unfortunately, this parameter is not set by default nor mandatory while installing the matrix server and there are some considerations to take in mind while applying this remediation (take a look at “Final considerations and thoughts” chapter).

undefined

Figure 13 Effective remediation

Final considerations and thoughts

There are some aspects to take in consideration while analyzing this vulnerability. This flaw is basically leveraging on a feature and, in some cases, a remediation like the one proposed in the “Remediation” chapter isn’t feasible.
A server like Matrix.org that SHOULD allow every federated server to connect and interact; the whitelist scenario may not be applicable.

DevOps should be aware that keeping the default settings and letting every federated server to inter-communicate with each other is a serious security risk. This, at the same time, would effectively help the “Federated network” concept to grow.

Fixing this vulnerability could be a serious hit on the matrix-synapse federated network’s idea; let’s imagine the case. The server blocks the ability to download files or perform requests to external locations, so files are not shared among others federated servers. The result is that User A on server 1 cannot share files with User B on server 2, and vice-versa.

Lastly, while the note on the configuration file “/etc/matrix-synapse/homeserver.yaml” states:

# N.B. we recommend also firewalling your federation listener to limit
# inbound federation traffic as early as possible, rather than relying
# purely on this application-layer restriction. If not specified, the
# default is to whitelist everything.

This clearly isn’t a good solution if the server needs to be inter-connected with the rest of the world.

(Cristian 'void' Giustini)