Earlier this year, NETSCOUT announced that it had identified a new attack vector that was being leveraged to launch DDoS attacks across the public Internet. The vector was a protocol incorporated into the Plex Media Server (Plex) application that – when the server was incorrectly left open to the public Internet – could be abused by attackers to launch reflection/amplification DDoS attacks.
Once Plex became aware of the issue, they reacted quickly to update their software so that it couldn’t be abused in this way, and encouraged all their users to upgrade. But attackers had been actively abusing this reflection/amplification vector for a considerable time prior to the release of the fixed version, and NETSCOUT had been working to assist network operators to mitigate these attacks.
To learn more about the Plex Media Server, how it was leveraged in DDoS attacks, and the impact of these attacks, we recently sat down with Roland Dobbins, a Principal Engineer on NETSCOUT’s ASERT Team. Roland is NETSCOUT’s foremost DDoS subject-matter expert and is one of the top DDoS mitigation specialists in the world, having spent more than 30 years securing, maintaining, and defending many of the highest-visibility networks.
In the first part of our two-part Q&A interview, we asked Roland about Plex in particular, and how attackers were able to abuse Plex Media Server instances to launch DDoS attacks. We also asked about how reflection/amplification attacks function to bring down networks, applications, and mission-critical services.
Here is what he had to say:
GovCyberHub (GCH): What is Plex? Can you tell our readers that may not be familiar with the application a little bit about it?
Roland Dobbins: Plex makes a media server application that has been around since approximately 2007 – it’s not a new service. The Plex application allows users to take their own physical media – if they own DVDs, or Blu-Ray discs, or another type of physical media – digitize it and use the Plex application to curate their now electronic media and stream it back to themselves.
Using Plex, a user could create a centralized mp3 song collection, or a centralized collection of mP4 movies, store it on a device that’s running the Plex server, and then play it back on another device on the network. So, a song or movie or TV show that resides on a desktop computer or laptop could be played back on a phone, or tablet, or smart TV. And they’ve also recently gotten into the online commercial streaming service business, as well.
So, as you can imagine, Plex has become quite popular, especially among folks with large physical media collects who are also somewhat technically-inclined. That being said, Plex isn’t difficult to use, but you do need to have a certain amount of understanding of networking, knowledge of electronic media formats, and how to establish your own media server.
GCH: What did NETSCOUT learn about how attackers were using the Plex application? How were they using it?
Roland Dobbins: The DDoS attack vector we discovered is an example of a category known as reflection/amplification attacks. These are among the most common types of DDoS attacks because they enable attackers to generate a relatively small amount of traffic, but then get a significant boost in attack output that turns that small amount of traffic into a lot of attack traffic.
More specifically, attackers can send out small packets that are reflected and amplified into enough traffic so that it pummels the actual intended target of the attack and causes an outage.
Reflection/amplification attacks leverage abusable applications and services to amplify that traffic and execute the attack. By abusing these applications, attackers also preserve the anonymity of their attack initiation infrastructure.
“Malicious actors are constantly scanning the Internet and looking for new, abusable services that they can leverage….[they] begin to talk shop in the digital underground, and, eventually, a critical mass of people in the digital underground understand the new attack methodology.” – Roland Dobbins
Reflection/amplification attacks begin with the attacker spoofing the target’s IP addresses. The attacker is most likely utilizing an attack infrastructure that’s sitting on a network that does not enforce source address validation (SAV), also known as anti-spoofing. The attacker uses the spoofed IP addresses of the target’s network, machine, server – whatever it is they’re attacking – to send out a request to a bunch of abusable reflectors/amplifiers, which are typically servers or network infrastructure devices running services which as misconfigured in such a way that they can be abused to launch DDoS attacks of this nature.
The attackers generate relatively small request packets, spoofing the IP addresses of the target, and send them to the abusable servers. When the attack infrastructure sends enough requests to enough abusable servers around the Internet they subsequently pummel the target of the attack with enough unsolicited “answers” that they take down the server, fill up Internet peering and core links, etc.
There are many different services that can be abused in this manner. What we discovered was that Plex incorporated one of those services that that could potentially be abused by bad guys – where they could send a relatively small request and get a relatively large amplification factor. In this case, the service that resided within the Plex application could amplify a request by approximately 6:1.
GCH: What led to the application being abusable in this way? And how do you think these bad guys found this service?
Roland Dobbins: Malicious actors are constantly scanning the Internet and looking for new, abusable services that they can leverage. Some of the more advanced attackers will find an abusable service and begin launching reflection/amplification attacks leveraging it. Then, these more advanced attackers begin to talk shop in the digital underground, and, eventually, a critical mass of people in the digital underground understand the new attack methodology.
Eventually, we see it getting added to DDOS-for-hire services; this ‘weaponization’ of new DDoS attack vectors means that anyone who can click a mouse or type an IP address or DNS name can launch sophisticated DDoS attacks, including new attack vectors.
In the case of the Plex application, these advanced attackers found that a population of Plex instances had been incorrectly exposed it to the public Internet.
The legitimate users of these media servers may have wanted to be able to stream media from their home media server to a location off of their home network – at their work, or in their car, or to play some music at a party at their friend’s house. To make their media collections available and accessible, they had actually configured the firewall rules in their home networks to allow these devices to be remotely accessible.
Unfortunately, there’s a variation of a protocol called SSDP that’s used by Plex as part of the media server discovery functionality of the application. And when it’s exposed to the Internet, the bad guys could send a request to all these Plex servers from a spoofed IP address and, in turn, get an answer that would then pummel their target.
“The Plex users were all innocent people that were doing nothing wrong. The Plex team didn’t do anything wrong. But people were exposing their media servers to the Internet because they didn’t understand the risk, and then malicious actors began abusing the bandwidth and applications of those Plex users to execute DDoS attacks.” – Roland Dobbins
This was actively taking place on the public Internet and we were actually alerted to it by one of our customers who were able to leverage their NETSCOUT DDoS mitigation solution to successfully mitigate the attack.
We then took a look at our data – and we have by far the broadest attack sensorium on the Internet – and we were able to see that this particular DDoS vector was being actively used by attackers, and had been for quite some time. At some point in time, after it was originally discovered and used by the more advanced attackers, it had been weaponized and added to these DDoS-for-hire services.
So now anyone who can basically click a mouse or enter an IP address can launch a Plex media server reflection/amplification DDoS attack.
GCH: Were the attacks leveraging Plex targeted against the application users, or other, third parties? Was there any negative impact to the user at all?
Roland Dobbins: The Plex users were all innocent people that were doing nothing wrong. The Plex team didn’t do anything wrong. But people were exposing their media servers to the Internet because they didn’t understand the risk, and then malicious actors began abusing the bandwidth and applications of those Plex users to execute DDoS attacks.
The main impact that this abuse could have on Plex server users themselves would be to consume enough bandwidth and resources that their Internet access would be negatively affected. This can cause disruptions to the users’ ability to work and learn remotely, and enough aggregate impact can cause more widespread disruptions for broadband access users of ISPs with a significant proportion of abusable reflectors/amplifiers present on their networks.
The people who paid the price for this vulnerability – and user error – were the companies, individuals, and organizations that were ultimately targeted by DDoS attacks that leveraging abusable Plex servers.
In part two of our two-part Q&A interview with Roland, we talk about the larger trends of abusing connected devices and applications for DDoS attacks, and how application developers and device manufacturers can keep their solutions from being leveraged in massive DDoS attacks.