Subscribe

Summary

Customer success engineer Dan Kelly explains how Race Communications used Kentik’s powerful network analytics to identify malicious traffic associated with the Mirai botnet, determine which of Race Communications’ customer IP addresses were being exploited, save its online IP reputation and fend off other types of DDoS and botnet attacks.

The Mirai botnet was first discovered back in 2016, but has continued to persist and abuse common vulnerabilities and exposures (CVEs) on IoT devices, including home routers and many other network-connected devices. In short, the Mirai network of bots was built by malicious actors who exploited remote access and control protocol ports over many different device types, producing damaging traffic levels and creating an advanced, powerful tool that can be used for large-scale DDoS attacks and many other nefarious purposes.

When Kentik customer Race Communications—a provider of reliable, high-speed internet and advanced communications to communities throughout California—learned of Mirai’s potential risk to its customers, the team knew it needed to act fast.

In this blog post, we outline how Race Communications was able to leverage Kentik’s powerful network analytics to identify malicious traffic associated with Mirai, determine which of Race Communications’ customer IP addresses were being exploited by the botnet, and ultimately, save its online IP reputation.

The Race to Stop Mirai

Race Communications was alerted to the potential Mirai risk when the team received a letter from another network online noting that IP address(es) owned by Race Communications acting maliciously over the internet. These addresses were apparently port scanning IP addresses that belonged to the company sending the letter. The letter took on the form of a formal complaint and asked that Race Communications cease this activity.

While the port scans were only coming from a few hosts, Race Communications noticed that the complaint was against an entire /24 IP block. This had the potential to lead to the entire /24 block getting blacklisted for malicious activity. This type of blacklisting could potentially cause other Race Communications customers to experience a loss of connectivity to services, due to poor IP reputation.

Race Communications knew that by utilizing network forensics capabilities from Kentik, the team would be able to quickly drill down into the incident and determine the root cause behind the formal complaint.

Network Forensics & Visibility from Kentik

The Race Communications network team turned to Kentik’s “Unique Destination Port” metric and was quickly able to see how many ports the address listed in the formal complaint might be hitting, and why it would be considered a scan. When this number revealed only 40 destination ports on average, the team again turned to the insights available with the Kentik platform.

Checking the pattern of all ports that were being utilized, Race Communications discovered that the vast majority of all destination ports were either port 23 (Telnet), or port 37215 (Huawei Remote Procedure Port). The Huawei port was of immediate interest to the network team as this port is part of a long-lived exploit cataloged as CVE-2017-17215. The exploit requires only a single authentication to work, and once it had been exploited, remote code execution is possible on the associated device. (Vuldb has additional information on the life of this exploit.)

The final port that Race Communications observed was port 2323, which was a tiny fraction of the total traffic, and always sourced from port 23 on protocol TCP. This is a potential sign of a Mirai variant botnet C&C traffic, as described in this article, and is additionally associated with CVE-2016-10401, an exploit associated with ZyXEL network devices and escalation of permissions. The method of exploiting both the Huawei CVE and the ZyXEL are very similar, requiring one authentication first.

In just a three-hour period, Race Communications was able to see a very large Unique Destination IP count-per-port. As shown in the image below, this rate was highly consistent:

Race Communications was also able to leverage the Destination Port information from Kentik to pinpoint backtrace several additional Source IPs on its customers’ networks that were participating in similar traffic patterns.

During this portion of the investigation, while working through the traffic, Kentik and the Race Communications network team found additional interesting and actionable information within the Kentik platform:

• Most destination 37215 ports were hitting IP addresses in the Asian market regions, such as China, Japan, and South Korea. To the team, this was a potential indicator that the attackers were targeting addresses within a region more likely to contain a Huawei product.
• Destination port 23 traffic was hitting an evenly distributed traffic pattern between the Asian markets and the United States. To the Race Communications team, this had the potential to imply that the attacker(s) were searching for new devices using Telnet, aiming to find vulnerable Huawei devices in the process, and scripting to additionally hit these devices with a vendor-specific attack.
• Each Source IP that was on its customers’ networks was reaching an hourly average of 3,500 Unique Destination IPs consistently, for weeks. Each Unique Destination is another potential report, and another hit against this Race Communications’s customer IP reputation.

An Interesting Forensic Finding

Perhaps the most interesting finding for Race Communications was that Kentik’s Spamhaus Botnet and Threat-List data feeds tagged all of the offending IPs that Race Communications had identified manually. Additionally, Kentik’s platform highlighted other suspicious activity among Race Communications’ customer subscriber IPs. This meant that there had been significantly more complaints against its customer IP blocks than Race Communications was initially aware of from the single, formal complaint the team received.

The Race Communications team knew it was not sufficient to wait for another entity online to formally complain. With insights from Kentik, the team could act proactively in order to detect malicious subscriber traffic heading to the larger internet.

Automated DDoS/Botnet Detection

Kentik was able to offer Race Communications an automated way to detect botnet activity, by creating Kentik DDoS-type policies with modifications. Kentik now alerts the provider whenever a subscriber IP communicating outbound via ports 23 and 32715 reaches more than 500 unique destinations.

These insights give Race Communications the option to proactive notify internal operational groups of events that could impact IP reputation in the long term. Additionally, Race Communications can now contact their customers to alert them of a possible infection on one of their Internet-connected devices.

Additional DDoS Detection Features

In addition to the capabilities offered by Kentik’s platform scrubbing partners Radware and A10 Networks, and our BGP-based Remote Trigger Black Hole (RTBH), Kentik now offers BGP FlowSpec mitigations. FlowSpec can be utilized to mitigate traffic with low-to-no collateral effects versus RTBH. It excels in situations where non-volumetric attacks like the ones associated with the Mirai botnet are compromising commercial security and reputation.

Summary

Using Kentik, Race Communications can protect its business and its online IP reputation by unlocking:

• Information needed to remediate not just the reported problem, but the entire detectable problem
• A path by which to contact subscribers and explain specifically why they have been the subject of a complaint (e.g., if they have infected machines on their network, response and action can quickly take place)
• An automated way of being informed of new IP addresses that form the Mirai pattern or other botnet traffic patterns
• Faster mean time to diagnose (MTTD)
• Faster mean time to repair (MTTR)

Kentik is able to offer insights that can help identify almost any malicious or unwanted traffic on any network and provides automatic notification and mitigation capabilities. For more information on network visibility from Kentik, we encourage you to request a demo or start a free trial today.

Related reading:

• Kentipedia Entry: DDoS Detection
• Kentik Solution Brief: Kentik DDoS Detection and Defense