Infosec on the Threadripper NVMe Drivers, a Level1 Diagnostic

Level1 Diagnostic: What would you expect to get when installing NVMe RAID drivers?

AMD’s new CPUs are formidable. Threadripper for high-end desktop use cases is one of the most exciting platforms I’ve seen in years. Even though most would describe AMD as a global multinational, there are many aspects of the company itself that remind me of an energetic and scrappy startup.

A few months ago, right after the Ryzen launch, I was on the AMD campus for a few days and we were able to take lunch in The Summit cafeteria, where most of the employees eat. Normally I’m not an outgoing person, but in one case where I overheard a conversation about git and source code, I had to introduce myself and ask a few questions.

I wasn’t after any privileged info; I was curious about the working environment, the culture and “the vibe” as it were. One of the bits of information I learned on my trip was that AMD was retooling its participation in eSports and how they approach eSports sponsorships, and that the "grown up nerds" had a lot of say so in the fire & passion of the company. It felt a bit like it feels when I'm consulting with startup companies on this and that. 

AMD is a global multinational company, but a remarkable job has been done to preserve that startup culture. AMD offering, for free, NVMe raid functionality is just the cool thing to do. It is the startup thing to do as an enterprise feature makes its way to our desktops (and for a fraction of the price). Unfortunately, it seems apparent that not a lot of engineering time could be put into repackaging those enterprise-ish drivers for the high-end desktop platform, or auditing what/how exactly the drivers and accompanying software programs are setup once installed.

Please don’t fault AMD too hard for this – the problems are easily fixed by simply hiring more smart people to spread the load a bit.

First, the drivers themselves seem pretty solid. It is perhaps a bit disconcerting that there is no upgrade path for anyone using the RaidXpert package with a SATA-drive-based array (you will have to tear down and recopy your array contents before upgrading). There are some platform teeting issues (and some performance regressions in some scenarios) but the board vendors and AMD are working closely together to line that out. I would guess that the NVMe raid drivers have come to the HEDT from the enterprise; that certainly explains the licensing functionality.

Along with the drivers themselves, a full-fledge installation of the Apache webserver is created, along with with a… developer installation of Xampp?

XAMPP is a completely free, easy to install Apache distribution containing MariaDB, PHP, and Perl. The XAMPP open source package has been set up to be incredibly easy to install and to use.

https://www.apachefriends.org/index.html

After some digging, I have determined that the AMD Raid driver implements the array management GUI with Apache and PHP. The Apache web server is running all the time on a target machine, offering up the password protected RaidXpert2 raid management GUI.

In addition to that, https://localhost will offer up the Xampp installation. From here we were able to determine many variables about the host, including pathing, PHP version, etc. This is bad from an information security perspective. 

Networking Access

Both the XAMPP process and the RaidXPert2 GUI do not restrict access from the LAN. That means that anyone on your local area network can access both Apache sites. While the RaidXpert2 has a password, the Xampp install does not. It provides some nifty developer tools, and phpinfo() reveals a lot about the system configuration.

Apache? PHP?

Given this is accessible to anyone on the network in a default install, how hardened is this installation? Not at all. From the phpinfo() directive, we can see the PHP version installed is out of date, and has a number of published CVEs. Fortunately, none of the CVEs are particularly severe.

Next, I checked how the Apache web server is configured. I discovered that the Apache process is running as the SYSTEM user; this is a privilege level on Windows systems above the Administrator user. This is unusual. Even so, it is possible configure both Apache and the PHP extension to prevent it from opening or manipulating directories outside the web root. However, it had not been configured this way.

We had no trouble crafting a PHP file to open c:\windows\system32\notepad.exe and to deposit .exe files at various locations around the system. A restrictive <Directory /> directive in the apache configuration and a PHP openbasedir restriction would have prevented this. As it stands, if someone were to find a flaw in the RaidXpert code, or even a problem with Xampp on port 443, it would be possible to execute arbitrary code as SYSTEM on the target computer.

Apache needs to be system because the PHP code provided as part of RaidXpert is executing the CLI raid management utilities with administrative/system level privileges.

Alternatives

There are alternative ways to accomplish this functionality. For one, it is trivial to write this type of service wrapper in modern languages like Go, from Google. It would be possible for AMD engineers to provide the very same (potentially cross-platform) PHP web gui, but instead of directly calling the shell or windows scripting host as we see here, it could call a service API. The service API interface would be a platform specific binary, sure, but Go has a lot of great support on both Linux and Windows so it can still be cross-platform. With that change, the Apache process can run as an unprivileged user without any issues.

For the immediate workaround, it is perfectly fine (and advised) to simply stop the Apache process when you don’t need to manage your raid array. In services, you can simply mark it as manual startup and that closes the hole.

Another recommendation from everyone at Level1 would be to update the apache configuration and change the Listen line in Apache so that it only listens on 127.0.0.1 – that would remove the Xampp and RaidXPert2 accessibility from your local LAN. Requests to https://localhost would still work just fine, however.

And, of course, you can elect to operate from the CLI only for managing your RAID array and incur no overhead penalty or background processes.

We did not perform any type of analysis to see if we could take over the windows scripting host by, for example, submitting raid configuration forms with escape sequences that might let us run arbitrary code that way.

Security is not easy. No doubt the engineering team that put this together was overworked and understaffed. Still, this is something we will keep an eye on because we hope it will be improved in future versions.