March 2nd, 2007

Hardware-based rootkit detection proven unreliable

Posted by Ryan Naraine @ 8:10 am

Categories: Black Hat, Data theft, Exploit code, Hackers, Pen testing, Responsible disclosure, Rootkits, Viruses and Worms, Vulnerability research, Zero-day attacks

Tags: Acquisition, Memory, Hardware, Rootkit, RAM, Ryan Naraine

For years, we've been convinced by companies like Komoku and BBN Technologies that hardware-based RAM acquisition is the most reliable and secure way to sniff out the presence of a sophisticated rootkit on a compromised machine.

Not so fast, says Joanna Rutkowska, a security researcher at COSEINC Malware Labs.

Rutkowska, an elite hacker who specializes in offensive rootkit research, has found several ways to manipulate the results given to hardware-based solutions (PCI cards or FireWire bus).

At this year's Black Hat DC conference, Rutkowska demonstrated three different attacks against AMD64 based systems, showing how the image of volatile memory (RAM) can be made different from the real contents of the physical memory as seen by the CPU.

Rutkowska's research, though purely theoretical, underscores the need for multiple solutions (hardware and software) to work in tandem during forensics. It also highlights just how scary the threat from sophisticated rootkits can be. If, as Rutkowska proved, forensic examiners cannot rely on images collected from RAM, then it's basically game over.

Jamie Butler, a rootkit guru who works with software- and hardware-based anti-rootkit tools, said he was "very impressed" with Rutkowska's presentation. "We already know that software isn't reliable and now we know that you really can't trust the hardware either. You really need to combine both and, even then, you just never know," Butler said.

"I really don't want to meet the attacker who is at that level," he said. "That is scary stuff," Butler said, referring to the techniques used during Rutkowska's presentation.

In three different scenarios, Rutkowska showed how an attacker can crash a machine during memory acquisition. In this case, it would be a denial-of-service against the forensics examiner looking to find traces of malware on a hijacked machine.

She also described a "covering attack" where the malware is programmed to present garbage data

to the hardware trying to read physical memory.

A third scenario is what Rutkowska described as a "full replacing attack" where the malware author not only hides malicious code from the memory acquisition tool but actually provides arbitrary/fake content to the examiner.

The overall problem, Rutkowska explained, is the design of the system that makes it impossible to reliably read memory from computers. "Maybe we should rethink the design of our computer systems so they they are somehow verifiable," she said.

Rutkowska suggests that hardware vendors come up with a special "auditing" interface dedicated only to memory acquisition.

"I'm thinking about motherboard manufacturers adding a special port which would allow for *direct* (this time really "direct") access to RAM and potentially some other critical resources like e.g. CPU system registers and maybe even caches," she said.

Here are the slides from Rutkowska's presentation (PDF).