When criminals get into your network, especially with ransomware, they push you to move quickly. It’s not because they want money fast. Instead, it’s because they are using psychological warfare tactics to make their victims more compliant. In the latest round of ransomware attacks, cyber gangs are going beyond the threat of stealing data; if victims don’t pay, they threaten to auction the stolen data on the Dark Web to the highest bidder, increasing anxiety for the company and its potentially breached victims.
That ticking clock is an effective tool to trick the victims into making bad decisions under stress while trying to investigate and remediate, or simply knuckle under and pay the ransom. But organizations that are ready for a breach can work fast, if they are prepared with remote Digital Forensics & Incident Response (DFIR) tools and have ensured that full endpoint monitoring has been properly deployed in their environment.
The sheer logistics involved in a DFIR investigation can be staggering: how can you pull hundreds of hard drives across multiple widely distributed physical sites and get them all to a forensic lab for analysis within a 48-hour window? And if you need to bring the experts physically to the machines they need to inspect, you have delays increased by things like air traffic, weather, and social distancing.
My colleague John-Luke Peck, featured in the video linked above, presented on this topic in his presentation Autopsies of Recent DFIR Investigations at BlueHat 2019. He predicted at 20:20 that the solution to these problems was to move to remote DFIR and do away with on-site forensics entirely, replacing them with cloud services that store all event data and are accessed via web browser. That requires endpoint monitoring clients must be installed on every single endpoint on the network. This allows administrators and security personnel to use centralized management consoles hosted either on-prem or (ideally) in the cloud to monitor and manage the network. With those EDR solutions in place, information about the current state of both the network and all the endpoints it contains is constantly being moved to a safe offsite repository that is safe and easy for authorized personnel to access remotely.
I recently had the opportunity to perform a forensics investigation using one of these new remote DFIR tools for the first time, specifically Microsoft Defender ATP’s Advanced Hunting console. My experience was like stepping into the cyberpunk world of the future. Instead of waiting days for hard drives to arrive, or shipping myself across the country, all the customer had to do was create a read-only administrator account for me on their Defender ATP instance. Upon logging in I had instant access to every network, user, and file event on every single host that had a Defender ATP agent installed.
I could write sophisticated queries on any Windows event I could think of and trace patterns of activity from one machine to the next. I had the ability to inspect, quarantine and download files for offline investigation on demand. With the customer’s permission, I could also instantly log into any machine of interest via RDP. In short, I soon had both the 30,000-foot view of the state of the network and the attacker’s path through it and could quickly pounce on a detail I wanted to investigate deeply to reconstruct the timeline of events before and after it.
It was trivially easy to search for files that shared the same hash as a malicious payload; I didn’t even need to write a query, just paste the hash into a search box and get a handy list of where and when it was last seen, and what user ran the command. I could see, search, and run queries on every PowerShell command run on every machine on the network and easily tie those events back to hostile logins.
The sense of freedom and the speed with which I could develop and confirm or refute a hypothesis were pure investigatory catnip. All I had to do was inhale, and I was rewarded with the rich, sweet scent of actionable information. Here’s the coolest part: within minutes of using the Defender ATP console for the first time in a ‘live’ situation, I was already able to spot activity the client hadn’t noticed and expanded the scope of our investigation to eliminate risks that had not previously been considered.
After that initial success, I pursued the investigation with an obsessive determination because I was making such constant progress. The role of exhaustion in impairing investigations should not be understated. I found that with these new remote DFIR tools, my stamina and creativity as an investigator both went up sharply because my frustration level was kept low.
More traditional forms of DFIR investigation are no longer sufficiently timely. When it was possible to board an airplane with little or no preparation, it was possible to get people on-site to respond to problems in a reasonable amount of time. Pulling and shipping hard drives also takes too long, as investigators must often coordinate with slow, unreliable and/or inexperienced MSPs to get vital information.
In a recent DFIR investigation, we found that the ransomware thieves were already releasing information well ahead of their threatened deadline. Client data was already potentially at risk while we were still waiting for them to compile and send the data required for us to begin work. That meant their own customers were now being contacted by law enforcement and were starting to demand answers as to how they had been breached well before there was an answer to give.
Had my team been able to jump right in and get to work on a fully-configured remote threat-hunting system, we would have been able to provide that customer with a much better sense of the root cause, the extent of the breach, and whether the remediation steps that had been taken were effective. That, in turn, would have allowed them to proactively contact customers and inform them of the breach rather than hearing it from the FBI, causing substantial reputation problems.
Since John-Luke gave that talk at BlueHat six months and what feels like an eternity ago, a global pandemic has made remote work mandatory for most people in the software industry. There is a dreadful irony in attempting to eradicate software viruses by traveling onsite to perform investigation and remediation only to spread deadly biological viruses in the process. Security practice is by nature holistic: we must consider all risks and every threat, including biological ones, or we are failing in the task of making our customers more secure. As a result, the case for remote DFIR has only grown. Your IT staff’s best PPE (personal protective equipment) is remaining in their own homes – and the management capabilities that remote DFIR tools offer is the only way to offer that.
Remote DFIR is not without flaws and risks, but they’re minor compared to the benefits. The endpoint agent must be correctly configured and running. Intruders can spoof and/or disable it, preventing it from sending and receiving updates. If that happens, the data is lost, and more hands-on investigation is still required. These risks are easy to mitigate, particularly with the assistance of consultants from Critical Insight who can walk you through the best way to deploy endpoint monitoring and protection services.
With ransomware incidents escalating daily in response to the pandemic, the risks of not implementing remote DFIR capabilities have become too high to accept. This is a powerful methodology, and its tools are now mature enough to do the work. Remote DFIR is no longer the way of the future. This is, as a famous fictional threat-hunter often notes, the way.