TLDR: You Don’t Know What You Don’t Know
The bad guys don’t sleep. They wait. In other words, ransomware is a lucrative business, and it’s not going away. With every engagement, cybercriminals learn and adapt to make the next version more insidious, more painful, and more difficult to untangle. They count on their victims being unprepared.
Why does it work so well? Ransomware affects your entire business. The pressure to take decisive action is immense. The chaos of lost data and locked file systems is suffocating. You’ve never experienced this kind of scrutiny before or with such urgency. If you’re not prepared for it, you will be crushed by it.
We know because we’ve seen it firsthand. We’ve helped countless clients recover from it. And, we’ve learned what they wish they knew before it ever happened to them. Here’s the truth about ransomware recovery.
Ransomware is subtle until it isn’t.
You may not realize you’re under attack until you start getting user complaints. The sooner you can determine something is going on, the sooner you can stop it and work on recovery. The whole point of ransomware is to back you into a scared corner where you’re willing to pay the money to get your data back.
The success of ransomware attacks depends on damaging or appearing to damage your data so that you think you have to pay the ransom to recover it safely. While there are dozens of ransomware variations out there in the wild, here are a few truths about all of them:
- Attacks are FAST, encrypting files faster than you can react.
- Attacks are SNEAKY, hiding until they’re well established.
- Attacks are WELL COORDINATED, making recovery as difficult as possible to ensure you have no choice but to pay the ransom.
Regardless of which variant is used against you, there are four common scenarios that can occur singly or in combination. Here’s what you need to watch for:
1) Files Encrypted, Extensions Changed
Users report they can’t open files. You discover some or many extensions have been changed
2) Files Encrypted, File Extensions Unchanged
Users will report they can’t open files, but the extensions haven’t changed.
3) Virtual Machines Encrypted
You’re unable to access your nodes and are locked out of your hypervisor environment. Your file system is completely inaccessible.
4) Cloud Bucket Compromised, Data Deleted
Files will not open. The metadata points to data blocks that should be in the cloud but cannot be read.
Your preparation determines your pain.
While Panzura’s immutable data object storage certainly ensures that ransomware won’t leave you paralyzed with data loss and financial damages, several critical factors depend on your preparation.
The steps you take today directly determine how an attack plays out and whether you’re an easy victim or a hardened target. Here’s what you need to assess immediately:
Secure your login credentials.
Some of this is basic cybersecurity, but it all bears repeating. Make your passwords incredibly strong, and never repeat them. Change them regularly. Never store them on your file system. And limit the users who know your most important admin and cloud bucket passwords to only a handful of people within your entire organization. A least-permissions protocol across your entire active directory will also limit exposure.
Keep systems patched.
Treat security releases as high-priority tasks. Don’t let the bad guys have an easy attack vector that was identified months ago. Pay careful attention to your VMs and hypervisors. Cybercriminals act on opportunity, so again: limit your exposure.
Check your snapshot frequency.
This is the key factor in your tolerance for data loss — don’t cut yourself short by aging out snapshots in the midst of a crisis. Work with Panzura’s support team to ensure you have the optimum mix of high-frequency snapshots during high-use times and longer-term images with the proper aging strategy. This lays the groundwork for easy recovery.
Enable cloud versioning.
Drop what you’re doing and talk to your cloud provider about the best versioning capabilities for your needs. Cloud versioning can be the difference between easy data restoration and a painful recovery process, depending on the attack vector used against you. Many cloud providers will actually refuse to restore files that they possibly could if you don’t have versioning enabled, so don’t shortcut this vital safety net.
Have a plan to act fast.
From the moment of attack discovery, your immediate actions to disable the CIFS license and make your system inaccessible will determine the extent of the recovery process. Be ready!
Panzura has a plan (and a team) for every scenario.
What does ransomware recovery look like with Panzura? Like it never happened to begin with. Like not paying the ransom. And, most importantly, like a team effort.
There are critical steps you must take during and after an attack to make the process as seamless and painless as possible. Here’s what the general flow of events will look like from the moment of attack to the restoration of normalcy:
1. Stop the spread.
Shut down the CIFS license to the affected nodes. This avoids flushing out good data from your cache that hasn’t reached the cloud store yet. If it’s your VM under attack, secure your VM hosts and reinstall the hypervisors. If your cloud bucket is under attack, your cache becomes critical to restoration. Raise a Priority 1 Ticket with us.
2. Stop the attack.
Identify the variant, the file extensions involved, the source of the attack on your network, and the corrupted/compromised user accounts. Depending on the attack vector, additional steps might be necessary before moving to Step 3.
a. Rebuild Your VMs.
If your VMs were the attack vector, you’ll need to redeploy your Panzura nodes before you begin the recovery process. This is where custom encryption certificates are vital, as they are the key to rebuilding your CloudFS.
b. Use cloud versioning to restore data objects.
If your cloud bucket was the attack vector, you can restore data objects from the previous cloud version rather than affected folders since data blocks are never overwritten by CloudFS. If you don’t have versioning turned on, you’ll be in for a much more time-intensive process leveraging your cache to find files that can be restored.
3. Create some breathing room.
Increase your snapshot limits to two months’ worth so that you keep the most granular snapshots preserved while you assess the situation.
4. Find the damage and identify what’s most urgent.
Scan for affected files and when they were affected. With Panzura Data Services, this is quick and painless. Once you have the best snapshot to restore from, rename it to preserve its name.
5. Restore the clean files.
We use a mass file restoration that points the clean metadata to the right files. Prioritize your most active files with a granular approach to get your users working again. Working in a temporary folder while the stress is high ensures you can move the files to their final destination once things are calm and reasonable again.
6. Clean up.
Once the restoration is complete, clean up snapshots from the moment of attack to the moment of restoration, shorten your snapshot schedules back to normal, and re-enable user snapshots. This could take between a few days and a few weeks, depending on the size of your file system, the speed of your VMs, and your cloud bucket provider’s bandwidth.
There’s a solid chance some cybercriminal is out there looking for vulnerabilities in your system as you read this. Don’t treat ransomware like an impossibility, but don’t let dread paralyze you either. Instead, take smart precautions, make critical preparations, and develop a solid plan to act with confidence and authority when the inevitable crisis knocks on your door. With Panzura, you’ve got a partner who’s been there, done that, and can show you what you don’t already know.