It started out as a normal Thursday for Tony Mendoza, senior IT director at Spectra Logic, a data storage company based in Boulder, Colorado. And then the ransomware attack began.
“We got some notifications of some system failings and it quickly turned into a lot of unrelated systems failing, which is really abnormal,” says Mendoza. He realised that the company was under attack – and that its files were being encrypted.
“When it hit, we ran to our server room and data centre and started pulling plugs out so it couldn’t propagate itself – which brought our entire infrastructure down,” he says.
In total, three-quarters of the production environment was compromised with ransomware. The hackers left a ransom note demanding a payment of $3.6 million in bitcoin in exchange for the decryption key.
“Figuring out what it was was fairly simple, because they tell you who they are, and they tell you where to send the money. It was NetWalker because it said so in the ransomware letter,” explains Mendoza.
Another problem: the attack came in May 2020, when many employees had just started to work remotely because of the COVID-19 outbreak, so there was no way of easily communicating what was going on outside the building.
Despite that, the IT team had to assess the damage that had been done and what the options were for getting data back – if it was going to be possible at all. There was some hope – the company had backups, which were separate from the rest of the network and safe from the incident.
“We’re still under attack, we’re still trying to stop the bleeding, we still don’t know what the extent of the damage was – but we knew we had data to work with,” says Mendoza.
Every organisation that falls victim to a ransomware attack ultimately has to face one major question – do they they give in to the ransom demand in order to retrieve their data?
- ‘Like playing whack-a-mole’: Do cyber-crime crackdowns have any real impact?
- Ransomware attack halts production at IoT maker Sierra Wireless
- Ransomware gangs have found another set of new targets: Schools and universities
Cybersecurity companies and law enforcement agencies around the world argue against giving into extortion surrounding ransomware attacks, because not only does it hand over hundreds of thousands or even millions of dollars in bitcoin to criminals, it proves that the attacks work, which encourages ransomware attackers to continue with campaigns.
However, some victims feel as if they’ve got no choice and they’ll pay the ransom, perceiving it to be the quickest and easiest way to get their data returned and the network back up and running – although that isn’t without issues. There are instances where attackers have either taken the money and ran, or taken the ransom then just returned with a second attack.
Spectra Logic had cyber insurance, which could potentially have covered the cost of paying the ransom. That might have been the simpler short-term decision for restoring the network, but it was quickly decided that with the backups still available, Spectra Logic wouldn’t give in to the ransom demand.
So instead of communicating with the cyber criminals at all, Mendoza contacted the FBI.
“I went from being in a panic to being reassured by them that they’d seen it before, we’re not alone in this and they’re going to put tools in place to start protecting us. That was the biggest thing, getting protected,” he explained.
The FBI also assigned a specialist team to help Spectra Logic deal with the immediate fallout from the attack over the course of the days that followed.
- Three billion phishing emails are sent every day. But one change could make life much harder for scammers
- FBI: Phishing emails are spreading this sophisticated malware
- Largest ransomware demand now stands at $30 million as crooks get bolder
Attempting to restore the network turned out to be a 24/7 job for the small team over the course of the following week. For much of that time, people were sleeping at the office in order to have the most time possible to focus on restoring the network.
“From the Thursday morning, we spent 24 hours everyday for the next five days working on this – we slept in shifts. Three of us would work through the night while two people slept for a few hours,” said Mendoza.
“There was no leaving and coming back, it was go sleep on the couch in case we need you. It was five days of all hands on deck.”
As well as this, he was having to provide the board with updates on the ongoing situation. They wanted answers about when the network was going to be restored and when business was going to be back to normal.
“I’m dealing with leadership in the company and I don’t want to lie to them and say I know when it’ll be up – I had to tell them I don’t know what’s going on or when systems will be up,” he says.
- Microsoft Exchange Server attacks: ‘They’re being hacked faster than we can count’, says security company
- Cyber security 101: Protect your privacy from hackers, spies, and the government
- The best antivirus software and apps
- The best VPNs for business and home use
- The best security keys for two-factor authentication
- Why some governments are getting cyber crime gangs to do their hacking for them (ZDNet YouTube)
It took days of working around the clock but eventually the IT department, with the aid of cybersecurity specialists, was able to restore some functionality to the network a week after the ransomware attack, without paying out to the attackers.
“Our cybersecurity team provided us with the expertise and tools, monitoring and logging to get the threat out of our system. Monday morning they give us a green light; it’s done, they’ve stopped it and removed it,” Mendoza remembers.
“The FBI told us we’re going the hard way, but the right way – and it ended up being the easy way when we came back and said we were back up eight days later; it was shocking for them,” he added.
But it didn’t mean everything was immediately back to normal – it took weeks more to bring back systems that weren’t critical to the business, and during that whole time careful attention was required just to make sure the attackers hadn’t somehow managed to spread the ransomware again, which meant constantly monitoring all activity on the network for another month.
A lot of ransomware attacks never become public knowledge, and examples of companies that go into detail about what happened are still few and far between.
But Mendoza says it’s important to be transparent about dealing with a ransomware attack, because it’s important to show that it is possible to recover from an attack without lining the pockets of cyber criminals.
“What we realised was we protected our data and there’s a way to thwart ransomware. We couldn’t find public information when we were looking for it, so we wanted to make it a common thing, that it’s okay to talk about being impacted by ransomware,” he said.
- Microsoft Exchange Server zero-day attacks: Malicious software found on 2,300 machines in the UK
- This malware was written in an unusual programming language to stop it from being detected
- This trojan malware is now your biggest security headache
So what is the key lesson Mendoza would say that other organisations need to take away from Spectra Logic’s experience? It’s backup your systems – and do so offline – so, if the worst happens and the organisation falls, you still have backups offline.
“You’ve got to limit your attack blast radius. Backup your data in multiple locations on multiple mediums and the key is to air-gap it. Whether it’s physical air-gap or virtual air-gap, you’ve got to put a wall between an attack and your data,” he said.
And how did the company end up falling victim to a ransomware attack in the first place? Analysis of the incident revealed a phishing email sent to an employee working from home was how hackers gained their initial access to the network.
In the aftermath of the ransomware attack, Spectra Logic has worked to improve its cybersecurity culture, both on-site and for remote workers in an effort to learn from the incident. The company is now actively looking for potential cybersecurity threats that might have been missed before.
“Initially after the attack, when the wounds were fresh, we talked about security. Six months later, we’re still concerned about security and we’re more aware of phishing attacks. We were kind of complacent before,” he says: now staff will notify him if a phishing email isn’t picked up by the malware system. “There’s more awareness now.”
MORE ON CYBERSECURITY
- Ransomware is now the biggest cybersecurity concern for CISOs
- How to protect your organization’s remote endpoints against ransomware TechRepublic
- Ransomware vs WFH: How remote working is making cyberattacks easier to pull off
- How to avoid a spear-phishing attack. 4 tips to keep you safe from timeless scams CNET
- Ransomware: How clicking on one email left a whole business in big trouble