Lessons from the SolarWinds Hack

p>It seems as if this was a lifetime ago, but it's only been about a month since the massive SolarWinds hack, in which foreign intelligence service actors made it into all sorts of sensitive government systems. SolarWinds is a prominent and reputable (at least until now) server monitoring and security firm that services infrastructure belonging to institutions of all sizes, including government and Fortune 500 companies.

From SolarWinds' perspective, some serious mistakes have been revealed. Among them, it appears that the attackers may have found a password shared in a public repository. It gets worse; the password in question was "solarwinds123". Furthermore, SolarWinds advised users of the Orion product (which was the one hacked) to turn off some antivirus scanning for optimal functionality.

If you had asked me a month ago, I would have said that contracting with SolarWinds would be a decisive step towards increased security. Instead, it appears that this sophisticated supply chain attack used SolarWinds to distribute malware onto the servers it was monitoring; these companies downloaded malware from their security provider.

As a site owner, that is a terrifying thought, that the very company you hired to protect you in fact led to an eventual compromise because they were so sloppy.

What are Supply Chain Attacks?

Take nearly any physical object like the computer or phone that you are using to read this post. To reach you, the following had to happen:

• Miners dug up metals and rare earth minerals that are used in the manufacturing process
• These metals were refined and sold to parts manufacturers
• Different parts were fabricated, including screens, computer chips, and hard drives
• These parts were in turn assembled into a phone or computer
• The finished product is shipped to distribution warehouses
• Warehouses distribute to specific retailers (who likely have their own warehouses)

This is, of course, a simplification of the steps, but the concept of a supply chain should be familiar. When it comes to software, websites, and other such digital "objects," supply chains are less discussed. But unless you spend enormous amounts of time and resources on building everything yourself (and perhaps even then), all software relies on other companies, services, and actors to function. At the most basic level, this includes hiring developers or contracting an agency to build something. But it also includes operating systems, programming languages, hosting platforms, SSL certificates, and other layered services, all of which are created and maintained by someone that isn't you.
 
A supply chain attack is a hack or intrusion that targets the least secure elements in the "digital supply chain" that every site or organization relies on. You are only as strong as your weakest links; usually, we would interpret that as applying to employees who might be targeted for social engineering or use weak passwords. Sometimes, it can apply to the links that we would have assumed are the strongest and spend the least time scrutinizing, such as SolarWinds.

Our usual advice is that most organizations have more to fear from a passive attack than a targeted one. For example, it's more likely that a WordPress site will be attacked in a semi-automated fashion based solely on an attacker attempting to exploit known vulnerabilities in any WordPress site that he can find. While it appears that this Solarwinds exploit was used to target a specific subset of customers (Cisco, governments, and other valuable targets), this type of vector can also represent the worst-case scenario for what a passive attack might look like.

The Risks of Interconnectedness

Our exposure to third parties in our supply chain is not necessarily limited to attacks. Last month, when Gmail went down briefly, it took down the email systems of thousands of companies that contract that service from Google rather than running their email servers. (NP Group is one of those companies.) And at least once or twice a year, AWS or Cloudflare sneezes, and the whole internet catches a cold (as other services like Facebook, Netflix, along any other site relying on the affected data center goes down).

It's not unusual within a website's code to find bugs (both known and novel) in open-source code libraries that have become indispensable. Even a reasonably simple site might rely on Wordpress's codebase and various plugins (both free and paid) to power most of the site's functionality. A bug or failure (and of course, a security vulnerability) within any of these elements can take down your site.

What Should You Do?

Just like in the physical world, we are all interconnected. You can't just move into a log cabin in the forest (or whatever the digital equivalent of that would be). It is impossible to eliminate these potential threats, but you can consider your risk tolerance and act accordingly. If your site stores sensitive personal information, it is rational to be more cautious than you would be with an informational marketing site.

Though some level of risk is a given, there are steps that you can take to mitigate and reduce the impact of any failures or attacks in your supply chain. The most important is redundancy. If a service that your site depends on were to fail, is your site built so that you could get back up and running without it (even if that means that your site is less functional temporarily). This also applies to basic server administration. Most organizations make regular backups of their site and database, but I can't count how many times I've seen the backups stored on the same server as the site. If a server goes down, there is no way to reach the backup and restore it to a new location. As part of NPG's 24/7 monitoring service, we also store remote copies of all backups that we can use to re-platform a site as quickly as possible.

This also means that you should consider contracting your server security to an agency you can reach by phone. While services like Solarwinds and Sucuri are excellent (and we make use of the latter ourselves), there is no substitute for having your own layer of security that is self-hosted and monitored by a human. While nothing is 100%, having multiple layers in place significantly reduces your risk of everything failing at the same time.

If you would like to review your exposure to supply chain risk, we are happy to help you evaluate and make recommendations. Contact us today.