Introduction

If you’ve been following breach news, there’s been a recent “Flurry” of large-scale breaches [TicketMaster / Santander @ The Guardian, TicketTek @ TheGuardian , many others], with a common denominator: Snowflake [link], the US-based cloud provider for data storage and processing. Some victims have pointed fingers directly at Snowflake, while security commentors note recent breaches at Snowflake as well as some questionable internal practices. Meanwhile, Snowflake itself denies being the source of the breaches [Story @ TheRecord]…

What the heck is actually going on? There’s been news stories, blog posts, etc. alleging a number of different angles [That Snowflake’s claim of “it’s the client’s fault” is right, That they may not be directly responsible but something’s off, etc. ] – let me help make heads or tails of it.

TL / DR

• Largest Recorded Breach?: Multiple companies began reporting large scale data breaches, including hundreds of millions of compromised customers, starting in mid-May
• Common Ground: All breached companies seemingly hosted and / or processed data on Snowflake cloud service
• How did attackers get in: Some victims cited Snowflake as the initial access point, whereas Snowflake claimed victims were compromised due to not enabling MFA, allowing unrelated infostealer campaigns to target them
• Worst Practices: Snowflake admitted to a recent internal breach due to the lack of MFA on a former employee’s associated accounts (not a good look)
• If not the direct cause, an enabler: Even if attackers didn’t pivot directly from Snowflake into clients, the provider may have been a source of intel for subsequent client breaches
• Conclusion: Lots of pie, on lots of faces: Like many “he said, she said” situations, the truth is there is likely some blame to go all around here – read on for details

The Facts, Summarized

Here’s a summary of the facts from the various sources, as well as key observations made by the security community.

The Breaches – Facts to date

*some commentators note that ShinyHunters may just be a data broker, and not the original threat actor – this remains unclear at this time

Snowflake’s Response and Comments from the Security Community – summary

Putting it all together & Making heads or tails of it

So what’s actually true? Hard to tell, as we’re dealing with imperfect information, but Kevin Beaumont’s take is likely closest to truth. I’d put an additional spin on things, though:

A chief factor here was the lack of MFA on the client companies’ side. This fact came up in multiple sources. Whether credentials came from a breach at Snowflake or an independent infostealer campaign, MFA, properly implemented, would have foiled the attacks.

That said, it also seems that not all is right at Snowflake. I say this for a few reasons.

First, the speed and prejudice with which HudsonRock had their post taken down, together with Snowflake touting that as indirect evidence they did nothing wrong, does not sit well. It suggests there was something potentially accurate and damaging in HudsonRock’s post – you don’t “Cease and desist” things that don’t matter.

Putting a tinfoil hat on, Snowflake’s statement was worded to note that the demo accounts “did not contain sensitive data” and “[were] not connected to Snowflake’s production or corporate systems”. Note they don’t specifically say “did not contain production data” – and fail to define “Sensitive”. This sounds like careful wording to avoid outright false statements…

Why is this potentially significant? Some of what HudsonRock’s post showed included files with account / client details. While passwords weren’t visible, other meta data attributes were – including what appeared to be real client names, onboarding dates, etc… it’s entirely possible “MFA Enforcement?” was one of the columns we didn’t see in the truncated screencaps.

Client names alone would be a good starting place for a “Target list” for an attacker, so much more so if other data, such as MFA settings, was also available.

Second, Snowflake doesn’t make it easy to be secure by default. Their own documentation notes that while MFA is an option, it is disabled by default, and required each user to opt in individually – no central policies or rules exist to force enrollment en-masse. While a client could get around this by using an external identity provider, the fact they designed their own product this way suggest a “security-second” engineering mindset in-house.

Lastly, the fact Snowflake admitted to themselves being breached, in the same way (lack of MFA enforcement), and for “accounts belonging to a former snowflake employee” evidences poor best practices at the provider (and is thematically consistent with the above point about MFA defaults). Not mandating MFA internally, as well as not having asset management adequate to the task of cleaning up resources of departing employees are both lapses from best practices.

It wouldn’t be much of a stretch to think that in such an environment, said former employee might have replicated some production data (e.g. client meta data, as above) into a demo environment for testing purposes, which was subsequently compromised.

Even if attackers didn’t pivot from Snowflake to clients, the provider’s policies and public statements don’t engender much trust in the company. Even if customers not enforcing MFA is ultimately to blame for the individual compromises, there isn’t a party involved that comes out of this looking clean or blameless.

Conclusion and Final Lessons

Given how fast things are developing in this story, and the number of potential targets if attackers were indeed cherry-picking Snowflake clients, it’s almost certain we haven’t heard the end of this.

While there’s still a lot unconfirmed, there are lessons for the rest of us … most notably, the top 3 rules of CyberSecurity

1. Use MFA, always
2. See Rule #1
3. To avoid all doubt, see Rules #1 – #2

More seriously, the importance of MFA, of consistent and effective onboarding and offboarding (e.g. to close down “accounts of former employees” BEFORE they become untended attack surface), and proper forensics and incident response (for all my critique, the fact Snowflake had Mandiant and Crowdstrike on the ground so fast tells me they already had them on retainer – which is a VERY good thing) are all solid takeaways for security practitioners.

As with many high-profile breaches – nothing truly novel here in terms of attack or prevention – the name of the game is best practices – there’s a reason fundamentals are called “fundamentals.”