It really comes down to beer (endpoint security redux)

572px-Dutch_beersUpdate: SentinelOne responds in the comments. Additionally, they claim (and I have no reason to doubt this claim) that the report I referenced was an older version that had incorrect information on the part of Tevanos.

As a follow-on to my recent post about endpoint security (see “A bomb just dropped in endpoint security…“), I thought I’d share some additional thoughts, conclusions and opinions.

It really comes down to beer. But more on that later.

Reuters
First off, Joe Menn at Reuters wrote a story this morning. It’s a good story, fair and balanced.  Worth reading. There will likely be more stories as well.

Beating up vendors isn’t really my thing
In the recent blog post, there were a lot of slings and arrows thrown at a few endpoint security players. My blog got trolled quite a bit. I cleaned it up.

Cylance certainly came under heavy attack by some commenters, and I removed those comments.

I’m not interested in beating the crap out of some company; I’m really just interested in writing about stuff that I find is interesting.

On Cylance
There was a possible confusion that got propagated that Cylance was using VirusTotal directly in their product. I now have information that this may be incorrect.

Cylance was using VirusTotal, as they said in the Reuters article. It’s possible they were using the service to download samples to train their engine, not directly from inside their product. It’s also possible that they used VirusTotal to help detect malware.

I don’t know for sure, and that’s why I expect to be talking to them in the next several days.

Note that Cylance is not a bad group of people. There are many very good people working there, and they run a good business. I’ve challenged them to get more public tests, and I hope they do so. So far, there have been two tests that I know of — a condoned test by Av Test, and another test, where AV Comparatives/Effitas had to basically break the rules to try and get a copy of Cylance (one can’t just download Cylance and test it, as Cylance keeps its trials very closely monitored).

So testing more, and being more public, would be a good thing.

At any rate, peace, people.

Other endpoint players
I don’t know about other endpoint players. I know SentinelOne, for example, has been open about using VirusTotal.

Specifically, this report (PDF) on SentinelOne’s capabilities in healthcare, highlight this point:

SentinelOne ensures that it is always up to date, checking file hashes against reputable sources such as Virus Total. Using this method, SentinelOne’s platform does not suffer the traditional time lapse in needing to push out new definitions…

[Edit – As mentioned earlier in this blog, SentinelOne tells me this quote is from a PDF of an older document that incorrectly noted VirusTotal as a source and has since been corrected.]

Regarding Palo Alto Networks and CrowdStrike, I really don’t know the involvement of these players. But as the Reuters article mentions, they have been users of VirusTotal (and have not been participating with the community). That’s not to say they’re bad people or have bad products (CrowdStrike and Palo Alto both make excellent products). It’s just something that has been addressed.

Endpoint security
My knowledge of the endpoint security market comes from the fact that I’ve been there in the trenches.

I’ve presented at conferences like VirusBulletin and submitted papers, etc., but that’s the fun stuff. Actually being in the day-to-day sweating bullets to try to keep your customers protected is a very difficult task.

You see, we created a full-stack antivirus product at my last company (Sunbelt Software’s VIPRE) and I personally ran the antivirus lab for some time.

When we released the product, it was probably mediocre, and we avoided tests. It got better when we started opening up to tests, and in fact, it got quite good (and then I sold the company and, well, I don’t recommend the product anymore).

We were on VirusTotal, and it was painful to see us miss detections. But at least we saw ourselves for what we were, and made our product better as a result.

Is there a “next-generation?”
The truth is there are only so many ways to skin a cat. The former Eastern Bloc countries were famous for producing some of the world’s most brilliant mathematicians and now we have companies like Kaspersky and BitDefender with just those same type of people. Yet even they have a tough time of it (no matter what they say publicly). It’s not an easy business, trust me.

I don’t entirely buy a lot of the “next-gen” security arguments. I think that there is room for innovation, but I’ve seen companies like Malwarebytes (of which I’m a board member and incredibly biased toward) and (recently) Symantec do some very impressive work in detections, without resorting to anything of the next-generation type. It really comes down to a lot of hard work, block-and-tackling type of stuff.

So, it’s no surprise that people do whatever it takes to get the best result possible. If this means using VirusTotal to do a hash lookup (which IMHO is fairly silly, since polymorphism makes hash lookups far less useful than people might think), or good old fashioned PR to paint lipstick on a pig, well, so be it.

The key is openness. If you have something special, open it up to the world for them to look at, to test, to validate. Be a part of the community and give back to it. It will make your product much better.

Which comes to the beer
Information sharing in security happens around conferences and beer. (When I brought in a new lab manager for my research team years ago, I urged him to spend as much time as possible going to conferences and drinking beer with other experts. He didn’t object).

Perhaps that is a bit tongue-in-cheek, and it’s not that we’re a bunch of alcoholics (well, mostly not), it’s simply that a lot of information sharing happens at conferences, when experts talk to each other freely. The swords of competition are put down briefly, people open up, and you hear a lot of interesting things.

And what I hear around the tables is what is reflected in some of my blog posts. There is good data, but it can’t be substantiated and it won’t ever be confirmed. So, I’m sorry I can’t be specific, despite many of you emailing me for much more detail than I am prepared to give. Trust is everything in security.

But beer? Yes, we can all share that freely. So, here’s to beer (in my case, the ever excellent Buckler Non-Alcoholic).

You can see who VirusTotal credits here. 

A bomb just dropped in endpoint security… and I’m not sure anyone noticed

wp84552171_01_1a
Pay no attention to the man behind the curtain…

Update: Reuters now has the story

Update 2: I’ve updated this post with additional information, here. 

VirusTotal just dropped a major bomb, and only people deep in the endpoint security ecosystem understand the ramifications of this announcement.

If you’re involved in endpoint security to any degree – as a customer or an industry person – you need to understand what just happened. It’s really, really big.

A bit of background.
VirusTotal is a multi-engine virus scanner. You upload a file, and it passes the file to a large number of commercial antivirus products, and it tells you which engines detected the file as malicious.

While there are other tools available, and some have come and gone, VirusTotal is the big dog in the space. It’s owned by Google, has massive computing and resource power and everyone in the security industry uses it.

VirusTotal shares the results with subscribers. So, you can pay to get extensive and detailed information on what has been detected at any moment of the day, and who detected it. 

How antivirus companies use VirusTotal to make better detections.
It’s common practice of antivirus companies to use VirusTotal as a tool to make better signatures.

For example, if a researcher finds that two high quality antivirus engines detect a file as malicious, he/she has a high confidence that it’s actually malicious without further analysis. As an antivirus researcher, it saves an enormous amount of time.

Now, there’s absolutely nothing wrong with using VirusTotal results in research, and many antivirus companies use VirusTotal to supplement their own labs. They get samples from VirusTotal, and along with the samples, what engines detected them. If they find that a couple of high quality engines are detecting a file, they can easily add the detection to their own signatures without much further thought.

Now, there’s a next step. You could set up an an API integration with your product. If you scan a user’s machine and find an unknown file, you could upload it through an API to VirusTotal and get a disposition on the file –who detects it. From this data, you can flag a file as malicious.

In other words, you can use VirusTotal to create your own antivirus program. Easily. 

Until now. 

It’s fine to use other engines. If you’re also contributing.
Using other engines to improve your detection rate is completely fine. If you’re also contributing back to the community yourself. In other words, if your antivirus product is also one of the participating antivirus engines.

The dirty little secret
And here’s the dirty little secret that very few people know. There are a number of endpoint products that use VirusTotal to determine if a file is malicious. Without any contribution to the communityWithout giving anything in return. 

They simply pay VirusTotal a subscription fee, and receive the information.

And some of these companies have been getting a lot of attention for their supposed prowess. But for some mysterious reason, they refuse to put their own engines on VirusTotal. Could it be because they don’t want to contribute back? Maybe. Or it could be that they just don’t want everyone else to see how poorly their products actually perform.

Unfair? Yes.
Using VirusTotal information without any contribution back to the community is patently unfair. The people who are actually writing detections are sharing their results with the rest of the community, while a small group of endpoint products have been boasting of their extraordinary abilities, while working off the backs of other researchers. 

So as a customer, perhaps you can ask the next endpoint security vendor if they’re on VirusTotal. If they are, they’re contributing to the antivirus community. If they’re not, they’re not. Whatever their PR story, that’s the simple truth.

Until now.
Well, the world just got a bit brighter for the many endpoint security companies that actually contribute to VirusTotal: Because VirusTotal just announced that they are requiring that all scanning companies that use their service must integrate their engines into VirusTotal. Furthermore, “…new scanners joining the community will need to prove a certification and/or independent reviews from security testers according to best practices of Anti-Malware Testing Standards Organization (AMTSO).”*

It’s big news. It levels the playing field. No longer will antivirus companies see their hard work taken by some sexy startup that’s raised millions of dollars on the false promise of “next generation” endpoint or other such nonsense, while bashing the very companies that they’re effectively stealing the intellectual property of. And perhaps, we’ll see what their products are really made of. Because without VirusTotal as a crutch, companies that rely on it are going to see their detection rates take a hit.

Poetic justice, indeed.

What does this mean for the IT manager?
If you’re an IT manager who has been duped by sparkly marketing materials to buy-in to one of these “next-generation” endpoint products, take a hard look at their actual detection capabilities. If they’ve been using VirusTotal results but not contributing back, their ability to detect malware just took a potentially serious hit. This is serious.

You don’t have to believe the marketing hype. Setup a virtual machine that’s separated from your corporate network, and go to a site like MDM to find all kinds of nasty stuff. In the words of Ronald Reagan, “trust, but verify”. One nasty piece of malicious software (especially ransomware) can have serious consequences.

In closing
My compliments to the VirusTotal team for seeing this disparity and unfairness and taking such swift action. A class act, indeed.

And now, perhaps, we can all finally see what is really behind the curtain.


* Disclaimer: I am a board member of Malwarebytes (a contributing member to the VirusTotal community), and an advisory board member to AMTSO.  The opinions in this blog post are my own and are not connected to these two organizations.