A bomb just dropped in endpoint security… and I’m not sure anyone noticed

wp84552171_01_1a
Pay no attention to the man behind the curtain…

Update: Reuters now has the story

Update 2: I’ve updated this post with additional information, here. 

VirusTotal just dropped a major bomb, and only people deep in the endpoint security ecosystem understand the ramifications of this announcement.

If you’re involved in endpoint security to any degree – as a customer or an industry person – you need to understand what just happened. It’s really, really big.

A bit of background.
VirusTotal is a multi-engine virus scanner. You upload a file, and it passes the file to a large number of commercial antivirus products, and it tells you which engines detected the file as malicious.

While there are other tools available, and some have come and gone, VirusTotal is the big dog in the space. It’s owned by Google, has massive computing and resource power and everyone in the security industry uses it.

VirusTotal shares the results with subscribers. So, you can pay to get extensive and detailed information on what has been detected at any moment of the day, and who detected it. 

How antivirus companies use VirusTotal to make better detections.
It’s common practice of antivirus companies to use VirusTotal as a tool to make better signatures.

For example, if a researcher finds that two high quality antivirus engines detect a file as malicious, he/she has a high confidence that it’s actually malicious without further analysis. As an antivirus researcher, it saves an enormous amount of time.

Now, there’s absolutely nothing wrong with using VirusTotal results in research, and many antivirus companies use VirusTotal to supplement their own labs. They get samples from VirusTotal, and along with the samples, what engines detected them. If they find that a couple of high quality engines are detecting a file, they can easily add the detection to their own signatures without much further thought.

Now, there’s a next step. You could set up an an API integration with your product. If you scan a user’s machine and find an unknown file, you could upload it through an API to VirusTotal and get a disposition on the file –who detects it. From this data, you can flag a file as malicious.

In other words, you can use VirusTotal to create your own antivirus program. Easily. 

Until now. 

It’s fine to use other engines. If you’re also contributing.
Using other engines to improve your detection rate is completely fine. If you’re also contributing back to the community yourself. In other words, if your antivirus product is also one of the participating antivirus engines.

The dirty little secret
And here’s the dirty little secret that very few people know. There are a number of endpoint products that use VirusTotal to determine if a file is malicious. Without any contribution to the communityWithout giving anything in return. 

They simply pay VirusTotal a subscription fee, and receive the information.

And some of these companies have been getting a lot of attention for their supposed prowess. But for some mysterious reason, they refuse to put their own engines on VirusTotal. Could it be because they don’t want to contribute back? Maybe. Or it could be that they just don’t want everyone else to see how poorly their products actually perform.

Unfair? Yes.
Using VirusTotal information without any contribution back to the community is patently unfair. The people who are actually writing detections are sharing their results with the rest of the community, while a small group of endpoint products have been boasting of their extraordinary abilities, while working off the backs of other researchers. 

So as a customer, perhaps you can ask the next endpoint security vendor if they’re on VirusTotal. If they are, they’re contributing to the antivirus community. If they’re not, they’re not. Whatever their PR story, that’s the simple truth.

Until now.
Well, the world just got a bit brighter for the many endpoint security companies that actually contribute to VirusTotal: Because VirusTotal just announced that they are requiring that all scanning companies that use their service must integrate their engines into VirusTotal. Furthermore, “…new scanners joining the community will need to prove a certification and/or independent reviews from security testers according to best practices of Anti-Malware Testing Standards Organization (AMTSO).”*

It’s big news. It levels the playing field. No longer will antivirus companies see their hard work taken by some sexy startup that’s raised millions of dollars on the false promise of “next generation” endpoint or other such nonsense, while bashing the very companies that they’re effectively stealing the intellectual property of. And perhaps, we’ll see what their products are really made of. Because without VirusTotal as a crutch, companies that rely on it are going to see their detection rates take a hit.

Poetic justice, indeed.

What does this mean for the IT manager?
If you’re an IT manager who has been duped by sparkly marketing materials to buy-in to one of these “next-generation” endpoint products, take a hard look at their actual detection capabilities. If they’ve been using VirusTotal results but not contributing back, their ability to detect malware just took a potentially serious hit. This is serious.

You don’t have to believe the marketing hype. Setup a virtual machine that’s separated from your corporate network, and go to a site like MDM to find all kinds of nasty stuff. In the words of Ronald Reagan, “trust, but verify”. One nasty piece of malicious software (especially ransomware) can have serious consequences.

In closing
My compliments to the VirusTotal team for seeing this disparity and unfairness and taking such swift action. A class act, indeed.

And now, perhaps, we can all finally see what is really behind the curtain.


* Disclaimer: I am a board member of Malwarebytes (a contributing member to the VirusTotal community), and an advisory board member to AMTSO.  The opinions in this blog post are my own and are not connected to these two organizations.

83 thoughts on “A bomb just dropped in endpoint security… and I’m not sure anyone noticed”

  1. Very interesting article Alex. I will have to check out virusTotal.

    Security issues that originate from applications rather than OS isdues seem to be of more concern lately, things like WordPress and Magento. Securi seemsto have this are under control. Do they use VirusTotal?

    1. “I will have to check out virusTotal”

      Tell me your kidding, don’t reply with what does malware mean, PLEASE!!!!!!!!!

      Alex!….You didn’t see this coming?…….I would be surprised to hear that indeed but ya sure caught a Cretemonster and a Blender offguard, Tammy will eat this up!!!! 😉

        1. You and Me both, Kimberly is still around too, I see her post to twitter every so often, along with a few others original SpywareWarriorTroopers, seems like a lifetime ago and just the other day, both at the same time too but man Mr Alex, did we ever pave the roads and show many others the way who are now working for Security Firms all over. Times are beginning to make that next change too, feels like Windows could soon be in the Smithsonian Museum and outa the picture all together, not sure why I feel so strongly about that either. You my friend were one of the first trailblazers when it comes to Original Antivirus, even taught me a few things with Adams help ofcourse. 🙂

    2. Alex – the real bomb was the Ninth Circuit court affirming that the two Settlements which control all online use of timestamping are good and the action of the Appellate Court perfected their enforcement standing.

      What this means is any and all timestamping inside of switches and routers (rotflmao) as well as inside Client/Server and a number of other applications are all now controlled no matter what country they are run in, by the laws of the State of California.

      How? Simple – my timestamping IP is controlled according to the Federal Courts by two settlement documents which require you and every other user to accept California Law. The two patents are US6370629 and US6393126.

      What this means to anyone using any infringing IETF Protocol inside their Netwiork Switch or Router now is operating solely under the Laws of the State of California. As to how – the settlements only allow for developmental uses without getting a release from the terms. IETF knew this and has been noticed through their IPR website filings since 2003. So there is no third party cop-out here. Everyone globally is tied to this now.

      Is that funny or what?

      1. Not sure how this make a real difference. CStartup companies don’t bother to contribute because they’d rather spend money than time. If they are forced to contribute, they can contribute with holdbacks, and who is going to find out?

    1. How is Cylance going to be negatively affected?

      From what i understand, Cylance is the only AV out there that does not rely on constantly referring back to an updated VirusTotal database to protect endpoints. Apparently they are using some sort of machine learning algorithm and does not need to refer back to VirusTotal to confirm malware is malicious, and also to block/quarantine the file.

      If anything, this sounds like it will boost Cylance’s market share by pushing out many vendors who are using VirusTotal. What do you think?

    2. I find it funny you can’t disclose this and a few scrolls down you are calling some EDR guys out. Truth is Cylance is eating everyones lunch and they don’t depend on VT. However ofcourse you wouldn’t say that since you seat on the board of a competing vendor.

        1. Let’s do this Mr. Board member with no vested interests. Why don’t you stand up your Malwarebytes product vs. Cylance and let’s see who has the better detection rates. Isn’t that the point you are trying to prove? Without VT. Your detection rates comment might be correct with the other EDR vendors you mentioned but it doesn’t impact the Cylance AI technology.

          1. You and I agree on something! I think we should do just that. Just put your Cylance engine on VirusTotal, so we can all see the engine at work. Then we can see how it stacks up to Malwarebytes, Symantec, Kaspersky, and all the rest of the companies you so readily dismiss.

            That’s the only honest way to really see, isn’t it? And then you can continue using VirusTotal as well.

            Sounds like a win-win to me.

        2. Cylance is garbage. It scores worse than the AV products it claims it can replace in standardized testing. It lacks protection for any kind of targeted or advanced attack that doesn’t use malware as the delivery mechanism (e.g. dropping a webshell on a web server). And they still don’t have an EDR component. I don’t understand why anyone would pay them money for their half-baked product. However, I do have to tip my hat to their marketing department. They are cashing in on the anti-AV sentiment in the market right now. Unfortunately, their early adopter customers are likely to continue suffering infections and breaches because the product is so ineffective.

          Just for fun, take any piece of malware and package it up with AutoIT https://www.autoitscript.com/site/autoit/ and then run it on a machine “protected” by Cylance … Cylance will fail to block it. Every time.

      1. Why doesn’t Cylance participate in 3rd party tests any more? Or do you think your product can only be tested in ‘controlled’ conditions? It’s easy to make ad-hominem attacks – but you are dodging some difficult questions.

        1. AV testing in any form is misleading in any form, Ill put my product up against any out there in a live box test performed fairly and in a contained environment exclusively.

          We still miss items daily, anyone know of a company in the industry that has a real 100% Average in any form of testing?

          Unless you get out there and act like a 13 yr old that’s trapped in his room on a Friday night, you haven’t tested much of anything in my opinion and not just once or one platform, I mean multiple OS and multiple machines with different OS Builds,Versions and build platforms (x86 or x64) NTFS or FAT32

          If you can catch me after the start, Ill give ya the win publicly but I doubt Ill hang around long enough to even see you back there, suckin wind! 😉 Damn Skippy my Clouds work and they work quite well!

    1. I should have said Cloud Based Antivirus Solutions to make my point but its a long day and I must go continue to make the AV that could never work, spank some more malwares around like a 2 cent, ah nm! 😉

    2. I think it won’t be a big problem for them, Alex, now they will switch to VirusTotal clones that allows them to do the same they were doing with VirusTotal until now.

    3. Unless you are just singling out all the new players, not sure why Carbon Black made your list? Clicking on a tab on file information shows they have been contributing to the community for a while.

      1. Yeah, you’re right. They’re not an engine on the scanner, but they do play nice. They also are very open about their integration with VT in the product. I’ll remove them from that little comment I made upstairs.

  2. I get the reasoning for the move…

    But for those of us who use VT as a kind of double-checking source of information, when it comes to looking up known malware signatures, this may or may not create more of a headache. As it is, there are already too many AV vendors contributing to VT that are constantly throwing up false positives as “malware” or “PUP”. Sure is going to be a bigger pain if more “Anti-Virus” companies add more noise.

    1. We simply suffer because of this, you can look but there I nothing good to be found in it and I have seen the worst side and the best side of the industry in the last 2 decades, you cant rationalize it, there are always leechs and ass clowns in every industry, if anything, I was actually surprised before disappointed but I’m sure it was destined to happen some day. Ya cant have much good without some bad around, spreading its stinch wherever possible.

      Good Luck JC,Emi and the rest at VT, hopeful solution comes in short time!

    1. Alex, I hlwas told that Fishnet Security has now removed SentinelOne from all their internal systems due to false postives, blue screens, and other application conflicts.

  3. Do you mean companies like enterprises? I think VT is really going after those players that are using scan results for their own benefit, to compete with the very people that they’re taking the results from… without being a participating scanner.

    Feel free to email me to discuss more.

  4. While I love a good mud-fight, I think this “bomb” is a bit misplaced. While there may be some cheating scum that uses VT as their primary detection “engine”, many of the EDR guys simply call VT to validate their behavioral detection [we call this “secondary detection”] and/or as a mean to name what they found.

  5. Hey guys,

    Hate to break up the fun Cylance bashing folks but we need to set the record straight. This announcement does *not* impact Cylance one iota. We have a completely independent conviction engine using math and algorithms, learning from the past to predict the future. We would be happy to educate anyone who is interested.

    Alex, we would not be against a retraction of your claims to the contrary. Completely up to you.

    Now, back to my weekend.

    Thanks
    Stu

    1. Stu — thanks for dropping by, and I genuinely appreciate it. This post was not intended to turn into a Cylance bashing fest. It was more targeted at the several players who are abusing VT. I am happy to retract my statements (in fact, I will remove a lot of the comments above, it’s getting a bit out of hand in general).

      [Edit — the following has been found to be incorrect information and should be ignored as a data point] However, I do have independent verification from other researchers that they have found when a unique file is placed on a machine, and that machine is scanned by Cylance, the unique file somehow finds itself on VirusTotal within minutes. I have a difficult time explaining this without the conclusion that somehow, Cylance is using the VirusTotal API to upload the file. If you can somehow disprove this (recognizing it’s nearly impossible to prove a negative), it would be very helpful to the community. [/end edit]

      I do have empathy for your position. I have been through the process of creating an endpoint product, but I suppose the difference is a) we were part of the community and b) we exposed our product to independent analysis. I’ve not seen either occur in the case of Cylance. Perhaps if Cylance was more open and forthcoming about its product, allowing it to be publicly validated, there wouldn’t be this air of suspicion and we could all go on with our lives – and our weekend.

      1. Did you/they consider that their cloud-based management platform might leverage VT’s collective “opinion” of a file in their reporting – but not in the actual analysis? Ie “we have x% confidence this file is malicious, and VT collectively reports X/Y unsafe” or something similar.

        1. Dude, horseguy, you do realize your point proves this fact, correct? Clearly either a buyer of this new garbage or a “trusted VAR” seeing his/her trust finally flushed away at long last. Un-freaking-believable.

          Wow, it is no wonder why the bad guys keep winning and the fat guys are getting fatter by any means possible, even at the detriment of their own customers.

          It’s like a telenovela where the evil trusted VAR who’s deep in debt due to some sort of repeated missed quota desperation or simple greed goes takes to what’s natural and slings more snake oil to the man with the horse with no name cause it just too darned easy.

          This is the real problem with this industry. You see FireEye up there either? Sentinal One? If I had more time and cared enough I’d deduce some more. We’re standing arm to arm in the world-wide battle against the bad guys as millions innocent lose millions every day for the same exact reason a few make hundreds of millions in the USA’s Sillycon valley while down on the ground where we’re seeing it happen, please let this be like the bright kitchen light that chases the value added roaches (aka VAR’s) with their duplicitous “I’m independent so buy this shiny new expensive thing cause it’s better than the last I sold you” back into the walls where they belong. Ever since FireEye turned up and publicly bankrupt all of their shareholders while snatching money out of their customers accounts payable, this industry’s changed for the worst. Industry discretion, honest, publicly verifiable head to head comparisons conducted in the clear light of day were swapped for get rich quick on the backs of those who worked under the honest promise of continuous improvement with fixed budget.

          Now I understand how many of these next-gener’s were able to allocate 95% of their revenue to sales and marketing and 2-3% to R&D and still manage to sit at valuations of 90PE and higher and turn up on Mad Money with honest Jim Cramer more than once…hmm weird timing on that honest Doug DeWalt step down?

          All that food for thought’n and reaffirmed disappointment’s got me hungry for a bowl of Stu…actually that thought made me immediately sick. La Cucaracha Stu.

  6. Alex,

    This is just not how our product works. At all. It doesn’t upload files to anything other than our private servers. And have never uploaded to outside services like this. If we did, our customers would have killed us long ago with their bare hands and I would have probably cheered them on. Please feel free to connect with me directly via email and I can prove it to you in probably 30 seconds. Also, we encourage every customer to “test offline” which means we have zero way of uploading anything to anywhere or accessing anyone else’s results, and yet somehow magically we convict with the same core engine and efficacy.

    1) if being “part of a community” means we need to share our algorithmic, unique conviction engine with Big AV so they can steal our convictions, then yes we will not be able to meet that criteria.

    2a) we did participate in public AV tests (AV-TEST) and we discussed the testing extensively in a blog here: https://blog.cylance.com/ai-vs.-av-gorillas-and-germans-and-gartner-oh-my

    2b) if “independent” testing actually was “independent” then we would *love* to participate in testing. Unfortunately this is not the case. I spent 3 years testing at InfoWorld Labs, probably the most “independent” testing facility I’ve ever come across and know how hard it is to be truly “independent”. The most “independent” test is the one the customer does, with their own designation of “bad”. Because no one in the industry truly determines bad “independently” enough to set the standard to test products with. As former Global CTO for a major AV company, I can assure you this to be the case.

    Have a great weekend!
    stu

    1. The main thing that the community approach defeats is the “I got this” model that some companies follow.

      That failed every time it was attempted. You might remember a company called McAfee…

    2. You feed it the samples you suck out of VT. How do you think everybody does not see that? There’s not even a curtain between you and the hack, you hack,

    3. So you’re saying that all of the vendors that have been participating monthly in AV-Test.org or Dennis Labs are being duped by unscrupulous and dishonest testers within those labs that have been conducting industry comparisons for over a decade?

      Those organizations and the actual testers (and many others like them) may find that to be a very insulting statement and a personal attack from a desperate huckster.

      Does this qualify as a more honest evaluation? That first slide doesn’t really qualify as a thorough and unbiased testing methodology. Where’s the detailed configuration information? And we should just trust the vendor conducting the test that the samples are what you say they are when in fact you’ve got another youtube showing downloading a hundred samples from where else but VirusTotal that appears to have been very recently pulled, hmmm…

  7. Wow! Whaaaaat!!! I’m so excited to hear about finally getting some equity in the Virus Total! Sounds pretty dope. Glad this really big bomb dropped.

  8. Ok, Cylance, SentinelOne are affected. I would really like to know how it affects Crowdstrike as well. I know they don’t present themselves as an AV vendor, but they do block “high confidence” threats just based on virustotal scores.

    1. It doesn’t affect SentinelOne. Check our CSO’s response to Eckelberry’s latest blog. This is a non-event as VirusTotal is 1 out of 7 vendors we use in our Cloud Intelligence, not including what we collect from our opt-in customers. Cloud Intelligence is a crowdsourced service we add in to filter out legacy threats. Has nothing to do with our Dynamic Behavior Tracking (DBT) engine that’s used to detect, prevent and remediate the 100k’s of new threats that are created weekly that have yet to be detected and examined by the traditional scanning engines that rely on VirusTotal. I think the bigger story is how is VirusTotal going to evolve such that they can take advantage of the intelligence we’re collecting every day on unknown threats. We’re more than happy to work with VirusTotal to integrate – if the AV guys will let them. Right now all indications are they have no desire or intention to work with the next-generation endpoint protection companies.

      1. “Lies, damned lies, and statistics”
        – Mark Twain

        Virus Total has 49 antivirus product contributors and 55 website/domain scanning engines (per a quick review of wikipedia). With no further access to Virus Total, Senitenelone cloud intelligence (signature-based dataset, cough, cough) has been reduced from 110 datafeeds to 6. I’d wager that the marketing spin from a marketing executive, is just that. Good luck using behavioral analysis to stop ransomware, by the way.

        1. well said!! Spin wizards including C-level execs are busy doing damage control. VT going away will impact them one way or the other despite what they claim.

  9. Anyone using opensource Cuckoo sandbox uses VT since Cuckoo does. Guess who uses Cuckoo? I wont say but they are all over the net doing damage control. They can claim they dont use VT (directly) but what say ye about honesty?

    Its simple-if you do not contribute to VT, you don’t have telemetry/data sets in-house to re-train your Machine Learning, nor keep False Positives low. And with M/L you have to retrain and constantly. Buy the way M/L is nothing new, many big vendors have been doing for years. Newbies claiming its next gen?

    Re VT API Down but not Up: Your basically scamming both customers and riding the backs of other companies whom really work hard-and then your competing against said customers.

    How does this hurt anyone? You have been scammed, you did not do your diligence, don’t blame VT for finally being fair.

    It’s been the dirty endpoint secret for quite some time and kudos to those vendors whom kept it quiet without naming names in print or when competing. Now that’s honesty.

    And yeah disclosing I do work at a major endpoint player.

    1. If your not a listed company “…have contributed somehow.”?

      https://www.virustotal.com/en/about/credits/

      …”This page acknowledges all companies and individuals that have integrated a product, tool or resource in VirusTotal, or have contributed somehow. VirusTotal is not tied to any of these companies or individuals in any way, hence, the results provided are not subjected to any type of bias.”

      Rhetorically speaking ask yourself some questions – Why are you not using VT then to submit and help the community and thus users and companies protect themselves w/ increasing efficacy and lowering FPs (across major and minor vendors that compete!!)? Is your dataset(s) in-house are that much superior, larger, and technology that much better – Your product stands alone on the mountain top of endpoint security looking down at all others? So you will not contribute, you will cut off VT feed using now, you are all alone from the wisdom of the community, you are vastly superior, have a bigger dataset, and you shout that from the peak. Maybe, its possible. Nothing is impossible. Probability of these claims being true?

      The machine learning does not need re-training!? And if it does need re-training, you use a solo dataset not using VirusTotal?

      If you were using/used VT to check hashes/download results without contributing why then was the reason to stop now, was it not loss of your tech then as well? But suddenly its now concern over IP loss?

      The statements and questions are contradictory no?

      If you cut off the VT feed – is that only directly in your software and is that including usage by any opensourced component like Cuckoo sandbox? All companies need to clarify their statements of fact here. We did use but not contribute, we cut off now VT entirely both in our software code and in any open sourced software employed in our product. Unequivocal statement. Then state you are not using nor switching to a similar VT feed from another company, either directly in company product code or indirectly in open source sw. By the way that may be the next endpoint bomb to drop (hopefully) if the VT like feed provider follows suit.

      Then submit to third party analysis for efficacy and false positives and confirmation of no VT feed indirect/indirectly and same for that other community feed offering.

      One does not have to name any names: just ask yourself questions, google, look at recent public articles, and ponder any ironies in statements on the matter.

  10. There are a few items bothering me here. In your blog you state that these companies are getting benefit without sharing. At the same time, you also state that a file appears on VT after being uploaded. Doesn’t that mean that these vendors are in fact sharing the file with all subscribers? If all the other vendors who supply engines get a sample feed, then they receive the uploaded files from Virus Total, no?

    So in a way, anyone who uploads a file to VT is participating, and thereby supplying all vendors with a potential piece of malware for free. Unless I am reading it wrong.

    Also, there is another thing to consider. The “engines” supplied by the vendors are only one component of their protection strategy. There are other engines and methods in use today as vendors try to find the “next big way” to detect and stop malware, other than the hashes and signatures that are prone to slow turnaround to the endpoints. Aren’t the engines on VT just file scanners that don’t execute the file ? These other methods are generally not represented on VT(as I understand it), which means a vendor may not detect a new piece of malware in the VT scanner, but may still block it in their application. Meaning that you are not getting a full representation of a vendor’s capabilities at blocking new types of malware. So not really stealing the important “next big thing detection methods”.

    These days it seems those types of technologies are really where the value is, rendering the capabilities of VT only as a comparison tool for researchers on known malware(and if that is all they are using for their definitions, they will be woefully inadequate). Also meaning the vendors that use these APIs are only really looking at older stuff that is not as important as newer types of threats that appeared today.

    1. ” In your blog you state that these companies are getting benefit without sharing. At the same time, you also state that a file appears on VT after being uploaded. Doesn’t that mean that these vendors are in fact sharing the file with all subscribers? ”

      No, what this means is that said company is taking the file hash and looking it up on VT for a match. If the match indicates known malware – then their artificial “intelligence” will convict it as malware. In a nutshell, this is the basic form or VT abuse. When an NG vendor relies on training their engines on VT data, where said data has been submitted by traditional vendors, thats when the whole NG argument starts unravelling. QED.

  11. Thank you for your blog post. Brown and I happen to be saving for a new book on this subject and your post has made us all to save our money. Your thoughts really clarified all our queries. In fact, greater than what we had recognized just before we stumbled on your superb blog. My spouse and i no longer have doubts along with a troubled mind because you have totally attended to our own needs above. Thanks

Leave a Reply