Results 1 to 5 of 5
  1. #1
    iNET Interactive
    Join Date
    Jan 2010
    Location
    Seattle, WA, USA
    Posts
    377
    Thanks
    1
    Thanked 29 Times in 24 Posts

    The complicated world of antivirus testing




    IN THE WILD

    The complicated world of antivirus testing

    By Robert Vamosi

    Don't shoot the messenger when it comes to AV test results.

    The fact that MSE barely got certified by AV-Test.org shouldn't be easily dismissed not without considering all the facts.

    The full text of this column is posted at WindowsSecrets.com/2011/05/19/07 (paid content, opens in a new window/tab).

    Columnists typically cannot reply to comments here, but do incorporate the best tips into future columns.

  2. #2
    New Lounger
    Join Date
    Dec 2009
    Location
    Pittsburgh, PA
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts
    My question is, "Who funds AV-Test.org?" They have to be paid by somebody. If the Anti-Virus companies are funding the testing, it's no surprise that free software comes out poorly in the tests. Just look at pharmaceutical testing as an example of self- or inappropriate funding sources which produce the results that the financiers are looking for.

  3. #3
    2 Star Lounger
    Join Date
    Dec 2009
    Location
    Sacramento, CA, USA
    Posts
    116
    Thanks
    7
    Thanked 4 Times in 4 Posts
    Valid concern, but did you read the article all the way?

    "MSE was not the only AV product to have poor zero-day results: McAfee Total Protection 2011, CA Internet Security 2011, Comodo Internet Security 5.0/5.3, Norman Security Suite 8.0, and PC Tools Internet Security 2011 (all paid products) earned lower scores."

  4. #4
    3 Star Lounger
    Join Date
    Jul 2010
    Location
    Raleigh, NC
    Posts
    211
    Thanks
    11
    Thanked 20 Times in 17 Posts

    Question MSE + Threatfire = Full Coverage?

    Robert, would you please address whether using PC Tool's free Threatfire app in conjunction with MSE adequately compensates for MSE's "zero day" weakness?

  5. #5
    Lounge VIP bobprimak's Avatar
    Join Date
    Feb 2009
    Location
    Hinsdale, IL, USA
    Posts
    2,482
    Thanks
    176
    Thanked 152 Times in 129 Posts
    Most noteworthy is that many of readers' favorite free products have not even been tested at all in these reports. Where is Malwarebytes? Super Antispyware? Threatfire? For that matter, where is Prevx? The lack of inclusion seems to me to be the most telling feature in the entire story. What are these testing labs afraid of??

    Granted, MSE by itself does not offer much by way of Zero-Day protections. And it does not do well at Rootkit detection and removal. But without testing these other programs, we are left with no data to interpret. And the combined effects of using several different antimalware products is completely untestable in a laboratory environment.

    Even without using the active components of Malwarebytes Pro, Malwarebytes and MSE 2 interact during scans with either engine, if the other product is running. I was recently scanning with Malwarebytes while MSE was active, and even though MBAM did not report any malware, and MSE had not flagged anything, as Malwarebytes scanned over a certain file, MSE popped up an alert and quarantined the file. It was a rootkit component as it turned out, and what neither program could detect on its own, the combined programs did find and remove. This experience has me really shaking my head at single-program AV testing reports.

    Add to all of this confusion the browser security features of IE9, Firefox 4 and Google Chrome 11, (not to mention some of the add-ons for each of these browsers) and we have no idea whatsoever what to make of the test lab reports. And Chrome behaves differently when installed into accounts without Administrator privileges than when installed into accounts with Administrator privileges. (IE and Firefox do not share this characteristic.)

    I also use a multi-engine in the cloud scanner called Hitman Pro (free). It uses (among other engines) GData. Again, the combined effect of a multi-engine scan is hard to reproduce, let alone evaluate in a test lab. VirusTotal is based on the presumption that a multi-engine scan will find more malware than a series of single-engine scans. Again, a hard hypothesis to test.

    Note to all Threatfire users -- test labs will never test all combinations of multiple security tools. They have a hard enough time trying to test security suites due to the inability to isolate specific components. And there are so many products out there that no one can afford to set up tests for every possible combination. Yet another reason not to trust or rely upon the current laboratory test results.

    If you have a security baseline which you believe has been working well, do not switch based on these test reports. But if you are choosing for the first time, or if you are dissatisfied with what you have, feel free to use these reports as part of your decision making process. Just do not make these reports the only basis for your choices -- professional reviewers are much more familiar with real-world antimalware performance than any test lab, and they do look at these reports when making recommendations. It all comes down to, "Whom do you trust?" I cannot decide that for anyone.

    I will stick with a multiple scanner approach. Super Antispyware, Malwarebytes, Hitman Pro and MSE combined has to be better protection than any one of these used alone. And I still say that Windows 7 does not need any third-party software firewall. Windows XP, on the other hand...
    Last edited by bobprimak; 2011-05-21 at 04:30.
    -- Bob Primak --

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •