McAfee reveals SiteAdvisor's retesting policy
By Brian Livingston SiteAdvisor.com, which rates Web sites on a green-yellow-red scale, is releasing details for the first time on how quickly it retests sites, after a story on the subject appeared in Windows Secrets on Feb. 12.
To its credit, the site-rating service — which was acquired in 2006 by the security firm McAfee Inc. — is publishing a phone number for complaints and says it will reverse within days any genuine rating errors that are brought to its attention.
The Feb. 12 article was headlined “SiteAdvisor ratings may be 1 year out-of-date.” The story was written by WS contributing editor Mark Joseph Edwards, and I was credited with research assistance for interviewing McAfee representatives.
The article gave as one example a site named HometownZone.com. This site, which had been legitimately rated “green,” showed up in SiteAdvisor’s analysis as hosting several harmless hyperlinks to “good” sites. No problem there, except that site owner Scott Thompson said he’d removed the links approximately six months earlier.
The implication was that SiteAdvisor hadn’t scanned the site in months, or the nonexistent links wouldn’t be shown.
McAfee research analyst Shane Keats was quoted in the article as saying the company wouldn’t comment on how often it scans sites for spam, infected downloads, and other threats. “We’ve made a public decision not to tell how often we test sites,” Keats said at the time.
Further, Keats was quoted as saying it would be difficult for a site owner to get a rating changed in fewer days than the “probationary period” published by SiteAdvisor. These periods range from 30 to 365 days if a site is judged to have a Web exploit, 60 to 270 days if its e-mails are considered spammy, and so forth.
The article asked: If a site’s e-mails were erroneously scored as spammy, but the site owner protested, is it true that the site wouldn’t be tested again for 60 to 270 days? “That’s correct,” Keats was quoted as saying.
Even paying U.S. $859 per year or more to be tested daily and certified as clean by the “McAfee SECURE” program wouldn’t reduce the probationary period, Keats added, after confirming this with other McAfee executives. “The probationary period is no different for a McAfee SECURE customer or a non–McAfee SECURE customer,” he was quoted as saying.
More precise wording in newly disclosed procedure
After the article appeared, McAfee representatives contacted me, saying the article contained inaccuracies.
In a subsequent telephone interview, Keats said he’d been asked not whether a site with an erroneous rating “wouldn’t be tested again for 60 to 270 days” but whether it “wouldn’t be out of the probationary period for 60 to 270 days.” I believe the original question included the words “tested again,” but that’s not important now. Keats said the article had raised several questions and that McAfee had decided to review and publicly release its policies on scanning and retesting sites.
I’m pleased to report that McAfee has e-mailed me a previously undisclosed set of complaint-resolution policies, and the documents reveal a great deal about the service’s re-evaluation process.
Keats tells me, “This is the first time, at least in some cases, that this information is going out to the public.” The documents I’ve seen are not yet visible at McAfee.com or SiteAdvisor.com. Since I’ve been given them in writing, however, I trust that they’ll be posted by SiteAdvisor soon.
SiteAdvisor is making a public commitment to quickly change to “green” a rating that a Web site owner can prove is in error. The published probationary periods of up to 365 days do not apply before a genuine false positive can be corrected in SiteAdvisor’s ratings.
A site owner who is the victim of a false positive must first file a complaint using SiteAdvisor’s feedback form. After that, according to the documents I’ve received:
- A review begins within “5 business days, usually less,” the documents say.
- If a Web site owner says a downloadable file on the site should not be rated “yellow” or “red,” contrary to SiteAdvisor’s automated scan, the complaint is scheduled for a false-positive evaluation. “We will evaluate within 10 business days to see if we agree/disagree,” McAfee says. If the complaint does not involve a false positive, such as a site that unintentionally hosted a suspicious download but no longer does, “we will evaluate within 5 business days to see if the file is still hosted.”
- If a complaint involves a false positive, a retest/re-evaluation takes 10 business days to complete for all categories of threats.
- If the complaint is determined by SiteAdvisor not to involve a false positive, a retest/re-evaluation requires 60 calendar days of monitoring if a site sent e-mails considered spammy. It takes up to 5 business days for all other categories, such as infected downloads, exploit code, and links to questionable sites.
- If the complaint is confirmed by SiteAdvisor to be a false positive, the situation can be patched in the scanning system (so the site won’t be wrongly flagged again) in “1 business day,” the documents say. In addition, “There is no probation period for confirmed false positives.”
- If a site was not the victim of a false positive, but was able to correct or remove a threat, the site will remain “in probation” and continue to bear a “yellow” or “red” rating. “First-time ‘reds’ will go green in as few as 10 days,” the documents say. “Repeat reds will stay red for as long as 365 days.”
Best of all, McAfee revealed to me a U.S. toll-free number that Web site owners can call to talk with a human being about erroneous ratings. The number — which hasn’t been visible at SiteAdvisor.com but has previously been used by McAfee.com — is 1-866-622-3911. (This number is not accessible or toll-free from every country.)
Keats emphasizes, “We will make it clear to site owners that this is largely a Web-based form and e-mail process.” It’s important that site owners feeling wrongly accused first submit a report via SiteAdvisor’s feedback form. But I believe it’s also essential for any system that’s subject to false positives to provide live human telephone support in special cases.
Information won’t necessarily be up-to-date
How about the example used in the article, in which SiteAdvisor claimed that a site was hosting hyperlinks that actually hadn’t existed there for months?
“We looked at HometownZone,” Keats explained to me in an e-mail after the original article appeared. “We originally rated it in March of 2008. We recrawled it several times subsequently. Most recently, we crawled and rated it again in December of 2008. … We didn’t find any red-rated links, and we didn’t find any heavier green-rated links, so we could continue to show that link analysis for up to a year.”
In a telephone interview, Keats indicated to me that some links could be given greater weight by SiteAdvisor’s crawler and therefore be considered “heavier” than other, newer links.
Hmm, old components of a site could remain in SiteAdvisor’s ratings “for up to a year”? As you recall, the headline on the Feb. 12 article said data might be “1 year out-of-date.” In certain cases, that’s absolutely true, and I stand by the wording of the original article.
Despite criticism, McAfee is proud of its rating service and wants it to be as responsive as it can be. Keats says, “Without equivocation, we can say that McAfee policy is that no site rating is a year old. Many of the sites in our database are tested at least weekly.”
The day the original article appeared, Keats told me that “some sites we test every day, some sites we test several times a day.” When I received McAfee’s newly released documents, he backed away from asserting that SiteAdvisor scans any sites that frequently, saying it was McAfee’s policy to make that claim.
Because SiteAdvisor’s policy statements are certain to change, I’ve posted the company’s original PDF files on dispute resolution and escalating a complaint at WindowsSecrets.com. Readers can judge for themselves whether the older documents gave much hope to site owners who were falsely accused.
In my view, a rating service should reverse within hours or days, not weeks, any negative ratings that are wrong. If a threat really has been removed from a Web site, I’d like to see the site upgraded to “green” but scanned every day, instead of bearing an inaccurate rating for up to 365 days.
But scanning the Web costs money, and SiteAdvisor is rating sites as frequently as it can within its budget. I hope independent test labs can soon give us objective scores for the accuracy of SiteAdvisor and all similar services. (See today’s Known Issues column for alternatives.) SiteAdvisor is a “white-hat” service and I commend it for revealing some of its previously unposted policies.
Brian Livingston is editorial director of WindowsSecrets.com and co-author of Windows Vista Secrets and 10 other books.
