SiteAdvisor ratings may be 1 year out-of-date
By Mark Joseph Edwards The free SiteAdvisor browser add-in claims to protect you by labeling Web sites green, yellow, or red to indicate that they are safe, questionable, or dangerous.
But a good or bad SiteAdvisor rating can persist for as long as a year after the site’s content has changed, raising serious questions about the service’s usefulness.
SiteAdvisor was initially launched as an independent, free service in April 2005 by Massachusetts Institute of Technology developers led by CEO Chris Dixon. The company built software to automatically crawl the Web and find sites containing virus-infected downloads and hyperlinks to suspicious addresses. Security giant McAfee Inc. acquired the company in April 2006, at which point the SiteAdvisor team said it had rated some 2.7 million pages, representing a majority of Web traffic.
Ratings from SiteAdvisor’s browser plug-in and its associated Web site, SiteAdvisor.com, are based on a variety of measures. Besides scanning sites for malware, the service enters customized e-mail addresses into registration forms to see whether this generates spammy e-mails.
The outcomes of these and other tests are used by SiteAdvisor to give a green rating to sites that score well and red ratings to destinations considered dangerous. Browser plug-ins are available for Internet Explorer and Firefox. Besides showing a rating for sites that a user visits, the plug-in also displays its color-coded symbols next to the links that appear in search engines such as Google, Yahoo, and MSN.
Unfortunately, I’ve found that SiteAdvisor’s ratings can persist for as long as one year after a site has been analyzed by its automated Web crawls. If a legitimate Web site falls victim to a false “red” rating, McAfee’s official policy is that months can elapse before a site is evaluated again. Conversely, if bad guys create a clean site that initially wins a green rating, and then immediately start offering infected games or other downloads, it might take SiteAdvisor months to notice.
McAfee certifies for a fee, but it’s no guarantee
At the time of its acquisition of SiteAdvisor, McAfee was widely expected to integrate the service into the corporation’s line of commercial products. McAfee soon announced SiteAdvisor Plus, a $24.99 download that added e-mail checking and other features.
Ratings such as SiteAdvisor’s can be helpful, but according to its own documents, McAfee allows up to 365 days between tests of individual sites, even if a Web site owner protests that a “red” rating is a false positive.
McAfee promotes a paid service to ensure that a site will be scanned for security threats on a daily basis. The site’s owner must pay a fee for “McAfee SECURE certification,” as described at the McAfeeSecure site.
For the smallest sites, SECURE certification costs $859 annually plus a $100 setup fee. If a site gets more than 2,000 page views per day — a tiny number for any serious e-commerce destination — the price rises. McAfee measures traffic by inserting a bit of HTML into the site’s pages.
After a site ponies up the cash, a security audit is performed, according to a description by McAfee. This audit (formerly known as McAfee HackerSafe certification) has long been criticized as permitting critical Web vulnerabilities, as outlined in a recent analysis by security researcher Mike Bailey.
Even paying for and passing SECURE certification, however, doesn’t guarantee that a site with a false rating in SiteAdvisor will get the red flag corrected immediately.
In a telephone interview, McAfee research analyst Shane Keats explained that SECURE certification will fail — even if a site passes all the SECURE security tests — if SiteAdvisor rates the site as “red.” In that case, he said, the site owner must wait for a period of time that’s specified in SiteAdvisor’s Site Rating Escalation Process (a PDF document).
I detail the waiting periods below, but an example will illustrate the procedure. The document says sites that request a re-evaluation are “subject to a rigid aging, or expiration, policy.” Something judged to be a Web exploit may be “aged out” in 30 to 365 days, e-mails that are considered spammy in 60 to 270 days, and so forth.
According to Keats, SiteAdvisor uses SpamAssassin, an automated scoring application, on messages that the service receives after its crawler signs up for a list. If SpamAssassin rated a site’s once-a-day e-mails as spammy, but they weren’t and the site owner protested, is it true that the site wouldn’t be tested again for 60 to 270 days? “That’s correct,” Keats said.
“The retest can happen tomorrow, quote unquote, whether it’s 24 hours or 4 days, for persistent site owners, particularly someone who says this is a inadvertent mistake,” Keats added. “But the probationary period is no different for a McAfee SECURE customer or a non-McAfee SECURE customer.”
McAfee doesn’t say how often the average site is scanned. “We’ve made a public decision not to tell how often we test sites,” Keats said.
The lack of a quick and easy retesting policy is hard to defend. Legitimate Web sites that erroneously receive “red” ratings might try to pay for SECURE certification to clear their names. But they could bear a scarlet letter for months before being rescanned and receiving a “green” SiteAdvisor rating. While waiting, their site couldn’t display the McAfee SECURE logo, because the certification would fail no matter how clean the site actually is.
Meanwhile, sites that initially garner a “green” rating but later go bad have no incentive to pay to be scanned — they can be labeled “good” indefinitely.
Ratings unchanged for 6 weeks, 6 months, or more
I called McAfee’s sales staff, posing as an ordinary Web site owner. My main question was: “If you rate my site green, and tomorrow it gets hacked and a lot of malicious stuff is put on it, how long will it be before you change the rating to red?” The answer I received was “about six weeks.” That’s a long time before a hacked site might be detected. But even that period is not the real story in many cases.
Web site designer Scott Thompson discovered this first-hand after his HometownZone.com site, known as Webster Weather, received its first SiteAdvisor rating in March 2008. At that time, the site justifiably earned a green icon. Six months later, however, Scott completely changed the site, to the extent that only its domain name remained the same.
SiteAdvisor today still shows links that existed only in the old design, according to Thompson. (See Figure 1.) Some of the links SiteAdvisor currently shows as being on the site had been removed even before the redesign.
Figure 1. SiteAdvisor shows that HometownZone.com has several links to sites rated green, but the site removed those links long ago and McAfee hasn’t updated its rating for months.
As of this week, SiteAdvisor still thinks the old links are there. The McAfee service doesn’t currently show the actual links on the site. These, of course, are what SiteAdvisor users assume are being evaluated to determine whether the site deserves a green rating.
Site re-evaluations can be agonizingly slow
According to McAfee’s Site Rating Escalation Process, SiteAdvisor “ages” its scanning criteria at the following intervals for individual sites that request a re-evaluation:
- Annoyances: every 10 to 270 days
- Downloads: every 10 to 365 days
- E-commerce: every 30 to 365 days
- E-mail: every 60 to 270 days
- Exploits: every 30 to 365 days
- Links: every 10 to 270 days
Unfortunately for legitimate Web site owners, SiteAdvisor is subject to criticism for false positives and unwarranted red flags, according to an analysis by The Register’s John Leyden.
Meanwhile, any bad guy on the planet can game the system by getting a green rating for a clean site and then changing the site into a vector for attacks. SiteAdvisor may display a green rating for months, leading its users to think the site is safe. At that point, it’s “game over,” and you lose.
Web surfers should consider alternatives such as Web of Trust (MyWot.com). This plug-in updates its ratings more quickly than SiteAdvisor, according to an interview with CEO Esa Suurio and several forum commentors, and incorporates feedback from a large community of users.
| UPDATE 2009-02-19: McAfee representatives have responded to the above article and released previously undisclosed documents that reveal SiteAdvisor’s timetable for scanning and retesting Web sites. See our Feb. 19 follow-up for details. |
Mark Joseph Edwards is a senior contributing editor of Windows IT Pro Magazine and regularly writes for its Security Matters blog. He’s a network engineer, freelance writer, and the author of Internet Security with Windows NT. Editorial director Brian Livingston contributed research assistance and interviews to this article.
