Readers say Google is losing its relevance

By Brian Livingston

I reported in eWeek on Feb. 17 that my readers and I had found numerous specific examples of the Google.com search engine failing to provide in its first 10 results good links for fairly straight-forward technical searches. The apparent decline in relevance is a very fitting subject for a follow-up story in today’s Brian’s Buzz, even though Google of course isn’t a Windows program. Google is used by so many people in the information technology biz to research PC problems that the search engine sometimes seems like an ever-present Windows utility.

I received far more reader responses than I expected after my column, entitled “Google grumbles,” appeared in eWeek. Today, I bring you a wide selection of readers’ experiences on this surprisingly emotional subject.

Subscribe to our Windows Secrets Newsletter - It's Free!

Get our unique weekly Newsletter with tips and techniques, how to's and critical updates on Windows 7, Windows 8, Windows XP, Firefox, Internet Explorer, Google, etc. Join our 480,000 subscribers!

PC Drive Maintenance (Excerpt)

Subscribe and get our monthly bonuses - free!

Your hard drives store photos, books, music and film libraries, letters, financial documents and so on. This ebook is aimed at helping you understand your hard drives, expand their capacities and length of life, and recover what you can from them when they fail. We're offering you a FREE Excerpt! Get this excerpt and other 4 bonuses if you subscribe FREE now!



In my column, I used as illustrations Google’s results on two different searches. These queries were designed to find information on (1) deleting Mac files that had been copied to Windows but couldn’t be removed due to illegal characters and (2) setting the default for Windows XP file-open dialog boxes to the Details view, respectively:

    windows delete mac files illegal names

    xp open dialog default details
Google had little of relevance on these topics in its first 10 results. Clicking a tab at Google to switch to Google Groups (an archive of Usenet posts), however, resulted in 10 links that were far better. And a competitor, Teoma.com, which now powers the Ask.com service, also had very pertinent results.

Google’s director of search quality, Peter Norvig, told me by way of explanation, “These are the types of questions that have always been best answered on Google Groups — there are more good technical answers there and fewer unrelated pages. I think this has always been the case and is not due to any recent changes to Google or the Web.”

My readers took both sides, some of them disagreeing with Norvig, others defending his search firm’s pre-eminence.

It’s been a long, slow slide
Reader Jean Ichbiah was typical of those who wrote me to say that they’d noticed a decline in Google’s relevance for some time:

  • “Your article on Google, while raising the issue, severely underestimated the huge degradation that has taken place in the past two years. There is a race between Google and companies who advertise their products, and in this race Google has been losing big, to the degree where it is now largely ineffective.

    “Here’s an example. Say you want to buy a DVD recorder and are looking for comparative reviews to fine-tune your choice. So you do an advanced search and use the phrase ‘DVD recorder’ and the word reviews. You get 256,000 results from stores selling you DVD recorders. In a nutshell, the merchants have learned the tricks of getting positioned to the degree where the service is now totally ineffective.

    “I did the test with Teoma.com and the results are more precise and useful: they are not as popular and merchants have not yet have the time to develop good positioning techniques. In essence, they are two years behind Google [in being gamed].

    “It appears that it is a battle that Google can only lose, given that they are outnumbered. On the other hand, when the word spreads, their IPO is not likely to be as glorious as could have been anticipated.”

Losing whole sites due to filtering
A similar perspective was provided by Simon Tam, the publisher of a blog site that once had high rankings but suddenly disappeared entirely from the index:

  • “I was so glad to see your Feb. 16 column on Google in eWeek. Let me share with you my personal experience with Google. The keyword is ‘personal’ because my own Web pages are involved.
    “You see, I publish a pop-culture blog on Pyra Labs’ Blogspot.com site on the side. So every month for the last year I’d type in a series of searches to check the Google rankings of some of my pages. Everything was fine until last September, when some of my previously top-ranking pages suddenly disappeared from Google’s listings. The irony is Google is the parent company of Pyra Labs.

    “The response I got from Google was the standard corporate denial: they claimed I was simply experiencing the normal fluctuations of rankings (I paraphrase). It’s one thing if my pages’ rankings fell, but when they became ‘invisible’ to Google, there’s a problem.

    “That’s when I noticed Google has added a filtering option. (Digression: I think Google shouldn’t be doing filtering at all — that’s what Cyber Patrol is for! But since the company is going public, I can imagine its legal team recommending a filtering option.) And the one thing my ‘missing’ pages had in common is some objectionable word in the form of a song title or artist name. The default filter setting is set to block explicit images only, but I went ahead and edited my pages to remove the offending words (yes, I censored myself). Sure enough, these pages are now visible to Google again. Hmmm….

    “So it seems it’s not just technical terms Google has problems with. I’ve been meaning to try Teoma and AllTheWeb; I’ll definitely do it after reading your column. I’ve never looked at Google Groups before. But your column only confirms what I tried to tell Google — it’s been going downhill since last fall.”

A rose by any other index would not smell as sweet
Nancy Kramer operates a photographic site, DigitalPhotoArtistry.com, which has given her the occasion to check regularly what Google is linking to:

  • “Google has become totally illogical in its results. Many spammy and very mediocre sites are in the top ten. Also, sites that have not been updated for years are often number 1. Put the query orchid pictures into Google and you’ll get a site that states it has not been updated since 1997 as the number 1 site. It is on free space and uses someone else’s pictures and their bandwidth for the pictures. The person doing that site has basically copied someone else’s content. The site that was copied is not even listed. Go figure. What I really need a search engine for is fresh content. Stuff that is unchanged since 1997 should be in people’s bookmarks. …

    “I have a site that is in the top ten for red rose pictures but does not even appear anywhere I can find for rose pictures. It has whole sections with nothing but photos of roses, but Google considers that spam, I guess, because I call them rose pictures. … The thing that bothers me most about this, since I am a programmer, is that this is totally illogical.”

Other search engines start to appeal to users again
Several readers volunteered the names of search engines other than Google that they now use more of the time or exclusively. The leading candidates were AllTheWeb and Dogpile.

Here’s what I heard from a blogger named Sal from Saltation.Blogspot.com:

  • “I read your article just now re Google’s result-relevance problems. I agree completely, and have noticed the problem growing over the last couple of years. This is a great shame because, for all its faults, I still found Google’s results better than earlier engines such as Metacrawler or AltaVista.

    “I’ve found a search engine which returns startlingly relevant results even to the most cack-handedly expressed queries: AllTheWeb.com. It’s a relatively recent Norwegian startup, and the results are spookily useful.

    “For example, your search for sites advising how to delete Win files with illegal Mac characters. Look at the first handful of sites returned by the same query in AllTheWeb:

    http://www.alltheweb.com/search?avkw=
    fogg&cat=web&cs=utf-8&q=windows+delete+
    mac+files+illegal+names&_sb_lang=pref


    “A little more apposite, no?

    “Alas, I have no financial interest in them. But I do like to see credit where credit is due. For me, this engine has replaced Google for any nontrivial search.”

Ron Rubel is a fan, by comparison, of a meta search engine, one that includes Google’s links in its results:

  • “I used to use Google quite a bit, but found the same problem you just described. Since then I have been using Dogpile. It gives answers by relevance to the question and even gives choices as to which search engine you may want to check out. I have just plain forgotten about Google, as Dogpile will include any of their results in my search.”

Brian, you used the wrong search string, you idiot
On the other hand, several readers wrote me to say that I hadn’t chosen the right search examples or should have tweaked them more carefully.

This was true of Bob Bailin, who provided the following:

  • “It really helps to use phrases when searching Google, rather than just a list of common words.

    “In your eWeek column, 2nd example, where you were looking to change Windows XP’s File Open dialog boxes to default to the Details View, you searched for:

      xp open dialog default details

    and got no useful results. I searched for:

      xp “file open” “details view” default

    (note the quotes) and got relevant answers among the top 10.

    “In other words, if you want to look for something about the File Open command, look for “file open” and you’ll automatically eliminate all those matches dealing with tripping over open file cabinet drawers.

Will Manzer re-organized the same query in a different way:

  • “With the examples given, it would be hard for the engine to sort out exactly what you were looking for. I was able to customize the query slightly, and it returned very relevant results. My query was:

      “open dialog” default “details view” “Windows XP”

    “Using the quotes helps greatly. Granted, the Teoma site may figure the relationships out better, but it may also miss an important article regarding Office XP. What do you think?”

Finally, Kenneth C. Sherwood used different searches also. Please note that his technique is to search Google using hyphens between word-pairs rather than putting quote marks around them (Google ignores punctuation but treats hyphenated words as a phrase):
  • “In my opinion, your ‘problem’ with Google lies in your apparent lack of skill in properly — or appropriately — choosing search words.

    “For your first example, I entered windows illegal-characters file-names fix. My solution was found within Google’s second major hit, at AB-archives.com [scroll down to MacNames, a shareware program —BL].

    “For your second example, I entered xp how-to-set default-view details. My solution was found within Google’s first hit, at Annoyances.org.

    “Appropriately choosing key words is a skill that must be developed through exprience, particularly experience with Google and its ‘ways.’ Too many key words can be as faulty as too few, for example. In your first example, ‘delete’ and ‘Mac’ don’t belong in a proper search … they may not be needed in text which will resolve your problem. In your second example, simple use of Google’s syntax rules (using hyphens or quotes to group words into phrases) is in order.

    “Friendly tip: Instead of clicking on Google’s link for the site you want to check, try clicking on the word “cached” within that listing. Your page shows up with your key words highlighted, easing your task considerably.”

All of this raises the question: If we now must re-arrange and tweak our searches to get satisfactory results from Google — but other search engines produce good results without tweaking — which is the better search engine? And what’s actually happened to Google?

Google still has far more features than its competitors, including a vast resource of “Cached” links, a dynamic spelling checker, and so forth. But the grumbles are getting louder and the natives are getting restless.

To send me more information about this, or to send me a tip on any other subject, visit WindowsSecrets.com/contact. You’ll receive a gift certificate for a book, CD, or DVD of your choice if you send me a comment that I print.

= Paid content

All Windows Secrets articles posted on 2004-02-26: