Google and Wikipedia, Revisited

2007.10.20   prev     next

There was a lot of buzz about a year or two ago that Wikipedia was going to do Google in. Juicy headlines like these led the way: “Wikipedia Eats Google,” “The Wikipedia Threat To Google’s Empire,” and “Wikipedia 3.0: The End of Google?” But read these articles carefully, and you find no realistic reason to think that Google is threatened by Wikipedia.

And when I say “no realistic reason” I mean exactly that — the “Wikipedia 3.0” article seriously claims that all websites will contain some sort of very advanced AI that will tell the search engine of the future (which for no particular reason will not be Google) when to return a page in a search result and when not to. Riggghht. We can’t even convince every web author to use simple metadata tags (I myself don’t use them for anything but telling iPhone how many pixels wide my blog is), and when we do get people to use them, we often wish we hadn’t.

The only relevant fact mentioned by these Wikipedia-over-Google articles is that a lot of Wikipedia pages are showing up as high-ranked Google search results. But what’s wrong with that? As I have described in this blog, Google’s inclusion of Wikipedia is a plus for Google. When you Google a topic, you get a prioritized list of the best the internet has to offer, which often (but not always) includes a relevant Wikipedia page. So what’s to fear at Google?

The whole Wikipedia-as-a-threat-to-Google concept has gained traction due to this kind of thinking: “If I Google a topic, and I get a link to a Wikipedia page, why shouldn’t I just go to Wikipedia first? Google is becoming obsolete! (Quick — write an article about that before everyone else does!)” The fallacy in that logic is, of course, that you don’t always get a link to Wikipedia (especially when your search is more specific than a typical hardback encyclopedia article title), and even when you do, you also get many links to excellent material that you won’t find on Wikipedia.

Suppose I walk into a public library full of books, and I ask the librarian, “Where can I find general information about snakes?”

And she says, “I suggest you start by looking up snakes in the encyclopedia over there on that shelf, then there are some other places where you can find whole books about snakes.”

As I read this nice encyclopedia article about snakes, I start thinking, hey, who needs a library or a librarian? I’ll just look in this encyclopedia from now on! But that would be silly, of course. The encyclopedia does serve a purpose in the library, but it’s only a tiny subset of the library’s content. The goal of the library is to cover everything — including someone else’s attempts to cover everything in .001 as much shelf space.

Even assuming the best writing and the most sensible dispute-resolution techniques (a huge assumption), an information source that limits itself to newspaper-style articles, about subjects that can be uniquely identified by short, general titles, can never contain but a tiny fraction of the useful information on the internet. And Wikipedia may be fast filling up that tiny fraction — witness this (very recent) SlashDot article, Has Wikipedia Peaked?, that reports Wikipedia content growth to be slowing. It asks, “Are Wikipedians simply running out of things to write about, or is the community collapsing under the weight of external vandalism and internal conflicts?”

Now, I don’t think Wikipedia is, in any sense of the word, “collapsing.” But, like an unfinished encyclopedia nearing completion, Wikipedia is nearing the limits of what it can provide in its chosen format. Hardback encyclopedias like World Book and Britannica have been around for many decades, but they did not grow larger and larger and take over the whole library, even though there was plenty of space in the library for them to do so. They reached a certain size and then they just experienced periodic updates, gaining new general-topic information as history progressed, and losing older, irrelevant material as it became out-of-date. (And have no doubt: There are many, many participants in Wikipedia who delete material the minute it can be plausibly declared out-of-date or irrelevant!)

Wikipedia serves a very specific function on the internet — the same function a hardback encyclopedia serves in a library. The library (the internet), and its librarian (Google), are not threatened by the encyclopedia in the slightest. They welcome it as good material for one of their many, many shelves.

Update 2008.03.10 — Check out this Slashdot link, “The Battle For Wikipedia’s Soul”