Google and Wikipedia, Revisited
THERE was a lot of buzz about a year or two ago that Wikipedia was going to do Google in. Juicy headlines like these led the way: “Wikipedia Eats Google,” “The Wikipedia Threat To Google’s Empire,” and “Wikipedia 3.0: The End of Google?” But read these articles carefully, and you find no realistic reason to think that Google is threatened by Wikipedia.
And when I say “no realistic reason” I mean exactly that — the “Wikipedia 3.0” article seriously claims that all websites will contain some sort of very advanced AI that will tell the search engine of the future (which for no particular reason will not be Google) when to return a page in a search result and when not to. Riggghht. We can’t even convince every web author to use simple metadata tags (I myself don’t use them for anything but telling iPhone how many pixels wide my blog is), and when we do get people to use them, we often wish we hadn’t.
The only relevant fact mentioned by these Wikipedia-over-Google articles is that a lot of Wikipedia pages are showing up as high-ranked Google search results. But what’s wrong with that? As I have described in this blog, Google’s inclusion of Wikipedia is a plus for Google. When you Google a topic, you get a prioritized list of the best the internet has to offer, which often (but not always) includes a relevant Wikipedia page. So what’s to fear at Google?
The whole Wikipedia-as-a-threat-to-Google concept has gained traction due to this kind of thinking: “If I Google a topic, and I get a link to a Wikipedia page, why shouldn’t I just go to Wikipedia first? Google is becoming obsolete! (Quick — write an article about that before everyone else does!)” The fallacy in that logic is, of course, that you don’t always get a link to Wikipedia (especially when your search is more specific than a typical hardback encyclopedia article title), and even when you do, you also get many links to excellent material that you won’t find on Wikipedia.
Suppose I walk into a public library full of books, and I ask the librarian, “Where can I find general information about snakes?”
And she says, “I suggest you start by looking up snakes in the encyclopedia over there on that shelf, then there are some other places where you can find whole books about snakes.”
As I read this nice encyclopedia article about snakes, I start thinking, hey, who needs a library or a librarian? I’ll just look in this encyclopedia from now on! But that would be silly, of course. The encyclopedia does serve a purpose in the library, but it’s only a tiny subset of the library’s content. The goal of the library is to cover everything — including someone else’s attempts to cover everything in .001 as much shelf space.
Even assuming the best writing and the most sensible dispute-resolution techniques (a huge assumption), an information source that limits itself to newspaper-style articles, about subjects that can be uniquely identified by short, general titles, can never contain but a tiny fraction of the useful information on the internet. And Wikipedia may be fast filling up that tiny fraction — witness this (very recent) SlashDot article, Has Wikipedia Peaked?, that reports Wikipedia content growth to be slowing. It asks, “Are Wikipedians simply running out of things to write about, or is the community collapsing under the weight of external vandalism and internal conflicts?”
Now, I don’t think Wikipedia is, in any sense of the word, “collapsing.” But, like an unfinished encyclopedia nearing completion, Wikipedia is nearing the limits of what it can provide in its chosen format. Hardback encyclopedias like World Book and Britannica have been around for many decades, but they did not grow larger and larger and take over the whole library, even though there was plenty of space in the library for them to do so. They reached a certain size and then they just experienced periodic updates, gaining new general-topic information as history progressed, and losing older, irrelevant material as it became out-of-date. (And have no doubt: There are many, many participants in Wikipedia who delete material the minute it can be plausibly declared out-of-date or irrelevant!)
Wikipedia serves a very specific function on the internet — the same function a hardback encyclopedia serves in a library. The library (the internet), and its librarian (Google), are not threatened by the encyclopedia in the slightest. They welcome it as good material for one of their many, many shelves.
Update 2008.03.10 — Check out this Slashdot link, “The Battle For Wikipedia’s Soul”
Update 2009.08.14 — More about Wikipedia slowing down.
Update 2009.08.24 — Wikipedia now requiring editors’ approval to edit any article about a living person.
Update 2013.04.27 — New York Times article on how someone is slowly moving all female authors out of Wikipedia’s “American Novelists” category, into an “American Women Novelists” category — so you won’t even see them if you innocently peruse “American Novelists.”
Update 2013.05.17 — Salon article on how a Wikipedia editor made numerous “revenge edits” against the bios of people he personally didn’t like, and went undiscovered for months if not years.
Update 2013.10.22 — In an MIT Technology Review article on “The Decline of Wikipedia” Tom Simonite says, “The loose collective running the site today, estimated to be 90 percent male, operates a crushing bureaucracy with an often abrasive atmosphere that deters newcomers who might increase participation in Wikipedia and broaden its coverage.”
Update 2015.01.01 — When visiting Wikipedia, you are now frequently assaulted with a huge, pop-over ad, in which Wikipedia itself begs for money.
Update 2015.05.26 — As the general election approached, “Wikipedia pages of dozens of UK politicians had references to sex scandals, fraud and opposition to same sex marriage removed ... The edits were made using computers with IP addresses registered from inside Parliament.”
Update 2015.11.12 — Wikipedia founder Jimmy Wales says that persistent declines in Wikipedia visits from Google search results are a “long-term issue.”
Update 2016.04.25 — Study finds that Wikipedia has degenerated from its ideals into something functionally identical to a corporate bureaucracy.
Update 2019.03.14 — Wikipedia’s pages about NBC, Axios, and Facebook were reportedly “whitewashed” of any unflattering content.
Update 2019.05.31 — The North Face quietly gamed Wikipedia to successfully improve its Google search standings.
Update 2019.10.28 — Wikipedia’s Windows Phone page (and all the rest of Wikipedia, apparently) contains no mention whatever of Microsoft’s widely publicized iPhone-effigy funeral parade — like it never even happened. But do a ten-second Google search, and ta-da, there it is, plain as day. Further, the Wikipedia page barely mentions iPhone at all, twice, starting about 2/3 of the way through the lengthy article.
This, more than anything, is why Wikipedia will never replace, or even eclipse, the web, and the ability to fairly search it (i.e. Google, for the foreseeable future). Until/unless Wikipedia finds a way to allow proponents of more than one side of an emotionally charged topic to present their case at all, then it will never be anything more than a big collection of one-sided pages in the grander scheme of the web, and the web’s de facto custodian, Google.