Google Now Tracks 1 Trillion URLs
By Anna Johnson on July 28th, 2008Just how many websites does Google track? Apparently, it’s more than 1 trillion unique URLs.
But just to be clear, although Google has identified 1 trillion URLs… it hasn’t indexed all those URLs. Google says this is simply because many of those sites are duplicates or based on auto-generated content. Another reason is probably the sheer expense of indexing too many sites, especially on top of the cost of indexing news sites, blogs and other sites which change every few minutes.
According to Tech Crunch, the true size of Google’s index is probably around 40 billion pages. That’s still pretty big, and quite a jump from the 26 million pages it indexed in 1998, and the 1 billion or so in 2000.
Sources: Jesse Alpert & Nissan Hajaj, “We Knew The Web Was Big”, Google Blog, July 25, 2008, Michael Arrington, “Google’s Misleading Blog Post: The Size Of The Web And The Size Of Their Index Are Very Different”, Tech Crunch, July 25, 2008, Frederic Lardinois, “Google Now Knows About 1 Trillion Pages”, Read Write Web, July 25, 2008
