September 2nd, 2008
Is Cuil killing the Internet?
While the world gets excited over Google’s new open-source browser, Chrome, a new story is emerging about one of the Web’s next-generation utilities: Cuil, the search engine heralded as a “Google killer,” might just be taking out the entire Internet, one site at a time.
As TechCrunch’s Don Reisinger reports, Cuil’s Twiceler indexing bot is so bandwith-hungry that it’s shutting down sites all over the Web:
“I don’t know what spawned it, but when Cuil attempts to index a site, it does so by completely hammering it with traffic,” the tipster wrote. “So much, that it completely brings the site down. We’re 24 hours into this “index” of the site, and I’ve had to restrict traffic to the site down to 2 packets per second, while discarding the rest, or otherwise it makes the site unusable.”
Meanwhile, the forums at The Admin Zone are raging over Cuil’s indexing method, and several posters are complaining that their websites have been brought down because of the Twiceler robot. Just one example: It “leeched enormous amounts of bandwidth — nearly 2GB this month until it was blocked. It visited nearly 70,000 times!”
The reason? Some say the bot seems to randomly hit a site, guessing and generating URLs in an attempt to find pages that aren’t accessible by links. In effect, the process renders sites unusable.
What do you think about Cuil? Cool, or catastrophe? Tell us in TalkBack.
Andrew J. Nusca is an associate editor for ZDNet and SmartPlanet.
See his full profile and disclosure of his industry affiliations.
Email Andrew Nusca 
Follow on Twitter
Subscribe to The ToyBox via Email alerts or RSS.
