Apache Nutch 2.0 indexes at web scale

The Apache Nutch developers have announced that version 2.0 of the network crawling and indexing search framework is now available. Built on top of other Apache projects including Solr, Tika, Hadoop and Gora, Nutch has been designed to crawl "at web scale" to allow organisations to create searchable indexes of their web-published content. Nutch adds web-specific functionality to Solr with a link-graph database and uses Tika to parse web pages and a number of other document formats.

Complete Story

Related Stories:

• IT Manager's Journal: Why the Future of Search May Be Open Source(Jan 26, 2004)
• SearchEngineJournal: Open Source Search Engines Give Non-Commercial Alternatives(Apr 26, 2004)
• Crawling in Open Source, Part 1(Feb 20, 2011)
• SearchEngineWatch: An Open Source Search Engine(Sep 12, 2003)