Scrapy is a fast high-level screen scrapingand web crawling framework, used to crawl websites and extractstructured data from their pages. It can be used for a wide rangeof purposes, from data mining to monitoring and automated testing.

For more information including a list of features check theScrapy homepage at: http://scrapy.org