The following gist is an extract of the article Building a simple crawler. It allows crawling from a URL and for a given number of bounce.
from crawler import Crawler
crawler = Crawler()
crawler.crawl('http://techcrunch.com/')
rkmunusamy
/ crawler.md
Created
December 11, 2017 12:17
— forked from typehorror/crawler.md
Simple Website Crawler (in python)