spidy Web Crawler

The simple, easy to use command line web crawler. Spidy (/spˈɪdi/) is the simple, easy to use command line web crawler. Given a list of web links, it uses Python requests to query the webpages, and lxml to extract all links from the page.