scrapy-cluster
на сайте с December 16, 2022 12:40
Uses Redis and Kafka to create a distributed on demand scraping cluster.