Web crawler
crawler, Crawler, crawlers, на сайте с 12 декабря 2022, 07:42
Software which systematically browses the World Wide Web
Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web.