The best of the Internet. Dedicated to the most funny, interesting, and cool stuff on the Internet today.
-
funnyinterestingcool
- Posts: 587
- Joined: Tue Jul 09, 2019 11:54 pm
-
Contact:
Post
by funnyinterestingcool » Wed Oct 25, 2023 6:52 pm
Cralwer that's not respecting robots.txt rules and using AmazonAWS services to crawl websites and mask (or re-route) their origin IPs. Their crawling rates are extremely high.
