and also that they are horribly designed and will slam you with several thousand requests in the span of a minute
anyway a hearty "fuck off" to amazonbot for giving us a brief downtime event1 while everything scaled up. enjoy being blocked
how the hell does this even happen. why dont they just use internet-wide breadth-first search or something so the next request is always on a different site or something. are they stupid?
If you click through the link Jae left, you get taken to a question and answer page about AmazonBot. One guy said his site had a bug where it was appending garbage to the url that wasn’t actually changing the page at all, and every new request from AmazonBot would create a group of these connecting to the various pages on his site with the same garbage string, thus trapping AmazonBot in an endless loop on his site. It wouldn’t ever notice it wasn’t getting any new content. Literally who the fuck coded this piece of shit?
