Yeah, I’m real torn. On one hand, I immediately want to scrape this site, but I also don’t want to beat the site up tying up their bandwidth. There seems to be a parent site db4p.org thats managing mirrors of this site, but I don’t see any sort of torrent or archive. If there’s something like that, I’d be very inclined to just archive the entire site/database.
Mmm… such a bot could run once every 24 hours either “visiting the site” and reading the HTML contents. Or using the DB directly if they have an API somewhere.
Yeah, I’m real torn. On one hand, I immediately want to scrape this site, but I also don’t want to beat the site up tying up their bandwidth. There seems to be a parent site db4p.org thats managing mirrors of this site, but I don’t see any sort of torrent or archive. If there’s something like that, I’d be very inclined to just archive the entire site/database.
Mmm… such a bot could run once every 24 hours either “visiting the site” and reading the HTML contents. Or using the DB directly if they have an API somewhere.
Either way it doesn’t cost them much.