this post was submitted on 24 Jun 2023
49 points (100.0% liked)
Reddit Migration
104 readers
2 users here now
### About Community Tracking and helping #redditmigration to Kbin and the Fediverse. Say hello to the decentralized and open future. To see latest reeddit blackout info, see here: https://reddark.untone.uk/
founded 2 years ago
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
One of the reasons they are doing this is because of the large language models being implemented. These companies are using Reddit to train the models. The reason is because of the voting on replies. Where else can you get millions of questions being answered with actual humans saying how good a response is?
The big boys in the current AI space will definitely pay for the API. They'll likely pay a lot for it as well.
Why pay the bloated and gouging costs for API access when you can just write a web parser and scrape the site the old fashioned way?
Scrapers can easily be disabled. Reddit won't look the same obviously. But this isn't a real obstacle.
then the scrapers start using residential proxy botnets
Then you just force them to change the syntax repeatedly and scraping will break with regular occurrence. Scraping is extremely fragile and not easily adaptable without human effort which costs money.