It's a little bit of everything, but the main contributor is the financialization of the Internet. Ads went from reasonable banners to ever more egregious and disruptive formats, which led to ad blockers. Then efforts went towards ways to embed ads in ways that couldn't be blocked or paywalls to keep revenue flowing, which hurts the value of the content. At the same time many companies started viewing the Internet as purely a sales and marketing platform, anything that didn't drive revenue was a cost center and eliminated. At the same time search engines were slowly getting worse due to increased ad presence and polluted results with seo garbage sites that aren't actually relevant. Everyone has slowly cannibalized their content to a point where the value isn't there anymore for actual users.
Things like AI and bot nets were more the final nail in the coffin. Another large contributor was the death of flash, which destroyed a large portion of the Internet at the time.