S01E08 - Scrapy, Python, web data and mushroom soup.

In this episode, John, Neha, and Ayan trade origin stories (requests and Beautiful Soup, a runaway crawl at IBM back in 2016, a wall of JavaScript that broke everything) and get into why a lean, extensible core is exactly what modern data extraction needs. They cover the anti-bot arms race, reverse-engineering back-end APIs, the recent httpx download handler and the move toward an async reactor, and the four plugins John reaches for on every production build.