Back homeBlog
Notes from the workshop
Practical writing on web scraping, data pipelines, and shipping software. No fluff, just what we have learned building this for clients.
Data EngineeringJune 28, 20266 min read
What a real ETL pipeline costs (and why)
Asking what an ETL pipeline costs is a bit like asking what a house costs. Here's what actually moves the number, so you can tell a fair quote from a cheap one that bites you later.
Read postWeb ScrapingJune 24, 20265 min read
How we scrape large sites at scale without getting blocked
Pulling a hundred rows off a website is a weekend project. Pulling a few million every week, reliably, is a different sport. Here's what actually keeps a large scraper alive.
Read postData EngineeringJune 12, 20265 min read
Web scraping or an official API? How to actually decide
Most teams reach for scraping when an API would have saved them weeks, or pay for an API that doesn't hold the data they need. Here's the decision we walk every client through.
Read post