#data pipeline

Crawl a website into BigQuery or Snowflake on a schedule

A scheduled website crawler that turns pages into rows and MERGEs them into BigQuery or Snowflake, loading only what changed and never mistaking a blocked page for a deleted one.

2 Nov 2026 · 8 min read