Data engineer. I build the pipelines that turn messy, hostile, rate-limited sources into clean data you can trust — and the automation that keeps them running without me.

focus      large-scale scraping · ETL/ELT pipelines · workflow automation · data infra
depth      I read the network tab before the docs, and the query plan before I blame the DB
philosophy correct data first, fresh data second, clever data last
currently  making pipelines that self-heal instead of paging me at 3am

• Scraping at scale that survives contact with the real web — rotating proxies, session/cookie management, TLS & header fingerprinting, adaptive rate-limiting, and retry/backoff that respects the target instead of getting me banned. Playwright when the JS fights back, plain httpx/aiohttp when it doesn't.
• Pipelines that own their state — idempotent loads, dedup on natural keys, incremental/CDC ingestion, and schema-drift handling so an upstream column rename doesn't silently poison the warehouse.
• Automation that removes me from the loop — scheduled jobs, event-driven triggers, alerting that fires on data anomalies (row-count drops, freshness SLAs) not just crashes.
• Warehouses that answer questions fast — partitioning, clustering, and EXPLAIN-driven query tuning instead of throwing more compute at it.

Languages

Scraping & extraction

Data & pipelines

Storage & infra

• Backpressure over buffering. An unbounded scrape queue is a memory leak with good PR.
• Idempotency is non-negotiable. At-least-once delivery means the consumer, not the source, decides "did this row already land."
• The query planner isn't your friend until you read its EXPLAIN. N+1s and missing indexes hide behind ORMs that "just work."
• Freshness is a first-class metric. Data that's correct but 6 hours stale is a bug nobody wrote a test for.
• Determinism is a feature. If a pipeline run can't be replayed, it can't be debugged.

backlog = [
    "Designing Data-Intensive Applications — re-read #3, still finding new things",
    "dbt incremental models + snapshots done properly",
    "async scraping throughput: httpx + asyncio semaphores vs a task queue",
    "columnar formats — why Parquet beats CSV everywhere that matters",
]

_{📧 [email protected]  ·  Open a PR if I'm wrong — I'd rather be corrected than confident.}