I make slow queries fast.
And messy pipelines boring — the good kind of boring. A data engineer with one superpower: database internals and performance — proven across six years in financial services, utilities, and telecom, most recently on AML transaction monitoring pipelines at TD Bank.
Python · SQL · PySpark · dbt · Airflow · Snowflake · DuckDB · Azure · AWS
Trusted with production data at TD Bank · Exelon · Audi · Volkswagen · Bank of Ireland
// six years, drawn as a DAG — obviously. click any node · hover to trace the flow
Slipstream: Legacy Teradata → Modern Stack
A production ELT pipeline I originally led at Teradata, rebuilt end-to-end on the modern data stack — same data, same business logic, every line rewritten and re-architected by hand.
Why this project: most portfolio pipelines run happy-path demos on toy data. This one migrates a real production system I built and operated — including the unglamorous parts: idempotent re-runs, file archiving, Windows path resolution, silent validation failures, and column-name mismatches that only surface on real files. Every bug is documented with its root cause and fix. Ingestion layer is complete and verified; the dbt transformation layer (replacing the original StgToTrn stored procedure) is the current milestone.
Projects
Individual repos linked per project · browse the main projects repo ↗
Uber ETL Automation — Airflow + Mage
Automated ETL over Uber trip data — ingestion, dimensional transformation, and analytics-ready loading, orchestrated end to end with Airflow and Mage. Demonstrates the orchestration patterns (DAG design, task dependencies, scheduling) that production pipelines live on.
Azure Olympic Data Engineering Pipeline
End-to-end Azure data pipeline over Olympic datasets — cloud ingestion, Spark transformations on Databricks, and an analytics serving layer. A compact demonstration of the same Azure patterns used at Exelon on 15TB+/day production workloads.
Real-Time Financial Data Streaming Pipeline
Real-time transaction streaming in active development: Kafka producers → AWS-hosted brokers → Snowpipe continuous ingestion → Snowflake Streams & Tasks for incremental transformation — with a Claude API multi-agent layer for automated orchestration. The next flagship alongside Slipstream.
// more in the repo — every link is real code
Experience
- Architected SQL pipelines integrating 15+ source systems powering AML transaction monitoring, regulatory reporting, and financial crime analytics
- Led analytics infrastructure migration to AWS — S3 data lake architecture, RDS, PySpark transformations, Airflow orchestration
- Engineered 30+ automated data quality validation frameworks — reducing data incidents by 40% and ensuring SOX compliance
- Automated reconciliation workflows saving 40+ hours/month via Python, Bash, and GitHub Actions CI/CD
- Designed Power BI and Tableau dashboards tracking 20+ financial KPIs and AML metrics for compliance leadership
- Built end-to-end pipelines with Snowflake and dbt — dimensional models reducing data prep time by 30%
- Optimized SQL queries and schema design, improving processing performance by 25%
- Integrated pipelines with Django backend via REST APIs and MongoDB for real-time product features
- Built PySpark pipelines on Azure Databricks processing 15+ TB daily with 95% SLA reliability
- Reduced pipeline execution time by 30% through bottleneck identification and optimization
- Engineered Azure Data Factory workflows reducing data latency by 45% in on-prem to cloud migration
- Led SLIPSTREAM — automated ELT pipeline processing 10M+ telecom subscriber records daily, improving revenue forecasting accuracy by 25%
- Architected 10+ enterprise ETL/ELT pipelines on Teradata Vantage, Talend, and SQL Server — 2.5TB daily at 95% reliability
- Optimized 100+ complex SQL queries, transforming response times from minutes to seconds
- Built 40+ Python/Bash automation scripts reducing mean time to resolution from hours to minutes
Certifications
All badges verifiable on Credly.
