I design and operate scalable data infrastructure and platforms — with a particular focus on fast, resilient pipelines, clean data models for analytics, and automation that keeps teams shipping confidently. I enjoy mentoring, documenting architecture clearly, and turning complex data flows into repeatable, trustworthy systems.
- 💼 Data Platform Engineer at Federated Engineers
- ✍🏽 I write about data platform engineering on my Substack
- 🌐 Connect with me on LinkedIn
- Architect end-to-end data platforms (ETL/ELT, Cloud Observability, Data Lakes).
- Build reliable orchestration & automation (CI/CD, Fault-tolerance, Alerting).
- Deliver analytics-ready datasets and data products for business stakeholders.
- Lead teams on scalable data engineering patterns, idempotent data pipelines, and production deployment best practices.
🏗️ Building Cloud Data Platforms on AWS with Apache Airflow
- Multi-tenant ETL/ELT flows with automated scheduling
- In-memory transformations using dbt and Apache Spark
- Event-driven architecture via Amazon S3 and AWS Lambda triggers
- Distributed Deployment on Kubernetes with auto-scaling capabilities
🔍 Data Observability & Monitoring
- DBT Tests for Row-level data quality testing
- Notification & alerting via Slack/Teams or Email integrations
- Comprehensive observability with Prometheus & Grafana
- Platform / Production Pipelines — built multi-tenant ETL/ELT flows with low-latency ingestion, in-memory DuckDB matching logic, and a Polars-first transformation layer.
- Monitoring & Validation — Stood up Grafana Observability Dashboard for cost and resource management with threshold alerts and notifications for EKS and Apache Airflow resources.
- Data Governance — Strong focus on data governance and operations principles. Employing software engineering and DevOps principles to standardise development and documentation practices for data engineering teams, leading to improved developer experience and development speed.
(See pinned repos on my profile for code samples and architecture docs.)
- 📖 I enjoy reading and writing about data, systems, and technology.
- ⚽ Big football fan, with side projects in sports analytics.
- 🌍 Passionate about communities, collaboration, and making data useful.