Best Bowler Award – MK AIR Cricket Club 2025 Season
Honored to receive the Best Bowler trophy for MK AIR Cricket Club 2025 season. From data engineer by day to cricket bowler by weekend!
Technical Blog
Deep dives into data engineering, cloud architecture, AI/ML, and the technologies that power modern data platforms.
Honored to receive the Best Bowler trophy for MK AIR Cricket Club 2025 season. From data engineer by day to cricket bowler by weekend!
Recognized by Senior Data Engineering Manager for close-to-customer mindset and driving business improvements during induction
Grateful to Gregory Sigaud for empowering me to build tools that transform how teams work. My journey from automation advocate to innovation builder.
It’s Not Goodbye, It’s See You Later – My Last Day at Sky
Received an award of £100 Amazon voucher! - Sky Star Award Feb 2025
Experimenting with Node.js - Part 01
Experimenting with Node.js - Part 02
Experimenting with Node.js - Sending Email using the html template
Complete comparison of GCS bucket synchronization methods in Airflow. Covers BashOperator with gcloud rsync, GCSSynchronizeBucketsOperator, GCSToGCSOperator, and Storage Transfer Service with pros, cons, and production code samples.
Production-grade ETL pipeline from GCS Avro files to BigQuery using Cloud Composer. Features partition gap detection, schema validation, data count verification, and Slack alerts for monitoring daily/hourly data loads.
Example
Production-ready async Python pattern for extracting millions of API records with automatic pagination, retry logic with exponential backoff, and seamless BigQuery integration using aiohttp and uvloop for maximum throughput.
Scala, Python, and Java are all popular programming languages, each with its own strengths and weaknesses. The choice between them depends on factors such as project requirements, ecosystem, performance needs, and personal preference.
Production-ready Apache Airflow setup with PostgreSQL, Celery Executor, and RabbitMQ on macOS. Covers database migration, worker scaling, Flower monitoring, and troubleshooting common distributed task queue issues.
Step-by-step Apache Airflow installation on macOS with SQLite. Covers PyPI install, AIRFLOW_HOME configuration, database migration, admin user creation, scheduler/webserver setup, and daemon process management for production pipelines.