Reference-data pipeline migration
I led the end-to-end migration of a core promotion-targeting pipeline from legacy Table Storage and Databricks to an event-driven SQL architecture that processes millions of records daily.
- Built a Spring Boot batching engine with idempotency, retries and fault-tolerant ingestion.
- Tuned indexes, queries and JDBC batching to hold throughput across millions of records per run.
- Reached 99.9%+ delivery to downstream systems, reprocessing unacknowledged events within a 5-minute SLA.