01
Lakehouse architecture
Delta-first platforms that turn fragmented batch and streaming estates into governed, reusable products.
Delta LakeUnity CatalogLakeflow
Databricks / Lakehouse / Production Data Platforms
Gaurav Kumar
Senior Data Architect.
I turn complex, distributed systems into calm, governed platforms that teams can trust in production.
Lakehouse architect
Databricks certified pro
Gaurav KumarSenior Data Architect · Aays
10YRSLed $40B Kimberly-Clark × Kenvue merger data integrationSAP HANA → Snowflake · zero data loss · 30% faster100+ TB legacy estate → Azure · 60% faster ETL$30M operating cost removed175+ countries on one governed platformDatabricks Certified ProfessionalNow Senior Data Architect at Aays
01What I build
The architecture
behind the outcome.
Good platform work is quiet. It makes the next pipeline faster to ship, the next incident easier to resolve, and the next decision easier to trust.
02
Streaming at scale
Event-driven ingestion with restartability, exactly-once semantics, and freshness measured in minutes, not hours.
Structured StreamingKafkaEvent Hubs
03
Governance by design
Least privilege, masking, lineage, quality gates, and observability built into the path to production.
LineageMonte CarloData Quality
04
Cloud platforms
Pragmatic architecture across Azure and AWS, from landing zones to the operational details that keep them healthy.
AzureAWSSnowflake
05
Delivery as code
Infrastructure and data products shipped through repeatable promotion paths, not one-off console work.
TerraformDABsGitHub Actions
06
MLOps & GenAI
Production foundations for feature data, model promotion, retrieval, and traceable answers over operational data.
MLflowRAGLangGraph
02Proof in production
Measured in more
than dashboards.
From performance tuning to platform-wide migrations, these are the signals that stay after launch day.
Execution-plan analysis, partition tuning, and cluster sizing
100+ TB migration with zero data loss
Legacy estate to Azure at enterprise scale
Global operations on one governed platform
Supply chain control tower data foundation
03A decade at scale
Ten years,
in numbers and domains.
Beyond the headline wins — the scale, the people and the industries behind a decade of production data platforms.
0+production pipelines built across a decade
0+field sensors ingested in real time
0ML models run on pipelines I built
0large-scale migration programmes delivered
0+tables governed under Unity Catalog
0+vendor teams managed
0regions served: APAC · EMEA · NA · LATAM · ANZ · Middle East
0.9%pipeline SLA on mission-critical feeds
$0.5Mannual cloud & warehouse spend optimised
0%Databricks compute cost reduction
0%faster queries after HANA → Snowflake
0%dashboard accuracy improvement
Industries delivered for
Consumer goods & supply chainManufacturingEnergy · oil & gasIndustrial IoTFuel retailTelecomDigital advertisingProcurement & sourcing
Recognition
APAC Global Distinguished AwardKimberly-Clark
Best Employee of the YearHarman
Brillian of the MonthBrillio
Spot Award ×2Mu Sigma
Quarterly Individual Star AwardCapgemini
04Flagship programmes
The programmes
with real stakes.
Merger integrations, estate migrations and cost programmes — the work where data platforms carry the business.
×KenvueKimberly-Clark × Kenvue — $40B merger integration
Led data-platform integration planning for one of the largest consumer-health combinations: mapping source estates, sequencing Day-1 / Day-100 data cutovers and designing the governed landing path for combined reporting.
$40Bdeal value
150+source systems mapped
3regions aligned
- 01Source-estate discovery
- 02System & data mapping
- 03Day-1 data cutover
- 04Day-100 consolidation
- 05Combined governed reporting
Migration programmes
SAP HANASnowflake
Legacy warehouseAzure Data Lake
Bespoke pipelinesDelta Lake medallion
Batch reportingStructured Streaming
Hadoop / HiveDatabricks Lakehouse
Ad-hoc reportingGoverned Power BI self-service
Where the $30M came from
$0Moperating cost removed · supply chain control tower
- Inventory & working capital$12M
- Maintenance & downtime avoided$9M
- Logistics & supply planning$6M
- Platform & cloud efficiency$3M
Control towerDemand sensingPredictive maintenanceInventory optimisation
Indicative breakdown
05Selected experience
Ten years of
shipping the hard parts.
A career moving from analytics foundations to global lakehouse ownership — with the same throughline: make systems legible, resilient, and useful.
2016201820202022Now · Aays 2026
01
Oct 2026 — PresentBengaluru · Current role
Oct 2026 — PresentSenior Data ArchitectLeading lakehouse architecture for global enterprise clientsAays Analytics
Senior Data ArchitectJoined Aays as Senior Data Architect to design and lead Databricks lakehouse platforms for global enterprise clients — bringing a decade of governed, production-grade delivery.
- Architecting Databricks lakehouse platforms for enterprise data & AI programmes.
- Shaping solution design from discovery and scoping through to production.
- Setting governance, cost and delivery standards across client engagements.
02
May 2022 — Oct 2026Bengaluru · Global Data Platform CoE
May 2022 — Oct 2026Senior Data ArchitectGlobal platform serving 175+ countriesKimberly-Clark
Senior Data ArchitectOwning the Azure data platform serving operations across 175+ countries — from architecture through rollout, monitoring, and tier-3 escalation.
- Standardised 20+ batch and streaming pipelines on a Delta Lake medallion architecture.
- Designed Unity Catalog access models, masking, and lineage across the estate.
- Led $40B merger integration planning and SAP HANA → Snowflake migration.
03
Feb 2021 — May 2022Bengaluru · RaceTrac & Lebara
Feb 2021 — May 2022Senior Data Engineer40% faster end-to-end Spark processingHarman
Senior Data EngineerBuilt high-throughput telemetry platforms and config-driven ingestion frameworks for real-time monitoring and alerting.
- Introduced Structured Streaming to cut freshness from hours to minutes.
- Reduced end-to-end Spark processing time by 40% through plan and partition optimisation.
- Built AWS streaming paths with checkpointing, retries, and dead-letter handling.
04
Jun 2020 — Jan 2021Bengaluru · Microsoft Bing Ads
Jun 2020 — Jan 2021Data Engineer100+ TB migrated with 60% faster ETLBrillio
Data EngineerMoved a 100+ TB legacy data estate to Azure while keeping source-to-target integrity visible through cutover.
- Reduced ETL processing time by 60% with reusable PySpark libraries and ADF templates.
- Partnered with stakeholders, QA, and business owners through migration validation.
- Shortened delivery timelines by roughly 30% through parameterised patterns.
05
Dec 2018 — Jun 2020Bengaluru · Petrofac Industrial IoT
Dec 2018 — Jun 2020Data Engineer$30M operating cost removedMu Sigma
Data EngineerArchitected data foundations for 20,000+ field sensors and the supply chain control tower behind a $30M operating cost reduction.
- Contextualised live sensor signals into a unified layer for downstream AI workloads.
- Operated pipelines behind 246 deployed models with parallel training and inference paths.
- Deployed reproducible, containerised pipelines on Amazon EKS.
06
Nov 2016 — Nov 2018Bengaluru · Procurement & Sourcing Analytics
Nov 2016 — Nov 2018Software Analyst40% fewer supplier duplicatesCapgemini
Software AnalystDesigned centralised data lake and analytics solutions connecting ERP, SCM, and sourcing systems.
- Built SQL data models and Power BI dashboards for procurement and finance.
- Applied Levenshtein, Jaccard, and Cosine similarity to supplier data.
- Cut supplier master-data duplicates by 40%.
06Deep dives
The systems
behind the signal.
Three examples of turning ambiguous platform work into reusable architecture — each one designed to be adopted, governed, and operated.
01 / DATA PLATFORM
Metadata-driven ingestion framework
Replacing bespoke pipeline builds with one configuration-led path for files, APIs, OLTP systems, and streams.
RESULTOne reusable ingestion contract
Read the architecture 02 / MLOPS
ML data & deployment pipeline
A governed production path from point-in-time feature data to CI-gated model promotion and traceable predictions.
RESULTProduction ML with lineage
Read the architecture 03 / GENAI
GenAI retrieval layer
A traceable retrieval foundation for a supply-chain chatbot, with incremental refresh and operational metadata per chunk.
RESULTAnswers teams can trust
Read the architecture 04 / FINOPS
SQL pre-flight for Snowflake cost
A Streamlit-in-Snowflake tool that flags expensive query patterns before they run — shifting cost control left to the developer.
RESULTCost control before execution
Read the architecture 07How I think
One platform.
Three layers of trust.
A medallion architecture is more than a diagram. It is a contract for how raw signals become decisions — with governance and observability carried all the way through.
Silver / Cleansed & enriched
Quality gates, MERGE upserts, business context, governed Delta tables.
BRONZEraw signals01
SILVERtrusted data02
GOLDactivated value03
QUALITY GATESLINEAGEOBSERVABILITY
STREAM ACTIVE / SILVER LAYER IN FOCUS
08Credentials & toolkit
The tools are only
useful in context.
Verified credentials from the platforms I architect with — plus the details that make them production-ready.
Claude Code 101Data Engineer Professional
Data Engineer Associate
Azure Data Engineer Associate (DP-203)Claude Code 101Anthropic Academy · Completed Aug 02, 2026View credential
Databricks Certified Data Engineer ProfessionalDatabricks · Professional certification
Databricks Certified Data Engineer AssociateDatabricks Academy · Issued Oct 27, 2026 · Expires Oct 27, 2028View credential
Microsoft Certified: Azure Data Engineer Associate (DP-203)Microsoft Learn · Azure data engineering credentialView credential
4verified credentials
2Databricks
1Microsoft Azure
1Anthropic
EducationB.E. Computer Science & EngineeringJSS Academy of Technical Education · 2016
LanguagesHindi (native) · English (fluent)Working across APAC, EMEA & Americas teams
Full toolkit
01Databricks & Lakehouse
Delta LakeUnity CatalogLakeflow (DLT)Auto LoaderDatabricks SQLWorkflowsAsset BundlesMLflow
02Spark & Streaming
Apache SparkStructured StreamingKafkaAmazon MSKKinesisEvent HubsExactly-once
03Governance & Security
Unity Catalog lineageColumn maskingLeast privilegeKey VaultService principalsIAM
04IaC, CI/CD & Orchestration
TerraformGitHub ActionsAzure DevOpsDockerKubernetes (EKS)AirflowControl-M
05Cloud · Azure
Data FactorySynapseADLS Gen2Event HubsFunctionsCosmos DB
06Cloud · AWS
MSKKinesisLambdaGlueS3EKSCloudWatch
07Databases & Warehousing
SnowflakeSAP HANAPostgreSQLSQL ServerBigQuery
08Testing & Observability
pytestMonte CarloGrafanaSplunkQuality gatesSchema-drift detection
09MLOps & GenAI
MLflow registryFeature pipelinesRAGEmbeddingsVector DBsLangChainLangGraphCrewAI
10Cloud · GCP
BigQueryCloud Storage (GCS)Apache AirflowCloud ComposerDataproc
11Languages & Modelling
PythonPySparkSQL (advanced)ScalaMedallionData VaultDimensional
12APIs & BI
REST APIsParquet / AvroStreamlitPower BISSAS
09Open channel
Have a platform
worth building?
If you’re working through a data platform decision, a migration with real stakes, or the path from prototype to production, I’d like to hear about it.