DATA ENGINEERING
•
LAKEHOUSE
•
STREAMING
•
RELIABILITY
I build verifiable data-platform reference systems around replayability, event-time correctness, contracts, lineage, governance, privacy, and operational evidence.
Reliable by design. Replayable by default. Observable when it matters.
engineering_profile:
domains:
- lakehouse and incremental processing
- streaming correctness and state
- data quality, lineage, and governance
- platform delivery and infrastructure-as-code
delivery:
- Python and SQL
- Docker and GitHub Actions
- Azure and Terraform patterns
evidence:
portfolio_systems: 10
automated_tests: 120
latest_ci: "passing across every portfolio repository"
disclosure: "All portfolio labs use deterministic synthetic data."| System | Engineering signal | Verification |
|---|---|---|
| Retail Lakehouse Platform | Medallion layers, contract gates, quarantine, reconciliation |  |
| Real-time Fraud Streaming | Event time, watermarks, keyed state, deduplication |  |
| CDC Lakehouse Reconciliation | Ordered CDC, SCD2 history, rerun safety, audit outcomes |  |
Inspect the engineering evidence
| Signal | What the repositories demonstrate |
|---|---|
| Correctness | Deterministic fixtures, idempotent reruns, duplicate suppression, and reconciliation |
| Architecture | Explicit boundaries, contracts, decision records, and failure paths |
| Operations | Quality gates, metrics, service objectives, manifests, and runbooks |
| Delivery | Automated tests, container checks, Terraform validation, and GitHub Actions |
flowchart LR
S["Sources<br/>events · APIs · files"] --> I["Ingestion<br/>batch · streaming"]
I --> B["Bronze<br/>immutable · replayable"]
B --> Q{"Contracts &<br/>quality gates"}
Q -->|pass| V["Silver<br/>validated · conformed"]
Q -->|quarantine| D["Diagnostics<br/>lineage · incidents"]
V --> G["Gold<br/>data products · metrics"]
G --> C["Consumers<br/>BI · ML · APIs"]
O["Observability<br/>freshness · volume · SLOs"] -. monitors .-> I
O -. monitors .-> V
O -. monitors .-> G
| Capability | Tools and patterns |
|---|---|
| Processing | Python, SQL, incremental and idempotent pipelines |
| Lakehouse | Medallion architecture, schema evolution, reconciliation |
| Streaming | Kafka concepts, event time, watermarks, deduplication, state |
| ML data | Point-in-time features, offline/online parity, leakage prevention |
| Platform | Policy-as-code, semantic metrics, data products, residency controls |
| Reliability | Contracts, quality gates, lineage, SLOs, runbooks, FinOps |
| Delivery | GitHub Actions, containers, Terraform, multi-version testing |
| Platform system | Focus |
|---|---|
| Point-in-time Feature Store | Leakage-safe historical joins and offline/online parity |
| Data Contracts & Observability | Contracts, SLOs, lineage, incidents, and runbooks |
| Semantic Metrics Governance | Metric contracts, grain validation, SQL compilation, impact |
| Data Platform Control Plane | Policy-gated onboarding, manifests, plans, and drift |
| Data Platform Cost Optimizer | Showback, anomaly detection, and explainable recommendations |
| Privacy-preserving Data Sharing | Pseudonymization, privacy controls, and audit policy |
| Federated Data Mesh Simulator | Product contracts, residency, dependency DAGs, SLO simulation |
These are transparent portfolio labs built with deterministic synthetic data. They demonstrate design and implementation choices without claiming client or production deployment history.
current_focus:
building:
- production-minded data-platform reference systems
- clearer operational evidence and architecture narratives
hardening:
- CI, contracts, idempotency, reconciliation, and failure paths
exploring:
- open-data ingestion, orchestration, and analytical serving
open_to:
- data engineering collaboration
- architecture review
- meaningful open-source contribution
correctness before cleverness · evidence before claims · systems before buzzwords