dataiq and AIQFlow
The AI data platform and pipeline orchestration behind 25+ AI products for the energy industry.
RoleLead Data & ML Engineer, then Lead Solutions Architect; managed 17 developers and 3 architects
Result25+ AI products on the platform
The problem
Oil and gas data comes in every form: sensors, images, seismic, drilling operations, well logs. Each AI product was rebuilding the same plumbing.
What I did
- Joined as one of the first 12 employees of the startup that became AIQ, and grew the engineering team to 20 people (17 developers, 3 architects) that I managed directly.
- Designed dataiq, the shared AI data platform: storage, permissions and archiving for every data type, with a data ontology to organize and expose it.
- Built AIQFlow, the orchestration of the ingestion, processing and inference pipelines around it.
- Built security, compliance and traceability in, for data and AI governance.
- Managed the migration from AWS to Azure: a lift-and-shift first, then a redesign onto native Azure services.
- Gave other teams their architecture on top of the platform.
Python · TypeScript · PyTorch · ClearML · Kubernetes · Helm · Terraform · AWS · Azure · Keycloak · RabbitMQ · Elasticsearch · Spark · PostgreSQL