Platform and MLOps Engineer
Collaborated with the London team to pilot the infrastructure of DeepPCB, a reinforcement-learning product for autonomous printed circuit board routing developed with Google Cloud and NVIDIA.
- Managed a shared fleet of approximately 170 on-premise H100 GPUs with team-based quotas and multiple GKE clusters.
- Supported 20 to 30 ML, SRE, and platform engineers on a multi-million-pound annual cloud footprint, while mentoring a junior platform engineer.
- Designed a Crossplane operator in Python and Jinja2 enabling each team to provision cloud resources from a single claim with integrated tenant isolation.
- Implemented dual-level GitOps (Flux and ArgoCD on the GKE Connect gateway) to prevent Kubernetes API exposure.
- Migrated DeepPCB to the new platform, reducing provisioning time from several weeks to half a day.
- Rewarded the blue/green upgrade of RayService to prevent loss of in-flight work during model deployments.
- Configured automatic rollback and cancellation of Argo Rollouts on Prometheus analysis above 5% errors or 5s p99.
- Migrated DeepPCB inference from Argo Workflows to KubeRay to package the model as an independent FastAPI endpoint.
- Served 6 client organizations simultaneously on the same hardware with 200 to 500 inference tasks per day per client.
- Configured DCGM with Prometheus and Grafana to alert on allocated but idle GPUs or ECC/XID errors.
- Deployed Kueue with gang scheduling on AWS EKS with the NVIDIA GPU operator and MIG partitioning.
- Migrated ingress from nginx to Envoy Gateway for native traffic splitting of the Gateway API.
- Led the migration of the LangChain Cooper agent backend to internally hosted H100 bare-metal servers with vLLM.
- Led the consolidation of Neptune to MLflow for over 10 AI engineering teams.
- Reduced annual GCP costs by approximately one third while building a per-client cost dashboard.
- Technical environment: GCP, AWS, Kubernetes, GKE, EKS, Crossplane, ArgoCD, Flux CD, Argo Rollouts, KubeRay, Ray Serve, vLLM, Python, Jinja2, Prometheus, Grafana, DCGM, Kueue, KEDA, Vault, Envoy Gateway, MLflow.