Hire ML platform engineers who have actually run GPUs under load.

An ML platform engineer owns the AI-specific infrastructure layer your product teams consume: inference serving, fine-tuning pipelines, GPU orchestration, the model registry, experiment tracking, feature stores, and the cost/latency dashboards that stop a Q3 OpenAI bill from becoming a board-level incident. They are not an ML engineer (who builds models), not an SRE, and not a generalist DevOps. They own the abstraction between your ML scientists and your Kubernetes cluster — and in 2026, that is the single highest-leverage role on an AI-native team of 8 or more engineers.

The stack has consolidated. We see the same toolchain in 80%+ of Series B–D engagements: Kubernetes (EKS or GKE) for orchestration, Ray or Anyscale for distributed training and batch inference, vLLM or SGLang for serving open-weight models, NVIDIA Triton or Modal for multi-model endpoints, Docker + Helm + Terraform for packaging, Prometheus + Grafana for infra observability, Langfuse or Arize for LLM-specific traces, and SageMaker / Vertex AI for managed pieces. The vLLM team's benchmarks (v0.6 release notes, Oct 2024) show ~2.7× throughput over naive HuggingFace `generate()` on a single A100 — an ML platform engineer is the person who captures that gain in production rather than in a benchmark table.

Demand is rising for a structural reason. NVIDIA's 2024 State of AI Infrastructure report found enterprises running more than one foundation model in production spend 43% of their AI budget on inference compute, and only 17% on training. Once a company graduates from one feature on openai.com to three features on a mix of OpenAI + Anthropic + self-hosted Llama or Qwen, infra complexity compounds faster than any product team can absorb as a side quest. Our rule of thumb from 30+ founder conversations: if you have fewer than five AI-focused engineers, this is an over-hire. If you have more than eight, the role is already overdue and someone is doing it badly on top of their real job.

Production signals matter more here than for any other role we hire. A strong ML platform engineer reasons natively in GPU utilisation, tokens-per-GPU-hour, cost-per-million-tokens-served, model-deployment MTTR, and eval-pipeline wall-clock. They know continuous batching from static batching, when KV-cache reuse earns its complexity, and have strong opinions on speculative decoding at the platform layer versus the application layer. If a candidate cannot draw the lifecycle of a single inference request from load balancer to tokenizer to GPU scheduler to response stream, they have not operated this layer; they have read about it.

The pattern we keep seeing in inbound CVs: 'MLOps engineer, 4 years' translated in practice to 'I wrote Airflow DAGs that ran scikit-learn on CPU nodes'. Legitimate platform experience, but not the same job as running GPU inference under real load. A CPU-bound batch scheduler never thinks about KV-cache fragmentation, GPU memory pressure under concurrent requests, spot-instance preemption during a fine-tune, or the difference between Triton's dynamic batching window and vLLM's continuous batching scheduler. We filter the two populations apart in the first 12 minutes.

Our sourcing channels for this role: the OSS contributor graphs of vLLM, Ray, Triton Inference Server, SGLang, KServe, and Hugging Face `text-generation-inference`; NVIDIA GTC speaker lists and MLPerf Inference submitter teams from the last 18 months (mlcommons.org); engineers with public Modal, Anyscale, or Together.ai production usage via conference talks or blog posts; KubeCon ML track attendees; and the private network of 640+ CEE AI engineers Nikita built during his time at Neurons Lab, which over-indexes on platform talent because the shop ran a shared ML platform serving 15+ enterprise clients in parallel.

Every candidate goes through a 12-minute AI technical interview tuned to this role. Sample probes: 'walk me through the last time you saw GPU utilisation sit at 40% on an H100 serving a 7B model — what did you change to get it to 80%?', 'your fine-tune job is evicted from spot every 90 minutes, checkpoint interval is 20 minutes — design the recovery flow', 'explain continuous batching to a backend engineer who has never touched a GPU'. The AI asks adaptive follow-ups. A human recruiter reviews before shortlist. Candidates who passed our filter in 2026-Q1 had a median 4 years of post-2022 GPU production experience and an 80% client-interview pass rate at our clients.

The last layer is role-specific: we require every ML platform engineer shortlisted to have personally operated at least one multi-GPU inference deployment that served >1M tokens/day for >30 days. We verify via public artifacts (GitHub, a talk, a published post-mortem, a benchmark submission) and a reference call with the engineering lead at their previous platform. "I optimised a fine-tune" is not the same thing.