About the company
Our client is a technology company.
The role
Raydar is recruiting for this role on behalf of our client. Run and scale highly available container clusters across several public clouds and specialist GPU providers, owning the production reliability of a model-serving platform. The work spans accelerator troubleshooting, deployment tooling, performance monitoring and automated failover. You will also partner with optimization engineers so performance gains hold up under live traffic.
What you'll do
- Build and operate highly available Kubernetes clusters across multiple public clouds and GPU providers.
- Own the production serving layer, troubleshooting GPU stacks from drivers through to the network fabric.
- Work with optimization engineers so performance improvements hold under real traffic and hardware.
- Extend deployment tooling so models roll out consistently across different accelerator types.
- Build monitoring that tracks latency, throughput and availability by model, hardware type and region.
- Design alerting, automated failover, self-healing systems and traffic routing for reliability.
Requirements
What we're looking for
- At least 5 years of infrastructure engineering experience.
- Currently at staff level or above in an individual contributor infrastructure role.
- Direct, hands-on experience running inference infrastructure day to day.
- Deep experience running production Kubernetes at scale across AWS and GCP.
- Hands-on skill debugging GPU systems, including drivers, CUDA and collective communication libraries.
- Hands-on experience with LLM serving frameworks.
- Experience with high-performance cluster networking.
- Track record of observability and reliability for latency-sensitive services, including metrics, alerting and automated failover.
Bonus points
- Experience deploying across mixed accelerator types.
- Contributions to open-source inference frameworks.
- Background on a production-scale inference platform.
Benefits
Compensation and benefits
- Base salary: USD 350,000 to 450,000
- Equity
Location and work model
- San Francisco, CA, United States
- On-site, 5 days per week in office
- Full-time