About the company
Our client is a software company.
The role
Raydar is recruiting for this role on behalf of our client. Take charge of the speed and efficiency of production machine learning serving, locating and removing latency and throughput bottlenecks across several model types. Create tooling that accelerates optimization work and partner with research and platform colleagues so new models launch with efficient serving.
What you'll do
- Lead inference optimization from start to finish across language, audio and image-generation models.
- Implement and tune memory management strategies for long-running sessions, including eviction, compression and efficient attention.
- Evaluate, deploy and extend open-source model serving frameworks for specific workloads.
- Profile and benchmark latency and throughput, then systematically eliminate bottlenecks.
- Build internal tooling such as profiling viewers and end-to-end test harnesses to make optimization faster and more rigorous.
- Speed up generative image model inference using techniques like distillation, caching and custom kernel work.
- Apply and develop quantization methods to reduce memory use and raise throughput while preserving output quality.
- Work with research and infrastructure colleagues so new models ship with optimized serving from the start.
Requirements
What we're looking for
- A minimum of two years in a full-time role running live machine learning systems.
- Hands-on skill using profiling tools to find and fix performance bottlenecks in large language or diffusion models.
- A record of taking trained models into live, large-scale serving environments.
- Familiarity with popular open-source inference serving frameworks.
- Proficiency in Python and either Rust or Go.
- Solid hands-on knowledge of container orchestration, infrastructure as code and public cloud environments.
- Background at a startup or on a highly selective team at a large technology company.
- Ability to design scalable infrastructure from scratch, compare technologies and debug distributed systems.
- Willingness to work on-site five days a week.
Bonus points
- Experience building multimodal systems for video, audio or multimedia models.
- Experience with low-level optimization such as GPU kernel work.
Benefits
Compensation and benefits
- Base salary: USD 250,000 to 350,000 per year
- Health coverage with employer HSA contribution
- Paid time off and holidays
- Commuter benefits
- Relocation support
Location and work model
- Seattle, WA, United States
- On-site, 5 days per week in office
- Full-time