Join Sesame as an ML Model Serving Engineer to enhance our serving layer for voice agents with cutting-edge techniques.
Posted by employer 1 year ago
First seen on Joblaze 1 month ago
Last verified on the company career page 5 hours ago
Skills & Technologies
Not disclosed in this posting: compensation, years of experience, work arrangement, visa sponsorship.
Benefits
Joblaze summary
In this role, the ML Model Serving Engineer at Sesame focuses on enhancing the serving layer for various machine learning models, including LLMs, speech, and vision. The position requires expertise in frameworks like PyTorch and a strong background in optimizing machine learning models for high throughput and low latency. Ideal candidates are experienced in systems programming and performance engineering, capable of managing complex projects independently. Sesame's team, composed of industry veterans, is dedicated to creating innovative voice agents that integrate seamlessly into everyday life.
Joblaze insights
Quick facts
- What's the tech stack?
- Joblaze extracted these technologies from the posting: Kubernetes, PyTorch, Ray, SGLang, vLLM.
- What seniority level is this role?
- Sesame targets senior candidates for this position.
- Is this full-time or contract?
- Full-time for this ML Model Serving Engineer role at Sesame.
From the original posting
Responsibilities:
Turbocharge our serving layer, consisting of a variety of LLM, speech, and vision models.
Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer to power a new consumer product category.
Modify and extend LLM serving frameworks like VLLM and SGLang to take advantage of the latest techniques in high-performance model serving.
Work with the training team to identify opportunities to produce faster models without sacrificing quality.
Use techniques like in-flight batching, caching, and custom kernels to speed up inference.
Find ways to reduce model initialization times without sacrificing quality.
Required Qualifications:
Expert in some differentiable array computing framework, preferably PyTorch.
Expert in optimizing machine learning models for serving reliably at high throughput, with low latency.
Significant systems programming experience; ex. Experience working on high-performance server systems—you’d be just as comfortable with the internals of VLLM as you would with a complex PyTorch codebase.
Significant performance engineering experience; ex. Bottleneck analysis in high-scale server systems or profiling low-level systems code.
Always up to date on the latest techniques for model serving optimization.
Preferred Qualifications:
Familiarity with high-performance LLM serving; ex. experience with VLLM, SGlang deployment, and internals.
Experience with a public cloud platform such as GCP, AWS, or Azure.
Experience deploying and scaling inference workloads in the cloud using Kubernetes, Ray, etc.
You like to ship and have a track record of leading complex multi-month projects without assistance.
You’re excited to learn new things and work in a multitude of roles.
Full-time Employee Benefits:
401 (k) max employer match: 3.5% of compensation
Unlimited PTO and sick time
Guardian Employee Assistance Program (EAP)
Benefits do not apply to contingent/contract workers.
Standard company text repeated across Sesame's postings is omitted here.