Inference Systems Jobs in New York

Staff + Sr. Software Engineer, Inference Remote Visa

Anthropic San Francisco, CA | New York City, NY | Seattle, WA $300k–$485k/yr Published 7 months ago

AWS Load Balancing AI/ML LLM inference distributed systems

Performance Engineer, Inference Systems Remote Visa

Join Anthropic as a Performance Engineer to optimize AI inference systems for throughput, latency, reliability, and correctness.

Anthropic San Francisco, CA | New York City, NY | Seattle, WA $350k–$850k/yr Published 1 week ago

pandas Load Balancing SQL TPU Python

Flexible on stack

Apply →

Software Engineer, Model Routing & Inference

Cursor New York Published 1 month ago

AI/ML distributed systems real-time data pipelines traffic routing inference serving

Apply →

Engineering Manager, Inference Routing and Performance Remote Visa

Anthropic San Francisco, CA | New York City, NY | Seattle, WA $405k–$485k/yr Published 2 months ago

continuous batching coordination planes prefill/decode disaggregation KV caching Trainium

Apply →

Engineering Manager, Inference Remote Visa

Anthropic San Francisco, CA | New York City, NY | Seattle, WA $425k–$560k/yr Published 1 year ago

ML framework internals GPU/Accelerator programming Machine Learning AI/ML distributed systems

Apply →

Staff + Sr. Software Engineer, Inference Deployment Remote Visa

Anthropic San Francisco, CA | New York City, NY | Seattle, WA $320k–$485k/yr Published 3 months ago

Trainium Rust TPU Python GPU

Apply →

Sr. Manager, Engineering - AI Gateway (LLM Inference) Remote

Databricks New York $228.6k–$314.3k/yr Published 1 month ago

Machine Learning AI/ML Generative AI

Apply →

Applied AI Engineer

Ramp New York, NY (HQ) Published 4 months ago

backend development AI/ML LLM inference web frameworks infrastructure

Apply →

AI Research Engineer - Datadog AI Research (DAIR) Remote

Datadog New York, New York, USA $140k–$400k/yr Published 9 months ago

C++ Ray JAX CUDA DeepSpeed

Apply →

Research Engineer, Frontier Speculative Decoding

Together AI San Francisco, New York City $190k–$270k/yr Published 6 months ago

DeepSpeed AI/ML Slurm PyTorch Python

Apply →

Data Scientist, Supply Remote Visa

Anthropic San Francisco, CA | New York City, NY $275k–$370k/yr Published 3 weeks ago

data visualization AI/ML Operations Research SQL Python

Apply →

AI Research Scientist - Datadog AI Research (DAIR)

Datadog New York, New York, USA $140k–$400k/yr Published 1 year ago

multimodal learning Machine Learning DeepSpeed AI/ML reinforcement learning

Apply →

Senior Applied Scientist, Credit Risk

Ramp New York, NY (HQ) Published 2 weeks ago

pandas Machine Learning scikit-learn PyTorch Economics

Apply →

Data Scientist, Developer Productivity Remote Visa

Anthropic San Francisco, CA | New York City, NY $275k–$370k/yr Published 3 weeks ago

A/B Testing data visualization AI/ML Statistical Modeling SQL

Apply →

Staff+ Software Engineer, Backend Remote Visa

Anthropic San Francisco, CA | New York City, NY | Seattle, WA $405k–$485k/yr Published 1 month ago

LLMs SO2 distributed systems cloud infrastructure agentic applications

Apply →

Software Engineer, Machine Learning Remote

Figma San Francisco, CA • New York, NY • United States $153k–$376k/yr Published 12 months ago

Spark MLlib scikit-learn XGBoost PyTorch TensorFlow

Apply →

Research Engineer, Frontier Safety Risk Assessment Visa

Google DeepMind London, UK; New York City, New York, US; San Francisco, California, US $136k–$245k/yr Published 5 months ago

AI/ML Python

Apply →

Research Engineer, Evaluations Remote

AssemblyAI Remote - New York $210k–$260k/yr Published 2 months ago

TTS ASR VAD Voice AI Machine Learning

Apply →

Staff Software Engineer - AI Research Infrastructure

Develop and run the research stack that powers Databricks AI Research, enabling rapid large-scale experiments.

Databricks New York City, New York; San Francisco, California $199k–$270k/yr Published 2 weeks ago

C++ Ray Java Go Slurm

Flexible on stack

Apply →

Staff Software Engineer - AI Research Infrastructure

Databricks New York City, New York; San Francisco, California $190k–$270k/yr Published 1 month ago

C++ Ray AI/ML Java Go

Apply →