Product engineer, full stack
Own end-to-end problems in building and improving AI agent learning infrastructure at Judgment Labs.
Posted by employer 1 month ago
First seen on Joblaze 1 week ago
Last verified on the company career page 1 day ago
AI in the day-to-day
Judgment ingests everything your agents do in production to improve their performance.
Not disclosed in this posting: compensation, years of experience, work arrangement, visa sponsorship.
Joblaze summary
In this role, the product engineer will take ownership of the entire development process, from engaging with customers to defining and building solutions that enhance AI agent performance. The position requires expertise in creating scalable production systems, with a focus on both backend and frontend development, as well as strong problem-solving skills in dynamic environments. Ideal candidates are those who thrive in a hands-on role, possess excellent communication abilities, and are comfortable navigating complex technical challenges. Judgment Labs emphasizes a collaborative approach, where engineers directly influence product improvements based on real-world data.
Joblaze insights
- Listed 9 days ago on Joblaze — tracked directly from Judgment Labs's career page.
- Judgment Labs has 4 other open roles including Backend/Infra Engineer, Founding Account Executive, Product engineer, Agent.
- 5148 active AI/ML roles on Joblaze right now.
- 20 active Fullstack roles on Joblaze right now.
- Typical AI/ML salary we track runs $140,000–$231,000 (median $180,000, across 2237 roles). See AI/ML salary trends
Quick facts
- What's the tech stack?
- Joblaze extracted these technologies from the posting: AI/ML, Fullstack.
- What seniority level is this role?
- Judgment Labs targets mid-level candidates for this position.
- Is this full-time or contract?
- Full-time for this Product engineer, full stack role at Judgment Labs.
From the original posting
Product Engineer — Full Stack Job Description
The Role
Judgment is the learning infrastructure for AI agents. Agents in production don't improve from prompts alone. They improve from experience: the tasks they attempt, the mistakes they make, the edge cases they hit. Here's how it works:
We ingest everything your agents do in production: traces, tool calls, decisions, outcomes
Judgment turns that raw experience into structured signals: failure modes, behaviors, rubrics, evals
Teams close the loop, shipping agent improvements validated against real production evidence
You'll own problems end-to-end: talking to customers, defining what to build, building it, and iterating until it's great. This is not a role where you implement specs handed down.
What You Will Accomplish
Investigation interfaces: Design how engineers understand what their systems did and why. Long traces, tool calls, decisions, failures. How do you make a complex sequence of events legible in minutes?
Verification: Build the platform for verifying system changes: hosted simulated environments, trajectory replay, and monitors for unintended behavior changes.
The improvement loop: Build the workflows that turn production data into datasets, evaluations, and regression checks, so the path from "found a problem" to "verified a fix" feels like one motion.
The platform underneath: Workspaces, roles, permissions, billing, usage, and limits for teams running many workflows across many environments.
What You'll Bring
Experience building and scaling end-to-end production systems, from data layer to UI
Strong technical problem-solving skills, especially in fast-changing, ambiguous environments
A builder and tinkerer's mindset with high agency - you find creative ways to overcome obstacles and ship
Comfort working directly with customers to understand their needs and solve real-world problems
Excellent communication skills - clear, direct, and persuasive across technical and non-technical audiences
Similar positions
Explore more