Judgment Labs
Product engineer, full stack
- Location
- San Francisco, CA, US
- Arrangement
- On-site
- Employment type
- Full-time
- Posted
- 14 July 2026 (about 2 months ago)
Checked about a month agoApplications go to the employer, never to RoleSprint
About this role
PRODUCT ENGINEER — FULL STACK JOB DESCRIPTION
THE ROLE
Judgment is the learning infrastructure for AI agents. Agents in production don't improve from prompts alone. They improve from experience: the tasks they attempt, the mistakes they make, the edge cases they hit. Here's how it works:
1. We ingest everything your agents do in production: traces, tool calls, decisions, outcomes
2. Judgment turns that raw experience into structured signals: failure modes, behaviors, rubrics, evals
3. Teams close the loop, shipping agent improvements validated against real production evidence
You'll own problems end-to-end: talking to customers, defining what to build, building it, and iterating until it's great. This is not a role where you implement specs handed down.
WHAT YOU WILL ACCOMPLISH
- Investigation interfaces: Design how engineers understand what their systems did and why. Long traces, tool calls, decisions, failures. How do you make a complex sequence of events legible in minutes?
- Verification: Build the platform for verifying system changes: hosted simulated environments, trajectory replay, and monitors for unintended behavior changes.
- The improvement loop: Build the workflows that turn production data into datasets, evaluations, and regression checks, so the path from "found a problem" to "verified a fix" feels like one motion.
- The platform underneath: Workspaces, roles, permissions, billing, usage, and limits for teams running many workflows across many environments.
WHAT YOU'LL BRING
- Experience building and scaling end-to-end production systems, from data layer to UI
- Strong technical problem-solving skills, especially in fast-changing, ambiguous environments
- A builder and tinkerer's mindset with high agency - you find creative ways to overcome obstacles and ship
- Comfort working directly with customers to understand their needs and solve real-world problems
- Excellent communication skills - clear, direct, and persuasive across technical and non-technical audiences
Work location
- San Francisco, CA, US
Related jobs
Ready to make a decision?
This role is either worth your time or it isn’t.
Analyze the posting against your experience, see the gaps clearly, and build the right materials only if the opportunity makes sense.
Nothing is submitted automatically. You choose what happens next.
About this listing
Published on Ashby under the board identifier Judgmentlabs, which is the name the employer’s own job board carries. RoleSprint has not verified the company’s registered or trading name, so it is shown exactly as published rather than tidied up.
RoleSprint is not the employer and not a recruiter. Applications are made on the employer’s own site and never reach us; what RoleSprint does is help you decide whether a role is worth your time and prepare for it if it is.
Published 14 July 2026, last checked about a month ago. A posting stops being advertised here 90 days after the employer published it, and one the employer takes down is marked closed rather than quietly removed.