AI Engineer – Trust & Explainability (AI Platform)

Accountabilities • Build end-to-end tracing across AI platform components, including gateways, orchestration, memory, tools, model calls, agent handoffs, parallel branches, and retries.

• Develop correlation capabilities that connect actions across multiple agents into coherent, readable workflow traces.

• Build developer-facing trace and debugging experiences that allow engineers to understand complete agent interactions.

• Develop explanation capabilities that transform raw trace information into human-readable accounts of what an agent did, what information it relied on, and why it followed a particular path.

• Create platform primitives for customer-facing trust, including explanation records, confidence and provenance metadata, and summaries of information used by agents.

• Partner with product engineering teams to integrate trust and explainability capabilities into production AI applications and iterate based on feedback.

• Evaluate and integrate open-source observability, tracing, and evaluation frameworks, extending them when existing capabilities do not meet platform requirements.

• Build missing trust and explainability capabilities and contribute useful fixes or extensions to open-source projects where appropriate.

• Develop and maintain evaluation tooling covering golden datasets, test runners, scoring pipelines, regression reporting, and model or prompt comparisons.

• Create processes for generating and versioning evaluation datasets using de-identified traffic and synthetic scenarios.

• Build automated quality checks for model, prompt, and tool changes so regressions are identified before production release.

• Develop adversarial and red-team testing for prompt injection, jailbreaks, tool misuse, and data-exfiltration risks.

• Build automated tenant-isolation tests to ensure agents cannot access another customer's data through memory, retrieval, tools, or model context.

• Collaborate with Security Operations to incorporate threat models and emerging attack patterns into platform testing.

• Participate in design discussions and code reviews, support onboarding of junior AI engineers, and contribute documentation that reduces tribal knowledge.

Requirements

• 3+ years of professional software engineering experience delivering production features independently.

• Bachelor's degree in Computer Science, Software Engineering, or a related field, or equivalent practical experience.

• Strong foundations in algorithms, data structures, software design, and modern engineering practices.

• Production-level proficiency with Python; TypeScript experience is a plus.

• Hands-on experience building applications that integrate LLMs, such as LLM APIs, agent frameworks, RAG pipelines, or comparable systems, either professionally or through substantial personal or open-source work.

• Active daily use of AI-assisted software development tools and an interest in applying them effectively to engineering workflows.

• Experience with Git, Docker, automated testing, and modern scripting or development tooling.

• Hands-on experience in at least one area such as LLM observability and tracing, LLM evaluation and testing, agent frameworks and multi-agent orchestration, or application security testing, with an interest in developing broader expertise.

• Experience with distributed tracing or observability tooling in a production environment.

• Strong automated testing practices, particularly for systems where outputs can vary between runs.

• Experience running workloads on AWS or Azure, including foundational knowledge of identity and access management, networking, and secrets management.

• Strong analytical and problem-solving skills, with the ability to work independently while knowing when to seek guidance on complex system design.

• Clear written and verbal communication skills and the ability to collaborate effectively across engineering, product, architecture, and security teams.

• Preferred experience with OpenTelemetry, GenAI semantic conventions, OpenLLMetry, or comparable tracing standards.

• Familiarity with LLM observability and evaluation platforms such as Langfuse, Arize Phoenix, LangSmith, Braintrust, promptfoo, DeepEval, or equivalent tools.

• Experience contributing to open-source AI observability, evaluation, or agent-framework projects is a plus.

• Experience with AWS Bedrock, Azure OpenAI, or other cloud-managed model services is beneficial.

• Experience building multi-tenant SaaS platforms where tenant isolation is a critical security requirement is preferred.

• Experience in financial services, fintech, or another regulated environment where explainability influenced technical decisions is advantageous.

• Experience building developer-facing debugging, monitoring, or visualization tools is a plus.

Benefits

• Annual compensation range of $104,148–$177,600 , with final compensation determined by experience, qualifications, and other job-related factors.

• Fully remote work within the United States.

• Medical, dental, vision, life, and disability insurance coverage.

• Flexible paid time off.

• Paid company holidays.

• 401(k) plan with company matching.

• Opportunity to work on AI platform infrastructure spanning agent tracing, explainability, evaluation, observability, and security.

• Exposure to emerging AI engineering practices and open-source observability and evaluation frameworks.

• Collaborative environment with opportunities for technical growth alongside experienced engineers and architects.

• Role supporting production AI systems in a regulated financial-services environment.

• Background, credit, and drug screening may be required as part of the hiring process.

How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1