Senior AI Agent Engineer

Senior AI Agent Engineer - Voice AI at Zendesk
Join to apply for the Senior AI Agent Engineer - Voice AI role at Zendesk .

Job Description

The Agentic Tribe is revolutionizing the chatbot and voice assistance landscape with Gen3, a cutting-edge AI Agent system that's pushing the boundaries of conversational AI. Gen3 is a goal-oriented, dynamic, and truly conversational system capable of reasoning, planning, and adapting to user needs in real time. Leveraging a multi-agent architecture and advanced language models, Gen3 delivers personalized and engaging user experiences, going far beyond scripted interactions to handle complex tasks and “off-script” inquiries with ease.

We are seeking a passionate and experienced Senior Voice AI Agent Engineer with a strong focus on Voice AI to join our team. In this role, you will be dedicated to innovating at the forefront of conversational AI, engineering intelligent, autonomous agents that can listen, understand, and speak with human-like fluidity. You will build the cognitive architecture for our voice applications, creating systems that can reason, plan, and execute complex tasks through seamless, low-latency spoken dialogue. A key part of your role will be to effectively communicate complex technical concepts to both technical and non-technical stakeholders.

What You Will Do

Design and develop robust, stateful, and scalable voice-first AI agents using Python, specifically optimized for real-time voice interactions, managing turn-taking, interruptions, and low-latency responses.

Integrate real-time Speech-to-Text (STT), Text-to-Speech (TTS), and Voice Activity Detection (VAD) services to create a seamless conversational flow.

Connect voice agents with existing enterprise systems, databases, and third-party APIs to create powerful, end-to-end automated workflows initiated and managed through voice.

Establish and own the evaluations for voice agent performance and behavior, and iterate over time to systematically improve performance, reliability, and the overall user experience.

Build end-to-end conversational flows with reasoning, planning, and dynamic tool use — beyond pre-scripted voice experiences.

Work cross-functionally with product managers, ML scientists, and engineers to deeply understand user needs and voice interaction goals.

Implement fallback, recovery, and error-handling strategies to deal with noisy audio input or speech recognition inaccuracies.

Define and track voice-specific evaluation metrics (e.g., word error rate, latency, conversational naturalness).

Develop observability tools and guardrails to monitor performance, ensure safety, and handle edge cases in spoken interactions.

Document development, architecture decisions, and research findings to share knowledge across the team.

Requirements

LLM-Oriented System Design: Strong experience building multi-step, tool-using agents (LangChain, Autogen). Familiar with prompt engineering, context management, and reasoning strategies like Chain-of-Thought and ReAct.

Voice AI Expertise:

Experience building low-latency, streaming voice applications. Expertise in integrating and managing real-time STT/TTS models and APIs. Proficient with techniques for Voice Activity Detection (VAD), noise suppression, and implementing robust barge-in/interruption logic.

Experience with integrating third-party voice AI APIs, including STT and TTS services from providers like OpenAI, Deepgram, ElevenLabs, etc.

Understanding of latency, timing, and streaming audio constraints.

Tool Integration & APIs: Comfortable connecting agents to external APIs, tools, databases in secure environments.

RAG (Retrieval-Augmented Generation): Building pipelines with vector stores, chunking strategies, and hybrid retrieval.

Evaluation & Observability: Implementing and using monitoring tools and evaluation frameworks (Braintrust) to score our AI Agents.

Safety & Reliability: Familiarity with techniques for prompt injection defense, guardrails (Rebuff, Guardrails AI), and failover logic.

Performance Optimization: Token budget and latency management using caching, model routing, etc.

Programming & Deployment: Expert in Python, FastAPI, and LLM SDKs. Experience deploying AI apps to cloud platforms (AWS, GCP, Azure) using CI/CD best practices.

Nice-to-have

M.S. / Ph.D. in Computer Science, NLP, Machine Learning, or related field

Background in spoken dialogue systems or conversational UX design

Familiarity with real-time streaming architecture (e.g., WebRTC, gRPC, socket.io)

Multilingual ASR/TTS pipeline experience

About Zendesk

Zendesk builds software for better customer relationships. It empowers organizations to improve customer engagement and better understand their customers. Zendesk products are easy to use and implement. They give organizations the flexibility to move quickly, focus on innovation, and scale with their growth.

Zendesk is an equal opportunity employer, and we’re proud of our ongoing efforts to foster diversity & inclusion in the workplace. By submitting your application, you agree that Zendesk may collect your personal data for recruiting, global organization planning, and related purposes. Zendesk's Candidate Privacy Notice explains what personal information Zendesk may process, where it may be processed, its purposes, and your rights.

Hybrid: In this role, our hybrid experience is designed at the team level to give you a rich onsite experience while also giving you flexibility to work remotely for part of the week. The specific in-office schedule is to be determined by the hiring manager.

The Intelligent Heart Of Customer Experience

Zendesk is an equal opportunity employer. We are an AA/EEO/Veterans/Disabled employer. If you are based in the United States and would like more information about your EEO rights, please click here. Zendesk provides reasonable accommodations for applicants with disabilities.

By submitting your application, you agree to Zendesk collecting your personal data for recruiting and related purposes. Zendesk’s Candidate Privacy Notice explains what information is processed and your rights.

Note: This posting is for a role in Ireland. Referrals increase your chances of interviewing at Zendesk by 2x.

#J-18808-Ljbffr