Job at a glance
About Invoca: We built the category that made conversations measurable. Now we're building the AI agents orchestrating those engagements. Invoca is the revenue infrastructure connecting marketing, commerce, and contact center teams — turning every buyer conversation into revenue, and attributing it all the way back to the media spend that drove it. Our multimodal AI agents operate at the moment of decision: engaging buyers across the journey with personalized interactions at scale for life's most considered purchases — without losing the empathy, explainability, and trust those decisions demand.
Every engagement is transparent and attributable — brands see not just what converted, but why, and what marketing dollar earned it. Come work on the layer where AI agents and real revenue meet. The Opportunity As an Agentic AI Analyst at Invoca, you will play a pivotal role in shaping and refining the emerging category of agentic AI applications. Within our Product organization, you'll collaborate closely with a cross-functional team, including Product Managers, Engineers, Data Scientists, and Customer Success, to identify, evaluate, and operationalize agentic use cases that directly improve customer outcomes and advance Invoca's strategic AI vision.
A core focus of this role is the integration layer that makes agents useful in the real world: designing and validating the tools agents call (via MCP and similar protocols), authoring the skills that shape agent behavior, building the evaluation harnesses that measure whether agents actually work, and running simulations that stress-test agents before customers ever touch them. This is an opportunity to grow your product skillset at the intersection of customer needs, agentic AI system design, and real-world AI performance.
You will contribute to building new features, evaluating agent performance in production, and distilling insights from field engagements and system data to inform product priorities. You should bring strong problem-solving ability, curiosity around emerging AI capabilities, and confidence working across stakeholders to translate ambiguity into clarity and execution. What You'll Do Design and Validate Agentic Integrations (MCP Tools): Define the tools our agents use to read and act on customer data.
Author tool specifications (names, descriptions, input/output schemas, error behavior), prototype MCP servers and tool integrations, and validate that agents select and invoke tools correctly across realistic scenarios. Partner with Engineering on tool ergonomics that reduce hallucinated calls and improve task completion. Author and Maintain Agent Skills: Design, write, and maintain reusable skills: structured instructions, workflows, and domain knowledge that govern how agents approach specific jobs.
Test skill triggering accuracy, iterate on instructions based on eval results, and manage a versioned library of skills for reuse across products and customers. Build and Run Agent Evals: Design evaluation frameworks for agentic behaviors, including golden datasets, task-level success criteria, grading rubrics, and LLM-as-judge pipelines. Run evals across model, prompt, tool, and skill changes to catch regressions, quantify improvements, and inform release decisions.
Report results in a way that both Engineering and GTM can act on. Simulate Agent Behavior Before Production: Build and operate simulation environments (synthetic callers, mock tool backends, adversarial and edge-case scenarios) to exercise multi-step agent workflows at scale. Use simulation results to uncover failure modes, calibrate human-in-the-loop handoff points, and establish confidence thresholds before beta and GA.
Design End-to-End Agent Behavior: Own how an agent reasons through a task from start to finish, including planning logic, state management across steps, escalation and human-in-the-loop decision points, and fallback behavior when tools or data are unavailable. Validate Use Cases with Customers a