Check out the conversation on Apple, Spotify and YouTube.
Brought to you by:
Kameleoon: Leading AI experimentation platform
The AI Evals Course for PMs & Engineers: You get $800 with this link
Vanta: Automate compliance, manage risk, and prove trust
Amplitude: Try their 2-min assessment of your company’s digital maturity
Product Faculty: Product Strategy Certificate for Leaders (Get $550 off)
Today's Episode
What do PMs need to know about AI agents?
That’s the question we break down from every angle with today’s guest.
This is another special in-person episode, shot in San Francisco with Armand Ruiz, VP of AI Platform at IBM.
He has been in AI for 16 years and has become one of the most-followed AI voices on LinkedIn. He spends his time meeting with CIOs from the biggest brands who all have AI as their number one priority - and agents as one of their core components.
In our conversation, he breaks down:
What you need to know about Agents
Why RAG systems power 90% of enterprise AI
How product management changes when agents do the work
(By the way, we’ve launched our podcast clips channel, so don’t miss out: subscribe here.)
Your Newsletter Subscriber Bonus
For subscribers, each episode I also write up a newsletter version of the podcast. Thank you.
We've put together the complete guide on AI agents:
Understanding AI Agents
What makes agents different from chatbots
The four building blocks of every agent
Development frameworks that actually work
RAG Systems in Production
Why 90% of enterprise use cases start with RAG
What teams get wrong implementing RAG
Vision RAG for multimodal data
The Future of Product Management
How PM team ratios change from 1:6 to 1:30
From writing PRDs to building prototypes
1. Understanding AI Agents
1a. What Makes Agents Different from Chatbots
The AI landscape shifted fundamentally when we moved from conversation to completion.
The core difference:
Chatbots respond to what you ask
Agents complete what you need
Consider this comparison:
Traditional: "What should our pricing strategy be?" Agent: "Research our top 10 competitors' pricing, analyze our customer feedback on price sensitivity, and create three pricing strategy options with projected impact."
The first gives you advice. The second delivers completed work.
"We got into this chatbot era, but agents really deliver the wall of automation that is going to unlock everyone and businesses to generate way more output."
1b. The Four Building Blocks Every Agent Needs
Armand breaks down every successful agent into four sequential phases:
1. Thinking - The reasoning layer where LLMs excel.
"That thinking step takes extra compute but gives you the chain of thought process that we used to do manually."
2. Planning - Transforms thoughts into actionable subtasks.
3. Action - Actual execution in real systems. Whether inputting data to a CRM or "interacting with information in a system like Workday."
4. Reflection - The learning loop that separates good agents from great ones.
"Reflection is what makes agents really good. Maybe at first they're raw, but with human input they iterate and become better over time."
1c. Development Frameworks That Actually Work
The framework landscape splits into two paths:
For Enterprise Implementations Requiring Sophisticated Control
You need coding frameworks that give precision for complex systems:
LangGraph - Dominates with extensive documentation
CrewAI - Focuses on multi-agent collaboration
LlamaIndex - Excels for data-heavy applications
Trade-off: These require Python knowledge but offer enterprise-grade control.
For Rapid Deployment and Experimentation
No-code platforms accelerate development:
Lindy - Recently impressed Armand as "very impressive"
n8n - Popular workflow automation platform
IBM's Langflow - Visual agent building
Benefit: Handle technical complexity while you focus on concepts.
Choose your framework based on complexity needs, not technical comfort.
2. RAG Systems in Production
2a. Why 90% of Enterprise Use Cases Start with RAG
RAG solves the fundamental limitation that makes LLMs powerful yet incomplete for business use.
While these models contain vast knowledge, they're frozen in time at training cutoff and lack access to your proprietary data.
"90% of the use cases we were doing for enterprises were RAG use cases" in Armand's first year after ChatGPT's release.
Businesses need AI that can:
Access current policies for customer service
Tap institutional knowledge for internal systems
Analyze feedback across multiple platforms
"It's a gold mine for most traditional companies that are sitting on a lot of valuable data."
Instead of searching documents, employees ask: "What were the key customer complaints about our Chicago launch?" and get synthesized answers from all systems.
2b. What Teams Get Wrong with RAG
The biggest RAG failures stem from misaligned expectations around accuracy rather than technical implementation challenges.
"The conversations I have with customers are frustrations about accuracy," Armand observes.
"In the consumer space, a little lack of accuracy is acceptable. But when we're talking about a customer service chatbot that needs to connect to RAG, it needs to be very accurate. 70% accuracy is not acceptable."
Teams typically make three critical errors:
Generic Templates - Applying vanilla solutions instead of customizing for specific use cases
Final-Stage Checking - Only evaluating outputs at the end instead of throughout the pipeline
Technical Mindset - Treating RAG as a technical problem rather than a data quality problem
"You need to build a strong practice to properly evaluate the outputs. It's really a data problem."
The solution requires building sophisticated evaluation practices that match business requirements.
This means:
Establishing evaluation frameworks at multiple pipeline stages
Not just the final output
Understanding that RAG accuracy depends more on data quality, retrieval precision, and evaluation rigor than on model sophistication
2c. Vision RAG for Complex Data
Traditional RAG processes text, but most valuable business information lives in visual formats that standard systems can't interpret.
"Vision RAG is taking classic RAG that is text-based and opening it up to multimodal scenarios."
This unlocks entirely new use cases across industries.
Key applications include:
Healthcare - Reading charts from medical equipment
Finance - Analyzing market visualizations
Research - Extracting data from papers with complex diagrams
IBM's DocLing framework exemplifies this capability - it extracts information from Word documents, PDFs, and PowerPoint files visually, then feeds that into RAG pipelines.
3. The Future of Product Work
3a. How Agent Technology Changes PM Teams
Product management faces a structural transformation that will reshape team composition and scope within the next five years.
Armand’s prediction?
"Usually the ratio I've seen is a product manager for six to 10 developers. I think with AI agents, we can get to a different ratio. Instead of one to six to 10, maybe one to every 20 or 30 developers," Armand predicts.
With agents handling research, analysis, and documentation, PMs can manage broader product surfaces simultaneously.
Agents can:
Process competitive analysis across hundreds of companies
Synthesize user feedback from multiple systems
Generate 80-90% of PRD content
Even build functional prototypes for user testing
PMs become orchestrators of AI systems rather than coordinators of human workflows.
3b. From Writing PRDs to Building Prototypes
Armand's career breakthrough illustrates why showing beats telling, especially when enhanced by AI capabilities.
Armand's career breakthrough illustrates the power of showing instead of telling.
"More than 10 years ago, I moved from Spain to the US. My English was rough. We had this big meeting with executives about reimagining the next machine learning platform. I was struggling to articulate my ideas, so I built a prototype. Everyone else was showing slides. I was the only one showing a product you could touch. I got to lead the project."
Traditional product development follows a linear path:
Write detailed specifications
Communicate vision through documents
Manage translation gaps between teams
AI-enabled development flips this model:
Build functional prototypes that demonstrate exact requirements
Iterate based on direct feedback
Eliminate communication overhead
"There is nothing that speaks better than a working prototype. Even if you write the most beautiful detailed PRD, a lot of information is lost in translation."
With AI tools, what previously required weeks of development can happen in hours.
PMs can:
Prototype ideas before writing specifications
Test concepts before committing resources
Communicate through working examples rather than written descriptions
The key is to balance rapid prototyping with deep customer problem discovery.
Start customer-first to understand problems deeply, then use AI tools to quickly build and test solutions.
Key Takeaways
Where to find Armand Ruiz:
Related Content
Podcasts:
Newsletters:
P.S. More than 85% of you aren't subscribed yet. If you can subscribe on YouTube, follow on Apple & Spotify, my commitment to you is that we'll continue making this content better.
