Check out the conversation on Apple, Spotify, and YouTube.
Brought to you by
Fingerprint - Identify good and bad visitors with industry-leading accuracy
Ariso - Ship AI agents and features faster, with fewer regressions
Customer.io - Send smarter messages using your product data
Land PM Job - My next cohort starts Monday! Join to learn AI job search from me
Product Faculty - Get $150 off their #1 AI PM Certification via code AAKASH150
Today’s Episode
I got a chance to sit down with the head of AI engineering at Salesforce.
Jayesh Govindarajan leads the teams behind the $190B SaaS giant’s AI products.
In today’s episode we cover:
Why he thinks the SDLC should be replaced with an ADLC, and what it looks like
The path to building an AI-native product team at Salesforce scale
What Salesforce looks for when hiring PMs
This was recorded live at Dreamforce, so it’s a bit different than my typical podcast. Should I do more like this?
1. Why the SDLC should be replaced with an ADLC
We’ve all lived the SDLC, software development lifecycle. For most of us, it looks something like plan it, build it, test it, ship it, look at the results.
The big change with AI is that every few months, a new model ships with new capabilities. For the past couple years, that’s meant a lot of progress on long-running agentic tasks.
What this means for product builders is that you need to extend the capabilities of the agent you built. You need to tune how it serves the existing capabilities, and you need to keep doing that.
Jayesh noticed that his teams would go back and rework every agent they built, often simplifying it because the new model can do more on its own.
That’s pretty different from prior tech we built on. Did you ever have to dedicate sprints to SQL getting an upgrade? It didn’t happen often, right?
And so that’s why we need a new software development lifecycle, which Jayesh calls the ADLC:
The reason you have to keep rebuilding the agent is because of drift.
The agent works. For instance, an agent answers refund questions grounded on the refund policy doc.
But then someone rewrites the doc, the update isn’t clear, and the code doesn’t change. As a result, the refunds come out too big.
That’s a big problem. So then you have to detect it.
You have to figure out what context changed, and whether it changed the right way or the wrong way. Then you have to correct it in production.
So the ADLC is built to accommodate that. We go from a linear process to a continual iteration.
Of course, we all sort of had a continual iteration, but here Jayesh is just codifying it. We go from observe, build, test, evaluate, deploy.
And so now we just continuously expect to rebuild our products.
That’s also why we’re so picky about what we put into our product, because we know it’s going to have a high maintenance cost when it’s an AI feature.
We’re going to spend a ton of money, time and energy on evaluating and observing that well.
So evals become incredibly important, and we need to embrace that continuous shipping methodology.
Every AI feature you ship is one you’ll keep rebuilding. Budget for evals and observability before you budget for the build.
2. The path to building an AI-native product team at Salesforce scale
One of the things I consistently see from my readers is that you guys are having to work on building AI-native product teams. You’re being asked to do that.
You’re the type of person who reads this newsletter. So I spent a lot of time with Jayesh understanding how he has rebuilt how 700 people work inside of Salesforce.
We broke the path down into 6 steps:
Step 1. Give teams a choice of tools
This is one of the most important takeaways I had from our conversation: execs shouldn’t pick the winner.
The goal for product leaders is to let teams choose. Do they want to use Codex?
Do they want to use Claude Code? Do they want to use Cursor?
That’s how Jayesh’s teams moved from Cursor to Claude Code without a mandate.
As he watches the performance of the best, he helps others learn from that. But he is enabling all tools.
Step 2. Make Slack the operating system
I think this is something a lot of people attempting this sort of tooling shift miss.
You want to find a tool where you are always available asynchronously. For many people that is Slack.
Slack can become the home for your company OS Slack bot. It can even become the home for user feedback from your customers, with Slack Connect.
In Slack, PMs can get feedback on a prototype directly from a customer.
Step 3. Make the agent deterministic
One of the things you’ll encounter when you build these is there are some points where you don’t want to have two answers.
In the case of Salesforce’s agents, for instance, they do not want to have two numbers for pipeline. You can’t have one say 25 million and one 20 million.
That’s going to be a big problem.
So they use something called Agent Script, which allows them to identify which parts of the agent should have deterministic answers.
This was a really interesting nuance he brought up that I think other builders might not have. It points to the importance of getting this question answered:
Where should I be using LLM AI? When should I be using determinism inside of my AI feature?
Step 4. Become eval first
Jayesh told the story of how his data science and engineering teams differed.
Data science teams were already eval native. Because they came up through the era of predictive AI, they always expected a model to be wrong some of the time and wrote tests against outcomes.
But most of his engineers, on the other hand, came from the deterministic world, where a test passes or fails. Those are the teams us PMs need to help become eval first.
What it means to be eval first is that you decide how you’ll grade the agent before you trust it. Essentially, each eval checks something.
And generic evals really don’t cut it. So as a PM, you develop or bring in the subject matter expertise to really win in the domain you’re delivering an agent for.
Step 5. Wire feedback to the roadmap
For Jayesh’s org, this means that every forward deployed engineer posts to one Slack channel. Each post is tagged as a paper cut or a feature request, and PMs match it against the roadmap.
When they do build something specific for a customer, they ask, should we be building this for everyone?
This allows you to learn directly from the field in order to improve your product.
Step 6. Roll out one win, then go wide
This is the classic pilot advice, but I was asking Jayesh where teams go wrong, and he brought it up as super important.
You have to find that area where you are going to show value.
Once you do that, then you go wall to wall with this sort of an ADLC instead of your normal SDLC.
3. What Salesforce looks for when hiring PMs
I’ve written about this a lot here. AI PMs have two skills that a regular PM doesn’t: AI prototyping and AI evals.
Jayesh agreed that those are the two most important, but when he’s screening, here’s what he’s looking for first:
1. Agency
This is actually the thing Jayesh finds in short supply, and I probably agree when hiring on my own team.
You want people who are high agency, and so that’s what he looks for first.
2. An agent-native mindset
Are you the type of person who, after doing a task a few times by hand, then teaches an agent to do it? That’s what he’s looking for.
3. Skepticism
You want to love the power of AI, but at the same time, you can’t believe everything.
This means you’re checking every output before you’re shipping it.
4. Craftsmanship
Craftsmanship applies to the product as much as the docs. You need to be able to sign your name on absolutely everything that lives in the product and the docs that you send out.
In particular, Jayesh finds red flags in people who send prototypes that look like every other prototype, or long docs that are just sent as a checkbox.
5. Setting the bar
This is probably the most important area for PMs, always. Can you say what good looks like?
Because if you can, that’s how you set up the evals and the test cases that we talked about being so important.
A good PM is an AI PM who understands AI prototyping and AI evals and brings these five character traits.
So if you want to get into Salesforce, where PM pay runs from $194K to $487K, with a median around $290K, this is what you’ll need.
I noticed they have a Senior Agentic Product Manager job open right now that’s really good.
If you reply to this email with your pitch to Jayesh for the role, I’ll make sure he sees it.
Going after a role like this? My Land a PM Job cohort starts next week.
I give you 2 one-on-ones, live negotiation and one-on-one coaching. That in and of itself makes it worth the cost.
Plus, I teach you how to apply to jobs with AI and connect with people automatically every day. Join the cohort here.
And that’s it for today’s episode.
To get my AI tool stack ($27,371 in tools, including Dovetail, Linear and Magic Patterns), become an annual subscriber ($150) and grab Aakash’s bundle.
To get my AI PM products (PM OS, Job Search OS, Prompt Library and quarterly product drops), become a founding member ($250).
Where to find Jayesh online
Dreamforce did pay for my travel to the conference, but they did not make any requirements about what content I post. In fact, they were kind enough to provide the entire recording space and access to Jayesh for this episode. So thanks to Salesforce!
If this was useful, take a moment to follow us on Apple, Spotify and YouTube! Your support helps me, and the team, keep bringing you high quality resources for free.
