It’s an exciting week here in Cleveland as MAICON kicks off, and we have just as exciting AI news and updates to share in this weeks episode. Join Mike and Paul as they unveil SmarterX’s newest tool, ProblemsGPT. The guys also break down use cases of Generative AI productivity benefits, plus, hear Karpathy’s insights on the journey toward automated intelligence.
Listen or watch below—and see below for show notes and the transcript.
Listen Now
Watch the Video
Timestamps
00:05:15 — ProblemsGPT
00:19:11 — The ROI of Generative AI
00:29:20 — Karpathy Podcast
00:39:42 — $1B Round for Sutskever’s Safe Superintelligence
00:44:41 — OpenAI Subscriptions
00:50:58 — Salesforce Agentforce
00:57:28 — Claude for Enterprise
01:00:28 — Copyright Laundering
01:02:15 — Time AI 100
01:04:40 — Apple Intelligence
Summary
ProblemsGPT
Paul Roetzer and SmarterX have launched ProblemsGPT, a new AI-powered tool designed to help businesses identify and prioritize problems that could potentially be solved more effectively using artificial intelligence.
ProblemsGPT uses ChatGPT technology to help organizations define problem statements and create preliminary strategic plans for building an AI-forward business.
The tool’s approach focuses on known pain points or challenges that could be solved more efficiently and at scale with AI, rather than just looking at use cases.
The tool then helps you craft strong problem statements that include a value statement, establishing the worth of solving each problem.
It can be applied to various problem types such as audience growth, innovation, churn, costs, efficiency, forecasting, personalization, and more.
And, after helping refine problem statements, the tool can draft an initial strategic brief to accelerate planning.
The ROI of Generative AI
We have some more interesting examples in the wild of real companies using generative AI to achieve real productivity gains…
First, a weekly tech newsletter from Tanay Jaipuria (called Tanay’s Newsletter), provides a number of compelling recent AI case studies, including results from companies like Klarna AI, Salesforce Agentforce, Microsoft, Ebay, Walmart, Amazon and more.
Second, BCG has some new work that reveals that generative AI can significantly expand workers’ capabilities, allowing them to perform tasks beyond their current skill set.
The company’s Henderson Institute teamed up with Boston University and OpenAI’s Economic Impacts research team to do a study on how generative AI actually impacts work performance.
They found that using AI leads to immediate aptitude expansion. In their tests, workers with no prior coding experience achieved 84% of data scientists’ benchmark when using GenAI for coding tasks.
And GenAI-augmented participants performed 49 percentage points better than those without AI on coding tasks. They also found that GenAI served as a valuable brainstorming tool for complex tasks like predictive analytics.
Karpathy Podcast
Ex-OpenAI and Tesla AI researcher Andrej Karpathy just gave an interview on “the road to autonomous intelligence” on the No Priors podcast.
In it, he covered a ton of ground, giving takes on AI in self-driving cars, robotics, education, and more.
He also speculated that we may see the development of “AI ecosystems” in business that resemble entire companies—where various models work together to achieve specialized roles that create value.
Links Referenced in the Show
- ProblemsGPT
- The ROI of Generative AI
- Karpathy Podcast
- $1B Round for Sutskever’s Safe Superintelligence
- OpenAI is Exploring New Pricing Strategies
- Claude for Enterprise
- Salesforce Agenforce
- Copyright Laundering
- Time AI 100
- Apple Intelligence
This week’s episode is brought to you by MAICON, our 5th annual Marketing AI Conference, happening in Cleveland, Sept. 10 – 12. The code POD200 saves $200 on all pass types.
For more information on MAICON and to register for this year’s conference, visit www.MAICON.ai.
Read the Transcription
Disclaimer: This transcription was written by AI, thanks to Descript, and has not been edited for content.
[00:00:00] Paul Roetzer: You realize when you pair AI, like the right AI tools with the right use cases, we are talking about 50, 70, 90 percent gains in some cases. And I think that’s where some people get caught up and think it’s hype. It’s not. People just, they get overwhelmed by the hype if they’re not experimenting themselves.
[00:00:15] Paul Roetzer: Welcome to the Artificial Intelligence Show, the podcast that helps your business grow smarter by making AI approachable and actionable. My name is Paul Roetzer. I’m the founder and CEO of Marketing AI Institute, and I’m your host. Each week, I’m joined by my co host. and Marketing AI Institute Chief Content Officer, Mike Kaput, as we break down all the AI news that matters and give you insights and perspectives that you can use to advance your company and your career.
[00:00:50] Paul Roetzer: Join us as we accelerate AI literacy for all.
[00:00:57] Paul Roetzer: Welcome to episode 114 of the [00:01:00] Artificial Intelligence Show.
[00:01:00] Paul Roetzer: I’m your host, Paul Roetzer, along with my co host, Mike Kaput. It is MAICON week. You’ve been hearing us talk about MAICON. We actually, I was like, kind of regretting this last night, Mike. It’s like, so today it’s September 9th, 10 a. m. Eastern time. And I was like, why did we decide to do an episode MAICON week
[00:01:17] Mike Kaput: Yeah, the million other things we both have going on?
[00:01:20] Paul Roetzer: But, uh, yeah. Here we are. We are actually recording this the week of MAICON. As soon as we’re done with this, I am packing my stuff. Now MAICON’s in Cleveland, which is our hometown, so luckily I’m just driving downtown. but I said, but I’m packing and , , we are in it.
[00:01:36] Paul Roetzer: The rest of the team, I think, is already there. I know Tracy’s on site and the Event Ops team’s there. So it’s happening. Hopefully some of you are, uh, joining us this week. I know there’s a lot of podcast listeners coming to the, uh, conference. So if you’re hearing this on Tuesday the 10th and you wanted to do a last minute thing, it, it kicks off with afternoon workshops on the 10th and then those are optional and then the full [00:02:00] event starts, the the morning of September 11th at 8 AM with my road to AGI keynote, which I finished that deck finally, Mike, like yesterday.
[00:02:09] Paul Roetzer: Although I did a run, I may have to cut a couple of slides, I may have to get it down a few minutes, but, I think. think it’s going to be kind of shocking to some people, honestly.
[00:02:20] Mike Kaput: Nice.
[00:02:21] Paul Roetzer: Like, in a good way, but
[00:02:23] Paul Roetzer: I do think that there’s just As I’m like, I’m putting it together and it’s basically like building off of episode 87 of the podcast where we laid out like reactions to the Sam Altman quote about, you know, AI doing 95 percent of what marketers do.
[00:02:37] Paul Roetzer: and I think it just makes it very clear to people how close we are to truly doing what we’re doing. transformative experiences in marketing and business and society and educational systems. And like, as I was building the presentation, I was becoming more convinced myself of the timeline of the theory [00:03:00] behind like where this is going.
[00:03:01] Paul Roetzer: Um, not intentionally, like I was just trying to kind of lay out what I think is happening, but I don’t know. I think it’s going to be fascinating. And then I did end it with these kind of like 10 or 12 questions specific to the future of marketing that you and I are that going to do in the closing keynote on the 12th, um, bring it back around. And what Mike and I are going to do is try and synthesize it. Insights and findings from the 60 some other presentations and sessions, conversations with throughout, um, we’re going to try and bring that around to like 15, 20 action items everybody can do to prepare for the future, like prepare for what’s next.
[00:03:37] Paul Roetzer: So I don’t know. I’m, I’m getting excited. Like, I think when I’m building up for this event, I just stay so focused on the agenda, my own sessions that I don’t get caught up in everything else. And we have this insanely. Talented event operations team that just makes everything go otherwise. So I always joke, like, I don’t even know what the main stage looks like.
[00:03:55] Paul Roetzer: It’s kind of a surprise to me. Like people would think I I know this stuff, but I don’t. Like [00:04:00] I, other than seeing like a, a blueprint of where it’s going to be, I don’t actually know what anything looks like. So it’s kind of fun for me to get down there like Monday night, um, and start to actually like see the event, the venue and everything.
[00:04:13] Paul Roetzer: So that is happening. Mike and I are. Deep in that this week, uh, we will be there all week, but we will be back next week to do episode 115, and we’ll be talking about MAICON and some of the key takeaways from
[00:04:26] Paul Roetzer: that. So if you want to learn a little bit more about it, last minute, go to MAICON.ai. that’s MAICON.AI. and will be, an uh, an demand option that’ll go live. Probably like a week or so after the event, that’s going to have the 10 general sessions. And then I think there’s about five to seven more. So I think it’s gonna be about 15 to 17 or so
[00:04:47] Paul Roetzer: on demand sessions from MAICON So if you can’t make it and you want to, you know, catch up on all the featured talks and keynotes, that will be available as an on demand and we’ll share that information with you as we, have it.
[00:04:59] Paul Roetzer: So again, [00:05:00] MAICON. ai if you want to learn more about the event and, yeah, hopefully we’ll see some of you there and make sure to come up and say hi to Mike and I. We always love to. to interact with our listeners and get a chance to meet everybody.
[00:05:11] Paul Roetzer: All right, with that, Mike, let’s get into the topics of the week.
[00:05:15] ProblemsGPT
[00:05:15] Mike Kaput: Alright, Paul, so first up is a new tool that you have launched through SmarterX, the AI research and consulting firm you have started in addition to the Institute. And we’ve talked about a couple of these tools.
[00:05:30] Mike Kaput: in the past couple episodes. There’s a third one now called ProblemsGPT, which is an AI powered tool designed to help businesses identify and prioritize problems that could potentially
[00:05:42] Mike Kaput: be solved more effectively using AI. So ProblemsGPT uses ChatGPT technology to help organizations define problem statements and create preliminary strategic plans for building AI forward business. businesses. So the approach behind this tool kind of [00:06:00] focuses on identifying your known pain points or challenges that can be
[00:06:04] Mike Kaput: solved more efficiently and at scale with AI, not just looking at use cases like some of the other. Tools enable us to do, and then the tool helps you craft a strong problem statement that includes a value statement establishing the worth of solving each problem.
[00:06:20] Mike Kaput: And it can also then help you after that. Draft an initial strategic brief to accelerate aI
[00:06:27] Mike Kaput: transformation. So, paul, maybe to kick things off here, can you talk about why is this tool significant? Like, why is it needed now? And maybe walk us through, like, what is the methodology behind it? And like, how have you been using that?
[00:06:42] Paul Roetzer: Yeah. So the, you know, if anybody’s read our book, the Marketing Artificial Intelligence book, there’s a chapter dedicated to the two approaches we teach, the two frameworks we
[00:06:53] Paul Roetzer: teach to adopt AI in an enterprise. And so the first, which we’ve talked a lot about on the [00:07:00] show recently is the use case model.
[00:07:02] Paul Roetzer: That is finding the things you already do, the tasks you already do, and leveraging AI technology to do them more efficiently, to do, be more creative, more
[00:07:11] Paul Roetzer: innovative, more productive. So that’s where the jobs GPT came from and the campaigns GPT that we’ve talked previously on the show about. So those were meant to help you find tasks that you already do and integrate AI to do them smarter, better, faster.
[00:07:28] Paul Roetzer: The second framework that we teach in the book and that we use in our workshops, and that’s actually the forcing function for launching this one today, is, uh, Strategic AI Leader Workshop that I teach and that I’m running at MAICON on Tuesday, teaches the problem based model. Now in the problem based model, I think of more as like the director level and above approach.
[00:07:48] Paul Roetzer: This is, You have goals, you have pain points, you have challenges in the organization, and we all have them, and they can be related to audience, awareness, customer churn, revenue growth, innovation, whatever [00:08:00] your, your challenges are, whatever your domain is. If you’re the CEO, it’s all of these things. And you’re, you’re looking at these problems differently.
[00:08:07] Paul Roetzer: So by understanding what AI is capable of, you start to look at problems in a different way and say, maybe there’s a smarter way to solve this. We’ve been trying to solve this problem for
[00:08:17] Paul Roetzer: months, for years, for decades in some cases. Maybe we should take a first principles approach to
[00:08:22] Paul Roetzer: this and revisit. How we think about solving this problem. And so that’s what the problem based model teaches. And that’s what our workshop
[00:08:29] Paul Roetzer: teaches is it kind of goes through this framework of how to define a problem statement, as you mentioned, make sure you have a value statement.
[00:08:36] Paul Roetzer: So you go through and you identify all the challenges you’re facing in your organization.
[00:08:40] Paul Roetzer: And then what is it worth to solve these challenges? Because what then happens is you, you may have a list. And so in our workshop, we’ll actually give people like a blank template, and we’ll go through and come up with all the different pain points you’re dealing with around these different areas. And then you want to be able to prioritize them.
[00:08:57] Paul Roetzer: by ranking them. And one of the best ways to do [00:09:00] that is, well, what is the value for us to solve this? And so, you know, give you a few examples. So, uh, an example problem statement for audience. The problem is we saw a dramatic spike in customer churn last quarter, resulting in a monthly recurring revenue drop for the first time in two years.
[00:09:15] Paul Roetzer: The value statement to solve 80, 000 per month or 680, 000, 960, 000 annualized. So that’s a problem statement.
[00:09:27] Paul Roetzer: The second option, we’ll do an innovation one. So, Our growth in existing verticals has stalled and we need to identify new markets and product ideas that can unlock massive value for the organization.
[00:09:38] Paul Roetzer: Great. That’s a, that’s a solid problem statement. What’s the value? The value based on historical data and market research, we believe there are two new verticals that could generate 10 million plus each over the next three to five years. So again, problem, value, and then we’ll give you one more related to customer churn.
[00:09:54] Paul Roetzer: We saw a dramatic spike in customer churn last quarter resulting, oh wait, no, I did that one already. We won’t do that [00:10:00] one again. So, but you get the idea, problem, value, problem, value. And so what I did with problems GPT is I basically just trained a custom GPT to create problem and value statements. So it’s trained on a template of what that looks like.
[00:10:16] Paul Roetzer: And it’s given a bunch of examples so that it knows to do it. And then it’ll actually help people craft a proper problem and value statement.
[00:10:24] Paul Roetzer: So that’s the first thing is it can help you identify and craft these problem and value statements.
[00:10:29] Paul Roetzer: But then it actually is trained to help you develop a strategic brief. And this was a big, it was a big, unlock for me in building this because mike,
[00:10:38] Paul Roetzer: you, you and I have done these workshops together before. What we often do in these workshops is we teach the framework. And then we get people to develop a list of all these problem statements, but then, as we do in the book, we say, okay, then here’s the 10 steps to then go figure out what to do about it.
[00:10:56] Paul Roetzer: So in the workshop, we don’t actually give people [00:11:00] the strategic brief side. We’re not saying, okay, here’s the 10 ways AI can help you solve that problem.
[00:11:05] Paul Roetzer: So when I was building this, Custom GPT, I thought, I wonder if I can give it that ability. I wonder if I can just train it, how to write a strategic brief.
[00:11:12] Paul Roetzer: So like instantly they have a starting point.
[00:11:15] Paul Roetzer: And so I devised a way to do it. And the first time I tested, I was like, Oh my gosh, it actually works. Like this is genius, not me genius. Like the fact that it does this is genius because it accelerates so much. The value people can unlock and like how quickly you can move from having this pain point to like, Oh, wow, this is a really smart way to maybe do this.
[00:11:37] Paul Roetzer: And so the way it’s designed, you know, it’s again, it’s the way the problems GPT, and you can go to smarterx. ai slash problems GPT, and you can go see this for yourself. Um, You need to be logged into ChatGPT then to use it, but just click on the link and go there. It’s trained to identify and prioritize the business problems, but then once you lock in your problem, it’s actually trained to then develop the [00:12:00] strategic brief.
[00:12:00] Paul Roetzer: And so you can go there, and the three options are help me identify problems. So you can just say, Hey, I’m trying to figure out a problem, and it’ll walk you through it, and it’ll ask you questions and kind of guide you through developing a problem statement. Or if you already have a problem statement, You can just drop it in there and then it’ll start creating a strategic brief for you.
[00:12:18] Paul Roetzer: If you’re not sure and you want some inspiration, you can just click, show me some sample problem statements. And the thing I found here, and I didn’t even train it to do this. It’ll give you example problem statements. I think I taught it to give it like three at a time. So you can just say, give me more, give me more.
[00:12:32] Paul Roetzer: You can just keep saying that and it’ll just keep doing it. But what you can do is say, listen, I’m in the retail industry. Give me some example problem statements in the retail industry. And it’ll do it. And so what I found is it itself learns or had the ability kind of emergent within it. To do this in specific areas to you, you can do it by job, you can do it by vertical, you can do it by industry, company type, whatever you want, and it’ll actually do it, identify the problem statements, work with you on them, and then develop the strategic briefs.[00:13:00]
[00:13:00] Paul Roetzer: So, you know, I think that The way I think about this one, like I did with JobsGPT and CampaignsGPT is think of this as a consultant. Like, think of it like you’re talking to a really talented consultant who can help you develop these problem statements and then can give you real time guidance for how to think about the strategic side.
[00:13:17] Paul Roetzer: Now, the strategy output is not a final product. And you need to have expertise in your domain to know if the strategic output’s even any good. But what I have found is, it again, gives you instant feedback to what it looks like. And these briefs, the way I trained it to do it, is it restates the problem statement.
[00:13:36] Paul Roetzer: It then goes through theoretical issues and drivers, like what’s causing this problem. So it kind of teaches you a format that we teach in our framework, Mike. It then gives you strategic recommendations. It gives you a preliminary timeline for an implementation plan.
[00:13:51] Paul Roetzer: expected outcomes, metrics for success, and then what resources are going to be needed.
[00:13:57] Paul Roetzer: So every time you ask for a strategic brief, it actually [00:14:00] follows that template. So I gave it an example and its knowledge base and kind of trained it on that model. So ‘s also helpful just to think about how a strategic brief should be structured and this is what it, how it’s trained. So, yeah, I mean, like literally we launched this thing.
[00:14:15] Paul Roetzer: Mike helped me build the landing page over the weekend, uh, in between being at a wedding. Thank you, Mike, for knocking that out. and uh, and we launched it
[00:14:23] Paul Roetzer: on Sunday afternoons. I mean, this thing’s been an out in the wild for like less than 24 hours now.
[00:14:28] Paul Roetzer: Um, so a beta product. It’s built on ChatGPT.
[00:14:31] Paul Roetzer: It’s not going to always get everything right. It may hallucinate some stuff, but I’ll tell you like, Mike, we’ve, we’ve sat in these workshops for like three plus hours and people get a ton of value out of them, but they, they leave with a list of problems. And now it’s like, okay, now what do I do? And I mean, my own experience with this thing is like within three minutes.
[00:14:51] Paul Roetzer: You can have a bunch of problem statements identified, and you can start playing around with some ideas around strategic planning, and I don’t know, I mean, it’s just, it’s [00:15:00] still shocking to me that someone like me with no coding ability can just take a framework we’ve been teaching for like six years
[00:15:08] Paul Roetzer: and in a matter of, like, a couple weeks, just build out this GPT that all of a sudden just has all these abilities.
[00:15:14] Paul Roetzer: It’s wild to me. Like, we couldn’t teach this to a strategic, like, a senior strategist. Like, this is the kind of thing back when we ran the agency that we would be, like, trying to teach people the ability to do this. And here, it just took a framework and some knowledge of how to prompt this thing, and all of a sudden it just does it.
[00:15:30] Paul Roetzer: So. Yeah, smarterx. ai slash problems GPT, or you can just go to smarterx. ai and click on tools. They’re all there. give it a try. I don’t know. I’d love to hear people’s feedback. It’s, you’ve played around with it, Mike.
[00:15:42] Paul Roetzer: Like, what did you think? You’ve been through it on both sides, like the manual side of trying to teach people this and now having a tool that does it. [00:16:00] Yeah,
[00:16:08] Mike Kaput: at scale, real magic happens. We’re not just saying like, Hey AI, go think for me about a problem
[00:16:16] Mike Kaput: using generic data training sets, like very specific to what we have seen work and what we know works. And. that’s where this is really opening my eyes to, especially from a
[00:16:26] Mike Kaput: consulting perspective,
[00:16:28] Mike Kaput: anyone who has taught anything in any context, this is definitely something to be exploring, I would say. Yeah.
[00:16:38] Paul Roetzer: of this, like, no code, future where anyone can build anything they can imagine. And this to me is just like
[00:16:47] Paul Roetzer: a very early window into what that is going
[00:16:49] Paul Roetzer: to look like. And I think that’s a future that’s coming very quickly through the work Replit and like Microsoft co pilots. I got to imagine, I mean, OpenAI hasn’t made changes [00:17:00] to custom GPT since they announced them last year. Like there’s gotta be a version 2. 0 coming and maybe it’s just a smarter model. I don’t know.
[00:17:07] Paul Roetzer: but. I mean, yeah, as someone who ran a marketing agency for 16 years, I I could sit here and probably list dozens of.
[00:17:17] Paul Roetzer: Frameworks, processes, things that we were doing manually that would take dozens or hundreds of hours sometimes that we could probably build custom GPTs for now that would like deliver immediate value to clients. And now what does that mean to
[00:17:34] Paul Roetzer: your financial model as an agency? Like that’s a whole nother ballgame. That’s why we have our AI for Agency Summit is to we have an event
[00:17:40] Paul Roetzer: for that? But, But, But, um, yeah, I mean, I would just encourage anyone. That, uh, if you haven’t built a custom GPT yet, or, you know, now you can do Google Gems with Gemini and Anthropic has projects, but like really take the one or two or three things you do over and over again, the frameworks you go [00:18:00] through, the processes you have in place and experiment with building one yourself.
[00:18:03] Paul Roetzer: It is, it is a very enlightening experience and it, I mean, this is the third one I’ve built. And I texted Mike. I was like, dude, I’m, I’m just like shocked that this works. Like I’m still having a hard time processing that we have this ability to now build these things. and they just work like on first trial.
[00:18:21] Paul Roetzer: It’s, it’s a wild future we’re heading toward.
[00:18:24] Mike Kaput: Yeah, it’s really incredible. And yeah, you mentioned like Claude artifacts slash projects. Like it’s incredible. I’ve like built several apps already that like, you know, not like selling them to
[00:18:34] Mike Kaput: anybody, but it’s like, I don’t know how to code and react. Like it’s, and you can literally iterate right in a window
[00:18:40] Mike Kaput: On
[00:18:41] Paul Roetzer: With words, like it’s, yeah, it’s just like, you just try and figure out a way to say it to the AI. So it understands what you’re trying to say. And I think next week we might dive deep into this new prompting video we were talking about, Mike. And it’s, it’s
[00:18:54] Paul Roetzer: just wild to me, like how human language is becoming the, the, the, [00:19:00] the you code. I don’t know. That’s really cool. It’s exciting.
[00:19:11] The ROI of GenAI
[00:19:11] Mike Kaput: is we have a few more examples of real companies using generative AI to achieve real productivity gains. So a few things have come out where some people have collected
[00:19:22] Mike Kaput: A couple of interesting examples I think are important. to relate. So, first, there was a, weekly tech newsletter from Tane Jaipuria called Tane’s Newsletter, which we’ll link to in the show notes. And they list out a number of compelling recent AI case studies. So I’m just going to go through these really quick, because I know that we have tracked a few of these individually, but I think hearing all these together is really compelling.
[00:19:47] Mike Kaput: So first, Klarna, which is a payment company. They have an AI assistant that replaced 700 employees in terms of reducing the
[00:19:57] Mike Kaput: work of 700 [00:20:00] employees, reducing resolution times on customer tickets from 11 minutes to two minutes while maintaining customer satisfaction. We’ll talk about this one again in a second, but Salesforce is AgentForce platform is showing double digit increases in customer satisfaction
[00:20:17] Mike Kaput: and case resolution rates. Microsoft expects to save hundreds of millions annually on call centers from adopting generative AI. Amazon’s Q Assistant saved what they are terming 4, 500, 4, 500 developer years worth of work, that it’s not a typo, and
[00:20:38] Mike Kaput: 260 million dollars in costs for a recent code migration task. eBay has an AI assisted selling flow that led to a 20 point increase in customer satisfaction. Rocket Mortgage has an AI assistant that transcribes calls and completes mortgage applications. It literally saves [00:21:00] bankers from inputting millions of data fields weekly.
[00:21:04] Mike Kaput: And a final example they mentioned in the newsletter, https: thebusinessprofessor. com Walmart used generative AI to improve over 850 million pieces of data in its product catalog.
[00:21:15] Mike Kaput: Now, on top of this, the consulting firm BCG also released some research around how GenAI can significantly
[00:21:24] Mike Kaput: expand worker
[00:21:26] Mike Kaput: capabilities. So they teamed up with Boston
[00:21:28] Mike Kaput: University and and OpenAI’s Economic Impacts Research Team to see how GenAI is actually impacting
[00:21:36] Mike Kaput: work performance. So,
[00:21:37] Mike Kaput: some of the notable things they found here is that
[00:21:40] Mike Kaput: AI leads to immediate, what they call, aptitude expansion. So basically in their tests, workers with no prior coding experience to our previous point achieved basically an 84 percent score on a benchmark that data scientists were using for a coding task. So you got like 84 percent of [00:22:00] the way there with no experience by pair programming with Gen AI. Gen AI augmented. Uh, participants in this work performed 49 percentage points better on those benchmarks than people without AI. They also found Gen AI served as a valuable brainstorming tool for complex
[00:22:20] Mike Kaput: tasks, And they kind of summed this up really nicely with this quote, they said,
[00:22:25] Mike Kaput: quote, a new type of knowledge worker is entering the global talent pool. This employee augmented with generative AI
[00:22:33] Mike Kaput: Can write code faster, create personalized marketing content with a single prompt, and summarize hundreds of documents in seconds.
[00:22:40] Mike Kaput: of Paul, I confess, like, this is like a near and dear topic to me because like I feel like I get heat online sometimes for people being like super skeptical that you can do anything. To achieve real world results with generative AI, which I get, there’s a lot of hype out there. I’m sure I contribute to it at times.
[00:23:00] Mike Kaput: It can’t do everything and it has limitations, but like, the benefits of this technology have been so immediately obvious in our own work that I’m just like having a, I have a hard time understanding the skepticism.
[00:23:10] Mike Kaput: Like, are you seeing that too? Is that how you look at stuff like this?
[00:23:13] Paul Roetzer: Its interesting, I saw Ethan Mollick tweeted this morning
[00:23:17] Mike Kaput: Hmm. Mm
[00:23:18] Paul Roetzer: he’s seeing is, the
[00:23:21] Paul Roetzer: companies where the C suite or the executives themselves are active participants in experimentation with LLMs are the ones that are racing ahead. The companies where the C suite isn’t using or has just tried it and like, you know, didn’t
[00:23:34] Paul Roetzer: see the, you know, what value was, are the ones that are lagging behind.
[00:23:38] Paul Roetzer: And I think that’s insanely true. Like, I really don’t understand how, if you invest the time in experimenting with these models,
[00:23:46] Paul Roetzer: You don’t connect the dots of the value you could be creating. And these examples that Tanja gave,
[00:23:53] Paul Roetzer: who the way is a partner at Wing Ventures, a VC firm that invests in early stage AI companies, um, he’s pulling [00:24:00] excerpts from earnings calls.
[00:24:01] Paul Roetzer: So this, and it’s interesting from an AI perspective, this,
[00:24:04] Paul Roetzer: I wasn’t following this guy. It 4U feed on Twitter. Like that was how I came across it. And then you’re like, ah, like whatever. It’s just another Twitter thread. Then I just
[00:24:14] Paul Roetzer: did a little digging and I was like, no, this guy’s got some like really valuable stuff in his feed.
[00:24:18] Paul Roetzer: Like he posts really good stuff and it’s like research based. It’s coming from good sources. So that was why I kind of flagged it for us to talk about. And then for you and I, Mike, like, You’ll remember this, like about four or five years ago, I
[00:24:30] Paul Roetzer: think it was either Mike or Tracy on our team, I said we need to build a database of brands actually using AI.
[00:24:37] Paul Roetzer: Because at that time it was really hard to find companies that were getting value from AI. And so we, we went out and search and we created a, it was just a Google sheet where we were tracking stories of brands that had actually had success with, with AI. And so that was when I saw this tweet, I was like, man, times have changed.
[00:24:56] Paul Roetzer: Like it, it’s still challenging sometimes to [00:25:00] find like great case studies, because I think a lot of people who are having success
[00:25:03] Paul Roetzer: just aren’t
[00:25:03] Paul Roetzer: talking about it intentionally from a competitive standpoint. But I thought these are all really practical examples. And the thing I liked is. In a lot of ways, it’s innovation or expansion of capabilities over replacement of human
[00:25:16] Mike Kaput: Right.
[00:25:16] Paul Roetzer: And so doing things like the 850 million, you know, the product example one for Walmart, you just wouldn’t do that. Like you wouldn’t hire humans to do what they did. It just wouldn’t happen otherwise. And so there’s this balance between automating, Tasks that already exist. The 20 things I do for my job, the 50 things we do for these five campaigns, like the known things, and then there’s looking at your work, your company and saying, what are all the things we’re not doing that AI
[00:25:47] Paul Roetzer: now enables?
[00:25:48] Paul Roetzer: And that’s why I love this, you know, these examples that he pulled out from these earnings reports, um, is it’s just really practical stuff that’s showing the potential for this, for AI to create value in all [00:26:00] these new ways. and the other thing I’ll say is. I think a lot of times people get caught up in these like finding 10, 20 percent productivity increases or efficiency gains.
[00:26:11] Paul Roetzer: And then you look at examples like this and you realize when you pair AI, like the right AI tools with the right use cases, we are talking about like
[00:26:20] Paul Roetzer: 50, 70, 90 percent gains in some cases. And I
[00:26:23] Paul Roetzer: think that’s where some people get caught up and think it’s hype. It’s not. Like if If you apply this to the right thing, like just think about our problems, GPT model,
[00:26:32] Paul Roetzer: That is honest to God, a thing that would take dozens of hours that you can do in three minutes, but get, get an initial.
[00:26:40] Paul Roetzer: Initial run, initial draft in three minutes. We are talking about hundreds of percentage of increases in productivity gains. So
[00:26:47] Paul Roetzer: it’s, yeah,
[00:26:48] Paul Roetzer: I think people just, they
[00:26:50] Paul Roetzer: get overwhelmed by the hype if they’re not experimenting themselves. When you’re in this and you’re finding that one use case you can get excited about or you’ve [00:27:00] seen it,
[00:27:01] Paul Roetzer: applied to something you were doing and you realize the potential of it, then it just becomes this game of looking at everything and matching AI with the things that are going to create the greatest value for you.
[00:27:12] Paul Roetzer: And that’s not hype. That’s just a practical approach to applying AI to your business. And I think there’s a massive Opportunities still for professionals and business leaders and companies that
[00:27:25] Paul Roetzer: do this in a strategic way, that take the use case model and the problem based model, that you’re running these things simultaneously, you’re looking at both ways to be more innovative and solve problems in a smarter way, and you’re being more efficient and productive in your tasks, like companies that do both of those things and build a roadmap for how to do it well over the next, you know, one to two years, they’re just going to leap ahead of their peer peer groups.
[00:27:46] Mike Kaput: Yeah, it’s really fascinating to kind
[00:27:48] Mike Kaput: of consider like, oh, okay, well, where do all these perspectives come from? And I also wonder too, we’ve talked about this a bit, how much of the skepticism is people approaching this [00:28:00] like traditional software, where they
[00:28:01] Mike Kaput: try one prompt, It didn’t work. They tried it a year ago on a much inferior model. It didn’t do exactly what they wanted or what they said it could do. And that’s kind of really where they stopped. Like it’s, well, okay, let’s look at the prompt. Let’s look at the model. Did you talk with it, coerce it into
[00:28:20] Mike Kaput: doing what you wanted? Did you give it examples? I mean, I’ve literally had night and day different results with pretty basic tasks by changing up that approach.
[00:28:28] Mike Kaput: So just validates everything.
[00:28:30] Paul Roetzer: I think for a lot of companies, like a lot of business users, their only experience is Copilot. And the reality is, from everything we’ve seen, Like Copilot within, you know, Microsoft, it’s just not at the level of using a Claude or a ChatGPT outside of that environment. And, and, And, your common use cases there are like rewrite my email or write this draft or create this presentation.
[00:28:55] Paul Roetzer: And I get how you could have an underwhelming experience if that’s all you’ve ever done [00:29:00] with a large language model is. TestCopilot or Gemini within Google Workspace. Like, it could be underwhelming,
[00:29:07] Paul Roetzer: but get, get outside of those domains and really push ChatGPT or Claude or, you know, Gemini outside of, you know, the productivity platforms that you use at work every day.
[00:29:17] Paul Roetzer: And I think you’ll find a totally different experience.
[00:29:20] Karpathy Podcast
[00:29:20] Mike Kaput: All right, so our third big topic this week, Andrej Karpathy, who we have talked about many times, ex OpenAI, Tesla AI researcher, just gave an interview about the road to autonomous intelligence on the NoPriors podcast.
[00:29:36] Mike Kaput: And in it, he covered a ton of ground in like 45 minutes. He’s just brilliant
[00:29:42] Mike Kaput: and giving all these great takes on AI for self driving cars, robotics, education, and much more.
[00:29:50] Mike Kaput: Interestingly, he also speculated we may see the development of kind of AI ecosystems in business where you essentially have like an AI [00:30:00] organization that resembles a company where various models work together to achieve specialized roles that create business. business. values. So definitely getting into like the future as now in this conversation.
[00:30:12] Mike Kaput: So Paul, you had flagged this and said you found it really notable and valuable. Like what jumped out at you here as particularly worth paying attention to?
[00:30:23] Paul Roetzer: This fits in that realm that Mike and I talk a lot about of, you know, listen, when, the leading AI researchers and entrepreneurs talk. Listen,
[00:30:31] Paul Roetzer: like there’s
[00:30:32] Paul Roetzer: always pieces of information within these interviews where you learn something new. It validates something you already believe to be true. but but if nothing else, you get a window into where these leading minds are and what they think is going to happen
[00:30:47] Paul Roetzer: next. So
[00:30:47] Paul Roetzer: again, as I’ve always said, anytime Sutskever or Ilya or Dario Amodei or Demis Hassabis or Sam, like, I’ll listen every time because there’s always something in that, in that interview. [00:31:00] So a couple of things I found interesting,
[00:31:02] Paul Roetzer: um, for me personally, Tesla versus Waymo, he spent a lot of time on this.
[00:31:06] Paul Roetzer: So if you don’t recall, he ran AI at Tesla for five years. He was in charge of building, uh, self driving at Tesla. So he knows. Ins and outs of that system, probably better than anybody in the world. And if you’ve been in a Waymo, or if you’ve been on the West Coast,
[00:31:21] Paul Roetzer: you’ve maybe seen a Waymo driving around without a human driver.
[00:31:24] Paul Roetzer: I was recently in Scottsdale, and that’s like epicenter of Waymo right now. That and then, out in out Silicon Valley. San Francisco, these
[00:31:32] Paul Roetzer: things are driving around everywhere. Well, the Waymos have these like huge contraptions, big piece of hardware on them that’s enabling them to drive without a driver on these city roads.
[00:31:42] Paul Roetzer: Tesla has gone the opposite direction. Tesla’s computer vision only. It’s got eight cameras. Those cameras have the ability to see the roads, see objects, and that’s how they learn and drive. Now I have had a Tesla. I’m on my third one, so I’ve had it for seven years and I’ve been monitoring the full self-driving all the way along.
[00:31:59] Paul Roetzer: If you’ve [00:32:00] read our book, I actually have a whole section of a chapter dedicated to how these models learn and why matters.
[00:32:07] Paul Roetzer: he, he said that, he actually, some people don’t believe this to be true, he thinks Tesla is way ahead of Waymo. Even though Waymo actually has cars driving people like taxi cabs basically,
[00:32:18] Paul Roetzer: or Ubers without drivers, um, he feels Tesla way ahead because.
[00:32:23] Paul Roetzer: He thinks Tesla only has to solve a software problem and that that is a very tractable thing to solve.
[00:32:29] Paul Roetzer: He feels that Waymo, which still has these really expensive devices, LiDAR and radar and all these cameras, that they have a massive hardware problem to solve. And that’s a much harder thing to get to.
[00:32:41] Paul Roetzer: So I just, I thought he had a really interesting perspective. And then he shared something I hadn’t heard before, which is Tesla does use LiDAR and they use all these things. Only in their training sets. So they have cars that have the LiDAR to go do all this mapping, but then the actual vehicle itself doesn’t have those [00:33:00] expensive systems.
[00:33:00] Paul Roetzer: And so they’re able to scale by focusing more on the training side and investing there and then like, Diluting that, u what’s the term? Dilating, diluting, whatever. Dilation, they call it. Taking it down to these, like, smaller models.
[00:33:15] Mike Kaput: Oh, okay. Yeah.
[00:33:17] Paul Roetzer: The other thing I thought fascinating is, uh, Transformers.
[00:33:20] Paul Roetzer: So we know Transformers are the basis for ChatGPT and large language models. Transformer, invented by the Google Brain team in 2017 with the Attention is
[00:33:29] Paul Roetzer: All You Need paper. He is very bullish on the Transformers. So he talked about these things
[00:33:34] Paul Roetzer: as being largely underappreciated for how well they have scaled over these seven years.And, with
[00:33:41] Paul Roetzer: very little change. And so while there’s some in the AI research community that think transformers are going to kind of like run up against these data walls and have other issues, he seems extremely bullish on transformers and the potential to continue building on top of them. He talked a lot about data.
[00:33:59] Paul Roetzer: And the [00:34:00] one thing I thought was really interesting is on episode 113 last week,
[00:34:04] Paul Roetzer: we talked about this idea. that the internet, which is where a lot of the training data comes for these models, is full of outputs. And what it’s, what the internet lacks is the reasoning process that humans go through to get to those outputs.
[00:34:19] Paul Roetzer: And that that is fundamental to building, like, smarter AI models that can do this, like, system two thinking. And so he talked exactly about that, about how, like, this internet data is lacking, but all the research labs are, like, working on solving that. That also led into memorization. I think this is a really important thing for people with the future of education and work.
[00:34:42] Paul Roetzer: The reality is like right now, these models have limited memory. So what 2 million context window I think is with Google Gemini is still the industry leading right now. Their, their research has shown they can get it to 10 million or more tokens, which is roughly
[00:34:59] Paul Roetzer: [00:35:00] 9.5 million words. Like it would remember everything basically perfectly in that environment.
[00:35:05] Paul Roetzer: I think most AI research labs assume that we are heading toward a day of infinite memory for these models, that they, they will just remember everything, um, they will have instant recall on this. And so it’s, it’s very important when we think about like, what do we teach? How do we teach? How do we,
[00:35:23] Paul Roetzer: develop skills professionals in our organizations?
[00:35:26] Paul Roetzer: You do this under the assumption that we are all going to have infinite memory on call. That, that these models are going to remember everything. And so memorization of topics, being able to regurgitate things you’ve learned, isn’t a human advantage. So when we think about how we train students, how we develop people, forcing them to just remember stuff, Isn’t going to be uniquely human at all.
[00:35:53] Paul Roetzer: And so he talks a lot about how these transformers actually have some advantages over the human brain, that
[00:35:59] Paul Roetzer: there are these [00:36:00] limitations, these inefficiencies in the human brain and how these transformers won’t and don’t have those inefficiencies. and that can be kind of, important moving forward.
[00:36:11] Paul Roetzer: Talk a little bit about smaller models. We’ve touched on this quite a bit, how these bigger models are going to train and oversee the smaller models, which actually led to what you were mentioning, this idea of like. AI models or LLMs making up companies that you may, your CEO is maybe like GPT 6. It’s the big massive model that took a hundred billion to train in a couple of years.
[00:36:31] Paul Roetzer: That’s your CEO. And then it manages a swarm of smaller models that are more cost efficient, that do specific
[00:36:38] Paul Roetzer: things. And that’s, that’s something that I think is very abstract for people, but. Andrej isn’t alone in talking about this. I went back to, for my opening keynote actually at MAICON this week, I went back and pulled the excerpt from July 2023 Atlantic article called, Does Sam Altman Know What He’s Creating?
[00:36:57] Paul Roetzer: And in that article, Ilya Sutskever, [00:37:00] Talked about this exact concept. He said, the way I think about the AI of the future is not as someone as smart as you or smart as me, but as an automated organization, a constituent AI is working together at a high speed like bees. In a hive, a single AI organization would be as powerful as 50
[00:37:16] Paul Roetzer: apples or Google. So this idea of like these big models and small models working together, I think that’s something we’re gonna be hearing a lot more about and could start to. Play into, like, I mean, Andrej is building Eureka Labs. My guess is, he’s in, he’s building in this Like, what’s going to happen is these leading AI researchers are going to start building their own companies this way, and that’ll set the groundworks for, you know, where we go.
[00:37:44] Paul Roetzer: And then the final thing, which is the thing I’d shared on LinkedIn, is I have tremendous respect for Andrej. Like, I think he teaches with a humility that I find inspiring. His, intro to LLM’s YouTube YouTube video from earlier this year is like this amazing [00:38:00] view into where everything is going. And so when he announced he, you know, he had left OpenAI earlier this year, when he announced he was doing education, I was like,
[00:38:08] Paul Roetzer: why? Like,
[00:38:08] Paul Roetzer: what is he, he could raise billions. I mean, Ilya just raised a billion last week. Like, we’re going to talk about that in a minute. Like, Andrej could raise billions to build an AI company. And yet he’s he’s not. seems to be focusing on education.
[00:38:20] Paul Roetzer: And so they asked him point blank, like why? And I loved his answer. He said, I would start with, I’ve always been an educator.
[00:38:26] Paul Roetzer: I love learning. I love teaching. And so it’s a space that I’ve been very passionate about for a long time. One macro picture that’s kind of driving me is there’s a lot of activity in AI to replace or displace people. It’s the theme of sliding away the people, but I’m always more interested in anything that empowers people.
[00:38:42] Paul Roetzer: And I feel like I’m team human, but And I’m interested in things that AI can do to empower people. I don’t want the future where people are on the side of automation. I want people to be very in an empowered state and I want them to be amazing, even more amazing than they are today. And so I just, I love that approach.
[00:38:59] Paul Roetzer: And my, [00:39:00] my basic belief is we just need more high profile AI researchers, investors, entrepreneurs. on TeamHuman, as would say, and we’ll have to see how it plays out, but I think people like Karpathy taking this position opens the door for a lot of other AI researchers to take a similar position of
[00:39:17] Paul Roetzer: innovation over replacement, basically.That, augmentation of people versus replacing the people. So
[00:39:23] Paul Roetzer: I, yeah, it’s a great interview. It’s very technical. It’s heavy on robotics for the first, like, 25 minutes. So if you’re not that interested in that, you can kind of skip ahead to the other stuff. But,
[00:39:31] Paul Roetzer: always, like I said, there’s insights when you hear these people Give Talks. talks.
[00:39:36] Mike Kaput: All right. So let’s jump into some rapid fire topics this week. And like you mentioned, one of them
[00:39:42] $1B Round for Sutskever’s Safe Superintelligence
[00:39:42] Mike Kaput: concerns Ilya Sutskever is that OpenAI’s former chief scientist, he’s making waves for his new venture called Safe Superintelligence, SSI for short. They have secured a billion dollars in funding despite only being [00:40:00] a few months old, and this investment values SSI at five Billion dollars.
[00:40:06] Mike Kaput: Now, their mission is to develop safe AI systems that far surpass human capabilities, I
[00:40:13] Mike Kaput: Right now they’re splitting their operations between Palo Alto and Tel Aviv and this investment round has you know, major people like Andreessen Horowitz involved, Sequoia Capital involved. in it really
[00:40:29] Mike Kaput: it’s something Sutskever co founded alongside
[00:40:34] Mike Kaput: two other AI experts, Daniel Gross and Daniel Levy, who is a former open AI researcher. And of course, this move comes after Sutskever was involved in the
[00:40:45] Mike Kaput: high profile leadership drama. of OpenAI last year, where he briefly supported ousting CEO Sam Altman before getting, uh, let out of the company.
[00:40:57] Mike Kaput: Now,Paul, this is, [00:41:00] of course, really interesting given Ilya’s background and pedigree, but I do want to, like, ask the obvious question.
[00:41:06] Mike Kaput: How is this company valued at 5 billion already? Mm.
[00:41:09] Paul Roetzer: it’s Ilya,
[00:41:13] Paul Roetzer: He could raise whatever he wanted. I mean, Elon’s got a company valued at 24 billion that had done nothing. It’s just right now there’s only, you know, so many top AI researchers that can attract talent. That’s a real key that they have the name and the, history that they’re able to go get other top researchers to come with them.
[00:41:33] Paul Roetzer: And that’s where a lot of the value is being accrued is in the talent within the teams and what they’re capable of building. the the interesting thing with safe superintelligence is he has very specifically said they have zero product plans. They are going on a straight line to superintelligence. They are not going to do a ChatGPT like bot.
[00:41:50] Paul Roetzer: They’re not going to do all these other things, They’re going to pursue the superintelligence and they’ll let us know when they get basically.
[00:41:57] Paul Roetzer: if you want if you want to, like, a little bit more [00:42:00] about this, so last week’s episode 13, we talked about Ilya related to that Project Strawberry.
[00:42:05] Paul Roetzer: And
[00:42:05] Paul Roetzer: I think you could go back to his work from spring of 23, the Let’s Verify Step by Step paper as as maybe a preview of what they’re working on, how they’re approaching this differently.
[00:42:16] Paul Roetzer: this was only Ilya’s third tweet of the year. I mean, after the OpenAI drama last fall, he kind of went MIA for a, for a a long time. Uh, his tweet was mountain identified, time to climb. That was the whole tweet. And it linked to the Reuters article about the billion dollars. In the article, he actually gave an interview, which he hasn’t done all year, that I’m aware of, and he said, they’re they’re going to take a different approach than he previously at um, building on maybe some of the things he had learned there.
[00:42:44] Paul Roetzer: But he said, everyone just says scaling hypothesis, meaning the scaling laws that they’re going to keep throwing more data, more compute, and it’ll
[00:42:51] Paul Roetzer: keep, you
[00:42:51] Paul Roetzer: know, improving. Everyone neglects to ask, what are we scaling? So he’s kind of being here. But, some people, this is a quote, some people can [00:43:00] work really long hours and we’ll just go down the same path faster.
[00:43:03] Paul Roetzer: It’s not so much our style, but if you do something different, then it becomes possible for you to do something special.
[00:43:11] Paul Roetzer: So. You know, there is a lot of belief, like, again, like a Yann LeCun is one of the people in this camp at Meta,
[00:43:17] Paul Roetzer: that we need other breakthroughs, that that super intelligence. Now, again, you could argue, should we even be pursuing super intelligence? Do we really want AI
[00:43:25] Paul Roetzer: that is smarter than humans at all cognitive tasks? Like smarter than expert humans at cognitive tasks?
[00:43:31] Paul Roetzer: I, we don’t get a say in it. It, they’re, they’re going to pursue it, whether we agree that it should exist not. So,
[00:43:38] Paul Roetzer: this pursuit, though, is that we’re going to need something more than the transformer.
[00:43:43] Paul Roetzer: That the transformer that’s the basis for today’s large language models will get us so far, but it won’t get us to maybe AGI or beyond AGI. And so Ilya here is kind of alluding to the fact that maybe they have a different way, that there’s another process to go through [00:44:00] other scientific breakthroughs that they maybe envision that’ll get us there.
[00:44:04] Paul Roetzer: So certainly worth following. They’re being very. secretive about what they’re doing and how they’re doing it. And I don’t expect that to change. Although history is, um, releasing research papers, like, you know, he’s, he’s kind of a a research, AI researcher through and through. So maybe they’ll publish some papers along the way, but I think they’re only going to release what they believe can be safely understood by society.
[00:44:30] Paul Roetzer: If, if they think they’re onto something that they need to solve the safety side for first, they’re not going to tell us about it.
[00:44:36] Paul Roetzer: So, I wouldn’t be surprised if we don’t hear anything familiar for another six months.
[00:44:41] OpenAI Pricing
[00:44:41] Mike Kaput: So, in other news, OpenAI is exploring some new pricing that could have a big impact on anyone who uses their technology. This comes from the information, and they say that internal discussions have reportedly touched on subscription prices [00:45:00] as high as 2, 000 per month for upcoming large language models from OpenAI. Though the information says it’s unlikely the final price tag will get that high.
[00:45:12] Mike Kaput: Now this comes, as we’ve talked about, to highly anticipated releases from OpenAI. Strawberry, which is some type of potentially advanced reasoning capability, and Orion, the codename for their new flagship model.
[00:45:28] Mike Kaput: Now, ChatGPT’s current 20 a month subscription has already generated an estimated 2 billion dollars in annual revenue, but it sounds like that might not be enough
[00:45:40] Mike Kaput: to cover all the costs that OpenAI also incurs. Now, of course, OpenAI, we also talked about, is in the midst of talks to do a massive fundraise as well,
[00:45:51] Mike Kaput: and secure billions in new capital as well from people like Microsoft, Apple, and potentially NVIDIA. Now Paul,
[00:45:59] Mike Kaput: like, the [00:46:00] information readily says don’t expect open AI to include a $2,000 subscription tier for their most advanced
[00:46:08] Mike Kaput: stuff, but like, the fact that they’re even considering this seems to indicate they think people would pay that much for access to their most advanced models. Like, how are you thinking about that? Is that true?
[00:46:21] Paul Roetzer: Yeah, so I think the market is set for what the current models are capable of. I mean, everybody’s in the 20 to 30 per month realm, and anything beyond that is Probably not going to fly. you
[00:46:33] Paul Roetzer: You know, I think you already have enterprises like a few episodes ago, like 105, 106, we talked about, was it Chevron and their 20, 000 Microsoft Copilot licenses
[00:46:42] Paul Roetzer: and questioned value of it.
[00:46:45] Paul Roetzer: you know, I think for enterprise adoption, you’re looking at that. Now, if they unlock. Advanced reasoning capability, where these things are much more, advanced in solving, uh, assisting in decisioning, driving [00:47:00] innovation,
[00:47:00] Paul Roetzer: and you can productize that in a very understandable way, then certainly, if we zoom out, 2, 000 a month is 24, 000 a year,
[00:47:10] Paul Roetzer: like that’s nothing if it’s applied correctly, like if if you look, if you can Handhold to use cases where you’re showing them the value.
[00:47:20] Paul Roetzer: And now you, unfortunately you have to equate it to FTEs. Like you’re, you’re basically saying like, Hey, instead of hiring this role or adding more headcount, this will do the job of three, Programmers that you’re paying 175, 000 a year each. And so now 2000 a month, doesn’t seem like that much, which up
[00:47:40] Paul Roetzer: until, honestly, you’re giving kind of the intro here.
[00:47:42] Paul Roetzer: I didn’t think about this, but if we connect this back to the automated organization that Ilya and Andrej talked about, I could 100 percent see, I don’t think OpenAI is going to do this, but somebody is going to do this with open source tech. They’re going to sell autonomous organizations. So if you think about what we sold as [00:48:00] an agency
[00:48:00] Paul Roetzer: for 16 years, we were an outsourced marketing team, basically.
[00:48:04] Paul Roetzer: Like, we would bring in SEO capabilities, content creation capabilities, brand capabilities, research capabilities. Like, you were plugging and playing, but as the CEO of that agency, I was building a team of people capable of doing
[00:48:16] Paul Roetzer: these things that I would then provide to clients as a service. I can completely see that idea of We’re going to have a CEO level LLM, like a really, really advanced reasoning capable engine that I’m going to pay 2, 000 a month for, and I’m going to have this like really powerful model.
[00:48:33] Paul Roetzer: And then I’m gonna have a swarm of like an SEO GPT and a copywriter GPT. And I am not saying it should happen. I’m telling you it’s going to happen. That someone is going to build this as a service, autonomous organizations as a service.
[00:48:48] Paul Roetzer: Where you might then be paying 2, 000 or 5, 000 or 10, 000 a month because you’re paying for the capabilities that otherwise you would have had to go get an agency or consulting firm for.
[00:48:59] Paul Roetzer: I [00:49:00] don’t, I’m not, as I’m, I’m thinking this as I’m saying this, so like, you know, this is, this is like real time
[00:49:06] Mike Kaput: yeah,
[00:49:07] Paul Roetzer: yeah, this isn’t next year. I’m, I’m not saying that next year we’re going to not need agencies or anything like that. I’m saying.
[00:49:16] Paul Roetzer: These researchers think this is possible and that this is a likely future where we have one to 10 person billion dollar companies being built.
[00:49:25] Paul Roetzer: so someone is going to look at that and start building solutions around. I mean, this is what Salesforce agent force that we’re going to talk about next is
[00:49:32] Paul Roetzer: basically doing. It’s what,
[00:49:35] Paul Roetzer: it’s what Google’s basically doing with building AI coworkers. Like no one’s calling it this yet, but in essence, that’s what they’re all doing is building up agents that are able to do these things.
[00:49:45] Paul Roetzer: That when you connect them together and they function like a swarm or like bees in a hive, they’re able to go do all these pieces. And if they report up to the really powerful LLM that you’re paying 2, a month for, And that LLM is able to manage the work of the worker bees,
[00:50:02] Paul Roetzer: man, like, and this is like, honestly, like this isn’t even timed this way, but like my opening keynote gets into this, like, I kind of go into the idea of the Ilya quote the, future.
[00:50:13] Paul Roetzer: And this is the stuff I just don’t think people are ready to talk about yet. Like years ago, I remember I wouldn’t talk about AGI on LinkedIn because I didn’t think like we were ready, that people were ready to hear it Right.
[00:50:22] Paul Roetzer: And I would put this autonomous organization idea in that realm from three years ago, where AGI was, where no one was like, wanted to hear it.
[00:50:31] Paul Roetzer: My guess is no one wants to hear this conversation right now, but this is absolutely it’s going. and and I I, think it, you know, again, it requires agents to become reliable and accurate. We may be a year or two away from, from that, but this is, this is reality. Like people are going to build these things and people are going to do things.
[00:50:48] Paul Roetzer: And, I think you’re going to start hearing about early forms of it in 2025, probably.
[00:50:54] Mike Kaput: Well, let’s carry this thread through to the next
[00:50:57] Paul Roetzer: It just became like a main topic
[00:50:58] Salesforce Agentforce
[00:50:58] Mike Kaput: No, yeah, I’m serious. I [00:51:00] think this, uh, it gels really well and I have some follow ups here. ’cause like the next topic that you mentioned is. Salesforce is creating a new platform called AgentForce that’s set to debut at their annual Dreamforce conference, which takes next week. CEO Mark Benioff is calling this a quote, hard pivot for the company. They’re basically re imagining Salesforce as, you All in
[00:51:23] Mike Kaput: on AI agents. So AgentForce goes beyond just a typical AI
[00:51:27] Mike Kaput: chatbot. It allows users to build and deploy autonomous AI agents that can make decisions and complete complex multi step
[00:51:36] Mike Kaput: tasks. So this operates on top of Salesforce’s existing business apps with a focus on automating customer service tasks.
[00:51:46] Mike Kaput: So it sounds like, at least on paper, the entire company is kind of Starting to reorient around this idea of AI agents and they are making it seems at least a big public bet
[00:51:58] Mike Kaput: That this is going to be the future.
[00:51:59] Mike Kaput: [00:52:00] So Paul like maybe carrying through this point like not only people inventing autonomous organizations You
[00:52:08] Mike Kaput: know, as startups that compete with agencies and service providers. But what happens when this stuff gets baked into, you know, anytime you mentioned Salesforce, I immediately think of HubSpot too, right?
[00:52:19] Mike Kaput: Like it’s an entire agency ecosystem and HubSpot,
[00:52:22] Paul Roetzer: It is how we started, if we were PR 2020, my agency was HubSpot’s first partner in 07. So yeah, I agree.
[00:52:29] Mike Kaput: So is that where this is going? Do you think?
[00:52:32] Paul Roetzer: Yeah, so like Dharmesh Shah, the co founder of HubSpot, owns Agent. AI, and they’re building a professional network of AI agents. Now that’s outside of HubSpot at the moment, to my understanding,
[00:52:43] Paul Roetzer: but they just rolled ChatSpot, was the chatbot that, that, that’s now part of HubSpot, um.And this is kind of how Dharmesh rolls, like he’ll build these kind of
[00:52:54] Paul Roetzer: projects on the side and then they eventually get kind of rolled into HubSpot. So I could see agent.
[00:52:59] Paul Roetzer: ai being a [00:53:00] similar path. So yeah, and I think agent is just this confusing term right now because so often when it’s talked about or it has been talked about, an AI agent meant like level.
[00:53:13] Paul Roetzer: Was it three on OpenAI’s intelligence rev? So once we get past reasoners, then we get to agents. So the ability for these things to take actions on our behalf. Right now, what I think we’re getting is a bunch of rules based agents that have some LLMs built into them. So it’s like,
[00:53:27] Paul Roetzer: it’s kind of blending the lines between just traditional automation, where someone writes some rules that
[00:53:33] Paul Roetzer: connect some Zapier zaps to do some stuff.
[00:53:36] Paul Roetzer: And I think people are kind of calling those agents right now. like in like some ways you could say, like, well, problems GPT as an agent. It’s like, it’s not like, not the way I would think about what an agent is going to be that can go do things for me,
[00:53:49] Paul Roetzer: take actions on my behalf that I don’t have to program. It just learns
[00:53:51] Paul Roetzer: it like what Replit’s doing right now with their
[00:53:54] Mike Kaput: Mm hmm.
[00:53:54] Paul Roetzer: like an early preview of that. So I think we’re in this kind of weird stage where we’re calling [00:54:00] agents. The same thing, but really not, but I do think that as these things get more intelligent, it’s going to be like just giving a prompt.
[00:54:07] Paul Roetzer: You’re going to be able to do prompt to agent and, and I’m not going to need to build seven zaps and Zapier and do all these other things. I’m just going to say, I want an app that, you you know, runs custom dashboards for me whenever I ask for data and it just goes and it. Learns how to do it. And it comes back and says, okay, I’m able to do this now.
[00:54:24] Paul Roetzer: What do you, what, what do you want to visualize? And the agent just does it, or I want you to send an email in HubSpot and here’s what I want it to do. And it just goes and it picks the list and it does the things. And like, that’s what I, what I think of agents as like, I think of this agentic future.
[00:54:38] Mike Kaput: It sounds wild to say, but what you’re describing, like, again, when or if it ends up working as envisioned,
[00:54:46] Mike Kaput: is literally like, you know, we have text to video, text to image, text to speech. This is text to employee.This is like, you
[00:54:55] Mike Kaput: literally spin up an employee, you know? It’s
[00:54:58] Paul Roetzer: if you can spin up an employee that [00:55:00] does things
[00:55:01] Paul Roetzer: like what was, oh, what was the example I just saw? Oh, Google has like, a new thing. I forget the name. I’ll see if I can look it up in minute, but, it’ll, it’ll just like turn a podcast into short, like, or, turn a research paper into a podcast and like, That’s an, like, maybe you have an employee that does that, and so yes, it’s like, how big is the scope of the task that we can build these things to do, and in some ways it may, it may just become an employee, and again, like, the labs can’t talk about it, like, you have to understand
[00:55:39] Paul Roetzer: They know that AI is going to replace people, like not, I’m not saying all jobs are going away or anything like that, but they know what they’re building will replace people.
[00:55:46] Paul Roetzer: They can’t say that. They cannot tell you that an AI co worker is a replacement to your co worker. They can’t position it that way.
[00:55:55] Paul Roetzer: So we’re going to keep talking about it as agents and co workers [00:56:00] and co pilots
[00:56:00] Paul Roetzer: because that’s what the marketing messages have to say. But they understand, trust me, they all do, that what they’re building is going to disrupt the workforce.
[00:56:10] Paul Roetzer: They also believe that there is a possibility it does grow the workforce by growing productivity, Like,
[00:56:19] Paul Roetzer: and so they’re trying to race to find the ways that it benefits the workforce before it disrupts it so much that the negative becomes obvious. That’s what’s happening. Like, it’s Trust me,
[00:56:33] Paul Roetzer: I’ve talked to
[00:56:34] Paul Roetzer: them, like, this is how it’s working. They know it’s coming. They’re just trying to find the ways for a soft landing, where we create an abundant, beautiful future.
[00:56:45] Paul Roetzer: But it’s not promised. Like we, we’re going to be in this, this realm where we’re just not sure, which is why we do what we do. It’s like, we got to talk about this because it can be good,
[00:56:55] Paul Roetzer: but we, we got to like make an effort to get it there first. [00:57:00] Yeah,
[00:57:01] Mike Kaput: Karpathy’s approach and motivation.
[00:57:04] Paul Roetzer: Yep.
[00:57:05] Paul Roetzer: And that’s, we need more people like him saying it out loud because he knows better than anybody what these things are capable of doing
[00:57:12] Paul Roetzer: and, and that we’re not that far from it and we got to be intentional about creating a better future with these things.
[00:57:18] Mike Kaput: All ight. Now that we’ve written your MAICON 2026 keynote, we’re gonna, yeah. all right. Just a couple more topics on the docket this week, but
[00:57:28] Claude for Enterprise
[00:57:28] Mike Kaput: next up is Anthropic has announced the launch of Claude Enterprise, which is basically an enterprise tier for Claude that helps organizations securely collaborate withClaude using their internal knowledge. So, some interesting features that come with the Enterprise plan. There’s a 500, 000token context
[00:57:47] Mike Kaput: window, so much, much larger than I believe the previous one is about 200, 000 on the commercial
[00:57:53] Mike Kaput: facing version of Claude. That’s equivalent to tons of transcripts, literally dozens of [00:58:00] hundred plus page documents if you want. There’s a GitHub integration, a native GitHub integration is being introduced in beta, allowing engineering teams to sync entire code bases with Claude. There are enhanced security features like single sign on, role access.
[00:58:17] Mike Kaput: some features coming soon include audit logs and a system for automated user provisioning. And they emphasize that Claude does not train on customer conversations and content at this point. This tier, so so ensures the privacy of your organization using
[00:58:35] Mike Kaput: I’m not sure. They did, they say, Anthropic emphasizes that Claude does not train on customer conversation and content. I am not a lawyer.
[00:58:43] Mike Kaput: Go read the terms and conditions of the free plans that you’re using, I would say. Don’t quote me on any of that. so Paul, like, We love Claude, others
[00:58:55] Mike Kaput: consistently tell me that they love it, like, I’m kind of actually surprised we’re only now [00:59:00] getting an enterprise offering, like, how do you, is this kind of where all these companies are moving to, like, go upmarket?
[00:59:05] Paul Roetzer: Well, I mean, look how much money OpenAI is making on it. It’s, it’s the obvious play. It just takes time to build an enterprise movement. Like, Anthropic is a research firm. They’re, they’re trying to figure out how to be a product company. Like it, it just takes time and it’s not going to work great to start.
[00:59:18] Paul Roetzer: I mean, they’ll probably make a bunch of money, but it, it, building an enterprise platform is no joke. Anybody who’s worked in enterprise software knows this doesn’t just happen. the the thing I find fascinating is.
[00:59:31] Paul Roetzer: How far ahead is Google that they have a 2 million context window already that five months ago, they said showed it worked at 10 million.
[00:59:40] Paul Roetzer: And yet the enterprise play from Anthropic is still only 500, 000, which is a lot. Like, don’t get me wrong. I mean, think ChatGPT is 128, 000. Like it’s, it’s a lot, but it’s like, Google’s huge.
[00:59:54] Paul Roetzer: Like what did Google unlock that like they, I don’t know.
[00:59:57] Mike Kaput: what they unlocked and also probably what they [01:00:00] can afford To support. they got a lot of ad dollars to throw at that. centers and their own chips and but it does
[01:00:08] Paul Roetzer: we’ve always Google’s got an advantage. Like people underestimate it, but
[01:00:11] Mike Kaput: for sure. And, but to your point too, I, it’s not, I,
[01:00:15] Mike Kaput: I’m be pretty confident saying it’s not just their budget. Like they have done some fundamental research breakthroughs,
[01:00:23] Mike Kaput: most fundamental Yeah. Right.
[01:00:26] Mike Kaput: So definitely don’t sleep on that.
[01:00:27] Paul Roetzer: Yeah.
[01:00:28] Copyright Laundering
[01:00:29] Mike Kaput: All right. So next up, we got introduced to a new AI term this week that we think is important to understand moving forward. The term is, quote, copyright laundering. And it comes from a post from Ed Newton Rex, who posted on X. He’s an X stability AI leader who actually left that company over copyright concerns.
[01:00:50] Mike Kaput: In his post, he wrote, quote, models will be retrained on synthetic data alone. Attempting to limit copyright risks and the need to license data. Of course, that [01:01:00] synthetic data will be created
[01:01:01] Mike Kaput: using models trained on copyrighted work without permission. The copyright laundering cycle will be complete. Paul, you raised attention to this term in a follow on post. Can you kind of unpack for us what was compelling about this from you? Basically, if we’re using synthetic data, that was trained on something copyrighted, but you can maybe get away with then using that data to train a new model.
[01:01:24] Paul Roetzer: Yeah, I just thought, so ironically, Ed, we talked about last week, I was saying like, he’s out there fighting the good fight on this stuff. So it was just funny timing. I saw this, but no, I just thought it was such
[01:01:33] Paul Roetzer: a great term. I think he kind of coined the phrase, but it’s this whole idea of like money laundering. It’s like you take the
[01:01:39] Paul Roetzer: illegal money, you funnel it through some real estate and comes out the other side, you know, clean. And that’s That’s exactly what’s happening. They’re training these models on copyrighted material. You can use the model, the bigger model, like
[01:01:51] Paul Roetzer: Orion or whatever it is,
[01:01:52] Paul Roetzer: to produce synthetic data that doesn’t have a copyright.
[01:01:55] Paul Roetzer: And then that synthetic data trains the next model or the smaller models. And that model all of a [01:02:00] sudden comes out clean with no copyrighted material it. in it. it’s just fine. Like it’s a very, it’s very visual, uh, easy thing to visualize. And so I thought it was a clever term and,
[01:02:09] Paul Roetzer: seemed extremely accurate to what’s going on and probably how this is going to be done moving forward.
[01:02:15] Time AI 100
[01:02:15] Mike Kaput: All right. So Time Magazine has unveiled its second annual Time 100 AI list. This showcases the most influential figures in AI, and they come from a diverse range of backgrounds, some tech giants, startups, policy makers, privacy advocates. The
[01:02:33] Mike Kaput: list, as you would expect, has a bunch of notable people we always talk about. Google CEO Sundar Pichai, Sam Altman, NVIDIA’s Jensen Huang, Mark
[01:02:42] Mike Kaput: Zuckerberg.
[01:02:42] Mike Kaput: Google DeepMind leader, Demis Hassabis. There’s also a bunch of other interesting people that might be flying a little more below the radar that are well worth checking out on the list. But there’s one high profile AI figure missing who has a little bit of a habit of attracting controversy and [01:03:00] commentary.
[01:03:00] Mike Kaput: So a bunch of stories have already come out about the fact that time snubbed him. And this is Elon Musk. So he is not on the list.
[01:03:09] Mike Kaput: People supposedly are up in arms about it. So,
[01:03:12] Paul Roetzer: the bots on Twitter.
[01:03:14] Mike Kaput: the, yeah, yeah, yeah, the autonomous organizations on Twitter putting out
[01:03:19] Mike Kaput: information. So did you, when you looked at this list, like, did any inclusions or omissions, like, surprise you or kind of make you sit up and take notice?
[01:03:28] Paul Roetzer: Well, I mean, the no Elon Musk thing is ridiculous. Like he shouldn’t be there. It’s kind of like when the White House had electric
[01:03:35] Mike Kaput: Yeah, yeah, right, right.
[01:03:37] Paul Roetzer: sometimes it’s just obvious he’s being snubbed intentionally. Now keep in mind, Mark Benioff, the CEO of Salesforce owns Time Magazine. So I’m not saying Mark. Made the decision to not have Elon on there, but, if Mark if Mark Benioff thought Elon should be on there, my guess is Elon would have been on there, or if there wasn’t something going on. So, that was one. The senator who’s pushing the [01:04:00] SB1047 bill, uh, he was on there, and the funny thing is he tweeted about it, and it’s like, I don’t think he realizes he’s probably on there for the wrong reasons.
[01:04:08] Paul Roetzer: And he’s like
[01:04:09] Mike Kaput: Yeah.
[01:04:10] Paul Roetzer: humble bragging about being like, dude, like, take the temperature of the room for a minute before you go,
[01:04:16]
[01:04:16] Paul Roetzer: go, you know, talking about that. But anyway, no, I mean, like any list. It’s gonna be controversial, and there’s gonna be people on there who, you know, probably shouldn’t be, and a bunch of people who probably should be.
[01:04:27] Paul Roetzer: But, it’s good, like, I mean, people are always trying to figure out who to follow in this space, so go check it out, read it, see if there’s some people on that list that, you know, maybe you weren’t aware of, or you’ve heard us mention a few times, but weren’t following them yet, that are worth, worth following
[01:04:40] Apple Intelligence
[01:04:40] Paul Roetzer: along.
[01:04:44] Mike Kaput: of next week, because, uh, today is Monday, September 9th, and this afternoon, Apple is going to have its scheduled Apple event. It is called, I believe it’s Glowtime, and it is going to be revealing things about
[01:04:58] Mike Kaput: the new iPhone, [01:05:00] some Apple intelligence, their AI features are going tobe talked
[01:05:04] Mike Kaput: about, and, Apple has been postponing more and more of those features and the announcements around them.
[01:05:10] Mike Kaput: So, we’re going to do a deep dive all that, because by the time you listen to this, the event will have happened. So next week, we’re going to go deep on what they do announce. But Paul, any kind of final thoughts there of what to expect?
[01:05:24] Mike Kaput: Or it seems like some of the AI stuff is getting, being pushed back a little.
[01:05:30] Paul Roetzer: Yeah, I’m not, not surprised.
[01:05:32] Paul Roetzer: Like they’re,
[01:05:33] Paul Roetzer: so they’re going to probably announce it all. Like, they announced it all already a couple months ago. We covered when they introduced Apple intelligence and said it was coming this fall. So I think everything they introduced is still probably coming, but it’s not. I don’t think that like two weeks from now, you’re going to get your iPhone 16. That’s going to have every AI capability baked into it. It sounds like they’re going to probably slow roll it for whatever reason, additional testing. It’s just not ready for prime time yet, but I would expect next [01:06:00] version of Siri will, will be in there.
[01:06:02] Paul Roetzer: which is where the glow time comes from, because if you’ve
[01:06:05] Paul Roetzer: seen the previews, when you talk to the new Siri, The beveled edges actually glow. So it’s actually like a much more prominent activity. And I think there’s like a wave function that happens across the screen, like things that grow. So they, that’s where glow the time is coming from.
[01:06:20] Paul Roetzer: So I’m anxious to do it. I will, um, I’ll, as I do with every Apple product, I will go buy it. And hopefully it’s not the Apple vision pro that I just realized yesterday. I hadn’t put on my head in like eight weeks. and I actually use it. I do expect it to be. Cool. I’m not so sure. It’s going to like the,
[01:06:40] Paul Roetzer: world changing infusion of AI that maybe, you know, we thought it could be earlier this year, but we’ll see, see what happens.
[01:06:48] Mike Kaput: All right, Paul. That’s all we got this week. Outside of, you know, Macon. Uh, so
[01:06:53] Paul Roetzer: Just that small little thousand plus person conference. We’ve got to go run for
[01:06:56] Paul Roetzer: the next few days.
[01:06:58] Mike Kaput: But as some final quick [01:07:00] housekeeping notes, go check out the Marketing AI Institute newsletter, which wraps up all the news we talked about and all the stuff that doesn’t fit in this episode. Marketing ai institute.com/newsletter. And if you don’t mind and have not done so and can do so on your podcast platform of choice, we’d love to have a review from you. Just let us know how we’re doing, how we can improve every single bit of feedback helps.
[01:07:25] Mike Kaput: So Paul.
[01:07:26] Paul Roetzer: those YouTube videos, like the, like my kids always like watching us, you can hit that subscribe button or whatever that is.
[01:07:31] Mike Kaput: Yeah, yeah.
[01:07:32] Mike Kaput: Right. Great.
[01:07:34] Mike Kaput: Paul. Paul. All right. Thanks so much for going through everything for us. Really appreciate it.
[01:07:38] Paul Roetzer: All right, everyone. We’ll talk to you soon. We’ll see you on the other side of MAICON.
[01:07:42] Paul Roetzer: Thanks for listening to The AI Show. Visit MarketingAIInstitute. com to continue your AI learning journey and join more than 60, 000 professionals and business leaders who have subscribed to the weekly newsletter, downloaded the AI blueprints, attended virtual and [01:08:00] in person events, taken our online AI courses, and engaged in the Slack community.
[01:08:05] Paul Roetzer: Until next time, stay curious and explore AI.