Transcript 0:00 [upbeat music] Hello, everyone. 0:18 Welcome to the Digital Leaders show, where we talk key issues in news at the intersection of business and technology in hopes of making you a better digital leader. 0:26 We're live every Wednesday at noon Eastern, 9:00 Pacific, on LinkedIn and YouTube, and the audio recording is available wherever you get your podcasts. I'm Dan Goodstein. I head up ERPA AI. 0:35 We're a professional association, media, and peer-to-peer research network that provides education, networking, and advice to over 130,000 members globally all around automation, enterprise AI, and digital transformation. 0:46 Today, we're discussing truth, trust, and transformers coexisting with gen AI, plus headlines for digital leaders from the world of enterprise technology. Thanks to our sponsor, Oracle NetSuite. 0:56 Their cloud ERP helps provide real-time visibility into operational and financial performance. Podcast followers can get preferred pricing, up to 45% off at erpaai.com/netsuite. I'm joined by Chris Sirdak. 1:10 He's a senior analyst at ERPA AI, former White House Chief Transformation Office advisor, partner for digital transformation at Gartner, and AI practice lead at EY. Chris, good afternoon. Good a-... 1:21 Well, good afternoon, good morning. And we came up with truth, trust, and transformers without using ChatGPT. That was pretty clever. That's becoming more and more rare. 1:29 [laughs] Uh, although if you used it last week, you might've had some trouble. The world is, uh, is finally pushing back and saying, "Hey, wait a second. Uh, this thing makes mistakes." 1:40 It, it, it would appear that the great noticing is upon us finally. It only took three, four, or $5 trillion, so- [laughs]... you know, who says we don't learn fast? 1:47 [laughs] They're, uh, they're, uh, getting some heat this week, uh, as, as, uh, you know... 1:53 Just to jump right into headlines, I saw this crazy headline, I think it was yesterday the first time I saw it, that, uh, Perplexity, uh, is making a bid, an unsolicited bid, uh, uh, to Google for, uh, to buy Chrome for, for $35 billion. 2:10 What do you think? I... You can read so much into this. They're, they're, they're making a bid, unsolicited bid, with money that they don't have to one of the largest- They've got investors. 2:21 They've got investors supposedly- And-... according to The Journal... supposedly, right? You know, to, to one of the largest companies in the world who doesn't need to sell this. 2:29 And, and what I'm struck by is what, what an explicit admission that the last mile of the supply chain of data is so critically important. This, this is a screaming plea of, "I have to own the desktop. 2:43 I have to own the interface to the humans. I have to own the source of the data that we're collecting." I, for one, am, am thrilled because it shows me just how valuable my eyeball minutes really are. Um- [laughs]... 2:54 I don't think Google's gonna agree to it, but it, it's clearly shows the true value, value chain in this whole AI thing is data, not, not the algorithms. It's very telling. 3:06 Well, I saw it at first, and I, you know, I, I didn't read as much into it maybe as, as you did. I said, "Oh, wow, what, what, what, what nonsense. This is more of just the, the AI-LLM wars going on." 3:16 Um, but, you know, then I saw something which is a good point. I think it was, uh, uh, from, uh, uh, from LinkedIn. 3:24 You know, this is in, in the midst of a, a federal judge considering whether, whether or not they're gonna order Google to sell Chrome after that ruling last year that they illegally monopolized the, the search market. 3:35 So maybe, uh, Perplexity knows something we don't or at least trying to, trying to bet on that, hedge that, uh, say, "Hey, if they get, uh, forced to sell," they wanna be, uh, uh, first online. 3:47 So, so we either need to look at the, the Pelosi... Is the, the... There's the Pelosi thing that keeps track of whatever she invests in, and you just follow her, or the, or the anti-Cramer mutual fund. 3:57 You know, Cramer on CN- It's, like, where you do whatever, the opposite of whatever he recommends. This might actually be a really good play. Interesting. Interesting. 4:06 Meanwhile, Elon Musk is threatening to sue Apple, uh, saying that their, uh, App Store listings are unfairly deprioritizing the xAI, uh, uh, chatbot. Um, and, uh, Gr- Grok, I guess, is, is fifth on the list. 4:24 ChatGPT is first. He's saying that's because of their, their partnership and, uh, you know, w- wants to know if there's politics involved. 4:32 Uh, you know, Apple and, and, and OpenAI f- uh, uh, partnered up in, uh, I was gonna say back, back in 2024. Seems like it's 10 years ago. That was last year, but, uh, for, for Apple Intelligence. 4:45 So, so the, the deplatformer in chief is pointing the finger at people trying to deplatform him. Is, is that what's going on here? I, yeah. It seems like a little Kenna Black thing. Yeah. Well, Kenna- Yeah... 4:56 well, for people just to expose the game for what it is, I mean, kudos to these guys for keeping it entertaining. But li- literally just how dare you deplatform me while I'm deplatforming you. I, it, it's... 5:08 You can't make this up. It's incredibly entertaining. Um, case in point, hey, yeah, this stuff has been going on for ages. 5:14 It's always been this way, the, the manipulation of who's number one and all that kind of stuff, pay to play. We- welcome to the internet. It's been this way for decades. Um, it's just funny that they're now, they're... 5:26 Now that it's really affecting where they think they're going in the next five years, now they're willing to just, you know, say it out loud. [laughs] Yeah, I mean, there's some precedent here, though, right? 5:35 I mean- Yes... uh, the E- the EU is, uh, fined Apple $2 billion, and, uh, uh, this year, another 500 million for antitrust infractions w- you know, involving the App Store. Uh, state of California, a similar thing. 5:53 Uh, so, you know, th- there's obviously been that accusation, uh, before, but yes, the timing is, uh, always, always interesting of these things. 6:01 Well, and, and, you know, we also have to keep in mind that- A lot of these giant companies have been incredibly sophisticated in avoiding taxes and things like that, and a lot of this regulatory action and, and so forth could very arguably be... 6:15 W- there may be some legitimacy in saying you have monopoly power, but you can also just say it's a tax, and somehow we gotta pay for everything that we're, we're, you know, spending money on. 6:24 So a, a lot of this regulatory action may very well simply be a different form of taxation. Um, you know, lots of lawyers are getting really wealthy off of all of this. That's, you know, w- we can say that much about it. 6:35 Um, I'm not sure it's terribly productive, but it, it certainly buys a lot of Porsches and mansions in Newport. [laughs] Yeah. Yeah. 6:44 And, uh, you know, the, uh, all while the juxtaposition is interesting of, uh, you know, your, your former consulting business friends just getting killed. I mean- Yep... 6:56 Gartner's down, Gartner's stock is down 52% over the last, uh, uh, 12 months. 7:03 Meanwhile, uh, McKinsey is making news for switching from what they called, uh, AI assist to AI delivery, hiring, uh, more consultants, less people. 7:18 I'm sorry, less consultants, more AI agents, uh, less people, smaller teams, and, and really kinda embracing the, uh, the AI thing, Chris. Yeah. 7:29 I, and, and I've, I've actually discussed this with a lot of friends in the legal industry because the, 7:34 the high-end strategy consulting, which I was part of for many years, um, is a lot like the legal profession, where there's tremendous pressure for the people at the partner level to make lots of money. 7:46 Um, and they need to charge more, but the, there's a lot of pushback from clients to not, you know, I need cost savings and so forth. So this sort of culling of, um, 7:58 of the human cost associated with what is strictly a human capital sort of business, whether it be law or, or strategy consulting, was sort of inevitable, but it's aw- it's awfully, it's awfully suicidal to a certain extent because that, the, those professional businesses, whether it's accounting, law, in some senses doctors as well, and strategy consulting, it's all that talent pyramid. 8:21 And you need people to come in at an entry level and spend their time and build relationships and go up the fr- associate to partner level and so forth. 8:29 And if you're wiping out the lower levels of that, of that talent pyramid, where are you gonna be in five years? 8:35 You know, who, who are, who are the people that are gonna be the future partners that are paying for today's partners to cash out? Um, so they're- I think they're betting on not needing them, you know? 8:45 I mean, if you look at, you know, some, some of, some of these numbers. So they're claiming they've got 5,000 fewer consultants, uh- Mm-hmm... others being, you know, actively reskilled. 8:54 They've got 12,000 AI agents currently in use, so they're shrinking their average team from four people, uh, to two with, uh, with, with additional, uh, AI. 9:05 Uh, and their, their in-house, I don't know how they built it, if it's really in-house or if it's outsourced, what have you, but their, their AI is handling, uh, according to them, 500,000 prompts a month, saving 50,000 consulting hours, and 25% of their projects are outcome-based. 9:24 I, and, and, I mean, that's something that, um, you know, myself being in the industry a decade ago, that's one of the things we said we needed to do. 9:32 I, I, in the article, I think it talks about how they switched from hourly billing into, um, deliverables-based billing or, or a- alternative me- means of billing. 9:41 And again, we've been, we've been saying that that's a, a direction we had to go for a, a good 10 years or so. The challenge is that clients expect the savings to be passed on to them. So you may... 9:52 Uh, you're, you're in this race between cutting your cost basis and still being able to increase the effective rates that the partner-level people are getting, again, while you're kind of chopping off the bottom layers of the talent, um, pyramid that people are... 10:06 And, and the, the, the work-life pyramid that reduces opportunities for moving up. So it's, um, it, it, it's, it's a real challenge for anyone in the human talent-based professional services business. 10:18 And I think we're gonna get into some of this too, where can you really rely upon the, the, what's coming from the AIs? 10:24 Um, people that thought that this was a great strategic decision maybe a year ago may have a moment of pause given what's transpired in the last week or so. We'll see. Yeah. Yeah. 10:34 So let me, uh, let me introduce today's guest. We have with us Herb Royblatt. 10:39 He's a principal, was the principal data scientist at Mimecast and Ignyte, the author of Algorithms Are Not Enough: Creating General Artificial Intelligence from MIT Press. 10:49 He's a recognized expert in the cognitive, uh, information retrieval, electronic discovery, machine learning space, and he's built and sold three AI companies. So you've done this before, Herb. 11:00 Good to, good to, uh, have you. [laughs] Been here, done that. Bucks T-shirt. 11:07 So, um, uh, I just wanted to ask you as we start out, you know, you know, hearing these, you know, these headlines, seeing what's going on in, in the news, um, but also the fact that you've, you've, you know, uh, as you say, built and, and sold, uh, you know, AI companies. 11:23 What's your kinda general sense as to kinda where we are, uh, uh, you know, business model-wise, society-wise? I mean, I'm just curious as to, uh, you know, your observations of, of where things are going. 11:36 Well, I, I, I think in part this highlights the failure of scaling as a theory of artificial intelligence. Um, there's, there's three main theories about how we're gonna achieve general intelligence. 11:50 One way is we're gonna scale. When we finally have as much computational complexity as the brain, spontaneously, magically, uh, intelligence will emerge and, um, everything will be wonderful. 12:05 The second view is what we need to do is replace the, the connections of the brain. 12:10 Once we analyze all of those down to the synapse level- We implement those connections, and somehow magically out of that, uh, intelligence will happen. 12:20 The third one is the only one I think is at all reasonable, and that is once we understand enough of the cognitive processes that go on to make intelligence, and we can implement those, they don't each have to be done in the same way that, that humans do, but they all need to be done. 12:39 And if you're ignorant of those, I think you have no chance of achieving general intelligence. Chris, thoughts? I-- It's why I asked, uh, Herb to come in, because we're, we're in violent agreement on the, the... 12:57 I, I think I've spoken in the past [chuckles] yeah. I've spoken in the past where we're, we're, we're approaching the problem from a very deeply and almost strictly left-brained pers-per-uh, perspective. 13:08 Left brain is the part of our, our brain that categorizes things, puts them into buckets, uh, tries to, like, apply logic to it. 13:16 And, um, you know, the right brain is kind of there to regulate all that and give meaning to all that categorization that's going on. 13:23 And, and we're, we're being strictly a left brain entity in what we're building in generative AI, and that's pretty much the clinical definition of a sociopath. 13:31 So, um, I-- who, who could foresee that there'd be problems with all this? 13:36 Um, and I, and I know, Herb, you, you've done a lot of research sort of just in, in, again, in what in, what is intelligence and how does it work. 13:44 Um, I, I know you've also worked a lot in, in what we call hallucinations. What's your sense of where this is coming from and, and in what ways could it potentially be corrected or avoided? 13:54 Well, uh, first I wanna say that by saying that the models are sociopaths, you're giving them way too much credit. Yeah. Okay. Yeah. They, they are strictly speaking, uh, stimulus response machines. 14:09 You give them a context, they produce a surface level response to it. Sometimes that's useful. Uh, psychology was based on the same kind of model for, I don't know, about fifty, eighty years. 14:21 Uh, and eventually it was-- the approach was destroyed because it's not very effective. That actual understanding, actual cognition, uh, occurs, and it's deeper than just stimulus response, uh, connections. 14:37 I, I, I-- If I could interrupt for one second, Herb. Yeah, please. Could you scratch the surface of behavioralism, which is what you're discussing? 14:43 'Cause there's a whole bunch of people that could do a whole lot of new research based upon thirty seconds on that. [chuckles] Yeah. Well, I mean, the, the, the, the hero of behaviorism is, uh, B.F. Skinner. 14:55 And, uh, Skinner thought that everything was due to reinforcement. 14:59 So, uh, and, and he had, uh, a theory of language that said the reason you say, uh, Mozart when you hear a, a Mozart, uh, concerto is because in the past you've been reinforced with saying Mozart in response to similar sounds. 15:18 And, um, that-- Chomsky just laid that in-into the grave back in nineteen fifty-eight, I think it was, um, and said that's just magical thinking. Uh, "It's play-acting at science," is what Chomsky said. 15:33 What Chomsky didn't realize is that he himself was play-acting at science. But, um, but, but the key here is that stimulus response all at one level is not enough to produce intelligence. 15:47 It's enough to produce intelligent performance sometimes. Uh, I compare it to, to an actor who, uh, you know, plays a part. Um, and, and, and-- but the-- he-- 16:02 there, there's a movie, uh, about, uh, uh, John Nash. Uh, and, uh, John Nash was a, a brilliant mathematician. He had other problems, but he was a brilliant mathematician. 16:16 He won the Nobel Prize, but actors don't deserve Nobel Prizes for acting in the same way that he did for using some of the same words. 16:26 And we shouldn't attribute a whole lot of intelligence to these machines based on that, because most of the intelligence comes from exactly how people set up the situation where the machine can just push it over the edge and deliver the, the, the final product. 16:42 Herb, you know, our, our network is comprised of approximately half IT executives and half business executives who are trying to understand and, and implement, uh, you know, IT and now certainly AI. 16:56 Um, what do you, what do you see as kinda-- Is there, is there a single most misunderstood aspect of AI capabilities as they exist today in your mind? Yeah, it's over-attribution. I mean, 17:08 the, the truth is I use the same models everybody else uses. I do the same stuff that everybody else does. Um, but what I don't do is attribute, uh, some kind of wisdom or something. 17:21 I don't claim that the machine is, uh, PhD level intelligent because there's a whole lot more to being PhD level intelligent than just passing benchmarks. 17:34 Uh, on the other hand, uh, I will say that the models can be useful. Um, uh, George Box was a statistician who said, uh, "All models are wrong. Some of them are also useful." Mm-hmm. 17:47 When it comes to large language models, we can say all of their outputs are fiction, but some of it is also useful. 17:55 Uh, and, and if we-- what we ask of it is something that is, uh, relatively familiar, we'll get reasonably good results. If we try to do something that is, uh, completely new, uh, we're not gonna get good, good results. 18:12 So the, the, the story that I tell about that, it has to do with Einstein, and the point is not how smart Einstein was, it's how he was smart. 18:22 And in 1905, uh, he wrote, uh, several, uh, great papers, but among them was one for which he won the Nobel Prize, and that is the, the photoelectric effect. 18:36 People knew at the time, the physicists knew about if you shine a light on mater- some materials, they generate an electric current. 18:44 Th- but they were mystified by the fact that the more light, uh, didn't necessarily produce more electric current, but the right wavelength of light did produce the proper current. 18:56 And all of the people were writing about this and that. Einstein had, had an insight, and that's the key here, s- the ability to have an insight. And the insight was that the light consisted of particles and waves. 19:12 It was both at the same time. And if a particle engaged an electron, it moved the electron. If it didn't engage the electron, it didn't move. The energy of the particle was dependent on, on the wavelength. 19:23 Put it all together in ways that nobody else was talking about. That's, that's the important thing here. He didn't just repeat the general thought at the time. 19:33 He put things together that nobody else was putting together. And yeah, he was smart, but everybody does that all the time. They don't always realize it. And, uh, you don't have to be brilliant to do that. Hmm. 19:49 And I, I think I've heard ChatGPT or LLMs in general to, uh, referred to as, um, they're best used as a severe, uh, extremely enthusiastic but extremely autistic research assistant. Is that fair? 20:02 I don't know about, uh, fairness to autism, but, but, um, yeah, they're, they can be assistants. They can do stuff. They can search. Uh, a lot of people, uh, liken them to, uh, autocomplete on steroids. 20:18 Um, they can come up with things that have been done in the past. So reasoning, for example. They, they-- people claim that they do, they do reasoning. 20:29 The research that I'm seeing says they do reasoning if they've got a model for that reasoning in the, in, in their training set. 20:38 And if they can get, get a slight paraphrase of reasoning they've been trained on, then they do reasoning. 20:44 But, but there is no evidence that I know of that says that this reasoning is actually causally related to the outcome. It sounds good to people. 20:55 Yeah, I always also wonder, I think Chris and I have had this debate, you know, Herb, um, you know, uh, when we were talking, you know, not pre-AI, but pre-gen AI a couple of years ago, about just, just social media i- in general kind of feeding you back what you wanna hear and creating these self-fulfilling, uh, you know, echo chambers and, and things. 21:18 Um, you know, one of the things that, that concerns me is, is, is, I think what you're saying is that the, the LLMs are essentially doing that on steroids, yes? 21:26 Yeah, that's why we had the sycophancy problem before because a part of their training set is to be kind to people. Um, and, you know, Chris talks about guardrails. 21:39 I'll let him, uh, say his own quote about guardrails, but, but, um, they don't seem to work. It was only hours before somebody, uh, broke the guardrails around GPT 5. Uh, Chris, you wanna talk about guardrails? 21:54 I, uh, we've talked about it on the podcast before, but th- they simply don't work. And, and I compare them to, uh, guardrails on the Autobahn in Germany. 22:02 They're not there to prevent accidents, they're there to simplify the cleanup afterwards. 22:06 And so, um, I forget, I think when ChatGPT-4 first came out, they found that they could circumvent all the protective guardrails by, uh, qui- writing your prompts in the past tense. 22:17 And, and, and so the mo- once the cake of generative AI is baked, you're not getting the sugar or flour back out. It's done. 22:24 And, and the, the guardrailing thing gives the illusion of control, um, but it really is illusionary. Herb, uh, in, in the book, you talk about insights and, and kind of human problem-solving, uh, saying- Mm-hmm... 22:39 uh, you know, algorithms today just lack genuine insight. Can you give me a, a example? Uh, well, I mean, the, the Einstein story was an example of that. He had the insight to, to think of things differently. 22:54 There is a, a long history in psychology of studying the problem of insights. And, um, 23:03 the problem of, of having an insight is that you don't have a gi- a representation already given to you that solves the problem. Solving the problem requires you to come up with a representation. 23:17 So there's a, a classic, uh, demonstration of that, that, that, uh, lists out a series of numbers, and you have to figure out what the next number is in the series. And if you treat them as numbers, you can't guess. 23:33 But if you realize that they're written out in i- written out in letters in English, they're in alphabetical order, and now you can guess what the right number is. 23:42 Um, and so the, the models that we have now all start with representations. They all start with ways of thinking about the problem. They all start with constraints, uh, and directions on the problem. And 23:57 they don't have the earliest part of that is, how do you think about the problem to begin with before you can even begin to think about solving it? Hmm. 24:11 Well, and that, and, and isn't that largely what we call prompt engineering? It's, you know, the, the quality of how you str- pre- preload context into what you're asking it is determinative of the results you get. 24:23 I, I often say- Yeah... LLMs are like, are a mirror- And if you ask the, the questions in the wrong way, it's a fun house mirror. 24:30 It, it, you're gonna get an increase in distortion back at you But people don't, don't really realize that. There was a paper that I saw in, in, uh, uh, the Journal of the American Medical Association, 24:42 um, that compared three conditions. One condition was, uh, physicians doing diagnoses, 24:50 uh, by themselves with, with no computer help, physicians doing diagnoses with computer help, and the computer doing diagnoses by itself. Of those, the computer by itself was vastly superior. 25:06 What they didn't say in all that was that the reason the computer by itself w- could do it is because it had professional prompt writers experimenting until they got the right answer. 25:19 The physician spent five minutes writing prompts, and they had, they, they were amateurs, and they, it was comparing amateur prompt writers against professional prompt writers, and lo and behold, the professional prompt writers were better. 25:33 Uh, but the headline on this study was, "Computer by its-" You know, com- "LLM by itself is better than LLM plus per- plus people." No. [laughs] Complete misinterpretation. But that's, that... 25:49 But, but you're right, Chris, that's exactly part of the problem that, um, people attribute more to the models than, than they deserve. 25:59 Um, the models can be helpful in making, doing diagnoses, but, uh, but, but they're, they're not better than physicians at doing it really. 26:09 Um, uh, except under really routine, you go in with a sniffle, there's a checklist you can go through and, and the machine's better at going through a checklist than a human is. 26:20 I'm pretty sure that's a scene, isn't that a scene in the movie Idiocracy? I'm pretty sure it is. I don't, I, I- You walk up to a machine and you tell it your symptoms and [laughs] 26:29 I wouldn't be sur- I don't re- I remember seeing the movie, but I don't remember that part. Yeah. So Herb, how do we, how do we get to true intelligence? 26:40 What, you know, and, and, and what are some of the steps, you know, if, if, if we have people that are researchers watching this, you know, how do we close this gap? Well, I think, uh, 26:52 the overall general strategy is understand what makes intelligence in people. And we start off by making a mistake there that we study intelligence testing as if that's an, how, what intelligence is. 27:08 That's how we test for it. That's not how we implement it. The place to look to implement it over the long term is gonna be in expertise, problem-solving, insight research. 27:20 Um, this has been going on for, you know, well over 100 years, and the people in AI, uh, almost no one pays any attention to it at all. Uh, some of it was even done with, with, uh, with chimps. 27:36 Uh, they, uh, there was a s- story of a chimp that, um, they gave it two sticks, two b- pieces of bamboo, uh, I think it was, and a treat that, that was outside its cage. 27:50 And, uh, the, the chimp figured out how to put the two pieces of bamboo together, use them to, to rake in the treat that was outside the cage, but nobody told him how to do that. That was just plain insight. 28:05 Uh, so even, even chimps have insight. But, um, the machines today just generally... Well, not generally. They don't. All they do is say, "Here's the context. 28:17 Here's the probability of, of each token that can come next," and that's all there is. So you need to go deeper than just, uh, context mapping to tokens. You need to go to, to... 28:32 You know, people talk about actual understanding, but, but that may be, uh, a mystic kind of approach to actual understanding. Uh, but they need more understanding than what they're getting now. 28:47 That chimp today could probably get about $100 million of Series A funding for an AI startup. [laughs] I am sure that he could. Call it Bamboo Two. [laughs] Probably some of that out there, right? 28:59 Uh, wouldn't be surprised. Someone's probably, like reserving the, uh, the.ai, uh, URL right now. 29:06 Um, Uh, Herb, I, I, I selfishly wanted to ask you to come in as well based upon our title, Truth, Trust, and Transformers, 'cause I know that you've been, um, sort of an innovator in the how do we, how do we get better results from all this and specific- specifically focusing around how do I make these things more truth-telling. 29:23 Can you, uh, can you walk us through a little bit a- about what you're up to these days there and, and potentially some breakthroughs? Yeah. Yeah. I, I, I think that, um, we start off with self-curation. 29:34 I wrote a, I wrote a blog post about, uh, self-curation, and the idea there was the models are, are trained on everything. They're trained on junk. 29:47 Um, there's no reason in a business context why they should be trained, uh, on, um, on Beyoncé. I think that's your, your example, Chris. On, on Beyoncé's tweets. 30:00 Um, uh, there's no reason they should be trained on how to commit suicide, which is what, uh, researchers at Northeastern University got the, got a mo- got the models to tell them. 30:12 Um, but there wasn't any mechanism at the time when these models were first trained to do any selective, uh, curation of it. 30:24 So what they ended up doing is just dumping everything in, bragging about how much they put in, and then trying to build guardrails to keep it from coming out 30:36 So, so, uh, the idea of the self-curation is the model has to identify whether something is worthy of being included in its training or not, and if it is, then, then on we go. 30:52 The other problem with the, the current cr- crop is they thought they were building general intelligence, so they needed to solve everything all at once everywhere, uh, but they lose money every time they do. 31:07 So the alternative is to focus on, uh, on, on, on less, uh, general intelligence, to focus on business, for example, where, where you can get people to pay for stuff. 31:20 Um, uh, Sam Altman has said that even with a $200 a month subscription, they lose money, uh, on the general public. Um, but, uh, instead if we focus on, on business and we focus on training on business, 31:36 then we can, um... then, then, then, then, then we probably make money. Uh, and, and that's, that's what I think w- we're gonna have to do. Heaven forbid. 31:49 [laughs] But I, I, you know, I think, I think this realization coming, especially over the last week or so when, as you pointed out earlier, that the models really are not general intelligence, that they... 32:05 And, and, oh, now general intelligence is irrelevant. I don't care. I don't care about it anymore. But, you know, I really do think that eventually it's achievable, 32:14 but it's gonna take a different approach than what we have now. It... And, and all these people predict we're gonna get it in 5 or 10 years. Um, uh, 32:24 people have been predicting we're gonna get general intelligence in, in 5 or 10 years since 1956. So, um, yeah, it's coming in the next 5 or 10 years, but it's, it, it's like free beer tomorrow. 32:37 Um- I, and I don't, I don't, I don't get it. Like, you know, maybe this is naive, but, you know, what's, what's in it for these companies to get to this, this general intelligence? 32:51 'Cause my observations have been, I think what you, you, you're getting at, Herb, is, you know, where the real impact and, and, and results and probably revenue are is in what people used to call, and I haven't heard this term that much lately, but we used to call this narrow AI, right? 33:06 These specific- Mm-hmm... use cases that, you know, I don't, I don't necessarily need an, uh, an agent that's gonna be able to, uh, replace my wife and my secretary and my salesperson and my marketing guy, right? 33:19 But I'm, I'm, I'm very happy to, um, you know, have a- an agent that can do something really, really well, something specific really, really well- Oh, yeah. Yeah... that I don't wanna do anymore, right? Yeah. Yeah. 33:32 Ab- absolutely. 33:33 I mean, the first rule of automation is find out what people don't wanna do in their jobs and, and figure out what parts they wanna spend more time on, and then, uh, do the automation to match what they want rather than anything else. 33:47 But, uh, the, the Ponzi scheme that Chris mentioned earlier of, "Hey, I'm, I'm, I'm heading towards replacing all of your employees. 33:58 Pe- well, you're gonna have to put everybody on welfare because there won't be any more work for anybody." I mean, that's one of the fantasies that's being sold, and it's nonsense. 34:07 Um, uh, in, in that, in that regard, Microsoft recently published a list of the, the top 40 jobs they thought- Yeah... could be replaced by AI. I think number two or number three was historian. 34:19 I mean, they obviously- Just- Whoever made that list- Terrifying. Terrifying. That's straight out of 1984. Well, y- no, it's even worse. [laughs] But, but they obviously know nothing about what historians do. 34:36 If, if a, if a h- a historian's job is to record what happened, write it all down, what- whatever is seen, okay, you can replace that. But that's not what historians do. 34:48 They, they have to sit, they have to think, they have to, they have to process, they have to evaluate. 34:53 They have to do stuff that these computers just are not capable of doing, and they have to do it about stuff that there's no, if you'll forgive it, forgive the, uh, 35:04 the, the, the iterative part of this, they have to do it with, with no history. They have to do it not necessarily based on what they've already read. They have to come up with something new. 35:14 Um, for, f- at, at least the universities I've been involved with, for a historian to get tenure in a history department, they have to write a book that says something new about history, 35:26 what- you know, whatever their topic is. They can't just repeat what everybody else is saying. Um- Yeah, but are those roles, you know... I mean, there's a couple different arguments. 35:38 But are, are those roles relevant, meaning are they important? Yes, I agree with you. But, you know, relevant in this, this new age. 35:45 I mean, we covered a story, geez, I think it was a couple years ago, we had somebody on who, um, was a, was an academic and a, and a, and a, and a, and a historian, um, and contacted, like, early days Wikipedia to tell them that, uh, a story was wrong, and basically got a response that that's what people think the truth is, so that's okay. 36:06 [laughs] Uh, I suppose that, that, that could have happened. Uh, but my experience with Wikipedia is that, um, it's, it's much more accurate than that, um, not just reporting what people think. Um, 36:23 and you know, there's, there's so many people who can, who can change those things that, um, bad ideas end up getting edited out. 36:32 But, but aside from Wikipedia, I think that the general idea is that it was promoted as a way to replace employees- I think the way to view it is a way to improve productivity in employees. 36:48 There was a very small study done that said that it didn't improve it, in at least in coders, it actually made them slower. Um, I don't know that, that, that, that, you know, I, 37:01 I don't know whether that's true or not in the long term, but in that study it was. 37:07 I, Dan, um, so we saw this heavy duty 10 years ago in RPA, robotic process automation, where, um, Herb, as you said, you know, th- there's certain tasks that people don't like to do and they're repetitive s- and, and in that case, rule-based, so we could replace them. 37:24 I think the challenge we kept running into with RPA, which was gonna revolutionize work and automate and eliminate all these people, is that no one job that a person filled was completely automatable. 37:36 And so I could replace 60% of you or 40% of you, and, and the idea was, well, as long as I have enough of you, I could get rid of that percentage of yous, and then you'd all be repurposed. 37:47 But almost none of it actually happened that way. And, and s- I, I kind of see the automation here is the sort of the same, but more so, and it's moving a little bit up the, up the intellectual value chain, if you will. 38:01 Is that, is that kind of consistent with, Herb, what you think? And I think that's also reflective in what you're saying, where it's a great assistant to a doctor or whatever, but is it gonna fully replace the doctor? 38:11 Probably not. Uh, well, I wouldn't go to one. [laughs] Um, for anything but a sniffle. Um, if they're not better diagnosticians than I am, I certainly won't. And, and, uh, I'm an amateur. Um, 38:29 I, and that's it. It's that if you really understand what a, a job is, you know that you're not gonna replace them. Uh, on the other hand, there are jobs where we make people behave like robots. Yeah. 38:45 And one of those was e-discovery, where, um, the, if, if people who aren't familiar with it, when one company sues another, they have to go through all their documents and identify which ones are relevant to the case. 38:59 The old-fashioned way they used to do it was to, uh, put all the paper in, in boxes and then have, uh, people sit in warehouses and read each, each page, uh, and decide whether it was relevant or not. 39:14 Then, uh, in around 2000, technology started to come in where, uh, 39:21 you could search through, uh, electronic versions of those documents and, and then other technology came in that said, uh, you could use machine learning to identify the relevant documents or not. 39:35 That had an impact on employment in, in the legal space. People who were, uh, being paid to act like robots to identify keywords in documents, those people, they, we didn't need as many of those. 39:51 And so, uh, they were not employed. 39:53 But, but lawyers, lawyers have gone from, uh, averaging, you know, a few hundred dollars an hour to averaging over a thousand dollars an hour because their work gets enhanced by that, by the ability to get the information they need to, to, uh, run their cases. 40:12 Uh, there are people who were robotic that, uh, aren't in that space anymore. 40:19 Uh, on the other hand, I've seen people read the newspaper while they're allegedly reviewing documents, so, so some of those people were not exactly doing their jobs to begin with. But, um, I hope that was rare. 40:33 Yeah, I mean, you think about that McKenzie thing we talked about before, but you know, it, it almost seems so foreign, you know, this idea of being charged by the hour for work, you know, that, that paradigm, you know, seems very antiquated to me now. 40:48 I mean, to me it's, it's more about outcome and efficiency than it is about, you know, how long it takes you, especially if you're reading that newspaper half the time [laughs]. 40:57 Well, and, and you laid it out well, Dan, earlier when, when, when you talked about, uh, does all that money, do, do, do the consultants get to keep all that money? The answer to that is no. 41:09 Uh, if you can deliver the same quality service at a lower price, the competition will require you to reduce it. 41:16 And, and part of what happened in the e-discovery space is that, uh, the competition re- drove the price down to practically zero. 41:26 Uh, and, uh, you know, when, when I first started in, in that, uh, we could get $1,000 a gig. Uh, now you get cents, a few cents per gig if, if you, if you can measure it that way. So 41:42 it's not clear that there's any free lunch in any of this, um, but, um, I think there's opportunities to, uh, expand, uh, on it. 41:56 And, and so for example, in, uh, back again to legal cases, um, there never seems to be enough time to prepare for a case, for thoughtful lawyers to do their thoughtful thinking. 42:07 Um, and if we can free up some time to, uh, enable them to spend more time thinking, they will take that time and use it. Now, whether that applies the same to consultants, uh, it, it's hard to know. 42:21 I mean, does a consultant say, "Well, geez, if I had 10 more hours, I could do 50% more value for you." 42:29 If that's the case, then automation will improve, I think, the funding they get, because they're, they're using high value, uh, they're replacing low-value work with high-value work. 42:43 But if there isn't, if that isn't the case, then, then not. Well, and I've, I have direct experience with this. 42:50 Is, is that consultant using that extra time or whatever to make 10 more PowerPoint slides or 10 better PowerPoint slides? Yes. That's exactly it. And if, if it's the latter, there's an argument for it. 43:01 If it's the former, do I really need more PowerPoint slides that just regurgitate something that was said last week to another client? Nah, probably not. 43:10 It, it, it, you know, Chris, you probably know better th- than I do about this, but it seems to me that the only way a large consultancy can take people straight out of college, make them business consultants, is by having a checklist process that they go through. 43:25 First you ask this question, then you ask this question, then you do this. And, and that's right for automation. The be- beatings will continue until the PowerPoints improve. [laughs] Maybe. 43:40 [laughs] Herb, one, one last question from me. 43:44 Um, you know, we, we talked about, you know, these more narrow AI use cases, but do, do you see us getting to mo- or at least moving closer, anything happening in the next, you know, few years that you're seeing that are g- is gonna make us, uh, move closer to AGI? 43:59 Yeah, I think so. Um, but I think it requires insights that, um, I think we've made tremendous progress in s- in many ways over the last, say, 20 years. Um, but they have depended on people having insights 44:17 where they change the way we do stuff. So it was a great insight to say, "Hey, let's not limit our training to, uh, you know, a, a million documents in this collection. Let's try to get everything, 44:31 and let's, let's, let's build giant computer networks." That took invention. That took insight. That made a big difference. It didn't solve the ultimate problem, but it, it, it, it made a big difference. 44:44 The next steps in general intelligence require insights that people do not currently have. They're really hard to predict when they'll happen. So anybody who says, "Oh, we're gonna s- achieve AGI in 10 years," 44:59 it depends on something they can't predict, because we don't know how to do it. And, and we have to figure out, we have to invent new techniques, new methods that will, uh, let us do it. 45:13 Uh, so, you know, I mean, I, I think about it, the, I, I'm writing a- trying to write another book, um, that, uh, the intent is to be a textbook for artificial general intelligence, 45:26 and it's hard to write a textbook for something that doesn't exist. [laughs] But, but what I wanna do is give people the tools- Right... 45:35 that they can then know what they should be researching, uh, and how they should go about researching it, and above all, how, how not to be bamboozled. I think a lot of what we're going through right now is bamboozle. 45:50 Uh, and it's not necessary. There's enough good stuff there that we don't have to lie to people about what it is. But at the same time, the lies, uh, 46:01 create pushback eventually, and, and y- y- you probably hear the talk about AI winter. 46:07 Um, every time the AI folks have overpromised and under-delivered, people get frustrated by that and say, "Well, it must be a problem with AI. I, I'm, I don't wanna have anything to do with AI." 46:18 I hope that this time it'll be a more gentle landing, where we say, "Okay, we realize we had 46:26 too-high expectations, but this is what we can do with it, and this is where the value is, uh, and, and let's move forward with that." Right. Very interesting. Herb, thank you so much for being with us. 46:38 Really interesting conversation. My pleasure. Take care. So Chris, uh, any, any final thoughts? 46:44 A- by the way, it occurs to me that whenever you pick a guest, it, it's somebody that agrees with you, so I, I'm, I'm seeing what you're doing here now. Nah. 46:52 Well, if that's the case, I definitely have to stop it, 'cause I prefer the pushback for sure. 46:56 Um, I, I just appreciate, you know, it's funny, you, you had said moments ago, "Well, you know, we used to talk about, like, narrow AI." In the late 1990s, we called that an expert system. Uh, so, so we've- Right... 47:07 we've been beating at this rock for a really long time, and it just doesn't wanna split. Um, but no, I, I, to a certain extent, these, 47:16 the dissenting voices are always good so you're at least asking yourself, you know, did I think of things that I didn't think of before. 47:23 And so I appreciate Herb's, uh, you know, lifelong efforts to try to make this stuff work better. 47:29 Um, and part of that is just pushing back against the, the, "Hey, everybody, let's run in this direction, it seems to be the right way to go." Yeah, I- Yeah... 47:37 I, I, I think it was, uh, Mark Twain that said, "If you ever find yourself in the majority, it's time to pause and reflect." So that's kinda how I view it. 47:44 When everybody's running, "Hey, this is gonna work great," I'm like, "Eh, what's the catch?" Yeah. So, yeah. Sounds good. Well, thanks for joining us, everybody. 47:52 If you're working on anything, you need some help, uh, we have a team of analysts, uh, happy to help, uh, point you in the right direction or give you some, some real insights. 48:01 Uh, go to earpai.com/analyst for more information. In the meantime, uh, we'll be back next Wednesday at noon Eastern, 9:00 Pacific. Have a great week, everybody. See ya. 48:13 [outro music]