Transcript 0:00 [upbeat music] If you don't know your numbers, you don't know your business. 0:21 I learned this the hard way with my first company, so I created NetSuite, the world's number one cloud business system. 0:28 NetSuite gives you the visibility and control you need to make smart decisions and grow with confidence. That's why companies who use NetSuite grow three times faster than the S&P 500. 0:41 Schedule your free product tour right now. 0:54 Hello, everyone, and welcome to the Digital Leader show, where we talk key issues and news at the intersection of business and technology in hopes of making you a better digital leader. 1:03 We're live every Wednesday at noon Eastern, 9 Pacific on LinkedIn and YouTube, and the audio recording is available on major podcast platforms the following week. 1:11 Today, we're discussing AI governance and three steps you can take to protect your AI investments, plus other tech business headlines for the week. Thanks to our sponsor, Oracle NetSuite. 1:19 Their unified cloud business management solution helps organizations operate more effectively and provide real-time visibility into operational and financial performance. 1:28 ERPA members and followers get preferred pricing of 40 to 45% off NetSuite. Just go to erpaai.com/netsuite for more information. I'm Dan Goodstein. I head up ERPA AI. 1:37 It's the Institute for Robotic Process Automation and Artificial Intelligence. 1:41 We're a professional association, media, and peer-to-peer research network providing education, networking, and advice to over 130,000 members globally all around automation, enterprise AI, and digital transformation. 1:52 My co-host is Chris Sirdak. 1:54 He's a senior AI advisor to ERPA AI, an author, consultant, and was formerly White House Chief Transformation Office advisor, partner for digital transformation at Gartner, and AI practice lead at EY. 2:05 Chris, good afternoon. Hey, good afternoon. How are you? I'm good. 2:09 Aside from, as I said before we went live, uh, you know, getting mad at my, my financial advisor who told me the, the market was gonna get killed, uh, because of the election, and, uh, here we are. 2:20 You don't, you don't follow the just do the opposite of Jim Cramer, uh, methodology for investing? Not yet, but maybe, maybe, maybe I need to, uh, maybe I need to start. 2:29 I, I, I understand there's an index fund that does that, and it actually outperforms the market. It's pretty funny. Oh, that's funny. I gotta, I gotta look into that. But, uh, yeah, interesting, uh, interesting week. 2:39 You know, uh, lots of excitement on, like I say, Wall Street. But, uh, you know, in the, in the tech space specifically, uh, you have a couple, couple announcements. 2:49 You know, Apple's getting into the smart home device market. Uh, they haven't named it yet, but it's sort of a, a, a home, uh, command center. 2:58 Uh, while kinda Amazon is, is gearing up with their, uh, Alexa Plus service to make the, the Amazon Echos in your, uh, in your house smarter. 3:06 Uh, A- Amazon's also launching a, an AI chip, uh, next version of its AI chip to try to compete with and, uh, mitigate the dependence on, uh, NVIDIA, which has been going gangbusters, I think we talked about, uh, last week. 3:22 Um, and, uh, you know, a few AI startups, uh, making announcements. It seems like that's, uh, an hourly, uh, announcement now instead of a, a daily, weekly, or, or monthly. 3:33 Just, uh, more and more coming out of the, the woodwork. I saw Alibaba now has launched their own AI search tool for small businesses. 3:41 Uh, I guess it's, uh, another Amazon, uh, co- competitor, so, uh, interesting marketplace. So, so Apple's gonna have a, a new in-home device. Are they gonna call it the Eye of Sauron, like from Lord of the Rings? 3:53 Or, [laughs] you know, just watching everything you do no ma- It, it's funny, I've, I've had multiple conversations over, geez, the decade about how these devices are, um, I call 'em PAS, persistent ambient surveillance, because they're quite literally constantly on, constantly listening to you, and yes, they record everything. 4:11 I think there's well over 100, uh, murder cases in the United States where evidence, um, is held by these devices, and the, and the manufacturers won't release the evidence 'cause they don't want, they don't wanna admit that they're listening to everything. 4:23 So, um, you know, A- Apple, Apple will do this just like they do with Siri right now. 4:29 And, um, I mean, e- even, I think it was 12 years ago, Samsung, uh, basically admitted that all of their t- flat panel TVs that you buy have a pinhole camera in the front of them, and they're watching you while you're watching them. 4:40 So this has been going on for a long time. 4:43 Yeah, you know, we had, uh, a while back, I think I had the, the head of, uh, Amazon Alexa Privacy on, and, um, you know, they, they, they had some, you know, what he referred to as kinda safeguards in place, the wake word and, and, and things like that. 4:58 What's interesting about the Amazon Plus service that I'm reading is it, it's supposed to be, as I'm reading it, it's supposed to be more conversational, and, and they, and they even hinted at the idea that you may not need, you know, the, the rep- the repetition of a wake word, which, you know, yeah, lends itself to, okay, so now if it, if I was wondering if it was listening before, is it listening, you know, all the time now? 5:20 Or how do you, how do you mitigate that, or, or do you, right? Is that just a, a, a risk? 5:24 But then you have the, you know, the other part of it, which is what I always tell people, which is, "Hey, you know, you may not have it in your house, but, you know, you go into your neighbor's house, they have it," right? 5:33 If you don't have the Ring doorbell, all your neighbors do, so you still have a camera pointed at you. So I don't know. It's, uh- Well, it's, yeah... 5:39 it's a balancing act of, uh, innovation versus, well, you know, inevitability. The, um, yeah, e- everyone's talking about how they're running out of training data for generative AI. 5:49 Well, what better way to generate a whole bunch more data than to constantly listen to people as they talk through their day? 5:55 So that's a way of, of, uh, generating vast additional quantities of, of language sampling that would allow them to further train these models. 6:05 So, um, yeah, I'm, I'm not the least bit surprised that they're going this way, uh, 'cause they're already tapped out on, on internet content, which is kind of staggering to think, but it's true. 6:14 That is, that is kinda crazy to think. Um, 6:18 the, there's a article this week also, uh, actually it was, it was last week, uh, uh, pr- this AI startup Perplexity, uh, CEO got into some, uh, some controversies, you know, during the New York Times employee strike, which looks like it's just ended yesterday. 6:34 But, uh, he initially said that he would, uh, offer Perplexity support to New York Times while, uh, you know, using its AI while tech- tech workers were, uh, o- on strike. 6:46 And, uh, you know, he claims it was just to, to help support and wasn't, wasn't a bargaining technique. 6:52 Um, but, uh, yeah, tho- those kinds of things are, I think, what a lot of people are worried about around the, the role of AI replacing, uh, jobs, right? 7:02 Those in the commoditized content creation business are, are in a lot of trouble. We saw this last year, I think, with, uh, Hollywood and the writers in Hollywood. Yeah. 7:10 Um, you can create halfway decent content almost instantaneously. 7:16 Uh, I- I've been doing it myself, just playing with it, some of the image generation stuff, and it's, takes seconds to come up with whatever you can create. 7:23 So the, those that generate, again, sort of commoditized content like the news, you're not, you're, you're, you're not long for this world. Time to start learning how to make sandwiches. 7:33 Yeah, and, and that was, that was more the technical, you know, the tech side of it, right? But you're right, I mean, on the, 7:40 on the news side, that's, you know, there's a bunch of lawsuits going on, as, as we've been, you know, uh, covering on this show about, you know, IP rights and, and what that new business model... 7:49 Some, some, some media companies seem to be embracing it, and it's kind of a resurgence for them of, you know, monetiz- monet- monetizing, uh, monetization techniques, monetizing the, the content that they're creating, especially some of the more s- the smaller and mid-market, I think, uh, regional, uh, players, while, you know, a lot of the larger ones are, you know, in, in, in lawsuits. 8:08 So it seems like a Tale of Two Cities there. You know, some, some are, are fighting it, some are embracing it. 8:14 Um, but, uh, listen, the good news is, uh, A- uh, Starbucks is working on a AI algorithm to sequence how orders are made and hand it out. Clearly, that's not controversial. 8:26 That's one area of AI that we can all agree, ought to be nice to, uh, clear up the process and make sure you get your drink on time and not get stuck in line on, uh, uh, uh, if you have a quick stop in, in, in Starbucks. 8:39 So there's a- I, I, I think when-... easy winning use case, right? Being able to buy your $12, uh, cappuccino more efficiently, I think, is, is game-changing for sure. 8:48 I, I can understand why we put $100 billion into this technology. [laughs] Well, it's a new, it's a new CEO. 8:54 Apparently there's a lot of complaints about, uh, I guess the, the experience, which is a big chunk of Starbucks. If you're gonna pay, if you're gonna charge $12 for a coffee, you better have a good experience. 9:03 But people are saying the, the experience has not been, uh, as good as it once was. So yeah, they're trying to, uh, uh, revive sales. Uh, so, you know, I'm all, I'm all for, uh, I guess those are the easy ones, right? 9:15 The, the high impact, high risk ones are, you know, use cases are, are, are different, right? But the, the low risk, potentially high impact, why not? 9:26 It, use case selection is absolutely key in having any measurable success with this technology. It's, it's... I, I posted to this effect earlier in the week. This is RPA squared. 9:37 You know, just as RPA was useful if you had the right use case, uh, generative AI is even more use case specific. So, you know, choose well, and you might actually get some be- benefit out of it. 9:47 Last thing I wanted to throw at you is, uh, interesting, uh, survey came out, research came out from, from Slack, their, their latest, uh, workforce index. 9:57 Um, and they're basically saying that the a- the adoption level of employees using, you know, things like ChatGPT and, and, uh, large language models is actually starting to, to, to drop off. 10:10 They, they surveyed over 17,000 global workers, and, uh, the top two reasons were, you know, one being uncomfortable admitting to managers [laughs] that they were, uh, uh, using it. 10:22 Um, and the other is, you know, and relevant to our topic today is, you know, fear over, you know, governance, policies, you know, within the company. Um, so I thought that was, uh, interesting. 10:35 Me- meanwhile, another, another study came out saying that business spending on AI vendors in the third quarter increased 38%. So it's interesting that, that it's kind of flipped, right? 10:47 It was, it was just kinda, I think, the individuals that drove the initial excitement around large language models, the democratization of this stuff, and now that seems to be slowing down a little bit, although I, I bet it's temporary, uh, while kinda companies are, uh, adopting not only, uh, OpenAI, but, um, they found 22% of OpenAI customers are spending money on rival, uh, large language model vendors at the same time. 11:15 And a- again, everything repeats itself. 11:17 This is exactly what we saw back in the RPA days, where one of the keys to being successful in an RPA implementation is getting the buy-in and input from the very people you were trying to replace with the technology. 11:30 And so, you know, those people are naturally gonna push back against the adoption curve. And then also using more than one platform, um, another lesson we learned with RPA. Um, the tools were not identical. 11:42 They, they had different strengths and weaknesses, and some, um, lent themselves better to certain use cases over others. 11:49 So while many organizations, when they're doing RPA, were looking for the one perfect silver bullet solution, those that succeeded with it generally had two or three different platforms and, and applied them to their best use. 12:02 So I, I'm not, I'm not surprised at all that people are coming to that same conclusion, uh, in the generative AI space. Yeah, yeah. 12:09 A- a- interesting also, half of the top 10 fastest growing AI software vendors were, were startups. So, you know, there was this kinda talk, uh, a few months back about investment in the AI space also slowing down. 12:24 It, it seems to not be the case, or is, is making a revival. Um- Uh, still fueling kind of VC, uh, uh, growth. 12:32 Uh, and o- of, of the startups, the top categories were probably no surprise, video generation and editing platforms, uh, and, uh, AI code assistant software development platforms. 12:45 Yeah, it's interesting following the code development side where, um, a lot of people are starting to find that it does, you know, it, it writes halfway decent code. Um, but then how do you troubleshoot it? 12:58 How do you test it? How do you confirm that there aren't, um, uh, inherent vulnerabilities in that code? 13:04 So when you actually, w- when you look at the full life cycle of using the tool for that sort of stuff, it, it's, it's sort of questionable if you're actually gaining anything in terms of time or, or time savings or efficiency, and you're also introducing a whole host of new risks that may not be quantifiable until they reveal themselves, which is not a great way to operate. 13:23 Um, but it's a great segue into our discussion about governance today. Yeah, sounds good. So, uh, today's kind of part two of our, uh, AI governance discussion, right, from a, from a few weeks back. 13:36 I wanna invite our guest today. We're joined by Joe Shepley. He's an AI governance expert and managing director at Alvarez and Marsal. Joe, it's great, g- great to be with you. Thanks for joining us. Hey, thanks, guys. 13:46 Always a pleasure. 13:48 So for those that don't know you, Joe, give, give us a little bit of your background and, and, you know, what kinds of things you're, you're working on, and then I know, uh, Chris has a couple tough questions for you. 13:57 Yeah. So, so I've been involved in broader kind of questions like information governance, records and information management, data governance, uh, for 24 years. 14:05 So I started in IT about eight or nine years, you know, doing a variety of things from application development to business analysis, and then got into consulting about 16 years ago and have been solely focused on helping organizations with information governance, although we haven't always called it that, right? 14:20 It used to be called enterprise content management, document management, whatever it is, but always had a passion for helping organizations understand the information and the data they have, understanding the value, understanding the risk, and helping them put governance in place, programs in place to manage that risk. 14:36 Um, you know, probably in the last six or seven years, uh, a lot of shifts from things like records management and information governance being sort of the red-headed stepchild and no one really caring about them. 14:48 Uh, with the rise in sort of cyber threat landscape, uh, and also the rise of privacy laws in Europe, in California, things like New York DFS in New York, um, this idea that, that the data behind the walls, uh, is part of your security profile, is part of your, uh, risk surface, like managing that risk surface is so important. 15:09 It's been amazing to watch over the last five or six years how much more seriously leadership is taking, you know, managing data properly and, in particular, making sure that we don't have data that we shouldn't have. 15:20 Um, and so it's been interesting to see with the sort of recent focus in the last two or three years on, on AI, particularly genitive, generative AI, and what challenges that poses for governance. 15:32 It's been a fun intellectual challenge to think about that and see what we can take from, from what we've been doing well and not so well in information governance, and how much does that help us when we try to apply this to the sort of brave new world of AI governance. 15:47 Chris, over to you. 15:49 [laughs] Um, as, as you sort of point out, Joe, it, it's, it's this whole governance question has been astonishingly unsexy for a really, really long time, but I tend to agree with you that, um, it's, it's coming more to the fore. 16:02 What do you think are some of the key drivers, um, beyond simply, you know, obviously AI requiring so much care and feeding, but are there other factors that you think tie into that? Well, I think, 16:13 a- and it's funny, in a way, in the US we're a little bit behind culturally places like Europe when we think about, um, like, like our personal data. 16:23 Like, I think I first heard about Google, like, blurring out people's addresses on search, like search pictures of streets like 15 years ago. I thought, "That's crazy. Why would... Like, huh? That's weird." 16:34 And then you talk to someone from Europe, and they would, they would talk about the way that they believe that, like whether they had a political sign in their window and those sort of things, like, that's my personal, like that's my personal information about me, and I own that, and I don't want Google having it. 16:48 And I think we all thought that was kind of weird 10, 15 years ago. Maybe not you, Chris, 'cause you've been ahead of the curve on this sort of stuff. But now we sort of feel like we have a right to our data. 16:56 Like, that's ours, and I'm lending it to you to get a service, or I'm lending it to you for value, or you're paying me for it, and you get that for a little while, but that's really my data. 17:06 And I think in the States, we've, we've had this increasing realization or, or awareness, change in attitudes that this data is ours. Like, the data about me is mine. It's not Apple's just because Siri is listening. 17:19 Um, they're actually kind of borrowing it or stealing it from me. 17:21 And so I, I have a right to have a say in how that's being governed, or at least I want the regulators to have a say in how it's being governed to help protect me. 17:29 And so I think that's been one of the big changes, and that poses a lot of problems for firms that have been collecting personal data in a variety of formats for 20, 30 years and never really gave much thought to how they were gonna do anything other than monetize it. 17:42 Um, it's a pretty big game of catch-up, and so that's... 17:44 A- and then I think the, the regulatory regimes and the fines and the, the bad publicity and the bad actors in the cyber world that, that leverage that sensitive data that we're storing, I think all that comes together, and it's, it's, it's making it impossible for firms to ignore the governance aspect of any kind of information, AI or not. 18:04 Well, and it's kinda hard to argue with the 10, what, trillion dollar market capitalization of those companies that have been harvesting our data and putting it to use. 18:13 Eventually you're gonna say, "Hey, I, I would like some of that value back maybe," beyond just getting free access to the platforms, which was the, the sort of, um, devil's bargain that we made with these companies 15 years ago. 18:25 Yep, that's right. And, and, you know, I think, um- 18:31 A- and again, it's funny 'cause people will say that they take privacy seriously, and they don't want their information misused, and then we have no trouble just, like, clicking okay to get an app, right? 18:39 [laughs] Like, so kind of wh- and, and, you know, we all say we wanna trust companies and that we, we feel betrayed when there's a breach and our data goes out, but, like, we all shop at Target, uh, we all get gas from BP, right? 18:50 P- people who remember Deepwater Horizon, right? So I think there's a really short memory. I think at this point, there's no company, there's very few companies left that are, are blameless. 18:59 And so I think we have sort of a, our, our indignation rises pretty quickly when something happens, and then a week later, I don't even remember who that notice of breach was from. 19:07 I mean, it's not like I'd do anything about it. So in a way it's almost become like credit card, um, theft. Like, it's a victimless crime. It's like in the '90s where I was liable for the person who stole my credit card. 19:17 Now it's just like I get the money back, they never get caught, the bank writes it off, and it's sort of like a victimless crime. 19:22 So it is this kind of tension where firms are taking it more seriously, and I think citizens are, say they take it more seriously, but I don't know how much we actually behave differently based on that yet, so. 19:36 Yeah, I mean, that's, that was gonna be my question, Joe. You know, I, I, I have this feeling, and it's not scientifically backed, but I, I, I have this feeling when I speak to some of the big tech executives that 19:51 they have this, um, attitude's the wrong word, but they have this kind of, you know, feeling that, you know, regulation is like, is like just another thing that they gotta deal with. That's why they have a legal group. 20:03 That's why... And that, you know, they're gonna kind of do what they gotta do, um, and pay the fines that they, especially in the US, right? The fines are a little bit, uh, more serious in, in Europe, I think. 20:13 But, you know, at l- at least until very recent history in the US, I just felt like, you know, it was just a cost of doing business. That was kind of one school. 20:23 Then there was another school where, you know, I'd speak to people at, you know, I'll beat up on Google or Amazon because they're the big guys, you know, where it was more, not, not necessarily conscious, but it was, it was, "Yeah. 20:34 Hey, this is a big animal, and we kind of can't keep up, and we're doing the best that we can." Uh, is that, are those accurate? 20:42 Uh, you know, do, do, do you think that these organizations are really proactively trying to fix these things or resolve these things? 20:49 Or is it just kind of a, a cost of business or, you know, we're trying to keep up the best we can, but we're not gonna, you know, we're not gonna shut down our ads business over it? 20:58 [laughs] Yeah, I mean, I, I think most of the Fortune 1000 firms that I talk to, or our clients, or I just meet out in industry, I, I can't think of really any that are just like, "Ah, who cares?" You know? 21:10 "We'll be, we'll be evil with people's data." Like, nobody's saying that. But I think it's a little bit like diet and exercise and being healthy. It's like everybody knows what they're supposed to do. 21:17 Um, everybody knows why they're supposed to do it. I think everyone agrees that it's a good thing to do. 21:22 I think everybody knows that there's probably some edge case bad things that could happen if I don't do diet and exercise that I'm scared about, but there's a low probability, but if it happened, it'd be really, really bad. 21:34 But there's a whole range of, like, general things that are gonna go wrong if I don't get diet and exercise right that I know I'm gonna face, and I'm not too excited about it, but I'm, I'm, like, also not gonna be that crazy person who, like, is annoying to be with 'cause I'm, I'm so healthy, right? 21:49 A, a lot of firms fall in that middle spot. And so the firms that I'm helping are trying to do better. Um, it helps if it aligns with some of their other goals. 21:59 So, like, for example, um, if you, if I happen to bump into a firm and start working for them on information governance while they're building new data centers and they're co-located and they're building two of them, and we talk about how 70% of the data that's on premises now is probably stale, and you can probably get rid of it if we can just get legal to help us, you know, in an efficacious way go through the preservation obligations, they love that because not only are they doing the right thing and getting rid of a lot of client, consumer, and employee data that they shouldn't have, but they're also buying, you know, 30% of the storage they would've normally bought for these two data centers. 22:32 So it's a win-win. So it's, it's kind of like if I'm going to my, my reunion, high school 30-year reunion, maybe I'll be healthy for a while 'cause I wanna look good when I go back. 22:41 So I'm not necessarily doing it 'cause I'm supposed to or I wanna be healthy. I've got, like, a benefit for me that's gonna make me be healthy. 22:47 Um, I, I s- feel like companies wanna do the right thing, and the ones that are, it typically aligns with some other corporate goals or objectives, um, that dovetail with good IG, and so then you get it. 23:00 If a firm doesn't have that, you, you see less of it. 23:03 So for example, um, in firms that don't have a lot of consumer data, so, like, in oil and gas and energy outside of utilities, you see much less movement for, like, getting rid of data and cleaning up 'cause there isn't a lot of pressure from regulators and from consumers 'cause, like, they just don't have a lot of that data. 23:19 They got their employee data, and they have asset data around, like, wells and refineries. 23:24 But anybody who's consumer-facing, there's a tremendous amount of pressure because there's just so much regulation and so much scrutiny. 23:30 And, and, um, they wanna be able to go to market, and they wanna be able to clear those compliance hurdles to innovate and get market share when their competitors can't in a way that's at least plausibly compliant. 23:42 So again, for me, um, Dan, it, it just depends on, um, everyone's trying, and the ones who are succeeding or trying harder usually dovetails with some other thing they want to accomplish, right? 23:56 So, so what do, what do you advise- I'm just thinking, I'm just thinking using the term efficacious. That's awesome. Well, you know, all those grad school loans. [laughs] Gotta get my value out of them. 24:04 [laughs] Joe, what, what, what do you advise... So, so you, you, you, you're working with some of these, these organizations and, you know, they're saying to you, "Hey, we're, we wanna do the right thing. 24:14 We wanna, we wanna try harder." Are, are there some particular recommendations or, or frameworks that you're, you're giving these, these organizations around, uh, balancing compliance and innovation? 24:26 Yeah, I mean, for me, and, and if you've heard me speak before or we've met, I, I really hate consultants, and I hate how consultants make things, like, com- more complicated than they need to be. 24:36 Um, and I really do always bring it back to, like, it is really diet and exercise here. Um, hold on. My computer's doing something weird. There we go. It's, to, to me, it comes back to diet. 24:45 It's, it's like there's complex, difficult problems, which would be like when they shot that probe to the asteroid and it landed, and they got a sample and it came back. Like, that's complex and difficult. 24:56 And then there's straightforward and difficult things. So diet and exercise would be one. We all know what we need to do. It's straightforward. 25:03 The, like, the rules of what you have to do are pretty obvious, but you just gotta do it, and that's the hard part. 25:08 And for me, information governance, AI governance, records management, all the stuff that I work on, is not phenomenally complex. 25:15 Like, um, and I, I feel like if folks are telling you it's complex, if they put slides up and you can't understand it, you should probably end the meeting and kick them out. 25:22 'Cause generally, what you have to do is pretty straightforward and pretty clear, and just about anybody could learn it in a few minutes. It's the doing it that's hard. 25:30 So, um, one of the things that I'm, that I really focus on with my clients, whe- whether it's information governance, AI governance, is let's get back to the basics, right? 25:39 Because a lot of what I'm seeing around, not just, uh, I've seen it for years in information governance with AI governance. It's almost like everything's a TED Talk. It's really pie in the sky. It's really amorphous. 25:50 Um, it's incredibly complex. It's hard to follow even what they're talking about sometimes. When for me, it comes down to some basics, right? 25:57 If, when, when we're helping a firm do AI governance, step one is what AI are you doing, right? 26:03 Um, because the only reason why we're doing governance is to, is to support the compliance requirements for whatever use case, Chris, to your point, that we're using AI for. 26:11 Is it generative AI that employees are using for productivity? Is it AI that's doing automated decision-making around the candidates we're hiring or loans we're underwriting? So step one is an inventory. 26:22 So you gotta, you gotta make a list of like, what are all the use cases and all the technologies that are using AI, right? So what are we doing with it? 26:29 And then step two is, given what we're doing with it, what are the legal and regulatory obligations that intersect that? 26:35 Because whether you're subject to, you know, and there's, there's only a couple of laws that speak to AI, you know, like Colorado AI Act or the EU AI Act, right? They have AI in their name. 26:44 Um, but lots of laws and regulations don't say AI, but they say things like automated decision-making, algorithmic decision-making, um, uh, or even just general privacy laws intersect AI depending upon your use case. 26:59 So when you've listed out all the different ways you're using it, all the different technology you're using for AI, the second step is a regulatory crosswalk. 27:06 Well, given my uses of AI, what are all the laws and regulations that I'm obligated or potentially obligated to? And once you've figured that out, you go, "Okay, I wanna intersect the two. 27:16 What are the requirements then based on my legal and regulatory obligations, based on how I'm using AI in these five scenarios, 10 scenarios, 15 scenarios?" 27:26 And then you have a list of, these are all the requirements that I have for AI governance. And so now it stops being a vague thing. 27:34 It stops being this kind of generic boil the ocean thing that I have to do, and, and all these words like fairness and transparency and ethical and... That honestly, I don't know how to, you know. 27:45 Those, those are, those are really big concepts and tough ones to put into specific governance directions. 27:51 But if you follow the three-step path I just laid out, which is really straightforward, um, you now have a list of specific compliance requirements that you can build your governance program around to give you a shot of demonstrating to regulators, one, that you're doing, you're making a, a good faith, best 28:09 chance, reasonable effort to figure out how the heck I'm supposed to govern this stuff. 28:14 I've documented what I'm doing, what laws I think I'm subject to, and what requirements I believe I'm subject to, and now I'm gonna document how I'm attempting to meet those requirements over the course of the next six months, 12 months, 18 months, two years, so that I can both reap the benefits of AI but also be a good corporate citizen and do my best to comply with what frankly is a pretty confusing welter of laws and regulations that overlap and are not coordinated, right? 28:40 That's the way I look at it. So for me, it's pretty straightforward actually conceptually. It's just, it's just difficult to do. Well, and that's where I come in, right? Is I help folks just like a fitness coach. 28:50 You, I've got a Peloton, I've got a recipe book, right? I don't necessarily do any of that stuff. 28:53 Well, if I have a fitness coach, that person rides with me, stays with me, helps me stay the course, and get through what I have to get through to achieve those objectives. So... 29:05 Chris, anything you would add or disagree with? I, I couldn't, I couldn't agree more with this, this notion that there's only so many things you can do with a piece of information. 29:16 And yeah, the regulatory landscape can be extremely complex because there's so many different laws, but all those laws or, or regulations are telling you to basically do the same thing with slight variations. 29:27 But once you, you know, you store it, you protect it, you encrypt it, you keep track of who's looking at it, there's not many more things that you're gonna do with a piece of information. 29:35 And I kinda liken it, it's sorta like Mexican cooking, right? If you look at every recipe of Mexican food, there's like seven ingredients. 29:41 They just change the order or the way in which those ingredients are introduced, then you have this cornucopia of, of cuisine. So it's sorta like that. 29:49 I, I think your, your point, Joe, about making it, simplifying it down and it's difficult but not complex, I think is, is absolutely true. 29:59 Once you map out all the different regulations that'll apply to you, the number of things you actually have to do to be compliant is actually remarkably small. 30:06 Yeah, and I think the, the, um, you'll also realize that there, there, there is a remarkably smaller set of things you have to do, and you'll hone in pretty quickly on the ones that you're not sure if you can do them. 30:17 So the one that I hear people talk about most with regard to, like, large language models is data subject rights. 30:23 So, you know, if I've got a platform in my organization and I'm loading in consumer data to train the model, and then the model's done at the end, and then, you know, I was one of the consumers whose data trained that model and I, I ask to know where all my data is being stored, um, or if I ask to be forgotten, to have my data deleted, or if I ask for my data to be s- sent to me, what's the status of the model? 30:46 And we haven't had a regulator, I haven't seen a regulator speak to that yet. Um, and I haven't gotten really, nobody that I talk to feels comfortable saying, "Oh, well," and then they give me an answer. 30:56 It's just kinda like, hmm. 30:58 And so, like, that's one of those areas where if you do that crosswalk and you have your uses of AI, and you have the legal obligations, right, and you have your legal requirements, you can flag those requirements where you don't have a warm and fuzzy about doing it, where, like, I don't really know how I'm gonna support data subject rights if I do this. 31:15 And so that's when you know to go talk to outside counsel to at least have a position on it, and say, like, "I'm not gonna not do it, but hey, regulator, we thought about this and our decision was that it didn't count or it did count or we were gonna do X, Y, or Z." 31:27 But Chris, to your point, when you get it down to that manageable set, you can flag the ones where it's like, I kinda have heartburn about that one. I'm still gonna do it, but at least I know. 31:36 I know that when we get a question about that one, you gotta be set for bear, right? I don't have a great answer for that. But at, at least you did your due diligence, right? You're kinda eyes wide open about that risk. 31:48 It's a governance burrito. It is. [laughs] Right? Hopefully it stays wrapped up. Doesn't open up halfway. Hold the beans for me, right? I don't want that indigestion. Yeah. 31:57 [laughs] You guys made it easy for me to figure out what I'm getting for lunch now. Thank you. 32:00 [laughs] Uh, so what, what, what do you, what do you think if you think about if we move away from the governance side a little bit and just talk about, you know, IP protection in, in, in general. 32:10 I mean, you have the models themselves, right? You have the outputs of the models, uh, and then you have, I guess, the privacy piece of that, you know, as to what the models are, are putting out or, or, or, or sharing. 32:25 Um, what do you advise a- a- around that? Because that's, that's almost a few steps ahead. It's, it's, it's not as backward-looking as some of this stuff has been in the past, right? 32:35 You gotta almost anticipate the future with some of these, uh, la- you know, large language models, for example. Yeah. And I don't do a lot of IP work. That frankly makes me super nervous for a couple of reasons. 32:45 Well, one, I'm not a lawyer. I know Alvarez and Marsal s- sounds like a law firm, but it's not. Um, and, and I think that I, I, I don't know that anybody knows. 32:54 I, I feel like on the one hand you have this land grab, like OpenAI and folks who are just like, "Screw it, I'm just gonna suck everything in and I'm gonna make a lot of money and I'm gonna write a check. 33:01 We'll figure it out later." Um, a- and then, a- and I think that if you look back at, like, streaming platforms, right, when we go back to, like, Napster 'cause I'm old, like, that always wins. 33:12 Like, even at first we're pissed and there's lawsuits but, like, once the horse is out of the barn, like, I, I sort of feel like, one, if I'm someone who's worried about that, like I'm a, a musician or a, 33:23 a, a writer, a, like, journalist or a movie actor or whatever, I feel like you gotta get in front of it and monetize it. 33:29 Those are the people who really, like, win in this case, in these cases, are the people who aren't like, "Ugh, streaming. I, w- let's go back to selling CDs," right? Those folks, like, kinda lost, lost. 33:40 The folks who realized it was coming, they're just like, "All right, it's here. I'm gonna figure out how to stay relevant and how to monetize it." 33:45 So I, I think without being a lawyer or really knowing much about this stuff, just my, like, armchair quarterbacking, I think this stuff always trends in the direction of just being open 'cause, 'cause that's what we want, right? 33:57 I, I don't wanna have less content available to me when I'm making- So you're, you're a, you're a play to win guy. 34:02 You're a let, let's innovate and we'll figure out the- I just think, I'm not, I'm not saying I am, but I think that's what usually wins, at least in the United States, right, I think that wins. 34:10 Um, I think for corporations, though, not on the IP side, I think on the, the taking care, thinking about governance I think is really important 'cause what I hear firms worried about is that I'm gonna build a core capability for my business and innovate and grow on an AI tool or tools. 34:28 And if I say to myself, "Well, there isn't really a law yet, and it's so confusing I won't think about it till there is a law," if in three years from now you've spent all this money on it and you have a core capability of your business built around it and you haven't thought about it, and you're now sideways with a regulator and they ask you to, say, disgorge your model or stop using it or rebuild it, that's gonna be devastating. 34:51 Um, and so I feel like while that might work for, like, the folks who are building AI platforms and kinda, um, ingesting content, I think for organizations like Fortune 1000 organizations, that doesn't work 'cause when there eventually is, like, more clear regulation, you wanna show that you've thought about it so that you have a better chance of not having to just jettison whatever you've been doing for the last one, two, three, four years, and you at least have a leg to stand on to say, "Hey, y'all didn't have anything. 35:19 We went out and tried to figure out what future-proofing this thing looked like and tried to take those seven things that make up the burrito and incorporate them to the best of our ability." Right? 35:29 I think you're gonna wanna have something like that, um, so that you have the best chance to not have to start from scratch once there is, like, more heightened regulatory clarity. 35:39 So I know I just talked out of both sides of my mouth like it's all- Joe, I've al- I've often said that, you know, compliance is one of those things where if you treat it like a, 35:48 a necessary cost of doing business, that's what it'll be. Yeah. And instead if you treat it as if it, it's an enabler to do this sort of disruptive innovation, then that's what it'll be. 35:57 It's, it's all how you approach it. Yeah. And it's, it's a hurdle for everybody. 36:01 So, like, the firms that I work with that I've, that I've probably, that I feel like have the most strategic value out of information governance are the ones who love the fact that it was hard to do this stuff 'cause that means that my competitors can't do it, and that means that if I do do it, I can operate knowing that I'm gonna be okay, and I'm gonna watch them in the news get hit by New York DFS or whoever, California, because I know that I've cleared that hurdle and now I've got space to operate and, and compliance is sort of taken care of. 36:27 And it is. It's an enabler. So the folks who look at it that way, you're right, they get tremendous value out of information governance, and I think those firms will get tremendous value out of AI governance or anything. 36:36 Um, but yeah, if it's, you know... If, if, you're right, if you look at... And I've never heard it said that way. If you look at it as a necessary evil, it'll always stay a necessary evil, right? Joe, is, is the, 36:47 at l- at least in my perception, it used to be that- The larger companies really carry the regulatory burden that the, the smaller companies didn't really have to, you know, worry about it quite, uh, uh, uh, as much. 37:00 Is that, is that changing knowing that so much of this growth is happening from the startup world? Yeah, I, I mean I think that's been changing for a while and it, it hasn't, I haven't seen it so much, um... 37:11 I mean, I think, yes, that is one factor, but for me it's the third party risk in cyber is what's causing it, right? If you look at a lot of the high profile breaches that I'm not allowed to mention 37:21 by my legal team at [laughs] my firm, but j- pick any of them. It's never, like, the firm that y- the name that you recognize that got hacked. It's the, like- Right... HVAC vendor. It's the, like, third party, right? 37:33 I m- and that's just the way it is. And so I think a lot of the, and the regulators understand that. 37:38 Like, you can regulate, name the top tier n- 10, top 10 banks in the US, but the real risk is not coming from one of them getting hacked. They have, you know, 4,000 people in cyber. 37:50 It's, it's that third party software vendor, or it's that professional services vendor that has email credentials in which is how it's gonna happen. 37:58 So I think the regulators who kind of have to do that because that's where the risk is for the financial system for, and name the 14 key infrastructures, right? 38:06 That's where the risk is, is that the breach happens at the little person, not the huge energy company. So that's part of... 38:13 And so as, as littler firms, I mean we're, we're, you know, 10, 11,000 people, but even when I've been at smaller firms, I've been at 15 person firms, we were held to pretty high standards when we did business with Fortune 500 companies 'cause we are part of their, we are part of their risk surface, and we made them vulnerable even though we're only 15 people. 38:30 So I think with AI it's, it's only, it's, it's gonna continue and maybe increase given how easy it's gonna be for a five person shop to build a tool that could be used by a Fortune 500 company. 38:42 So- Were you held to, were you held to a high standard by the company you were doing business with? Yes. Or from the regulatory, yeah. By the company we're doing business with b- um, because of the regulation. 38:53 So if you think about it, like, um, it's just been on my mind this week, ISO 2701, right, which is a cyber standard. 38:59 There's guidance 2702, um, which in 2022 introduced a little sentence that said you have to delete data when you no longer need it, kind of like New York DFS, and we're starting to see firms coming up, starting to use that 2022 guideline recently to, like, recertify. 39:14 And that one's a little bit of a sticky point 'cause nobody's doing it. And so, like, uh, sometimes it'll be that. 39:20 It'll be that, um, it'll be that, that they have an ISO certification, they have, you know, SOC 2, whatever. And so anybody that they do, they do business with, those, those third parties get, have certain obligations. 39:31 And certainly, like, things like CPRA, if I'm handling sensitive data from a client, I'm subject to CPRA in a range of ways that I better be clear on. 39:40 So I think it's driven a little bit by the regulators driving it to my client, my client pushing down to me as part of their third party risk management, um, capabilities. Interesting. 39:54 Chris, any- Well, and I think a- a- as we're seeing with AI, AI is gonna be such a massive vector for additional, um, vulnerabilities, and it, it's showing itself more and more. 40:05 And, and like often the case with these sorts of bugs or w- or vulnerabilities, um, you're not gonna recognize it, it's there until it actually hits. 40:14 And I know a lot of the research papers that I'm looking on this topic now, the, the, the, even, like, open AI is rife with these sorts of additional vulnerabilities. 40:23 And how, how are you gonna protect yourself, um, when they're not even seeable or visible until they actually execute? Well, and it's a blessing and a curse, right? 40:33 Like, I know that cyber professional services firms and cyber teams and companies use AI to help find vital vulnerabilities. 40:40 And of course, the bad, the bad folks are using AI to c- create vulnerabilities, but that's, like, welcome to cyber, right? It's always asymmetric. You have to win... I have to get it right every single time. 40:50 They only have to get it right one time. Once, yeah. Or my employee only has to click on a link once. That's it. Yep. So it's like cyber's a special kind of crazy, right? You gotta really enjoy, [laughs] 41:01 you have to enjoy that asymmetricity. Um, and you only- You, you only have to be right once through the absolute weakest part of the whole chain. Right. Yeah. It's, it's exhausting, right? 41:09 That's why, a- and it takes a certain kind. If you know people who work in cyber, there's a certain sort of, like, footprint personality that sort of thrives in that world. Um, so I work with them a lot. 41:19 [laughs] Joe, my, my last question is around, uh, decision-making and bias. Uh, do, do you, are you seeing that in, in the regulations, and, and how are you advising clients to, to deal with that? 41:32 Is it, is it still keep humans in the loop or are there other ways to mitigate those, those risks? 41:38 Yeah, I mean, it's always keep humans in the loop and, and I advise them it only in the sense that, right, so again, that's a really a legal question. 41:45 So we can help clients with knowing that that's a requirement because of the way they're using AI. We can direct them to the kind of requirements that we are seeing in a regulation or a law or a standard. 41:56 But our advice to clients is that when you're doing, whether it's AI governance or just plain old information governance, that outside counsel is really critical. Um, because again, it's, 42:07 the answer to those questions is really a legal one for them. 42:10 Um, but I see clients, the, probably the, the one of the first places we saw folks using AI was embedded in tools for hiring, hiring decisions and performance analysis for employees. 42:20 And so that's just, like, ground zero for, for these kind of, like, bias and discrimination. And firms, and I haven't seen a single firm that isn't worried about that when they're using it. 42:28 Like, to their credit, you know, firms have a really good ethical north star around that. Nobody wants to get that wrong. Um, everybody wants to do a better job hiring. 42:37 Everybody wants to have a better workforce, attract the best folks, you know, really make good hiring decisions. So it's, I, um, I just think, uh, I don't know that... I think it's always gonna be a moving target, right? 42:49 I think, um, it's, it's certainly made it much harder- To identify potential bias and to, and to evaluate it. 42:55 Um, and I think that it's even more important for firms to be seeking good legal guidance on this, and you really wanna find a firm that has AI experience, right? 43:04 Um, it's easier now than it was maybe three or four years ago, but to find those larger law firms that have been working with your peers, uh, and have folks who really understand automated decision-making, algorithmic decision-making, uh, who potentially are not only lawyers but have a technical background, I think is really important because it, it, it, it is significantly different in what you need to be thinking about than just good old-fashioned hiring and discrimination. 43:27 And so you really wanna find a law firm, um, as your advisor that, that specializes in that, which is getting easier to do, so. Yeah. No. Sorry I didn't have a good hot take on that, but again, I'm not a lawyer, so. 43:40 No, that's good. Well, that's [laughs] even better. I'm certainly learning. At the risk, at the risk of, uh, getting hate mail from all my lawyer friends, I, I think that's, uh, you know, uh, to... 43:49 These, these decisions have to be made holistically, right? If you, if you look at, if you ask a l- a lawyer a legal question, you get one perspective. 43:55 You ask a, you know, procurement officer, you get one perspective, right? But I, I think, I think part of the challenge we have in all of these areas is that, uh, it's a more holistic conversation. 44:05 It's a weighing of pros and cons and, you know, you need, you need multiple point of views at the, uh, at the table. So really appreciate it, Joe. Thanks so much for joining us, and, uh, hope we can have you back. 44:16 Uh, I used to tell people, like, you know, in a year or so, but things change so quickly now, I think we gotta have you back on in a few weeks or maybe a couple months. [laughs] Any, any time. 44:24 I really appreciate it, gentlemen. Thanks so much, Joe. Take care, Joe. So Chris, this is the second guest, uh, I think we've had in, uh, the last few weeks that you haven't disagreed with at all. 44:32 You're gonna, you're gonna lose your reputation. [laughs] Well, no, we're just being more selective of who we're letting on the, on the platform, right? [laughs] AI bias. No, well, I, it's, um... 44:43 I, I think it's good to have both, you know, the, the common understanding and so forth, just to reinforce some of the points, but then also I'm, I'm very fond of having the, uh, the knock-down, drag-out fights with folks as well. 44:54 I think they're both valuable. That's the way we get, we make progress, right? Yep. Well, thanks for joining us, everybody. We're, we're back, uh, next Wednesday at noon Eastern, 9:00 Pacific. 45:05 In the meantime, if there's anything you're working on and want some free advice, or just wanna learn some art of the possible from our team of subject matter experts, you can schedule a free networking call at erpaai.com/networking. 45:16 In the meantime, have a great week, everybody. Enjoy your burrito, Dan. Yes. Thank you. [laughs] Take care. 45:22 [upbeat music]