1 00:00:00,942 --> 00:00:01,562 You're welcome. 2 00:00:01,562 --> 00:00:03,246 It's so great to talk to you. 3 00:00:03,246 --> 00:00:04,516 It's so great to be here, Kevin. 4 00:00:04,516 --> 00:00:05,734 Thanks for having me. 5 00:00:06,112 --> 00:00:10,592 You've been involved for a long time at a number of different organizations, public 6 00:00:10,592 --> 00:00:16,452 sector, private sector, nonprofit, on issues around internet policy, privacy, 7 00:00:16,452 --> 00:00:18,112 and so forth. 8 00:00:18,232 --> 00:00:24,072 How is it different or is it different in your perspective what's happening now with 9 00:00:24,072 --> 00:00:24,861 AI? 10 00:00:25,646 --> 00:00:33,326 I think that I just heard a ding, so I'm going to have to start over again to 11 00:00:33,986 --> 00:00:36,106 silence my notifications. 12 00:00:36,706 --> 00:00:37,426 Okay. 13 00:00:38,186 --> 00:00:41,286 You know, Kevin, I think there are a lot of conversations right now about whether 14 00:00:41,286 --> 00:00:46,826 this is truly different and kind of the really monumental change in how we live 15 00:00:46,826 --> 00:00:47,746 our lives. 16 00:00:47,746 --> 00:00:51,966 I'm a little bit of a skeptic about that in that I guess perhaps because you and I 17 00:00:51,966 --> 00:00:54,766 have been working in this area for so long, we've seen 18 00:00:54,766 --> 00:01:01,886 the trains of large language models and automated decision making and just 19 00:01:01,886 --> 00:01:04,766 software that runs really fast in the background. 20 00:01:04,766 --> 00:01:09,726 Why I think this feels really different to people is that, first of all, this has 21 00:01:09,726 --> 00:01:10,506 come into the wild. 22 00:01:10,506 --> 00:01:17,486 I mean, it's come into the public use very quickly and that it seems to be alive. 23 00:01:17,486 --> 00:01:21,866 People seem to feel like this is making decisions about them or it's talking back 24 00:01:21,866 --> 00:01:22,670 to them. 25 00:01:22,670 --> 00:01:29,030 And although we know cognitively it's not alive, having a converse with you, having 26 00:01:29,030 --> 00:01:34,910 an inanimate object suddenly talk your language feels different than previous 27 00:01:34,910 --> 00:01:37,210 iterations of technology. 28 00:01:37,650 --> 00:01:41,070 Why I think it's important that we scrutinize it is, and I've said this 29 00:01:41,070 --> 00:01:45,210 before, it's not only the data that we're putting into it, it's the decisions it's 30 00:01:45,210 --> 00:01:47,214 making about us in the world. 31 00:01:48,462 --> 00:01:52,062 What about the development of AI governance and policy? 32 00:01:52,062 --> 00:01:55,882 How have you seen the regulatory developments in this area around AI 33 00:01:55,882 --> 00:01:59,782 relative to what we saw a decade ago or two decades ago with the internet and 34 00:01:59,782 --> 00:02:00,490 privacy? 35 00:02:00,782 --> 00:02:06,162 I think because it's moved so quickly from not known to known in the public, the 36 00:02:06,162 --> 00:02:10,602 policymaking world is trying to catch up as usual to technology and doing it 37 00:02:10,602 --> 00:02:11,802 quickly. 38 00:02:11,882 --> 00:02:17,262 What I've also seen is that good companies, I think, are ready to apply 39 00:02:17,262 --> 00:02:20,882 governance principles to new technologies because this is not the first one. 40 00:02:20,882 --> 00:02:22,766 We have internet, we have... 41 00:02:22,766 --> 00:02:27,806 social media, we have lots of policies around other uses of technologies by our 42 00:02:27,806 --> 00:02:30,746 own, what we call employees, our associates. 43 00:02:30,746 --> 00:02:36,466 And so we really, we in particular, my company, we're ready. 44 00:02:36,466 --> 00:02:41,006 We had already thought about and started to build and started to benchmark with 45 00:02:41,006 --> 00:02:42,146 other companies. 46 00:02:42,146 --> 00:02:44,974 What does good governance look like? 47 00:02:44,974 --> 00:02:46,204 for these tools. 48 00:02:46,204 --> 00:02:49,224 And of course, you know, just say AI, it's almost like saying sidewalks, right? 49 00:02:49,224 --> 00:02:52,154 It encompasses so much, right? 50 00:02:52,154 --> 00:02:57,114 But to be general, kind of looking at emerging technologies, biometrics, AI, and 51 00:02:57,114 --> 00:03:03,154 a whole bunch of other de novo uses of technology in the real world, proximate to 52 00:03:03,154 --> 00:03:08,334 human beings, we did set about looking at what the tech companies were offering, 53 00:03:08,334 --> 00:03:11,474 looking at what other companies were recommending, looking at what trade 54 00:03:11,474 --> 00:03:15,118 associations and think tanks were saying about, 55 00:03:15,118 --> 00:03:17,598 governance and the possibilities of governance. 56 00:03:17,598 --> 00:03:20,918 I also sometimes like to call it trying to govern the ungovernable, right? 57 00:03:20,918 --> 00:03:26,178 Because it's so opaque in so many ways, but it's also, we shouldn't engage in that 58 00:03:26,178 --> 00:03:30,038 kind of tech exceptionalism that we are so special and so new and so different that 59 00:03:30,038 --> 00:03:36,058 we shouldn't play by some reasonable rules or that it's just too hard to figure out. 60 00:03:36,058 --> 00:03:38,798 If it's too hard to figure out, we shouldn't be using it, right? 61 00:03:38,798 --> 00:03:44,334 So what we did was look at ordinary processes that companies put into place. 62 00:03:44,334 --> 00:03:50,374 what the company's principles are, and then how we apply those as people. 63 00:03:50,374 --> 00:03:55,974 And it's usually our engineering, our data scientists, our tech folks who want to 64 00:03:55,974 --> 00:04:00,174 deploy new tools that will affect human beings. 65 00:04:00,174 --> 00:04:04,994 We pull those out of the work streams and say, if this tool is going to be making 66 00:04:04,994 --> 00:04:09,744 consequential decisions about a human being, their rights, their knowledge, 67 00:04:09,744 --> 00:04:12,686 their abilities to move forward, in most particular, 68 00:04:12,686 --> 00:04:19,206 their employment and advancement, that tool is going to require additional close 69 00:04:19,206 --> 00:04:24,786 scrutiny by technologists and by lawyers and policy people before it goes live. 70 00:04:24,786 --> 00:04:29,206 And then at waypoints, you know, six months a year, and then perhaps annually 71 00:04:29,206 --> 00:04:33,476 after that, because as you know, these tools drift, they change, that, you know, 72 00:04:33,476 --> 00:04:35,756 it is a constant governance model. 73 00:04:35,756 --> 00:04:39,706 And we're still building towards automation in our governance and 74 00:04:39,706 --> 00:04:42,574 monitoring of tools of all kinds. 75 00:04:42,574 --> 00:04:46,714 And, but again, I don't think just because it's new and funky and cool, it should 76 00:04:46,714 --> 00:04:48,394 escape some kind of scrutiny. 77 00:04:48,394 --> 00:04:50,758 In fact, maybe, maybe more so because of that. 78 00:04:52,046 --> 00:04:57,956 So you issued a Walmart Responsible AI pledge in October of 2023. 79 00:04:57,956 --> 00:05:00,666 And first of all, I'm curious, it's interesting that you call it a pledge. 80 00:05:00,666 --> 00:05:02,766 It's not just a set of principles. 81 00:05:03,286 --> 00:05:07,270 Tell me a little bit about the process of developing that. 82 00:05:07,918 --> 00:05:12,478 And like all of our kind of internal policies, statements, pledges, it's really 83 00:05:12,478 --> 00:05:15,328 a multifunctional endeavor. 84 00:05:15,328 --> 00:05:18,598 I don't think the lawyers should sit in a room and write things. 85 00:05:18,598 --> 00:05:21,418 I don't think the technologists should sit in a room and write things. 86 00:05:21,418 --> 00:05:23,158 The comms people shouldn't be working by themselves. 87 00:05:23,158 --> 00:05:27,698 It was really a kind of collaborative approach that a number of our teams came 88 00:05:27,698 --> 00:05:32,558 to my team and said, we'd like to say something publicly about the guardrails 89 00:05:32,558 --> 00:05:37,078 and the guidelines and really the values that we live by as a company. 90 00:05:38,158 --> 00:05:40,318 And I said, really, who's going to listen to that? 91 00:05:40,318 --> 00:05:42,198 Again, somewhat the skeptic. 92 00:05:42,198 --> 00:05:45,458 But in all seriousness, the pledge is really to our customers and associates, 93 00:05:45,458 --> 00:05:52,618 people who we are engaging with to both be transparent and accountable in our uses. 94 00:05:52,998 --> 00:05:57,838 And that sort of really reflects one of the things I say to my team all the time. 95 00:05:57,838 --> 00:06:03,178 When we are introducing new technologies into the environment of human beings, 96 00:06:03,178 --> 00:06:06,990 whether it's online or in the stores, in the real world, 97 00:06:06,990 --> 00:06:13,630 in offices and in warehouses anywhere our associates work, we have a duty to explain 98 00:06:13,630 --> 00:06:18,010 because in many cases it may be the first time someone's come into contact with 99 00:06:18,010 --> 00:06:18,650 this. 100 00:06:18,650 --> 00:06:23,050 And so I always say to our team, inform, educate, entertain. 101 00:06:23,050 --> 00:06:24,500 And let me tell you what I mean by that. 102 00:06:24,500 --> 00:06:28,230 Inform, obviously notice that, you know, that something is happening in the 103 00:06:28,230 --> 00:06:29,410 environment that is different. 104 00:06:29,410 --> 00:06:32,450 It's particularly in the stores, which were, you know, 20 years ago, you could 105 00:06:32,450 --> 00:06:35,690 walk in, you could walk out, you could pay cash, you would be unobserved to any 106 00:06:35,690 --> 00:06:36,846 system in the world. 107 00:06:36,846 --> 00:06:41,286 That's changing, obviously, with your cell phones, with electric cash registers, with 108 00:06:41,286 --> 00:06:44,646 cameras in the stores for security and safety. 109 00:06:44,785 --> 00:06:46,806 So I think we have a duty to inform. 110 00:06:46,806 --> 00:06:50,626 We are meeting people where they are in their comfort level with technology, in 111 00:06:50,626 --> 00:06:53,926 their educational level, in their language abilities. 112 00:06:53,926 --> 00:06:56,806 And we have to speak to all of those people. 113 00:06:56,806 --> 00:06:57,846 So that's inform. 114 00:06:57,846 --> 00:06:58,866 Educate. 115 00:06:58,866 --> 00:07:00,566 Tell them a little more. 116 00:07:00,566 --> 00:07:04,622 Tell them a little more about what the technology does, how it works. 117 00:07:04,622 --> 00:07:07,652 When I grew up in New York, there was a company called Sims. 118 00:07:07,652 --> 00:07:11,282 I actually had a summer job there one summer and their phrase was, an educated 119 00:07:11,282 --> 00:07:13,522 consumer is our best customer. 120 00:07:13,522 --> 00:07:17,302 I believe the same thing about technology and the technology and responsible 121 00:07:17,302 --> 00:07:18,882 technology use of our company. 122 00:07:18,882 --> 00:07:23,122 If they know the responsible choices we're making about these technologies, it will 123 00:07:23,122 --> 00:07:28,422 engender trust and mutual respect and hopefully really benefit the ongoing 124 00:07:28,422 --> 00:07:30,422 customer store relationship. 125 00:07:30,422 --> 00:07:32,342 And the last part is entertain. 126 00:07:32,342 --> 00:07:33,742 And honestly, 127 00:07:33,742 --> 00:07:34,532 This is fun. 128 00:07:34,532 --> 00:07:38,462 You and I were talking a little while ago, you know, I, in all of the years we've 129 00:07:38,462 --> 00:07:41,822 been working, we, I get to learn something new almost every day. 130 00:07:41,822 --> 00:07:44,782 And, you know, certainly every year there's a new, you know, or every few 131 00:07:44,782 --> 00:07:47,602 years there's some monumental change in the technology. 132 00:07:47,602 --> 00:07:48,902 That's a joy. 133 00:07:48,902 --> 00:07:54,122 It's really a joy to be a part of that and to learn and to be surrounded by people 134 00:07:54,122 --> 00:07:58,702 who are just so, so smart and dedicated and both want to do the right thing and 135 00:07:58,702 --> 00:08:02,574 also want to use the latest technology to do whatever it is their job. 136 00:08:02,574 --> 00:08:03,814 requires. 137 00:08:04,314 --> 00:08:05,534 So and it should be fun. 138 00:08:05,534 --> 00:08:09,404 We had a long time ago, we had these sweepers in some of the stores. 139 00:08:09,404 --> 00:08:12,374 And if you go into Sam's Club, you might still see them. 140 00:08:12,374 --> 00:08:18,354 And they forget the brand name, but they were about the height of a, you know, I 141 00:08:18,354 --> 00:08:19,784 don't know, 10 or 12 year old boy. 142 00:08:19,784 --> 00:08:22,394 I'm thinking about my son is probably, you know, a few years ago. 143 00:08:22,394 --> 00:08:24,974 And but almost almost eye level. 144 00:08:24,974 --> 00:08:25,874 And they had cameras. 145 00:08:25,874 --> 00:08:30,234 And the purpose of the camera was actually to scan the shelves to say what's out of 146 00:08:30,234 --> 00:08:31,342 stock so that 147 00:08:31,342 --> 00:08:36,442 people in the back store room would go, okay, on aisle seven, we need to get more 148 00:08:36,442 --> 00:08:37,622 cereal. 149 00:08:38,362 --> 00:08:42,702 But a reasonable person would think, was that camera looking at me? 150 00:08:42,702 --> 00:08:43,652 What's that camera? 151 00:08:43,652 --> 00:08:44,492 What's it recording? 152 00:08:44,492 --> 00:08:46,122 What's it gonna do with that information? 153 00:08:46,122 --> 00:08:49,262 So I said to the guys, first of all, I thought we should have put vests on them 154 00:08:49,262 --> 00:08:54,482 and made them, anthropomorphize them and also have disclosures right there on the 155 00:08:54,482 --> 00:08:56,522 machine that says, I'm not looking at you. 156 00:08:56,522 --> 00:08:58,766 In fact, we had blurred out any facial. 157 00:08:58,766 --> 00:09:05,106 features of any human being to just be looking at boxes and QR codes and the 158 00:09:05,106 --> 00:09:05,946 like. 159 00:09:06,366 --> 00:09:07,666 But it can be fun. 160 00:09:07,666 --> 00:09:11,986 I have no problem with anthropomorphizing the technology if it helps people 161 00:09:11,986 --> 00:09:14,470 understand and feel comfortable with it. 162 00:09:16,494 --> 00:09:17,264 me just stop a second. 163 00:09:17,264 --> 00:09:20,934 Do you want to see if you can shut off that other thing that's dinging? 164 00:09:22,314 --> 00:09:24,214 If it's too much of a pain, it's not a big deal. 165 00:09:24,214 --> 00:09:26,564 This is not Hollywood, but yeah. 166 00:09:26,564 --> 00:09:28,014 I don't know what that is. 167 00:09:28,494 --> 00:09:29,374 okay. 168 00:09:30,022 --> 00:09:32,182 how do I shut off the notifications? 169 00:09:32,182 --> 00:09:34,430 Let's take a look on my 170 00:09:37,006 --> 00:09:39,066 Okay, I think it's gone. 171 00:09:39,766 --> 00:09:42,150 I think it's all off now. 172 00:09:42,638 --> 00:09:43,148 Alright, thank you. 173 00:09:43,148 --> 00:09:44,526 Yeah, it was not a big deal, but... 174 00:09:44,526 --> 00:09:47,246 No, no, we don't want it in the background. 175 00:09:47,646 --> 00:09:47,966 Okay. 176 00:09:47,966 --> 00:09:48,845 Back at it. 177 00:09:48,845 --> 00:09:49,505 Okay. 178 00:09:50,546 --> 00:09:51,758 Hope that works. 179 00:09:51,758 --> 00:09:52,668 No, OK, we'll see. 180 00:09:52,668 --> 00:09:54,378 All right. 181 00:09:58,094 --> 00:09:59,164 That's a good. 182 00:10:07,982 --> 00:10:08,902 first. 183 00:10:09,418 --> 00:10:10,098 no, no, you're okay. 184 00:10:10,098 --> 00:10:13,458 Yeah, it's, I mean, whatever this, this tool Riverside is recording each of us 185 00:10:13,458 --> 00:10:14,698 individually and uploading it. 186 00:10:14,698 --> 00:10:15,418 So it should be fine. 187 00:10:15,418 --> 00:10:16,390 Even if the. 188 00:10:18,686 --> 00:10:20,566 Okay. 189 00:10:22,190 --> 00:10:26,870 Yeah, it's interesting that one of the values that is in that pledge is customer 190 00:10:26,870 --> 00:10:30,760 centricity, which is not typically what people see in terms of AI. 191 00:10:30,760 --> 00:10:33,570 You've got things like fairness and so forth. 192 00:10:33,570 --> 00:10:39,110 But why is customer centricity for Walmart a value in terms of responsible AI? 193 00:10:39,566 --> 00:10:44,786 I think so many times companies roll out technologies that they say are for the 194 00:10:44,786 --> 00:10:48,366 customer, but are really for the company or really have some other purpose. 195 00:10:48,366 --> 00:10:52,726 And in this case, when we say customer centricity, it really goes to what I was 196 00:10:52,726 --> 00:10:53,866 saying before. 197 00:10:53,866 --> 00:10:55,525 Does the customer understand? 198 00:10:55,525 --> 00:10:57,156 Do they feel comfortable with it? 199 00:10:57,156 --> 00:11:00,866 Is it a use case that enhances their experience? 200 00:11:00,866 --> 00:11:03,966 That's, I think, some of the values I think about when I think about customer 201 00:11:03,966 --> 00:11:04,766 centricity. 202 00:11:04,766 --> 00:11:08,654 But ultimately, it's about does this deployment... 203 00:11:08,654 --> 00:11:12,914 serve the humans with which it is proximate. 204 00:11:13,494 --> 00:11:15,954 And if it doesn't, why are we doing it? 205 00:11:15,954 --> 00:11:20,614 And if those customers or associates don't understand it, then we have failed in our 206 00:11:20,614 --> 00:11:24,974 duty of care, our duty to inform, educate, and explain. 207 00:11:25,254 --> 00:11:30,814 So I think it really goes to, we have recognized that people will be impacted 208 00:11:30,814 --> 00:11:35,470 even just having it in their environment, having it nearby, and that... 209 00:11:35,470 --> 00:11:40,210 There may be different levels of comfort with it and that we have a responsibility 210 00:11:40,210 --> 00:11:43,430 to those, to all of our customers, regardless of their comfort. 211 00:11:44,718 --> 00:11:50,038 What else do you think is distinctive about the way you approach these AI 212 00:11:50,038 --> 00:11:54,398 issues, given that Walmart is a retailer that touches literally hundreds of 213 00:11:54,398 --> 00:12:00,098 millions of customers directly as opposed to enterprise service providers, for 214 00:12:00,098 --> 00:12:00,742 example? 215 00:12:00,974 --> 00:12:05,554 Well, again, it's because it's a frontline human machine interface, right? 216 00:12:05,554 --> 00:12:09,914 We get to live in the real world and physical world as well as the internet and 217 00:12:09,914 --> 00:12:13,254 see how people feel comfortable with these tools. 218 00:12:13,254 --> 00:12:16,554 It's funny that we were focused on customers because I would say our most 219 00:12:16,554 --> 00:12:23,334 effective rollouts initially were in tools that support our associates in what 220 00:12:23,334 --> 00:12:23,974 they're doing. 221 00:12:23,974 --> 00:12:28,074 You know, I give the example and I don't think it's a real one, maybe a real one 222 00:12:28,074 --> 00:12:29,134 now. 223 00:12:29,390 --> 00:12:33,550 when I was doing a Sam's Club tour and they let me decorate cupcakes, which I 224 00:12:33,550 --> 00:12:34,450 think is hilarious. 225 00:12:34,450 --> 00:12:38,550 Now, anyone who's had a young child knows when you're ordering a birthday cake, you 226 00:12:38,550 --> 00:12:42,250 want to make sure you get the red one or the blue one or the right number of roses 227 00:12:42,250 --> 00:12:44,670 or the right number of decorations. 228 00:12:44,670 --> 00:12:49,710 And so there are guidebooks that tell associates, this is how many roses you put 229 00:12:49,710 --> 00:12:53,930 on an eight by 11 sheet cake and this is what it should look like for a new 230 00:12:53,930 --> 00:12:57,750 associate or even for someone experienced who hasn't done this particular task 231 00:12:57,750 --> 00:12:58,350 before. 232 00:12:58,350 --> 00:13:01,450 That means they run to the back room, they have to flip through the thing, they have 233 00:13:01,450 --> 00:13:04,550 to page through and say, you know, what is, and then read and then decorate and 234 00:13:04,550 --> 00:13:05,050 then read. 235 00:13:05,050 --> 00:13:09,090 Just one example of all the many thousands of things associates do in a day. 236 00:13:09,850 --> 00:13:14,990 With AI, there is a potential and I believe it's deployed already in some 237 00:13:14,990 --> 00:13:20,110 places in the country to simply say, and we have a tool called Ask Sam, we have 238 00:13:20,110 --> 00:13:23,870 another tool, we have tools named a number of things that are kind of in the Walmart 239 00:13:23,870 --> 00:13:25,646 legend and language. 240 00:13:25,646 --> 00:13:30,806 And say, you know, Sam, how do I decorate an eight and a half by 11 cheesecake for a 241 00:13:30,806 --> 00:13:34,406 young, you know, for a boy, for a girl, for, you know, in a fire truck, whatever. 242 00:13:34,406 --> 00:13:38,006 And that will save time, saves effort. 243 00:13:38,006 --> 00:13:40,026 They can say, you know, repeat that. 244 00:13:40,026 --> 00:13:43,896 It will really be an efficiency builder for an associate. 245 00:13:43,896 --> 00:13:48,726 And again, that's a very, I think, fun and upbeat example, but there are lots of, you 246 00:13:48,726 --> 00:13:53,582 know, paper based or knowledge worker examples as well that improve efficiency. 247 00:13:53,582 --> 00:14:00,322 One of the things we have put online are a number of our employee manuals, our 248 00:14:00,322 --> 00:14:02,412 benefits manuals, basic information. 249 00:14:02,412 --> 00:14:06,042 When you've got millions of associates, many of them are going to be asking the 250 00:14:06,042 --> 00:14:06,962 same question, right? 251 00:14:06,962 --> 00:14:10,882 So saves time, makes sure the information is accurate, every associate is getting 252 00:14:10,882 --> 00:14:12,302 the same information. 253 00:14:12,302 --> 00:14:17,358 That's a good example of where regular old AI is a good enough tool. 254 00:14:17,358 --> 00:14:20,188 do not need generative AI because you actually don't want it learning. 255 00:14:20,188 --> 00:14:21,698 You don't want it learning and changing the content. 256 00:14:21,698 --> 00:14:25,297 You want that content to be static and accurate and correct for everybody. 257 00:14:25,297 --> 00:14:28,818 That's, I think, one of the other points I would make is people have gotten really 258 00:14:28,818 --> 00:14:30,148 excited about generative AI. 259 00:14:30,148 --> 00:14:31,548 And it's a fascinating tool. 260 00:14:31,548 --> 00:14:33,558 And there's lots of good and bad about it. 261 00:14:33,558 --> 00:14:37,498 But you don't necessarily, it's not necessarily the right tool for every job 262 00:14:37,498 --> 00:14:38,978 that you might think of. 263 00:14:38,978 --> 00:14:43,138 And so fit for purpose, to me, is a really big question. 264 00:14:43,566 --> 00:14:48,586 you know, and accuracy and truth of the information to me, those are some of the 265 00:14:48,586 --> 00:14:51,322 biggest worries I have about gender AI. 266 00:14:52,814 --> 00:14:58,554 How do you structure the work on responsible AI in your organization or 267 00:14:58,554 --> 00:15:00,678 your organization relative to the rest of the company? 268 00:15:00,750 --> 00:15:01,750 Thanks for asking. 269 00:15:01,750 --> 00:15:07,070 So digital citizenship is, as we define it, the responsible use of technology and 270 00:15:07,070 --> 00:15:09,130 data, period, full stop. 271 00:15:09,130 --> 00:15:10,530 That's the mission. 272 00:15:10,530 --> 00:15:14,370 We have now got the privilege of leading a team of several hundred people. 273 00:15:14,370 --> 00:15:19,780 And that includes lawyers, technologists, data scientists, policy people. 274 00:15:19,780 --> 00:15:23,410 It's basically half kind of legal and half compliance and operational. 275 00:15:23,770 --> 00:15:25,990 And so we went about setting up a team. 276 00:15:25,990 --> 00:15:30,158 I really think that the woman who has to lead that team, when we created it, 277 00:15:30,158 --> 00:15:35,178 had one of the best jobs in the company because her first year was doing all the 278 00:15:35,178 --> 00:15:38,858 research and the benchmarking and finding all the best practices and then standing 279 00:15:38,858 --> 00:15:41,018 up the team for us. 280 00:15:41,018 --> 00:15:45,398 And again, it is the people who created the original policy went right up to the 281 00:15:45,398 --> 00:15:50,758 leadership of the company to say, these are the rules for rolling out an AI tool 282 00:15:50,758 --> 00:15:53,258 that has an impact on a human being. 283 00:15:53,258 --> 00:15:58,734 And then created a kind of a gate in our rollout processes to 284 00:15:58,734 --> 00:16:04,454 to try to explain it without too much jargon, that says, is this an AI tool? 285 00:16:04,454 --> 00:16:05,774 Does it impact humans? 286 00:16:05,774 --> 00:16:11,614 If so, you've got to be reviewed and answer some questions and engage with the 287 00:16:11,614 --> 00:16:13,104 Digital Values Team. 288 00:16:13,104 --> 00:16:14,994 That's the name of the team. 289 00:16:14,994 --> 00:16:19,594 And again, that team includes, it is led right now by a data scientist, and they 290 00:16:19,594 --> 00:16:24,074 will kind of probe and scrutinize the algorithm itself, the purpose for the 291 00:16:24,074 --> 00:16:28,878 tool, who's rolling out, who is the responsible kind of business owner of it. 292 00:16:28,878 --> 00:16:33,818 and then schedule, not just do the initial scrutiny, but schedule kind of follow up 293 00:16:33,818 --> 00:16:35,578 monitoring and oversight. 294 00:16:35,578 --> 00:16:40,338 So kind of all the tools of a traditional compliance program, but applied to this 295 00:16:40,338 --> 00:16:41,718 new technology. 296 00:16:41,718 --> 00:16:43,178 And so far, so good. 297 00:16:43,178 --> 00:16:45,678 And we've worked with quite a few teams. 298 00:16:45,678 --> 00:16:49,438 I don't have an exact number of how many tools they've scrutinized, but the team is 299 00:16:49,438 --> 00:16:50,388 busy and it's growing. 300 00:16:50,388 --> 00:16:55,038 So I think the metric is, you know, as we do more in this area, they have more work 301 00:16:55,038 --> 00:16:56,538 to do. 302 00:16:56,558 --> 00:16:59,778 But the biggest, I think the biggest challenge and the biggest opportunity is 303 00:16:59,778 --> 00:17:04,458 really one of education, of getting out and making sure that all of our 304 00:17:04,458 --> 00:17:09,678 technologists understand what's expected of them and how we expect them to think 305 00:17:09,678 --> 00:17:13,298 about the tools that they're deploying from the beginning. 306 00:17:13,298 --> 00:17:16,918 So, you know, if we educate everybody, actually it will cut down on the work for 307 00:17:16,918 --> 00:17:21,078 this team, but all to the good because there's plenty of other work to do. 308 00:17:21,078 --> 00:17:23,534 And I really want... 309 00:17:23,534 --> 00:17:27,954 each of our associates to think of themselves as a responsible user of the 310 00:17:27,954 --> 00:17:29,222 technology and data. 311 00:17:30,543 --> 00:17:32,743 This is always a challenge with compliance though. 312 00:17:32,743 --> 00:17:36,283 How do you convince them that this is valuable and it's not going to slow down 313 00:17:36,283 --> 00:17:37,478 what they're trying to do? 314 00:17:37,582 --> 00:17:38,852 Such a good question. 315 00:17:38,852 --> 00:17:42,612 And it's funny because I don't say I bristle at those questions. 316 00:17:42,612 --> 00:17:47,482 It's always been a perennial, like is privacy getting in the way of productivity 317 00:17:47,482 --> 00:17:49,742 or of profits or whatever. 318 00:17:50,802 --> 00:17:56,842 We really do believe in customer trust and associate trust and putting people first. 319 00:17:56,842 --> 00:18:03,742 So I really posited as an issue of trust and responsibility and that that engenders 320 00:18:03,742 --> 00:18:06,246 a better relationship with. 321 00:18:06,446 --> 00:18:12,346 the company and that we are in a competitive environment where this is a 322 00:18:12,346 --> 00:18:12,696 benefit. 323 00:18:12,696 --> 00:18:15,346 It's considered a good thing, I think, by our customers. 324 00:18:15,346 --> 00:18:19,546 We see more and more educated customers about technology and asking the questions 325 00:18:19,546 --> 00:18:21,966 about how it's being used. 326 00:18:21,966 --> 00:18:23,106 So I really don't try. 327 00:18:23,106 --> 00:18:28,526 I don't think of it as as antithetical, but rather as kind of enhancing our 328 00:18:28,526 --> 00:18:30,016 relationship with our customers. 329 00:18:30,016 --> 00:18:31,906 And we do try to be lean. 330 00:18:31,906 --> 00:18:35,438 Obviously, we try to be really efficient and not 331 00:18:35,438 --> 00:18:39,858 every AI tool that's just deciding the example I always use and it is a real one 332 00:18:39,858 --> 00:18:43,138 is, are the bananas on the conveyor belt right? 333 00:18:43,138 --> 00:18:48,678 We've got AI that looks at fruit and vegetables and meat and things and says, 334 00:18:48,678 --> 00:18:52,518 based on what it has learned over the last how many years and what a good thing, 335 00:18:52,518 --> 00:18:52,738 right? 336 00:18:52,738 --> 00:18:57,998 We want good quality food to be eaten and safe and that sort of thing. 337 00:18:58,298 --> 00:19:02,258 That we do not consider, it's impactful to humans obviously indirectly, but we're not 338 00:19:02,258 --> 00:19:03,418 making judgments about humans. 339 00:19:03,418 --> 00:19:05,166 They do not need to go through the digital. 340 00:19:05,166 --> 00:19:06,566 value scrutiny. 341 00:19:07,546 --> 00:19:11,056 But I think frankly, in my experience, most people want to do the right thing. 342 00:19:11,056 --> 00:19:15,166 They're not always sure how their job, you know, affects that right thing and the 343 00:19:15,166 --> 00:19:17,666 service to the customer and associate. 344 00:19:17,706 --> 00:19:19,736 So it's a big company. 345 00:19:19,736 --> 00:19:23,966 And so education and outreach across multiple languages, multiple parts of the 346 00:19:23,966 --> 00:19:29,318 world, multiple job descriptions, that to me is an exciting educational challenge. 347 00:19:30,030 --> 00:19:36,450 Yeah, I mean, it sounds like you've got a screen that is similar to what many of the 348 00:19:36,450 --> 00:19:41,170 governments are coming up with in terms of there's high -risk AI or whatever it's 349 00:19:41,170 --> 00:19:41,650 described at. 350 00:19:41,650 --> 00:19:43,090 This is what we really need to worry about. 351 00:19:43,090 --> 00:19:47,142 Then there's all kinds of other uses of AI that maybe don't need the same scrutiny. 352 00:19:47,726 --> 00:19:55,346 I hope people do look at it as a spectrum that not everything deserves or merits or 353 00:19:55,346 --> 00:19:58,316 needs the same level of scrutiny, but some of it very much does. 354 00:19:58,316 --> 00:20:04,146 And in the corporate world, the impactful decision -making that we would have would 355 00:20:04,146 --> 00:20:09,786 obviously be in hiring, advancement, people's rights and responsibilities as an 356 00:20:09,786 --> 00:20:10,506 employee. 357 00:20:10,506 --> 00:20:14,062 In the customer world, 358 00:20:14,062 --> 00:20:18,882 it's actually been even more fun because it's things like our AI chat bots and 359 00:20:18,882 --> 00:20:22,962 putting together different fashion pieces on the website and saying, you bought a 360 00:20:22,962 --> 00:20:24,972 red sweater, maybe this blue skirt goes with it. 361 00:20:24,972 --> 00:20:26,982 This is fun. 362 00:20:27,402 --> 00:20:28,402 Obviously, we want to be careful. 363 00:20:28,402 --> 00:20:29,532 We don't want to get it right. 364 00:20:29,532 --> 00:20:34,972 But it is not a consequential decision, depending on where you're going that day, 365 00:20:34,972 --> 00:20:37,322 whether the skirt matches the sweater. 366 00:20:37,322 --> 00:20:43,602 But again, of a different ilk than financial or career or other things. 367 00:20:45,198 --> 00:20:49,257 How does the team stay up to date with the fast changing nature of technology in this 368 00:20:49,257 --> 00:20:49,710 area? 369 00:20:49,710 --> 00:20:51,490 That is such a great question. 370 00:20:51,490 --> 00:20:57,370 And it is, I think, a constant part of everyone's job to be reading, to be 371 00:20:57,370 --> 00:20:58,090 engaged. 372 00:20:58,090 --> 00:21:03,120 We are members of quite a few, not only professional trade associations in retail, 373 00:21:03,120 --> 00:21:04,850 in law, and that sort of thing. 374 00:21:04,850 --> 00:21:09,150 I do find the IAPP, the International Association of Privacy Professionals, to 375 00:21:09,150 --> 00:21:11,190 be increasingly aware. 376 00:21:11,190 --> 00:21:14,638 And they just today put out a big report on AI governance. 377 00:21:14,638 --> 00:21:18,518 and we're members of something called the Data and Trust Alliance, which was out 378 00:21:18,518 --> 00:21:19,788 front very early on. 379 00:21:19,788 --> 00:21:24,558 We were one of the founding members in looking at cross -industry guidance. 380 00:21:24,558 --> 00:21:30,398 So it's not just one area of industry, not just one size of institution, but how do 381 00:21:30,398 --> 00:21:33,278 we make governance real for all companies? 382 00:21:33,278 --> 00:21:38,118 And in fact, we, as well as many small and medium enterprises, sometimes are 383 00:21:38,118 --> 00:21:40,958 procuring things from third parties, from vendors. 384 00:21:40,978 --> 00:21:41,702 And... 385 00:21:42,062 --> 00:21:46,102 just like every other new technology, we see a flourishing of vendors coming in and 386 00:21:46,102 --> 00:21:53,262 saying, this is the key thing, this is the perfect tool for everything, solution for 387 00:21:53,262 --> 00:21:54,542 everything you've ever needed. 388 00:21:54,542 --> 00:21:57,322 And many people don't know how to scrutinize that, right? 389 00:21:57,322 --> 00:22:03,622 And we've had salespeople in our office saying, we can't really disclose the 390 00:22:03,622 --> 00:22:06,642 algorithm and we can't really explain how it works. 391 00:22:06,642 --> 00:22:09,742 And we're like, well, come back when you can, because that's not gonna be good 392 00:22:09,742 --> 00:22:11,482 enough for our customers or associates. 393 00:22:11,482 --> 00:22:11,854 So. 394 00:22:11,854 --> 00:22:16,774 So rules around how to engage with vendors and how to scrutinize those tools as well 395 00:22:16,774 --> 00:22:19,954 as the ones you're building internally I think are really important and that's a 396 00:22:19,954 --> 00:22:21,714 role that DTA has played. 397 00:22:23,150 --> 00:22:27,870 Do you have benchmarks or goals or a way to assess whether you're being successful 398 00:22:27,870 --> 00:22:28,834 in these efforts? 399 00:22:29,006 --> 00:22:33,246 I think benchmarking, especially when it's in the policy world, is hard. 400 00:22:33,246 --> 00:22:37,406 But certainly some of our benchmarking is how many tools have gone through, how many 401 00:22:37,406 --> 00:22:40,186 secondary reviews we've done. 402 00:22:41,146 --> 00:22:45,886 Customer complaints are also a metric if things are making the wrong decision or 403 00:22:45,886 --> 00:22:47,306 the right decision. 404 00:22:48,306 --> 00:22:51,426 And so that's what we're looking at right now. 405 00:22:51,566 --> 00:22:54,306 But if you've got any ideas, I'm all ears. 406 00:22:54,350 --> 00:23:00,990 Well, no, it's an interesting challenge because on the one hand, there's value in 407 00:23:00,990 --> 00:23:03,420 coordination and standardization. 408 00:23:03,420 --> 00:23:07,510 And on the other hand, what makes sense for an organization at the scale of 409 00:23:07,510 --> 00:23:12,630 Walmart is not going to be at the right thing for a startup or for a company 410 00:23:12,630 --> 00:23:14,850 that's in a completely different industry. 411 00:23:14,850 --> 00:23:18,170 And so that's why I'm curious about how different organizations think about and 412 00:23:18,170 --> 00:23:19,534 see progress in those areas. 413 00:23:19,534 --> 00:23:20,604 Yeah, that's such a good point. 414 00:23:20,604 --> 00:23:24,434 I think there's some core values that probably cut across in terms of 415 00:23:24,434 --> 00:23:29,494 transparency and kind of fit for purpose or, you know, scrutinizing high risk tools 416 00:23:29,494 --> 00:23:31,034 more than others. 417 00:23:31,034 --> 00:23:35,114 But we recognize also the place of privilege that I sit in and being able to 418 00:23:35,114 --> 00:23:38,934 stand up an entire team to think about and then go do and create a whole new 419 00:23:38,934 --> 00:23:40,574 compliance program around this. 420 00:23:40,574 --> 00:23:42,204 And so well aware. 421 00:23:42,204 --> 00:23:47,514 And that's why DTA set about creating kind of products and workbooks and things for 422 00:23:47,514 --> 00:23:49,294 smaller medium enterprises that. 423 00:23:49,294 --> 00:23:51,654 They're kind of off the shelf guidance as well. 424 00:23:52,622 --> 00:23:57,802 Are there any major gaps that you see either in terms of what resources or tools 425 00:23:57,802 --> 00:24:01,408 are available to you or in any other area? 426 00:24:01,838 --> 00:24:03,558 I think we can always do more. 427 00:24:03,558 --> 00:24:08,118 Again, back to the comment about governing the ungovernable, we can all do more to 428 00:24:08,118 --> 00:24:14,778 really think about how to automate and really probe the tools in a consistent 429 00:24:14,778 --> 00:24:16,118 fashion. 430 00:24:16,918 --> 00:24:20,658 And like we were saying before, there's obviously a lot of concern around bias and 431 00:24:20,658 --> 00:24:22,878 about accuracy. 432 00:24:23,358 --> 00:24:28,218 But how do you prove that it is doing the thing that it's supposed to do and that 433 00:24:28,218 --> 00:24:30,998 it's not doing the thing it's not supposed to do? 434 00:24:31,406 --> 00:24:34,486 There's not a lot of standardization in that yet. 435 00:24:34,486 --> 00:24:37,616 And so standard setting bodies, where are you? 436 00:24:37,616 --> 00:24:42,526 I'm sure someone's working on a framework for us to all follow at some point. 437 00:24:42,526 --> 00:24:47,026 And I know you and I were going to talk about legislation and government 438 00:24:47,026 --> 00:24:48,746 interaction. 439 00:24:49,446 --> 00:24:54,806 I get very concerned when I think government actors are saying, we have to 440 00:24:54,806 --> 00:24:56,326 regulate this one thing. 441 00:24:56,326 --> 00:24:58,958 Again, as we were saying, AI is, you know. 442 00:24:58,958 --> 00:25:01,958 it's like the air or the, you know, the sidewalks. 443 00:25:01,958 --> 00:25:05,108 It's just a tool and a platform for doing something else. 444 00:25:05,108 --> 00:25:09,318 And I'm usually the one saying, is there a law that prevents this already? 445 00:25:09,318 --> 00:25:13,818 You know, is there something on the books that simply hasn't been enforced in this 446 00:25:13,818 --> 00:25:17,458 area or hasn't been applied to tools like this yet? 447 00:25:17,458 --> 00:25:23,018 If it's really, really that new, you know, certainly it may need some level of 448 00:25:23,018 --> 00:25:25,218 regulation or legislation. 449 00:25:25,218 --> 00:25:28,618 But I'm usually the skeptic saying, 450 00:25:28,654 --> 00:25:34,794 We may have rules about this already, about being unfair to people based on race 451 00:25:34,794 --> 00:25:35,894 or gender. 452 00:25:35,894 --> 00:25:39,056 And let's look at what's on the books already. 453 00:25:40,750 --> 00:25:45,730 Yeah, I mean, that's a big challenge that we're seeing a lot where I talk to lots of 454 00:25:45,730 --> 00:25:50,010 people who say, well, companies can't be trusted and their incentives are not 455 00:25:50,010 --> 00:25:50,590 aligned. 456 00:25:50,590 --> 00:25:53,570 And we know this technology is so new. 457 00:25:53,570 --> 00:25:58,910 So how do you convince people that the resources and the mechanisms are there to 458 00:25:58,910 --> 00:26:00,422 address the problems that arise? 459 00:26:00,962 --> 00:26:06,542 I certainly hear the same things and I certainly know that big companies are just 460 00:26:06,542 --> 00:26:09,492 whatever, big companies are made out of people, right? 461 00:26:09,492 --> 00:26:11,726 And so... 462 00:26:11,726 --> 00:26:15,566 It's not been my experience, not here and not in other big and small places I've 463 00:26:15,566 --> 00:26:17,946 worked that people want to do the wrong thing. 464 00:26:17,946 --> 00:26:21,746 In fact, I think our incentives are entirely aligned because if we lose the 465 00:26:21,746 --> 00:26:25,206 trust of our customers, if we lose the trust of regulators in the countries in 466 00:26:25,206 --> 00:26:30,106 which we do business, then the consequences are much greater than just 467 00:26:30,106 --> 00:26:32,126 failing in one program or product. 468 00:26:32,126 --> 00:26:32,718 It's a... 469 00:26:32,718 --> 00:26:34,778 you know, an enterprise wide impact. 470 00:26:34,778 --> 00:26:36,968 So I, again, I'm skeptical. 471 00:26:36,968 --> 00:26:41,138 I understand it's good to have a healthy skepticism of large institutions in 472 00:26:41,138 --> 00:26:41,458 general. 473 00:26:41,458 --> 00:26:42,698 And I, and I respect that. 474 00:26:42,698 --> 00:26:46,558 And that's why part of back to why we came out with the AI pledge is like, that is 475 00:26:46,558 --> 00:26:50,998 something that we feel is worth talking about that we are committed to walking our 476 00:26:50,998 --> 00:26:55,658 values forward in the digital world, in this tool, in this setting and in others. 477 00:26:55,658 --> 00:26:58,766 And if people feel like we're not doing it. 478 00:26:58,766 --> 00:27:03,006 They should let me know, they should let our CEO know, and they do occasionally, 479 00:27:03,006 --> 00:27:05,626 including some of our own associates, which I actually love. 480 00:27:05,626 --> 00:27:09,586 I love it when people raise their hand and say, this doesn't feel quite right to me. 481 00:27:09,586 --> 00:27:13,646 And I've got a letter once from a pharmacist in Michigan, and he said, I 482 00:27:13,646 --> 00:27:15,045 don't like that policy. 483 00:27:15,045 --> 00:27:19,286 And it actually started a whole review from the ground up of how we were dealing 484 00:27:19,286 --> 00:27:20,966 with a particular issue. 485 00:27:21,246 --> 00:27:24,306 So I think we try to be very responsive. 486 00:27:24,306 --> 00:27:27,552 Something I've tried to talk about within the company is, 487 00:27:27,630 --> 00:27:33,410 and all of the good governance we've put in on these tools and others, mistakes may 488 00:27:33,410 --> 00:27:34,330 still happen. 489 00:27:34,330 --> 00:27:37,670 That's the really big challenge I find, especially for a company that prides 490 00:27:37,670 --> 00:27:40,530 itself on its reputation and doing things right. 491 00:27:41,230 --> 00:27:45,990 It's not an environment where we'd like to fail fast and move on. 492 00:27:45,990 --> 00:27:49,130 We really do kind of bet our reputation on a lot of things. 493 00:27:49,130 --> 00:27:54,726 And what I try to encourage my team and others in leadership is to realize that... 494 00:27:55,054 --> 00:27:59,954 If a mistake happens, we're going to have to work hard and fast to fix it. 495 00:27:59,954 --> 00:28:03,634 But I think people will give us grace if we are transparent about it. 496 00:28:03,634 --> 00:28:05,994 If we say, this is what happened. 497 00:28:05,994 --> 00:28:09,424 Here's what we're doing to fix this incident. 498 00:28:09,424 --> 00:28:12,414 And here's what we're putting in place to have it not happen again. 499 00:28:12,514 --> 00:28:18,414 So especially in kind of generative AI, when it moves and talks and it moves fast, 500 00:28:18,414 --> 00:28:24,264 we need to be ready to both be humble and be nimble in that area. 501 00:28:25,102 --> 00:28:28,702 Yeah, I mean, one of the challenges with generative AI is what you said touches so 502 00:28:28,702 --> 00:28:34,602 many people that now it's not just the data scientists who are building things. 503 00:28:35,102 --> 00:28:38,312 What does trust really mean to you in that context? 504 00:28:38,312 --> 00:28:42,542 I think it's a really important and valuable point that this is all about 505 00:28:42,542 --> 00:28:44,202 creating and promoting trust. 506 00:28:44,202 --> 00:28:46,342 But what does that actually boil down to? 507 00:28:46,478 --> 00:28:52,698 On the website, it means we've correctly described a product. 508 00:28:52,698 --> 00:28:57,098 Say you've got a child who's allergic to peanut butter, you better be sure that the 509 00:28:57,098 --> 00:28:59,438 product you've bought does not have peanut butter in it. 510 00:28:59,438 --> 00:29:02,658 That's really important to us, and that was important to us before there was 511 00:29:02,658 --> 00:29:03,308 generative AI. 512 00:29:03,308 --> 00:29:07,838 But now if you have generative AI creating product descriptions, you've got to make 513 00:29:07,838 --> 00:29:08,728 sure they're right. 514 00:29:08,728 --> 00:29:11,938 And as we know, generative AI is not always right. 515 00:29:12,178 --> 00:29:16,710 So in some tools we're just about to put out in the... 516 00:29:17,070 --> 00:29:24,290 We used outside consultants to red team, to probe it, to try to crash it, and 517 00:29:24,730 --> 00:29:30,270 things still do go wrong, but I think putting in the effort before it goes live 518 00:29:30,270 --> 00:29:36,430 to really try to stress test it and then to stay, again, really vigilant about how 519 00:29:36,430 --> 00:29:39,090 the AI might change is essential. 520 00:29:39,090 --> 00:29:42,650 I feel, again, a deep sense of responsibility, not just because it's the 521 00:29:42,650 --> 00:29:46,030 catchphrase of our team, but I think... 522 00:29:46,030 --> 00:29:50,030 all kind of purveyors of technology should be thinking about the impact it's having 523 00:29:50,030 --> 00:29:53,030 on the world and on the people it comes into contact with. 524 00:29:54,318 --> 00:29:58,958 And going forward, are there either developments that you're looking to or 525 00:29:58,958 --> 00:30:01,758 anything that you see coming in the next few years that are going to be 526 00:30:01,758 --> 00:30:03,430 particularly important in this area? 527 00:30:04,782 --> 00:30:08,522 Well, I can tell you the things I'm the most worried about, unfortunately, it's 528 00:30:08,522 --> 00:30:10,762 not the way to start to answer that question. 529 00:30:10,762 --> 00:30:18,302 But the things that I worry about are the appearance of accuracy and truth when it 530 00:30:18,302 --> 00:30:19,682 is not there. 531 00:30:19,682 --> 00:30:25,482 It's too cute to say fake news, but just simply the authoritative answers that 532 00:30:25,482 --> 00:30:30,042 often are provided by these tools that are simply incorrect. 533 00:30:30,642 --> 00:30:34,574 Do all of our citizens in this country or elsewhere? 534 00:30:34,574 --> 00:30:40,374 have the ability to be good consumers of this kind of media savvy or media, you 535 00:30:40,374 --> 00:30:48,414 know, digital kind of citizenship in the true sense of can we consume and analyze 536 00:30:48,414 --> 00:30:53,014 information for its content and for its provenance? 537 00:30:53,014 --> 00:30:57,234 And do we have the tools to help us do that? 538 00:30:57,234 --> 00:31:03,846 I saw a great study years ago about kind of government issued propaganda and 539 00:31:04,078 --> 00:31:10,158 Americans were more likely to click on something that was, it kind of seemed to 540 00:31:10,158 --> 00:31:15,338 be true, but it was a little bit off than many parts of Eastern Europe. 541 00:31:15,338 --> 00:31:17,658 And I thought, wow, that kind of shocks me. 542 00:31:17,658 --> 00:31:19,758 You know, we have free press and we have all this stuff. 543 00:31:19,758 --> 00:31:23,858 And it was because they had lived with propaganda, perhaps in more recent times 544 00:31:23,858 --> 00:31:25,158 than we had. 545 00:31:25,158 --> 00:31:29,418 And I thought, well, how do we get to, you know, a place where we have really 546 00:31:29,418 --> 00:31:30,500 empowered? 547 00:31:30,606 --> 00:31:37,406 Internet users of all ages, shapes, sizes, everything to scrutinize the information 548 00:31:37,406 --> 00:31:42,586 or perhaps have penalties for people who put out things knowingly that are 549 00:31:42,586 --> 00:31:43,726 incorrect. 550 00:31:43,806 --> 00:31:48,666 That's to me one of the hardest questions for the Internet community, for the world, 551 00:31:48,666 --> 00:31:54,196 and one of the consequences of gendered AI that worries me the most. 552 00:31:54,196 --> 00:31:57,146 The other one is our kids using it to write term papers. 553 00:31:57,146 --> 00:31:59,054 But that's me. 554 00:31:59,054 --> 00:32:05,414 The exciting thing to me is always, not just the general utility for human beings, 555 00:32:05,414 --> 00:32:08,194 but I have a son who's on the spectrum. 556 00:32:08,194 --> 00:32:11,874 We have lots of associates who are disabled or in some way we have many 557 00:32:11,874 --> 00:32:13,014 customers. 558 00:32:13,014 --> 00:32:17,454 We've just started, you may have seen quiet hours in the stores where every 559 00:32:17,454 --> 00:32:22,134 morning for two hours, everything in the sound wise and the lights are lowered. 560 00:32:22,674 --> 00:32:27,774 And it's been a real, it's been a joy to watch that evolve and people really get 561 00:32:27,774 --> 00:32:28,974 excited about it. 562 00:32:28,974 --> 00:32:32,814 I always call it kind of the ancillary effects as well. 563 00:32:33,774 --> 00:32:39,794 People of all kinds who may not fit in its traditional kind of disability community 564 00:32:39,794 --> 00:32:41,844 have said, some people just like it quiet. 565 00:32:41,844 --> 00:32:44,894 My husband says I would much rather shop when it's quiet. 566 00:32:44,894 --> 00:32:46,374 And whatever. 567 00:32:46,434 --> 00:32:50,834 And so the thing I'm the most excited about, not just about AI and gender AI, 568 00:32:50,834 --> 00:32:57,094 but technology is the ability to assist humans who are physically, cognitively 569 00:32:57,094 --> 00:32:58,062 otherwise. 570 00:32:58,062 --> 00:33:01,282 diverse or challenged in some way. 571 00:33:01,522 --> 00:33:06,822 And Gen .AI, especially in voice enabled, is just a boon for many, many people. 572 00:33:06,822 --> 00:33:08,362 And I'm so excited about that. 573 00:33:08,362 --> 00:33:12,322 What that means for our workforce, enabling more people to join our 574 00:33:12,322 --> 00:33:18,402 workforce, generally join the workforce, and enabling customers to find the 575 00:33:18,402 --> 00:33:21,172 information they want and the goods and services they want. 576 00:33:21,172 --> 00:33:24,810 I think that's one of the best and highest uses of technology. 577 00:33:25,294 --> 00:33:25,874 No question. 578 00:33:25,874 --> 00:33:30,454 No, it's all tremendously exciting and it's sort of the paradoxes in and of this 579 00:33:30,454 --> 00:33:34,374 technology that what makes it exciting is also what in some ways makes it scary. 580 00:33:34,374 --> 00:33:39,314 I mean, you talked about sort of two sides of citizenship in both the corporate and 581 00:33:39,314 --> 00:33:43,574 the public governmental context where this technology can be so great at 582 00:33:43,574 --> 00:33:45,811 personalizing, but that can also be abused. 583 00:33:46,414 --> 00:33:47,814 So true, so true. 584 00:33:47,814 --> 00:33:52,114 And transparency is necessary but not sufficient answer. 585 00:33:52,462 --> 00:33:53,032 Absolutely. 586 00:33:53,032 --> 00:33:54,312 Nuala, really great to talk to you. 587 00:33:54,312 --> 00:33:57,016 Thanks so much for sharing your experiences and what you're doing. 588 00:33:57,134 --> 00:33:57,904 Delighted to be here. 589 00:33:57,904 --> 00:33:59,534 Thanks for having me.