1 00:00:00,916 --> 00:00:03,473 Shea, thank you so much for joining me on the podcast. 2 00:00:03,714 --> 00:00:05,396 Thank you for having me here. 3 00:00:06,494 --> 00:00:10,410 How does an astrophysicist turn into an AI auditor? 4 00:00:11,968 --> 00:00:15,599 Yeah, that's a really strange story. 5 00:00:17,320 --> 00:00:24,203 my research in astrophysics was primarily in data analysis. 6 00:00:24,203 --> 00:00:29,986 So I worked on detecting black holes in distant galaxies, measuring magnetic fields, things like this. 7 00:00:29,986 --> 00:00:39,930 And the problem with astrophysics is that there's so much data that we're getting from the sky now that literally if every human on earth spent their entire lives looking at it, 8 00:00:39,930 --> 00:00:40,994 they would not 9 00:00:40,994 --> 00:00:42,034 be able to get through it all. 10 00:00:42,034 --> 00:00:50,818 And so it sort of necessitated me getting into machine learning and artificial intelligence as a tool to sift through this data. 11 00:00:51,058 --> 00:00:55,900 And I decided that I was really excited about AI and I wanted to get into it. 12 00:00:55,900 --> 00:01:01,983 And so I sort of started to do a little discovery of like, what, where could I have a unique impact? 13 00:01:02,383 --> 00:01:09,486 And I felt like AI auditing and was kind of a unique place where my brain just sort of fit. 14 00:01:09,486 --> 00:01:17,026 you know, the critical scientific approach to let's dig and uncover what the problems with, with this, these systems are. 15 00:01:17,026 --> 00:01:22,906 And it was around the time of like Cambridge Analytica and ProPublica's article about biased algorithms. 16 00:01:22,906 --> 00:01:24,326 And it just felt like the right time. 17 00:01:24,326 --> 00:01:26,026 So around 2018. 18 00:01:26,026 --> 00:01:33,666 And so I sort of made a switch to sort of spend a lot more time focusing on how do we audit systems? 19 00:01:33,666 --> 00:01:37,994 What are AI systems and what are the ways we can kind of systematically uncover? 20 00:01:38,082 --> 00:01:41,415 the risks that were beginning to become apparent. 21 00:01:42,676 --> 00:01:51,074 So what are the basic technical skills or capabilities that are needed in order to effectively audit an AI system? 22 00:01:52,066 --> 00:01:52,336 Yeah. 23 00:01:52,336 --> 00:01:58,250 So I think that a lot of the, there's not that many technical skills per se. 24 00:01:58,250 --> 00:02:07,256 So, so we spent a lot of time trying to train auditors and, and, upskill them into, into this field. 25 00:02:07,557 --> 00:02:12,870 And it's not like you have to learn to code in Python or be able to build a neural network or something like that. 26 00:02:12,870 --> 00:02:21,944 It's a much more about understanding the types of systems you're going to interact with that you might be auditing in terms of how they work roughly. 27 00:02:21,944 --> 00:02:28,116 What kind of data they use, what are the risks associated in particular with that kind of technology? 28 00:02:28,417 --> 00:02:33,159 And then a lot of sort of societal risk analysis. 29 00:02:33,159 --> 00:02:43,964 So we have a lot of philosophers actually are really great at thinking through sort of what issues are really germane to this problem, sifting through irrelevant information and 30 00:02:43,964 --> 00:02:47,725 getting down to the core of what's important. 31 00:02:47,725 --> 00:02:49,366 And that's really the... 32 00:02:49,656 --> 00:03:06,129 That's really the skill and of the technical skills, would say statistics or an understanding of how many factors can influence a data point that there's a big, we call 33 00:03:06,129 --> 00:03:08,040 it parameter space thinking. 34 00:03:08,401 --> 00:03:09,862 That's really the skill. 35 00:03:09,862 --> 00:03:19,880 It's much more about understanding the little bit of statistics and appreciating the complexity of data and how AI processes data. 36 00:03:20,842 --> 00:03:23,374 What does that exactly mean, parameter space thinking? 37 00:03:23,682 --> 00:03:33,445 So parameter space thinking, so a parameter space is just that there are lots of different factors that influence a particular result. 38 00:03:33,445 --> 00:03:37,886 Let's say it's a number, let's say it's a behavior that AI system has. 39 00:03:39,627 --> 00:03:44,878 most of the time when you're auditing these systems, some of those parameters are important, others are not. 40 00:03:44,878 --> 00:03:50,670 So that's like the first way of thinking, like understanding what the parameter space is. 41 00:03:50,670 --> 00:03:52,610 So an example might be 42 00:03:53,467 --> 00:03:59,351 I have some sort of activity detection in video recordings, right? 43 00:03:59,351 --> 00:04:07,038 So maybe I want to detect from a camera, a video feed, whether someone's getting robbed or when someone's stealing something. 44 00:04:07,438 --> 00:04:10,771 So in that scenario, there's a whole bunch of parameters. 45 00:04:10,771 --> 00:04:13,183 It's like, what kind of camera did it use? 46 00:04:13,223 --> 00:04:16,206 What kind of model am I using to predict this? 47 00:04:16,206 --> 00:04:17,437 What kind of training data? 48 00:04:17,437 --> 00:04:19,568 Where did that training data come from? 49 00:04:19,629 --> 00:04:21,373 Is it different than where I'm? 50 00:04:21,373 --> 00:04:22,083 using it now. 51 00:04:22,083 --> 00:04:28,215 Maybe I'm doing it in parking garages, but all my data is from like YouTube channels, really high quality video. 52 00:04:28,235 --> 00:04:39,859 Those are all parameters that are relevant for understanding the outputs of some testing that you might have done or the risk profile that you might have. 53 00:04:39,859 --> 00:04:47,704 And so being able to parse that out and actually, I don't know if quantifies the right word, ideally quantify would be great. 54 00:04:47,704 --> 00:04:56,730 But sometimes it just qualitatively parse out what are the things that are important of this massive parameter space and then getting down to here are the relevant things and 55 00:04:56,730 --> 00:05:01,718 this is how I can sort of partition what factors are important for risk. 56 00:05:02,930 --> 00:05:06,075 Okay, let's take a step back before we go on. 57 00:05:06,337 --> 00:05:16,194 What exactly is an AI audit and what basically are the kinds of services that your company Babel provides to organizations? 58 00:05:16,846 --> 00:05:17,846 Yeah. 59 00:05:17,966 --> 00:05:21,786 So in 2018, we started AI audit wasn't really a thing. 60 00:05:21,786 --> 00:05:24,176 And so it's, it's now becoming more of a thing. 61 00:05:24,176 --> 00:05:25,866 So what is it? 62 00:05:26,446 --> 00:05:32,206 it's very akin to, let's say a financial audit, like a typical audit that you would expect. 63 00:05:32,206 --> 00:05:33,446 So what is that? 64 00:05:33,446 --> 00:05:42,326 There's some standard or some normative guidance for like what a company or a person ought to do or ought to have in place. 65 00:05:42,326 --> 00:05:45,828 And an auditor's job is to come in and to 66 00:05:46,292 --> 00:05:55,789 look at what a company or a person has done and has in place against that standard and weigh in on with some judgment. 67 00:05:55,989 --> 00:05:57,871 there are different types of audits. 68 00:05:57,871 --> 00:06:00,532 It could be like binary where it's like you pass or fail. 69 00:06:00,532 --> 00:06:07,737 It might be more like an assurance engagement where it's like, I have reasonable assurance that this company adheres to the standard. 70 00:06:08,058 --> 00:06:14,022 So that's exactly what we do, except that it's over AI systems. 71 00:06:14,022 --> 00:06:21,465 And the sort of processes around those AI systems, how they're governed, how they're managed, how they're tested. 72 00:06:21,666 --> 00:06:23,526 And that's really it. 73 00:06:23,667 --> 00:06:28,669 The complication here is that you can't just, you wouldn't just get a financial auditor to do this, right? 74 00:06:28,669 --> 00:06:36,172 There's certain capabilities that you need to have to recognize when something is in place or not. 75 00:06:36,172 --> 00:06:40,534 You know, is that risk management framework that that company has the right one? 76 00:06:40,692 --> 00:06:42,653 Is it actually effective at managing risks? 77 00:06:42,653 --> 00:06:45,586 For instance, is that testing the right kind of testing? 78 00:06:45,586 --> 00:06:49,188 Is that information you get relevant? 79 00:06:49,188 --> 00:07:02,458 And so it's the same as that sort of auditing, except it just has added complexity because the complexity of the system is so much higher than just like QuickBooks or some financial 80 00:07:02,458 --> 00:07:03,328 records. 81 00:07:05,386 --> 00:07:11,210 So you mentioned when you got started, there were not a lot of people and organizations in the space. 82 00:07:11,210 --> 00:07:21,047 Now there's a range, everything from individuals to some of the very biggest professional services and consulting firms providing AI audit and assessment services. 83 00:07:21,047 --> 00:07:26,160 And there's also researchers doing audit studies from the outside without firms' permission. 84 00:07:26,160 --> 00:07:29,342 So how do you look at the space? 85 00:07:29,342 --> 00:07:35,396 don't know if industry is too strong a word, but the community that exists around AI auditing today. 86 00:07:36,366 --> 00:07:46,812 So there is a strong ecosystem around AI auditing that's growing and there are different sort of players and actors and places where people naturally fit. 87 00:07:47,052 --> 00:07:54,206 And in this space, I would say there are kind of the technical assessor space, right? 88 00:07:54,206 --> 00:07:59,749 So these are individual people or firms that are gonna really come in and do some sort of technical assessment. 89 00:07:59,749 --> 00:08:01,270 They're gonna test for bias. 90 00:08:01,270 --> 00:08:02,691 They're going to check robustness. 91 00:08:02,691 --> 00:08:05,056 There might be some cybersecurity elements or... 92 00:08:05,056 --> 00:08:08,407 red teaming of large language models would fit into this. 93 00:08:08,648 --> 00:08:21,393 And there are a lot of players in that space from individual consultants, academics that might work on the side to all of the sort of large assurance firms, the big four and 94 00:08:21,393 --> 00:08:24,714 consultancies will have some services around this. 95 00:08:25,274 --> 00:08:31,957 And then there's like the, those same people are also going to offer services around like risk management, things like this. 96 00:08:31,957 --> 00:08:34,098 These are the sort of consulting 97 00:08:34,665 --> 00:08:37,507 capability building players. 98 00:08:37,507 --> 00:08:50,154 Then there are the of the technology players where it's might be a technology platform that will automatically run tests or might be like a GRC platform that will organize how 99 00:08:50,154 --> 00:08:52,155 you might manage your risk. 100 00:08:52,475 --> 00:09:02,401 And they will kind of have the connective, that will be like the connective tissue, the tech assisted layer for getting this done. 101 00:09:02,521 --> 00:09:04,194 And then there's sort of the 102 00:09:04,194 --> 00:09:08,235 the external sort of third party audit assurance services. 103 00:09:08,235 --> 00:09:10,516 And that's where Babel sits primarily. 104 00:09:10,516 --> 00:09:12,336 We do very little consulting. 105 00:09:12,336 --> 00:09:13,617 We try to avoid it. 106 00:09:13,617 --> 00:09:17,238 We prefer to be like an independent third party auditor. 107 00:09:17,238 --> 00:09:20,598 And there are a number of smaller organizations like Babel. 108 00:09:20,598 --> 00:09:22,479 Eticus is another one. 109 00:09:22,979 --> 00:09:28,601 There are sort of mixed like holistic AI is a platform and they do audits. 110 00:09:29,661 --> 00:09:33,474 And then of course there are the big players that are already in the 111 00:09:33,474 --> 00:09:46,551 sort of regulatory assurance space of the big four plus, you know, the other ones, they're all sort of getting into this in various levels, but the maturity is still quite low 112 00:09:46,551 --> 00:09:49,352 because there are very few standards around it at the moment. 113 00:09:50,398 --> 00:09:54,960 Yeah, so how does a small firm like yours compete against those giant consultants? 114 00:09:55,448 --> 00:10:00,001 So, well, we start early, which we did. 115 00:10:00,983 --> 00:10:03,765 The way we compete is that we're hyper -focused. 116 00:10:03,765 --> 00:10:09,710 And so we're not doing, we don't do ESG assurance. 117 00:10:09,710 --> 00:10:13,012 We don't do financial auditing. 118 00:10:13,012 --> 00:10:14,844 We don't do cybersecurity auditing. 119 00:10:14,844 --> 00:10:24,261 We do one thing and we do it very well, which is the sort of AI assurance risk management also includes technical testing. 120 00:10:25,282 --> 00:10:31,244 we approach most of the sort of senior team, we were professors and other things we know about research. 121 00:10:31,244 --> 00:10:36,305 And so we started early on recognizing that we don't know what we're doing. 122 00:10:36,305 --> 00:10:38,466 And so we have to figure out what to do. 123 00:10:38,466 --> 00:10:49,169 so early on we weren't auditing, we were consulting and getting our hands dirty, trying to figure out how do you actually systematically detect risk in these systems? 124 00:10:49,169 --> 00:10:50,629 What is a good risk assessment? 125 00:10:50,629 --> 00:10:54,240 What kind of technical testing and how should they be informed by each other? 126 00:10:54,822 --> 00:11:04,912 and we published a lot, know, so we're publishing our research, we're really coming at it with ideally sort of intellectual humility that there are a lot of things to be figured 127 00:11:04,912 --> 00:11:16,003 out and it's that niche aspect and so now I mean we have people from the big four coming through our training program to learn from us how to do these things because we're just 128 00:11:16,003 --> 00:11:19,542 focusing on one thing and that's probably 129 00:11:19,542 --> 00:11:24,708 Any, all the business people listening will know that that that's a good strategy in general. 130 00:11:24,708 --> 00:11:35,241 If you're small and don't have resources to niche down and just make sure you're very good at a very specific thing that hopefully will have a big market at the end, which I think 131 00:11:35,241 --> 00:11:36,712 is becoming apparent. 132 00:11:38,378 --> 00:11:42,447 Can you describe what makes a good AI audit? 133 00:11:43,448 --> 00:11:50,100 So a good AI audit, so this will be my opinion, because some people will think different things. 134 00:11:50,480 --> 00:11:55,612 I think a good AI audit is going to have a clear objective, right? 135 00:11:55,612 --> 00:11:57,792 So you have to know what the audit is for. 136 00:11:57,792 --> 00:11:59,542 Like why are you doing this? 137 00:11:59,542 --> 00:12:01,823 Is it because there's a law in place? 138 00:12:01,824 --> 00:12:09,836 Is it for some sort of internal risk management that you want senior management to know how their company is performing, for instance? 139 00:12:09,836 --> 00:12:10,416 What's the goal? 140 00:12:10,416 --> 00:12:14,688 So it has to have a very clear goal and a very clear intended audience. 141 00:12:14,829 --> 00:12:16,270 And that's not always obvious. 142 00:12:16,270 --> 00:12:19,431 And so if you have that, then that's a good, good start. 143 00:12:19,512 --> 00:12:23,213 There has to be some kind of standards in place. 144 00:12:23,234 --> 00:12:24,725 Now it doesn't have to be standards. 145 00:12:24,725 --> 00:12:36,852 That's, that's like totally set or that aren't changing or they don't even, they could be standards that the auditors themselves had put in place, but they have to be there ahead 146 00:12:36,852 --> 00:12:37,666 of time. 147 00:12:37,666 --> 00:12:40,647 And they have to be clear and clearly articulated. 148 00:12:40,767 --> 00:12:43,588 And then there has to be independence. 149 00:12:43,588 --> 00:12:48,270 think this is a, it, can of course have internal audit, which is not fully independent. 150 00:12:48,270 --> 00:12:52,752 There's some independence because they report, say up to the board or something. 151 00:12:52,752 --> 00:13:03,496 But I think independence or impartiality is a pretty important component because what we've seen is that it's so easy to manipulate the results of these things like technical 152 00:13:03,496 --> 00:13:03,877 testing. 153 00:13:03,877 --> 00:13:06,098 For instance, I have a big data set. 154 00:13:06,098 --> 00:13:07,758 I'm measuring bias. 155 00:13:07,894 --> 00:13:10,316 I run it through, I don't like the results. 156 00:13:10,316 --> 00:13:12,937 say, what is it that's causing these results to be bad? 157 00:13:12,937 --> 00:13:15,729 Well, maybe this data point is not appropriate. 158 00:13:15,729 --> 00:13:22,944 So I'll just take that out and slowly the results start looking much, much better and the way I want them to look. 159 00:13:22,944 --> 00:13:24,394 That's not a great situation. 160 00:13:24,394 --> 00:13:33,420 so having that impartiality, having the clear goals ahead of time, knowing what the process is will avoid those sorts of problems. 161 00:13:34,050 --> 00:13:36,351 and so I think those are kind of the key components. 162 00:13:36,351 --> 00:13:43,183 And then the final thing I'll say is auditor competence because you have to understand. 163 00:13:43,183 --> 00:13:47,474 mean, so we, our audits are kind of assurance engagements. 164 00:13:47,474 --> 00:13:52,256 So we follow sort of assurance, international assurance standards. 165 00:13:52,256 --> 00:14:00,768 And one of the things you need to do is really assess the audit risk or where, where it's the risk of material misstatement that could happen. 166 00:14:01,014 --> 00:14:06,918 where they just gave us the wrong information, maybe on purpose, maybe by accident. 167 00:14:07,079 --> 00:14:14,065 If you don't understand the system and how it works, then it's going to be hard to detect where those risks are. 168 00:14:14,065 --> 00:14:17,627 so auditor competence is sort of the final thing that like seals the deal. 169 00:14:17,627 --> 00:14:21,490 That feels to me like a good package of things that would make a good audit. 170 00:14:22,602 --> 00:14:26,169 Great, so there's a lot in what you just said to unpack. 171 00:14:26,169 --> 00:14:29,814 Let me start at the beginning with the rationale. 172 00:14:29,955 --> 00:14:36,506 What do you find today are the main reasons that companies come to you for those audit services? 173 00:14:37,336 --> 00:14:40,767 So there were sort of three stages. 174 00:14:40,768 --> 00:14:43,769 The first stage was reputational risk. 175 00:14:43,809 --> 00:14:49,732 So early on, the people who came to us were under sort of severe kind of reputation scrutiny. 176 00:14:49,732 --> 00:15:00,356 Either the articles were written about them, accusing them of bad practices or AI that was bad, the Senate was looking at them or something like that. 177 00:15:00,597 --> 00:15:02,187 That was sort of the first phase. 178 00:15:02,187 --> 00:15:03,682 People were just panicked about. 179 00:15:03,682 --> 00:15:08,345 having a bad reputation because a lot was being written about it, let's say five years ago. 180 00:15:09,106 --> 00:15:11,167 The next is that laws actually got passed. 181 00:15:11,167 --> 00:15:19,732 so things like New York city has a local law one for four, which requires bias audits of automated employment decision tools. 182 00:15:19,732 --> 00:15:28,418 So the tools that would like sort through your CV or your resume, when you apply for a job, there's many of those, almost every job you apply to will have those. 183 00:15:28,559 --> 00:15:30,520 If you're operating in New York, they have to be audited. 184 00:15:30,520 --> 00:15:33,432 So there was like a compliance or regulatory pressure that 185 00:15:33,432 --> 00:15:39,496 brought a new wave of clients to us and to other auditors, because I had to get this done because of that. 186 00:15:39,496 --> 00:15:44,450 That's not going to stop because there's a lot of laws coming that are going to require that. 187 00:15:44,450 --> 00:15:48,932 But there's a new wave now, which is procurement pressure. 188 00:15:48,933 --> 00:15:57,198 So even before these other laws have come into effect, the majority of people who come to us now are trying to sell to, say a large enterprise. 189 00:15:57,198 --> 00:16:02,902 That large enterprise is, has already worried about compliance and reputational risk from before. 190 00:16:02,902 --> 00:16:07,038 is now pushing back on their vendors to get some sort of auditor assurance done. 191 00:16:07,038 --> 00:16:10,573 They need something that's gonna limit their downside risk. 192 00:16:11,014 --> 00:16:13,708 And so that's the pressure we're feeling right now. 193 00:16:13,708 --> 00:16:19,456 That's what's driving most of the activity in this space, from my experience. 194 00:16:19,668 --> 00:16:21,340 Yeah, no, no, no, it's really interesting. 195 00:16:21,340 --> 00:16:24,303 So the second thing you said before was standards. 196 00:16:24,303 --> 00:16:31,610 And sometimes, so if we're about local L -144, specifies an adverse impact report, what that test is. 197 00:16:31,610 --> 00:16:36,493 But if you don't have something like that, where do the standards come from that you use? 198 00:16:37,614 --> 00:16:40,825 So that's a really good question. 199 00:16:40,825 --> 00:16:43,616 So there are some emerging standards. 200 00:16:43,616 --> 00:16:50,037 So NIST has their AI risk management framework. 201 00:16:50,037 --> 00:16:53,248 That is a fairly kind of, I wouldn't say comprehensive. 202 00:16:53,248 --> 00:16:58,559 It has a lot of options for people to manage risk. 203 00:16:59,019 --> 00:17:04,721 And so if we are trying to audit somebody for their risk management practices, 204 00:17:05,710 --> 00:17:07,850 That's, it's not really a standard. 205 00:17:07,850 --> 00:17:17,130 It's not written necessarily like a standard, but it does have enough elements and enough specificity that we could use it as sort of criteria for auditing. 206 00:17:17,170 --> 00:17:29,030 There are now ISO has now released 42 ,001, which is a, an AI management system standard, which has gotten a lot of interest recently. 207 00:17:29,030 --> 00:17:33,262 And that is because it's an ISO standard is written like a standard that is 208 00:17:33,262 --> 00:17:34,382 meant to be auditable. 209 00:17:34,382 --> 00:17:35,702 even have guidelines. 210 00:17:35,702 --> 00:17:42,962 It's 42 ,006, which are in draft, but they're meant to specify how that should be audited against. 211 00:17:43,022 --> 00:17:46,022 So that's a great framework to work from. 212 00:17:46,562 --> 00:17:58,922 And then there are other laws that like EU AI Act, Colorado now has another law, Illinois passed some law, California has some new laws that are likely to be signed very soon. 213 00:17:59,022 --> 00:18:03,222 All of those will work, but there are always details 214 00:18:03,886 --> 00:18:13,242 that are complications for an auditor that we have to sort through that does cause, there's extra risk and there's a ton of development work. 215 00:18:13,242 --> 00:18:16,554 So we spend probably half our time working on audits. 216 00:18:16,554 --> 00:18:25,880 The other half of the time working on standard, our own internal processes and internal standards and risk management and procedures to manage the risk of those audits. 217 00:18:25,880 --> 00:18:30,152 it's R and D, half R and D at the moment. 218 00:18:31,762 --> 00:18:37,678 No, I would think that there is a danger in what you described in the things go the other way and there are too many different standards. 219 00:18:37,678 --> 00:18:41,683 There's all these different competing options out there with financial accounting. 220 00:18:41,683 --> 00:18:48,360 We've got GAP and international equivalents and financial accounting standards board and stuff and there's some clarity. 221 00:18:48,360 --> 00:18:54,256 Do we need more coordination about what the standards are for doing an AI audit? 222 00:18:54,708 --> 00:18:56,639 I think we do need that. 223 00:18:56,639 --> 00:18:59,031 And there are some organizations that are trying to do that. 224 00:18:59,031 --> 00:19:01,423 I'm a fellow at For Humanity. 225 00:19:01,423 --> 00:19:13,221 That organization is trying to do exactly this, have a central kind of repository of certification schemes that might be informed by laws or might be informed by standards, 226 00:19:13,221 --> 00:19:17,853 but are sort of developed in a unique sort of coherent way. 227 00:19:18,174 --> 00:19:21,776 But I don't think that we're going to get there. 228 00:19:21,776 --> 00:19:24,526 I think if you look at cybersecurity, for instance, 229 00:19:24,526 --> 00:19:30,986 There's just so many different certification schemes, so many different types of ways of auditing types of audits. 230 00:19:30,986 --> 00:19:35,036 You've got SOC 2, you've got ISO, there are other organizations. 231 00:19:35,036 --> 00:19:38,546 Some companies have just had their own certification that they develop. 232 00:19:38,546 --> 00:19:41,746 And I think we're going to see it evolve in that way. 233 00:19:41,746 --> 00:19:54,510 Now, one thing I'll say is that the EU AI Act is so comprehensive in its requirements, or at least so onerous, maybe is the word that some people would use. 234 00:19:54,734 --> 00:20:01,374 that it will probably be the most difficult quote unquote standard to adhere to. 235 00:20:01,374 --> 00:20:10,934 And so if you adhere to the most rigorous standard, it's probably gonna cover you for 95 % of all of the other regulations. 236 00:20:10,934 --> 00:20:15,694 And so I think we'll see people gravitating towards that as a benchmark. 237 00:20:15,834 --> 00:20:23,030 then it'll be just filling in the gaps for some individual local law or individual kind of... 238 00:20:23,030 --> 00:20:25,693 association that might want to put something together. 239 00:20:27,050 --> 00:20:37,033 Although it's interesting you mentioned the work that you have to do internally on R &D and assessment on standards with something like the AI Act, there's still tremendous gaps 240 00:20:37,033 --> 00:20:41,185 in terms of the interpretive guidance and what those standards mean and so forth. 241 00:20:41,185 --> 00:20:51,918 So do firms like yours really have to fulfill a role for making some of those decisions under conditions of uncertainty as these standards are evolving? 242 00:20:52,212 --> 00:20:53,442 Absolutely. 243 00:20:53,563 --> 00:21:08,469 we do have to interpret if there aren't standards and the EU will have, Sensenlack is going to come out with more detailed standards and we are sort of keeping track of that 244 00:21:08,469 --> 00:21:12,310 and providing input when we can. 245 00:21:12,470 --> 00:21:17,332 But a lot of times we are going to have to make some calls on our own. 246 00:21:17,332 --> 00:21:21,694 What do we think is good enough given the current language of the law? 247 00:21:22,040 --> 00:21:33,439 But I think that's okay because the level of maturity for most organizations frankly is not anywhere near even the laws it's written, much less some detailed standards. 248 00:21:33,439 --> 00:21:45,307 And so our sort of general strategy, and I think it's a good strategy for other organizations that they want to try to audit, is to fall back on transparency and say, 249 00:21:45,307 --> 00:21:48,970 listen, this is exactly what we are using as a... 250 00:21:48,970 --> 00:21:55,774 as a standard, these are the criteria that we use, these are the procedures we went through to check things and disclose that. 251 00:21:55,774 --> 00:22:00,717 And then people will know exactly what happened and what you did. 252 00:22:00,817 --> 00:22:10,063 And it may not be exactly compliant with the law as the new standards might indicate, but you can always go back and redo it. 253 00:22:10,063 --> 00:22:15,456 And it just, provides a lot more clarity on what exactly is going on. 254 00:22:16,850 --> 00:22:20,725 about certifications and accreditations for the auditors themselves? 255 00:22:20,725 --> 00:22:23,354 I know that's something you've been involved in, in the AI space. 256 00:22:23,896 --> 00:22:28,258 So that's a difficult thing because there isn't a central body for that. 257 00:22:28,258 --> 00:22:36,461 And so I mentioned for Humanity before, they have some sort of certifications based on their certification scheme. 258 00:22:36,461 --> 00:22:40,853 So they'll kind of accredit auditors for their certification schemes. 259 00:22:40,853 --> 00:22:44,855 And so we've, for instance, gotten some of those as our organization. 260 00:22:45,135 --> 00:22:51,818 There's a new organization that I'm one of the founding members of, which is the International 261 00:22:52,130 --> 00:22:54,111 Association of algorithmic auditors. 262 00:22:54,111 --> 00:22:58,832 I might've gotten one of those words mixed up, but it's I AAA, I AAA. 263 00:22:58,832 --> 00:23:05,264 And so the goal there is much more about identifying what are the capabilities. 264 00:23:05,264 --> 00:23:07,074 It's in a professional association. 265 00:23:07,074 --> 00:23:16,557 What are the capabilities we think auditors need to have and how could they acquire the sort of bona fides that they need in order to do this? 266 00:23:16,557 --> 00:23:20,672 It might be, we haven't got to the point where we have an examination yet or anything like that. 267 00:23:20,672 --> 00:23:28,387 It probably will get to that point, but I have a feeling it will be a mix of things from experience to education. 268 00:23:28,387 --> 00:23:30,629 We don't have any degrees in this at the moment. 269 00:23:30,629 --> 00:23:32,010 That's something that will come. 270 00:23:32,010 --> 00:23:39,775 I'm sure just like, just like financial auditing has accounting degrees that will lead to financial auditing that was going to happen eventually. 271 00:23:39,775 --> 00:23:42,657 But right now it's a little bit of the wild West. 272 00:23:42,657 --> 00:23:48,001 And I think we rely on our own training for our own auditors. 273 00:23:48,001 --> 00:23:49,982 And so we have our own program. 274 00:23:50,306 --> 00:23:59,287 where we've outlined the capabilities and we train internally our own auditors and we just make sure that we are transparent about that and everyone can see what we are expecting in 275 00:23:59,287 --> 00:24:00,978 terms of capabilities. 276 00:24:01,039 --> 00:24:10,890 But eventually, probably everybody's gonna have to get some sort of certification whenever one of these organizations kind of rises to the top in terms of credibility. 277 00:24:12,468 --> 00:24:18,364 And then going back to when you answer the question about what makes a good audit, think the third thing you said was independence. 278 00:24:19,025 --> 00:24:24,421 Isn't there though an inherent conflict of interest if a company is paying you to provide a service? 279 00:24:24,421 --> 00:24:31,288 How can you provide enough of an assurance of independence in what you respond to them? 280 00:24:31,616 --> 00:24:32,046 Yeah. 281 00:24:32,046 --> 00:24:39,789 So we basically approach this the same way that, that financial auditors approach this or assurance professionals approach this. 282 00:24:39,789 --> 00:24:43,531 So there's a, there's a standard kind of code of conduct. 283 00:24:43,531 --> 00:24:50,034 And we, in fact, the standard we rely on is the same one that the, auditors rely on. 284 00:24:50,034 --> 00:24:57,557 And so we look at, because this happens, of course, you know, a company pays Deloitte to come audit their financial, they have to pay them, right. 285 00:24:57,557 --> 00:24:59,677 Deloitte is not going to do it for free. 286 00:24:59,717 --> 00:25:01,506 So there, there are. 287 00:25:01,506 --> 00:25:11,070 different ways of mitigating the sort of fundamental issue of impartiality and independence. 288 00:25:11,070 --> 00:25:14,751 know, Sarbanes -Oxley came along and had some additional requirements there. 289 00:25:14,751 --> 00:25:18,053 We keep an eye on that and make sure that we're following that as well. 290 00:25:18,353 --> 00:25:22,124 But there, it's not easy and it's a difficult task. 291 00:25:22,124 --> 00:25:30,946 And what makes it more difficult is it's not like financial auditing where you can show up and expect that the company understands how 292 00:25:30,946 --> 00:25:38,371 bookkeeping works and understands how they need to track their financial transactions. 293 00:25:39,012 --> 00:25:40,273 That's not the case here. 294 00:25:40,273 --> 00:25:47,716 We are dealing with organizations that it's shifting so much that they don't know what they need to do. 295 00:25:47,898 --> 00:25:54,422 And so we have to play a balancing act of education, like what is it that you need to do? 296 00:25:54,623 --> 00:25:57,885 But then we also are gonna come in and ensure that you do that. 297 00:25:57,885 --> 00:25:59,308 And that's not easy. 298 00:25:59,308 --> 00:26:10,172 And I think that the level of independence is by necessity slightly less than it would be if there was a very strict standard that everybody understood and they go off and get it 299 00:26:10,172 --> 00:26:13,023 done and we are only coming in to check their homework. 300 00:26:13,324 --> 00:26:18,686 But I think that that's just a function of the time we're in right now. 301 00:26:18,726 --> 00:26:28,802 And as this becomes more mature as a industry, we're gonna see that independence is actually not gonna be that different than 302 00:26:28,802 --> 00:26:30,311 than financial auditing. 303 00:26:31,754 --> 00:26:43,460 So right now, how do you convince clients or prospective clients to accept your approach to it, to, you know, the value of what you're doing, the way that you put it together for 304 00:26:45,391 --> 00:26:48,072 sometimes it's difficult. 305 00:26:48,753 --> 00:26:49,903 what are the challenges? 306 00:26:49,903 --> 00:26:55,036 The challenges are the way we approach it is that we are not consulting them. 307 00:26:55,936 --> 00:27:00,368 we're, are simply, they have to do all the work, right? 308 00:27:00,368 --> 00:27:02,020 So they're the ones who do testing. 309 00:27:02,020 --> 00:27:04,421 They're the ones who have to manage the risk. 310 00:27:04,421 --> 00:27:11,855 We have governance requirements, like for New York, New York law doesn't require risk assessments or governance or any risk management at all. 311 00:27:11,855 --> 00:27:13,600 We have that in our audits. 312 00:27:13,600 --> 00:27:23,435 And so there's this super augatory nature of some of the things that they have to accept that they're going to be doing the work and we are going to be checking their homework and 313 00:27:23,435 --> 00:27:28,177 validating the everything they've done, everything they said they did, they've actually done. 314 00:27:28,457 --> 00:27:30,018 That's a tough call. 315 00:27:30,018 --> 00:27:38,218 The way we, we convince them is that every law besides New York, every law that's coming 316 00:27:38,218 --> 00:27:42,780 is going to require robust internal governance and risk management and technical testing. 317 00:27:42,780 --> 00:27:43,860 They all are. 318 00:27:43,860 --> 00:27:50,543 So if you were going to be a mature organization using AI in the near future, you have to do this anyway. 319 00:27:50,543 --> 00:27:59,707 So why not start now where the bar is much lower, get your feet wet, get used to having people hold your feet to the fire. 320 00:27:59,707 --> 00:28:03,548 And that's going to mature you as an organization and then you can grow from there. 321 00:28:03,548 --> 00:28:05,909 That's the, that's the selling point. 322 00:28:05,989 --> 00:28:08,130 Not everybody wants that of course. 323 00:28:08,130 --> 00:28:13,734 But at this point, we're only looking for the people who are willing to do that. 324 00:28:13,734 --> 00:28:16,405 And so there's a filter effect a little bit. 325 00:28:16,405 --> 00:28:25,762 And so there's probably a large swath of the market that is going to get serviced by other kinds of organizations or other kinds of companies that don't approach audit in the same 326 00:28:25,762 --> 00:28:26,762 way we do. 327 00:28:29,803 --> 00:28:33,176 How broad do you think the practice of AI auditing is going to get? 328 00:28:33,176 --> 00:28:42,474 Because obviously we're seeing now companies of every scale in every industry deploying AI systems and often many different AI systems. 329 00:28:42,474 --> 00:28:46,767 So do you envision a world where all those things are subject to auditing? 330 00:28:46,767 --> 00:28:52,962 sure that would be good for a firm like yours from a business standpoint, but obviously there's some overhead at some point. 331 00:28:53,388 --> 00:28:57,280 Yeah, so I don't think so. 332 00:28:57,280 --> 00:29:11,223 think it's going to evolve into much more of a, and again, I don't know, like I don't have a crystal ball, but my hunch is that a lot of the processes for all of these algorithms 333 00:29:11,223 --> 00:29:22,242 are gonna have to be automated and there will be centralized risk management and governance and testing of these systems that will be 334 00:29:22,688 --> 00:29:37,730 more specific and more based on standards and that the role of the auditor is to audit that system, that quality management system with the extra bits that are really relevant 335 00:29:37,730 --> 00:29:39,312 for AI systems. 336 00:29:39,312 --> 00:29:42,635 think that's going to be, because that's much more manageable. 337 00:29:42,635 --> 00:29:48,980 don't have to check that every, you know, for this particular algorithm that you're using internally to do something is compliant. 338 00:29:48,980 --> 00:29:52,182 What I'm checking is you have that program in place. 339 00:29:52,268 --> 00:29:56,119 and you have the processes in place, the controls that are going to manage that. 340 00:29:56,759 --> 00:30:06,482 Now, the caveat here is that some laws like the EU AI Act, there'll be high risk systems which have requirements for the AI system itself. 341 00:30:06,883 --> 00:30:11,204 Luckily, a lot of those requirements are also organizational based. 342 00:30:11,204 --> 00:30:14,205 do you have a risk, have you done risk management for that? 343 00:30:14,205 --> 00:30:14,545 Right. 344 00:30:14,545 --> 00:30:18,806 So one risk management system can cover multiple high risk algorithms. 345 00:30:19,074 --> 00:30:27,257 The technical testing is going to be unique to a particular algorithm, but that can be centralized and audited at a sort of organizational level. 346 00:30:27,257 --> 00:30:39,712 so anyway, that's what I see it's emerging as organizations get audited and the individual AI systems are being controlled internally in a robust way by those organizations. 347 00:30:41,374 --> 00:30:50,436 And going forward, what do you see as the greatest challenge for the auditing community in AI or specifically for you at Babel? 348 00:30:51,086 --> 00:31:00,029 So the biggest challenge is right now, as you alluded to, the explosion of standards and laws. 349 00:31:00,029 --> 00:31:09,202 there are just so many, so we spent a long time for New York City, for instance, developing how are we gonna test, and it's very simple, it's like one metric, disparate 350 00:31:09,202 --> 00:31:09,692 impact. 351 00:31:09,692 --> 00:31:10,733 How are we gonna test that? 352 00:31:10,733 --> 00:31:13,794 How are we gonna provide assurance that they have actually done the right thing? 353 00:31:13,794 --> 00:31:15,494 And it was a lot of development. 354 00:31:16,158 --> 00:31:23,320 And if you look at these laws, like the digital services act, which is already out and, and, and, and people are being audited for that. 355 00:31:23,320 --> 00:31:29,042 The EU AI act, Colorado, all the state laws, some of the federal guidance that is, that is coming out. 356 00:31:29,042 --> 00:31:34,903 There's just so many kind of normative frameworks for, for what you need to do. 357 00:31:35,124 --> 00:31:38,785 That it's going to be difficult to have that level of development work. 358 00:31:38,785 --> 00:31:43,818 And so the way we're approaching this is that we are focusing on. 359 00:31:43,818 --> 00:31:47,599 our internal process of like, how do we provide assurance? 360 00:31:47,599 --> 00:31:54,101 How do we audit these systems and these organizations in general based on that normative guidance? 361 00:31:54,441 --> 00:31:56,552 What is, and then it's like, what is the process? 362 00:31:56,552 --> 00:32:02,263 If someone comes to us with this brand new thing, like in Taiwan, we need to, this is the regulation and here are the standards. 363 00:32:02,263 --> 00:32:06,604 What's our process for determining is this auditable? 364 00:32:07,385 --> 00:32:09,565 What are the capabilities we need to audit that? 365 00:32:09,565 --> 00:32:12,152 What are the extra resources we might need to 366 00:32:12,152 --> 00:32:15,125 to execute that audit at what level of assurance. 367 00:32:15,125 --> 00:32:20,871 And that's sort of a more unified framework that we're working on. 368 00:32:20,871 --> 00:32:32,082 And that's the way it's probably going to have to proceed until there's like really niche down into like, I'm an auditor for just this one law and then just this one industry. 369 00:32:32,242 --> 00:32:33,924 have to be kind of, there has to be that flex. 370 00:32:33,924 --> 00:32:36,276 That's the challenge, that flexibility. 371 00:32:37,802 --> 00:32:40,665 Great, well this has been a really fascinating conversation. 372 00:32:40,866 --> 00:32:43,719 Really interesting to see how the whole space of all is going forward. 373 00:32:43,719 --> 00:32:45,380 Jay, thank you very much for your time. 374 00:32:45,506 --> 00:32:47,157 Thank you so much for having me.