1 00:00:01,358 --> 00:00:05,134 Shaun O 'Noe, thank you so much for joining me on the road to accountable AI. 2 00:00:05,580 --> 00:00:07,214 Thank you for having me. 3 00:00:08,654 --> 00:00:12,434 So what is responsible AI or AI ethics? 4 00:00:12,434 --> 00:00:14,314 Pick the term that makes the most sense to you. 5 00:00:14,314 --> 00:00:16,614 What does that mean to you? 6 00:00:17,740 --> 00:00:24,770 So AI ethics, or for us, it's also how we call it, digital ethics under data ethics 7 00:00:24,770 --> 00:00:30,900 and AI ethics for me is the responsible handling of algorithms and AI. 8 00:00:30,900 --> 00:00:37,580 So how can we responsibly use and deploy this technology in the world so that key 9 00:00:37,580 --> 00:00:41,910 ethical principles are safeguarded so that we create trust within our solutions? 10 00:00:41,910 --> 00:00:45,260 That may be trust from partners, trust from... 11 00:00:45,260 --> 00:00:50,830 even our own employees plus from customers, that we offer solutions that 12 00:00:50,830 --> 00:00:53,580 they trust us to buy solutions or work with us. 13 00:00:53,580 --> 00:00:55,460 That's the whole idea behind this. 14 00:00:55,460 --> 00:01:00,600 And to do this, actually, even besides regulation or an absence of regulation or 15 00:01:00,600 --> 00:01:02,860 developer regulation, we want to do the right thing. 16 00:01:02,860 --> 00:01:03,650 And how do we do this? 17 00:01:03,650 --> 00:01:06,120 By behaving ethically in our AI solutions. 18 00:01:06,120 --> 00:01:10,284 That's the not so short answer to responsible AI. 19 00:01:11,438 --> 00:01:15,588 Before you had a digital ethics program, as I understand it, you had a bioethics 20 00:01:15,588 --> 00:01:17,018 advisory panel. 21 00:01:17,138 --> 00:01:22,930 How did that earlier work inform what you're doing on digital ethics and AI? 22 00:01:23,596 --> 00:01:27,496 So yeah, actually, so with the company, the bioethics work started already in 23 00:01:27,496 --> 00:01:28,926 2011, which is some time ago. 24 00:01:28,926 --> 00:01:32,026 And this arose from, let's say, bioethics challenges for us. 25 00:01:32,026 --> 00:01:33,496 We have a strong health care pillar. 26 00:01:33,496 --> 00:01:34,896 One of the pillars is health care. 27 00:01:34,896 --> 00:01:38,106 And it actually has quite some sort of like obviously, we're going to try the 28 00:01:38,106 --> 00:01:42,636 programs, but also our fertility research, one of the leaders in female fertility. 29 00:01:42,736 --> 00:01:46,996 Actually, this brought quite some questions around sort of like research on 30 00:01:46,996 --> 00:01:49,484 embryos, embryonic stem cells with it. 31 00:01:49,484 --> 00:01:51,304 And how do we deal about this? 32 00:01:51,304 --> 00:01:55,884 And sort of at that point, 2011, there was quite a big debate around this. 33 00:01:55,884 --> 00:02:01,314 And so we decided to get external help in the sort of like the Merck Bioethics 34 00:02:01,314 --> 00:02:02,964 Advisory Panel was then called. 35 00:02:02,964 --> 00:02:06,314 And they really helped us to come up with principles and structures and how to 36 00:02:06,314 --> 00:02:07,944 address ethical questions. 37 00:02:07,944 --> 00:02:12,764 And then when five years ago, we first ventured really into digital health with 38 00:02:12,764 --> 00:02:16,004 the collaboration that we have with Palantir called Syntropy. 39 00:02:16,004 --> 00:02:18,892 We really looked into sort of like, okay, 40 00:02:18,892 --> 00:02:19,922 How do we do this ethically? 41 00:02:19,922 --> 00:02:23,712 Because as I mentioned, the trust element is really critical for us. 42 00:02:24,012 --> 00:02:27,402 And then we looked at sort of like, OK, what do we have we learned from the 43 00:02:27,402 --> 00:02:28,732 bioethics? 44 00:02:28,732 --> 00:02:30,932 How do we operationalize it? 45 00:02:30,932 --> 00:02:32,152 How we put this forward? 46 00:02:32,152 --> 00:02:35,612 And that's sort of like how we borrowed from the bioethical principles and 47 00:02:35,612 --> 00:02:39,612 frameworks and everything that we had done, and also these advisory panel work 48 00:02:39,612 --> 00:02:43,092 into digital ethics and then also AI ethics. 49 00:02:43,092 --> 00:02:47,142 And that's something that I guess why pharmaceutical companies have a little bit 50 00:02:47,142 --> 00:02:47,786 of a... 51 00:02:47,786 --> 00:02:51,586 Head start here because we have been dealing with these ethical questions all 52 00:02:51,586 --> 00:02:54,316 along and we are a highly regulated sector at the same way. 53 00:02:54,316 --> 00:02:57,576 So that's not, we're not the first, not the first time we encountered this. 54 00:02:57,576 --> 00:03:01,916 So we can borrow some ideas we've had before, other than maybe let's say big 55 00:03:01,916 --> 00:03:06,986 tech who have not been in this situation before that there's like also like a very 56 00:03:06,986 --> 00:03:09,556 critical public having a look at you. 57 00:03:09,556 --> 00:03:12,886 So like, how do you behave doing this rightly and really being careful that you 58 00:03:12,886 --> 00:03:15,182 do the right thing in your innovations. 59 00:03:15,182 --> 00:03:22,072 What's different about AI ethics from bioethics? 60 00:03:22,072 --> 00:03:25,852 Why not take the existing advisory board that you had and just extend it into this 61 00:03:25,852 --> 00:03:26,462 area? 62 00:03:26,462 --> 00:03:31,442 Yeah, so we realized we needed very specific expertise. 63 00:03:31,442 --> 00:03:36,002 Actually, we went with our first digital ethics questions to this existing advisory 64 00:03:36,002 --> 00:03:37,832 panel and said, OK, can you advise us how to do this? 65 00:03:37,832 --> 00:03:38,542 And they said, yes. 66 00:03:38,542 --> 00:03:41,662 We can give you a few hints, but this is out of our realm. 67 00:03:41,662 --> 00:03:43,042 This is not something we can help you with. 68 00:03:43,042 --> 00:03:44,702 We can give you some basic principles. 69 00:03:44,702 --> 00:03:48,822 But first of all, you actually, for yourself, you need to set yourself, let's 70 00:03:48,822 --> 00:03:52,822 say, ethical standards or ethical principles that then you can derive your 71 00:03:52,822 --> 00:03:53,302 tools from. 72 00:03:53,302 --> 00:03:55,897 Because we always come from the 73 00:03:56,234 --> 00:04:00,334 bio and medical side, and we have no experience at all in the digital sea. 74 00:04:00,334 --> 00:04:03,284 I mean, we have some that have sort of like big data, medical, big data 75 00:04:03,284 --> 00:04:05,814 experience, but the skill set is very different. 76 00:04:05,814 --> 00:04:09,384 And that's also why our digital ethics advisory panel is, other than the 77 00:04:09,384 --> 00:04:12,834 bioethics one, is consisted also out of industry experts. 78 00:04:12,834 --> 00:04:16,654 So they're all external, but they're, let's say, a mixture of academic and 79 00:04:16,654 --> 00:04:22,124 industry experts, while the bioethics purely academic, because... 80 00:04:22,124 --> 00:04:26,964 there's hardly enough, let's say, academic insight right now into all the AI. 81 00:04:26,964 --> 00:04:28,854 And the innovation is happening within the company. 82 00:04:28,854 --> 00:04:32,444 So it's very difficult to get the right expertise there. 83 00:04:32,444 --> 00:04:34,184 So we write more on this. 84 00:04:34,464 --> 00:04:38,384 And then, yes, I said also, for example, our legal experts, they really come now 85 00:04:38,384 --> 00:04:44,224 from the FDA regulatory medical devices, AI side, to really help us with this. 86 00:04:44,404 --> 00:04:49,204 And also, the factor of fairness became much more important in this regard. 87 00:04:49,204 --> 00:04:51,564 So we have an anthropologist, actually, from Johns Hopkins. 88 00:04:51,564 --> 00:04:58,264 pockets, Khadija Ferryman joining, because he has a special eye on neglected 89 00:04:58,264 --> 00:05:03,034 populations and how do we really take care that there's not a bias in our systems and 90 00:05:03,034 --> 00:05:06,364 how can we take care that these people are rightfully addressed also. 91 00:05:06,704 --> 00:05:08,744 So that's some of the points. 92 00:05:08,744 --> 00:05:10,624 And then also the principles are not the same. 93 00:05:10,624 --> 00:05:14,124 I mean, they're similar, but then there's different emphasis. 94 00:05:14,304 --> 00:05:16,266 And also one thing is that... 95 00:05:16,266 --> 00:05:20,096 Bioethicists are not too keen on going to principle frameworks and these kind of 96 00:05:20,096 --> 00:05:24,886 this and the digital ethics are much more open to this because they see at a large 97 00:05:24,886 --> 00:05:25,926 scale, how can we do this? 98 00:05:25,926 --> 00:05:31,236 Because within bioethics, it's usually a few larger topics that we discuss. 99 00:05:31,236 --> 00:05:34,556 And within digital ethics, it's a huge amount of topics that we go into on a 100 00:05:34,556 --> 00:05:35,356 regular basis. 101 00:05:35,356 --> 00:05:39,606 I mean, we have more than a thousand analytics use cases, for example, in one 102 00:05:39,606 --> 00:05:40,456 unit alone. 103 00:05:40,456 --> 00:05:42,246 And how can we screen this at a high throughput? 104 00:05:42,246 --> 00:05:44,130 And this will not work without. 105 00:05:44,848 --> 00:05:50,438 assessments and frameworks and these likes, and will not just by like be a one 106 00:05:50,438 --> 00:05:52,168 -on -one discussion with an SSS. 107 00:05:52,168 --> 00:05:55,248 But I will go into this in more detail probably later on. 108 00:05:55,470 --> 00:05:55,940 absolutely. 109 00:05:55,940 --> 00:05:57,010 No, it's really interesting. 110 00:05:57,010 --> 00:06:01,570 How do you find people who have the right kind of expertise? 111 00:06:01,570 --> 00:06:07,020 It probably was easier when you were doing it, getting started a few years ago, than 112 00:06:07,020 --> 00:06:11,210 now where every company now feels like they need to bring in people that have AI 113 00:06:11,210 --> 00:06:11,970 ethics expertise. 114 00:06:11,970 --> 00:06:15,490 But how do you find people that have the right expertise, but also are a good 115 00:06:15,490 --> 00:06:17,124 enough fit for your organization? 116 00:06:17,644 --> 00:06:21,344 Yeah, so I mean, sort of like we had the, we had started a little bit because we 117 00:06:21,344 --> 00:06:25,274 went through obviously our existing panel where we have to, okay, do you know 118 00:06:25,274 --> 00:06:26,484 somebody and something's happening? 119 00:06:26,484 --> 00:06:29,884 But then five years ago, this was a little bit more exotic. 120 00:06:30,024 --> 00:06:33,424 Nowadays, it's also more difficult because everybody is all of a sudden an AI ethics 121 00:06:33,424 --> 00:06:34,064 expert. 122 00:06:34,064 --> 00:06:36,964 So you're really difficult to find this. 123 00:06:37,284 --> 00:06:42,508 And so when we try to go through this, despite going through our... 124 00:06:42,508 --> 00:06:45,388 let's say, our esteemed experts that we already have. 125 00:06:45,388 --> 00:06:49,498 We go through our business that often has already the context to sort of like the 126 00:06:49,498 --> 00:06:56,108 right people in our digital unit, EMV Digital, that's located in Cambridge, 127 00:06:56,108 --> 00:06:57,128 Massachusetts. 128 00:06:57,128 --> 00:07:01,548 They have really good contacts already, let's say, to the academic sphere due to 129 00:07:01,548 --> 00:07:02,268 this innovation part. 130 00:07:02,268 --> 00:07:06,548 And so we got somebody along as John Alamka, who works for Mayo Clinic, but was 131 00:07:06,548 --> 00:07:09,528 like a CIO on Harvard for many years, to give us insights. 132 00:07:09,528 --> 00:07:12,008 And then from there, we went bit by bit forward looking. 133 00:07:12,008 --> 00:07:12,174 Also, 134 00:07:12,174 --> 00:07:12,958 you 135 00:07:12,958 --> 00:07:13,348 do we need? 136 00:07:13,348 --> 00:07:17,188 So we scan a little bit the academics here, looking into, OK, what kind of 137 00:07:17,188 --> 00:07:18,788 academics do we need? 138 00:07:18,788 --> 00:07:20,128 Is there already something? 139 00:07:20,128 --> 00:07:21,588 And what kind of expertise do they have? 140 00:07:21,588 --> 00:07:22,828 Have they published in this field? 141 00:07:22,828 --> 00:07:27,588 This is obviously very important that we go by publications and see, OK, speakers, 142 00:07:27,588 --> 00:07:28,418 what's the background? 143 00:07:28,418 --> 00:07:32,328 And then usually we invite them for a few test sessions and see, does that work for 144 00:07:32,328 --> 00:07:33,408 panel meetings? 145 00:07:33,408 --> 00:07:39,218 And through them, we then get exposed to other experts again, also to more industry 146 00:07:39,218 --> 00:07:39,768 experts. 147 00:07:39,768 --> 00:07:40,364 So it's the. 148 00:07:40,364 --> 00:07:41,464 a bit of growing exercise. 149 00:07:41,464 --> 00:07:44,754 And I think we started with three experts and then one change, then we were only 150 00:07:44,754 --> 00:07:45,054 two. 151 00:07:45,054 --> 00:07:47,564 And then by now, I think we're seven experts. 152 00:07:47,564 --> 00:07:50,744 And what's important to see that we also have a linker between these two panels. 153 00:07:50,744 --> 00:07:55,344 So there's one bioethicist, Jeremy Sugarman, he's also from Johns Hopkins. 154 00:07:55,344 --> 00:07:56,334 He's a bioethic professor. 155 00:07:56,334 --> 00:07:58,544 He has done some work for the Obama administration. 156 00:07:58,544 --> 00:08:03,064 So he's very well renowned in the US and he's, let's say, a generalist in bioethics 157 00:08:03,064 --> 00:08:05,184 and he's also sitting on the digital ethics panel. 158 00:08:05,184 --> 00:08:09,384 So there's a link between them and they can also learn from each other. 159 00:08:09,388 --> 00:08:10,988 Also share best practices. 160 00:08:11,662 --> 00:08:15,552 You talked about the distinction between bioethics and digital ethics, although you 161 00:08:15,552 --> 00:08:17,622 mentioned there are some connections. 162 00:08:17,622 --> 00:08:20,662 Is AI ethics just a subset of digital ethics? 163 00:08:20,662 --> 00:08:23,412 for us, it's a subset of digital ethics. 164 00:08:24,092 --> 00:08:30,202 Because when we started to develop, let's say, ways to ethically handle data, 165 00:08:30,202 --> 00:08:33,532 because our first business ideas, this started then with, as mentioned, product 166 00:08:33,532 --> 00:08:34,572 centropy. 167 00:08:34,992 --> 00:08:38,782 This was about health data and sharing health data and how can we analyze data in 168 00:08:38,782 --> 00:08:40,672 larger data banks. 169 00:08:40,672 --> 00:08:44,412 And we decided, OK, we need something for ethical handling of data. 170 00:08:44,412 --> 00:08:45,812 This was 2018. 171 00:08:45,812 --> 00:08:47,404 Then AI ethics had popped up. 172 00:08:47,404 --> 00:08:48,584 but it was not really present. 173 00:08:48,584 --> 00:08:54,074 But we said from the beginning, OK, we want to do something that encompasses AI 174 00:08:54,074 --> 00:08:58,624 ethics and data ethics at the same time, because for AI, you will always need good 175 00:08:58,624 --> 00:09:02,944 sets of data and sets that are handled the data the right way for all the training, 176 00:09:02,944 --> 00:09:04,744 especially now for generative AI. 177 00:09:04,744 --> 00:09:08,034 And so we decided to do this under the term digital ethics as one. 178 00:09:08,034 --> 00:09:13,054 Many might just be also responsible AI, maybe for some the more, especially in the 179 00:09:13,054 --> 00:09:15,044 IT field, the more. 180 00:09:16,204 --> 00:09:19,394 understandable term for them, but for us, this is sort of like digital ethics 181 00:09:19,394 --> 00:09:20,484 encompasses this all. 182 00:09:20,484 --> 00:09:24,094 So that we really also make sure for data, this is something as a German company or 183 00:09:24,094 --> 00:09:29,754 German headquartered company, this is also something where there's a high, at least 184 00:09:29,754 --> 00:09:34,994 within the country, there's a high view on how do we handle data or within the EU in 185 00:09:34,994 --> 00:09:37,684 general, the GDPR, you have heard about this probably. 186 00:09:37,684 --> 00:09:42,364 So that's why also we said, okay, we also need a strong footprint on data ethics. 187 00:09:42,364 --> 00:09:45,084 And so we encompassed this as two together. 188 00:09:45,230 --> 00:09:52,700 Other than the heightened sensitivity and the regulation, as you mentioned, what's 189 00:09:52,700 --> 00:09:57,130 different about thinking through these issues in a healthcare and life sciences 190 00:09:57,130 --> 00:09:58,170 company? 191 00:09:59,240 --> 00:10:03,670 mean, it's obviously been a heavily regulated environment already, which makes 192 00:10:03,670 --> 00:10:05,640 it more difficult a little bit to operate. 193 00:10:05,640 --> 00:10:08,600 And then it's also the most sensitive area you can imagine. 194 00:10:08,600 --> 00:10:13,540 So the trust element is even more critical for us than for other providers, I'd say. 195 00:10:13,540 --> 00:10:20,740 I mean, maybe it goes similarly for banks and the sensitivity or insurance. 196 00:10:20,748 --> 00:10:23,928 But then health is usually the most personal information people have. 197 00:10:23,928 --> 00:10:29,908 And we see this even in the US, that there's a higher, let's say, alertness 198 00:10:29,908 --> 00:10:33,528 towards health data than maybe other data being used. 199 00:10:33,528 --> 00:10:35,468 And so this was really for us the strong push. 200 00:10:35,468 --> 00:10:37,248 OK, we need to get this forward. 201 00:10:37,248 --> 00:10:40,808 And also, we're discussing with other companies in the health field. 202 00:10:40,808 --> 00:10:44,518 And actually, the most reach out from other companies I get is from actually in 203 00:10:44,518 --> 00:10:48,688 the health care field, because they see they have to be extra alert and extra 204 00:10:48,688 --> 00:10:49,452 early prepared. 205 00:10:49,452 --> 00:10:53,152 this because this is something that might help them otherwise in the long run. 206 00:10:53,152 --> 00:10:59,422 And also if you look at the UAI Act applications within healthcare, we'll most 207 00:10:59,422 --> 00:11:05,362 likely get in the high risk group and always will need more detailed analysis 208 00:11:05,362 --> 00:11:07,032 and oversight. 209 00:11:08,206 --> 00:11:12,056 Cutting the other way though, even within your corporate group, there are many 210 00:11:12,056 --> 00:11:16,346 different businesses, different units, different products, a range of different 211 00:11:16,346 --> 00:11:17,706 kinds of issues. 212 00:11:17,706 --> 00:11:22,166 So how can you, as a large company, you develop one set of guidelines or one 213 00:11:22,166 --> 00:11:25,646 process around digital ethics that spans all of them? 214 00:11:25,680 --> 00:11:29,110 so, I we have, I mean, I lead a small team of experts. 215 00:11:29,110 --> 00:11:35,210 I mean, that's we're the parts directly to the board of directors and we have, we are 216 00:11:35,210 --> 00:11:38,340 sort of like owning the topic over all our businesses. 217 00:11:38,340 --> 00:11:42,920 So the healthcare, the life science and our electronics business and also are what 218 00:11:42,920 --> 00:11:45,000 we call the enabling functions. 219 00:11:45,000 --> 00:11:46,994 So all our, let's say, 220 00:11:46,994 --> 00:11:48,824 finance and HR for everything. 221 00:11:48,824 --> 00:11:53,388 So, and we actually take care of questions for all of them, but we, and we have one. 222 00:11:53,388 --> 00:11:57,288 of digital ethics for the company that defines our principles. 223 00:11:57,288 --> 00:12:00,128 And it's not do's and don'ts, it's specifically principles. 224 00:12:00,128 --> 00:12:03,688 A set of in total 20 grouped under five major principles. 225 00:12:03,688 --> 00:12:07,538 And from these, we can develop guidelines and do's and don'ts for certain 226 00:12:07,538 --> 00:12:09,268 applications, certain areas, certain fields. 227 00:12:09,268 --> 00:12:13,748 So that was very important for us that we do something for the entire group. 228 00:12:13,748 --> 00:12:16,488 But then obviously, these sectors have very different needs. 229 00:12:16,488 --> 00:12:20,998 I mean, for our life science sector, for example, for the analytics unit, I already 230 00:12:20,998 --> 00:12:23,390 mentioned it before, we have one that's called the 231 00:12:23,390 --> 00:12:24,280 center of excellence. 232 00:12:24,280 --> 00:12:26,280 We have a platform here on Foundry. 233 00:12:26,280 --> 00:12:27,540 It's a technology platform. 234 00:12:27,540 --> 00:12:34,260 And here people can develop analytics use cases from ideation to proof of value to 235 00:12:34,260 --> 00:12:35,600 industrialization. 236 00:12:35,980 --> 00:12:39,820 And they, for example, asked us, can you quickly build us a filter so we can 237 00:12:39,820 --> 00:12:42,600 identify those cases that have ethical challenges? 238 00:12:42,600 --> 00:12:47,460 And within life science, which is sort of like our business that sells, let's say, 239 00:12:47,460 --> 00:12:51,890 any kind of lab materials, but also tools like genome editing tools or 240 00:12:52,396 --> 00:12:59,016 mRNAs and lipids, so also offer services, their data might not be as sensitive as 241 00:12:59,016 --> 00:12:59,396 they have. 242 00:12:59,396 --> 00:13:05,016 So we don't think that there's so many use cases that are critical. 243 00:13:05,016 --> 00:13:09,236 So we just wanted to drive with a filter to find out the views that are critical. 244 00:13:09,236 --> 00:13:13,096 And so we based on our critical ethics, we develop ethical markers that we then put 245 00:13:13,096 --> 00:13:16,876 into questions that the use case owners can actually have to answer as part of the 246 00:13:16,876 --> 00:13:19,036 process in their different stages. 247 00:13:19,036 --> 00:13:21,292 And we can see, OK, is that ethical risk? 248 00:13:21,292 --> 00:13:24,892 yes or no, and then we can, if there's an ethical risk or the score is high, then we 249 00:13:24,892 --> 00:13:26,992 go and analyze it. 250 00:13:26,992 --> 00:13:28,492 So this is, for example, one way. 251 00:13:28,492 --> 00:13:34,362 But then, for example, in HR, which is right now one of my closest customers, I'd 252 00:13:34,362 --> 00:13:38,532 say, with all the Gen .AI solutions that come along, or AI solutions in general, 253 00:13:38,532 --> 00:13:41,712 whether we insource them or if we develop them ourselves. 254 00:13:42,472 --> 00:13:46,028 For these, instead of like, these are all. 255 00:13:46,028 --> 00:13:49,188 Yeah, these are not so many, but then they all have very individual needs. 256 00:13:49,188 --> 00:13:52,478 So we're currently developing a different framework, a little bit based on what 257 00:13:52,478 --> 00:13:56,288 we've done before, but much more concrete for HR, which comes with certain 258 00:13:56,288 --> 00:13:57,048 questions. 259 00:13:57,048 --> 00:14:01,498 And then we also developed something similar, again, for healthcare, really 260 00:14:01,498 --> 00:14:02,528 tailored to them. 261 00:14:02,528 --> 00:14:08,128 So that's really, yeah, the code is a pretty good for all, but then it's like, 262 00:14:08,128 --> 00:14:10,248 yeah, how do we market our different customers? 263 00:14:10,248 --> 00:14:15,148 So that's really running a small consultancy here rather than, yeah. 264 00:14:15,148 --> 00:14:20,408 doing a one -size -fits -all solution, or doing a do and don't for everything in one 265 00:14:20,408 --> 00:14:21,508 document. 266 00:14:26,828 --> 00:14:30,548 So for those who are really keen on this, they can read about this. 267 00:14:30,548 --> 00:14:34,808 There's actually a publication in AI Society about this, how we put this. 268 00:14:34,808 --> 00:14:38,128 They have published this with a partner from the University of Fittenherdeck in 269 00:14:38,128 --> 00:14:39,708 Germany, Sarah Becker. 270 00:14:39,788 --> 00:14:43,688 She has been on our advisory panel also before as one of the leading experts. 271 00:14:44,028 --> 00:14:45,132 And she's also 272 00:14:45,132 --> 00:14:49,042 we really went and looked into being a European company. 273 00:14:49,042 --> 00:14:52,182 We looked into, okay, what kind of principles and frameworks have been 274 00:14:52,182 --> 00:14:54,992 published in the EU in the last five years back then. 275 00:14:54,992 --> 00:14:58,732 So this was between 2015 and 2020 when we started this. 276 00:14:58,732 --> 00:15:02,472 And what has been published towards data ethics and the AI ethics. 277 00:15:02,472 --> 00:15:07,882 And I think we had like, and may they be from governments, may they be from 278 00:15:07,882 --> 00:15:09,662 corporations, may they be from... 279 00:15:09,662 --> 00:15:13,232 NGOs or think tanks, and we found a total of 62 frameworks. 280 00:15:13,232 --> 00:15:15,872 And this was back then, I guess, now with several hundred. 281 00:15:16,052 --> 00:15:20,712 And we tried to identify principles and all of them to have a most holistic view. 282 00:15:20,712 --> 00:15:23,502 And often they were called differently, but we really tried to look into all 283 00:15:23,502 --> 00:15:29,892 details and to see, OK, is there sort of like one word behind this that we can 284 00:15:29,892 --> 00:15:30,372 frame with us? 285 00:15:30,372 --> 00:15:34,962 Because some of them are really marketing speech or semantic differences, but we 286 00:15:34,962 --> 00:15:37,372 really could narrow them down to the different details. 287 00:15:37,372 --> 00:15:38,540 And then we... 288 00:15:38,540 --> 00:15:42,960 After that, we actually grouped them into the five major principles that you see now 289 00:15:42,960 --> 00:15:44,880 when you access the code on our homepage. 290 00:15:44,880 --> 00:15:46,650 You can see five different principles. 291 00:15:46,650 --> 00:15:50,940 And actually four of them are identical with the bioethical overall principles 292 00:15:50,940 --> 00:15:52,060 that exist. 293 00:15:52,060 --> 00:15:56,770 And one of them actually is new, is transparency, which is because this is 294 00:15:56,770 --> 00:16:00,210 actually more or less the key principles when we go to digital ethics. 295 00:16:00,210 --> 00:16:03,780 This was a larger endeavor that took us more over a year. 296 00:16:03,780 --> 00:16:07,510 What we also did after is actually we checked whether, okay, how does this work 297 00:16:07,510 --> 00:16:08,784 within the US? 298 00:16:08,812 --> 00:16:12,492 principles that have been formulated there, also within Asia and China, for 299 00:16:12,492 --> 00:16:13,572 example, especially. 300 00:16:13,572 --> 00:16:18,262 And we see that it actually matches pretty well because it really just defines ethic 301 00:16:18,262 --> 00:16:19,692 principles in a philosophical manner. 302 00:16:19,692 --> 00:16:23,392 So it's really, if you would read this as normal employee, it's rather complex and 303 00:16:23,392 --> 00:16:28,512 we have to easing it down for people to explain, but it really helps us to 304 00:16:28,512 --> 00:16:30,032 identify challenges. 305 00:16:30,032 --> 00:16:34,092 And we saw that, for example, we're working well in the US, but also in China. 306 00:16:34,092 --> 00:16:37,392 And maybe if we look at the Asian... 307 00:16:38,284 --> 00:16:42,834 geographies, it might more be that the traditional principle of harmony coming 308 00:16:42,834 --> 00:16:46,374 along, that the autonomy is not as important and there's harmony coming up a 309 00:16:46,374 --> 00:16:46,864 little bit more. 310 00:16:46,864 --> 00:16:50,724 Societal harmony is a higher value, but beyond that, we have something that is 311 00:16:50,724 --> 00:16:54,184 actually quite checked globally also. 312 00:16:55,736 --> 00:17:00,064 Why is transparency so distinctively important for digital ethics? 313 00:17:00,716 --> 00:17:06,716 So we think about this for us, the key element or why we do this is really to 314 00:17:06,716 --> 00:17:09,916 create trust in our solutions that we offer trustful solutions. 315 00:17:09,916 --> 00:17:17,286 And I think with this high opacity that AI has or the whole big data topic has behind 316 00:17:17,286 --> 00:17:21,646 it, people, it's really extremely difficult to grasp what is happening and 317 00:17:21,646 --> 00:17:24,156 everybody knows they're putting the data somewhere on the net and something's 318 00:17:24,156 --> 00:17:26,436 happening with it, but nobody knows what's really happening. 319 00:17:26,436 --> 00:17:29,456 And you always imagine these kinds of things that are happening with your data 320 00:17:29,456 --> 00:17:30,664 or there might be... 321 00:17:30,664 --> 00:17:31,784 used in a certain way. 322 00:17:31,784 --> 00:17:35,544 So this is for us that we say, okay, we want to be transparent, we really want to 323 00:17:35,544 --> 00:17:40,654 share sort of like what we do and how we do it and be as open as possible about our 324 00:17:40,654 --> 00:17:45,584 sort of like maybe AI solutions and what's happening within them so that people can 325 00:17:45,584 --> 00:17:49,984 actually trust in these solutions and lose the fear and skepticism of these 326 00:17:49,984 --> 00:17:50,424 solutions. 327 00:17:50,424 --> 00:17:55,054 I mean, and this goes not only externally, but also internally with our employees 328 00:17:55,054 --> 00:17:59,984 here at Merck KGA Darmstadt, Germany, because this is... 329 00:18:00,412 --> 00:18:04,222 really something that, I mean, obviously we have the whole variety of society 330 00:18:04,222 --> 00:18:05,192 working for us. 331 00:18:05,192 --> 00:18:09,752 And also here there's quite a level of mistrust, especially when we look at blue 332 00:18:09,752 --> 00:18:13,862 color workers, sort of like, okay, the CI may cause my job, but also our white color 333 00:18:13,862 --> 00:18:18,142 workers are threatened by CHPT and think, okay, what does that mean and what kind of 334 00:18:18,142 --> 00:18:19,552 solutions are we using? 335 00:18:19,552 --> 00:18:25,902 And so this is something where we think, okay, we really want to educate here. 336 00:18:25,902 --> 00:18:29,836 This is where this transparency even plays a very important, 337 00:18:29,836 --> 00:18:31,136 role in -house. 338 00:18:31,136 --> 00:18:37,006 So we really educate and be transparent with our own employees so that they 339 00:18:37,006 --> 00:18:38,122 understand this. 340 00:18:40,142 --> 00:18:43,592 One of the reasons that I was interested in speaking with you is that you have 341 00:18:43,592 --> 00:18:47,632 published a couple of these papers, which we'll put in the show notes, talking about 342 00:18:47,632 --> 00:18:48,262 your approach. 343 00:18:48,262 --> 00:18:52,582 And I was particularly interested in one that talked about what you call principle 344 00:18:52,582 --> 00:18:55,722 at risk analysis for operationalizing AI ethics. 345 00:18:55,722 --> 00:18:58,612 Can you talk a little bit about what that means? 346 00:18:58,858 --> 00:18:59,978 Yeah, so this comes a little bit. 347 00:18:59,978 --> 00:19:01,938 So why did we actually put this code to digital ethics? 348 00:19:01,938 --> 00:19:03,328 I mean, why do we need this? 349 00:19:03,328 --> 00:19:06,718 And I couldn't we just do with the bioethicists that advised us because we do 350 00:19:06,718 --> 00:19:09,288 not have like a code of bioethics for the company. 351 00:19:09,288 --> 00:19:13,848 And this comes to the fact that there is not yet a fixed sort of like set of 352 00:19:13,848 --> 00:19:15,868 principles for digital ethics out there. 353 00:19:15,868 --> 00:19:19,598 And bioethics are very well established since 30 years or so, but for us, it's not 354 00:19:19,598 --> 00:19:19,978 the case. 355 00:19:19,978 --> 00:19:24,708 And so we realized, okay, we cannot just simply form the digital ethics panel, but 356 00:19:24,708 --> 00:19:27,072 they need something to orient themselves on. 357 00:19:27,180 --> 00:19:33,630 And these principles that we formulated in the code, they are as guidance for the 358 00:19:33,630 --> 00:19:36,390 experts to take decisions, but also, obviously, as an overall guidance for the 359 00:19:36,390 --> 00:19:36,980 company. 360 00:19:36,980 --> 00:19:41,720 But also, and this is why it's really designed in this way, that we can build 361 00:19:41,720 --> 00:19:43,100 analytical tools from it. 362 00:19:43,100 --> 00:19:46,940 And this is the para, as we call it. 363 00:19:47,160 --> 00:19:52,950 This tool, actually, we developed directly from the code, which is a way to analyze 364 00:19:52,950 --> 00:19:56,480 the certain questions and see whether in a certain business model, 365 00:19:57,772 --> 00:20:00,292 things are at risk or principles at risk. 366 00:20:00,292 --> 00:20:03,332 And with this outcome of this analysis, we could say, OK, if we have a certain 367 00:20:03,332 --> 00:20:07,712 question, for example, are we responsible how things are done in a certain way and a 368 00:20:07,712 --> 00:20:08,432 certain technology? 369 00:20:08,432 --> 00:20:12,722 And then we can, an expert, not everybody in the company, but our expert, my team 370 00:20:12,722 --> 00:20:14,112 can do this analysis. 371 00:20:14,212 --> 00:20:19,432 And then we find out, OK, five principles of our total 20 are at risk. 372 00:20:19,432 --> 00:20:23,752 And then this for us is the discussion basis then with the panel. 373 00:20:23,752 --> 00:20:25,388 I mean, that might not be a. 374 00:20:25,388 --> 00:20:28,528 complete identification of all ethical risks, but it's really a good head start 375 00:20:28,528 --> 00:20:33,508 because often at the beginning of discussion around an ethical, a certain 376 00:20:33,508 --> 00:20:36,468 ideology, I think challenges, it's all over the place and you need to structure 377 00:20:36,468 --> 00:20:36,858 it well. 378 00:20:36,858 --> 00:20:41,508 And so this really helps us to go sort of like in the risks on certain principles 379 00:20:41,508 --> 00:20:43,788 and how can we address them bit by bit. 380 00:20:43,788 --> 00:20:46,468 And so this is also like a bit of a loop. 381 00:20:46,468 --> 00:20:48,808 So a question comes to us from the business. 382 00:20:48,808 --> 00:20:52,178 We sort of, if it's a complex question, we run this through this analysis and the 383 00:20:52,178 --> 00:20:53,420 report then goes to the panel. 384 00:20:53,420 --> 00:20:55,080 And then they can prepare themselves. 385 00:20:55,080 --> 00:20:59,710 And then we sit down with a panel and discuss all of these risks and how can we 386 00:20:59,710 --> 00:21:02,500 mitigate these risks, what has to be put in place. 387 00:21:02,500 --> 00:21:05,060 And then this goes back to the business as a recommendation. 388 00:21:05,060 --> 00:21:09,670 And then we see that a loop of reporting several times to see, OK, is there 389 00:21:09,670 --> 00:21:11,640 actually a solution that's working? 390 00:21:11,700 --> 00:21:15,900 So this is sort of like, how can we actually operationalize this? 391 00:21:15,900 --> 00:21:19,870 And this is, let's say, for larger complex questions, how can we do this in a better 392 00:21:19,870 --> 00:21:20,090 way? 393 00:21:20,090 --> 00:21:22,348 I mean, if we would run the power on any kind of 394 00:21:22,348 --> 00:21:26,268 of most questions, then I would need like 20 people in my office. 395 00:21:27,790 --> 00:21:31,400 But for those couple of discussions, it's a focusing mechanism, it sounds like, for 396 00:21:31,400 --> 00:21:33,996 the discussion that's an input to the advisory board. 397 00:21:33,996 --> 00:21:37,206 And also really shows you the risk that might arise and something that you might 398 00:21:37,206 --> 00:21:39,836 not just not have on your radar that might happen. 399 00:21:39,918 --> 00:21:43,038 And then the output, as I understand it, of the advisory board, as you said, is a 400 00:21:43,038 --> 00:21:43,918 recommendation. 401 00:21:43,918 --> 00:21:48,078 What happens if the business says, we don't agree, or the business benefit of 402 00:21:48,078 --> 00:21:50,330 this is too great, we're going forward anyway? 403 00:21:51,244 --> 00:21:53,164 So this is always the problem. 404 00:21:53,164 --> 00:21:55,854 There are two different approaches to how do you digital ethics. 405 00:21:55,854 --> 00:21:58,874 I mean, there's the way of ethics -based auditing, which is more through 406 00:21:58,874 --> 00:21:59,254 compliance. 407 00:21:59,254 --> 00:22:03,334 So there has been a paper of Yako Mukanda on this with AstraZeneca. 408 00:22:03,334 --> 00:22:05,144 I think this has been done. 409 00:22:05,204 --> 00:22:09,064 They showed this value of audit every project. 410 00:22:09,284 --> 00:22:11,274 And in our company, we run it a bit different. 411 00:22:11,274 --> 00:22:13,674 This is also an experience with bioethics. 412 00:22:13,674 --> 00:22:15,284 We are awareness -based. 413 00:22:15,284 --> 00:22:20,140 So we do trainings, and people know that they exist, that they can address us. 414 00:22:20,140 --> 00:22:25,040 And so we are sort of like, people are not forced to ask us, but sort of like they 415 00:22:25,040 --> 00:22:27,240 come to us for questions. 416 00:22:27,240 --> 00:22:29,930 And throughout the years, or with the bioethics, we have built a very good 417 00:22:29,930 --> 00:22:30,430 reputation. 418 00:22:30,430 --> 00:22:34,710 So people come to us because they need the help and they see this is actually helpful 419 00:22:34,710 --> 00:22:35,740 for their business. 420 00:22:35,740 --> 00:22:39,760 They could ignore us and they could also ignore the advice, but it's well 421 00:22:39,760 --> 00:22:43,810 documented within the company and sort of ignore the advice and later on, then run 422 00:22:43,810 --> 00:22:45,020 into problems. 423 00:22:45,020 --> 00:22:48,432 Then this actually, I mean, obviously we'll heard. 424 00:22:48,844 --> 00:22:54,514 very much the decision -takers, let's say, reputation with the company or with the 425 00:22:54,514 --> 00:22:54,804 company. 426 00:22:54,804 --> 00:22:57,464 So that's sort of like indirect pressure. 427 00:22:57,464 --> 00:23:00,644 And it also goes to the way that we say, OK, for us, we're a family -owned 428 00:23:00,644 --> 00:23:03,754 business, or let's say we're on the stock market, but 70 % is off -stock market and 429 00:23:03,754 --> 00:23:04,704 family hands. 430 00:23:04,704 --> 00:23:10,484 There's a higher sense of, let's say, generational thinking. 431 00:23:10,484 --> 00:23:14,354 So we want to have a sustainable business in 15, 30 years for the next generation of 432 00:23:14,354 --> 00:23:15,004 the family. 433 00:23:15,004 --> 00:23:18,316 And so this whole of responsible business has been longer. 434 00:23:18,316 --> 00:23:25,236 thing, and it has also been, it's on the responsibility is on everybody's hand. 435 00:23:25,236 --> 00:23:31,076 So it's not just with the data scientists or AI developer that's on the all end, but 436 00:23:31,076 --> 00:23:34,496 it's also with the middle manager, the high manager and the board of directors. 437 00:23:34,496 --> 00:23:36,816 So it's really the entire chain. 438 00:23:36,816 --> 00:23:39,836 And this is sort of like a certain culture we have established with the company. 439 00:23:39,836 --> 00:23:43,666 And this helps that actually people come to us and they take these recommendations 440 00:23:43,666 --> 00:23:44,186 seriously. 441 00:23:44,186 --> 00:23:46,348 I mean, they ask us for certain questions. 442 00:23:46,348 --> 00:23:50,998 And I must also say some recommendations, obviously, we are not able to implement 443 00:23:50,998 --> 00:23:53,348 because just they may be not realistic. 444 00:23:53,348 --> 00:23:56,328 But so we come back and we ask, OK, can we have a more pragmatic solution? 445 00:23:56,328 --> 00:23:57,198 But do you think about this? 446 00:23:57,198 --> 00:23:58,908 We approach it in a certain way. 447 00:23:58,908 --> 00:24:03,178 But sort of like how you think it, it's not working because probably they do not 448 00:24:03,178 --> 00:24:07,168 have that insights in our processes and business. 449 00:24:07,168 --> 00:24:10,428 And then we really go iterative way to find a solution that's best fitting. 450 00:24:10,428 --> 00:24:12,908 And I also have to say that our advisors are rather pragmatic. 451 00:24:12,908 --> 00:24:14,476 I mean, so we're not going to. 452 00:24:14,476 --> 00:24:17,826 We don't have ivory tower from an analyst that comes a certain way, but it's really 453 00:24:17,826 --> 00:24:18,356 OK. 454 00:24:18,356 --> 00:24:19,996 This is a pragmatic solution. 455 00:24:19,996 --> 00:24:23,096 And obviously, we cannot aim for total perfection. 456 00:24:23,096 --> 00:24:29,528 But let's say risk mitigation on a larger level, we do go a pragmatic way. 457 00:24:31,246 --> 00:24:35,296 What has changed over the time that you've been doing this digital ethics and AI 458 00:24:35,296 --> 00:24:35,766 ethics work? 459 00:24:35,766 --> 00:24:39,866 Obviously there's now a lot more adoption of AI, there's generative AI. 460 00:24:39,986 --> 00:24:43,974 How has that influenced the nature of the work you do or things that you've learned? 461 00:24:44,012 --> 00:24:49,952 So actually, five years ago, I took over the team of digital ethics or bioethics, 462 00:24:49,952 --> 00:24:50,332 actually. 463 00:24:50,332 --> 00:24:53,372 I built this up before it was supposed to side job of the chief medical officer and 464 00:24:53,372 --> 00:24:55,752 I said, okay, we need somebody skilled for this. 465 00:24:55,752 --> 00:24:58,652 And I saw this sort of like coming up. 466 00:24:58,652 --> 00:25:04,442 It was just me as a lone player back then and sort of pushed this little bit forward 467 00:25:04,442 --> 00:25:09,452 with the company, but it was seen as a rather, let's say, esoteric topic or 468 00:25:09,452 --> 00:25:13,676 something like data governance or data anthology that nobody wants to do. 469 00:25:13,676 --> 00:25:16,636 It's something of a side business. 470 00:25:16,636 --> 00:25:23,366 And I've really seen the, let's say, in the last year, I mean, the people that 471 00:25:23,366 --> 00:25:27,146 come to me and ask me questions has been risen exponentially within the company. 472 00:25:27,146 --> 00:25:28,216 I mean, this is Chagy B .T. 473 00:25:28,216 --> 00:25:32,056 finally showed people things are happening and actually I need to address these 474 00:25:32,056 --> 00:25:34,406 things and I want to have certain solutions and how can I have them? 475 00:25:34,406 --> 00:25:36,076 How can I have them responsibly? 476 00:25:36,076 --> 00:25:39,536 This obviously has also been a sort of like under the attention of the board. 477 00:25:39,536 --> 00:25:40,588 And we now have a... 478 00:25:40,588 --> 00:25:44,758 since a few years also a chief data and AI officer who's really taking care of this. 479 00:25:44,758 --> 00:25:46,668 And he's actually the chair of the panel. 480 00:25:46,668 --> 00:25:50,668 So he's a very close partner in crime with me. 481 00:25:50,668 --> 00:25:52,068 He's really pushing this forward. 482 00:25:52,068 --> 00:25:53,378 And we see a big momentum. 483 00:25:53,378 --> 00:25:58,398 We have a big data culture team within the company now that's really pushing this 484 00:25:58,398 --> 00:25:58,708 forward. 485 00:25:58,708 --> 00:26:00,168 We do a lot of awareness. 486 00:26:00,168 --> 00:26:03,378 And I mean, I can see that there's not only me pushing and standing in the 487 00:26:03,378 --> 00:26:06,308 corner, but there's also a big pull effect from different businesses. 488 00:26:06,308 --> 00:26:09,964 So I don't need to stand on everybody's door anymore and say, OK, I'm 489 00:26:09,964 --> 00:26:13,814 I'm very sure you have some questions for me, but people are coming running to me 490 00:26:13,814 --> 00:26:15,764 and ask questions. 491 00:26:15,764 --> 00:26:17,284 And it's also externally. 492 00:26:17,284 --> 00:26:23,884 I was just yesterday sitting on a panel also with somebody from Google Cloud 493 00:26:23,884 --> 00:26:26,594 Service, actually, from Munich. 494 00:26:26,594 --> 00:26:30,184 And she was also saying that all of a sudden since a year, this whole data 495 00:26:30,184 --> 00:26:33,004 governance, data ethics thing has moved out of the shadow. 496 00:26:33,004 --> 00:26:37,634 And she always says to people, this is unsexy in the beginning, and people need 497 00:26:37,634 --> 00:26:38,894 to get their basics right. 498 00:26:38,894 --> 00:26:39,500 And it's sort of like, 499 00:26:39,500 --> 00:26:41,540 eating your veggies every day. 500 00:26:41,620 --> 00:26:44,680 But this is something you have to do in order to be able to innovate fast. 501 00:26:44,680 --> 00:26:51,680 And this is also what our chief AI, data AI officer who worked with automotive 502 00:26:51,680 --> 00:26:55,660 sector before always says, I mean, he always comes to me and always say, so why 503 00:26:55,660 --> 00:26:57,860 do we have brakes in the car? 504 00:26:57,860 --> 00:27:02,640 So the brakes we have, so we can go safe, but fast. 505 00:27:02,640 --> 00:27:07,380 So it's like, okay, this what we have enables us to innovate really rapidly. 506 00:27:07,620 --> 00:27:09,246 But sort of like in a... 507 00:27:09,246 --> 00:27:11,872 safe matter so we actually can hit the break if needed. 508 00:27:13,102 --> 00:27:18,062 One of the issues with generative AI is these foundation models, which get 509 00:27:18,062 --> 00:27:22,662 developed by big tech companies, OpenAI, Google, and so forth. 510 00:27:22,662 --> 00:27:27,302 And then an organization like yours might be using that model and bringing in some 511 00:27:27,302 --> 00:27:31,102 of your own data and building queries and specialized applications, but you're not 512 00:27:31,102 --> 00:27:33,902 the ones who are controlling the original training of the model. 513 00:27:33,902 --> 00:27:38,042 Have you had to confront that issue, or how would you think about that in terms of 514 00:27:38,042 --> 00:27:39,174 your ethics work? 515 00:27:39,340 --> 00:27:44,050 For example, so we actually use an insourced model within the company that we 516 00:27:44,050 --> 00:27:48,680 have our own, we have our own GPT, my GPT that we have in the progression where we 517 00:27:48,680 --> 00:27:52,800 can actually, let's say it's cut off so we can put in our own data or let's say 518 00:27:52,800 --> 00:27:54,600 confidential data from the company. 519 00:27:54,880 --> 00:27:57,920 And, but obviously there are questions about, okay, how do we do this? 520 00:27:57,920 --> 00:28:02,090 And obviously the IP questions and the larger frame sort of like how it was been 521 00:28:02,090 --> 00:28:04,510 trained, it's becoming much more prominent now. 522 00:28:04,510 --> 00:28:06,540 So how do we, how do we deal with this? 523 00:28:06,540 --> 00:28:06,732 But. 524 00:28:06,732 --> 00:28:11,192 But one question that also came up in this whole thing that in our speak up line, 525 00:28:11,192 --> 00:28:15,212 actually somebody asked us, so you see, I'm a compliance speaker. 526 00:28:15,212 --> 00:28:21,142 We got the questions that, okay, there were those reports that the OpenAI had 527 00:28:21,142 --> 00:28:28,552 workers in Kenya, actually, that sort of like were exposed to violent materials or 528 00:28:28,552 --> 00:28:30,832 pornography materials and didn't get the right counseling. 529 00:28:30,832 --> 00:28:34,672 And so they did this for like $60 a day and they were really suffering a lot. 530 00:28:34,672 --> 00:28:35,628 And how can we... 531 00:28:35,628 --> 00:28:37,138 what kind of thing is this? 532 00:28:37,138 --> 00:28:41,468 And so we really looked into, okay, is this a technology we can use? 533 00:28:41,548 --> 00:28:45,748 Has this been developed under human rights violations? 534 00:28:45,748 --> 00:28:47,978 And obviously we did a survey, and we see it's not the case. 535 00:28:47,978 --> 00:28:51,608 And also these kinds of programs have been halted. 536 00:28:51,628 --> 00:28:55,568 And so OpenEye is no longer pursuing this, so that's why we can use this technology. 537 00:28:55,568 --> 00:28:59,288 So this is some of the things that are in our radar. 538 00:28:59,428 --> 00:29:02,748 And I think you can read about this, it was a time article, I think, some time ago 539 00:29:02,748 --> 00:29:03,532 on this. 540 00:29:03,532 --> 00:29:07,752 And it's something that we take seriously and survey. 541 00:29:07,752 --> 00:29:13,232 But obviously, then we try to also when we in -source solutions, especially now 542 00:29:13,232 --> 00:29:15,672 within HR is a big topic right now. 543 00:29:15,672 --> 00:29:20,472 And we actually have developed also ways to assess the vendor of a product. 544 00:29:20,472 --> 00:29:27,832 So in the final, sort of like when we have like two or three finalists in our vendor 545 00:29:27,832 --> 00:29:30,060 selection processes, actually we come now. 546 00:29:30,060 --> 00:29:34,800 for AI solutions with the certain kind of questions that we need to check also, not 547 00:29:34,800 --> 00:29:38,830 only sort of like how does the company behave, what kind of principles they have, 548 00:29:38,830 --> 00:29:43,620 but also how they address issues, how do they have like bodies, the digital ethics, 549 00:29:43,620 --> 00:29:48,520 so they have sort of like a crisis model for this or certain things, just do they 550 00:29:48,520 --> 00:29:50,620 take this topic as seriously as we do. 551 00:29:50,620 --> 00:29:54,560 So that's also a big thing because most AI solutions, I believe we will insert, I 552 00:29:54,560 --> 00:29:58,828 mean, we will use the few skilled people on this we have, maybe like 200 or so. 553 00:29:58,828 --> 00:30:02,668 to develop solutions for research, clinical research, prediction tools, and 554 00:30:02,668 --> 00:30:03,448 these kind of things. 555 00:30:03,448 --> 00:30:06,628 We're not going to use our in -house experts to develop HR tools. 556 00:30:06,628 --> 00:30:10,028 Obviously, these will be mostly in -source and maybe adapt. 557 00:30:10,028 --> 00:30:13,148 And here we need to find a good way of sort of like, OK, how can we ensure that 558 00:30:13,148 --> 00:30:17,462 this aligns with our principles and general vision for AI? 559 00:30:17,870 --> 00:30:24,040 You mentioned earlier on the European AI Act, European Union AI Act, as a company 560 00:30:24,040 --> 00:30:29,510 that was doing this digital and AI ethics work for a long time before that 561 00:30:29,510 --> 00:30:30,790 regulation was adopted. 562 00:30:30,790 --> 00:30:35,710 How do you anticipate that influencing and impacting your work? 563 00:30:36,812 --> 00:30:43,162 So this was also a case where our work was suddenly highlighted and was seen as very 564 00:30:43,162 --> 00:30:43,972 valuable. 565 00:30:43,972 --> 00:30:50,962 A few weeks ago, the consultancy highlighted the survey of sort of like a 566 00:30:50,962 --> 00:30:53,202 UAI readiness within the company. 567 00:30:53,202 --> 00:30:56,782 And we saw that we have very strong foundation, thanks to all the work that my 568 00:30:56,782 --> 00:30:59,232 team has been doing in the last five years. 569 00:30:59,232 --> 00:31:00,772 So that's a good head start. 570 00:31:00,772 --> 00:31:02,992 I mean, it's also good for us to see, OK, we're doing something this. 571 00:31:02,992 --> 00:31:05,036 Because the whole thing was for. 572 00:31:05,036 --> 00:31:08,776 as I said, how do we behave rightly in the absence of regulation? 573 00:31:09,016 --> 00:31:12,296 So we had the sort of like future regulation probably upcoming already a 574 00:31:12,296 --> 00:31:15,356 little bit in mind when we design certain things. 575 00:31:15,456 --> 00:31:18,666 And so the influence on us is not going to be that strong. 576 00:31:18,666 --> 00:31:23,286 I think it's a little bit more about building sort of like one of the things 577 00:31:23,286 --> 00:31:27,716 that we need for now is for to identify high risk cases and really document them 578 00:31:27,716 --> 00:31:30,386 in the right database. 579 00:31:30,386 --> 00:31:31,116 And so I have a look at this. 580 00:31:31,116 --> 00:31:32,716 So this is going to be a little bit of more. 581 00:31:32,716 --> 00:31:36,906 building a broader AI governance that we do not have in that strict way right away, 582 00:31:36,906 --> 00:31:38,756 because right now it's more decentralized. 583 00:31:38,756 --> 00:31:40,396 So to really enable innovation. 584 00:31:40,396 --> 00:31:46,316 So we will move to a little bit more centralized AI governance that also gives 585 00:31:46,316 --> 00:31:49,986 like a consolidated mandatory review, I guess, for high risk cases. 586 00:31:49,986 --> 00:31:53,406 So it was no longer voluntary, as in my case, but then there were probably like a 587 00:31:53,406 --> 00:31:59,196 legal ethics and whatever other body review of certain projects and areas that 588 00:31:59,196 --> 00:32:02,804 we identify really in response to this. 589 00:32:02,804 --> 00:32:03,324 regulation. 590 00:32:03,324 --> 00:32:04,744 So it may slow down a bit in things. 591 00:32:04,744 --> 00:32:06,704 And let's say it's not a perfect regulation. 592 00:32:07,424 --> 00:32:12,414 But I mean, I obviously have two hearts, one of them in the company, I want to be a 593 00:32:12,414 --> 00:32:14,944 bit more free, but I'm also a citizen. 594 00:32:15,224 --> 00:32:19,284 And sort of like I as a citizen, I really much appreciate that the European Union is 595 00:32:19,284 --> 00:32:20,354 moving forward with this. 596 00:32:20,354 --> 00:32:24,264 It's not an ideal framework, but I think it's still better than having no framework 597 00:32:24,264 --> 00:32:25,064 at all. 598 00:32:25,064 --> 00:32:27,814 And you will see how this plays out. 599 00:32:27,814 --> 00:32:31,484 But I think it's also going to be a little bit of a role similar to the GDPR, which 600 00:32:31,484 --> 00:32:32,652 we have pushed forward. 601 00:32:32,652 --> 00:32:35,412 which is lived very differently in European countries. 602 00:32:35,412 --> 00:32:38,992 So in Estonia, it's lived very differently and much more openly than in Germany. 603 00:32:38,992 --> 00:32:42,052 So I assume it's similar going to be with the AI. 604 00:32:42,052 --> 00:32:44,072 But I think it's a good thing moving forward with this. 605 00:32:44,072 --> 00:32:50,102 And I think also other world regions will look at us pushing this forward and how 606 00:32:50,102 --> 00:32:51,942 have we done this and what are some of the key elements. 607 00:32:51,942 --> 00:32:54,432 And obviously, everybody wants access to this market. 608 00:32:54,552 --> 00:32:57,632 So let's see how this plays out. 609 00:32:58,272 --> 00:33:02,654 And also how the documentation requirements in the end really are. 610 00:33:02,654 --> 00:33:06,744 For us as a larger company, obviously it's easier than maybe for like a hundred 611 00:33:06,744 --> 00:33:07,824 people player. 612 00:33:09,102 --> 00:33:13,902 Shona, thank you for a really enlightening and instructive conversation. 613 00:33:14,380 --> 00:33:14,900 Thank you. 614 00:33:14,900 --> 00:33:18,370 I mean, for all those who see this, I mean, don't hesitate to reach out to me 615 00:33:18,370 --> 00:33:19,380 through LinkedIn. 616 00:33:19,380 --> 00:33:22,360 Learn more about how our company is doing this. 617 00:33:22,360 --> 00:33:23,930 And we're willing to share. 618 00:33:23,930 --> 00:33:27,820 We actually do a lot of cross -company initiatives also to share best practices, 619 00:33:27,820 --> 00:33:35,500 pushing things forward, educating, yeah, really policymakers, also exchange with 620 00:33:35,500 --> 00:33:37,060 academics that do case studies on us. 621 00:33:37,060 --> 00:33:41,160 So feel free to reach out to me or my team members and discuss. 622 00:33:42,062 --> 00:33:43,222 Wonderful.