Azeem, welcome to the podcast.
Kevin, it's great to see you again.
I think it's getting into our 25th year of knowing each other.
This is very frightening and you've done a lot of different things, worked for
various companies, started various companies.
How do you explain to your parents what you do now?
Yeah, well, I, they're quite excited because, or my mum is now, my dad passed
away last year, but he was even excited because I spent my time researching ideas
on my own with my, well, with my team.
And I have a couple of academic affiliations, research them, thinking
about the intersection between this rapid technology change, things like artificial
intelligence and societal change from the economy to our legal institutions and
systems of government governance.
And then I bundled that up.
produce a newsletter from it, write the odd book, help the odd investor or
company, maybe with a speech here or there.
So I'm, I guess, a one-man band, but I do have an amazing team of researchers who
work with me.
Yeah, it's one of those jobs that probably didn't exist before, but now you can be a
real force multiplier, which is wonderful.
So let me ask you, when did AI first come on your radar screen as a technology you
should focus on?
Obviously, AI has been around for a very long time, but at what point did you
notice that there was something really important going on?
You know, I actually, I mean, my first AI deal was done in 2000 to 2001.
So yes, it's been around for a long time, but then there was this additional shift
because of deep learning.
I first came across it in about 2012 or 2013 when someone in my team, my head of
machine learning, wants to take a job at some funky London startup called
Deep mind.
I don't know if you've heard of them and I persuaded him not to do that.
And, and then when we sold my company, I had some time to think, and then I, I dug
into things and I saw something was happening and that was about 2014.
So about a decade ago.
What's changed in terms of either fundamentally AI or how we think about AI
today?
you know, the technology has really, really matured.
And what happens in the process of maturation is things that become so hard,
that only the very, very best researchers in the best research institutions can make
sense of them.
And then over the course of four or five years, they moved to being available as,
you know, an executive ed class for a couple of thousand dollars at a business
school, right?
And that is...
That is how a technology deploys into the economy.
So a lot of those things have happened with deep learning based technologies and
traditional machine learning systems like classification and discrimination and so
on.
But the other thing that has happened of course is that the waves of discovery have
continued.
Alongside that, the underlying infrastructure, the enabling technologies
have become
much, much more capable.
And that means that for a business, they need to have less expertise than perhaps
they did a decade ago to get into these technologies.
And they're supported of course, by the systems integrators, the consultants who
are the ones who really drive enterprise IT spending.
And so what we see, which I think is pretty fascinating is not just, you know,
maturation,
of the technology that makes it more accessible to people, but also this
continued drumbeat of absolute cutting edge breakthrough innovation to the point
that, you know, you and I have been in the tech industry for a while.
It is very much very sort of moments of Lenin, you know, decades are happening in
weeks, sometimes in the same day.
I mean, you finished reading one tweet and something else enormous has happened while
you're reading those 200 odd characters.
But people in the tech world always like to say that this is the transformative
time.
This is the destructive moment.
This is the inflection point.
Are we really at one of those points?
What is it that makes you think that now it's really true?
We always say that because it's self-serving.
And what's different, I think, this time is the, you know, let's look at a
corporation.
If you've got a, the C-suite, the C-suite are people who are likely in their mid-50s
through to their early 60s, which means most of them have had careers using
computers.
And they've had computers on their desks and they know how to use Excel.
They probably did some time doing PowerPoint.
And so they're much more computationally savvy than C-level execs, even when the
iPhone came out.
And they've had 20 or 30 years to digitize their businesses, which is what we've been
telling them to do since forever.
We're out of that moment of J-curve that you see when you have to install a new,
implementing technology.
So what that means is that companies are ready to deploy this.
But let's then talk about the technology.
So what happened with large language models was we turned computers into that
scene.
I think it's Star Trek IV when they're trying to move the whales.
And Scotty picks up the mouse and says, computer, design me a da-da-da-da-da.
We finally got there.
We can talk language to computers, and computers talk language back to us.
And since humans essentially have
construct their entire world through the passing of messages in the form of
language, that has allowed us to bring the capability of computers into those
moments.
And one of the things that is so beautiful about using something like ChatGPT is that
in the old days, so November 2022,
If you wanted to do a particular type of analysis, you had to configure that
analysis into a way that a computer understood.
So you took your mental model of the world.
And that's why some people are good at programming and others are less good,
because those people who can convert this sort of squishy human world into, you
know, flawless logical code, do so and they do that translation.
But now with ChatGPT, I can say, hey, listen, I kind of want you to do a MISI
analysis.
on this list of issues, do it for me.
And it comes back and it presents as well as a grad student, which is, I think is
pretty impressive.
So I think those two things lend us to think that there are the ingredients for
an inflection.
So my main focus in this podcast is on responsibility and accountability for AI.
Let me just first ask you, generally, how do you think we're doing in those
dimensions?
Can we mention that this is the recording after the Google Gemini fracas, which
has...
Okay, so we'll get to that.
I think we are doing...
The awareness of the issue is much, much higher than it was seven or eight years
ago.
I mean, I remember in 2015, writing about all these issues with Google images and ML
systems that couldn't recognize dark skin.
There were no teams, there was no discipline, there was not much academic
study.
We didn't do the kind of evaluations either a priori or ex ante, pardon me, or
ex post of the systems.
I think that has changed over the last seven or eight years.
That's worth paying attention to.
In 2014, the reason these things weren't...
done was because people didn't know how to do them.
No one really even thought to do them.
And so the systems behaved the way they did.
Now, after a lot of work by civil society, by academia, by activists, we do think
about these things.
And then the question is, of course, it's often embedded into sort of the belief
systems and the narratives in the world at the right time, right?
And just so, so I think that some of the things we've seen have been
whatever the rung above the ladder, above the, the rung on the ladder, above the
idea of performativity is, we're getting to that stage with some of this where
people beat the responsibility drum really, really hard, perhaps harder than
it needs to be beaten.
I think Sundar Pichai at Google is somebody who does that a little bit more
than perhaps he needs to.
Yeah, so let's talk about this example with Google Gemini, where for those who
aren't familiar, they built an image generator model and a large language
model.
And addressing exactly those issues you're talking about, concerns about bias, went
too far.
That in some cases, the image model would produce literally non-historically
accurate pictures of people.
The text model would just refuse to answer totally anodyne questions, saying they're
too controversial.
So.
People have different reactions to this.
What's your takeaway or main takeaways from that incident?
Well, it's complicated.
I mean, I think the absurdity of some of the outputs of Gemini can't be denied,
right?
So the idea that you might say, give me a picture of soldiers, German soldiers from
World War II, and it doesn't come back with them looking at all how they should.
Or one I saw on Twitter recently, which said,
who caused more deaths, Elon Musk's tweets or Adolf Hitler.
And Gemini sort of prevaricates and ums and ahs and says it's an ambiguous
question, which is just sheer absurdity.
So I think that it highlights two issues.
The first issue is that large language models are very difficult to control.
They are not like machines that we have built in the past.
And so in the moment of
trying to control them, you are choosing between helpful and harmless.
And there seems to be right now on the methods that are used by these companies,
really a trade off between the two.
But it's a bit more complicated than that because you don't exactly know how much
helpfulness you have to give up in order to make it harmless.
And in a situation like that, I think you will always, I mean, use a British phrase
here, over egg it, right?
You'll always err on the side of caution.
in order to avoid the harmful nature of it.
And then you sort of end up slightly in this absurdity.
But I think there's a second issue, which is that, you know, we have to recognize
where we, that the world in which we currently live, not the one we want to
live in, the world we live in is mediated by a small number of large technology
companies.
And that means that the AI they build will become our interface between us and the
real world and the resources that we want to.
access.
And so somebody has gone off and stuck this resource in between me and the world.
I mean, you have to go to Gemini today, but in two years time, everything you do
on Google will be filtered through it.
And they've gone off and made certain decisions.
It's harmless, helpless trade-off, other decisions in particular about what they
consider harm.
And to me, this is a public infrastructure.
This is a public infrastructure.
It's a utility.
Maybe it's a type of good, public good.
And Google did talk a little bit about what went wrong, but not in the degree of
sufficient transparency of this new role they have of guarding something that is
not purely a private asset.
So I think that there are these two issues that we have to have to talk about and
make sense of.
Okay, no, that's a good point.
And that's, I want to talk about the issue about the power of the intermediaries and
the big tech companies.
But let me just first push you a little bit more on the question about the
capability of controlling the models.
Your book is called exponential age.
And you talked about how technologies like AI and rootables are on these exponential
growth curves.
Is there a concern that the improvements in the technology are an exponential curve
but our ability to control them?
are something like arithmetic, and I'm not just talking about here in terms of the
broad kind of alignment that the AI doesn't kill us all when it becomes super
intelligent, but even on what seems like a fairly mundane problem, like not having
biased image generation.
Is that potentially an unsolvable problem?
So I think that this isn't a case of the type that I describe in my book.
I mean, the ones I don't describe in my book are really about how the state and
the institutions of the state and the people who comprise that are able to
respond to this pace of change.
I think what we're seeing in Google Gemini is much, much closer to...
what happened in the early years of jet powered passenger planes.
So these emerged in the early 1950s.
And the first of one of the first of those was made by a British company.
We don't have names like this for jet aircraft companies anymore.
De Havilland.
It's so elegant.
It's straight out of Downton Abbey.
The De Havilland Comet.
And I think something in the order of 40 of these crashed.
I mean, they make Boeing look like, you know, an absolutely
pristine whiter than whites safety record.
And about 40 of these crashed, though 20 of them had fatalities and about 500
people died when these planes kept falling out of the sky.
And the reason was that they had square windows and the corners of the windows at
jet speeds in the thinner atmosphere heated up much more, created lots more
metal fatigue than they'd ever experienced in propeller planes that were a lower
height.
traveling more slowly.
And they couldn't do, you know, they didn't do have computational fluid
dynamics, right, to simulate all these things.
And, and, and we figured that out.
And yes, 500 people died and air travel is now so much safer.
Now, I think that we are that's partly what we're dealing with within the world
of the large language models, right, which is that these are these will be made safe
in some way, they will become progressively safer.
Now, not everyone agrees with me, there's a
an academic called Gary Marcus, who I think would take a different view to that
and say that's kind of really hard, if not impossible, to make them safe and they're
poorly designed.
But I think they can be made safe and we will learn how to do it.
Now, the challenge is that Google is in a race.
It's in a race that is both strategic, that in some sense is almost existential,
that is commercial, that is also about...
corporate and personal pride, which is driving them really fast.
And we have constructed a world where we are looking for some hype to drink from as
consumers of the media.
And so this pushes this process of actually figuring out how to make these
things safe out into the limelight much, much faster.
And so in some ways, I'm a little bit sanguine about the fact that this...
happened.
I'm kind of annoyed.
I feel it should have been caught in testing.
But I think it is a case of it's this technology is, you know, it's seven years
now since the first academic paper.
It's, it's, it's really early.
And to be honest, honestly, no harm was done, except for Google.
Google is embarrassed.
It shut the thing off.
It'll figure out a fix.
We won't see it.
Is anyone really missing having an AI based image generator for an extra few
days?
None of us are.
But won't that competitive dynamic you talk about continue?
Because the company that has succeeded the most so far is OpenAI, partnered with
Microsoft, which took the kind of Mark Zuckerberg, move fast and break things
approach.
While Google waited to deploy its LLM because it was concerned about all of
these issues, if that's always going to be the case, isn't there going to be
constantly commercial pressure to push the envelope and not even to take the time to
develop the solutions that are possible?
That's business.
I mean, when we read the stories of what business people did over the last 100, 150
years, I mean, they make trade-offs and they have to apply their judgment because
nothing is perfect, no one has perfect resources.
They're always competing needs.
There's also always ego and selfishness and profit motive in it.
It's just, it's happening in...
a in a public space, which, of course, the business people behind them, and that's
what they are, let's not call them founders, let's just call them business
people, they're just like the green grocer down the road.
They benefit from the publicity that it's that is being created by all of this, and
they have to make difficult choices.
I'm
I would say, having spoken to him a few times, but not knowing him because he's
very hard to read, Sam Altman, I would say that Sam is extremely, probably very keen
to make all his products factually accurate and generally pretty safe, not as
safe as a nervous Nick might want, but pretty safe.
And he's just waiting for them to develop.
those capabilities.
And he at the same time, at that moment, he wants to go out and win market share.
And what's happening is it's happening in this public domain, with all the noise by
the way, Kevin of this idea that these LLMs will crawl out of our fridge, like if
you've seen the film Gremlins, and you feed them and give them water, and they
turn really nasty, you know, like that and take over our world, which again, I don't
think they will.
So I think we're in this funny stage where we should just kind of look at this and
calm down a little bit and say,
And in that calmness, it can still be a transformative technology.
I mean, that can all be true at the same time.
One of the important things about this wave of technology is that we have these
big platforms that are building the AI systems.
But every company on Earth probably is going to use them in one way or another
with some amount of either their own technology or just taking things off the
shelf and deploying them.
You talk to executives of a lot of companies about AI.
How are they thinking about these questions of responsibility and accuracy?
And what does it take to put them
persuade them to make the right trade-offs.
I think in general, they take it very seriously.
And it comes up surprisingly unprompted in the conversations that I have.
And people talk about their responsible AI teams.
Or they'll say, well, we have to roll this out responsibly.
I mean, there are some variations.
And for some, it's a.
responsibility with a bit of a wave of a hand, say, don't talk to me about it with
others, it's more, more engaged.
But I think that comes back to a little bit earlier in our discussion, where I
said that the way things have moved over the last seven or eight years, is that
companies are taking this quite, quite seriously.
And perhaps, you know, the challenge is that how much of that is, is simply just
lip service.
and how much of it is really, really deeply embedded in what companies try to
do.
And of course, the thing is, Kevin, how much do we want our companies to do?
The founder of, and suddenly the name escapes me, the outdoor wear company,
Patagonia, perhaps, is that?
Who was sort of famous, he says, business will...
you know, business will always leave a footprint, right?
And we can't pretend it won't.
And our job is to try to make it as small as we can.
And I don't think it's necessarily a company's job to act responsibly or act
ethically.
That is the job of, I mean, it's the job of the law and whatever is delegated to
regulators to say, these are the parameters within which you should act.
Now you should act in that way and comply to this law.
Otherwise we're going to come in and find those breaches and we are going to enforce
actions against you.
So yes, I'm excited that CEOs care about it and they're not spending their time
trying to figure out how to kind of wind their way out of their obligations.
But I think it's a bigger issue that those obligations are not clear.
They're not clearly defined.
And so that the companies really know what the rules of the game are.
Yeah, that gets back to the point that you made earlier and you talk about in your
book.
Is there going to be enough capability in government to make those rules
appropriate?
Well, I mean, the government's getting better.
It's definitely getting a lot better.
And we, I think we tend to.
have a slightly sneery view of government, but we can go back to the first consumer
privacy legislation which came out in the late, early 1970s in California on the
basis of the arrival of computer databases.
I mean, that is phenomenally far-sighted.
I mean, if that person had gone off and said, computer database is gonna be huge,
I'm gonna build a company, they would be, you know, yachts are plenty by now.
And in other areas, governments have been
pretty impressive.
I'll give you another example.
My house was built in 1925.
The cable that carries the electricity is rated at 100 amps and is still more than
my house uses a century later.
We've got two electric cars and we've got all the electronics you might imagine.
genetic modifications in humans, right, was well ahead of the technology.
But we have, governments, I think, started to slow down for historical reasons.
And in my book, I talk about Reagan and the air traffic controllers and so on.
But of course, that process of deregulation had started under Carter.
And if you come at the back end of 25 or 30 years where
The notion of good in the context, the intersection between government and
business is about pulling regulations back, as opposed to recognizing that it is
a presence of clear regulations, clear enforcement, a mandate for compliance that
has allowed the pharmaceutical industry and the financial services industry to
grow as big as they have.
Of course, we're always going to feel that government is going to not have that
capability.
Now, I do think that they have really...
moved and I can speak about the UK, I mean the quality of people in policy, that is
the civil service who will brief government on these issues is really deep.
Now I just don't know if there are enough of them, but they're getting really quite
good and I wouldn't sort of besmirch them in any way.
Our government on the other hand, maybe that's for an offline discussion.
enough.
But with regard to AI, different governments are taking different
approaches.
And so you mentioned the UK.
It seems that actually this is one set of issues where Brexit has actually made a
pretty dramatic difference.
On a lot of things, there's not a great deal of substantive difference between
what's happening, you know, the laws in the UK and Europe.
But the European Union has adopted is in the final stages of this very extensive,
very prescriptive AI act.
UK government, at least from my perspective, more like the US, has taken
more of an approach of saying, let's look at what existing laws out there, let's try
and figure out and have different kinds of rules and how to balance innovation with
regulation.
Seems like a very different approach.
So first of all, I guess, do you see that?
And then what do you think is the best approach for governments to take?
I definitely do see that.
And I certainly see that gap emerge.
And there was something else that was about the sense of victory that was coming
out of the various parts of the EU.
And we haven't created a $100 billion company since SAP, and now Novo Nordisk,
which has got rich on Americans.
having an obesity problem, right?
I mean, because they make semaglutide, the GLP-1 agonist.
And so it's a really odd thing to celebrate.
The other thing is it's a really big act.
And so I think that means that in a way, some parts of it may have been hurried,
especially the bits that relate to the...
large language models and those uses.
But in reality, that has to now go through processes which will detail how things
will get enforced.
So we know that really bad laws can be created.
I think, for example, many people probably view the Patriot Act hastily put in place
in the US after 9-11 as something that has resulted in some overreach and might need
some pairing back.
But I think that the issue that I have with the EU Act is less about the
specifics of the detail, but it was more about the posture of the participants
towards this remarkable technology and what was really embedded within that,
which was, I think, a bit of a distrust with American firms.
but there wasn't within it a sense of, we'll create an alternative.
And so then the question is what actually happens over this period of time.
So I think in these moments of really, really rapid change, you need to have
principles-based approaches to regulation, right, rather than hard and fast rules.
So I would say that that's probably more a British style than...
you know, an American approach and you need that because the underlyings change
so frequently.
So I think that feels like it's sensible.
But the other thing that I think will end up happening is that there will be an
increasing harmonization of the ways in which nations look at certainly economic
and industrial applications of
of AI.
And the reason I think that they'll start to harmonize will be because we're in a
search process of what method works best.
Every country will benefit if technology companies can access their markets on
equal terms.
So that will also start to drive some degree of consistency across the piece.
Where I think it may differ
will be, and this is probably going to be quite a large part of AI to be honest,
aspects that connect to culture, identity, or potentially national security issues,
which was also the first sticking point, of course, with the internet, where
countries sort of started to cleave themselves away.
But I think what we saw with privacy was that there was a lot of harmonization.
So basically every global company is complying with the European GDPR either
because they have operations in Europe or because there's just no point in building
two completely different systems.
But I don't think anyone, even the people in the EU, in the European Commission,
would say things are great on privacy.
The practices have tremendously improved because we have this agreement on this
comprehensive set of standards.
Mm.
How do we get to a point where the actual substance improves, which, as you noted,
companies, they want to hear government telling them what to do, but they have a
role in helping governments understand what the capabilities are and what could
happen.
You know, I think a really great example of that was what happened with Apple and
the App Store.
So under the Digital Services Act, no, the DMA, the Digital Markets Act regulation,
because Apple operates this App Store that is sort of so big and so dominant, it has
to provide alternative means of payment and alternative App Stores.
So if you're in the EU and you have an...
and I felt this is going to be all to American listeners, you can kind of get
apps from a different store run by someone else and you don't have to pay through
Apple.
And of course, Apple has gone off and complied with this and involved something
like writing 500 new APIs, which are kind of quite complex ways in which you use
computer services over the internet.
And they introduced new pricing mechanisms because what they've done is they've
absolutely complied to the letter of the law.
as you'd expect because they can afford good lawyers, and people are arguing they
haven't complied to the spirit of the law.
And this is the old problem of the sorcerer's apprentice, right?
The Goethe, the old Goethe short story, right?
Where we've got this technology, we try to control it, we give it rules, and it
follows into the letter.
And so I think that raises issues about when and how is the right time to...
to write these rules and how do you get people sitting around the table at the
same time?
Now, I was actually quite a supporter of the Digital Markets Act because my framing
is that there is a tendency towards dominance in digital markets for reasons
that I initially probably Kevin learnt from you 25 years ago.
And that force of gravity continues.
And because there's this tendency to dominance, you will have concentration of
power.
So you need to find mechanisms to be more onerous with bigger firms than smaller
ones, because the bigger ones are so much bigger.
So I was at a high level, I was a supporter of it.
I think that the...
issue is, and there's another analyst called Benedict Evans, who I think
probably made a better call on this than I did, where he said, well, the problem is
that even if Apple is dominant, the process is safe and it's easy on the App
Store.
Consumers love it.
No consumer is going to go off the App Store to the dodgy flea market with men in
gray coats of saying, want to buy an app, when they can safely buy an app from the
App Store.
I think has a lot of momentum behind it as an idea.
And that I think brings to bear the fundamental issues of capability within
government and who did you have the right sorts of people who really were able to
have the right kind of dialogue to get around the problems of corporate capture,
which is what we saw with GDPR and have that conversation.
And I don't think we did.
So.
So, I mean, that's a very long answer to say, it's quite hard.
I'm not sure it's necessarily gonna serve European customers particularly well.
I think what's hidden slightly below this is that these are all American companies.
I mean, the rules are not written to discriminate against American companies,
but it just so happens they're all American firms and will...
continue to be if what we do is celebrate passing laws rather than building
companies.
One more question.
You're pretty optimistic, it sounds like, both about the path of the technology, the
business option, and working through these regulatory issues.
What do you worry about?
Where might you be wrong about any of these aspects that you think about?
That's something that might put us on a bad path.
You know that there are really interesting things that have happened that I wouldn't
have predicted, like that a trained scientist running a major nation would
shut down the nuclear program, as Angela Merkel did.
I could run that tape a hundred times.
I would always say she's not going to do it, and she did.
So I guess I get...
slightly concerned about the growing dominance of these firms.
I mean, less so Nvidia.
I mean, they're going off and making a lot of money because they're so far away from
the customer.
But the power and the capability that these companies will increasingly have,
and then what that means in terms of their ability to
control what's left of the public square and to do so in ways that aren't really in
the interests of the people who live there.
And I think that we've seen some aspects of that play out with the way targeted
advertising works in the US and Facebook over not necessarily the last couple of
years, but the years before that.
I'm always a very simple man.
And so I simply go back to...
ideas of concentration of power, where, and wherever I see that processes and
mechanisms that can reinforce concentration of power, it makes me
nervous because I can't believe that if you are that powerful, you aren't somehow
skewing the rules or taking a kind of shifty vig on the side that we don't know
about.
So where I could be wrong is that we don't put in things that slow down this
agglomeration.
And the other workplace I could be really wrong is whether it's even possible to
slow it down.
So some days I wake up and I think, I'm just not sure it's possible to slow it
down.
It doesn't matter that Google and Apple are a bit scared of Lena Khan at the
moment and aren't buying companies.
I don't think it's gonna slow down how quickly they grow.
And I could also be wrong about that.
What I'm not worried about so much is I'm not worried about AI accelerating and
getting out of control.
And the reason I'm not is because I think we technologically co-evolve systems to
keep it safe, even if we can't co-evolve the regulatory systems as quickly.
But yeah, but that's really...
That's really about it.
And of course, Kevin, I'm worried about the elections in the US and elsewhere.
And I'm worried about, you know, what might happen with China and what might
happen with Russia.
I mean, those things are just background noise for 2024, really.
They're the track to which we're going to be dancing.
Absolutely.
No, a lot out there to worry about a lot more that we could certainly talk about,
but we're out of time at this point.
Azim Azhar, thank you so much.
Always great to hear from you and appreciate your sharing your perspectives
with the road to accountable AI.
That's been wonderful, Kevin.
Thank you so much.