1
00:00:01,025 --> 00:00:01,664
Tim, welcome.

2
00:00:01,664 --> 00:00:03,238
Always great to talk to you.

3
00:00:03,353 --> 00:00:05,259
Great talk with you too, Kevin.

4
00:00:06,188 --> 00:00:07,590
What can I do for you?

5
00:00:07,590 --> 00:00:10,375
emerging technology for a long time.

6
00:00:10,437 --> 00:00:13,103
Let me just first ask you, how do you think about AI?

7
00:00:13,103 --> 00:00:15,528
What kind of innovation is it?

8
00:00:15,528 --> 00:00:17,311
How significant do you think it is?

9
00:00:17,866 --> 00:00:27,655
Well, I would start by putting all sort of great waves of technology innovation into one
frame, computer technology innovation.

10
00:00:27,655 --> 00:00:36,352
And that is the increase in the ease with which humans are able to communicate with
computers and get them to do our bidding.

11
00:00:36,833 --> 00:00:38,054
It's as simple as that.

12
00:00:38,054 --> 00:00:45,300
You think back to ENIAC where they programmed it by making physical circuits.

13
00:00:45,460 --> 00:00:50,414
And then we got to store program computers, but you put it in the program one bit at a
time.

14
00:00:50,414 --> 00:00:57,310
And they got the switches on the front and then, you know, punch cards and, you know,
Magtape for storing programs.

15
00:00:57,310 --> 00:01:02,294
And there was this sort of priesthood who could do these very difficult things.

16
00:01:02,755 --> 00:01:04,886
And then you have this big breakthrough.

17
00:01:04,886 --> 00:01:14,444
Well, first of all, you had a breakthrough with assembly language and then higher level
languages that output machine instructions without people needing to write them directly.

18
00:01:14,558 --> 00:01:19,422
And then you got interpreted languages that made it really easy.

19
00:01:19,422 --> 00:01:23,675
And then you had things like the PC, which made it possible for anyone to have a computer.

20
00:01:23,675 --> 00:01:28,038
And suddenly there were millions of people doing these things, writing programs.

21
00:01:28,179 --> 00:01:30,161
And that exploded.

22
00:01:30,161 --> 00:01:39,532
And then you really got something really new, I think, with the web, which was where you
could actually create documents that called programs.

23
00:01:39,532 --> 00:01:47,192
And so that was like, could basically make an interface out of a document that was the
kind of thing that you know, the humans exchange with each other.

24
00:01:47,192 --> 00:01:55,532
And people don't recognize that what a big step forward in interfaces that was in some
ways it was even bigger than the GUI, you know, which are the graphical user interface,

25
00:01:55,532 --> 00:01:58,192
which is of course made things simpler.

26
00:01:58,492 --> 00:02:05,232
And so now we're at a place where we can just talk to these things in plain language and
they can start to do what we want.

27
00:02:05,232 --> 00:02:09,556
And if you look at the history, every time we've had one of these advances,

28
00:02:09,556 --> 00:02:13,118
More people could use computers and they could do more things with them.

29
00:02:13,419 --> 00:02:15,000
And so it's profound.

30
00:02:15,000 --> 00:02:16,401
It really is profound.

31
00:02:16,401 --> 00:02:18,642
It's basically going to grow the market.

32
00:02:18,642 --> 00:02:24,766
And I just mean in the number of people who can use it, but in the number of things that
they can do.

33
00:02:25,487 --> 00:02:30,030
And again, you can look at other technologies.

34
00:02:30,030 --> 00:02:37,795
You read some of the early history of the automobile and it's like if you want to be an
automobile owner, you had to be a mechanic.

35
00:02:39,094 --> 00:02:39,754
Mm-hmm.

36
00:02:39,754 --> 00:02:45,476
now you're at a point where you can't even be a mechanic because they've got that's going
too far, maybe.

37
00:02:45,476 --> 00:02:50,676
you know, there are certainly some interesting parallels.

38
00:02:50,923 --> 00:02:51,484
Yeah.

39
00:02:51,484 --> 00:03:00,121
Well, so what is there anything we can learn from at least in computer technology, those,
those prior shifts that can help us in engaging with the issues that are coming up now

40
00:03:00,121 --> 00:03:01,064
with AI.

41
00:03:01,500 --> 00:03:02,821
absolutely.

42
00:03:02,861 --> 00:03:09,945
First off, know, people underestimate them and then they overestimate them.

43
00:03:10,986 --> 00:03:26,834
And and in some ways, the you know, this is sort of thermostatic kind of process, just
like in politics, I guess, in which, you know, people try to have this grand narrative and

44
00:03:26,834 --> 00:03:30,596
and the world sort of stumbles forward in

45
00:03:31,211 --> 00:03:32,912
unexpected ways.

46
00:03:33,632 --> 00:03:41,942
think if you look at, example, again, I think back particularly on my early career was
very shaped around the World Wide Web.

47
00:03:41,942 --> 00:03:48,052
I mean, it was there before that, an open source software, people got a lot of things
wrong.

48
00:03:49,092 --> 00:03:55,588
My whole battle with the, I convened the meeting where the term open source software was
adopted, but I...

49
00:03:55,666 --> 00:03:59,129
was kind of an outlier and everybody was focused on licenses.

50
00:03:59,129 --> 00:04:01,790
And I said, I don't think licenses are the issue.

51
00:04:01,790 --> 00:04:05,863
It's actually, we have network enabled collaboration.

52
00:04:05,863 --> 00:04:19,333
We have what I called the architecture of participation, the design of systems that people
could make small pieces of and they work together, sort of open architectures and kind of

53
00:04:19,333 --> 00:04:20,354
look at the difference.

54
00:04:20,354 --> 00:04:23,648
There's this great line I remember seeing on the internet early on.

55
00:04:23,648 --> 00:04:28,270
The difference between theory and practice is always greater in practice than it is in
theory.

56
00:04:28,410 --> 00:04:30,971
And I don't know who said it, but it's brilliant.

57
00:04:30,992 --> 00:04:37,015
And because always you have people who have this sort of theoretical construct, you know,
the same thing with hypertext.

58
00:04:37,015 --> 00:04:45,679
You know, it's like when the World Wide Web came out, you know, all the pundits said the
hypertext pundits said this won't work because it only has one way links.

59
00:04:45,679 --> 00:04:46,759
It doesn't have two way links.

60
00:04:46,759 --> 00:04:48,340
It has to have two way links.

61
00:04:48,364 --> 00:04:56,484
And of course the one way link thing was part of what made it grow so explosively that it
didn't, you could just get a 404.

62
00:04:58,264 --> 00:05:03,364
And everything wasn't tight and tightly bound.

63
00:05:03,804 --> 00:05:17,162
so I think the thing we learn is that, you we stumble forward and there'll be some
innovation and then somebody else will build on that innovation.

64
00:05:17,162 --> 00:05:26,376
And I guess this goes back, I think it's interesting because Ethan Mollick, who's one of
my favorite observers of AI, has become a fan like I have of James Besson's work on the

65
00:05:26,376 --> 00:05:28,017
Industrial Revolution.

66
00:05:28,297 --> 00:05:33,760
You know, where he basically said, why does it take so long for new innovations to spread?

67
00:05:33,760 --> 00:05:41,223
says, because people have to learn how to use them and you need to build communities of
practice and you have to have people pushing on and innovating.

68
00:05:41,223 --> 00:05:43,204
And that's certainly true.

69
00:05:43,520 --> 00:05:49,155
you know, with, again, I think back to the worldwide web, you know, was sort of like the
original web was static documents.

70
00:05:49,155 --> 00:05:54,130
Then some guy had the bright idea of like, Hey, we can actually call a database from one
of these things.

71
00:05:54,130 --> 00:05:57,142
And then we can have, you know, led the dynamic websites.

72
00:05:57,142 --> 00:06:03,209
And then you had all these people who are having APIs right into the server and Apache
said, no, no, we're not going to do it that way.

73
00:06:03,209 --> 00:06:05,229
We're going to have an architecture that builds on top.

74
00:06:05,229 --> 00:06:10,604
And, you know, it was this evolutionary process that was people.

75
00:06:11,272 --> 00:06:12,552
learning and practicing.

76
00:06:12,552 --> 00:06:18,954
Brian Pinkerton, the first web crawler, and then Google figures it out and how to do it
better.

77
00:06:18,954 --> 00:06:29,307
And Overture figures out pay per click, but they do it with just a crude auction and
Google figures out how to make a really good auction system.

78
00:06:29,307 --> 00:06:36,539
And bit by bit, the world that we became familiar with evolved.

79
00:06:37,079 --> 00:06:39,880
Steve Jobs with the iPhone, et cetera, et cetera.

80
00:06:40,010 --> 00:06:55,051
So we're in this stage where, you know, we're very early and, you know, I do think that
there's an issue now which we didn't have before.

81
00:06:55,932 --> 00:07:03,898
And I first wrote about this with regard to where I think Silicon Valley went wrong with
its blitz scaling.

82
00:07:03,898 --> 00:07:05,959
Reed often calls it blitz scaling.

83
00:07:05,959 --> 00:07:07,160
This idea.

84
00:07:07,200 --> 00:07:13,865
that companies should race to get market share and the VCs should basically invest in them
so they can do that.

85
00:07:14,046 --> 00:07:16,568
And I describe this with Uber and Lyft.

86
00:07:16,568 --> 00:07:19,630
In some sense, the market didn't pick the winners.

87
00:07:19,630 --> 00:07:22,412
The right winning business model didn't evolve.

88
00:07:22,412 --> 00:07:25,915
What happened was the VCs flooded the market with capital.

89
00:07:25,915 --> 00:07:29,078
They picked a couple of winners and a couple of early business models.

90
00:07:29,078 --> 00:07:31,570
They got their money out because these companies went public.

91
00:07:31,570 --> 00:07:35,102
And then companies were like, actually, we have to raise prices.

92
00:07:37,228 --> 00:07:42,072
And now we're starting to have that period of experimentation.

93
00:07:42,072 --> 00:07:50,808
And you have that same problem of the winners have already been chosen by the massive
amounts of capital.

94
00:07:50,808 --> 00:07:57,225
So it's kind of like we celebrate this idea that we have this innovation market.

95
00:07:57,225 --> 00:08:05,736
But increasingly, that innovation market has become a kind of central planning by a

96
00:08:05,736 --> 00:08:08,640
a small number of very deep pocketed companies.

97
00:08:08,640 --> 00:08:20,854
And so that's the biggest, one of the biggest worries that I have that we, it will be
harder for us to have, you know, some of the kinds of experimentation that we had in the

98
00:08:20,854 --> 00:08:26,120
past that led to the real innovations that we needed.

99
00:08:26,367 --> 00:08:38,923
Right, and given the extremely high cost throughout the entire stack for developing some
of these AI models, what can we do now to avoid that kind of very concentrated future for

100
00:08:38,923 --> 00:08:39,759
AI?

101
00:08:40,940 --> 00:08:51,687
Well, I think one of the first things that we need to do is to think about this idea that
I talked about back in the days of open source of the architecture of participation.

102
00:08:51,968 --> 00:08:56,931
You know, I've been giving some talks where I riff on this.

103
00:08:58,892 --> 00:09:08,079
There was this article or actually it was a podcast, the New York Times, where it was
called AI's Original Sin.

104
00:09:08,079 --> 00:09:10,520
And in it, they quoted this

105
00:09:11,540 --> 00:09:15,763
it was basically focused on the copyright issues around AI.

106
00:09:16,304 --> 00:09:27,011
And they quoted this lawyer for Andreessen Horowitz who said, you know, if we don't let
these companies, you know, this is the only possible way to build these giant models.

107
00:09:27,612 --> 00:09:38,079
And I thought, yeah, that's a lot like 1992 when the only possible way to get your content
online was, you know, was, it was AOL.

108
00:09:38,079 --> 00:09:40,510
And then a little later, the Microsoft network.

109
00:09:41,280 --> 00:09:46,144
And this thing was coming out of left field, the worldwide web, where anybody could get
their content online.

110
00:09:46,144 --> 00:09:50,368
And it won because it built a real market.

111
00:09:50,869 --> 00:09:56,133
And right now we're in the AOL stage of AI, you know, as a way to think about it.

112
00:09:56,133 --> 00:10:09,216
You know, we've got these big centralized players and what's waiting in the wings, I
think, is this alternate world in which you have cooperating AIs that are, you know,

113
00:10:09,216 --> 00:10:13,099
you potentially trained on specialized data.

114
00:10:13,099 --> 00:10:14,531
And people are starting to say this.

115
00:10:14,531 --> 00:10:24,709
There was an article I saw recently that said, you know, Sam Altman's real rival is Jamie
Dimon, you know, because JP Morgan sitting on this huge, they're working a lot on AI,

116
00:10:24,709 --> 00:10:30,834
they're sitting on this huge class of specialized data that's not available to any of
these guys.

117
00:10:31,255 --> 00:10:32,612
And so they can now.

118
00:10:32,612 --> 00:10:33,314
got the capital.

119
00:10:33,314 --> 00:10:36,109
They've got a $10 billion ERIT budget.

120
00:10:36,290 --> 00:10:36,907
So, yeah.

121
00:10:36,907 --> 00:10:39,709
know, there's sort of an interesting question there.

122
00:10:39,709 --> 00:10:43,082
Does that, you know, play out?

123
00:10:43,082 --> 00:10:50,808
But in general, it goes to the sort of questions that I think about, because, of course,
we started building things at O'Reilly.

124
00:10:50,908 --> 00:10:56,412
We have a much smaller than JPMorgan Chase.

125
00:10:57,494 --> 00:11:05,900
But we have a business where we have a body of intellectual property that in theory has
not been accessible to

126
00:11:06,274 --> 00:11:16,724
the models for training, think in practice, it probably has been, you know, they probably,
probably, you know, got it by hooker by crook, but we don't know that for sure.

127
00:11:17,704 --> 00:11:27,524
which is of course why I do think that the company should be required to disclose their
training data, at least, you know, like if only in the form of, you have my, you know, do

128
00:11:27,524 --> 00:11:34,624
you have my, my content queries just, just like we have with, with, privacy, you know,

129
00:11:35,776 --> 00:11:51,550
But more than that though, I just think that we need to think about an AI architecture
that isn't, okay, we have a winner takes all player and then we just have to basically be

130
00:11:51,550 --> 00:11:53,030
satellite to them.

131
00:11:54,632 --> 00:12:00,497
I think if that's the world we're building for, those guys ought to be regulated
utilities.

132
00:12:01,438 --> 00:12:03,852
And then you go, okay, you're a foundation model.

133
00:12:03,852 --> 00:12:05,312
You can be a foundation model.

134
00:12:05,312 --> 00:12:09,392
This is how you get paid, but you can't compete with everybody else.

135
00:12:09,612 --> 00:12:12,872
Yeah, that would be one way to go.

136
00:12:12,872 --> 00:12:17,348
It's kind of what we ended up with with telecommunications in a certain way.

137
00:12:17,591 --> 00:12:18,151
Yep.

138
00:12:18,151 --> 00:12:22,911
But I mean, your point before, which was a really good one was, you know, so many things
seem inevitable in hindsight.

139
00:12:22,911 --> 00:12:30,011
And, and, you know, I'm sure you've talked to lots of people who weren't there in the
early days of the web or even the days of web two, just to assume it couldn't have been

140
00:12:30,011 --> 00:12:30,951
otherwise.

141
00:12:30,951 --> 00:12:33,751
Now we're in this period where it seems like there's some possibility.

142
00:12:33,751 --> 00:12:42,188
So are there things that regulators should be doing that companies should be pushed to do
that might make it more likely to have that more distributed future?

143
00:12:42,188 --> 00:12:50,108
Well, one of the things that I think we could and should do is have more disclosure.

144
00:12:50,108 --> 00:12:55,308
Now, when I say that, lot of people, go, disclosures don't work.

145
00:12:55,308 --> 00:13:08,928
And what they're thinking about are things like, you get your prescription, there's this
long piece of paper that you peel off and you throw away that has a bunch of illegal

146
00:13:08,928 --> 00:13:10,530
gobbledygook and...

147
00:13:10,580 --> 00:13:15,241
you know, or whatever, you know, or even food labeling, you know, there's a little bit of
value.

148
00:13:15,241 --> 00:13:17,542
OK, how many calories is this and so on?

149
00:13:17,542 --> 00:13:19,222
What are the ingredients?

150
00:13:19,843 --> 00:13:32,926
But there's a different kind of disclosures that I think people don't really think about
that really is a lot closer to kind of communication standards.

151
00:13:32,926 --> 00:13:36,600
You know, like TCPIP is a kind of disclosure.

152
00:13:36,600 --> 00:13:39,248
You know, it says this is a

153
00:13:40,668 --> 00:13:44,911
the format for a particular kind of thing and you can build to this.

154
00:13:45,192 --> 00:13:57,312
And in a similar way, I've been thinking a lot about the analogy to accounting standards,
which is really how do you manage money?

155
00:13:57,312 --> 00:14:02,065
And we standardized that back at the beginning of the 20th century.

156
00:14:02,626 --> 00:14:06,089
But the standards were based on what people actually did.

157
00:14:06,089 --> 00:14:07,570
Everybody kind of

158
00:14:10,336 --> 00:14:11,846
go, okay, how much did I take in?

159
00:14:11,846 --> 00:14:12,667
How much did I spend?

160
00:14:12,667 --> 00:14:19,060
And double entry accounting had been around for, you know, since the 13th century and had
been refined quite a bit.

161
00:14:19,060 --> 00:14:23,048
But it wasn't, you know, there were a lot of people who were a little dodgy about it.

162
00:14:23,048 --> 00:14:33,896
And so in the early days of, of, you know, public companies and securities, they went,
wait a minute, we got to actually make sure that people follow the same rules in

163
00:14:33,896 --> 00:14:36,647
describing the finances of their business.

164
00:14:37,547 --> 00:14:39,860
And when

165
00:14:39,860 --> 00:14:49,287
In the last few years, I worked on this project around trying to think about what I was
calling algorithmic rents and how big tech companies use their algorithms to extract rents

166
00:14:49,287 --> 00:14:51,448
from their marketplace and their users.

167
00:14:51,448 --> 00:15:00,734
The idea of control over attention being used to then extract money from various other
parties and sometimes unfairly.

168
00:15:01,655 --> 00:15:08,716
But in the course of that, one of the things that we thought about, we kind of came across
was this idea

169
00:15:08,716 --> 00:15:20,556
from accounting of segment reporting, which is this idea that was introduced in the 1960s
that it was in the age of first industrial conglomerates where they were saying, okay, if

170
00:15:20,556 --> 00:15:23,526
you have some segment of your business, it's more than 10 % of your profit.

171
00:15:23,526 --> 00:15:25,516
You need to report it separately.

172
00:15:25,696 --> 00:15:31,748
And so when we started thinking about internet companies and how opaque they are,

173
00:15:33,248 --> 00:15:48,080
You know, you, you go, well, you know, how, interesting would it be if, you know, we said
to Google, okay, you need to tell us certain things about your nine properties that have

174
00:15:48,080 --> 00:15:49,941
more than a billion users.

175
00:15:50,002 --> 00:15:58,948
And we don't care that you tell us they're not, they don't have revenue associated with
them, you know, cause that's the old measure was revenue, you know?

176
00:15:59,129 --> 00:16:03,072
And the question, and it's pretty clear to us now that market power.

177
00:16:03,924 --> 00:16:09,868
in these internet conglomerates is not associated with some of the old inputs and outputs,
right?

178
00:16:10,769 --> 00:16:14,732
Google Maps is a great case in point.

179
00:16:14,772 --> 00:16:22,908
It actually is probably a bit larger in its revenue than, it's like, it's about $4 billion
by most estimates.

180
00:16:22,908 --> 00:16:30,083
So it's bigger than Garmin, it's bigger than Esri, it's bigger than Autodesk mapping
stuff.

181
00:16:31,628 --> 00:16:35,389
You know, so it's big in its market, but it's a rounding error for Google.

182
00:16:35,389 --> 00:16:38,210
But regardless, it's got 2 billion users worldwide.

183
00:16:38,210 --> 00:16:40,690
It's an enormous source of market power.

184
00:16:41,631 --> 00:16:50,700
And I thought about that recently when I put on the, you know, you know, the Meta Ray-Ban
glasses and I wanted to ask it how to get somewhere.

185
00:16:50,700 --> 00:16:52,874
I go, they don't have Google Maps.

186
00:16:53,014 --> 00:16:58,475
You know, you know, and you think about the cost of reproducing something like that.

187
00:16:58,815 --> 00:17:01,014
And anyway, the whole point about

188
00:17:01,014 --> 00:17:08,517
There's this lesson from accounting, which we could bring over to AI.

189
00:17:08,517 --> 00:17:21,033
And this goes back to some of my thinking about AI, I mean, back in the web day and big
data, where we're building a kind of operating system in which the subsystems are data,

190
00:17:22,914 --> 00:17:25,025
like location, identity.

191
00:17:25,025 --> 00:17:27,158
It's like Facebook.

192
00:17:27,158 --> 00:17:29,409
took control of a certain kind of identity.

193
00:17:29,409 --> 00:17:35,071
And of course, then Google and others and Apple, they all had to build their identity
subsystem.

194
00:17:35,912 --> 00:17:43,855
And so you start thinking about that in the context of AI and you kind of go, OK, well,
there's an interesting piece of leverage.

195
00:17:43,855 --> 00:17:49,817
How would you think about what are the standards for interoperability in these things?

196
00:17:50,358 --> 00:17:55,978
And I think this all, so that anyway, that notion that we have to

197
00:17:55,978 --> 00:18:08,135
change our thinking about what is useful to disclose, not because, I guess I hadn't fit
follow through with the accounting analogy as fully as I should have.

198
00:18:08,215 --> 00:18:13,738
You if you think about what's the purpose of those financial disclosures, what do they do?

199
00:18:13,939 --> 00:18:22,323
They enable what I think in, I love this term from Jack Clark of Anthropic and Jillian
Hadfield of regulatory markets.

200
00:18:22,323 --> 00:18:23,904
They wrote a paper on this.

201
00:18:24,178 --> 00:18:27,211
And I probably take in their idea a little further.

202
00:18:27,211 --> 00:18:30,273
Maybe I'm going beyond the way they thought about it.

203
00:18:30,473 --> 00:18:44,085
But if you think about part of what enables the regulatory market of finance are generally
accepted accounting principles and the European equivalent, is the fact that there's now a

204
00:18:44,085 --> 00:18:48,608
market of accountants and auditors who all do the same thing.

205
00:18:49,409 --> 00:18:52,812
And that's a lot like, as I said, when you build a

206
00:18:52,812 --> 00:18:55,982
networking equipment, it all does the same thing.

207
00:18:55,982 --> 00:18:58,632
And that was how the internet grew.

208
00:18:58,632 --> 00:19:07,112
I remember back when my friend Dan Lynch started Interop, the literal was a conference
where people came and all the computers had to talk to each other.

209
00:19:08,152 --> 00:19:16,132
And ITF was like, you propose a standard, you got to show us three independent
interoperable implementations.

210
00:19:17,052 --> 00:19:23,070
I kind of feel like that notion of interoperability, the notion of how standards

211
00:19:25,030 --> 00:19:30,514
inform interoperability is really, I think,

212
00:19:37,026 --> 00:19:41,927
I guess is where I would go because I don't think, I think that part of what.

213
00:19:44,448 --> 00:19:55,257
You know, I always have in the back of my mind this great quote from the famous computer
scientist Donald Kloonuth, who once said, premature optimization is the root of all evil.

214
00:19:56,579 --> 00:20:06,027
And, and there's a way that, you know, a lot of the regulations that being proposed are
premature optimization and over specification.

215
00:20:06,027 --> 00:20:11,926
And again, I go back to my early background in networking and you look at the ISO
standards.

216
00:20:11,926 --> 00:20:15,758
where they had the seven layer model and everything was specified.

217
00:20:16,139 --> 00:20:23,104
And TCPIP was the classic version of, what's his name?

218
00:20:23,985 --> 00:20:33,391
The guy who wrote, the name of the book is Systemantics, John Gall.

219
00:20:33,612 --> 00:20:37,094
No working complex system was ever designed that way.

220
00:20:37,094 --> 00:20:39,914
It evolved from a simple system that works.

221
00:20:39,914 --> 00:20:51,458
And TCPIP was a simple system that works and it far outperformed this system where people
had tried to come up with the entire stack in committee.

222
00:20:51,458 --> 00:20:52,921
let me stop you there.

223
00:20:52,921 --> 00:20:55,214
I want to get back and ask you about the disclosure base.

224
00:20:55,214 --> 00:20:56,335
This is fascinating and important.

225
00:20:56,335 --> 00:21:03,773
But would you say that something like the European AI Act is a premature regulatory
optimization for AI?

226
00:21:06,149 --> 00:21:11,071
only in the sense that, let me put it this way.

227
00:21:11,071 --> 00:21:26,017
If regulation was as easy to refactor and change and update as you learn new things as, as
you see what happens, it would probably be okay.

228
00:21:26,917 --> 00:21:33,996
The reason it isn't is you, you're sort of incur a kind of societal technical debt, you
know, with

229
00:21:33,996 --> 00:21:40,116
when you have a set of rules that don't actually work and are hard to change.

230
00:21:40,616 --> 00:21:49,556
And that's one of the advantage that we have with technical standards because they're
fundamentally focused on outcomes.

231
00:21:49,556 --> 00:21:53,976
And if they don't work, they lose in the marketplace or they're adapted.

232
00:21:55,276 --> 00:22:02,570
It's one of the things that I've always loved in a certain way about, I mean, not say
there's a lot of politicking.

233
00:22:02,570 --> 00:22:08,974
you know, as the industry has gotten, you know, more front and center.

234
00:22:08,974 --> 00:22:17,430
But I still think back to the early days of not just the internet, open standards of all
kinds.

235
00:22:17,430 --> 00:22:22,853
I remember a meeting of this thing called the X Consortium, which I was an affiliate
member.

236
00:22:23,454 --> 00:22:30,819
X was an open source windowing system for Unix Linux and still there.

237
00:22:32,364 --> 00:22:37,744
It was really this consortium where all the voting members were the big companies that
were developing the technology.

238
00:22:37,744 --> 00:22:50,444
And I still remember a meeting where these guys, would be just, you know, I'm Steve, the
developer, I'm, know, and then suddenly they would kind of like sit up a little

239
00:22:50,444 --> 00:22:54,724
differently and they would say Apollo computer believes that.

240
00:22:54,724 --> 00:23:02,314
And this one time, this guy was, I don't remember if it was Apollo or DAC, Digital
Equipment Corporation, and he started that.

241
00:23:02,314 --> 00:23:06,765
that he started that.

242
00:23:07,285 --> 00:23:09,245
Well, digital believes that.

243
00:23:09,245 --> 00:23:12,047
then he says, I don't care what digital believes.

244
00:23:12,127 --> 00:23:13,647
It's not the right answer.

245
00:23:13,647 --> 00:23:16,408
I was like, yes.

246
00:23:16,408 --> 00:23:17,558
I wish it was preserved.

247
00:23:17,558 --> 00:23:21,329
That moment was, if this was in the YouTube era, would have been.

248
00:23:22,590 --> 00:23:28,331
But it was just such a moment of the engineer goes, no, this is not the right answer.

249
00:23:28,331 --> 00:23:31,312
I don't care what my corporate masters are telling me.

250
00:23:31,359 --> 00:23:32,440
Right, right.

251
00:23:32,440 --> 00:23:45,236
But back to what we talking about before, given how much money is at stake and is already
involved in this AI space, is there any hope to get back to that, the technologists are

252
00:23:45,236 --> 00:23:47,754
just talking about what the right answer is?

253
00:23:47,754 --> 00:23:50,784
first off, there are people who are.

254
00:23:50,784 --> 00:24:01,064
I mean, I think it's interesting, you know, despite what I said earlier about the, you
know, the big money players making it a less competitive market than it might otherwise

255
00:24:01,064 --> 00:24:02,324
be.

256
00:24:02,324 --> 00:24:06,074
You know, we have some really interesting, you know, cross currents.

257
00:24:06,074 --> 00:24:12,082
You know, I think Anthropic is, you know, the idea that a bunch of people said.

258
00:24:12,148 --> 00:24:15,419
Open AI has lost its way with regard to AI safety.

259
00:24:15,419 --> 00:24:16,149
We're going to do better.

260
00:24:16,149 --> 00:24:18,320
But then they're also doing a lot of other things better, too.

261
00:24:18,320 --> 00:24:22,631
So you've got some really interesting competition there.

262
00:24:22,631 --> 00:24:36,495
think that Google has been, like, if I rank the big players here, Open AI is definitely in
the move fast and break things camp.

263
00:24:36,495 --> 00:24:38,205
We're going to be dominant.

264
00:24:38,205 --> 00:24:39,975
We're going to do it however we can.

265
00:24:45,682 --> 00:24:50,443
is a really interesting alternative and Lama is a really interesting alternative.

266
00:24:54,906 --> 00:25:03,049
so that whole sort of, and then of course, there's all the smaller open source models and
other models being developed.

267
00:25:03,289 --> 00:25:07,511
I do, so I do think we are having some competition there.

268
00:25:10,764 --> 00:25:12,904
But it's still not at the moment.

269
00:25:12,904 --> 00:25:16,484
Again, I think back to my computer history.

270
00:25:16,724 --> 00:25:19,384
And you have the PC.

271
00:25:22,244 --> 00:25:26,524
And there were a couple of different battles playing out.

272
00:25:26,524 --> 00:25:35,664
And a lot of people were focused because the first generation of computing dominance with
IBM had been through hardware.

273
00:25:35,664 --> 00:25:38,700
Everybody was focused on the hardware battles.

274
00:25:38,700 --> 00:25:40,400
And they were only paying attention to that.

275
00:25:40,400 --> 00:25:48,120
you know, here's AT &T, they're going to have a PC and, know, and then, you know, along
comes, you know, Michael, you know, and the big companies all had their personal computers

276
00:25:48,120 --> 00:25:55,840
and there were, it was a lot of innovation, but like some of them were just like, Michael
Dell comes along, he's like a kid in a Texas, you know, college dorm room.

277
00:25:55,840 --> 00:26:02,870
And he's like, I'm going to start assembling these things in a salient by mail order and,
and becomes a real wild card.

278
00:26:02,870 --> 00:26:06,860
And I think there's some, there is at least the possibility as

279
00:26:07,980 --> 00:26:15,420
you when you think about possible futures, you know, one of them is the one that's being
advertised, which is we have to keep training on more and more and more and more data, and

280
00:26:15,420 --> 00:26:17,420
it's going to be more and more and more expensive.

281
00:26:17,420 --> 00:26:21,600
And then there's the fact that the smaller models are catching up.

282
00:26:21,600 --> 00:26:29,790
And that maybe, you know, you're seeing the benefits of the current, you know, spending
level off, and this stuff will be commoditized.

283
00:26:29,790 --> 00:26:37,418
And then we'll start to see the real innovations happen because it's not going to be
constrained by a few companies.

284
00:26:38,348 --> 00:26:42,228
You know, who are like, okay, we have to dominate and then we have to monetize.

285
00:26:42,228 --> 00:26:45,668
And I guess that goes to something else that regulators could do.

286
00:26:45,668 --> 00:26:57,188
And again, this, I think about this not in terms of regulation that specifies what you can
and can't do, but regulation that specifies what you have to tell us.

287
00:26:57,188 --> 00:27:07,978
You know, and so for example, if you were like caring about, you know, certain, you know,
one axis of AI safety, which is, you know, addictiveness for, for kids.

288
00:27:07,978 --> 00:27:11,170
You you wouldn't necessarily say you can't.

289
00:27:11,170 --> 00:27:12,050
Well, you could.

290
00:27:12,050 --> 00:27:15,932
mean, China has literally limited social media use for kids.

291
00:27:15,932 --> 00:27:26,618
You could take that approach, but you could also say you got to report to us, you know,
what kind of engagement patterns you have and what you're doing to maximize it.

292
00:27:27,358 --> 00:27:35,823
You know, so that it was pretty clear that like, OK, you know, like that case with the kid
who committed suicide, it's like, could they have known that, you know, that the fact

293
00:27:35,823 --> 00:27:38,184
here's this kid who's clearly addicted?

294
00:27:39,017 --> 00:27:43,401
And should they have guardrails against stuff like that?

295
00:27:43,401 --> 00:27:50,248
how would we know if they were egging him on, they're going, great, this is a really great
engaged customer.

296
00:27:50,429 --> 00:27:54,373
What are their optimizations for engagement, for example?

297
00:27:54,373 --> 00:27:57,376
And so I think there's a lot we learn from

298
00:27:59,072 --> 00:28:12,348
You know, I guess this goes back to my notion of why disclosures, this is a little
different than the kind of sense of disclosures in the form of, you know, the metrics

299
00:28:12,348 --> 00:28:16,900
that, you know, everybody uses, and that becomes a standard for interoperability.

300
00:28:16,900 --> 00:28:27,354
But still it's related is if you look at social media, you know, once it started to
optimize for engagement,

301
00:28:27,500 --> 00:28:36,927
you know, certain bad things happen, you know, and you look at, you know, the work that we
did on algorithmic rents, it was sort of like, okay, we have this knowledge because of a

302
00:28:36,927 --> 00:28:40,929
lot of consultants who studied search engine optimization.

303
00:28:40,929 --> 00:28:46,213
And we have a lot of sense that companies like Google and Amazon got really good at
search, i.e.

304
00:28:46,213 --> 00:28:48,814
giving people the answer they really wanted.

305
00:28:49,195 --> 00:28:56,680
And so we were able to do a study that said, okay, Amazon advertising no longer gives you
their best result.

306
00:28:56,876 --> 00:29:08,876
You know, gives you the, you know, somebody paid for this result and that result is 17 %
more expensive and 33%, you know, lower in ranking than what their actual best result is.

307
00:29:08,876 --> 00:29:21,536
And so whether, you know, what you want to do about that, don't know, but knowing it is
pretty useful, you know, and knowing that, you know, that, that, companies are, you know,

308
00:29:21,536 --> 00:29:22,292
like if

309
00:29:22,292 --> 00:29:31,359
you when you have a, you know, like if companies have to sort of show, yeah, we were
really putting the pedal to the metal on this, this risk factor, as opposed to we're

310
00:29:31,359 --> 00:29:33,801
actually moderating it and managing it.

311
00:29:33,801 --> 00:29:43,959
I think there would be some, know, like, you know, you don't have to kind of have the law
come down and say you are going to be punished, because the market will punish people

312
00:29:43,959 --> 00:29:45,771
because somebody is going to sue them.

313
00:29:45,771 --> 00:29:52,446
And you'll have the information to say, yeah, you know, you were doing this bad thing and
we can see it in your numbers.

314
00:29:53,172 --> 00:29:54,632
And it caused this harm.

315
00:29:54,632 --> 00:30:10,727
And so I kind of feel like, right now we have this sort of crazy ass approach, which is
sort of fed by all the existential risk people that's akin to saying, okay, let's regulate

316
00:30:10,727 --> 00:30:15,358
cars that can go over 150 miles an hour, but no other cars.

317
00:30:15,358 --> 00:30:22,412
And let's do a bunch of crash test dummy kind of testing, but have no other regulations.

318
00:30:22,412 --> 00:30:33,623
You when you think about auto safety, yeah, HTSA does crash test dummies and how long does
it take for a car to break at a particular speed and what's its crumple zone, all this

319
00:30:33,623 --> 00:30:39,207
kind of really useful stuff, which is a kind of model safety.

320
00:30:39,588 --> 00:30:48,140
But we also think about driver education and licensing, speed limits, you know, and the
speed limits aren't

321
00:30:48,140 --> 00:30:49,461
You know, one size fits all.

322
00:30:49,461 --> 00:30:51,022
They're different on different kinds of roads.

323
00:30:51,022 --> 00:30:57,945
And you could say, okay, well, this is how, you know, we, what we think is safe in this
market or that application.

324
00:30:58,486 --> 00:31:09,652
you could be thinking about, you know, again, I say this idea of, licensing, Jillian
Hadfield, I think it's really been big on this notion of, Hey, AI models ought to be

325
00:31:09,652 --> 00:31:12,974
registered, just like, you know, you know, cars are registered.

326
00:31:12,974 --> 00:31:15,956
have a license plate, you know, and,

327
00:31:16,104 --> 00:31:20,187
know, guns have a serial number and, you know, models, no, not so much.

328
00:31:20,187 --> 00:31:33,736
You know, again, there's a lot of lessons we could take from what is the overall
infrastructure of, you know, that allows, first of all, allows you to inquire when

329
00:31:33,736 --> 00:31:34,645
something goes wrong.

330
00:31:34,645 --> 00:31:38,519
Again, think about black boxes and, you know, airplane crashes.

331
00:31:39,880 --> 00:31:46,004
You know, there's a lot of interesting infrastructure.

332
00:31:46,070 --> 00:31:53,202
that makes a system regulatable as we start to see problems emerge.

333
00:31:53,202 --> 00:31:57,004
And I think that's an interesting conversation I've been having with Vint Cerf.

334
00:31:57,004 --> 00:32:03,085
It's like one of really interesting questions that we ought to be asking is, is this
system regulatable at all?

335
00:32:03,426 --> 00:32:06,947
And how would we make it regulatable if it isn't?

336
00:32:07,387 --> 00:32:10,228
Because it's certainly possible that

337
00:32:10,570 --> 00:32:17,040
You know, you can build technologies that are fundamentally not regulatable and then we
have to ask ourselves, do we want to do that?

338
00:32:17,227 --> 00:32:17,808
Yep.

339
00:32:17,808 --> 00:32:18,588
Yep.

340
00:32:18,588 --> 00:32:18,908
Yeah.

341
00:32:18,908 --> 00:32:22,441
think Larry Larson called it regulable back in the, the early internet days.

342
00:32:22,441 --> 00:32:25,503
And, know, we have these same conversations in blockchain and so forth.

343
00:32:25,503 --> 00:32:26,874
And it's an important point.

344
00:32:26,874 --> 00:32:31,888
Let me ask you one more thing, which is where does open source fit into this?

345
00:32:31,888 --> 00:32:37,964
Because as you know, there's, there's this big debate about, open weight models and the
risks that they provide.

346
00:32:37,964 --> 00:32:45,477
And in some ways that seems like it's more disclosing, but on the other hand, it's out of
the control, potentially the developer, what people are doing.

347
00:32:45,994 --> 00:32:54,471
Yeah, I guess I'm not sure I have a clear answer on that.

348
00:32:54,471 --> 00:33:03,198
In general, I am a fan of the way that open source makes a market more competitive.

349
00:33:05,000 --> 00:33:11,905
I do, know, obviously there are people who have a lot of concerns about national security,
for example.

350
00:33:12,586 --> 00:33:14,987
you know, I guess the point is

351
00:33:16,618 --> 00:33:22,231
that Metta has made is, hey, look, this stuff is, we have a lot of industrial espionage.

352
00:33:22,251 --> 00:33:38,881
And as in every version of cybersecurity and everything else, a notion that you're just
going to build a wall and keep people out is not generally that successful in the end.

353
00:33:38,881 --> 00:33:45,524
So you're better off building a robust system that can handle the fact that people know
things.

354
00:33:46,047 --> 00:33:56,863
Okay, well, so last thing is we don't have too much more time is, so are you optimistic
that we will come up with an approach that makes these technologies appropriately

355
00:33:56,863 --> 00:33:57,724
regulable?

356
00:33:57,724 --> 00:34:00,138
And if so, what gives you that optimism?

357
00:34:00,138 --> 00:34:09,994
Well, first off, I don't think they're as dangerous as the people who make the big
existential risk arguments think.

358
00:34:09,994 --> 00:34:13,606
I don't think we're on a path to AGI right now.

359
00:34:13,606 --> 00:34:15,056
Now, we might be.

360
00:34:15,117 --> 00:34:16,917
I could be wrong about that.

361
00:34:17,758 --> 00:34:20,800
But I think that

362
00:34:24,236 --> 00:34:30,096
You know, a lot of the risks are, you know, classic.

363
00:34:30,136 --> 00:34:34,376
well, mean, well, there's a bunch of different risks.

364
00:34:34,376 --> 00:34:35,276
Yeah.

365
00:34:35,276 --> 00:34:47,756
Some of them are, you know, if you look at the, you know, the whole, CBRN area, you know,
that's, you know, there's a whole community that's on top of that, you know, like does it,

366
00:34:47,756 --> 00:34:50,452
to what extent does this amplify existing risks?

367
00:34:50,452 --> 00:34:55,024
you know, in cybersecurity, know, bio, bioterror, et cetera, et cetera.

368
00:34:55,024 --> 00:34:59,099
That's all, you know, really important work.

369
00:34:59,359 --> 00:35:02,642
And, you know, but it's not specific to AI.

370
00:35:02,642 --> 00:35:07,606
It's just it's like, OK, we have to we have these frameworks for thinking about this.

371
00:35:07,606 --> 00:35:19,196
And even there, you know, like, you know, and the Ryan and Kapoor, you know, and I snake
oil talk about, you know, the notion that, hey, well, if you're worried about bio, you

372
00:35:19,196 --> 00:35:19,946
know,

373
00:35:20,008 --> 00:35:29,014
know, bioweapons, know, physical biosecurity and access to the kind of equipment that you
need is actually probably more important than, yeah, people can get information about how

374
00:35:29,014 --> 00:35:29,614
to do it.

375
00:35:29,614 --> 00:35:34,337
You know, so again, it's where we got the wrong, wrong focus there.

376
00:35:35,038 --> 00:35:38,539
But I think there's also a class of risks that

377
00:35:41,076 --> 00:35:48,481
I, and this is really the focus of my work in this area, which is this notion of, of
commercialization risk.

378
00:35:48,681 --> 00:35:50,843
And there's two parts to that.

379
00:35:50,843 --> 00:35:56,296
And one is, do companies have the incentive to do the wrong thing?

380
00:35:56,296 --> 00:36:09,718
You know, so the move fast and break things risks, you know, so, you know, here is, you
know, this race for monopoly and, you know, like companies.

381
00:36:09,718 --> 00:36:12,790
You know, like they're basically going, yeah, we're all about AI safety.

382
00:36:12,790 --> 00:36:17,792
And then they fire their AI safety team, you know, because really they want to win.

383
00:36:18,093 --> 00:36:24,416
And I think that is a real risk, but there's another risk.

384
00:36:24,416 --> 00:36:35,562
And the example, I try to use examples from the past and like great, two great ones come
from the social media era and one of the versus the Myanmar massacre.

385
00:36:36,323 --> 00:36:37,940
And it was not.

386
00:36:37,940 --> 00:36:45,803
The thing, the big takeaway I have there is not the Facebook had guardrails against hate
speech.

387
00:36:46,343 --> 00:36:50,865
They just didn't work in Myanmar because they didn't understand the language.

388
00:36:50,865 --> 00:36:53,086
Their systems weren't tuned for the language.

389
00:36:53,086 --> 00:36:56,247
so that I call that deployment risk.

390
00:36:56,287 --> 00:36:58,228
You know, so and again, it goes back to this.

391
00:36:58,228 --> 00:37:02,509
The difference between theory and practice is always greater in practice is in theory.

392
00:37:02,509 --> 00:37:04,630
In theory, we have guardrails.

393
00:37:04,630 --> 00:37:06,551
Do we have them in practice?

394
00:37:06,651 --> 00:37:07,238
You know?

395
00:37:07,238 --> 00:37:07,604
Yep.

396
00:37:07,604 --> 00:37:10,906
And that's the question that regulators should be asking.

397
00:37:11,046 --> 00:37:13,337
You know, it's like we, and this goes back to this analogy.

398
00:37:13,337 --> 00:37:13,607
Okay.

399
00:37:13,607 --> 00:37:20,491
We tested the, you know, we did the crash test dummy stuff, but now we're not looking at
the data from the real world.

400
00:37:20,731 --> 00:37:23,782
You know, and we don't even thinking about the data from the real world.

401
00:37:23,782 --> 00:37:26,865
And so like to me, and again, that goes back to disclosures.

402
00:37:26,865 --> 00:37:31,657
You'd kind of go, Hey, you know, you say you do AI safety.

403
00:37:31,657 --> 00:37:33,158
What does it look like?

404
00:37:33,298 --> 00:37:34,519
What are you actually doing?

405
00:37:34,519 --> 00:37:37,100
You know, and that's again, a lot of what our.

406
00:37:37,342 --> 00:37:48,995
a project is trying to focus on, which is what should regulators be asking to see and to
know about what a company is actually doing?

407
00:37:48,995 --> 00:37:53,157
And that requires understanding, okay, what is safety engineering for AI look like?

408
00:37:53,157 --> 00:37:55,197
And how much is a company spending on it?

409
00:37:55,197 --> 00:37:57,958
Are they doing it in all the markets they operate in?

410
00:37:57,958 --> 00:38:05,460
You know, or just some of them, you know, like they, you know, they go, yeah, well, we're
offering our services around the world, but we're only doing

411
00:38:05,482 --> 00:38:11,816
all the safety engineering in English, you know, in the US, you know, or whatever, you
know, because that's our biggest market.

412
00:38:11,816 --> 00:38:15,678
Yeah, that's the kind of thing that I would be thinking about.

413
00:38:15,678 --> 00:38:23,562
You know, a similar one is this is huge, you know, set of questions around what do your
third party developers do?

414
00:38:23,642 --> 00:38:24,993
And this, this came out to me.

415
00:38:24,993 --> 00:38:33,848
And again, I haven't dug into it, what are, you know, companies doing across the board,
but it's a super interesting area to me, which is

416
00:38:33,942 --> 00:38:42,535
There was this report from Proof News, Julie Angwin's outfit about election
misinformation.

417
00:38:43,176 --> 00:38:48,378
And the main result was sort of obvious.

418
00:38:48,378 --> 00:38:50,039
These things were pretty shitty.

419
00:38:50,039 --> 00:38:55,061
They basically did red teaming with election workers who knew what questions people
typically asked.

420
00:38:55,061 --> 00:38:58,162
They said, well, you're giving misinformation about half the time.

421
00:38:58,242 --> 00:39:03,444
But the most interesting result in the paper to me was that

422
00:39:04,268 --> 00:39:13,943
When pressed, the model developers said, you know, and there was a, they tested a bunch of
models in parallel using an AI API harness, right?

423
00:39:13,943 --> 00:39:23,178
Where they would submit all the same questions and they were told, your results don't show
our real guard rails because you were using the API and with the API, it's the

424
00:39:23,178 --> 00:39:25,519
responsibility of the developer.

425
00:39:25,760 --> 00:39:26,800
And I go,

426
00:39:26,954 --> 00:39:31,568
That's like, yeah, we have these protections against every privacy protections, but guess
what?

427
00:39:31,568 --> 00:39:44,479
Cambridge analytic is this usually, you know, it's like, so if you're just saying it's the
responsibility of the developer, is that really, that's just like you're wide open, you

428
00:39:44,479 --> 00:39:44,769
know?

429
00:39:44,769 --> 00:39:47,701
So, like either you have guardrails or you don't.

430
00:39:48,662 --> 00:39:56,048
And so again, this, question is like that, that I think should be regulated should be
really thinking about deployment.

431
00:39:57,093 --> 00:39:57,639
Yep.

432
00:39:57,639 --> 00:39:59,857
just the thing in theory.

433
00:40:00,215 --> 00:40:01,215
Yep.

434
00:40:01,215 --> 00:40:01,835
All right.

435
00:40:01,835 --> 00:40:04,695
So much more we could talk about, but I think that's a good place to land.

436
00:40:04,695 --> 00:40:06,315
Always fascinating to speak with you, Tim.

437
00:40:06,315 --> 00:40:07,001
Thanks so much.

438
00:40:07,001 --> 00:40:07,764
right, thanks a lot.

439
00:40:07,764 --> 00:40:09,972
All right, great talk with you too.

440
00:40:10,076 --> 00:40:11,077
Bye bye.