1
00:00:00,000 --> 00:00:02,850
Well, well, Well, it's Saturday
the 8th of March, and this

2
00:00:02,850 --> 00:00:05,490
is episode 25, 0 5 of 3 0 1.

3
00:00:05,490 --> 00:00:09,958
Permanently moved to online, a personal
podcast, 301 seconds in length, written,

4
00:00:10,040 --> 00:00:12,792
recorded, and edited by me at the jmo.

5
00:00:13,927 --> 00:00:14,677
I continue

6
00:00:14,677 --> 00:00:15,097
to play

7
00:00:15,097 --> 00:00:17,887
close attention to local AI models.

8
00:00:18,027 --> 00:00:19,437
Most mainstream coverage

9
00:00:19,437 --> 00:00:21,807
focuses on the frontier
models and the hype.

10
00:00:21,863 --> 00:00:24,303
OpenAi, Anthropic, Deepseek, et cetera.

11
00:00:24,398 --> 00:00:25,409
But it rarely explores

12
00:00:25,409 --> 00:00:27,359
the wider industry and the context.

13
00:00:27,494 --> 00:00:29,338
I mean, all online services

14
00:00:29,368 --> 00:00:30,268
are embedded within

15
00:00:30,268 --> 00:00:33,399
a vast stack of global
industrial compute technologies.

16
00:00:33,498 --> 00:00:35,268
XAi's Grok 3 was trained on the

17
00:00:35,268 --> 00:00:36,841
world's largest first

18
00:00:36,841 --> 00:00:38,791
fully water cooled data center cluster.

19
00:00:38,848 --> 00:00:42,088
And that alone is worth an article
on its engineering and technical

20
00:00:42,088 --> 00:00:45,289
innovation, regardless of what you
think of the man that funded it

21
00:00:45,384 --> 00:00:45,864
also,

22
00:00:45,864 --> 00:00:46,374
yes,

23
00:00:46,374 --> 00:00:47,604
Microsoft just pulled out

24
00:00:47,604 --> 00:00:49,404
of its planned data center leases.

25
00:00:49,454 --> 00:00:50,534
And you could report

26
00:00:50,534 --> 00:00:51,404
that this implies

27
00:00:51,404 --> 00:00:52,694
the AI race has peaked,

28
00:00:52,722 --> 00:00:55,692
but it also signals excess
compute capacity ahead,

29
00:00:55,716 --> 00:00:56,406
which means

30
00:00:56,436 --> 00:00:57,936
prices are gonna come down.

31
00:00:57,997 --> 00:01:00,007
It wouldn't surprise me if in a few years

32
00:01:00,007 --> 00:01:02,827
we see dedicated compute
clusters for training or updating

33
00:01:02,857 --> 00:01:06,388
checkpointed open models at prices
that most large organizations

34
00:01:06,388 --> 00:01:08,518
could justify as reasonable CapEx.

35
00:01:08,622 --> 00:01:09,452
Model Crèches.

36
00:01:09,619 --> 00:01:10,609
But even still,

37
00:01:10,669 --> 00:01:13,879
I think local AI is a more
important area to be tracking.

38
00:01:13,943 --> 00:01:15,413
For the first time in a decade.

39
00:01:15,443 --> 00:01:17,433
I'm excited about hardware again.

40
00:01:17,563 --> 00:01:20,593
Last week framework announced
their desktop computer, which

41
00:01:20,593 --> 00:01:22,603
comes with 128 gigs of ram

42
00:01:22,633 --> 00:01:25,826
and a rapid Ryzen GPU all for just $1999.

43
00:01:25,849 --> 00:01:26,659
And crucially,

44
00:01:26,659 --> 00:01:28,489
you can chain multiple units together,

45
00:01:28,539 --> 00:01:30,518
forming your own local compute cluster.

46
00:01:30,610 --> 00:01:33,580
Scaling capability at
significantly lower costs.

47
00:01:33,697 --> 00:01:34,777
The new Apple studio,

48
00:01:34,777 --> 00:01:35,947
was also just announced,

49
00:01:35,947 --> 00:01:36,487
which when

50
00:01:36,487 --> 00:01:41,420
fully maxed out with an M3 Ultra,
512 gigs of ram, and 16TB of

51
00:01:41,420 --> 00:01:43,600
storage is priced at $14,099.00.

52
00:01:43,757 --> 00:01:44,927
Now this is a lot of money,

53
00:01:45,047 --> 00:01:46,517
but the new Mac Studio

54
00:01:46,517 --> 00:01:47,807
can comfortably run

55
00:01:47,807 --> 00:01:51,330
a 4-Bit Deepseek R1 model
locally with room to spare.

56
00:01:51,430 --> 00:01:52,150
Think about that.

57
00:01:52,210 --> 00:01:55,691
You can now run frontier models
on a desktop machine which a month

58
00:01:55,691 --> 00:01:57,941
ago needed an entire data center.

59
00:01:58,079 --> 00:02:01,259
What's the bet that future versions
of the mac Studio or even Mac

60
00:02:01,259 --> 00:02:04,375
Mini will offer something like
frameworks, modular scalability,

61
00:02:04,375 --> 00:02:06,055
enabling them to be chained together?

62
00:02:06,115 --> 00:02:08,815
Imagine super compute
clusters in every office

63
00:02:08,851 --> 00:02:11,101
The new Mac studio specs also now provides

64
00:02:11,101 --> 00:02:12,271
open source model makers

65
00:02:12,271 --> 00:02:13,471
a target memory, size,

66
00:02:13,471 --> 00:02:15,661
and hardware platform to optimize for.

67
00:02:15,775 --> 00:02:19,755
The same is also true of the new
iPhone 16E, which just got a ram bump

68
00:02:19,755 --> 00:02:22,136
to 8gbs with its updated processor

69
00:02:22,166 --> 00:02:23,906
apple now has a new minimum spec

70
00:02:23,906 --> 00:02:26,816
platform for local on-device
Apple Intelligence.

71
00:02:26,957 --> 00:02:29,117
which alongside Google's tensor platform

72
00:02:29,139 --> 00:02:31,989
shows there's an industry-wide
push towards making devices,

73
00:02:31,989 --> 00:02:35,169
AI inference platforms
rather than cloud clients.

74
00:02:35,435 --> 00:02:37,661
Last year I talked
about us heading towards

75
00:02:37,661 --> 00:02:39,971
maximal intelligence at all levels.

76
00:02:40,050 --> 00:02:41,610
Local inference will emerge

77
00:02:41,610 --> 00:02:43,050
and immediately sink below

78
00:02:43,050 --> 00:02:44,250
the user interface

79
00:02:44,310 --> 00:02:47,370
rather than being directly
exposed to the user as a chat bot.

80
00:02:47,578 --> 00:02:49,978
I've recently been really
impressed by how useful

81
00:02:49,978 --> 00:02:54,508
brave searches AI overviews have become
and find myself opening links directly

82
00:02:54,508 --> 00:02:56,398
from its references list all the time.

83
00:02:56,517 --> 00:02:57,777
I've also been making use of

84
00:02:57,777 --> 00:02:59,517
Perplexity's Deep research tool

85
00:02:59,517 --> 00:03:01,047
for more complicated searches,

86
00:03:01,153 --> 00:03:01,558
and now that

87
00:03:01,558 --> 00:03:03,643
I've tried openAI's deep research

88
00:03:03,643 --> 00:03:04,453
for myself,

89
00:03:04,514 --> 00:03:06,674
I sort of have a sense
where things might be going.

90
00:03:06,929 --> 00:03:09,269
My own use cases for deep search so far

91
00:03:09,299 --> 00:03:10,169
has mostly been for

92
00:03:10,169 --> 00:03:11,999
creating souped up tutorials

93
00:03:12,047 --> 00:03:14,117
as I'm currently learning
a new piece of software.

94
00:03:14,339 --> 00:03:15,449
And I've decided that

95
00:03:15,449 --> 00:03:16,349
help systems

96
00:03:16,529 --> 00:03:19,139
are going to be where we
see a ton of innovation.

97
00:03:19,390 --> 00:03:22,270
Every single application we
use has a built-in help system.

98
00:03:22,570 --> 00:03:23,740
Some are better than others.

99
00:03:23,920 --> 00:03:24,850
Microsoft Excel,

100
00:03:24,850 --> 00:03:25,480
for example,

101
00:03:25,510 --> 00:03:28,900
has some of the best help
documentation of any consumer software.

102
00:03:29,213 --> 00:03:30,983
But even with great documentation,

103
00:03:30,983 --> 00:03:31,823
finding exactly

104
00:03:31,823 --> 00:03:33,623
what you need is often a pain.

105
00:03:33,940 --> 00:03:34,450
Powerful.

106
00:03:34,450 --> 00:03:38,560
Local AI is going to be built into
the operating systems directly

107
00:03:38,595 --> 00:03:41,445
and will let us ask
software for help directly.

108
00:03:41,583 --> 00:03:45,753
Soon, app help data will load as
a knowledge object atop the base

109
00:03:45,753 --> 00:03:49,873
intelligence, letting you ask it how to do
something or why something's not working.

110
00:03:50,035 --> 00:03:50,815
For example,

111
00:03:50,912 --> 00:03:53,462
How do I implement an
anamorphic lens in Blender?

112
00:03:53,682 --> 00:03:54,162
Or

113
00:03:54,208 --> 00:03:58,848
I'm using this software to do X, Y, and z.
I know that I need these specific plugins,

114
00:03:58,848 --> 00:04:00,797
which require some JavaScript knowledge.

115
00:04:00,915 --> 00:04:02,214
Give me a step-by-step guide

116
00:04:02,214 --> 00:04:03,144
on how to implement them.

117
00:04:03,310 --> 00:04:04,420
These are things that I've asked

118
00:04:04,420 --> 00:04:08,140
both deep research tools recently and
got back extremely useful results.

119
00:04:08,200 --> 00:04:09,310
In both cases, I was

120
00:04:09,310 --> 00:04:11,620
able to follow the steps
they gave me quite far

121
00:04:11,620 --> 00:04:13,780
before switching to a
reference YouTube video

122
00:04:13,870 --> 00:04:15,340
that got me across the finishing line.

123
00:04:15,514 --> 00:04:18,574
Tutorials, youTubes and blogs,
et cetera aren't going away,

124
00:04:18,680 --> 00:04:19,610
but the ecosystem

125
00:04:19,610 --> 00:04:21,140
is definitely going to change

126
00:04:21,310 --> 00:04:22,270
and all software

127
00:04:22,330 --> 00:04:25,467
is going to need better
documentation, but when does it not?

128
00:04:25,622 --> 00:04:29,852
Apple has, its emerging on-device
AI strategy, and so does Google, and

129
00:04:29,852 --> 00:04:33,545
I think Microsoft has probably canceled
all that cloud compute because really

130
00:04:33,545 --> 00:04:35,585
good local AI models are coming soon.

131
00:04:35,802 --> 00:04:38,982
Software won't necessarily become
more useful with all of this,

132
00:04:39,012 --> 00:04:40,602
but it will become more helpful.

133
00:04:40,982 --> 00:04:45,072
Giving computers the ability to explain
themselves is a UX game changer,

134
00:04:45,522 --> 00:04:46,782
And lightweight debugging

135
00:04:46,812 --> 00:04:47,772
will only improve

136
00:04:47,772 --> 00:04:49,572
and be built into everything.

137
00:04:49,792 --> 00:04:51,652
Tutorial engines are coming

138
00:04:51,831 --> 00:04:55,401
and they'll run atop local
models embedded into our devices.

139
00:04:55,521 --> 00:04:57,351
The entire AI conversation

140
00:04:57,389 --> 00:04:58,635
is going to be very different

141
00:04:58,757 --> 00:04:59,039
when

142
00:04:59,039 --> 00:05:00,509
AI moves out of the cloud

143
00:05:00,599 --> 00:05:01,808
and onto our machines.