Well, well, Well, it's Saturday the 8th of March, and this
is episode 25, 0 5 of 3 0 1.
Permanently moved to online, a personal podcast, 301 seconds in length, written,
recorded, and edited by me at the jmo.
I continue
to play
close attention to local AI models.
Most mainstream coverage
focuses on the frontier models and the hype.
OpenAi, Anthropic, Deepseek, et cetera.
But it rarely explores
the wider industry and the context.
I mean, all online services
are embedded within
a vast stack of global industrial compute technologies.
XAi's Grok 3 was trained on the
world's largest first
fully water cooled data center cluster.
And that alone is worth an article on its engineering and technical
innovation, regardless of what you think of the man that funded it
also,
yes,
Microsoft just pulled out
of its planned data center leases.
And you could report
that this implies
the AI race has peaked,
but it also signals excess compute capacity ahead,
which means
prices are gonna come down.
It wouldn't surprise me if in a few years
we see dedicated compute clusters for training or updating
checkpointed open models at prices that most large organizations
could justify as reasonable CapEx.
Model Crèches.
But even still,
I think local AI is a more important area to be tracking.
For the first time in a decade.
I'm excited about hardware again.
Last week framework announced their desktop computer, which
comes with 128 gigs of ram
and a rapid Ryzen GPU all for just $1999.
And crucially,
you can chain multiple units together,
forming your own local compute cluster.
Scaling capability at significantly lower costs.
The new Apple studio,
was also just announced,
which when
fully maxed out with an M3 Ultra, 512 gigs of ram, and 16TB of
storage is priced at $14,099.00.
Now this is a lot of money,
but the new Mac Studio
can comfortably run
a 4-Bit Deepseek R1 model locally with room to spare.
Think about that.
You can now run frontier models on a desktop machine which a month
ago needed an entire data center.
What's the bet that future versions of the mac Studio or even Mac
Mini will offer something like frameworks, modular scalability,
enabling them to be chained together?
Imagine super compute clusters in every office
The new Mac studio specs also now provides
open source model makers
a target memory, size,
and hardware platform to optimize for.
The same is also true of the new iPhone 16E, which just got a ram bump
to 8gbs with its updated processor
apple now has a new minimum spec
platform for local on-device Apple Intelligence.
which alongside Google's tensor platform
shows there's an industry-wide push towards making devices,
AI inference platforms rather than cloud clients.
Last year I talked about us heading towards
maximal intelligence at all levels.
Local inference will emerge
and immediately sink below
the user interface
rather than being directly exposed to the user as a chat bot.
I've recently been really impressed by how useful
brave searches AI overviews have become and find myself opening links directly
from its references list all the time.
I've also been making use of
Perplexity's Deep research tool
for more complicated searches,
and now that
I've tried openAI's deep research
for myself,
I sort of have a sense where things might be going.
My own use cases for deep search so far
has mostly been for
creating souped up tutorials
as I'm currently learning a new piece of software.
And I've decided that
help systems
are going to be where we see a ton of innovation.
Every single application we use has a built-in help system.
Some are better than others.
Microsoft Excel,
for example,
has some of the best help documentation of any consumer software.
But even with great documentation,
finding exactly
what you need is often a pain.
Powerful.
Local AI is going to be built into the operating systems directly
and will let us ask software for help directly.
Soon, app help data will load as a knowledge object atop the base
intelligence, letting you ask it how to do something or why something's not working.
For example,
How do I implement an anamorphic lens in Blender?
Or
I'm using this software to do X, Y, and z. I know that I need these specific plugins,
which require some JavaScript knowledge.
Give me a step-by-step guide
on how to implement them.
These are things that I've asked
both deep research tools recently and got back extremely useful results.
In both cases, I was
able to follow the steps they gave me quite far
before switching to a reference YouTube video
that got me across the finishing line.
Tutorials, youTubes and blogs, et cetera aren't going away,
but the ecosystem
is definitely going to change
and all software
is going to need better documentation, but when does it not?
Apple has, its emerging on-device AI strategy, and so does Google, and
I think Microsoft has probably canceled all that cloud compute because really
good local AI models are coming soon.
Software won't necessarily become more useful with all of this,
but it will become more helpful.
Giving computers the ability to explain themselves is a UX game changer,
And lightweight debugging
will only improve
and be built into everything.
Tutorial engines are coming
and they'll run atop local models embedded into our devices.
The entire AI conversation
is going to be very different
when
AI moves out of the cloud
and onto our machines.