[MUSIC PLAYING]
Welcome to Data Nation.
I'm Munther Dahleh, and I'm the Director of the MIT's Institute
of Data, Systems, and Society.
Today on Data Nation, Liberty and Scott
are diving deep into the metaverse.
[MUSIC PLAYING]
When we decided to do an episode on the metaverse,
I have to be totally honest and say that I had to Google,
what is the metaverse?
And what I saw is that most people have
tons of questions about it.
Is the metaverse our future?
Are we going to be living with virtual reality headsets
and completely disregard the real physical world?
But before we can answer those bigger questions,
we have to figure out and define what actually is the metaverse.
Well, hopefully it's Ready Player One.
But I guess if you would ask the average person
on the street with the metaverse,
you get a lot of blank stares and wildly different answers.
But I guess that's why we're doing this episode.
So to be honest, we're probably much closer to the metaverse
than we realize.
We're going to start by figuring out
what that actually is, what we mean when we say the metaverse.
And we are going to take these questions to Joe Paradiso.
Joe is the Dreyfoos Professor at MIT
and directs the MIT Media Lab's Responsive Environments Group.
And on a quick side note, the MIT Media Lab has in the past
really defined the future for society
when it comes to media and technology.
And Joe has also written over 350 publications
and really specializes in how to augment and mediate
the human experience.
In the fall of 2021, Facebook renamed itself
Meta, which was really the first introduction
I had to the word metaverse.
But I at least understand that the metaverse isn't Facebook.
So from your perspective, from the MIT Media Lab perspective,
what do you see the metaverse as?
These ideas go way, way back.
So the basic ideas are not new.
When we think, we don't really call
it metaverse because that's become such a loaded term now.
You say it, everybody knows what it is, but that in a way
hems it in to this kind of narrow definition
that maybe of networked game environments
where you kind of play games and can
get a call in virtual reality.
It's so much deeper than that.
We call it Xverse at the Media Lab.
And we've got lots of programs that have been
exploring the edges of this.
We're looking at now kind of theming them
a little bit more than we have.
But it's about information coming from you
and coming to you in different ways at different times.
And already, we're living a bit in that.
I have my watch that's connected to my doorbell.
And somebody walks by, it may buzz or I'll see a picture.
So there's a whole continuum there that's really intriguing.
And I think that's how it's going to play out.
It's not just going to be this virtual reality
headset we put on, suddenly, we're somewhere else.
It's going to be a whole continuum
of really abstracted presence.
I think that's one of the interesting things
about the future.
As we become more and more connected,
it is about connection.
We start blurring the sense of where we are.
A lot of people are actually already
familiar with this blurred sense of reality.
When COVID-19 pandemic swept the globe,
many individuals started working from home.
And for many, I guess the typical work
day looked like this.
You'd sign into Slack.
You'd update your status to active,
so your employer knows you're there.
You make sure you stay productive
because your workplace monitors your project progress
and senses for computer movements
to make sure that you haven't left your desk.
You join an afternoon video call via your laptop
with colleagues who are also sitting in their homes.
And finally, you end your workday by signing out of Slack,
shutting down your computer, and you immediately
head back into your home life after having just
worked an entire day from your kitchen table.
So the line certainly became blurred between what was real
and what was virtual.
So, Joe, I'm curious, is this the kind of direction
the metaverse is going?
Because I know that there are other
examples that I might qualify as a tool to metaverse.
For example, if I have my Apple iPhone
and I've got a filter on it, and it's
in superimposing a dog over a picture or something I'm
looking at, am I using a tool of the Xverse verse or metaverse?
Or is that part of that blurred reality that you're talking
about.
Would that fit the definition?
It's definitely on ramp to it, and I'd say it's a part of it,
especially when it's networked, because you do it in Snapchat,
like my daughters do, then all your friends see you as that.
So that's your image at that point.
So using the dog filter on my Snapchat
is an on ramp to the metaverse.
And this would be an example of what we
call augmented reality or AR.
AR uses things like real landscapes and digital scenes
to theoretically enhance our current reality.
Augmented Reality, or AR, really came to the world's fingertips
in 2016.
And with the emergence of Pokemon Go,
players would open the app, which would then
use their camera and location to display
whatever the camera pointed at.
The app would display Pokemon on screen
as though they were right there in front of them.
Players could then catch the monsters.
Eventually, developers added trading
where players could directly interact and trade
Pokemon with each other from the comfort of their own homes.
It's funny to say the metaverse and the future of humankind
may look a lot more like the world of Pokemon Go
than you might think, where the metaverse uses reality
to support its platform rather than purely
be a virtual experience.
The most common way people are proposing
this future is through the use of special AR glasses.
And these glasses are being developed
by brands like Meta or Apple or Snapchat or even Ray-Ban.
But I can't help but wonder if existing in these AR and VR
landscapes will actually be what the metaverse consists of, or is
this just some idea purported by science fiction?
So Joe, when you say the word "metaverse,"
the average person on the street,
their first thought will probably be, oh,
like Ready Player One?
Or oh, I get an Oculus and I'm playing here.
You might get a more sophisticated nerd like myself
and be like, yeah, I've been part of the metaverse before.
I play a game called Eve Online, or, yeah,
I owned a house in Second Life.
The definition is fluid, so to speak.
Like, I don't think we would agree that Facebook
is inventing the metaverse.
I think we should say that Facebook
is trying to participate in the concept of the metaverse.
Yeah.
They're trying to give it a big kick.
Yeah, exactly what the concept is.
They're still figuring it out as we are.
You made some good points.
And of course, you could have your house in Minecraft,
or you could have it also in Animal Crossing, which
is very interesting, because people come and visit it.
You work on your property, you share it.
And also the weather in your island
can depend on the weather that's happening locally.
So you already see this folding of reality
into this other universe.
And I think that's where a lot of my work has been.
And I think that's where there's a whole interesting frontier
interpreting the real world through virtual constructs.
So you can actually have a closer connection to it.
I like that.
I'm going to steal interpreting the real world
through virtual constructs, which again, it's
not quite a direct link to the augmented reality we talked
about before.
So there's this concept of uploading your brain
to a computer in the future.
When I hear something like this, I
envision we're going to upload our brain to the metaverse
via an Elon Musk brain link and do a Zuckerberg-owned community
and play virtual football owned by the NFL.
Are we basically building the metaverse
that I can do all the things in a non-physical world
that I can't do in the real world?
That's a very loaded question.
You can break down 20 different ways, all of which
are kind of interesting actually.
So I think at the first level, yeah, uploading
your brain is complicated.
Making a connection to your brain is still complicated,
but it's doable.
People have done it, and there's going to be much more of it.
So if you want to look at-- you go out,
let's say, you talk 50 years.
This is a comfortable number.
It could even be much sooner.
We don't know.
People will connect to virtual realms directly, I suspect.
Lots of different takes on it.
Already, you have people that are
paralyzed that can move things with their motor neurons.
Elon's very involved in that, but there
are lots of other people working on that problem,
including people at the Media Lab and at MIT.
But yeah, this is going to be first for people
with disabilities.
Eventually, I think it will have broader impact.
And yeah, like in the Matrix and in Simulacron-3 and World
on a Wire, stuff like that.
When we talk about the metaverse,
we're talking about creating another environment
in which people interact everything
from economies to how we live.
Do we think this is a good thing for society,
that this technology is coming?
Are you excited for it or are you afraid?
It's a bit of both.
Even here at MIT, Tom Malone used
to talk a lot about collective intelligence, at least
to our point of view.
We're afraid of any kind of a socialism.
I think identity and individual identity is very important.
So if we approach this kind of a collectivism,
what's the role of the individual?
I think it's critical.
Or could individuals be dynamic things that just form out
of combinations of people?
I don't know.
It's going to be very different if you go out
far enough, maybe even in the near-term of a decade or so.
And I think it's going to lead to a lot of capability.
On the other hand, it leads to tremendous opportunity
for indoctrination because you know a lot about people,
you know a lot about their state.
You can approach them with something at the right time.
And for learning, I think that's great because you learn,
the cloud learns, we all learn together.
It's another future for this.
It's already happening to some extent,
but now it's brought up close.
So the whole division of what the cloud learns about I learn
and exactly how I learn it and what context
is something that's going to be very dynamic.
To wrap this up, we've just talked about the negatives.
What are some examples of the true, incredible positives
that are going to be able to come out of this for society?
When we did our hour Here&There project at Tidmarsh,
the cranberry farm, that's a wetland that we have sensors all
over the place and tunnel them through the virtual worlds,
we had an app where you wear bone conduction headphones.
It didn't cover your ears, it just pushed into your head.
So you could hear through vibrations on your skull,
essentially.
It was fairly innocuous.
You really didn't notice it.
But you heard these things as well as hearing
through your ears.
And we had it channeled into the microphones.
We had all over the place to pick up the wildlife.
And it knew where you were because you had GPS and knew
the angle of your head, because we had inertial measurement unit
to track it.
And we tunnel in the audio from these sources
as if you were hearing them.
So they move with your head, so on and so forth.
And it was remarkable.
I was expanded.
I was not diminished.
I didn't want to take it off.
I put on a VR headset, it's kind of fun to play some stuff,
but I get fatigued, I get sick.
I'm relieved when I take it off.
I didn't want to take this off.
It augmented me rather than diminish me.
And I think we have a chance to develop interfaces and systems
that really augment people in different ways
and don't diminish them.
In all aspects, I think that's what we have to go for
and I think there's tremendous promise there.
[MUSIC PLAYING]
I think we definitely have a clear view on the metaverse.
But I also think it's fair to say that a lot of people
have mixed feelings about how this will change society.
Joe mentioned in his experience with virtual reality
that he felt that technology added
something refreshing to life.
He didn't feel that it took away or cheapened reality.
But are there valid reasons for people
to be concerned about the increasing
use of metaverse technology?
So that's a good question.
One of the things people are concerned about
is how hard will it be for people
to access the metaverse from a financial standpoint.
In order to make the metaverse inclusive,
businesses need to be able to securely exchange
funds that are not exclusive to any one country or continent.
So it seems likely that the metaverse will
turn to cryptocurrencies like Bitcoin or Ethereum,
and some transactions may also take the form
of non-fungible tokens or NFTs.
And cryptocurrencies aside, will the cost
of the equipment like VR and AR glasses create an equity gap?
Another thing that people are really worried about
is obviously safety.
Recently, Meta was testing their VR platform
called Horizon Worlds when users alerted them
of this totally new kind of threat, this virtual physical
and sexual harassment.
And meta responded by adding a feature that created a 4-foot
virtual bubble between non-friend players.
But what happens when a user does what's called modding
or modifying the game and they create a program that allows
them to violate this 4-foot barrier and impose on the player
in some way?
How will the metaverse punish virtual crimes?
There are clearly a lot of questions,
concerns for what the metaverse would actually
look like day-to-day.
So we're going to take these questions to Eric Ravenscraft.
So when you Google, what is the metaverse,
Eric's article is the first one to come up.
Eric is a writer for Wired, where he has guided readers
on how to use technology for nearly a decade for publications
like Lifehacker, oneZero, and The New York Times.
The metaverse is such a big term,
and it takes a bit to wrap our minds around what the word
metaverse actually means.
So we're asking everyone.
We talked with Joe Paradiso from MIT's Media Lab
about what the metaverse means to him.
We wanted to talk to you as well.
How would you explain the metaverse?
I sort of have two branching answers to that.
One is that it is sort of a collection-- companies
are referring to a collection of technologies ranging
from virtual reality, augmented reality, a collection
of different types of sensors and networking devices
that enable motion tracking or whatnot that
go beyond the capabilities of just what we typically think
of as the internet, which is your phone
or your computer connecting to a website.
And that is a very wide array of technologies, some of which
even work in opposing directions to each other.
The second definition that I think
is more accurate for the metaverse
as a phrase is that it is a marketing term.
The conversation around the metaverse
kicked off when Facebook announced
that it was rebranding to Meta because it's planning
to build the metaverse.
So this idea of a virtual world where
everyone will socialize and buy stuff
and there will be a lot of transactions for digital clothes
and a merchant to take a cut of those sales
is a narrative that Facebook has a vested interest in promoting.
But the narrative of what the metaverse is
and the technology that actually exists
are very often divergent from each other.
So one of the problems that we've
seen-- so if we describe the metaverse just
for the purposes of this question as what I imagine
it to be, where it's like VR headsets or Oculus headsets
and you're living in this virtual reality,
I would imagine that to do interesting things,
you're going to need stuff.
Like you're going to need equipment,
you're going to need software, you're going to need stuff,
and that there's a huge cost as things goes on and on to that.
Is there a chance that the metaverse or what
I imagine the metaverse to be is actually
going to end up creating sort of an equity situation?
It's going to make it so that people
who have money or who can buy stuff
are going to be able to participate in these things.
And people who can't.
Or is there any issue with the metaverse in terms
of the socioeconomic aspect of our country or our world?
Absolutely.
Yes.
I think that we already see that a little bit
in the internet of today.
You need a laptop or a phone.
You need a cell connection in order to connect.
And those things are relatively cheap and relatively portable.
You can put a laptop on your lap,
you can carry a phone in your pocket.
But let's say that the metaverse is going to be VR-based.
If you have a VR headset--
I have a Quest 2 in my house.
And in order to use that, I need about a living room's
worth of space to set up and us, that. which means,
among other things, I need a living room.
I briefly lived in DC, and there are very small apartments there.
I'm not sure I could allocate space
for a metaverse workstation.
And headsets are already like they're getting less expensive,
but they're not useful enough to justify
the expense in the same way that a phone does.
And they're on top of a phone, they're on top of a laptop.
So it's one more device to buy.
There's space needed in order to use them.
We don't know what bandwidth requirements would be for these
if we got to a really high detailed virtual analog
for a physical world.
So I do think that there are absolutely
socioeconomic challenges to getting people
to use the hardware for this.
And then it sort of creates a chicken and egg problem
that if not everyone can use the hardware,
how do you develop the software for it?
And if no one's developing the software,
why buy the hardware for it?
So there are a lot of challenges in getting
even a hypothetical virtual world like this up and running.
But as we defined earlier, the metaverse
is not just what a headset you put on,
augmented reality phones, glasses.
If you and I and Liberty are talking about this 10 years
from now, where do you think it's going to be?
Is it going to be closer to Ready Player One?
Or is it going to be closer to we
have a lot of individual gadgets that augment our world
and allow us to interact differently,
but not necessarily ensconced or envelop our lives?
Yeah.
And I love talking about what may happen,
but the one thing I am comfortable
saying is we will not be living in Ready Player One.
It's challenging because part of the issue
is that we don't really have a good sense of scale
of the technological requirements for a Ready Player
One style metaverse.
Intel came out recently and described
that the amount of computing power
needed for something like that is
like a thousand-fold beyond what we have now.
But in the next five years, we might get 10-fold,
it's just the scale required.
And as processing power gets more powerful,
we tend to fill that up.
There's a reason that video games don't load instantly,
even though we have massively more powerful systems now
than we had 20 years ago.
It's because when the computers get more powerful,
what we ask of them gets more intense.
So Fortnite could be photorealistic,
but you couldn't do it with 100 people in an instance.
So that I don't think is likely or even possible.
But I do think that we have a lot of exciting technologies
that could help us augment our world.
Like one thing I'm really interested in
is a photogrammetry.
There's an app for an iPhone called polycam.
You basically take a few hundred photos of an object
or you just take a video walking around it.
And it will spit out a 3D model of that object.
You could take that into a video game,
you could make a movie out of that.
You could build your own virtual sets.
Those kind of things are really exciting because that
lets you connect your phone to a virtual environment,
to real practical things you can make.
And that I think, is very exciting.
But will that rise to the level of Ready Player One?
I don't know if it's technically possible in the next decade,
and I'm not even sure people want that.
So just to get back to this idea of-- my research area
is photogrammetry.
And so just to get back to this concept of, OK, you can take--
the metaverse could allow you to take a 3D image of your-- create
a 3D model of yourself and, all of a sudden, or of anything.
There's really cool aspects to that.
But you also then get into the really potential dangers
of what we already see with deepfakes
or if people making it look like our president
is saying one thing or doing one thing.
And you get into some serious danger.
So could you talk to us a little bit about what
may be the dangers that we have to be
aware of as we move forward into the next 10 years
or whatever it might be?
Yeah.
And I think that's especially an interesting problem
because, for example, we already have a problem today with people
going on Twitter and pretending to be someone else.
Someone pretending to be the president is obvious.
Like if the president starts saying a bunch of swears
on Twitter, that's probably not-- it's probably
not the president.
But it is very easy to impersonate someone else,
like someone you don't like at your school, someone
you don't like at work.
You could create dozens of accounts impersonating someone
with just a photo and a couple of tweets
in a virtual environment like we haven't really considered,
could you pretend to be someone else?
There's also new forms of digital harassment.
It took almost no time for Facebook
to have to create new rules about how close to people
could stand next to each other in a virtual environment,
because originally someone could walk right up to you
and just be right in your face, making
it impossible to see what's going on through your camera.
So Facebook said or Meta, I suppose,
said, well, now we'll create a barrier.
So you have to be X distance away from each other
and you can't push into that space.
But then that raises questions of could five people surround
you and make it impossible to move in a virtual space?
Could they physically blockade you.
And these are just questions that the systems,
assuming they even take off at all,
have to be built with the possibility of harassment
and impersonation and abuse and crime.
I guess just my last question is, are you hopeful?
Do you have a positive outlook on what
the concept of the metaverse brings?
Or are you more worried?
Not that it's all rosy, but do you
think it's going to be a net positive the influx
of these technologies and these experiences,
which is what the terms I will use to describe the metaverse?
Or are you more worried about the problems they bring,
which we've been kind of talking about for a little while?
I think in terms of the technologies
that we see today, augmented reality, motion tracking,
photogrammetry, for example, those
are all technologies that excite me.
I love seeing what you can do with those.
I've been obsessed with, like virtual production.
I'm somewhat of a film guy, and I'm
obsessed with virtual productions and stuff.
That excites me.
I also get excited about platforms
that allow people to express themselves creatively.
If there becomes a virtual reality platform that
enables creative expression in a way we haven't seen before,
that does excite me.
However, I think that when we talk about the metaverse, that
is where I get a little more cynical, I'll admit,
because when we talk about technology
in vague and conceptual terms, it
becomes easier to obfuscate problems
with systems like harassment or discrimination or whatnot.
I don't think we discuss the concept of discrimination
in the metaverse.
And it makes it difficult for people
who have to buy this stuff, people
who have to sign on to the platforms
to know what they're being involved with.
I've talked to a lot of people about the metaverse.
And just like you, they have almost no idea what it means.
And that's a tough place to be in when someone's
asking you to buy a product, to use it every day, to sign on
to a service when you don't know what it's going to mean.
So what I try to do in my work is focus on
the specifics of what actually exists as opposed
to grand visions.
The tech industry as a whole generally relies on futurism.
They rely on being able to tell you that this is the future,
this is what's coming next.
This is the grand shift that's happening.
And sometimes, there are big shifts that happen.
But a lot of times there's a lot of technology
that's just good for some things and that's fine.
It's OK for VR and AR and all these technologies
to be useful for a few cool tasks
rather than being a big shift that fundamentally changes
society every 10 years because we
do need to consider the challenges society already faces
before rushing to apply technology to a social task it's
not suited for.
[MUSIC PLAYING]
We asked today, what is the metaverse?
And while I feel that we know a lot more about it than we
did at the start of the episode, it
is very clear that the metaverse itself is still being defined.
So the experts we talked to agree we
aren't going to be living in a Ready Player One world
anytime soon.
I know that's hard for you, Scott
Yep.
Shedding tears right now.
It's not because of lack of vision or missing
link in the technology, it's just because we're
in the very early stages, where computers and software haven't
yet catch up to the vision.
It's a path many researchers and companies are working on today.
Just hopefully they can move faster.
I think that the more that the metaverse becomes
part of our reality, the more we're
going to have to consider the challenges
it's going to present.
Like, how could it manifest similar problems
we see in the real world, like harassment or theft
or discrimination?
And being ahead of these problems
is a huge part of the battle.
And I think we're going to see companies like Meta
handle it going forward.
So while disappointing, we're not
going to be in Ready Player One immediately,
in the meantime or at least the near future.
It's interesting, the metaverse could allow disabled people
to interact and experience things like seeing,
hear, and interacting with others
in a way they currently can't in their non metaverse lives.
And eventually, the metaverse will fully define itself,
and technology will start living up
to the science fiction novels.
Based on what we learned today, I'm
expecting and hoping a much bigger impact
on how we work, play, and go about our day to day lives.
Thanks so much for listening to this episode of Data Nation.
This podcast is brought to you by MIT's Institute
for Data Systems, and Society.
And if you want to learn more about what IDSS does,
please follow us at MIT IDSS on Twitter
or visit our website at IDSS.Mit.edu.