Elham Tabasi, welcome.
Glad to have you on the podcast.
Thanks for having me.
Glad to be here.
But first, why don't you tell us what NIST is, because it's an important agency, but
one that maybe people are not necessarily familiar with, even if they know something
about the US government and AI policy.
Oh, yeah, happy to.
And that's a great starting point.
So NIST, or National Institute of Standards and Technology, is a non
-regulatory agency.
We are under the Department of Commerce.
And NIST was established back in 1902 with a unique mission to advance US innovation
and industrial competitiveness.
Here at NIST, we have a very broad portfolio of research.
from building the most accurate atomic clock to cybersecurity to artificial
intelligence, advanced communications, but most importantly, a long tradition of
cultivating trust in technology.
And we do that by advancing measurement science and standards, measurement science
and standards that makes technology more reliable, secure, private, fair, or in
other words, more trustworthy.
And that's exactly what we have been doing in the space of AI.
Absolutely.
And so you were one of the architects of the NIST AI Risk Management framework,
which came out in January of 2023.
Other than the general goal of trust and technology that you just described, where
did that project come from?
How did that come about?
Yeah.
So AIRMF that was released a little over a year ago, it was directed by congressional
mandate.
It is a voluntary framework for managing the risk of AI in a flexible, structured,
and measurable way.
The whole point is that
We harness the power of AI for good and AI can serve all people in a beneficial, fair
and equitable manner.
These three attributes that I talked about, flexible, structured and
measurable, we wanted to be flexible to be able to keep up the pace with this rapid
change of the technology, but also so that the different organizations...
with different resources be able to implement that.
We didn't want to be a heavy thing that only the big organizations that have AI
governance or a lot of people doing that be able to operationalize it.
Structured in the terms that when we start working on AI RMF, the concept of the
trustworthiness, the question of what constitutes trust were not quite answered
when there has been a lot of really high value
high -level value -based documents that talk about these things.
None of them try to get those principles into sort of the building blocks that the
developers and the employer of the technology can use that.
So working with the community, we try to come up with unpacking the concept of the
trustworthiness, and come up with the components of the AI system that's
trustworthy, which those characteristics are.
valid and reliable, safe, secure and resilient, privacy -enhanced, explainable
and interpretable, fair with harmful bias, managed and transparent and accountable.
Bring the community on a shared understanding around each of them to get
to the next attribute of the measurable to have a good understanding of what we want
to measure to be able to measure them.
So the importance of being able to...
define trustworthiness into technical and socio -technical terms that developers of
the technology, deployers of the technologies, users of the technology can
understand, and then build a community together to come up with the scientific
underpinning for how to measure those because if you're really serious in the
business of improving things, first we have to be able to measure them.
We can't improve things that we cannot measure.
So I want to get back to that question about measurement because it is really
critical.
But first, let me just ask you, that's a big challenge talking about
trustworthiness around AI and all those principles you described.
They make sense.
But how did you and the team get your arms around that?
How to decide what kinds of aspects to focus on and how to operationalize these
broad principles?
Great question.
So how we do our work in this case and also in many other cases is to work
collaboratively with public and private stakeholders.
So basically everything that we do, we start with a formal process of request for
information.
That's where the government put an announcement into a federal registered
notice, announce to the whole public that we're doing this and ask for comments.
Then we work with the community to do rounds of workshops, listening sessions.
to get answers to all of these questions.
And in doing so, we very early on realized that we should definitely listen to a
community with the expertise that are building the technology or designing the
technology, those that have the expertise in computer science, mathematics,
statistician, but also the community that study the impact of the technology,
psychologists, sociologists, philosophers.
but also the community that are going to be in charge of either top down or bottom
up, try to operationalize and implement it and figure out how it basically fits into
the bigger risk management of the organization, so business groups.
And we also did listening sessions with the AI users, one of our best listening
sessions when we went to community colleges, when we went to minority serving
institutions and talked with the students.
So by getting a lot of input from the community, input from a very diverse sets
of backgrounds and expertise, we came to where we are in terms of the
trustworthiness characteristics, but also the whole document.
Doesn't that broad amount of input though make it a challenge to come up with a
consensus about what direction to go, what the principle should be?
It does.
And I think that's part of the reasons that we want to do that, because it is
really important to keep all of the voices included.
And when you do that, and if you really go for the diversity, of course we're going
to hear inputs both sides of the spectrum and trying to figure out the right space.
And I don't want to say middle ground, because there's a lot of the things to
consider.
You want to be scientific, high scientific integrity and scientific validity, right?
So we want to make sure that whatever we put there is technically robust and
empirically validated.
It is operationalizable and implementable today, but also it doesn't get expired in
a very short amount of time.
So we heard when we start this process from everything and everywhere, from the
civil society groups that wanted everything to be absolutely
public and open and audited and a lot of vigorous tests and transparency
requirements, a lot of really good reasons for that.
And we also hear from the technology developers, importance of the IP and
innovations and how those things have to be protected.
But that's the beauty of this, when you give everybody the voice and we are
objective and want to convene.
try to make value for every participant in the discussions to see how those consensus
are being built.
And the other thing is that when we get input from a very vast group of the
community, NIST has the pen in writing this.
So we would kind of make a decision where we have to go.
Of course, we go out and explain and make sure that to the extent possible, we have
the buy -in.
Those buy -in are important because that's
That's where the adoptions happens, but that is part of the process, building the
consensus.
How can we measure the trustworthiness or responsibility of an AI system?
That's a very difficult, you know, that's a technical challenge.
That's a question that the community is looking at this.
It's difficult at many different dimensions.
So our institutions, our work, the group that I have been a part of this, and by
the way, I joined this in June 7, 1999.
So in a few months, it's going to be my 25 years and all my career here, I've been
working on computer vision, machine learning type of efforts.
and a lot of tests and evaluations.
So if some of your listeners are familiar with the face recognition tests or
fingerprint work that we have done, I've been part of that team doing that type of
evaluation.
So with a rich history and experience and understanding of how to do evaluations,
evaluations of the AI system poses a new set of questions and challenges.
One is, of course, what we mean by
safe AI, what we mean by secure AI.
But even if we can come up with some understanding of those, AI systems are not
just about data computing algorithm.
There are complex interactions of the data computing algorithm with the environment
they're operating in, the human that operates that.
So the solutions and the answers are not only technical and technology.
but they all had to be answered through the socio -technical lens.
What that means is that the evaluations that we have done before to get systems
out of their context of use, bring them to the laboratory, run the test data through
them and test them, and those are important.
We have to do them.
They give us some baseline of the capabilities and limitations and failure
modes, but they are not going to be sufficient to surface all the sorts of the
problems, negative consequences that can come.
out of the AI systems.
We are hearing a lot about the red teaming of AI systems with spot checking and
basically trying to see how AI systems can get into undesirable behavior.
But beyond those things, we believe that another set of testing where AI systems
are being tested in their native context of use with the humans, individuals that
are
going to use it, it's really important.
We call them field testing.
We have seen examples of that in sort of, you know, pharmaceutical.
FDA is doing this.
So it's, there are all these difficulties of exactly what we are testing for.
And then the spectrum of the test that we are doing is not going to be enough.
The laboratory testing is not going to be enough.
We have to augment it with other testing.
And then the...
The other thing is that we are not going to be able to test multi -purpose AI
systems for all of the different use cases that are being used.
So how to scale all of these things?
So a difficult problem.
It is something that we have been working on since the release of the AI RMF.
One of the four functions on AI RMF is the measure function.
AI RMF provides, in addition to giving the structured approach to what we mean by
trustworthiness, the meat of that is...
72 recommendations categorized in four functions of map, measure, manage, and
govern to how understand and identify AI risks, how to measure them based on those
measurement results, what type of risk responses and mitigations are to be
implemented, and the govern functions give recommendations basically for the
processes and procedures, all of the embeddings that's needed for the right.
risk management processes.
So after the release of the AIRM, we put a lot of our effort into the measure
functions, which quite aligns and become the foundation work that our latest
assignment as part of the EO is built on top of that.
So a lot of work is happening as part of the task that we have with the EO as part
of the AI Safety Institute.
We are standing up.
and its consortium to work with the community on figuring out the evaluations.
The last thing I will say in this space is that it's a tough problem and we are not
going to get the answers in the first go.
So we have to be on the understand that it's going to be iterative and as we do
more of this research, run evaluations, you know, write guidelines based on those
guidelines, conduct evaluations.
The evaluations will tell us that what was the gaps in the guidelines, the evaluation
can also say the gaps in the
So hopefully we are going to generate a positive feedback loop that the
measurement science is getting strengthened by empirical evidence and
science is going to be built around that.
But also the practice and adoption of the AIs are going to get stronger because of
the test results.
Can you talk about some of the directions that you and the community are going to
try to come up with those benchmarks for some of those issues?
All right, so October 30th, Executive Order 14110, Safe, Secure, and Trustworthy
AI was released.
It's a very comprehensive set of instructions to many of the federal
government agencies on how to promote safe, secure, trustworthy AI.
Section 4 on safety and security tasks several agencies, but predominantly NIST.
on advancing the measurement for and evaluations for safe, secure, trustworthy
AI systems.
In particular, it directs NIS to develop guidelines for evaluations, reteaming,
safety, cybersecurity.
It asks NIS to facilitate development of consensus -based standards.
It also asks NIS to provide testing environment for
evaluations of the AI systems.
These guidelines and infrastructures will be voluntary resources for the community
to use.
So right now we had our heads down drafting the documents, which we are
hoping to go out for public comment.
Based on the public comment, we're hoping to improve it to meet the deadlines of the
EO.
And the EO gives us a 270 days deadline that gets us to the end of the July.
But all of these could work.
All of these actions that the EEO sets in the motion, we think that should not and
cannot stop in end of July.
So we're looking into the AI Safety Institute and its consortium to give a
long lasting approach to all of these efforts in the research and evaluation
builder.
So what we are working with the community is first develop the guidelines that we
have asked through the EEO, working through the consortium with the consortium
members.
when it's out to refine them, improve them, conduct evaluations, and based on
those kind of lived experiences, improve the guidelines.
Eventually, we definitely look like to have sort of standardized test
environments, standardized methods and metrics and methodology for testing so
that the community can use them and help with the scaling of the evaluations that
needed to be done in this space.
Have you seen good examples of either private sector companies or public sector
organizations taking the RMF and operationalizing it?
We have heard a lot and we are really grateful for that.
So when the AIRMF was released January 26, 2023, a lot of different companies
voluntarily and entities, I should say, gave us some sort of testimonies on how
they are planning to use AIRMF.
And that again goes back to building that engagement because when it comes out,
everybody sees themselves in it.
And then since then, we have heard from...
large medium, small size businesses that are using it.
Some of those have given us, we call them use cases, how they're using it
internally.
Some of those are posted on our website.
Some of the other one has been talking to us and letting us know that, particularly
for the larger organizations, how they are aligning their internal processes with AI
or MF.
And we're always happy to work with the smaller size businesses to...
help them with the operationalization.
So one of the things, for example, we did in March of 2023, we put out the
trustworthy AI resource center.
So in addition to the release of the ARRMF, with the goal of keeping ARRMF, we
call it as evergreen as possible, with a revision cycle of three to five years, we
put out a playbook out that basically gives a lot of...
informative references and information and descriptions on exactly what to do for
each of those 72 recommendations, what they can do today, what are the tools and
standards that can be used today for implementations.
So the playbook, we put a revision cycle of every six months so we can keep it up
with the latest changes in the technology as tools and information becomes
available.
We...
We made sure that this is sort of interactive, filterable based on the role
of the AI actor, designer, developer, deployer, the area that they want to get
information out from bias or privacy.
And we're also working with the community to add more tools and support, supportive
documents, informative references, implementations that can help with the
implementation of that.
But across the board, we have seen...
It's being used.
Another interesting or welcoming happening is that at the time of the release of the
AI RMF, we also released Crosswalks.
So AI RMF was not the first document being out, and we want to definitely make sure
that we build based on the knowledge of the community and leverage.
For example, the definition of AI system in the AI RMF is based on the OECD
definition.
So if we...
based on the input of the community.
We improved OECD definition and then OECD took some of our improvement, put it back
into their definitions, definition of AI systems.
So we want to make sure that we are not adding confusion to the market by yet put
another document out.
So we put out crosswalks that says that, for example, how AI RMF relates to OECD AI
recommendations, the draft of the EU AI Act.
few weeks ago, but at that time it was a draft with some other documents that's
coming out of the White House, but also with ISO, the standards that were coming
out of the ISO international standard organizations.
ISO Insights is the technical tag to ISO, US technical tag to ISO.
Basically pick that up and as ISO is coming up with other standards, they
develop crosswalks with AI RMF.
And hopefully all of these things that...
I think is an indication that AI -RMF are being used, so that's why they need those
crosswalks, but also hopefully bringing more semantic interoperability in the
short run.
You mentioned the executive order and the AI Safety Institute that was one of the
things that was created out of that process and you're now taking over as CTO
of that USAI Safety Institute.
How is that different from the work that NIST has done before and what's on the
agenda for the Institute?
Yeah, we look at both the EO and the Safety Institute as basically, that's the
phrase I use, supercharging our efforts in advancing trustworthy responsible AI.
So I think in terms of the goal and vision to advance the science, practice, and
policy of AI systems,
increasing adoptions of the AI is the work that we are doing.
And all of these things are giving us more capacity and strengthens our engagements
to do better and more.
Eventually, we also want to keep an eye on the, or help with the adoption.
So as a measurement science agency, we want to make sure that the right
measurement science tools, standards, guidelines are there.
for anybody that wants to design, develop, or use AI systems.
And hopefully, by doing this, we actually help the adoption rates of AI in science,
in health, climate, agriculture, all of those areas that we think that that's
where AI technology can bring many, many benefits.
What do you think the biggest gap is right now or the biggest area where we haven't
seen enough progress on some of the elements that are going to be necessary to
promote that?
I might be biased coming from a measurement science agency, but I would
say measurement and evaluations.
I think everybody wants to save, secure, trustworthy AI, but we don't know how to
measure them.
We don't have the metrics there.
We don't have the methodologies there.
There are documents, policies they like to talk about, assuring that AI systems are
safe.
You know, we saw it in the EU AI Act.
They want to see...
some sort of assurance or conformity assessment, but those specifications
hasn't been written yet.
So what do we mean by safety?
And policies usually stay at the abstract high level that AI should be safe or non
-discriminatory, but bringing it down, getting everybody on the same page, that's
what it means, and how to assure for them.
I think that's the biggest, that's one of the biggest gaps.
In general, in this AI space, I will say that we know a lot less than what we
should.
So anything that we can do to be able to know more what to expect, how to
characterize the behavior functionality of the systems, these are the good partners.
How much are those really scientific questions though?
And part of what I'm thinking about is you get to some issues like bias, where
there's various different tests about algorithmic fairness, but some of them are
fundamentally incompatible.
And you mentioned the socio -technical aspects of the systems where there may be
things that are not just about yes or no.
So how do you and others in the community work to identify what it is where we can
actually get an answer through the kind of science you're talking about?
Yeah.
Thank you for that question.
So back in March 2022, we put out a document out, special publication 1270.
It has a name towards standards for understanding harmful bias in AI systems.
But these are some of the foundational work that we're doing with the community
that.
Again, as you said, there are tools for bias, but what are really measuring?
Are they measuring the right thing?
What should we really measure?
What is bias across the life cycle of AI anyway?
So that document talks about a lot of these things.
And if I want to summarize, the most important to me, the most important point
of that document is that bias is not just about disparity and demographic disparity.
A lot of us, when you're talking about
bias, the first thing that comes to our mind is demographic disparity.
The document talks about computational and statistical bias as one type of bias, but
there are also other types of bias, systemic biases.
If you go and do a statistically valid data collection of the community and
society today, that's going to be biased and the biases baked into the society will
get into the systems because our training data will carry those biases.
It also talks about another type of bias, which is cognitive biases.
How the different individuals process information given by AI systems or
interact with AI systems.
And based on that, how we react to those is different.
So there's biases there too.
Understanding all of these types of biases, having a wholesome understanding
of now in this topic what the bias is and...
how biases showed itself at the different stages of the AI lifecycle become really
important on other steps of the evaluation.
So the science part of that, again, becomes to having a sort of a wholesome
approach to what could be the sources of the biases, where they're coming from, and
that information can help us to do the measurement.
You asked the question about the socio -technical.
So all of these problems are socio -technical.
So there is not just a technical or technology solution for them.
We just talk about cognitive science biases, for example.
And the fact that one of the components of the responsible AI is human centricity.
So you have the humans there.
And that gets to another challenge that we have with building the science.
for the measurement that a lot of things that we are used to on, for example,
measuring length, mass, electricity, those areas that NIST work on coming up with
standards for measuring them at the turn of the last century, which were critical
for the underlying technologies of that time.
Right now we have...
quantitative metrics to measure them, intropable ways of measuring them.
We are at the distant of technology, we are trying to do the very same thing for
the technology of our time.
As you mentioned, some other jurisdictions, the European Union in
particular, have now a comprehensive legislation.
states.
Talk a little bit about what the function is of the work that NIST does in that
context where there aren't necessarily specialized regulations.
So NIST is a non -regulatory agency.
We don't work with regulators.
None of the things that we put out is for mandatory use.
Everything is for voluntary use.
And even the evaluations that we do while we evaluate for, for example, accuracy or
security or privacy or any of these things.
We don't make any judgment on how safe is safe, how private is private, how accurate
is accurate, because there is no one size fits all and it depends on the context of
use.
What we are trying to do and our work with the policy makers is to provide empirical
evidence to help with development of the policies that are clear and also make sure
that the underlying science and technical tool and standard exists.
to help with the enforcement of the policy.
We just talked about it, that the laws can come and say that AI must be safe, but
safety means.
And we see it what happened with the EU Act as well that after its passing, now,
since then, like European centralization body are tasked with coming up with the
standards for basically checking the conformity assessment to the requirements
in the law.
So...
So we are not regulatory.
Everything that we put out is for voluntary use.
But we give sort of empirical evidence for clear policy making, evidence -based
policy making, and the technical tools and the measurements and evaluations needed
and standards needed for enforcement of that.
I use the word standards several times, and it may be.
helpful if I clear that while standard is our middle name, we are not standard
development organizations such as ISO or IEEE.
There are really good policies in the US in effect and in existence that the job of
the federal government is to help industry develop the standards.
So they are in the driver's seat.
We are supporting their work because, again, we bring those objective technical
contributions.
in the development of the standards.
Back in 2019, we had a task by an executive order to write a plan for
federal government engagement in development of AI standards.
With this current EL, we have a task to develop a plan for global engagement to
promote consensus industry standards.
And...
So we work a lot with the community, both industry and standard development
organizations on promotion and facilitate development of the good standards, but we
are not developing standards.
What is your sense about the current state of global cooperation and coordination on
AI response?
So a lot of really, a lot of conversations has been happening for the past many years
that I have been involved and a lot of really good work is happening.
OECD has, OECD AI Governance Party is doing a lot of good work on bringing
communities together but also provide a lot of technical clarity for policy
meeting through a lot of the work that they're doing, a lot of documents that
they're putting out.
We saw.
a really good progress through the G7 kind of code of conduct that try to bring more
sort of a global interoperability on sort of voluntary commitment that happened in
the US.
Bottom line is that global conversations and global interoperability for the AI
space is really important.
It is for any type of standards, but particularly in the space.
The technology is obviously very global and the users are across the globe.
So we are thinking about really maximizing the benefits of AI while minimizing its
negative consequences, harnessing the power of AI for good.
We need more global engagement, both at the research side to build.
the scientific underpinning on the standard side.
So we all are kind of using the same standards and also on the implementation
and operationalization side.
So we can all benefit from this technology.
So a lot of really good work is happening and a lot more can be done, I guess.
Absolutely.
Well, that's a really great place to wind up.
Thank you so much for a really fascinating conversation.
Thanks for having me.
Enjoy the conversation.