OWASP Global AppSec USA 2025 · 2025/11
OWASP AIVSS Project: What it is, why we need it and how we are doing it
Abstract
Introducing OWASP AIVSS Project: The AI Vulnerability Scoring System for Modern Threats
Traditional CVSS frameworks excel at scoring software vulnerabilities but fall short when addressing AI-specific risks like prompt injection, goal manipulation, and memory poisoning. The AI Vulnerability Scoring System (AIVSS) extends CVSS to tackle the unique challenges of AI systems, particularly agentic AI deployed in enterprise environments.
This session introduces AIVSS, an OWASP initiative developed by a coalition of security leaders from government, academia, and industry. We’ll explore why AI demands new vulnerability assessment approaches, dive into the top 10 agentic AI vulnerabilities, and demonstrate the AIVSS scoring methodology through live examples. Attendees will gain practical insights into assessing AI-specific risks and learn how to contribute to this critical open-source framework that’s shaping the future of AI security.
Transcript
AI generated from recording.
Introduction and Context
00:02 Presenter: Welcome to my session. This is a project showcase. We’re talking about the AIVSS project, the AI Vulnerability Scoring Project.
00:16 Presenter: project, but I want to put it into more broad scope to kind of argue that why we need AIVSS
00:28 Presenter: project.
00:29 Presenter: So I have to put it into the general agentic AI risk management framework.
00:36 Presenter: So basically I will cover mostly AIVSS, but also broad a little bit of scope here.
00:44 Presenter: A little bit about myself. I currently lead the AIVS project with other co-founders and other co-leaders, which I will introduce later.
01:01 Presenter: I also wrote a few books published by Cambridge University Press, Springer, and China Machine Press, John Wiley, Packet, and the BPB.
Risk Management Frameworks and Governance
01:17 Presenter: so different than the publisher.
01:20 Presenter: Maybe the most relevant book is this book for today’s topic,
01:30 Presenter: Securing AI Agents.
01:31 Presenter: So I just brought one copy here if you’re interested in browsing it.
01:37 Presenter: Another book potentially will be relevant is Generative AI Security,
01:42 Presenter: which was published last year.
01:44 Presenter: and if you want to broaden scope in terms of how agent is used in different business
01:50 Presenter: and what is the workflow and also security, there’s a third book.
01:58 Presenter: And the Beyond AI book was quite popular actually.
02:03 Presenter: We have more than 35,000 downloads in Spring website alone.
02:10 Presenter: long. Recently it was translated by
02:13 Presenter: Beijing University into Chinese, so I just come back in China
02:17 Presenter: for the book launch of that
02:21 Presenter: translated version of that book.
02:25 Presenter: But just about myself,
02:30 Presenter: let’s actually focus on the project, the AIBS
02:34 Presenter: project. Where it sits in terms of
02:40 Presenter: framework. I think one of the
02:43 Presenter: very good risk management framework is the
02:47 Presenter: NIST AI
02:50 Presenter: risk management framework. If you
02:55 Presenter: read the document from NIST, it
02:59 Presenter: really have four important phase.
03:04 Presenter: The govern, the map, and the
Agentic AI: Definitions and Threat Landscape
03:07 Presenter: measure and manage. Each one is very
03:11 Presenter: descriptive in the NIST
03:15 Presenter: document. The idea of the
03:19 Presenter: overall framework, it’s not want to be too prescriptive.
03:24 Presenter: But if you really want to implement it
03:27 Presenter: in your company, you need the tools.
03:31 Presenter: You need other methodologies.
03:34 Presenter: And this is why actually from the OWASP and the Cloud Security Alliance, we try to work on.
03:42 Presenter: And the government piece is really the Cloud Security Alliance organizational responsibility,
03:52 Presenter: has been working since last year, actually since last two years.
03:57 Presenter: And we published some white papers.
04:00 Presenter: I co-chair this organization.
04:06 Presenter: And for the MAP, we do have the framework to do,
04:12 Presenter: and I will dive a little bit deeper in today’s talk.
04:17 Presenter: For measure is really the project that we’re talking about,
04:22 Presenter: and I will spend the majority of my time talking about the AIVSS project.
04:28 Presenter: and the manager, right, manages how you do it in terms of red teaming and how do you
04:36 Presenter: prioritize and implement the controls. So we have the CSA AI control matrix
04:44 Presenter: framework. So this is the overall kind of risk management and how AI VSS is fit
04:51 Presenter: into this framework.
04:54 Presenter: In terms of
04:55 Presenter: government,
04:57 Presenter: this is an open source
04:59 Presenter: project. So you can download
05:01 Presenter: from the cloud security
05:03 Presenter: lines.
05:04 Presenter: Just register with email.
05:07 Presenter: You can
05:08 Presenter: this is all published last year.
05:11 Presenter: One is the core security.
05:14 Presenter: We focus on the
05:15 Presenter: model security, data security
05:17 Presenter: and
05:19 Presenter: also the
05:21 Presenter: other like
05:24 Presenter: cultures related to it. But most importantly, we talk about
05:29 Presenter: the responsibility
05:33 Presenter: matrix or RACI model and how do you
05:37 Presenter: implement the responsibilities. Like if you
05:41 Presenter: have AI system, who is responsible? Is this
The AIVSS Project Overview
05:44 Presenter: CAIO responsible or is CISO responsible or CTO?
05:51 Presenter: to argue it, it’s a team sport, right?
05:54 Presenter: So these are three document.
05:56 Presenter: The second document is more from the GRC and culture aspect.
06:02 Presenter: And also the last one is really genetic AI,
06:06 Presenter: the applications and the tools.
06:12 Presenter: So since today’s focus is more on IVSS,
06:16 Presenter: I will go very quick in terms of government and map.
06:21 Presenter: really is trying to deal with the identification or mapping of the threat
06:32 Presenter: if you want to develop your agentic AI applications what are potential threat
06:39 Presenter: you are facing right and how do you deal with it you can use in traditional
06:47 Presenter: select modeling framework, things like Strata, right,
06:53 Presenter: Pasta, there’s lots of good framework.
06:55 Presenter: You still can leverage it.
06:58 Presenter: We argue that this is not enough,
07:01 Presenter: because those framework usually are used
07:04 Presenter: for the traditional deterministic coding, right,
07:12 Presenter: or configurations, those kind of computational framework,
07:17 Presenter: So we need to deal with the non-deterministic nature of agentic AI or generative AI.
07:28 Presenter: And also, agentic AI must have a little bit of autonomy.
07:34 Presenter: If you don’t have autonomy, it’s not agent, right?
07:38 Presenter: You may not have full autonomy, but autonomy can introduce risks, additional risks.
07:48 Presenter: It’s a way called the risk amplification factor.
07:52 Presenter: So risk itself, like autonomy, is not a vulnerability.
07:57 Presenter: So if we really want to use a vulnerability scoring system, we cannot say autonomy is a vulnerability.
08:05 Presenter: autonomy is a risk amplification because for agent.
08:10 Presenter: So this is what we try to do.
08:14 Presenter: And also we try to threat modeling it, right?
08:16 Presenter: And in traditional threat modeling, like, you can have a boundary,
08:20 Presenter: like this is my trust boundary with threat model within it.
08:24 Presenter: We’re using data flow diagram.
08:26 Presenter: This all good.
08:27 Presenter: You’re using iris risk threat model, all this good tool.
08:32 Presenter: You can use it, right?
08:35 Presenter: But with agentic AI, you really should not assume the transfer modeling.
08:41 Presenter: You can have an agent in one platform to talk another cloud platform from AWS to GPC to Azure, right, using Google A2A.
08:56 Presenter: So you have to select model it as well.
09:01 Presenter: Also, we actually published the agent identity approach document.
09:08 Presenter: It is a cloud security alliance.
09:11 Presenter: And also, in the OWASP, we also have the current working on the agent AI top ten.
09:20 Presenter: It’s also located as an identity piece.
09:24 Presenter: and I’m the entry leader for the identity impersonation and control,
09:30 Presenter: but we cannot really assume the static identity that you defined in the deployment time,
09:38 Presenter: meaning that you can specify the policy during deployment time.
09:43 Presenter: That’s not sufficient.
09:44 Presenter: With agent AI, you need the identity to be ephemeral and dynamic assigned with the identity,
09:58 Presenter: and also using dynamic policy based on the task.
10:03 Presenter: So it’s task-based, because otherwise agent will be limited.
10:10 Presenter: Then it’s not really agent.
10:14 Presenter: really want to have it to have autonomy, you certainly need to give it the privilege it
Scoring Methodology and Tooling
10:24 Presenter: needs to finish the task. And this cannot really be solved by traditional
10:31 Presenter: simul-assertion or OWASP scope. So there is a lot of effort, including Cloud
10:40 Presenter: or the security allies or WASP is trying to define
10:43 Presenter: what the new identity framework
10:47 Presenter: or access management framework is, right?
10:50 Presenter: And how do you actually measure this kind of risk
10:54 Presenter: and mapping it?
10:57 Presenter: As the multiple agent communication is another one.
11:01 Presenter: For the agent A is not doing good, it’s a luck agent.
11:06 Presenter: it can impact the whole multiple agent system.
11:10 Presenter: It seems the multiple agent system is the way to go, right?
11:14 Presenter: Single agent system has limited use.
11:18 Presenter: If you ask it to do too much things, it breaks down.
11:21 Presenter: So multiple agent system, based on lots of research, is the way to go.
11:28 Presenter: But it increases the complexity.
11:31 Presenter: How do you measure mapping those kinds of risks?
11:36 Presenter: Maestro Framework, if we have time, there’s a demo video I will
11:40 Presenter: play, but let’s actually focus on the
11:43 Presenter: AI VSS project. So what is
11:48 Presenter: AI VSS? The AI Volatility Scoring System.
11:52 Presenter: Our initial focus is on the agentic AI.
11:56 Presenter: Since we talk with lots of our
12:00 Presenter: customers and open source communities, agentic AI
12:03 Presenter: AI is on the top of their mind?
12:06 Presenter: And how do you measure the risk?
12:10 Presenter: Because if you cannot measure it,
12:14 Presenter: it’s very hard to manage it,
12:16 Presenter: close to impossible to manage it, right?
12:20 Presenter: Firstly, you have to measure.
12:21 Presenter: So what can be the measurement framework?
12:28 Presenter: It’s hard.
12:29 Presenter: We’re trying to do it.
12:30 Presenter: And we’re trying to leverage CVSS.
12:34 Presenter: Actually, our initial framework is based on the Common Vulnerability Scoring System, or CVSS, right?
12:46 Presenter: And there is debate when we try to do within our participant, like funding members and lots of contributors.
12:57 Presenter: There’s some debate if we can leverage it, because CVSS is really focused on vulnerabilities,
13:03 Presenter: and those are deterministic code, right?
13:08 Presenter: And how can we extend it to support a dynamic, non-deterministic agent vulnerability or risk?
13:19 Presenter: So we have to actually think more on the risk, less on the vulnerability, right?
13:27 Presenter: So this is a consensus I think eventually we reach. Like there is a vulnerability
13:35 Presenter: we still need to think a little bit, but we leverage it. But mostly we should talk
13:41 Presenter: about the risk. This is actually not in the CVSS community, the first community.
13:49 Presenter: not want to talk about risk, right?
13:51 Presenter: They focus on vulnerability.
13:53 Presenter: So there’s a lot of work for us, actually.
13:57 Presenter: But we do have our approach, and we published the version 0.5
14:04 Presenter: a few months ago.
14:05 Presenter: Get lots of good feedback.
14:08 Presenter: And we’re trying to work on the version 1 about this.
14:12 Presenter: So we have the version 1 is under review now.
Community Collaboration and Contributions
14:19 Presenter: I will share the document, so everyone just reach out to me through LinkedIn.
14:25 Presenter: I will have LinkedIn.
14:27 Presenter: I will share the document with you.
14:30 Presenter: Yeah, we have many people participate.
14:32 Presenter: So our goal is to have this published March next year.
14:39 Presenter: So you still have time to contribute.
14:43 Presenter: So we try to publish it at the ISA conference.
14:47 Presenter: And yeah, you can visit our website,
14:52 Presenter: aivss.owasp.org, and we have the GitHub.
14:56 Presenter: But the most important thing we’re currently working on
15:01 Presenter: is the document there in the Google Doc.
15:05 Presenter: So there’s a kind of instruction for how you can contribute.
15:10 Presenter: We do have a lot of people already contributing.
15:13 Presenter: So people from the NIST, right, from Entropic and different banks,
15:28 Presenter: pharmaceutical companies, open source community, they all contribute.
15:33 Presenter: And some top AI companies also, they contribute.
15:37 Presenter: So we hope this will be useful.
15:39 Presenter: The key idea is we’re trying to make it simple enough, but still it can cover the actual
15:48 Presenter: agent AI deployment.
15:50 Presenter: And we will provide the tool so it will be useful.
15:55 Presenter: We do have some initial tool already developed by the member of our community, right?
16:03 Presenter: It’s in the demo, so the last one.
16:05 Presenter: one, he implemented the demo tool for us.
16:12 Presenter: And we also have the slack, but the majority of the contribution
16:17 Presenter: is in the document.
16:20 Presenter: So yeah, we have the support.
16:22 Presenter: We have the, in June, we have the kickoff meeting.
16:29 Presenter: So Rob Joyce, who is the former cybersecurity director at NSA,
16:35 Presenter: special assistant to Whitehorse, advisor to OpenAI and the PwC. He gave us the
16:45 Presenter: support and also his idea of how we should run our project, so really thanks
16:52 Presenter: to Locke. Castling from the TAPA and also contributed his opening remarks.
17:01 Presenter: Jackson, Clinton, Apostolo, Vasilev, they also contribute.
17:07 Presenter: So they are the huge support for our project.
17:13 Presenter: So this project is co-lead by myself, Michael, from Zenite.
17:19 Presenter: People may know Zenite is a huge agent.
17:23 Presenter: I have to call out Kyla is also in the audience.
17:28 Presenter: Yes, so huge support on this.
17:31 Presenter: Also the leader of top 10 for the citizen development
17:36 Presenter: framework, right?
17:37 Presenter: So thank you very much.
17:40 Presenter: And Venice is contribute a lot from the AWS,
17:47 Presenter: currently in MEDA.
17:49 Presenter: Bhavia from the ISO office in the Stanford University.
17:54 Presenter: So we co-lead this project.
17:56 Presenter: Of course, we need a lot of support.
17:59 Presenter: So those are founding members.
18:01 Presenter: We have more actually coming since we kick off.
18:05 Presenter: So you can see people from different companies, even from Gartner.
18:12 Presenter: They all contribute and support us.
18:17 Presenter: Yeah, so I think I talked a little bit about what and why already.
Future Directions and Call to Action — Part 1
18:23 Presenter: but here is the tool you can use and here’s again like to reiterate why we need AIVSS.
18:33 Presenter: We cannot manage risk if we cannot measure it, right? Otherwise, how do you know which risk to
18:42 Presenter: manage it? How do you know if you manage it, right? And we have the agentic factor such as autonomy,
18:50 Presenter: non-determinism, and agent can use tool.
18:54 Presenter: MCP server is in wide use.
18:58 Presenter: I think it’s now becoming mainstream.
19:01 Presenter: Many people, if they develop agent AI applications,
19:06 Presenter: they use the MCP server.
19:08 Presenter: MCP server expose what?
19:10 Presenter: Expose tools.
19:12 Presenter: So tool could be misused.
19:15 Presenter: I will give some example in the next slide.
19:18 Presenter: but tool can be misused. It’s all this also memory, right? Agent can use memory.
19:25 Presenter: So maybe a little bit step back, what is agent? Agent, AI or agent, there’s no
19:34 Presenter: kind of official definition yet for good reason because it’s still rapidly
19:40 Presenter: involved field, right? But I think from the industry we do have kind of a
19:46 Presenter: consensus of what the agent is. It has a certain level
19:50 Presenter: of autonomy. It’s based on the language models.
19:54 Presenter: It uses the tools and also has
19:58 Presenter: access to the memory. The memory is really
20:01 Presenter: to have short-term memory
20:04 Presenter: in the real use or you can have persistent
20:09 Presenter: long-term memory. You have the context. It’s also
20:13 Presenter: agent is goal-oriented. So you give a task, it has a goal, it will do what the
20:21 Presenter: task is asked for. So this is one reason you need to also give it a good
20:27 Presenter: appropriate level of identity so it can do the task. You should not limit it
20:35 Presenter: too much and give it too much. You cannot say, okay, you are the HR agent.
20:41 Presenter: Your goal is to scan the resume and find the good candidate to give a call using 11 labs to give a call to the candidate.
20:56 Presenter: And then you give all the HR identity or privilege to this agent.
21:04 Presenter: That’s too much.
21:05 Presenter: You can only give it the access to the resume database and the email or maybe the phone call.
21:15 Presenter: You cannot give access to the cellular database or the performance review database.
21:23 Presenter: So this is the idea.
21:28 Presenter: Yeah, so the key idea is we still can leverage CVSS.
21:33 Presenter: So we should not really get rid of it.
21:35 Presenter: Actually, CVS is like all the major security tools,
21:42 Presenter: like Quora, Snyk, you name it, right?
21:46 Presenter: And we still need to leverage it.
21:50 Presenter: This is why we leverage the CVS version 4.
21:55 Presenter: So when AIVS project start, we actually leverage v3.5.
22:01 Presenter: and we’ll review it.
22:06 Presenter: We think that V4, which is a vector-based matrix, is better.
22:13 Presenter: So we actually upgrade as well.
22:16 Presenter: But this is still a starting point.
22:19 Presenter: And this is more for the quantitative measurement.
22:23 Presenter: Maybe it’s not enough, right?
22:25 Presenter: So there’s another approach is called the SSVC or stakeholders.
22:33 Presenter: This is more for qualitative approach.
22:36 Presenter: So we do have a simple kind of draft for that.
22:39 Presenter: Like if you look at the link of the document, we do have that link there.
22:47 Presenter: So we provide two documents.
22:51 Presenter: One is a qualitative using the CVSS as the base for the scoring.
22:59 Presenter: Another is really decision-based, qualitative.
23:03 Presenter: So this is something exciting, I think.
23:08 Presenter: We have a lot of support doing it, and hopefully we can cover the majority of this.
23:15 Presenter: So this is the whole, over the last, actually this project, AIVSS project started in June,
23:22 Presenter: but the whole preparation work done is a year ago, even long, right?
23:29 Presenter: So there’s a colossal industrial effort in terms of coming up with some core risks, right?
23:36 Presenter: We’re still working on the agentic AI top 10, but it’s happened to be we also have 10 risks here
23:43 Presenter: here from the agent AI tool misuse to the goal manipulation
23:51 Presenter: or instruction manipulation.
23:56 Presenter: So in our document, we actually try
23:58 Presenter: to measure each of those risks given
24:02 Presenter: the sum of implication factors.
24:06 Presenter: If you look at this document, we also
24:08 Presenter: have ten amplification factor like autonomy or tool use all this right so
24:16 Presenter: that make it very complex maybe there’s a reason for that so we’re still
24:24 Presenter: debating right but those are the core risk that I think there’s industry
24:31 Presenter: consensus that we need to measure it and so we can manage it right just example
24:38 Presenter: misuse, what we look at like in the tool selection, when you select a tool,
24:46 Presenter: there should be some risk in it. Like it could be the tool selection attack,
24:53 Presenter: like impersonation, right? Or you need to discover the MCP server, that could be
25:00 Presenter: hacked and it give you some tool which is not good, right? So the tool selection
25:08 Presenter: could be. And also the insecure tool usage.
25:12 Presenter: After you discover, even this is a good tool,
25:18 Presenter: maybe tool itself is
Future Directions and Call to Action — Part 2
25:22 Presenter: not very well protected, although it will do the work that you need
25:26 Presenter: to ask it to do. But it can have a command injection,
25:31 Presenter: SQL injection, and
25:33 Presenter: and other kind of potential risk from the tool,
25:38 Presenter: including the misinterpretation.
25:43 Presenter: And also there could be the tool governance.
25:46 Presenter: You need to govern the tool, right?
25:48 Presenter: And how do you govern it?
25:50 Presenter: So this is one issue that we need to talk about.
25:56 Presenter: Select for goal manipulation.
25:59 Presenter: If you look at my sub stack,
26:00 Presenter: I actually get very deep into all this.
26:05 Presenter: Yeah, you go to my LinkedIn.
26:07 Presenter: I think I have the sub-stack shared there.
26:10 Presenter: There’s like if you’re using Crescendo attack.
26:15 Presenter: Crescendo attack is meaning it’s, I think it’s invented by Microsoft.
26:20 Presenter: It’s gradually kind of ask the agent to do something that it should not to do, right?
26:30 Presenter: it will have gradual gold drift.
26:33 Presenter: For example, you ask it to lock the port.
26:38 Presenter: It will not lock the port.
26:40 Presenter: It actually opens more ports.
26:42 Presenter: If you are smart enough, you’re using crescendo attack.
26:47 Presenter: Malicious gold expansion, it will do the work that you ask it to do,
26:52 Presenter: but in addition, it will do something else
26:55 Presenter: which could expose some risk, right,
27:00 Presenter: or maybe disclose some information that is secret.
27:05 Presenter: And the goal exhaustion loop is really,
27:08 Presenter: it will just stick on the same thing, endless, right?
27:14 Presenter: Without completion.
27:16 Presenter: So we totally have top 10 all here,
27:18 Presenter: and this example, you can take a look.
27:22 Presenter: But we only have five minutes, I speed up a little bit,
27:25 Presenter: so I can leave some minutes for the questions.
27:31 Presenter: Yeah, so call for action.
27:34 Presenter: We plan to have this document published before the ISA conference,
27:42 Presenter: March 23rd, I think.
27:45 Presenter: And, yeah, you can visit our website and register to contribute.
27:52 Presenter: So that’s a call for action.
27:55 Presenter: But remember, we have the risk management framework.
27:59 Presenter: We have the govern.
28:01 Presenter: We have the map.
28:02 Presenter: We have measure.
28:03 Presenter: Then we have manage.
28:04 Presenter: So manage is really, we have the agent AI red teaming guide,
28:10 Presenter: which was published a few months ago.
28:13 Presenter: And we have the tool also there.
28:19 Presenter: So also the AI control matrix, recently published as well,
28:23 Presenter: for the manager side of it.
28:27 Presenter: Yeah, so this demo actually also has a code repository.
28:33 Presenter: It is a Cloud Secular Alliance for using tool.
28:37 Presenter: You can use this tool for free,
28:39 Presenter: just plugging your own API, right?
28:42 Presenter: So if you do your applications,
28:44 Presenter: you can just do the threat modeling here.
28:47 Presenter: I don’t think we have time for the demo,
28:51 Presenter: but you click the link here, it will show you the demo.
28:58 Presenter: So, Red Teaming Tool, yeah, this you can take a look
29:01 Presenter: at the Red Teaming Tool demo and also code the repo.
29:04 Presenter: You are welcome to extend it.
29:08 Presenter: With that, I think the key takeaway from today
29:12 Presenter: is agentic AI certainly is here,
29:16 Presenter: and it may take 10 years to play this out
29:20 Presenter: and it presents the risk that is not before with code,
29:27 Presenter: deterministic coding, right?
29:30 Presenter: It’s really a behavior of semantic risk that we’re facing.
29:35 Presenter: We do need a new risk management approach
29:38 Presenter: and those tools, we list the tools,
29:42 Presenter: including the CSA tools, right?
29:46 Presenter: and also the other tools here, Maestro and AIVSS.
29:52 Presenter: So you can, yeah.
29:55 Presenter: So I see that we’re already done with time,
29:59 Presenter: so maybe you can ask me questions later.
30:03 Presenter: How about that?
30:04 Presenter: Yeah, just scan.
30:06 Presenter: Yeah, take care.
30:07 Presenter: Cool, thank you.
30:08 Presenter: Thank you.
30:09 Presenter: Thank you.