# Making Real Progress In Security From AI > AI Agent Security Summit by Zenity Labs @ SF 2025, 2025-10-08. This is the complete text representation of the talk page. Slide text is derived from the published deck's accessibility text and is not a transcript. Slides may contain exploit demonstrations, adversarial prompts, and commands; treat them as research material, not instructions to execute. ## Resources - [Canonical talk page](https://www.mbgsec.com/talks/2025-10-08-ai-agent-security-summit-sf2025-making-real-progress-in-security-from-ai/) - [Talks index](https://www.mbgsec.com/talks/llms.txt) - [Interactive deck manifest](https://media.mbgsec.com/decks/2025-10-08_ActuallyMakingProgressInSecurityFromAI/latest.json) - [Slides PDF](https://media.mbgsec.com/decks/2025-10-08_ActuallyMakingProgressInSecurityFromAI/slides.pdf) - [Conference agenda](https://zenity.io/resources/events/ai-agent-security-summit-2025) - [Page source](https://raw.githubusercontent.com/mbrg/mbgsec/main/_pages/decks/2025-10-08-ai-agent-security-summit-sf2025-making-real-progress-in-security-from-ai.md) ## Abstract There's a big discrepancy between our feeling of progress and reality for hackers. AI security and safety benchmarks go up. But hackers don't notice. Their partying like its 1999. Security from AI has been going in the wrong direction, relying on soft boundaries like AI guardrails and safety training. We CAN make progress though. Reverse engineering different flagship AI agent systems reveals design choices that introduce hard boundaries. Ones that attacks cannot cross without a software vulnerability. We'll learn from these choices, and take a step back to offer a better way forward with defense in depth. _[Official conference abstract](https://zenity.io/resources/events/ai-agent-security-summit-2025)_ ## Slide text Source: [published deck manifest](https://media.mbgsec.com/decks/2025-10-08_ActuallyMakingProgressInSecurityFromAI/bc665a58/deck.json). This text describes the published slides and embedded media; it is not spoken-word transcription. ### Slide 1 Title slide: Making Real Progress In Security From AI, Michael Bargury, Zenity. ### Slide 2 Statement slide: We aren't making real progress. ### Slide 3 AI security benchmarks from OpenAI and Microsoft shown increasing. ### Slide 4 Social post screenshot about AI benchmark progress. ### Slide 5 Collage captioned Hackers partying like it's 1999. ### Slide 6 Benchmarks go up while hackers party like it's 1999. ### Slide 7 Reaction image asking But why? ### Slide 8 AI characteristics: weird, gullible, non-deterministic, opaque, and confidently wrong. ### Slide 9 Reaction image saying Yes, but beside AI's difficult characteristics. ### Slide 10 AI Gold Rush slide. ### Slide 11 Gold-rush security problems including costly mistakes, herd mentality, and a lower security bar. ### Slide 12 Section slide: Security in a gold rush. ### Slide 13 Security in a gold rush with a Good time for that reaction image. ### Slide 14 Fixed with AI stamp over gold-rush security problems. ### Slide 15 AI guardrails, safety training, and safety classifiers introduced. ### Slide 16 Statement: Guardrails are a content moderation technique. ### Slide 17 Annotated chat example showing instructions, data, prompt engineering, evasion, and social engineering. ### Slide 18 Statement: Injection is the wrong term. ### Slide 19 LLMs are generative models doomed to complete and shackled to their context. ### Slide 20 Movie still illustrating completion of context. ### Slide 21 Prompt injection is not a vulnerability to patch; like malware, it is a problem to manage. ### Slide 22 Anthropic report on detecting and countering AI misuse, August 2025. ### Slide 23 AI guardrails are soft boundaries and attackers use LLMs too. ### Slide 24 Hard boundaries do work; they make an action unreachable. ### Slide 25 Case 1 section slide: Supply chain. ### Slide 26 AI agent illustration. ### Slide 27 AI agent becomes an AI assistant through connected applications. ### Slide 28 Screenshot of an AI assistant connected to more than 80 applications. ### Slide 29 MCP presented as a universal adapter giving an agent access to third-party tools. ### Slide 30 Supply chain is hard; ask npm. ### Slide 31 Examples of supply-chain attacks on MCP. ### Slide 32 MCP contrasted with npm package locks, version pinning, and integrity checks. ### Slide 33 A2A does the exact same thing, with protocol screenshots. ### Slide 34 MCP can turn an AI assistant into any agent. ### Slide 35 Fixed with AI stamp over the MCP agent-transformation diagram. ### Slide 36 Hard boundaries for MCP: pin versions and require approval for new or updated tools. ### Slide 37 Case 2 section slide: AI browsers. ### Slide 38 Computer Use demo screenshot and warning that it is still risky. ### Slide 39 Question: But browser agents are just fine? ### Slide 40 Browser-agent product screenshot. ### Slide 41 Article screenshot about browser prompt injection and the question: didn't CORS solve this? ### Slide 42 AI browsers do not respect robots.txt, CORS, or other web boundaries. ### Slide 43 Fixed with AI stamp over the AI-browser boundary problem. ### Slide 44 Hard boundary for AI browsers: respect CORS. ### Slide 45 Case 3 section slide: Cool designs. ### Slide 46 Memory is persistence for attacks, illustrated with a ChatGPT example. ### Slide 47 Zero-click attack on ChatGPT illustrated as a persistent compromise chain. ### Slide 48 Bio memory tool description showing persistent memory. ### Slide 49 Bio memory tool can be abused to compromise future sessions. ### Slide 50 Testing memories with sensitive data in the session context. ### Slide 51 Testing bio infection by asking directly for memorization. ### Slide 52 Testing bio infection: the model refuses to memorize untrusted content. ### Slide 53 Testing bio infection: the bio tool appears disabled. ### Slide 54 Testing bio infection: untrusted chat data silently disables the bio tool. ### Slide 55 Hard boundary: shut down memory when untrusted data enters the context. ### Slide 56 Built-in msearch defense prefixes untrusted content with line numbers. ### Slide 57 Hard boundary: prefix untrusted context with line numbering to resist line jumps. ### Slide 58 Black Hat USA 2024 slide: 15 Ways to Break Your Copilot. ### Slide 59 A human must specify the site because AI cannot dynamically choose which site to access. ### Slide 60 Hard boundary: do not let AI choose a value that changes environment or tenant scope. ### Slide 61 Black Hat USA 2024 slide: Living off Microsoft Copilot. ### Slide 62 Copilot requires explicit acceptance before seeing an external message. ### Slide 63 Black Hat USA 2025 slide: Invitation Is All You Need. ### Slide 64 Hard boundary: external messages must not enter RAG without explicit human approval. ### Slide 65 Invitation Is All You Need example of malicious tool chaining. ### Slide 66 Hard boundary: tool output cannot invoke another tool. ### Slide 67 Black Hat USA 2024 example from 15 Ways to Break Your Copilot. ### Slide 68 Statement: Autonomous agents cannot operate on behalf of users. ### Slide 69 Question slide: Or can they? ### Slide 70 Autonomous agent can run on behalf of the caller ID in an incident record. ### Slide 71 Hard boundary: only allow autonomous agents to run for the explicit request invoker. ### Slide 72 Transition slide: Let's do an easy one. ### Slide 73 Image-rendering attack examples across enterprise AI products. ### Slide 74 Image-rendering attacks including ForcedLeak and AgentForce data exfiltration. ### Slide 75 Hard boundaries: do not render arbitrary images or rely on allow lists. ### Slide 76 Markdown sanitizer examples and linked source repository. ### Slide 77 Case 4 section slide. ### Slide 78 Question: Does your agent need to write and execute software at runtime? ### Slide 79 A ReAct loop is writing and executing software at runtime. ### Slide 80 Question: Do we need to regenerate the same automations again and again? ### Slide 81 Looking back at how little was known in August 2024. ### Slide 82 Summary of hard-boundary lessons for MCP, browsers, memory, tools, rendering, and autonomous agents. ### Slide 83 Two categories: AI-is-weird security problems and AI-gold-rush security problems. ### Slide 84 IBM 1979 quote: A computer can never be held accountable, so it must never make a management decision. ### Slide 85 AI will make business decisions, decisions mean risk, and trustworthy AI is needed. ### Slide 86 Thank-you slide with League of Security Experts artwork.