重要ポイント
-
効果的なセキュリティテストを行うには、非決定論的な性質を理解することが重要です。
-
AIのリスクを内面化するためには、広範囲を網羅する実践的なトレーニングが必要です。
-
セキュリティチームは、AI主導の開発速度の向上に合わせて自動化する必要があります。
-
「Toxic Trifecta」は、企業に重大なアーキテクチャリスクを生じさせるものです。
-
戦略的な設定によって、創造性が求められない、予測不可能性を緩和できます。
動画の文字起こし
Elizabeth Heathfield, Chief Corporate Affairs Officer, FS-ISAC: Welcome to FS-ISAC's podcast, FinCyber Today. I'm Elizabeth Heathfield, Chief Corporate Affairs Officer at FS-ISAC. As generative AI moves from a cool, innovative approach to an absolute economic imperative, firms and teams need to learn to think not just about AI, but think like AI. Patrick Sullivan, SVP and CTO at Akamai, spent some time geeking out with me on how security teams can learn to harness the non-deterministic nature of AI tools.
Heathfield: Thanks so much for being here. I appreciate it. I'm super excited to talk to you about this because I know that we're both AI geeks. So let's talk about managing non-deterministic risk. Lay the ground rules, set the basics out here. What is non-determinism in GenAI models?
Patrick Sullivan, Senior Vice President and Chief Technology Officer, Akamai: Perfect. Yeah, I think it's important to set the table first. You know, I think when we look at the generative AI models that we're all so excited about, at their core, what they're doing is a lot of complex matrix multiplication and then trying to complete the next word with the most probable outcomes. Depending on how you configure things, it will be the most likely [word] or you could sample a less likely alternative.
But what that means and why we say it's non-deterministic is, if you run [matrix multiplication] with a set of inputs – and even though that model is completely static – there's no change to the system. The next time you run that, you're going to get a different set of results, right? And you run it again, a different set of results still.
In some ways, that's different from a lot of the systems that we traditionally have run. So there's a mindset shift that people need to wrap their heads around for things like security testing. If you have maybe a latent payload for pumped injection that doesn't detonate once, that does not mean that you have assurance that you're not vulnerable. Because the very next time you play back that exact same payload against the exact same application, it may detonate it, and you end up with a negative result. So I think it's important for people to wrap their heads around that.