Name: Beyond Generic Guardrails: Implementing Custom Policies for AI Agents
Uploaded: 2026-02-12T19:31:33.371Z
Duration: 1 h 4 s
Description: Beyond Generic Guardrails: Implementing Custom Policies for AI Agents

Transcript for "Beyond Generic Guardrails: Implementing Custom Policies for AI Agents": Hello, everyone. My name is Michael Ortega. I'm a director of AI marketing here at Rubrik. Super excited for our webinar today. This is our safe harbor statement. Take it in. But now let's get going with the content you came to tune in for. Alright. So our webinar today is Beyond Generic Guardrails, Implementing Custom Policies for AI Agents. We have our awesome team, engineering team, and leaders who are working on agentic policies, governance engines, and we'll be talking about a lot of really great concepts for you today. We'll be talking about what is an agentic policy? How should we think about guardrails? What are the best practices as you mature from a pilot phase to having hundreds of agents in production? And then most importantly, when do custom policies make sense? How can you do that? What does it take to do it successfully so you're securing your environment, keeping your agents innovating and your business innovating without opening the door to risk. So we're gonna show you all of that, and of course, we'll talk a little bit about Rubrik Agent Cloud. So with that, let me hand it off to our speaker. Our first one joining me today is Arnav Garg. He leads our AI engineering team here, specifically working on all this really cool governance functionality. He's joined by Sameer and Jeff who are implementing these new features and working directly with our customers on what are policy best practices and how to design them successfully for their business. So, with that, you're in good hands. I'm gonna hand it over to Arnav. Take it away. Thanks, Michael. Sorry. One moment. Alright. Yeah. Welcome, everyone. Super excited to talk to talk to you about Rubrication Cloud, sort of the age of AI agents, and sort of trends we're seeing and why you need a way to govern these agents in a a way that that's looked very that looks very different from the way we've traditionally done governance and software. So why are we here? I think one common trend maybe we've all seen is that companies across the world are hiring thousands of new employees who have no managers, no performance reviews, and most importantly, no fear of being fired. And we're essentially in the era of what we call nonhuman identities or NHI, and businesses are rushing to buy this agentic power while simultaneously, dodging the responsibility of who actually owns the fallout when things break. And it's really just a classic accountability shell game. You know, the buyer said it's not their problem when an agent goes rogue, And the answer to who is responsible is always the terrifying nobody knows. And, really, we're here today to learn about how we can stop the buy now, figure the liability out later cycle and put humans back in the loop, not to slow agents down, but to keep the business from flying off the rails while still leveraging AI agents as best as we can. To put this into perspective, you know, this the thing that we're talking about is not a future prediction. It's here and now today, and this is the current reality. So as of mid twenty twenty five, nearly 50% of y combinator y combinator's latest batch of startups are agentic companies. We've seen a massive shift from simple AI features being built in products to entire business models that are in that are agentic within the last eighteen months. And if this data point is not enough, we can actually shift our focus to the broader community. And the community's adoption of AI agents is actually staggering. And I'm sure you've all heard about the OpenCLO mold bot revolution that's kind of kicked off in the last month or so. But maybe what you don't know is that this is one of the fastest growing open open source projects out there and it gained a 150,000 GitHub stars in less than a single week between Jan and Feb this year. Both developers and consumers are ready for AI agents, and the code in general is moving much faster than the rate at which we're able to govern these agents. If you think this is just for start ups, think again. 80 we've surveyed over 200 companies in the global two k, and what we've learned is that 86% of enterprises already have agents in production. And more importantly, 57% of companies we interviewed are managing 10 or more agents in their live environments right now. So this is no longer a pilot stage curiosity. And to ground this even further, you know, agents are are actively doing a lot of work within enterprise. Clot code currently accounts for 4% of GitHub public commits, yeah, commits. They're kind of across all open source. And by the 2026, we expect agents to be responsible for 20% of all software commits globally. So if you look at this chart, what's really fascinating is that we're we're literally watching software write itself at a rate of a 135,000 commits per day. Now maybe to wrap this up, you know, the pressure to adopt AI agents kind of across consumers, across developers, across enterprise, both in your daily workflows and in your products is mounting. And Gartner predicts that by 2028, a third of all enterprise apps will be agentic, and at least 15% of decisions will be made completely autonomously. So if you don't have a plan for these AI agents to make autonomous decisions, you're looking at a major governance gap. Before we go further, I think it's really important that we align on some shared terminology. So what's an AI agent? Broadly, an AI agent consists of two components, an LLM and a set of tools. And you can think of the LLM as the brain. It has a reasoning engine. It's able to think through a user prompt and come up with a way to diagnose or solve a problem or complete the request. And tools are essentially the hands, and, they connect to different systems within your different tools within your system. So Gmail, SAP, Salesforce, Slack, could be many other tools like your back end. And, together, what we're trying to do is we're giving LLMs a way to tell us what, tools to use to solve a given problem. And, with with these tools, it's able to take actions on your behalf, such as reading emails or customer support tickets, writing marketing emails, even sending Slack messages on your behalf. And what's uniquely, you know, what's kind of unique about agents is that they're superhuman, but not always in a good way. So humans are generally if you think about how we do things, humans are generally thoughtful, responsible, slow, and very good at decision making and getting work done. And sometimes, you know, humans are not perfect. We can be slightly unpredictable and make some decisions or make mistakes. However, over decades, we've built enforcement frameworks to limit the scope of damage that may come from nonresponsible human behavior, particularly enterprise such as using strong RBAC systems with clear roles and permissions. However, agents are autonomous and rapid acting at scale. They make thousands of decisions every minute across the fleet of agents. And the big thing the big difference is that compared to humans, they are weakly supervised, which makes them unpredictable and, frankly, irresponsible when compared to deterministic traditional software. And, really, what I'm trying to get to is that this poses a new set of challenges that we will still need to solve for. And and just to sort of ground us in the reality of things we've seen, you know, there's been many, many reports of agents going rogue, not just for local development or side projects or, you know, fun hackathon projects, but in actual products that are used by millions of users today. And I'd say that the two that I really want to emphasize on are Replit and and air the Air Canada chatbot. So Replit is a no code, low code tool to build, you know, full web stack full full stack apps. And there was a report of a founder who was using Replit, and Replit decided to drop their entire production database. And I'm sorry. The agents used by Ruplit decided to drop the entire production database even though there was a code freeze. Right? And so that is something we absolutely don't want happening. Similarly, like Air Canada had a chatbot used for customer support, and there was a user who basically invented said, you know, they deserve a refund, and they convinced the the chatbot to create a new refund rule. And and, actually, Air Canada had to honor it. They were forced to honor it in court. So it hallucinated, did some things that it wasn't supposed to, and, yeah, led to bad action. So loss can go wrong when we use these agents in in large scale production, and we really do need a system to govern them better. And and so how do we how do we really, really solve for this? Right? I think the core issue is that it's not about giving agents too many permissions, nor is it it nor is it putting agents in production. The big problem is that IT teams have no single pane of glass view of what's happening in organizations that are actually using agents, either ones that you buy or ones that you build. So we don't know which agents are running, what data they can touch, how to stop a leak once it starts, and how you can recover from mistakes. So if an AI AI agent makes a mistake in your AWS or Jira environment today, quest the question is, how do you recover? How do you how do you stop these issues from happening even before? And we believe, fundamentally, that having the single pane of glass view is critical to successful AI agent adoption. And most and and just to kinda ground this a bit more, you know, a lot of people have this conceptual belief that building AI agents is is quick and easy. And to be honest, they're actually right. Building an agent is is easy with frameworks like Copilot Studio or ADK. These days, it only takes about a week or two to actually build an agent that can do the task that you want with reasonable performance. However, the part that no one really talks about is how long it takes to put this agent into production. And the reason for this is you have you know, you need to get design docs approved because agents are using tools that have permissions that touch different data systems within your organization. There's also this emerging trend of AI governance committees across the global two k, where it's a group of IT, infosec, legal engineers who need to align on the overall design and and kinda, like, reduce the downside risk of bad actions happening or being taken by these agents. And once you go through this cycle several times, you kinda end up in this place where you can finally deploy your agent. And so by the time you get there, even though it only took you two weeks to build it, it can take up to five months. And if you see where the time is being spent in this workflow, it's primarily around building guardrails, feeling really good about what this agent can and can't do, building, making sure there are ways to govern these agents, have guardrails in place, and make sure that they don't go rogue. And if they go rogue, there is either a way to remediate or limit the scope of what damage they either immediate the action or limit the scope of what they can do. And so we thought, well, why don't we take all of this and just bring it into our product? And so with that, we we sort of decided to build Rubrik agent cloud, and it's essentially your trusted platform for your agents. So Rubrik Agent Cloud brings together all of your applications, your users, the agents you're building, and all the AI models that they talk to in one place so that you have a single pane view of everything that's happening. With this, we're hoping to answer give IT teams, infosec teams, legal teams, and even engineering teams the ability to answer three core questions. One, what agents actually exist in my world? We actually capture and register all agents based on the most popular AI agent building tools that are used within your organization or even with custom models. And so this gives you access to things like an agent registry where you can see which agents are deployed across your enterprise, how active and efficient each one how active and efficient each one of them are, as well as what key risks they might have. The second is what's is what guardrails do I need to have in place to govern my agents? When you need to control your agent's behavior, allow or disallow certain actions from taking place or other things, Rubrik agent cloud lets you enforce policies and detect these errors in real time. And finally, sometimes, even despite best efforts, governance is nonperfect, and people will find ways to cheat the system. This has been true from the from the since the existence of most software systems in enterprise. And when these when things go wrong, you can actually Rubrication Cloud provides a way for you to remediate these bad actions by undoing undoing them to the data and ensuring that your systems are safe, and there's no the the the I guess, the blast radius of damage is not very big. Today, we're gonna focus on the core aspect of governance. So how you enforce policies, how you can quantify risk, and detect agent errors in real time. And maybe you haven't come across this term policies before when it comes to agents, and so I wanted to spend some time actually defining this term because it's sort of a key term you'll feel to the rest of this webinar. And very simply put, a webinar a a policy is a machine readable rule that you can define, that can be used to both monitor and enforce policy enforce guardrails in real time. And unlike traditional rules, they define who they like, you can use policies to find, like, who can use what, when. So you can govern specific agents with specific tools and rules and also define what conditions they should actually when they should kick in. And why is this different from guardrails? Because guardrails have existed in agentic AI systems and even in AI systems generally for a long time. The big difference, bluntly put, is that guardrails have traditionally used have traditionally been used to sort of as, like, a a a politeness layer to sort of ensure that the agent's input and output responses are safe. It doesn't do anything that's very strange. Whereas policies go beyond that, and they make sure the agent actually does work safely. We can understand this better by by sort of splitting turning this up into three levels, looking at how their focus, the level, and the risk mitigation that they actually perform. So guardrails generally prevent toxic language or bad words. Sometimes they even control things like making sure you don't talk about your competitors. That's assuming you know all the competitors that exist. And they typically live at the application layer, so what you just see in chat windows. And, really, what it's it's helping you mitigate is reputational risk. So things that, you know, no harmful content coming back out or preventing, yeah, biased outputs or basis comments, things like that. However and and I think this is important. Right? Like, guardrails are fundamentally important to making sure that your AI agents behave in a predictable way. But what what it's missing is a layer of policies. So policies act prevent bad outcomes from happening at all in the loop of AI agents, such as deleting a database. They work at the system layer rather than the application layer. So they control what the agent does in the back end when it interacts with your systems that are in the loop of your AI agent workflow. And finally, they help mitigate operational risks. So data loss, exfiltration, financial liability, terms of service, you know, violations of breaches, things of that nature, which guardrails traditionally have not been built to do. And, really, maybe one good way to understand why we think that policies are the way to go or sort of as an extension of guardrails, you kind of need both, is we can go back to some of the incidents that we've seen in the last year. So Replit, right, let's say they had guardrails, would it have stopped the agent from deleting a company's production database during a code freeze? Guardrails would not because guardrails are traditionally built for the politeness layer. But we could have a real time tool blocking policy that intercepts the request that the agent wanted to run, npm run db push command, and actually blocked us from event executing. So policy would have prevented this, guardrails would not. Similarly, for the Air Canada chatbot inventing refund rule, would a guardrail help? No. Because what the what the model said was not damaging in any in the traditional sense. It was a hallucination. But we could write a compliance policy that prevents the agents from making financial decisions or refund based decisions and ensure they can only do this by verifying the official policy that the company has via trusted API. And so, hopefully, just like through these examples, you can kind of start to appreciate the difference between policies and guardrails. And maybe to sum this up, we think about policies as a three tiered system. The first is we want to help we want to make sure there are policies that help you with early risk identification and create something called violations. So, essentially, we want to flag configuration based issues by analyzing tool permissions, ownership, making sure that the agents are provisioned and configured correctly out of the box when you first start to use them. So we have a set of policies in the product that catch these early risk identification early risk indicators for you. The second is active detection. So when when your agent is actually, you know, doing things in the product, but maybe then they don't seem, like, high risk, but they are noncompliant, we are able to provide visibility into these into these types of actions that were taken and help you diagnose the intent without stopping the process itself. So even something like, you know, if you have a belief that your agent is making financial decisions on your behalf, you could write policies that help you catch these without stopping them from happening. And when you see that this is happening consistently, you can then move on to the third tier of policy enforcement, which is real time prevention or which we also call tool blocking. And you can instantly block these critical risks, such as deleting files, accessing PII, financial decisions, etcetera, by intercepting and stopping these tool calls before they even execute. And a Rubrication Cloud offers policies and policy creation frameworks that help you every step of the way through your journey. With that, I'll hand it over to to Sameer to to give you a bit more to tell you a bit more about our policy enforcement framework. Sami, you might be muted. It's hard enough. I'll be talking a bit more about, the challenges with aging governance and how we are thinking about the problem. So I think in the day one, right, like, a lot of people are dealing with the tension between what is the perfect solution to governance where they can be sure that any agent that's being deployed won't go rogue or won't have the wrong access permissions and things like that. But having this process by committee, manual approval can really stifle innovation and the ability to deploy agents quickly and keep up with their competitors. And on the other side, you have people just kinda deploying agents in the wild and seeing what will happen without any governance so that they can build quickly and get results, but this can create all sorts of unacceptable risks and gaps that need to be solved. So, obviously, both of these solutions are not ideal, and we need to find the middle ground here. And so the way we think about it at Rubrik is on this day one when you're first trying to get your feet well with agents, is there foundational governance that we can offer you that will continue to grow as your maturity in the agentic journey continues. So we try to think about this in four different stages. So the first stage is gonna be experimentation where there might be one or two agents in pilot modes. And, you know, from our research, we've seen roughly half of organizations we've talked to are still in this phase. So it's really important to get this right. And we see the primary needs here are going to be being able to limit, like, certain more dangerous capabilities, like general, like, write access and being confident you can deploy an agent that won't change database or write to certain files that are important as well as observability. And then you're gonna move into the formalization stage. Like, you're able to have successful pilots with agents, and you wanna increase investments. And now you wanna expand the capabilities and access. Like, it would be great if they could actually write to the code base or write to documents, but you still want you won't need control to protect the most vital assets and maintain security borders. So here, we see that the needs are real time governance and alerting of potential risky actions so that you can actually at least see if there's any risk from the agents who deployed and prevent that from causing harm before it's too late. After that, you might enter the proliferation stage where you have now hundreds of agents in production. And what you really need is a centralized set of tools for governance that will give you observability into what all these agents are doing and the ability for also recovery and rollback, which is something Rubrik is also offering. And then this is kinda looking to the future. Like, you know, there's not many companies that are at this stage, if at all. But when there's thousands of agents being deployed in the system that are making much more autonomous decisions without human in the loop oversight for every decision, then you'll need even more intelligence on top of your basic policies and kind of handwritten governance. So most organizations find themselves in these first two stages, and we've already built solutions for this stage. In this in this at this time, you're trying to figure out, you know, going back to the governance challenge, how do we let our, let teams deploy agents freely and build with high velocity while mitigating the biggest risks and adhering to compliance. Right? And so the way we think about that is you should have some foundational policies on day one that will at least protect you from the biggest risks with agents. So with Rubrik agent cloud, we have some pre canned day one policies that we think cover some of the most vital attack points. And first is going to be, like, a read only tools policy. So you can be sure with this policy deployed that any tools that make right actions will be blocked from being happened. And so other policies, the unauthorized tools. So this way, the kind of AI governance committee can come up with tools and accesses that are completely unauthorized, and this can help a lot with the problem we're seeing around ShadowAI where people are, you know, installing things like OpenClaw or different tools and just giving it Slack access or giving it Jira access. And so at the governance level, the you can completely ban access to these sort of tools, and we'll make sure that agents can only interact with approved tools. And then third and probably most critical for departments like finance or legal is sensitive data detection. And so this is an always on filter that will try to automatically catch and redact PII credentials before they leave your sensitive environments. And so with the combination of these policies, we think that you can already start deploying all sorts of powerful agents without having to trust the user, you know, building the agents or using the agents to make perfect design decisions in how they've configured it. So, you know, here are just some examples of how these policies look within Rubrik Agent Cloud. And, yeah, so now I wanna talk about kind of looking ahead beyond this earlier stage that a lot of companies are at and thinking about what happens when you start scaling to first, you know, hundreds of agents, but even more agents, and you start expanding their capabilities and access privileges. So at this stage, right, these sort of universal policies, like no read read only access, is going to start breaking down. They're safe, yes, but very restrictive. They're gonna really restrict the set of what sort of workflows your agents can enable. And so to really govern agents more intelligently, you have to also understand the context in which they operate and the context for that particular interaction. And so, like, for example, you know, what is safe for a finance agent, which is you know, it's able to read certain, like, bank and financial information could be considered a breach for a marketing agent. Just it's very similar to how we organize, you know, our teams as people. Right? Like, different teams or different types of agents have very different access privileges, and we ideally just want to, give things the least permissions it has to get the job done rather than either stuff that's too restrictive or too open. And this also becomes a gap with so many agents where we talk about, like, the unauthorized tools list policy. This requires some sort of, like, constant manual approval of tools and maintaining of these lists, and that's not very scalable once you have many agents in production. People are building new ones daily with new capabilities. You need something more intelligent that will adjust to the context of what is going on. So this is where we really see that, like, simple rule based governance is not going to be enough. So we wanna think about how we can enforce policies like one Arnav mentioned. Like, do not provide a financial advice or guarantee returns. Like, let's say, you know, I have a help bot for Fidelity. I think, you know, they don't want it to offer too much advice on, like, how to trade because that may not be, like, the role they want for it. Or you have more nebulous kind of long winded policies that need to be interpreted, really, like the EU AI Act. So how can you kind of bridge this gap between these simplistic rule based policies and these more semantic policies that require understanding both what the policy means semantically and what the agent is doing. So just for an example, let's say a policy like the do not provide financial advice or recommendations. Now if you try to build a static rule like a regex that tries to prevent certain phrases from being said, buy, sell, stock, then this is a very basic check. And so if we had a real world question like a user asked, what asset classes would outplace inflation? And the agent responds, allocating funds into low cost index trackers is effective for wealth preservation. Right? I think there's some interesting things about just this example. First of all, you know, it's actually up to kind of the people who are governing the agents to decide, is this necessarily violate you know, is this a policy violation? Will we consider this financial advice? So it's not obvious. Like, there's not a good way to write a rule that can do this. And, really, only by describing the different types of behaviors that are allowed or not allowed can you really define a policy like this in an unambiguous way. So we actually enable you to just write a policy in a simple natural language definition just like that. Like, do not give financial advice. And if we go back to this example, you could really specify in the description, like, even recommend a recommending indirectly by saying allocating funds in a certain way is beneficial, constitutes advice, or you could say not so. And our engine is gonna dynamically understand what the policy means in natural language and decide whether that's a risky thing or not. And so it's gonna understand, like, the intent of the action. Right? It's also gonna look at what the agent was doing in the lead up to responding that way to the user or using a certain tool. And it's not fixated on these are a set of tools or a set of words that are approved for the for the agent to use. And as we look ahead, right, when we this is kind of thinking about the proliferation stage where you might have hundreds of agents. But you as you have thousands of agents in production or more, you need even more intelligence to be able to flag the risky things for you because you can't manually review, all the behavior that is happening in your system. So here, we're looking at, how we can build an detection to go beyond even just policies that you define, but also catching things that you wouldn't expect, really the unknown unknowns. So a system, this can automatically learn, like, what normal behavior is for an agent, looking at the interactions it's had, the access privileges it has. And, for example, you know, you can automatically file flag outliers like a customer support agent accessing a database and downloading its details at a strange time that's not normal. And so the idea with anomaly detection is even more intelligent security that will detect threats that you didn't even know were threats before they happen and alerting you to them to manage manage unknown risks that may occur. So now I've talked a bit about the different elements of agentic maturity and how we're thinking policy enforcement fits into those different parts of your use for agents. And now I'm gonna hand it over to Jeff to talk more about the natural language policies and how that works today in Rubrik Agent Cloud. Thank you, Sameer. Okay. So for the next few minutes, we're going to talk about how policies and observability handled within Rubrik's agent operations platform. We're going to cover, you know, what what the existing policies, the the sort of standard out of the box policies look like, how you can see violations policies, and how you can create your custom policies that are tailored to your org's needs. So here, what we're looking at is the is the policy list, for our organization. You can see that we have a blend of custom and system policies. System policies come out of the box with with the agent operations platform, and custom policies are created by you to tailor agent behavior to your needs. Out of the box, we have a read only tools policy. This is suitable for sort of starting out in your agentic journey, limiting the amount of risky behaviors that can be taken by making sure that agents can only perform read only options. We have a tool use hallucination policy that looks for attempts to call tools that don't exist. Hallucination being a standard problem that generative models run into. We try to catch that early and make sure that, your models are not trying to take actions that they cannot take. We have a PII detector. So if you have sensitive information in that that your agent may have access to, we look at the outputs it's it's returning and ensure that no sensitive information, exists in those responses. We have a, this minimal authorized tool scope, policy that looks at the tools made available to an agent and informs you of whether or not those tools are appropriate for the tasks that the agent is, try is tasked with carrying out. We have a mandatory ownership attribution, which states that all tools need to, all agents, all tools need to have an associated owner, either either organization level, one, person one, user owning owning the agent, what have you. Just somebody has to be responsible for it. And lastly, there's no unauthorized tools, so you can maintain a list of tools that you always want to block and, that you would always like to block and, make sure that they are never called. You can see that some of these have, have violations, have alerts. So these are instances where agents have taken actions that, that you don't want them to take, and we have either blocked them or noted that the agent tried to, tried to take an action you don't want, and have, raised some flag about that. If you'd like to dig into these, the these alerts and violations, you can go over to the violations tab. We have, we have a list of every violation that we have run into. You can see that, as of now, we have several, PII detection and mandatory mandatory ownership attribution violations. So let's take a quick look at this violation. So this is a mandatory ownership violation. It's a it just lets you know, hey. We found this agent. There's no owner associated with it. Here's the name of the agent as PII Health Care agent. And, yeah, the action that you need to take is that you need to assign an owner to ensure accountability and proper access controls. And you can you can open you can leave these open. You can, make the remediation mark the remediation is in progress or, or done, or you can dismiss them if you're not worried about it. Similarly, we have, this PII detection, violation where a social Social Security number was found in an agent response, and this is raised to you as potentially sensitive data exfiltration. So you can come in and, work on, resolving that issue. So, thus far, we've looked at these these sort of standard tools that we have. We, these, like I said, come out of the, come with, with the platform, and they're fairly generic. They're they're designed to help you get started, help you, take say, help guarantee your safety as you're taking your first steps into the world of agents. But this also means that they're they are restrictive. They're not well well tailored to individual needs. And so, like Samir said, we also allow you to create policies. So on on this policies page, we're going to, we're going to create a new policy here. As an example, we're going to create a policy that says, agents should not send emails to recipients outside of the rubric.com domain. And this would be sort of as a, as a next step after you have verified that your agents that are, that are communicating by email, right, are able to do so are able to do so moderately safely, but you still want to put some restrictions on what they're able to do. So to start, we put a short description of the, of the policy in natural language. You just describe what behavior you would like to enforce. Let me hit next. You'll see that we're we're doing some some, loading here. And what this is doing is it's, it's going to the, to our to our our our own agent behind the scenes and generating some some initial details about the policy that will help to, to define the scope, to define, the behaviors that you wanna see, to define, examples of of safe or violating interactions. And you'll see that what we do what we give to you is a, an assessment of how strong the policy is and some recommendations on how to improve it. And we'll go through each of these, each of these steps and strengthen this policy before actually enacting it. So the the high level instructions here, what the policy or what what are our, what our agent decided that the policy the policy description you provided mean I provided means is that this policy prevents the agent from sending emails to any recipient whose email address does not belong to the rubric.com domain. The intent is to restrict outbound email communications to internal only recipients for data security and privacy. That's a pretty good summary of the intent behind behind the the policy I wanted to create. So let's see what recommendations we have to improve this. Right? So, here under the key definitions tab, we have, a list of of terms that appear, and and an interpretation of what, these terms might mean. You can see that that, we have a couple of recommendations here about how to improve the the, understanding of these terms. So what we're going to start with is, we're going to confirm the definition of the of what, you know, an external recipient is. So I'm going to start with this recommendation here, any email address not ending with rubric dot com. It also said wants us to confirm some, email sending tools, and we're going to take this recommendation as well. Yes. And then finally, the recipient is defined as an email address appearing in the two c c or b c c fields, which is, sounds correct. We're then going to come down to this behaviors tab, and what this tab, does is it has a list of, of of behaviors that violate the policy that are safe. So we're going to we're gonna go through these really quickly. So violating content, sending an email where any two cc bcc field contains an email address that does not end with rubric dot com. Fantastic. Using an email sending tool to message both internal and external addresses together. That's a good catch. We, we didn't explicitly say that, but that's a use case that we would want to account for. Now in terms of safe behaviors, sending emails exclusively to recipients with addresses ending in rubric.com. Yes. That is very explicitly what we want this to do. Sending emails to multiple recipients, all of whose addresses end with at rubric dot com. Also great. The last one, not invoking any email related tools. That isn't quite what we're looking for. So we're going to remove that from the behaviors, list, and we're going to reanalyze. And by following the definitions here, we have, by following the recommendations here, we have strengthened the policy. We've hardened it, to make sure it is more specific that it is, it has examples of email send, tools, and the it has very specific fields to look for in the emails. You can see that there are still recommendations, so we can clarify whether subdomains are considered internal. That's also a good recommendation. So we can so we can say or any subdomain of to try to to try to to make the make it more applicable. And then we have some recommendations for some examples that we can that we can provide to the model. So this last tab that we didn't look at, before reanalyzing is a list of generated examples of, safe and violating tool calls. Right? So example of safe one, we have a very generic send email to alice@rubric.com. Great. That's safe. That's an internal email. We then have a violating example. That's smtp.sendtobob@gmail.com. That unsafe very clearly. So these are these are good good examples, that have that capture the intent of the policy we are trying to write. I would also like to add one. Actually, going to we're going to test another example. So we added, we added handling for, subdomains. So let's test that and make sure that the policy is working as intended. So we we run our test here. And, normally, what would happen is, we'd pop up with the actual response you would receive from our, agent our our policy guard agent. For now, we are going to, just add this to the safe examples because we're pretty confident that that is, that that is a safe example, and it does capture the behavior that we wanted to to deploy. So now that we have, now that we have these examples, now that we've refined our policy to be a bit stronger, we're going to move forward and finalize the policy by saying no e calling it no emails outside of rubric.com. And we can add a description. We're going to enable this out of the box, the source. So the this policy will be judged will be enforced using LM as a judge to analyze tool call content. And then lastly, we can, we, as of now, can say say that we just want to monitor for instances of emails outside of, sent to recipients outside of rubric.com and to, to raise an alert whenever this happens. Coming soon, we'll also be able to block these types of emails. We'll be able to to say, my custom policy, I want to I want to block an action from happening, rather than just raising an alert. And we're we're currently working on, making making these real time LLM as a judge custom policies a reality. So let's save this, and and there it is. We now have our new policy. It's enabled. It should be it will be shortly active in your environment, actively monitoring agent operations, for, instances of emails outside of rubric.com, and you can start to look for violations for alerts here. Now, just a few other things that you can do with this policy. You can you can disable it if, say, it starts if it starts generating false positives, you can turn it off, say, okay. We're we're gonna work on this a little bit and then come back to it. You can also edit the definition. So let's say that you have another, another domain that you'd like to whitelist. You can go in, add that domain, and, rerun tests, add examples to, refine your your policy. And then lastly, if, if you no longer need it, you can archive the policy, and it will be removed. You won't see it in this list anymore. So that is a quick overview of what we can do with policy of the types of policies that come with the with agent operations and how you can create a custom policy to meet your org's needs. So we are going to just very quickly move over here. To close out, just a few best practices regarding custom call policy creation. So firstly, be explicit. You saw that my first attempt at creating a custom policy, was a bit too generic. So, you know, rather than saying, like, saying something like agent should not communicate outside of my org, you wanna say you wanna say agent may only see send emails to recipients in the rubric.com domain, or or similar. Right. We're better able to to generate policy descriptions, behavior descriptions, examples if you are if you are explicit about what you're trying to accomplish. Similarly, you want to be precise. So our first attempt to write the to write the custom policy, we just said agent may only send the emails to recipients in the rubric.com domain, and there were some ambiguity toward what rubric.com domain means. So we went back and and at the recommendation of, agent ops, we said, okay. Let's also include subdomains and just be be specific about including subdomains, including adding an example to show that that's safe. You want to provide a variety of real world examples if possible. So on that examples tab, we generate a number of synthetic examples based on your description. But if you have real examples of violations, of policy violations, of actions taken that you do not want the agent to take, you can provide those as examples, and those will be submitted to the policy guard, as context for for carrying out its tasks. You're going to want to test frequently. So as you're monitoring, as you're seeing alerts and violations come up, you're going to want to go back and test your agent more. You're gonna want to see where its breaking points are, where it, doesn't quite catch edge cases, and just generally try to try to harden its, its behavior as, as much as you can. Similarly, refining based on observed violations, go back and and add actual examples of, of violations that you've seen. And finally, we suggest improvements. Take those suggestions. We try to we try to provide quality quality suggestions for making sure that your agents are are acting in safe and and predictable ways. Just to wrap up here, LLM based, based policy enforcement, can be could potentially be slow. It is probabilistic. So, so there's a chance that it could get a policy wrong. It could it could raise false positive. It could it could ignore a false negative. So what we did was we we put our policy guard head to head with GPT 5.2. We created a a set of policies and a dataset of, safe agent behaviors and violating agent behaviors for those policies. And then we we did this head to head test where each model was required to correctly classify the, the each, action as, safe or violating one of the policies. And what we found was that Rubrik PolicyGuard outperforms GPT 5.2. It is, coming up on 95% accuracy, and we'll only get better as we continue to refine it. And it's also much faster. So it's about 4.5 times faster. Ruvert PolicyGuard is about 4.5 times faster than GPD 5.2, which is great for you if you're running, your own, say, your own instance of, the policy guard, that'll save you money. And, also, we're closing in on on real time policy enforcement, and we'll be, we'll be implementing that for, natural language policies for custom policies in the near future. So with that, I'm gonna hand it back to Michael. Michael, I think you're muted. The awkward mute button. Big thank you to our speakers. We have a few minutes, so we can do live QA. Before we jump in live QA, just wanna reiterate again that the product is just about to exit, actually, the private preview period into GA. But you can go to the website, and we'll put that in the chat, you can learn more about the product. There's a self serve demo there, and you can also sign up to, you know, start getting access and learn more about it. With that in mind, though, let me jump right into questions. We have a few minutes left. Let's try to get to them. So there was one initial question just about, like, where does actually Agent Cloud sit within the AI stack? And it it enables it to enforce these policies. How does it, you know, embed itself within sort of the routing process? And, obviously, there's different types of agents, endpoint agent, applications. How do we kinda get all that? So how does that actually that piece work? Yeah. Happy to happy to take that. So I think, Michael, I might need your help. I think there were, like, multiple parts to your question. that you you brought back. Let's take it one by one. So first is where does Rubricate Cloud sit in the stack? It sits sort of between the agent implementations and where the models live, where your you make, yeah, LLM calls. And the idea is that it's sort of like a proxy server, so we can see all the requests that your LLM wants to make and all the responses that the LLM actually made in your agent workflow. Because it lives in that part of the system, we're able to derive a lot of information such as what the sort of purpose of your agent is, what tool that has access to, what tool called it actually made. And equally, because it lives in that part, we're able to enforce policies either at the input time, at the tool call taking time, or at the response time kind of across various different agent builders, including custom agents that you might have built. And what was the second part to that question? Yeah. And so it sits at, like, a proxy level. Can it integrate into, like, existing gateways, its own gateway? Like because there's and, obviously, there's, like, endpoint agents and stuff like that. Yeah. Yeah. So so we we support a few different things. You you can bring your own agents if you build them through custom frameworks for your own. We we offer sort of a Rubrik agent cloud managed gateway. If you have a light LLM gateway, which is a pretty popular choice that a lot of people use, you can also integrate it with that. And we support a lot of popular agent builders, Copilot Studio today with the goal to add about eight to 10 more in the next few weeks. So all the popular builders will be in there. And regarding endpoint endpoint agents, for those who are unfamiliar with that term, essentially, those are the AI agents that live on your desktop or laptop. So you can think about things like Cloud Code or Cloud Desktop or even ChatGPT Desktop, maybe other tools that that also are agents and have access to tools. And we actually let you, you know, integrate with with those as well so that you can at least, you know, at the IT infosec level, see which users have access to what endpoint agents. And soon, we're also gonna be able to govern those kinds of agents as well. Awesome. Only have a couple minutes left. There was a question about policies for, like, compliance frameworks and stuff. Curious your thoughts on on that and support for that. Yeah. So I I can take this one. The so the the out of the box policies that we have created were all, based on AI compliance frameworks like OWASP, like MITRE Atlas. And we're we're continuing to work on developing more policies that are that are derived from that directly address risks defined in those frameworks. Yeah. It's coming soon. At some point, we will we will want to, actually, link those policies to the, to the risks, in in the corresponding risks in frameworks, the corresponding, compliance items that they address just to make life easier for for everyone who has to worry about compliance. But also for now for the time being, you can if you have a specific need that is derived from one of those lists, you you can go in and, create a custom policy that tries to, tries to address that risk. Yeah. So two two in particular that we, that that we have actually sourced from a wasp are the tool hallucination detection, policy and the PII detection policy. Those are addressing tool hallucinate the tool hallucination risk and the data exfiltration risk, specifically. And, yeah, there will be more explicit links to to those, those items, compliance items, going forward. Awesome. So I know we're up on time, and I wanna be sensitive. We did try to get to most of the questions in chat. And if we didn't, like, I saw someone asked about a technical architecture. We definitely have those. You can reach out to us later. I know some folks also asked about, is the platform deployed? Answer is yes. We actually Rubrik is actually customer number one. So it's in deployed with our AI governance committee, we have at least half a dozen others that are or early partners that are deploying large mid sized enterprises. So those are some quick hits. But if we didn't get your question, you can email any of of this, any of our speakers here or myself, michael.artega@rubric.com. But big thank you to our speakers. Wonderful session. Love all the context on how to think about policies, maturity, and how to then build custom policies. And then thank you to our audience for joining today. And we'll catch you, next week on the next one. Thank you.