Transcript for "If a Bad Delete or Rogue Agent Wipes Your Iceberg Tables, What's Your Recovery Plan?": Hey, everyone. Welcome to If a Bad Delete or Rogue Agent Wipes Your Iceberg Tables, What's Your Recovery Plan? I'm Anthony Fiore. I'm a cloud architect with our cloud product management team. And I'm joined by Hey, everybody. Mike Preston. Work in technical marketing here at Rubrik in charge of cloud and automation. Thanks, Mike. And we're really excited to tell you more about, Iceberg and recovery and how Rubrik does that, for you. So let's take a look at our agenda. So we're gonna talk about why you need to protect your lake house infrastructure, how Rubrik protects your data lake house protection. We're gonna get deeper into some of the technical contents, kinda go, a little further into architecture diagrams, and then we're gonna finish up with a demo showing exactly how this works, in our Rubik's Security Cloud console. With that said, let's get started. So here, your business really runs on data lakes, and we we called it data lakes in the beginning. Now it's kind of evolving into lake houses. Essentially, we're seeing customers just put a heck of a lot of storage into their object storage repositories, like Amazon s three or Amazon s three tables, for example, right, for for iceberg protection. And we're seeing that everything lands there. Right? So, it used to be just one monolithic pile, but now it's a lot of different data doing a lot of different jobs. And what we're seeing is that enterprise unstructured data growing at about three x faster, than structured data, as per Gartner. So, every every event stream, every file or or object, depending on how you wanna refer to it, backups your application data, every source is essentially flooding, your your object storage, and it just makes it harder to predict. And you do have to protect it. Both AWS and Azure guarantee durability. 11 nines in s three and up to 16 on Azure. But durability isn't the same as resilience. Durability is really there to help you understand, okay, if a availability zone and AWS were to go down, in in a region, would you still have access to that at... To that s three bucket, for example? Right? But that's not gonna help you from things like ransomware, corruption, or deletion. So shared responsibility model definitely applies here. Recovery is your job as the customer, not theirs, if data is deleted or there's ransomware. And this is where it becomes even more critical, if you've got a data loss. Right? So now that lake houses have become something from a nice to have, whereas before, say, ten, fifteen years ago, analytics was, say, a tier two or a tier three application. It's not something that you would wanna check check a box for, during a annual Doctor test. Right? Analytics was just there. It was more for business intelligence. But today, that has changed significantly. Right? Your data lake, your lake house runs your business. So it's it's part of your business. So if there is some sort of an unfortunate event, whether it be, a bad actor, piece of ransomware, or just as we see these days, what's what's more common is something like, where you need to do an operational restore because someone mistakenly deleted something, right, as part of the deployments or sort of a fat finger sort of scenario there. Right? And this... All of this can really impact, your business. And, you know, as you see on the bottom there, some enterprises report, you know, losing more than $300,000 an hour in losses, some exceeding a million. So that that can really be, impactful. And we see that this critical data is is rarely protected. So AWS has native tools like versioning and replication with s three. That's great, for general purpose... For s three general purpose buckets. Right? Your standard classic s three bucket. So... But that really versioning and replication, especially replication, that solves for a different problem. Right? If we're talking about ransomware, for example. Right? A ransomware appears in your source s three bucket as an object. Right? And if you got replication enabled, all replication is gonna do is replicate that, that piece of ransomware over to your destination s three bucket. So now you're gonna have the ransomware in two places. And if you needed to do a point in time recovery, whether it be using replication, whether it be versioning, even with object lock enabled, that's not gonna protect you because neither of these mechanisms are suitable, for point in time recovery, with s three. Plus, if we we start looking at things like s three tables, which is Amazon's managed iceberg offering, that's very different. That that doesn't have versioning. It does have replication, but it's, but it's really just a one way rep... Replication to a read only replica that cannot be promoted. So, that becomes even dicier. Right? Then there's... On the other side of that coin, if we look at Iceberg... So Iceberg does have the ability to do time travel where you can query snapshots and whatnot. Right? So, this is this is helpful, but if someone was to delete your your your iceberg data, the s three bucket or the s three tables bucket that contains it, it can be it can be an issue, because now there's nothing... Time travel essentially requires those snapshots and that underlying data and the metadata of iceberg to exist. Right? So if if that's gone, so is your ability to to do time travel. And furthermore, we just see... Especially with object storage. Right? Because so much is being put in there. Right? Think of, think of your cell phone, for example. Right? If you... When you take pictures and you upload those pictures on your phone to a cloud, to to to some sort of a cloud provider, like iCloud, for example. Right? How often do you back that up? No. It's in the cloud. It's secure. What what we've seen is typically enterprise customers follow that same sort of process where, okay, it's it's it's there. It's it's in object storage. It's it's off of the source system now. So it's it's good. Right? Again, this goes back to that shared responsibility model. Yes. It's good from a durability perspective, but if it's not being protected and not being backed up, That's a true, business risk. So this is where we introduced Rubrik data lake house protection. So this is this is essentially complementing our existing, Amazon s three protection, where you can protect the underlying objects within an s three bucket, but we add an additional layer on top of that. So for self managed iceberg databases that run-in s three general purpose buckets using glue data catalog, we now support that catalog aware backup and recovery, of that self managed iceberg database. So prior to this, you could just back up that underlying s three data. So the data and the metadata with your iceberg tables, that was that was there and that was restorable, But the Glue data catalog component didn't exist. Now we bundle in that Glue data catalog, piece with that. So that offers you true catalog aware backup and recovery. And it also supports the ability to back up s three table buckets. So this is something brand new. So we can back up s three table buckets with using s three table's native catalog or using Gluedata catalog. That also is supported with s three tables. So, so this this is really important, and I think one of our key differentiators is that the customer, you, the customer, can decide where that data is stored. If you want it to be stored in, what we call Rubik Cloud volts, which is our immutable... Air gapped immutable storage that's hosted by Rubik. You can do that, by by sending your backups to RCV. Or, as we see with a lot of our highly regulated customers in health care and financial services, there's a need to keep that data in your own customer account. So we support the ability to allow you to, to, to back that... Back up your iceberg data into one of your, into one of your your accounts. And, typically, we see customers doing that where it's kind of like an isolated account, with... That doesn't have the same level of access as the applications. So this this can be very cost effective. So we're doing, you know, we're we're... What we're doing here is we're doing incremental forever backups. We're doing... We're backing up your compact snapshots only, and we're also auto tiering these backups to lower cost storage. In addition to some of the data reduction algorithms that we do. So, if you had, say, just for the sake of discussion, if you had a 100 terabytes of iceberg data, the... We we wouldn't expect you to have a 100 terabytes, on backup storage, if if that makes sense. So, and, really, what's nice about this is, again, this is a single platform with Rubik. Right? So it's a single... I know this this term gets gets used a lot, but it's single pane of glass, to be able to manage your... Not now... Not just your compute and storage, like, right, like, e c two, s three, RDS databases, and and Dynamo. But now we're adding that component now, of of Iceberg that's all on the same console and can extend to DevOps protection like like GitHub. Or if you're multi cloud and you've got Azure or GCP components, you can do it all from, from that single platform. This is where we're gonna switch to kind of a more technical, how it works sort of a piece. And for that, I'm gonna hand it over to Mike. Alright. So, thanks, Anthony. That was kind of a great overview as to, you know, why we need to protect our lake houses and some of the features that that Rubik brings to that protection. I'm gonna go a little bit deeper here for the nerds in the room now so we can actually take a look at sort of under the hood how Rubrik, actually protects and restores, our iceberg data on AWS. So I'll start with backup, then we'll move into recovery. But before we get too far, I just wanna bring some attention to a few things on here. We can see a role within the protected customer's account here. This is basically a cross account role. This is how Rubrik gains access into your environment. Right? So we have a trusted Rubrik account where we assume that role. And that cross account role and things like exo compute, which we'll we'll get into, These are all deployed automatically by Rubrik when you onboard that AWS account, into our platform. And onboarding's pretty simple. I mean, we use a cloud formation template that you can simply execute everything that needs to get deployed, gets deployed. You can automate that with things like Terraform, you know, if you're an IAC shop or different things like that support for things like AWS organizations, if you have, you know, a 100 or a thousand different accounts. So there's lots of different options to, to get your AWS account onboarded into the, the Rubrik platform. But the fact of the matter is once you're onboarded, we're gonna go out, we're gonna scan your account. We're gonna discover sort of all of those S three table buckets that you have, you know, all those glue catalogs that you have, configured within your account and present them in sort of a nice list and allow you to protect them. And we'll go through all that in a demo. But before we get in there, let's take sort of a look at at how it all works under the hood. So the first thing we're gonna do is we're gonna go and grab some credentials that are specific to you as a customer. They're gonna be stored encrypted within a trusted rubric AWS account. We use those customers with specific credentials to assume that role and assume that cross account role, which has all the permissions and and different things and policies within your account to allow us to do things like discovery and backup and recovery and all that. So... But it is all at least privileged too. Right, Mike? So it's so it's not... We're not we're not having Rubrik. Customer wouldn't have Rubrik assuming, say, an admin level rule or or something like that. So it's all it's all least privileged in the I'm piece. Absolutely. Yeah. It's all scope.... Security. Actually, like, right down to the workloads you're protecting. So, like, iceberg will be scoped to s three tables and glue. E c two will be scoped to e c two. So there's Yep. You know, if you don't onboard, protection for RDS, we're not going to, you know, grant ourselves widespread permissions for RDS because why would we? You're you're you're obviously not wanting to protect RDS with us. So so we assume that role, and then we just start calling some some native AWS endpoints. Right? So we're gonna go ahead and we're gonna call the endpoints into the the Glue data catalogs, those those self managed buckets that you might have, the s three table buckets that you might have. And we're gonna look basically for, one of three sort of different snapshots. So, when we get into the demo, I'll show you SLA domains, and we'll kinda go through these. But, basically, you know, by default, we're gonna target that latest compacted snapshot. So if you just set up an SLA domain by default, we're gonna look for that latest compacted snapshot. We do have some failback. If if that doesn't exist, we'll we'll failback just to the latest snapshot as well. So that's what these API calls are for is to identify, you know, exactly which table files, which data files have actually changed. Another option is gonna be the latest tagged snapshot. So this is especially useful if you do things like, I don't know, like month end or something like that. And you tag those those snapshots and you wanna back those up before you actually, you know, do a a risky change or anything like that, we're able to target those tag snapshots as well. So we'll figure out what's changed, what needs to be backed up. Obviously, first backup's a full, like Anthony mentioned. From there on, it's it's incremental or changes only. And then we spark up what we call Rubrik exo compute. So this is, essentially, a bunch of pods that run-in an EKS instant... Instance. They spin up when we have work to do like indexing or data transfer, compression, things like that. And then when they're done, they spin back down. So they're not constantly running. There's no long running process. They're all transient. So we'll we'll basically spin that up, perform, you know, all the the data processes that we need to do, grab all those different files, and then finally, you know, the backups land on some sort of storage. So that storage, like Anthony said, could be within your own account, could be in the exact same account as the source, probably not recommended. I mean, it's best to create another account if you need to host the data and and save it there. That way it's kind of outside of that identity domain. Or that could also be on Rubrik Cloud Vault, which is essentially object storage in one of our accounts that that we fully manage for you. So that's sort of the easiest path and probably one of the most secure paths to take, in terms of where to store your backup data. So we'll transfer that. We object lock that so it's immutable. So your backups are always there, you know, when you need them. If a bad actor gets into your account, they're not gonna be able to go and and purge all your backups as well because they definitely spend a lot of time in discovery and definitely try and mitigate your chances of not paying any ransom or not not, getting their payday, in terms of looking for backups and and things like that. Finally, we just send metadata back to RSC. The data itself, again, always managed by you either in your account or, you know, stored within a Rubrik Cloud Vault account. But our SaaS solution, Rubrik Security Cloud, the management plane only ever receives metadata around things like table names and, you know, point in time dates, different things like that. So that's essentially backup. Recovery, pretty much, you know, just the opposite of that. Right? We're gonna select, a restore point within RSC. We're gonna go ahead and get those credentials. We're gonna assume that role again, and then we're gonna spin up EXO compute to actually pull that point in time out of the immutable storage. So out of either one of your accounts or out of Rubrik Cloud Vault. And then from there, we're gonna transfer all that back to whatever target you, you give us. So in place, basically instructs us to overwrite what you have currently. You can overwrite and create a new branch as well if you don't actually want to, like, overwrite the actual production. Or you can use export, which can create a new table, or a new name space in a completely different, you know, account if you want to or things like that. So that's gonna be a completely different instance of that s three table or AWS Glue catalog. From there, you know, a lot of vendors will stop there. And Anthony talked a lot about this as well. Like, yeah. We we restore the data files. That's great. But they're pretty useless without, you know, rewiring that catalog, that's sitting in Glue or sitting in s three tables. So we're actually gonna take that sort of that next step to basically rewire everything. So after after you perform a restore with Rubrik, you can actually go in and query the data immediately using Athena or, you know, whatever it is that you use to query that data. This is, again, a pretty big differentiator for us, and it lets you get back to business, you know, a lot quicker without having to, you know, grab an engineer or data scientist or whoever manages that to to basically perform that manual work after the fact. So with that, I think we'll just jump into a demo now, and, we can see this in action. So we're in one of our demo environments here. And I do wanna say before we jump in, if you wanna go to rubric.com/explore, like, you can run through this demo on your own on your own time, at your own pace, as well as many, many others. So maybe you wanna look at how we protect RDS or how we protect Azure VMs or Salesforce or M three sixty five, or, you know, VMware, basically everything is, is up there on rubric, explore. And you can walk through all these demos yourselves to to sort of get a handle and a feel for what it's like to protect, your data with the Rubik's Security Cloud Platform. But in terms of Iceberg, we're gonna start with what we call basically, policy based protection or what we we achieve through creating and protecting our workloads with an SLA domain. So an SLA domain in Rubrik, you know, at its basic, is a policy that we create that contains all the data protection constructs that you might wanna set on a certain workload. So things like, you know, how often you wanna back up, how long you wanna keep those backups, do you wanna perform replication, do you want to, you know, archive older data off to different tiers of storage, you know, where those initial backups are gonna go, all that sort of gets wrapped up into a policy, and then we just assign that policy, to our to our workloads within Rubrik. So there's no concept of jobs and chaining jobs and and sort of that legacy sort of data protection, scenario that we used to go through where one job fails and the whole thing fails, and you gotta go start all over again and do everything. So let's walk through sort of creating an SLA domain and what that looks like. Right away, we can see, you know, we can select pretty much whatever we want here. There are some limitations, like, in terms of, being able to protect certain workloads within the same SLA. But for the most part, a lot of customers will create an SLA for, you know, a specific workload and go from there. So we've selected glue tables and s three tables here. And then in this feature, retention lock is probably something we should pause and talk about a bit because, as I mentioned, the backups themselves are immutable. So if a bad actor gains access to your account, you know, they're not gonna be able to go and delete those. However, you know, those bad actors are are smart folks and they've got to thinking, you know, well, if I can't delete them, then maybe, you know, I'll get some credentials to Rubrik somehow, you know, steal those credentials or elevate, you know, permissions and things and get into Rubrik. And maybe I'll just change how long Rubrik keeps those backups around for. Right? If, say, you're keeping them for five years, maybe a bad actor comes in here and says, well, I'm gonna change that to zero. So then Rubrik will expire those snapshots. What Retention Lock does is it prevents that from happening. So even if they do get credentials into the Rubrik platform, if this SLA is retention locked, they won't be able to, change the retention in such a way that will end up expiring backups prematurely or deleting backups. So, if it's a change that you're trying to make, then there is something called form authorization where a couple different people within your organization that you specify will need to come in here and approve that change. And, and only at that point can you actually, go through with with modifying the retention. But for the most part, like, if if you're really security conscious, I'd probably retention lock your SLAs. That way, even if Rubrik credentials are compromised, you know, they're not gonna be able to instruct Rubrik to delete those backups. Next piece is our frequency and retention. Right? And this is pretty flexible. You know, how often do you wanna take backups and how long do you wanna keep them? And that's in terms of hours, days, weeks, months, quarters. Like, there's a lot of different flexibility within here in terms of setting, you know, your frequency and retention. You could control things such as snapshot windows. If you only want backups to occur at certain times, you could control when that first full backup gets taken. You know, because, obviously, it's the one that's gonna take the longest. So there's a lot of different options here in terms of telling Rubrik when to actually start processing all that data. And next, we just say where we want that data to go. Right? So this is a backup location that I've set up within Rubrik Security Cloud. It's basically an AWS s three bucket. I think this one's actually in a different account, so it's not stored in the same source account. So we have this set up in a different account. This could be Rubrik Cloud Vault. This can leverage, you know, different tiers of storage within AWS to kinda keep your cost down. So if you, you know, wanna, you know, tier down to a certain type of storage, you know, maybe cold storage or something like that to ensure that, you know, you're not paying an insane amount of money for, you know, backups that you might not restore but you're keeping for compliance reasons, you can do so that way. And then we get into, you know, what we're gonna target. So here we have our tag snapshots. It's a simple, like, regex pattern. So here we're gonna target, you know, any any snapshot that's tagged with month end, and that's where we'll go to to grab the data. And then, of course, there's compacted snapshots as well. So that's gonna, you know, target the latest compacted snapshot and fail back to just the latest snapshot if it doesn't find one. Here, we simply, you know, give our give our SLA domain a name, a description, create it, and we're good to go. Right? And that's all it takes in terms of of creating an SLA domain. But, really, the next piece is gonna talk a bit about, like, how we protect those. How do we assign those SLA domains to to our workloads? So we'll come into Rubik here. We can see our inventory of of essentially everything we have. We'll go into s three tables. Here's that list we were talking about that that comes up after discovery. So Rubrik's gonna go and grab all of this information for us. And in terms of assigning SLA domains, you know, there's a number of different ways we can do it. First, we could see here we could we could select an AWS account and click manage protection and assign an SLA Domains to the entire account. And this is super handy if you want to target basically every single, you know, workload that you have within there. Every single iceberg, table that you have within that AWS account will automatically inherit the SLA domain you assigned to the account, as well as any newly created ones. It's super good if, you know, I was a backup admin for years and, you know, I need more than two hands to count the amount of times that somebody came up to me and said, hey. Can you restore this application or this service? And, you know, I'd scratch my head and be like, I have no idea what that is. It's you'd be like, well, that's something I built last month and never told anybody about. So guess what? We don't have backups for it. So this would catch that. So a lot of times, what customers will do is assign sort of a a blanket level SLA on the actual account to maybe take a daily back up or a weekly back up even just to make sure that we actually have back... Some sort of backups, of everything within that account. Similarly, you can do it on the namespace itself. So, again, all the tables within, you know, a namespace that we assign an SLA will automatically inherit that. Any new tables automatically inherit that, or we can assign directly to the table. So a lot of times, in that scenario where we sign a blanket SLA, you know, we may come in here and take our more critical tables and assign a more aggressive SLA to them, and that will override, the inheritance. And that way, we can get, you know, more frequent backups of our critical infrastructure while still maintaining, you know, that blanket level of protection. So assigning is pretty simple. I mean, we come in here, click manage protection, select the SLA that we, that we just created. We can select to apply that to existing snapshots. So if this workload had existing snapshots before, we can reapply the new changes to those existing snapshots if we wanted to. And we just hit assign. And as soon as we do that, Rubrik's gonna go out. It's gonna automatically sort of start scheduling the backups adhering to all those constructs, within that within that SLA. If you don't wanna wait for a scheduled snapshot, it is possible. We could see we come in here, drill into our drill into our table and actually, select to take what we call an on demand snapshot. And again, for that on demand snapshot, it's gonna need an SLA. And what this is gonna do is it's gonna take a snapshot basically right now. So, if you're making a big change and you don't wanna wait around for the schedule, you know, you can go ahead and and and kick off an on demand snapshot so you have that have that backup right at this exact point in time. Everything's logged. Everything's monitored. You could come into the events, you know, watch as everything completes. And we can see our snapshot here is completed. And that's essentially backup with Rubrik. That's create an SLA, sign the SLA, take an on demand snapshot if you want to. It's pretty simple, pretty intuitive. But as we know, like, backups, SLA, policies, they're all... It's all great. But, you know, when it when it comes down to it, really what matters is being able to actually recover your data. So we're gonna actually go through a scenario here, as we're in Athena, and we can see this query that's run. We're selecting basically everything from our session events table. And And this is just to kinda show we have some data here. Right? So let's go ahead and, you know, put our bad actor hat on and drop that entire table. So that query is ran now. Entire table's gone. And just to illustrate, like, what Anthony was talking about with time travel, time travel's great if the data is still there. So it's great for, like, an accidental mistake or something like that that just modifies the data. But if we drop the entire table, here's another query. We're actually looking for snapshots for that table, and, obviously, there's nothing there. Right? We can't find those snapshots. That table doesn't exist anymore. So time travel here is not going to help us. You've warped space and time. It's, it's gone. So let's head back to RSC and let's actually show you how easy it is to recover this stuff. So here's our session events table. Let's dive into it. We're gonna go ahead and select a date, you know, in the past. So in this case, the August 26, it's gonna list all the different restore points that we have for that day. You can easily come in here, click recover on one, and here's sort of our different, two different recovery options. So we have in place, which will recover the the table right back to its original state. I should say that, you know, we're just recovering one table here. So if you have a lake house with 350 tables and you come in and just recover one table, we're not gonna overwrite the entire data set. We're just gonna overwrite that table, that you've selected. If you select the name space to recover, we will basically... If you select all the tables, we will basically overwrite all of them. But since we've just selected the one table in place, we'll just overwrite that one table back at the original source. And, export basically allows us to create, a new table with that point in time data that we have or recover it into an existing table if we want to. So I'm gonna... This this scenario, we're gonna go ahead. We're gonna run an export on a new table. We're gonna give it a new branch name so we're not protect... Or affecting that production sort of branch or that live data that we have. And this is all it's really gonna take. Right? So we go ahead, finish the finish the recovery here. We can monitor it again. But when we go back to that AWS console and run sort of that same query, we can now see we have our data back. We actually have more data because we selected a point in time that was that was older than the one we run the query on. So, again, fully queryable as soon as that restore takes place. So, this was literally Anthony and I... Anthony and I did this, and this was, like, after the restore, it was, like, pretty much immediate you could come in here and run queries against that data. So the catalog's all wired up. There's no sort of manual work to do, after the fact. So in essence, that's that's our iceberg protection. So, there's lots more features and lots more coming down in terms of this. When we look at our road map, you're gonna see us expand into other lake houses as well. I don't know how much I'm allowed to say, but, you know, we have lots of different stuff coming out in terms of some of the other clouds as well as AWS too. And like I said, if you wanna run through this demo or any other demo, Rubrik Explorer is kinda your place to be. You can go there, sign up, and take as many demos as you want. You could probably spend a week on that site going through all these different sort of labs and, use cases. So with that, I mean, that's, you know, what we have for you today. I mean, thanks everyone for for coming out. I really really appreciate your time. And, Anthony, do you have any last words? I just wanted to reiterate the thanks. I I I hope you found this this session useful, and hope you're you're as excited about how we protect iceberg as we are. So thank you.