Building Secure Machine Learning Environment in Amazon SageMaker

okay so um my name is Wendy hja I'm a devops engineer and as you can like guess here I have like a passion for cyber security especially for cloud security uh and today we're going to discuss about machine learning and how to build a secure one in Amazon Sage maker um so first I want you guys to imagine this um let's say that you have like just built an amazing machine learning model that's potentially going to revolutionize your business or it's like a nice model you want to sell or I don't know something you're searching for PhD and stuff like that and we get like we get we get the we get the trend like creating a machine learning model

very nice model okay but there is something we should keep in mind which is how secur is your machine learning model and this is something that's not always considered 100% % and um that's why we're going to discuss together how we can build a machine learning model in Amazon Sage maker that's your project not going to be only efficient but also secure like we don't want to have those unauthorized access that everyone just talked about like destroying our BL brilliant ideas so um of course uh we have to firstly understand the importance of security in machine learning like as we all of us uh that we are here we understand this the importance of security in our

projects because I see security as um I'm going to take an example like as a house uh you can have a very like beautiful well-designed house but if your house does not have strong walls that you don't even have to wake to wait for an earthquake to happen because it's just going to collapse anytime soon so the same thing applies for security in machine learning and in machine learn learning is more delicate because for example uh we might have a healthc care company that uses machine learning model to predict clients diagnosis based on um medical data so what will happen if someone is gonna get access to somewhere he should not and uh let's say predict

how uh the model expects the pattern of the input first um privacy violation 100% And probably legal issues here but most importantly if someone um with an authorized access can understand uh the pattern of the input then it will resolve in invalid diagnosis right they can manipulate the diagnosis hous es and we don't have a trustworthy a trustworthy system anymore thus probably Financial damages uh for even the company but also for the client and possibly it might be dangerous cuz the client might even like uh die so um there are challenges in security machine learning especially in Cloud it comes with lots of benefits but it has it own challenges so first is data privacy protecting sensitive data

sets from leaks or unauthorized assets later on we're going to see like a flow of building a machine learning in sagemaker and we're going to understand that data sets uh it's a very important part of building it data sets contain very sensitive information that we don't want to we don't want to get leaked um then we have model Integrity so ensuring that the model hasn't been tempered with during training tring or deployment training or and deployment are uh both two very important processes when creating a machine learning model in Sage maker we're going to see it later on in details and also access control something that was mentioned a couple of times during this conference so managing

who can interact with your data and models and here I don't only mean uh people which is of course important because it's like uh it's like giving your out nothing personal with the a is here but giving your out the keys of your house she doesn't have to to have the keys of your house even though she's going not going to do anything with it so um but here I also mean um managing data within your project within the components of your projects which we're going to see later on so here is like a more uh generate uh example data how the data is being processed and the steps that it follows uh in machine learning so the

fetch clean and prepare is the the process when the data comes when it's still in storage um in Sage maker we store the data in a service called S3 buckets so here we just like take the data we clean we analyze it and stuff like that and then we prepare the data to create the model then we evaluate the model and we prepare it to deploy in produ ction or whatever you want it and then we have the monitoring collecting data and everything goes on and on uh we should understand that the security risk is in every phase of this one like the security risk can even be when your data is like stored somewhere someone can

take advantage of like the security issues you're having in your storage and can take advantage of it this is a more in detail but later on we're going to see a more Det da one uh schema of how the machine learning model in Sage maker is working so here we can see that we have two different clients one has a direct invocation to the sage maker endpoint in Sage maker um the model is deployed into an endpoint so straightforward to the endpoint and then we have um the model which is deployed to the endpoint as I just said then we have another client which actually is us is using another service which is which is uh API Gateway

and then it will go straight to the end point here we also have uh AWS nro and slaves which is a service we don't have to go much deeper inails into it but we have to understand that this service uh is used to um isolate a part of the model or something that you want to isolate within uh your project uh and later on we're going to see uh why that's that's important so um here we can we we can make the connection at what I just said earlier that we have the case in which our model is interacting with another service which in case is API gateaway so what should we do into into um the scenarios

here so here we have how um the model is created in Sage maker so we are creating an instance which in Sage maker is called a notebook um after creating the instance we then like store the data somewhere as mentioned uh before and we create the model which is the algorithm the patterns that we want to use for our um machine learning project after the model is ready and what do we mean with ready we will see uh we deploy to a sagemaker endpoint and here we can see the walk through of the data until it goes to the client we can see that we have an encrypted result and it goes to the client we can see

that the client uses a private key to decrypt the data that comes from the model and the same thing goes here so we can understand that the client is the only one who can see the encrypt the decrypted data using its private key and this means that the data is scripted starting from the scratch so starting from um after creating the notebook instance it is stored creating the model Etc so this is a more in detail schema so training the data set we have the data in S3 bucket stored we are analyzing probably experimenting with it until we make sure that's the data we want then we store to S3 bucket which is encrypted starting from them then we are

using Sage maker to create the model then we are pushing to the end point so here we can see the link that the data is encrypted even uh when the client sends the data to the model here we can also see the reverse side of what I just explained so we have the data being encrypted from S3 to the client so we also have to mention some security features that might be um basic but I've had cases in which um personally I've been asked just give him the access like he's not going to do anything right which I personally don't think it's a good practice everyone should have least privileges as possible and no one should have access to

everything so um we have the access control with as as mentioned I think by Daniel he gave a very good explanation of how I am service functions in AWS which we use roles and policies to assign to the user so we have two um possibilities here you can either create um a role of yourself like create a customer role or policy that you want or you can use um policies that are already created and managed by AWS so it is just a matter of choice depending on like the infrastructure that you're going to build then we have encryption which in Amazon Sage maker goes in two phases so at first we have at rest encryption here sagemaker is

using KMS which is a service to create keys and uh they are used to encrypt data in S3 buckets or ABS volumes then we have the data in transit where data moving between different components are encrypted using TLS next we have Network isolation so it's very important to um isolate your project or even components within your project so that let's say if someone is attacking a part of it not all of it it's going to fall down so in this case we can use um vpcs and we can use security groups there in which uh is a set of inbound and outbound rules so you can Define the the range of ips or services that you

want to access your model and then you can also use Access Control lists uh as usual so you want to keep training jobs endpoints and other resources out of the public internet and next we have um secure deployment so after everything is like done of course we don't want to mess at the last step which is deployment so we need to keep that level of protection uh also there so in this case we have um encrypted in uh end points which makes sure that the data flowing in and out are uh that are deployed are secured and protected then we have access control on endpoints in which you can configure your endpoint to receive access uh from whatever you want

let's say that someone is attacking your endpoint and you're going to it's going to say okay I'm receiving request from here am I expecting to get that request if yes okay walk through if no go away so um it's a pretty pretty nice feature to have and of course some best practices for building a secure machine learning environment of course apply least priv uh least privileges uh access so as mentioned um we must ensure that users have only the permissions that they need um this might sound very basic thing but I've seen this as something that's really um not considered that much as it should and as a result is like causing damages that might take time or even

might not take time to to solve um encrypt everything so that's the data that we want that we want to protect the data should be encrypted and protected at every phase of your um machine learning learning model so that no one can take advantage of your security um issues and um yeah we don't want to have problems there then we have regular audits um of course monitoring is a very important part of um identifying the issues and this is something also I've noticed is lacking um sometimes my we might uh just depend on daily monitorings or manual stuff but it's very important to have as much things as you can being automated this means creating alarms not only when

things are failing or you predict things that are going to fail but also when things are like acting as usual like you have you might want to understand even the pattern of how your model is functioning but also when there is an issue it will be easier for you you to identify uh why is why why it is failing what's causing this to fail and stuff like that so in AWS are uh a lot of nice Services one of them is cloud trail uh you can use cloudwatch or even AWS glue um for reporting and also for um yeah also for reporting purposes um then um secure the entire machine learning pipeline so we have to

understand the security isn't just part of the process so in this data driven world uh having a secure machine learning model is not a nice to have thing it's a musthave thing so secure your data your training jobs and points and your overall overall infrastructure um so that we we don't want to we don't want to get things messy uh we don't want to be repetitive of um the issues that we um are encountering insecurity so you can see this schema where there are real world examples of machine learning uh we should understand that these are very uh Dynamic Industries and it's very it should be um very important to follow best practices of security in cloud and not

only so thank you for your attention I try to keep it as short as possible so that everyone can enjoy um if you have any questions you can reach out to me or yeah during lunch break maybe [Applause]