Federated Learning: Schemes, Privacy, and Security by Md Morshedul Islam, Suraj Neupane

[music]

[music]

[music]

[music]

Good morning everyone. Um welcome to our presentation. So in the morning um I mean we listen maybe two three industry talk. So this going to be a kind of u a bit academic talk and yeah I'm used to um when I take the class I see couple of seats are back in so it's always happen so it's the same here as well. Okay. So in this presentation we basically talk about the federated learning and scheme is a privacy and security. So here is the outline of our presentation. Uh I mean um we divide the presentation into group. Um in first part I we will try to give you a bit overview about the federated learning. Maybe it's the new

concept for some of you. So after then we'll talk about our contribution in this area and if we have the times then we'll take few questions from you. So that's the plan. So I think all of us are familiar with the traditional machine learning right? So most of you use that traditional machine learning model that you collect all the data in a central server and then you train the model based on that data and you have a model. So most of the case you can expect that that model will be generalized model and you can deploy the model as a service to provide the service to the different clients. So this is the traditional setup. Uh but it have um it

has some problem I mean a few problems. So one of them let's say the data privacy. So we will bring all the data to a central server. So if we share the data to a central server if the server is compromised uh maybe there is a privacy issue. And there is one new dimensions that we see. This is the regulatory uh concern that you cannot share your data outside a geographic boundary. So in that concerns you may not have the settings where you will be able to share all the data to a central server and even if you have a large set of data so we are in the big data age. So we need to we have a lot of

data we need to transfer a lot of data. So it's also not that cost efficient. So there is a new technique that we call it federator learning. So it's basically let's say that uh the client or the parties let's say in this case it can be a hospital. So that's the best example. They can keep their data locally and train their model based on that local data and then they share the model update not the data with the server and there is a cooperation between server and the client. So after a couple of round of um training so the server as well as the client all will have a generalized global model. We call it as

a global model. If uh the parties train the model then we call it local model. So we will have the local model and by using the local model we aggregate it and then have the uh generalized global model. So that's the general idea of the federator learning. So the one of the important role um in the federator learning training is um one of the algorithm that we call the aggregation algorithm. So we have the server side. So server actually get all the weight or all the gradient from the client. So there is the challenge that how you going to combine those weight and then um let's say the mass them together and assign them to to to a model. So there

are two classic um K algorithm that widely used. One is the fab and another was um it's SGD versions. But uh the idea is simple that you have all the weight from the local model users averaging them together um and then you have a you have a a combined weight and you transfer it um to the client. So it's is it's going to happen couple of R. I'm not talking too details about this uh the mathematical concepts. So that's actually happened couple of round and after then we have the expectation that uh we'll have a generalized global model. So yeah there is frameworks and applications. So can you talk a bit? >> Oh hello uh thank you professor moridul

for the uh comprehensive review on feritative learning. So now that we know what federated learning is, let's look at what open source frameworks there are so that we can actually use it. Uh probably the most popular one is TFF which is a short form for TensorFlow federated. It is used by Google for uh next word prediction and emoji prediction. Um if you use Google uh Google's G uh Gboard uh this is the one they use. Next we have Fate which is I believe uh developed by a company called Weebank. It is a uh decentralized uh federated learning uh framework. So it's not the usual client server one. Instead, it's a peer-to-peer one. Uh it has built-in security features like uh

differential privacy. So it's mostly used for financial transaction and uh for data related to for data that needs uh that extra layer of security. The [clears throat] next one is IBM's framework IBM FL. U they famously use it for uh predicting the outcome. the geographical outcome of covid 19. Uh the next we have openfl, pi swift and leaf which have seen a lot of improvements uh which have seen a lot of use cases in convolution neural networks. So uh they are mostly used in CNN domains. Okay. So if we talk about the application of the for data learning you say see that there are a lot of different applications but most of the case we can see it um healthcare is one

of the main domain because they have some regulatory restriction not to share the data. So um IoT is the another domain and the finance. So we can use the data learning in those domain. Um so I mean when we see um we focus on the challenge in the federal learning we can see all the problem in three different perspectives. So some of those problem that are directly related with the uh machine learning um related let's say most of the fatal learning they actually use the deep neural network but um there is an options to use the classical uh machine learning model um to do the federated learning and based on the data distributions there are

defined way that the data are distributed. So we can divide that one in horizontal vertical and there is um one thing that when we train a global model that single model cannot fit for all the clients. So in that case um uh there is a concept that we call the personalized learning. So let's talk a bit details about the issue that are related with the um I mean the machine learning perspective. So this is simple the way that I discussed before how to train a federated learning. So we can use the our uh traditional machine learning model um to train the federator learning. In that case we'll have the input x uh let's say and y is the output

or predictions. So we have we will have a linear model. So our assumption is that to fit a model linear model um that locally and then share the weight and bias with the server and server will get those weight and bias and combine them together and send it back to the uh client for the next update. So the similar way that it also have the um two uh version one is the uh weight and uh bias version or is also have the HCD version as well for the aggregations. So this is the one of the interesting um way to do the fedal learning the way that we see the training of the fed learning. So is traditionally the

horizontal federated learning because the data is um partitioned in horizontally. Let's say if I am a client I have all the features and I have the data uh from the couple of client or couple of classes. So in that case I will train my own model uh by using my data as a local client and share it with the server. But let's say if the data is not distributed horizontally if uh there is a uh another way that data can be distributed that we call the vertical let's say that I don't have the all the feature of a particular client I have a subset of features and other client have the other subset of features maybe I

don't have the level of that particular client okay other client have that uh level of client so anyhow we need to be combine these two data set uh together or logically so that we can together train as uh general global model. So that is why we see this type of um I mean the data dividation we call it as a vertical partitioning and to train a fed learning in the vertical partitioning uh is a bit tricky as I said that there is uh one of the step that we need to be ensure that is called the private site interaction. Anyhow me and my cl other client we need to have a kind of interactions to uh not the merging the data together

we can keep the data [clears throat] in our local uh devices but still we can assign a logical identity of that data. Let's say that this is the ID one for my data and this is the ID one for his data as well. So just assign a logical ID and then share that ID with the server. So if you share that ID with the server uh you see at the top so the problem will be will be straightforward. So in this case we just uh use our own model. It can be a single model or it can be a part of the model. We just will do the forward pass based on the data to train

that particular part of the model and then when server get the features or the back for this from this forward pass he will this back as well as the level to train the model and we'll get the loss from that one and then he will do the backward pass from top to bottom. So it will be run couple of uh round then at the very end everyone will have a generalized global model. It's simple and straightforward just logically combine the data. So then uh the personalized data learning model. Okay. Uh now that we have established that federated learning is a pretty straightforward uh framework of doing machine learning. You basically have clients sending their weights and biases

to a global server. The global server aggregates those bi biases and creates a global model and it sends that model back to the clients and this process goes on. So what is personalized FL? I like to think of personalized FL as a solution to a problem in federated learning. The problem is is that uh each of the clients have unique distribution of data. What do what we call by that is non IID nonI [clears throat] data. So um basically this difference in distribution comes to the fact that some clients might be missing some classes of data while other clients might might have more of those uh data sets of those classes. Uh picture a data sets having four classes.

Let's say client one might not have any data examples of class 4 but they might have a lot of data of class 1 2 and three. However, another client, say client number 13, he might only have a few examples of class one uh but not enough so that the model can generalize. So, personalized FL uh tries to fix this issue by making sure that most of the clients roughly have the same distribution of data in them. So, the first tactic that we can use is something known as data augmentation. U it's just a fancy word of saying that uh say a client if if a client does not have enough data samples for a class

what we can do is we can take the pre-existing data of that class we can apply some transformations on that data like say if it's an image data we can change the pixel values uh we can do some rotation some scaling and create new data sets create new data examples for that class so that's what basically data augmentation is next is a term called regularization. Regularization is a uh tactic used in traditional machine learning to make sure that the model's weights uh do not get very high and that the model does not overfit. Uh it is basically to uh make the model as general as possible. Uh in federated learning as well uh we can do it very easily by appending a uh

cost appending a new term to the cost function. So we so the cost function plus we uh append the global models weights and the local models weight uh to make sure that uh it doesn't uh the local models don't overshoot uh and uh overfit on its own data. Uh we have some other concepts as well like uh client clustering uh which is to train clients with similar data sets uh together. So it's a more specialized towards federated learning. uh we have some other smaller concepts like metalarning uh multi task learning and distillation which we'll be talking about more in the coming slides. Okay. So it's about the personalized learning. So if I give you an example

let's say that um biometric authentication. So in the biometric authentication everyone have his own biometric data. So it's a single class data. uh we call sometime is a positive class problem. So if you have your own data and you use that data to train your model as a local model. So it will uh basically can decide it can decide based on your data. It it it never see the other data. So when you uh move that data to the server, move that model way to the server uh it just get the view of your data. when others model will come and then you will combine them togethers uh it will not give you the good performance

because all of the model basically bias on their own data so you will not have a generalized global model so this is one way that we can define the nonidness of the uh machine learning but there are other example as well so some cases even um when we talk about the personalized learning uh we see that the scarcity of the data maybe client may not have the sufficient data how we can um give the clients um or how the client can get enough data uh to train that for data learning. So there is one way uh maybe the data augmentation that is the easiest way uh to do or generative machine learning model to to train uh get get more data

set and there is other approach that that say the global data sharing. So the restriction is then not to share the data with the server but they said that okay you can share the data with the server by private preserving way so that other client can have that data to use and then later they use this data uh to train their local model. So there is one of the approach they said the probabilistic approach how to share the data with the uh um with the server in the probabilistic way and then client get those data so that they can use those data to train their model and another one he said that the regularization. So regularization

concept is um a bit straightforward here that when you have a global model and you send back to the train uh local client so client use their data to update the model in that case maybe that model will be biased on that particular data. So you need to be give a guarantee that the model at the local side that will not refer from the global model. If your single update dip far from the global model when you get those model and combine them you will get the different results. Um so there is a restriction that okay get the Google global model but when you train um update the model a bit so is the um baby

step we call so that they don't refer from uh the already trained global model. There are other um personalized federated learning as well but uh due to the time limitation maybe uh let's talk about the some issue uh from the security perspective when you see the federator learning we have the client so this is the one side another side is the server and another is the communications in between these two. So when you see the client side there are uh different type of client they have their own requirements. So maybe the model architecture will be different for the different client. Let's say I'm using the deep learning model, other one using the linear model. How we can combine

them together to have a uh general global uh model and obviously data distribution is the big issue in the federated learning. Um and the resource so when you use the def devices some of the devices can be the IoT devices. So they don't have the enough resources to be part of the uh um training process. So we need to be considered that one as well. At the server side uh there are not too many challenge. The main challenge is the aggregations but still um you need to be considered that if the server is not trusted. So in this case there is a concept we call the secure aggregation. how we can do the secure aggregation at the server side and

obviously the communication um is a big factor because not all the device have the same bandwidth and some device may be dropped due to their their resource demonstrations. So there are a lot of communication issue that we can um see in this area but um we will not focus all of them. We just will focus some of those um simple approach based on the client side and server side problems. Can you talk a bit about >> so uh the FL architecture is loosely categorized into uh two uh into uh two types based on the types of clients that we're dealing with. So the first one is the cross uh device FL uh which is characterized by having uh a massive

client population but but most of those clients are IoT devices or handheld devices like smartphones. Um uh we have a term heterogeneous devices. So it basically means that what kind of the device it is is pretty ambiguous. Uh as long as it has a processor, battery, memory, and it can talk through a network, it can participate in federated learning. Um there's a couple of challenges uh with uh cross device FL. Uh just the fact that they are uh handheld devices, they are IoT devices. Uh clients may drop out during mid-training due to connectivity or battery issues. uh we have a concept of com communication efficiency as well because most of them are wireless devices. So they may not have the most

fastest uh network bandwidth. So uh trying to send all the models weights and parameters in every single update might be uh might be a bit heavy on the network. Uh next is uh system uh robustness. So uh we need to make sure that our FL is very robust and it can actually handle the dropouts and the partial updates coming from those clients. Um privacy of course IoT devices um uh not all of them are very secure against cyber attacks. So we need to make sure that our FL architecture is uh secure uh in both the client side and the server side using tactics such as uh differential privacy and homorphic encryption. Uh so one of the examples of

cross device FL is uh uh the Google Gboard uh which they use uh cross device. So uh the opposite of that is cross silo. So this is a uh this is a much more reliable form of FL where we have a few clients but those uh clients like I said are reliable. So they are large organizations like hospitals, banks and research institutes. Uh uh these participants these clients are usually always online. Um and they have a very good system. So they have a uh good comput uh good compute efficiency. They have good network uh and uh like I said they're very very reliable. Uh however there are some challenges with uh this architecture as well because uh

uh the concept of data heterogenity is also important. Uh again since these clients are let's take an example of hospitals. hospitals might have a lot of data of of um a particular type of disease. So different hospitals see a different uh subset of patients and those patients we are not sure uh about uh the type of data like if it is evenly divided into uh the the diseases or not. So um we need to make sure that the distribution of the data is uh uh made sure. Uh next is trust and governance. So uh these uh large organization mostly have confidential data. So using these data for training machine learning models there needs to be a kind of a

legal framework and legal agreement that goes on uh in this case. So uh the the example is like I gave of a hospital where uh a lot of hospitals throughout the country they can train a model on the MRI images of their patients and that model can detect uh the tumor or what kind of tumor it is uh by combining all the models from all these hospitals. So we talk a lot about the data distribution. So this is the this slide should come I mean that earlier but you see that um one of the paper that uh they did the experiment based on nonid data one and two so that is the two different setting they said that in

nonid data one setting they said that client will have the single class data and nonid 2 is uh that is client will have the two class of data then they try to train the model based on the well-known amnest um c4 and other data they have uh more than uh 10 class. So these two data they have the 10 class and other one maybe the uh 15 or something. So if the client only have one or two classes so the performance reduce a lot. So this is the one form of nonidiness um in the uh data distribution but but there are some other form as well that let's say that I have a um two class data but some

class um I have too many sample but for other classes maybe I have one or two sample if we talk about the um intrusion detection we we may have that type of scenario so this is the one things um that we can call it non ID and even we see the distribution of the feature let's say I am a client I'll have a feature it have a particular distribution other client may have the different distributions even they might have the different level as well and over the time the feature may be change their distributions so there are a lot of different nonidness but the we are dealing the first two I mean that that's are a deal but there are some nonidness

as well so then when we talk about the aggregation algorithm they are not only uh two uh aggregation algorithm. Um we see that these are the two classical but there are other they are trying to focus on the different area. So the let's say that first group basically they focus on the heterogeneity um data and resource heterogeneity. There are some algorithm that actually do the personalized federal learning. Um I mean this is the time limitation. So we just just show you where what are the different algorithm that are available. Knowledge distillations [clears throat] and privacy preserving. So this is interesting. So there are a couple of algorithm that actually work at the server side to ensure the uh privacy of

the data. One is the sack act. So this is one of um the most popular one. Another is use the differential privacy with the aggregation and another use the homorphic. So that is our main interest. recently there are some other recently published aggregation algorithm as well. So you may find maybe 20 more than 20 aggregation algorithm in this area. So let's talk a bit about the privacy security because our main contribution is basically in this area. So we always see that in the federal learning um it have two main concern one is the privacy and another is the security. So and obviously we need to be consider how to defend them. So there are different def

different def different def different def different def different def different def different def different def different def different defend mechanism. So if we uh talk about the security issue so most of the case that we call is manipulation attack that actually happen during the training. So you have the model weight and you are sending that weight to the server. So in the middle maybe the attacker can manipulate that one uh to have his own advantage. So one of the attack is to create the back door and another one let's say that he would like have interest to reduce the performance of the model. So we call that type of attack is the benton attack. It can be one client or a group

of client can manipulate that weight not to uh get a generalized fat learning model or global model. So this is the one directions that we can uh see the attack and other attack is basically the privacy attack we call it as a inference attack to infer the informations infer the informations about the training data from the model training. So there are different type of inference attack that we see and even we we will talk about that one here as well. So you see the manipulation attack. The goal is basically the attacker is the insider. He's the insider and he can um manipulate the update to create a backdoor or uh to degrade the overall performance. So there are some attack

that we call level flipping and others. So what is the challenge to uh tackle those attack? Uh so one of the challenge that the server uh who actually aggregate everything he doesn't have the original data client data he doesn't know that which update is malicious and which update is not malicious. So there is a way to detect uh that one. Let's say that um you you can analyze the behavior and then see that which is far from the traditional behavior but still it's not that easy to to to find that which is uh malicious which are not malicious. So uh and you see that they said that filtering approach adapted training accountability but um I mean

that uh single defense is not enough. So another attack that we call the uh privacy attack. Our main focus is basically on the privacy attack because this privacy attack is easy and straightforward. So it can do by the insider or outsider as well. So there are different type of privacy attack that we also see that additional machine learning model in a membership attack in um model inversion attack and extraction attacks property inference attacks. So the problem to defend the system from those attack is that when you have the update so it seems like innocent but it may carry a lot of sensitive information about your local data. You don't know that how much information that update

carry about your local data. So if the attacker have that update, he may use that um update or um use that information to recover uh the private private information or even the attacker somehow can have the access of the model or if attacker can interact with the model so there is a chance that he can uh recover the privacy sensitive training data. Let's say that for the medical case. So there are uh different different approach. Some of them use heavy cryptographic approach but we usually like the non-cryptographic approach. Let's have the differential privacy that is the one of the uh widely approach that we are also using in this area but there are couple of other

approach as well. So this is the overall uh I mean that idea about um the fat learning. So uh I think we have uh 15 more minutes to talk about our contribution in this area. So we have two projects. So one projects we are using the medical data uh MRI data to train the federated learning but our goal is not to train a federated learning model. Our goal is to uh see that how uh generative attack or canvas attacks uh can use to uh recover the privacy sensitive data and how we can defend that one by using the differential privacy and second project is uh to use the resource constraint IoT device. So IoT device they have the data

uh but they don't have the sufficient powers uh to participate in the federated learning. So how we can use them uh to do the fat learning in this case we consider the intrusion detection system. So we face some issue the one of them is the nonid issue the scalability issue and the privacy challenge. So we'll we'll show how um show you how we deal that one. So can you go a bit from? >> So the whole idea behind federated learning is that uh individual clients on even the server does not have access to the private data the private training data of those clients but more but a concept known as model inversion challenges that it says that given the

weights and param weights and biases of a model you can actually uh re reverse engineer it and get the training data from it. So we put that to the test. So we we we proposed a DC GAN based model inversion attack for FL systems capable of generating synthetic samples that closely resemble private training data for specific training class. So basically what he's saying is that our attack will be given a target model with its weight and parameters and given a DC GAN. So DC GAN falls under the family of generative networks. It is basically a deep convolutional uh generative adversarial network which uh which when you train it on a data set it captures the distribution of that data. So when

you give it a latent vector it can actually generate uh synthetic images that look very close to the images that it was trained on. Uh so using the GAN and a target model we have shown that it is indeed possible to um recover some of the training images that was used to train that target model. Um as a part of this research we have also showed a couple of defense uh strategies. The most uh the most obvious one is we use differential privacy uh and we showed that uh we can mitigate some of the model inversion uh some of the uh [clears throat] effects of model inversion if we use differential privacy. Uh we also analyze this with

different uh federator learning algorithms to see how different algorithms work in this uh domain. Uh and we performed all of this experiment in medical MRI data set. take care of the rest. So, okay. So, we are talking about the model inversion attack but let me explain we are not the first who is used again in the traditional machine learning people use again a lot to recover the data. So, we just try to apply that one in the federal learning domain and differential privacy people use a lot in the different areas. So, we just use that one in our domain. uh the new thing is that we just we see that there are different aggregation algorithm. So we

see the effect of these two things dig and and the differential privacy in on those algorithm u and this is the one things that we see and another things that we try to use uh medical image um in this domain to see how to do that one. So you know the left figure if you see the figure in the left side is the traditional machine learning model I'm sorry traditional fedal learning we did the fedator learning uh but the right one is the attack so in that attack site we have a pre-train dig so why we say that this is the pre-trained dig so let's say that um we have some auxiliary data let's say that uh this fed learning

is based on the MRI data so anyone have the access of the MRI image from the online and they can use that uh publicly available MRI image to train a pre-train a digan to have some MRI image but the problem is that that MRI image generated by the digan it will not be enough to have the access let's say that if the fedal learning is the classifier it will not be those image will not be enough to uh be a part of a particular class because the feature space there is a feature space so that feature may not be close to the original features. So in that case um so how to how the pre-trained is again actually uh

generate the image is basically from the noise is we call it as a lat vector. So we feed the noise and the pre-training and can generate the MRI image. But the problem is that to attacking this system we need to get a feedback from the global model because global model have the information about the private data that are used to train this model. So that is why we need the feedback from the global model. So in that case let's say I'm a client um global model provider service. I can uh give image to him as a query and he can uh give me a feedback. So in our case the consideration a bit relax. We just

considered the loss. So most of the case is said that yes or no. But in our case we said that let's say that anyhow the attacker have the access of the loss functions. So he will use that loss function to update his uh latin vector. So that is the main goal. So if we have the good enough latin vectors so that latin vector will be sufficient to uh um uh that latin vector if you provide it to the pre-trained again so the image that we generated that will be sufficient uh uh sufficient close uh to the uh original MRI image that we have. So we'll show that one in the experiment. So if you see this uh result the left

image so that are the real MRI image the right group of image that you see that are the dig and generated image but without the feedback let's say that is the preend again so it's look like these two image are close uh but they are originally not if we consider it in the feature space so if I provide that image to the model model will will classify it as a random class so because of the feature space so we need to have the feedback. So this is the feedback that we get and we have the again the image but in two different uh condition. One is the nonDP conditions and the global model is not trained with the

differential privacy and other one is the global model is trained with the differential privacy. So if you see the trough image that is the original image and the bottom row are those image that are generated from the digan base attack but uh uh the uh global model they use the different aggregation algorithm. So we see the effect that how different aggregation algorithm effect on that one to generate the image but this um doesn't give you that much sense but if you see this distribution that's going to help you that how close the artificially generated image with the original image. So what we did we uh calculate the cosine similarity of the uh features that the real image and the

artificially generated image for the different aggregation algorithm and we then draw them in the box plot. So we see that um not all of them let's say the sum of them almost 80% close. So 80 or 70 above 70% close but when we see the diff uh we add the differential privacy with the model then we see that the closeness between uh these two means they become uh less. Okay. So this is the expected but obviously so there is a cost related to that one because uh if you don't use the differential privacy you have a chance to get a high accuracy from the federated learning model but if you added the noise so we see that

around 20% of the performance that are reduced because of the complex data. So this is image data. So we add a lot of noise. So that's why it's reduce uh the performance. But later we'll see the another example uh uh for the uh IoT data it it will have the less effect. So let's talk about the IoT data. So this is the simple example uh for the IoT based ideas. So we don't have the uh IoT device does not have the sufficient power. So that's why uh we bring someone in the middle. we call it as a fork. Let's say that fork have the enough power. So some of the IoT device that form a cluster and share the data with

that fork and our assumption is that that fog is trusted for this IoT device but not trusted uh um by the other other cluster. So in this case that fork have those data and then he use uh that data to train the federated learning and he can even include uh the differential privacy there as well. Okay. So then we did some experiment and there is an one issue that we also introduced the scalability let's say that uh a group of IoT device come to the system how we can use their data um um and incorporate their data and then added them there. So this is the simple we can from they can from another cluster and then there will

be a new uh fork node he will get the fat learning uh global model from that server and then we'll update that one based on the based on that local data uh but it's not that straightforward that we said so we we tackled the nonidity scalability privacy and we did some experiments so for our experiment we use these two data set uh one is the RTI data and one is the CIC data. So you see that they kind of if you see the class distributions so is a kind of non ID even we set up a settings in a way that that non all not all client will have the uh all class data maybe some of

those client will have the class missing. So based on that experiment we we calculate the performance uh for the different classes as well as the different uh client. So we see that all the cases the performance is above 90% but some cases you see the extreme case uh there are few extreme case they have the lower performance. So this is the one of the case uh why this happen I will tell it later but there is an option that we call it personalized federated learning. So in this case we can use the personalized ferated learning uh to improve the performance but still you need to be considered the bias and everything in that case as well. So this is the effect

of the DE if you add the differential privacy still we have the high performance uh let's say that only the 1 to 5% degradation. So why this happen? Basically the data set that we take uh is not that complex compared to the image data. uh this data maybe have the 46 features and other one have the 79 features. So it doesn't added too much noise in the system. So that's why we don't see too much performance degradations. So this is the effect of the personalizing. Let's say that the blue line is the initial performance. Though these figures show a high performance but if you estimate the performance of each individual client you'll see that that performance have a

lot of variation. So you need to be use the personalized uh training to improve that performance. That is the one of the way that we can improve the model performance. And there are the few feature directions that we can consider. uh one of the biometrics that is the extreme case. So positive class data only one class data we have. So how to train uh the federal learning by using the biometric or behavioral biometric data how to make that system scalable and we also familiar with the multimodal data that have the more than one mode. So how we can feed them in the fedal learning and people talk a lot about the last language model. Uh is there any way

to maybe the rack or something? I'm I'm not sure uh I'm I'm just trying to exploring that is there any way to uh do the federator learning at the rect rack part not the uh I mean the last language model part and obviously there is a explanability so people are working on that area as well and when we talk about the defense there are a lot of work that are happen in the defense but differential privacy the problem is that when you have a high dimensional data uh you need to add a lot of noise. So is there any way that we can reduce the dimension of the data by um maintaining the feature or something not to change

the feature that much or something like this. So this is the another detection that maybe the random projection I I work on that area that we can use um to do that one. So this is the concluding slide. I mean uh yeah. So we have the concord university the funded us um in our seat program. We started this project not long ago just one years and we have the couple of collaborators uh most of them from the academic uh most of them from academic and we need uh maybe a few collaborator from the industry uh so that we can apply our resource in the uh real applications. Yeah that's all about my presentation. Thank you

again for >> Thank you. [applause]