PG - Deepfakes, Deep Trouble: Addressing the Potential Impact of Deepfakes on Market Manipulation -

all right welcome everybody this is deep fakes deep trouble analyzing the potential impacts of deep fakes on market manipulation my name is Anna Skelton and I am pleased as punch that y'all have joined me on this Wednesday afternoon so start with a little bit about me I studied global security and intelligence studies at a really tiny nerd school in the middle of the desert I worked for GoDaddy for a little while and while I was working for them they actually sent me to my first Def Con Def Con 25 which I went to as a total and complete InfoSec newbie that was where I met my close friend and mentor Mike who gently guided me towards the path of

information security I was picked up by a large financial institution first and an info SEC generalist role then I moved on to doing strategic cyber threat intelligence which is what I do now and absolutely adore in my free time I like to romp around in my f-150 pickup truck Harvey or snuggle up with my Siamese kitty cat Eleanor so before we get started I wanted to kind of do an intro on how I'm going to be delivering this talk when I explain new concepts I'm going to do so first in a technical way and then in a more laypersons way I believe that the barrier of entry to technical concepts should be lower and I

want to and I also believe that even if you're an expert on something hearing it explained in a different way can help shift and mature your perspective so we'll start by just going over deep fakes what are they how do they work we'll move on to an intro into market some market manipulation we'll talk about past examples of cyber influence on the stock market we'll talk about my own personal adventures and misadventures in deep fake creation we'll talk about future possibilities and then we'll wrap it up by exploring some solutions so what even are deep fakes well in the traditional model generative adversarial networks or Ganz use - machine learning models the the generator and the discriminator and they

basically just Duke it out so the generator will create videos using data and then the discriminator will try to call it the discrepancies in those videos it just keeps going until the discriminator can't tell the difference anymore it's so it's you can think of it kind of like the Tom and Jerry of deep fakes right so I'll show you a video right now that's an example of the traditional deep fake model your lot ready today yes hey we're the forecast said it's gonna snow to me what do you want me to do about it come on honey don't be so grouchy I'm not being grouchy I just want to finish my work okay I understand I'll come back

later on with a couple of sandwiches for you maybe you'll let me read something then Wendy explain something to you whenever you come in here and interrupt me you're breaking my concentration you're distracting me taking time I understand so I've never seen the shining and honestly I probably never will but if you had told me that Jim Carrey was in the shining just after seeing that I would have been like yeah I dig it sure it kind of shows you like the quality and even when people are moving and pieces are moving it retains the structure of the deep fake but there's another model that's being developed coming out of a research team in between Cornell and Russia unlikely

pairing I know but they're looking at a new sort of format for deep fix right so it's it performs lengthy meta-learning on a large data set and then uses high quality generators and discriminators to create and frame one or few shop models so in layman's terms it basically just watches a ton of videos of just people talking all sorts of different facial angles and then applies the learning from those people talking to the target images so that looks a little bit more simple and the most famous one was done actually on Mona Lisa if you manage to watch an ad platform to track everything any managers in the room it's probably good advice

so you can see obviously the Mona Lisa never said any of those things or never did that moving face thing but that kind of gives you an example of how the deep fake threat continues to evolve over time so I believe just like the malware threat continues to evolve we'll keep seeing different types of deep fakes as we continue to evolve the technology and Washington's starting to take notice of the deep fakes threat right so Marco Rubio actually recently stated that he considered the threat of deep fake videos as serious and significant as this as the threat of nuclear war so yeah it's pretty serious stuff as far as the legality of deep fakes goes it's

kind of across the board right so in Virginia they recently passed laws outlawing deep fakes as part of their revenge porn Legislature in Texas on September 1st a law will go into effect outline deep fix is part of election security but obviously this is a very piecemeal approach and it's never going to keep up with the aggressive threat of deep fakes on the federal side there's two pieces of legislature in the House of Representatives right now but neither of them gate have gained very much traction and even if they did we know how long and arduous the federal legislation process can be especially with the divided Congress so let's take a minute now to talk about the market

I'm gonna start by introducing it right so the market has three components there's the currency markets the equity markets like the stock market and then bonds which are backed by the Treasury ultimately it'd be really hard to impact the currency market using deep fakes so we're just going to go ahead and take that out of the equation they use such a complicated entanglement so the relationships of two countries that it would take too much effort essentially to really create an impact now within the within the equity market you have three different components you have the Dow Jones which is very banking heavy the Nasdaq which is tech heavy and the S&P 500 which is essentially sort of

just a mix of everything it is worth noting here that there is a trading kirb rule in the equity markets it's a fail-safe so just in case there's some extremely suspicious shady stuff going on it'll automatically come into place and halt trading that's important and we'll come back to that later so market manipulation is really narrowly defined as artificially impacting the supply or demand of a security so this is basically using misinformation about an individual maybe a company to impact the stocker performance of a trade or shares so we'll be talking about market manipulation a little bit why it more widely today both on the micro level of how you can slide underneath the failsafe benchmark and still exert

market impact as well as in a macro way which is basically bypassing the failsafe by causing enough damage that even if the failsafe was triggered it would be too late also I like to appreciate this very unsettling photo that comes up on Google Images if you google image market manipulation so let's take a minute to talk about past examples of cyber threats that impacted the stock market so you might remember in 2011 when The Associated Press's Twitter account was hijacked and they posted a tweet saying that there had been an explosion at the White House and the president was injured automatically the Dow Jones plummeted and the SP 500 lost one hundred and thirty six point five

billion dollars in market capitalization which is calculated by the number of shares divided by the price of those shares or you may in this case you can see that the markets quickly recovered but a lot of the damage was caused by computer algorithms that automatically use social pulls from social media and news sites and make trades based on predetermined rules earlier this year in April there's a huge run on the UK's Metro Bank when a whatsapp rumor went viral it said that Metro Bank was liquid right and people were running to Metro Bank and pulling out all their assets all because of a rumor it was completely false and this one got physical there were several reported robberies of

people who would come to the bank to pull out everything they had there additionally in the deep fake vector already there have been reports of people using deep faked audio of CEOs from big corporations to call on the phone lower-level employees and ask them to transfer large sums of money around already there's been three million dollars stolen just from this method alone and that report was at the end of June so we can imagine that between what hasn't been reported and what has happened since then that number is probably much higher so we can talk a little bit now about my own personal adventures and misadventures in deep fake creation so when I started this talk I assume that

the barrier to entry for depicts was like here like literally like anyone could just like walk up and like I have a computer I'm a dude I'm gonna make a deep fake now that I've gone through the process I would put it more probably like here so if you have the time and especially if you have an inkling of like a technical background and you have the right technology I still think that it's really relatively easy to do essentially there's two main schools of thought there's deep face lab which is an active github repository and then there's fake app which was taken offline in early 2019 but it's still available by googling it interestingly both of

these run on a Windows dependency so not only that but you also have an admit you must have an Nvidia graphics card to really create a high quality deep fake so that actually was surprising to me it's not worth noting here that you can't run deep fakes through a VM because VMs can't use the hardware of the computer they're running on especially in this case the graphics card so you need to have a system that's running Windows OS and has an Nvidia graphics card or you need to use an external and video graphics card I also ran into some fun legal issues when I was creating my deep fake so I got yelled at by a corporate lawyer and told

that I could only do a deep fake of myself that would be really boring also I could just make a video of me being like Parmesan cheese tastes bad and then I could stand here and be like that wasn't me I would I would never say anything like that so I cajoled with a boy a little bit longer and he eventually said acquiesced and said that I could make a Twitter video or sorry idea fake video if I used somebody who worked at the same financial institution I work at and who would write provide written consent to let me do this to their face and I finally found someone so you might recognize this guy here David Mormon

runs the CFP here at East Heights Las Vegas he's very involved in all the beasts I'd sing he's kind of an OG I a big fan and he graciously allowed me to use him in my fake experiment so Mormons know Kim Kardashian right but it took a very simple Google search to pull up two relatively high quality videos to use as my extraction content what we're looking for here is from the head up with nothing in front of the face so in actuality this microphone would make that video tough to use so it's to dive a little bit deeper into the two different types there is deep face lab which offers three different packages that are a better available in Google

Drive and they're dependent on your CPU specs and your graphics card it's actively updated most recent that I saw was July 23rd of this year so it's still being actively updated and worked on it is however less user friendly especially when compared to fake app you're manually running the commands on your computer and it takes a little bit more technical knowledge to be able to go through this process it also has you work mainly off of a tutorial and when you go to use those techniques on your own video it gets a little bit more dicey there are YouTube guides but the main one doesn't have any audio and I feel like if you were coming to this as

somebody who had never trained on technical YouTube videos like me it's a little bit harder to follow it's also apparently just run by one guy out of Russia and he Google translated his entire readme which was hilarious my favorite part is actually right here I know it's hard to read it says deep face thought created on pure enthusiasm one person yeah this guy is awesome miss read me it was the most fun part of this project by far whoops that click nope okay so the other option is using fake out so fake out was taken down to February this year the last public published version is version 2.2 the biggest issue that I ran into a fake app

is mostly around the second point there it's really dependent on the programs that were current at time at publishing so it does have some significant dependencies for example CUDA it requires you use CUDA version 9.7 dot one with specific patching sequence after that and if you do not have that exact one it will not work so the issues that I ran into with fake app I mostly think are probably contributed to by that I mean it uses 2015 Windows Visual Studio which you can only get if you have a Windows license etc so overall though fake app is a lot easier to use so you can see on this I don't want to get from the

projector but you can see on this video on the side you essentially just direct it towards the content you want to pull from click extract you go to the content you want to train from click train and then you create it so it takes a while but it does all the work for you if you set it up correctly the YouTube videos awesome there's like religious one guy and he knows exactly what he's doing and he's like here let me just slowly talk you through this stuff and some of his videos have over 400,000 views right so people are watching this guy like I said the application interface is really easy to use but there's also an issue with I

like to refer to as the dreaded forum so at some point on fake app org which is where this was originally lived there was a forum I guess where you could ask all your questions right and somebody would come in and you know offer you a beer some water and be like hey yeah I'll help you like all answer your questions and now it's not there anymore and not only that but it wasn't captured on wait on the wayback machine which didn't really surprise me but the issue is that every time that this interface it has an issue a very very polite notice comes up that says feel free to post it on fake app door slash forum for

help obviously the forum's not up anymore these are the two areas I ran into about 50 times each and what's interesting is reddit posts for these two errors still exist but it's basically like one guy going like hey man I have this error and then another guy comes in it's like hey man me too and you're just like guys you know and then I'm like they must have figured out somehow and then I realized when that was posted before him was still active so that's fake app and ultimately programs like fake up worry me a lot more than deep face labs so ultimately this is the program that could be used by somebody with less

technical experience to really go through and create their own deep fix so let's imagine that you were able to create a really high quality deep fake and talk about the possibilities from there this is my shift pause so we're shifting now so if you wanted to impact the equity markets you would need to start with one specific part right so we're gonna start with the dow jones so i mentioned it was heavy with banks perhaps you make a deep fake of a CEO of a large consumer bank right and you release it and say whoops you release it and say and in the video he's saying that the firm is no longer liquid so even if it didn't take effect in the

stock market liquidity issues are a huge problem which is what we saw with the Metro Bank issue so right away people are running into the streets we're gonna see a huge enterprise impact to that Bank alone so that's just if it doesn't take hold in the stock market but let's say that it does and let's say that right after that you release another deep fake video maybe even of the same CEO blaming a specific tech company for the damage that was caused by the first video so now all of a sudden you're affecting not only the Dow Jones but the Nasdaq as well and you can see how quickly that could spiral out of control

if you wanted to go below the threshold what you could do is you could pass around a deep fake a really negative deep fake sentiment on Twitter perhaps even just using bought accounts when the AP is from that piss specific to stop connect it to the Twitter to the Twitter to pull down the information to change the price of that stock you can you could really impact the price of that stock and still slide right under the benchmark maybe not even raising too many flags at once so that's how you would do it if wanted to stay below if you wanted to impact the bonds market you would need to call and look into question the

ability of the United States to pay off its debts right because the bonds are backed by the Treasury so in this case maybe you release a deep fake of the chairman of the Fed really especially with the upcoming budget renegotiations that are already raising red flags you can see how a really sophisticated deep fake could have a significant impact on the bonds market and ultimately all this adds up to is a giant domino effect we're all if all of this happened at one time you can see how everyone would be chasing in different directions trying to pin this down and it could completely it could cause a significant amount of damage very quickly so hopefully you're

all significantly scared out of your socks we can talk about some different solutions that are on the table so right now the solutions are all longer latency right and what we're looking for is something that can get ahead of technology that exists to avoid detection right is exist to avoid those discrepancies being called out so leas longer latency solutions may not be practical for instance the instant the University of Rochester is looking at creating integrity stores to use on their on videos released to the Internet so the video you'd have a browser extension the video is color-coordinated and you're able to say like ooh that's a green video that one's been messed with or that one's maybe misinformation

there's a startup called Amber authenticate which is using assent isset II based on cryptographic hashes there is very vague start up called new knowledge and they're saying that for the low low price of five hundred thousand dollars they will keep the false information about your company from taking hold on social media not sure how they do that honestly I'm not don't think they're sure either and of course we couldn't stop talking about deep banks without mentioning blockchain fat comm is one company that is with mixed success looking at using blockchain to detect deep fix right now the only shorter latency solution we have is to keep monitoring both above and below forums for the development and release

of another accessible deep fixed software like fake app that in my eyes is what we have the potential that C caused the most damage and make the technology the most accessible but in order to really take care of this threat we're going to need community carburation human review this is a nathaniel glyco code quote i did not say this community collaboration human review and a whole slew of other factors to really tie it all together but ultimately this community doesn't really have a choice so I'm looking forward to it so I was too blessed to be stressed during this whole process I was stressed anyway John Seymour is my proving grounds mentor and he was totally

awesome through the whole thing these are a few other people that helped me that is the guy who sat next to me and patiently tried to troubleshoot with me all day for several days so I'll go ahead and wrap it up for any questions comments qualms or general existential tidbits that's my Twitter handle and thank you guys for being here do you think there's any quit sentence between the fact that the app was called fake app and face app so I think we know where a lot of the deep fix technology is coming from and going especially with our Russian buddy from deep face lab I don't think they're directly tied together but we do know that that is

obviously being sent straight into a server in a restaurant for facial recognition so I don't think there's a direct correlation but maybe a little bit I'm so I was just thinking when you were talking about market manipulation let's say that like a CEO of like as a very powerful stance releases a statement that they're gonna like on video that they're going to release or they're going to sell the stock for $420 per share to all the stockholders but then later comes out and says no that was a result of a perfect deep fake what would be you know strategies to you know counteract like reverse market manipulation do you think that would even be possible in the news a really

good question and especially for that specific use case so I don't know a lot about how you would backtrack from that I financial institutions to their credit are starting to at least acknowledge that the effects are a threat but it's like with every other you know every other part of our democracy and whatever else that's under threat from deep fakes you know this is a community issue so but I think it's important that they are starting to look at it from that specific angle I haven't seen any movement but I'll bring it up so I've been thinking about a lot about debates you recently a more on the social science side and in a perfect scenario

right so assuming we have a perfect 100 percent deep fake where it's like there's no lighting frames nothing like a just looks 100% on point and then on the other hand we also have a 1 percent accurate defect detection algorithm right saying that this is a falsified video or audio I often think what comes after that if we have videos that are deep fake and we have a perfect detection algorithm does that matter in the end because we have things right this is the kind of question is you know you know this is a fake video like it tells you someone on the screen it's like yep this is fake but does it really matter because that seed is

implanted in your head like you've checked I kind of like individuals with OCD like they've you know checked their doors with their locks they have checked that they have turned off the gas stove but then you drive you know ten minutes you're like did I turn off the gas stove then you have to drive back and so I wonder what your opinions are on that in that realm is do you think it will ultimately matter to the people who these defects are being targeted towards so I think it's interesting that you bring that up I mean that's really kind of the misinformation side we've seen already that false information spreads faster and wider than real information so even

when social media companies are now looking at it and you know Facebook will gently direct you otherwise lightly you know do you really want to watch this like you said I think to some extent you can't impact the way that people in take the information we can just do our part to make sure that that information is as limited as possible okay give it up on a speaker thank you very much [Applause]