28 Seconds Later: The Epidemiological Triangle & Other Models for Influence Ops

besides DC would like to thank all of our sponsors and a special thank you to all of our speakers volunteers and organizers thank you everyone for coming to my talk on influence operations yeah my name is Daniel Gordon I am a Lockheed Martin employee I'm a contractor to the Department of Defense cybercrime Center or dc-3 and Halloween is this week and I think it makes sense that I'm going to work zombies heavily into this presentation so here we go a couple things about me I am a threat intelligence analyst and instant responder by profession I published an article and war on the rocks on this topic and I have a degree in political science and modeling simulation which

has actually become in pretty handy for this and obviously I take myself very seriously all right so let's get started I did want to acknowledge a number of folks who contribute content to this presentation so there's a number of people in the MIS InfoSec community as we refer to ourselves who are kind of focused on this issue I wanted to recognize a lot of researchers who I encourage you to follow these are people who track campaigns who track activity professionally and you know better than professionally and I also want to recognize Rob connect and Carl Jack or Carl Miller who contributed key ideas to the presentation so here we go I'm going to go through some definitions because

not everyone's familiar with influence operations I'm gonna talk about why this matters for folks in the InfoSec for those of you in InfoSec and talk about social media and how its kind of changed the game I'm going to go through frameworks because they help us explain and understand this problem and it's very chaotic and very complex so the frameworks are helpful and I'm gonna go over state actors because they're kind of the thing that is most concerning most interesting in this field and also find response options because talking about is great but we want to do things to stop it or to counter it or to deal with it properly so let's talk definitions first off disinformation is false things

that people are spreading on purpose whereas misinformation is things that people are spreading without knowing that it is false these are both problems that are part of this issue but they are separate they are different the deliberate spread of this of false information is a much harder problem to tackle because you have to make them stop and not everyone who is doing it on purpose is gonna stop I also want to emphasize that true content can be part of an influence operation so that means that hey I am spreading something that is factual but it's framed in a way that doesn't give the full picture or makes it seem worse or better than it might be

or I'm giving it without context so now it has a different meaning or maybe this is private information maybe this is emails that are on your network that are being shared publicly hey that's not public information that's maybe embarrassing it's not it's still true but it's now part of an influencer operation to explain influencer operations which I've already been talking about but let's let's actually dive into it the easiest way is synonyms so there's a whole list of synonyms of influencer operations coordinated in authentic behavior is the name for it that Facebook uses lots of other folks use these in other ways but these are all kind of talking about the same thing there is a possibly made-up story that

Eskimos have a lot of words for snow and I want to point out that the last five terms on this list are from Russian and my explanation for the reason for that is that Russians have a lot of interaction with influence operations the same way Eskimos would presumably have a lot of interaction with snow their government does a lot of this kind of thing and they do it domestically they do it externally but it's in their playbook and it's gonna continue to be in their playbook so let's keep going all right so why does why did this matter to you guys there are a number of different reasons but the big one is that a big portion of what we do in our

life now happens on the internet it it's the way a lot of people work it's how they interact with each other socially it's how we take in the information the internet plays a huge role in our life now and when things happen on social media or on the internet that could have real-world consequences so here on the far right is a military exercise called Jade helm that basically had to be canceled because of conspiracy theories these other two examples up are where people died from things that happened on social media that's part of why it matters more specifically for you guys it matters because as I mentioned before content on a network that has been hacked and

released is part of now part of an influencer operation and that is a tactic in the tool set of adversaries now this particular example is some Russian GRU agents that were arrested while attempting to hack into the organization for prohibition of chemical weapons and when they were captured along with their equipment it became obvious from looking at stored wireless networks on the laptops that they had been involved in attempts against a lot of these other organizations now if you look through this list there's a lot of just disinformation around this but more specifically there's been a lot of leaked content from hacks so this is part of their playbook this is something you guys should think about if you're

defending a network that folks are going to conduct an influence operation using your data if the if they that is part of their goal and it clearly is for the Russian GRU this is a false flag example and false flags are also influence operations they're interesting to us because InfoSec practitioners are kind of the target of this kind of activity so this particular one was olympic destroyer so that's targeting the Olympics in South Korea it this was definitely a false flag attempt they used code similar to code previously used biases but suspected North Korean actor they slapped a rich header on top of some of their executables they were using encoding and encryption techniques that had

previously been observed used by North Korean activity but this was conducted by a Russian group this is yeah but North Korean code on the left and Russian code on the right they were clearly trying to imitate previously conducted activity they're trying to fool us as InfoSec professionals and they were actually successful in some cases I think now everyone realizes what actually happened so yeah false flags are something we often think about and I wanted to share this quote my favorite quote about this from Machiavelli a military operation involves deception even though you are competent appear to be incompetent though effective appear to be ineffective this isn't actually a quote from Bach you know II in fact this isn't

even a picture of Machiavelli that's Isaac Newton but the point is that the way information is presented and framed can definitely convince you of something and I'm guessing some of you did not know that that was not Machiavelli when I read it so I this is actually a quote from Sun Tzu because it's basically mandatory to have a Sun Tzu quote in a cybersecurity presentation so I so now I've talked about why this should matter to you guys we're going to talk about social media and how its kind of changed the game a little bit first off I wanted to bring up that you know there are a lot of platforms out there a lot

of platforms out there you know some are us-based but there are a lot of big ones that are not the us-based ones often have significant populations in other countries that means they they have to balance concerns of doing business in those countries with doing business in the US and you know you have laws that are different you have norms that are different it gets very very complicated I also wanted to point out that tick tock is a Chinese up tick tock is the fastest growing up in the u.s. so something to be aware of China definitely has different rules than we do everyone's paying attention right I can take this thing off right

alright so this problem is not new influence operations existed with previous forms of media there are there are things that have been spread over radio over TV over newspapers that or via newspapers that would definitely qualify as an influence operation but social media is different it's different in a few different ways one is that it's less expensive in a lot of ways so in order to send out TV signal you need a TV station to broadcast radio you need some kind of radio station as well but for sharing content on the internet you need a phone maybe or a computer whatever whatever means that you have to to send stuff out this also means that

control of information is decentralized you don't have a review process you may not have fact checkers you may not have folks calling up sources to double-check information you have lots of people contributing things information has been democratized and obviously that in some cases lowers the quality yes there are lots of folks contributing content to social media yes influencers are a real thing we want we're gonna kind of focus a little bit on state-sponsored activity because that's the one that's scariest and kind of most malicious sometimes but there's definitely people doing it for a profit spreading the same kind of thing because it gets them clicks it gets the money I also wanted to point out that

advertisers have a disproportionate voice in this medium because they're literally spending money to get your attention and yes there is a monetary value associated with your clicks with your eyeballs all of these folks are competing for it and also this picture makes me think of zombies so I wanted to include it anyway speaking of zombies it is a truth universally acknowledged that a zombie and possession of brains must be in want of more brains this is from the documentary Pride and Prejudice and Zombies and this is identical to influence operations which are targeting brains the activity on the Internet is a means of transmission but the brains are the goal they it spreads from brain to

brain there are more actors developing capabilities in this realm because they think it's working people have seen past worldwide events they say oh this is something we can do cheaply that we're gonna get into this game you know it used to be just Isis internet research agency maybe that folks I'd heard of but this is just a list and this is just a subset of ones that have been officially taken down in some cases by platforms and it's likely gonna grow a lot more so now now that I've talked about social media a little bit let's talk about some frameworks first let's talk about why frameworks because not everyone's on board frameworks give us a way to do

defense-in-depth it gives us a way to describe this activity in a systematic way so people don't get confused talking to each other about what what's going on this example is the Lockheed Martin site cyber kill chain which I know everyone laughs because it is now a little bit long in the tooth but it was groundbreaking it gave people a clear idea of conventional hacking activity and how to counter it and treat it you know defense and depth means approach and kind of make the whole thing systematic instead of chaotic and we're going to talk about some frameworks for influencer operations to attempt to do the same thing leading into that what's micro-targeting is a way to specifically

send information at individuals or a large group of people or small group of people depending on your desire people self reveal a lot of information about themselves online and also accidentally reveal that information but there some platforms give you the means to send information to a specific demographic specific geography and tailor your information accordingly this is a micro targeting campaign and this is one way that social media has changed things but it's also a reflection of adversary activity where they have delivered information they see immediately how you react to it and they can then change their tactics accordingly in 2016 the Russian IRA was had a a camp campaign pattern like this of about 24 hours and

for those of you who have spent time in the military or are familiar with this kind of thing this is an odor loop it's meant to describe what pilots go through when they're trying to shoo down an attacker an enemy aircraft this the decision-making process reaction and this is the same kind of thing but obviously applied to the Internet and influence operations the next framework is we're going to talk about bot activity so some bots are easy to spot like this one in the lower right that says Eric Young just a woman who loves Jesus there's there's a lot of will say low-hanging fruit in that in terms of BOTS that people can easily spot but

there that's not necessarily always the case bots are still a tool used by adversaries to amplify things that wouldn't necessarily have travel organically so but not all the activity is automated so I created a scale to describe this spectrum of automation based on the pilot authorization and control of tasks framework which again talks about how pilots control planes either with autopilot or you know using manual controls and the idea is that there's some fully automated bot activity that literally is just taking in information from other things happening on the internet and reacting to it but there are a number of cases where activity is both automated and manual or some parts of it are automated

and at the other end of the spectrum is sockpuppet accounts which is fake bank accounts or just official accounts that are being used to spread inflows operations alright so back to the loop for a second is you know this is the adversary this is what they're doing they've got longer term goals in mind but and those are in the middle you know they want to build up an audience so their message spreads more effectively maybe not too much of an audience because if it gets too big platforms will may notice and take action in general they want to evade attribution but there's an asterisk on that which I'll explain in a second adversaries have started realizing that

it's cheaper if they don't have to develop stuff themselves if they can just find somebody who has a message that they want to spread a maybe they want to just create division is that's that's a common tactic so they that allows them to kind of cut out the development process and hey now they're just amplifying folks that are carrying the message that they want to share yeah so the reason I I put availeth an asterisk on evade attribution is because bad guys have heard of the Streisand effect or you guys familiar with the Streisand effect so this is we're trying to hide something or make it not spread makes it spread more to the point where

this is now called the Streisand effect so sorry Barbra Streisand the bad guys realized that when one of their their campaigns gets taken down and a platform writes an article about it hey we took this down that may provide more coverage than their campaign actually had so it it makes it it it may accomplish their task on incidentally the next model is a recruitment funnel so Isis when they're trying to get people to participate in their and their activity would recruit people on major platforms would funnel them to less populated platforms to you know get them involved in their community with then using Krypton messaging to communicate with them more directly and then finally physical

contact this mirrors the staircase to extremism which you know put that's the progression of somebody from peripheral involvement or feelings of sympathy to indirect involvement and then you know taking action on behalf of a group so that can describe gang activity you can describe white supremacist groups it can describe any kind of community that is doing things and that that same kind of radicalization process and this mirrors the marketing funnel so let's say you like iPhones and you know they're cool but you know no big deal but you know the more you use it you're like oh this is kind of neat let me tell my friends about it let me go stand in line when it

comes out let me just post about it non-stop or adopt that lifestyle become part of the iPhone community I'm just picking on iPhones but this this applies to a lot of different things was that so yes sure Subaru there are an infinite number of examples of this but the I'm not saying that marketing and Isis are the same thing but sorry Claire the the techniques involved in these things the psychology involved in these things are definitely related these are communities and they are bring people into the community and giving them a sense of belonging and involving them in in this process and that that psychological phenomenon is definitely related this also happens with politics which I will touch on later

carefully so this is the the title the titular triangle I'm going to attempt to say it'd be a pity my logic triangle this is from the Centers for Disease Control and Prevention the CDC this describes how infectious diseases spread the triangle does I have here thrown attributes that describe why influence operations can spread as well in the same kind of viral net way these are all some factors but there are lots and lots of other things you can apply this to a significant influence operation a single influence operation sorry so hey this this message spread in this community because the people have these attributes this happened on this platform which is connected in this way this is a way to

map out that activity the triangle can also be used to describe things like asthma which aren't necessarily infectious but still have attributes explaining why something happened the next framework is the amat framework this is modeled on the mitre attack framework which I'm sure a lot of you guys are familiar with this is yeah this is a screenshot of the github page so I encourage you guys to go out and check that out this is put together by folks in the MIS InfoSec community and the credibility coalition and the idea is to I know I'm sure you guys can't read that sorry but I encourage you to go out and check out the website or the github page

this is intended to granularly describe this activity and then give options and ideas around how you could counter it how you could respond to it and you know more accurately describe it so now that we've talked about frameworks I'm going to go into some state actor activity starting with Russia shocking I know so in 2016 Russian influence operations use proxies such as WikiLeaks though they're also official media outlets involved including Russia today and Sputnik they are still very active they they used combined I'm going to do air quotes for this too because that's a term I came up with myself but pretty much to basically hack leak and disseminate information the hack and leak part was definitely

conducted by the Russian GRU that's been publicly attributed now the dissemination activity was from the internet research agency which isn't technically associated with the GRU but there's probably coordination advertising purchases so the average advertisement on social media has a one to three percent click-through rate the IRAs advertisements were getting at 10 percent click-through rate so they were very effective with their advertising they rapidly evolved their TTP's so the IRA experimented with a hundred different things they were on Pokemon go they were on a zillion different platforms they were trying videos they were trying lots and lots of different things 2016 there was targeting across the political spectrum that is still the case there was an article published very

recently of targeting across the political spectrum so there were black lives matters pages there were texas secessionist pages and everything in between there are people who try to make this a partisan issue it is not a partisan political issue in the u.s. this affects everyone and yeah I meant endeavor series want their operation to be revealed this was definitely the case with the IRA they did a reddit thread where they just said hey we're the IRA ask us questions alright so this is a false flag operation 2014-2015 a PT 28 which is a suspected Russian apt group was performing some hacking activity pretending to be the cyber caliphate they attempted to conduct a destructive attack of a TV station France they

successfully took over the u.s. CENTCOM Twitter account which and there's an actual screenshot I'm sure they weren't pleased by that more recently though I'm guessing people heard about this because it's very recent there was a revelation of a false flag operation where Turla which is a suspected Russian FSB had been performing false flag activity as oil rig which is an Iranian group this is kind of mind-blowing but they had been developing tools using the Iranians code had been using their infrastructure and were stealing their victims which is kind of mean not gonna lie going back to apt 28 though so yeah 2016 there's this famous breach of the DNC and you know the Associated leaks

thousands of conventional media outlets were publishing leaked information on purpose or they didn't know where it came from in some cases but the traditional media was part of that that activity the dissemination piece of this was huge there was thousands of accounts and BOTS there were hours of video there were 15 websites set up and there was a video game this is Hillary it's a flash game that they created because they are definitely creative according to the Senate report and this is just based on data from Facebook and Twitter 126 million Americans were exposed to content from this dissemination effort I do not know the impact that this had and to be honest nobody does because despite

pollsters best efforts you cannot measure what's going on in people's brains so this this probably had an impact just because of scale but nobody knows exactly what it was so staying on politics a little bit we're gonna try and be the center of here I don't normally talk about politics politics in my presentations but these are helpful examples this is from a from the White House transcripts on them to use air quotes because it's not an official transcript this was a conversation between the US president and the president of Ukraine and it's not always easy to tell what the president is talking about but in this specific case he mentioned CrowdStrike a server and Ukraine so it appears from

this that the President believes this conspiracy theory that a server was a physical server was taken from the DNC by CrowdStrike and taken to Ukraine because according to the conspiracy theory CrowdStrike is Ukrainian company apologies to anybody here who works from CrowdStrike that you have to deal with this but CrowdStrike is an American company there was no physical server taken but this is a conspiracy theory that the president it looks like believed enough to involve use it as part of a conversation with ukrainian president this is he's acting on this belief so how does he end up believing this and the answer is this quote I think if you tell a lie big enough and keep repeating it people will

eventually come to believe it Lenin didn't actually say this either this is actually Joseph Goebbels a famous Nazi and to be honest gurbles didn't say this either but once again you know information presented in a convincing way can be you know something that we all believe we are all susceptible to influence operations this is not just oh that group of people or oh this group of people we are all taking information presented to us and framed a specific way and accepting it based on our understanding of the world and that is not something that we can do anything about so now we're gonna jump into China some scholars have described Chinese influence operations as in three

Warfare's so that's you know trying to influence public opinion trying to psychologically influence what how people will perceive a problem and also they'll argue things in court as much as they possibly can especially when it comes to South China Sea which they will say belongs to them China uses astroturfing which is the fake generation of grassroots activity this is not the 50-cent army is not related to the rapper so it's speculated that folks do performing this activity on behalf of China get paid 50 cents per post I don't think that's correct because you could probably make a lot of money too at that rate trying to use these human leverage where they can they use the there they

have a an organization called the United Front which it has a number of different programs and elements that helps help spread their influence internationally there's a lot of documentation about their economic and cultural influence especially in Hollywood movies and lately with tech companies because there's a lot of concerns around the things having to do with Hong Kong and how people are dealing with that those protests China influenced operations will promote their or often promote their official media outlets they have both ones that are straight propaganda and ones that are more news like but just you know still have their their take on things trying to use this paid advertising there's a famous example where they ran an ad in the Des Moines

Monitor intending to influence sanctions policy they that prompted a tweet from the president in response and more recently there's been some evidence of stolen or sockpuppet accounts this particular one is a conservative news outlet it had more followers than I will ever have in my entire life but it was taken down as part of the effort to reduce their influence operation regarding Hong Kong I'm not going to go too far under Iran I did want to mention though that they were sending letters to the editor posing as local politicians or local celebrities and that was a way to get their voice in traditional media outlets by pretending to be somebody and just sending a letter to a newspaper and

saying hey you know can you publish this in your opinion section so how do we respond to this hopefully with this car if one of you has this car I wanted to drive it but this is this is the right way to respond I think it is I think it is a definitely a Subaru and this person is probably a big super roof and so platforms have a bunch of tools in their toolkit and part of that is because they they kind of own the territory of social media so they they can make changes to the environment section takedowns are kind of their biggest one they'll pump they'll take a whole bunch of accounts and just

shutting down or groups and often write up the results the way they find out about them is you know tips from other folks but also people using a platform abuse reporting mechanism so some platforms have a button you can press to say hey this is suspicious please use that not everyone actually realizes it exists and on some platforms it doesn't exist but when it's there use it platforms they use content filtering or shadow banning to reduce the reach of some things that they find undesirable they use they frequently make changes to algorithmic amplification stories because these are often gamed historically they were very very gamed but platforms have gotten better about not amplifying things that really should

not be trending we talked about BOTS a little bit there are a whole bunch of ways to detect bot BOTS humans tend to be awake during some hours and you know do certain things at certain rates those are the low-hanging fruit for four platforms to detect activity but you know they still take down I think Facebook announced something like a million a day in the name of stopping BOTS some platforms that putting things like CAPTCHA to make it harder to just create random accounts they'll give you more context around an advertisement or a news story to hopefully let you make your own decisions about it I don't know how many people actually click through to see who's sending them information

but at least that option is there for you and then human moderators there are there are news articles about human odd moderators getting PTSD or becoming radicalized from the terrible things that they have to look at all day that is really not good but in order to combat this kind of problem they kind of have to exist unfortunately there is some requirement for human direct decisions on some content so what are platforms actually doing yeah we talked about takedowns and the articles that they write up about them some of the platforms are actually taking action on bot activity Twitter is still 20% BOTS at least but they are finally taking action to kind of cut down on that

Pinterest realized that they had a problem with anti-vaccination rhetoric so they changed how their search results works of people who were not spreading this information about vaccinations would also have a voice this is the most common platform reaction now they don't want to deal with this problem it's a drag on their resources and it's a headache in the media and they really would rather not have to deal with it so there are some a number of platforms that just don't they just don't deal with it so what other options are available law enforcement can infiltrate some distributed groups like Isis like anonymous anonymous still exists but they're not as effective as they used to be or as scary no offense if anyone from

anonymous is in the room please don't mess with me but it's this this solution is unlikely to work against you know a state-sponsored activity coming from a building in Russia because it's from a building in Russia and infiltrating it is not easy speaking of the building in Russia though indicted people and you know conducting hacks on their on their infrastructure is a way to respond these are of limited utility in debt these nobody indicted in this indictment in this particular one that I've got a screenshot of here is likely to go anywhere with well where they will be extradited so they'll they'll never see jail time from this but it'll still call attention to the activity and hopefully

taking down their internet does actually have some impact so what do I recommend for you guys specifically first off please think about what you post on social media think about what you believe on social media and what you read use resources like media bias fact check to look for bias to look for reliability of information both of those are something that you can get from that website but also by just kind of double-checking things I encourage you guys to tip things that you find to platforms or to other folks that can actually respond to it please go to this github page and contribute if you want to fork it if you want to build something from it that would be awesome

and please support researchers people who do fact-checking because they are also helping solve this problem or solve some aspects of the problem please check out DFR labs they they're part of the Atlantic Council they do fantastic work around influence operations so quick summary of what I talked about definitions why this matters to you guys we talked about social media we talked about the frameworks to help describe this we went through some state actors and we talked about response options and I just want to leave you guys with this quote it's definitely real quote from Taylor Swift that is actually from Swift on security who is a fantastic Twitter account that everyone should follow that's my presentation does

anyone have any questions yeah nobody knows it's it's lots of people have said similar things but that actual quote nobody actually knows who originally said it so they had they made a bunch of mistakes they they had some things that made no sense that they had tried to false flag there was a piece of code inside that was encrypted but there was no reason to encrypt it in that manner they had some a number of techniques that overlapped with previous destructive attacks from apt 28 in general or the subgroup that I think Olympic destroyers attributed to which I believe this black energy or sandworm team there's a number of different ways and then I think there were some

governments that kind of weighed in and governments have their own methods of attributing things so yes

yes that is a very good philosophical question which I think I I'm not going to be able to give a like this is the correct answer but I can say that there are things that can be objectively proved to be true and there are also ways to convince people of things though it can be extremely challenging when people have completely different frames of references and completely different set of facts that they're operating from there are tricks with empathy there are ways to appeal to logic there are various ways to approach that but there are definitely arguments that I don't bother getting into with people because I'm just not going to be able to change people's mind as well so the there's no

objective and correct answer to your question but I do think and maybe I'm just being optimistic that this is not a lost cause and we don't have to just live in separate bubbles and just pretend that everything everyone who doesn't agree with me is crazy I don't think that's the correct way to operate

you're asking what activity being performed by US government is that correct aye-aye I'll be honest I don't actually know because I have no visibility into that I will say that you know that the u.s. government does some things officially through like websites through you know communication with the media so like my slide earlier about Turla and oil rig was published by the NSA along with the NCSC which is you know the UK's version and they you know they're putting this message into the media through official channels but I've no no idea what the US is doing covertly yes thank you I how do you tell the difference now I'm kidding yeah yeah I know I I I do want to

emphasize that there are some some things about you know every single outlet and every newspaper that are are almost every single newspaper and outlet that are actually like they're doing good reporting sometimes it so Shepard Smith just left Fox News and I have nothing but admiration for Shepard Smith reporting there are other folks in that organization who I think do great work and I don't want to just dump on them as like a source of conspiracy theories and craziness there are folks in Russia that suck at what they're doing as well so I I'm gonna say it's at I know you just I don't know I I can't I can't give a real answer because they're they have they

are different and you know they're both doing what they see is right even if I disagree often yes other questions yes sir yeah yes so I'm gonna repeat the question cuz as an excellent question honestly these evolved in excellent questions um so this this or in the back was talking about how Scandinavian countries and Taiwan have been received an incredible amount of targeting of influence operations and figured out kind of strategies around handling it the best strategy that they have is having societies that do not have giant fissures in them for adversaries to take advantage of they there are also things you can do with your educational system to help people develop media literacy help people think

critically about what they're reading and consuming but the biggest thing is to not have gigantic vulnerabilities for your adversaries to take care of take advantage of and that kind of applies in InfoSec as well if you have a gigantic vulnerability in your infrastructure than your ability to stop your adversaries limited is that is that helpful okay yes sir hey John thanks [Music]

[Music]

yeah the the comment was about there was some news articles about how the Russian IRA had tried to develop protests basically across the street from each other for competing organizations and also get like individuals to physically start protests in the US so I think there was a a black rights group that they were attempting to actually you know have a real protest and maybe turn it violent it's they had limited success with that but it's absolutely a significant goal for adversaries to have real tangible real impact that isn't just changing minds on the internet that will actually probably manifest itself more in the future though maybe not in quite as overt ways of let's have two

protests across the street from each other and see if we can start some violence that way yes sir Jenny and put on they were absolutely trying to create more effective targeting based on all the information that it they had collected I I don't know how much impact they had because again I I can't see into the brains of the people that would have voted one way but ended up voting another way because of advertising targeting from the Cambridge at analytical data but they're an example of a for-profit organization that's you know definitely trying to get into this kind of game and that is also likely to continue but I I don't know how much impact they had sorry

any other questions okay thanks you guys thank you for coming [Applause]