Social Engineering at Scale, For Fun

so I'm Sarah I'm a data scientist with a small hacking habit and for the past couple of years I've been working on ways to counter misinformation especially online misinformation at scale and I've generally been working with platforms with journalists with political data scientists with other affected communities but one community they really want to see working here is you what can ordinary people do what can small communities do so social engineering at scale given to you guide here's some things we can find useful and definitions so one thing misinformation lots of different definitions it's basically false information that is deliberately propagated so it's not whoops I accidentally said something wrong on internet it's not like like

small-scale stuff this is big scale lies and here's a couple of examples so you might have heard about Macedonian teen sites not teenagers at all but so basically people who produce fake new sites so that you can get web traffic and if you get web traffic to a site you can get advertising dollars it's worth money and on the left at the moment this is one of the recent ones that they're pretty much on the this medicine will cure your problems and some weird asked alien Putin stuff they're probably switched to political as they come up towards the midterms all the way on the other side is a more typical piece of misinformation so Muslim breakdowns

Toronto so aimed at at Canada and one of the things I've seen recently is a switch from mostly us to US plus Canada so we watched a bunch of bots and trolls from other nations pile on to what was domestic misinformation so that's what it looks like you see examples all the way through here social engineering so you're happy sides this you've probably seen some social engineering but it's this idea of manipulating people so psychological manipulation the classic definitions are for gaining access to information and getting people to form actions this is a third part so you're not just getting people to form actions you're getting people participate other people to perform actions so this is a scaling so scale

these are some of the Facebook groups that went up to Congress recently so these were other nation state created a set of activist groups like activists and you'll note these aren't kind of like sort of crazy flag wavers the LGBT and the Bernie Sanders coloring book is part of this but the important thing is that these were misinformation sites sites designed to cause friction chaos and the shares there I mean 103 million chess we're talking large-scale campaigns it's not a few people we're doing this at big big and at those scales the mechanisms are different the mechanisms were countering a different and community response and this is the the little black box I'm going to talk about some

community responses from elsewhere I haven't seen that much from the US yet it would be nice to see it happen so impact why is Miss information Point important so what's what's the risk here what's the problem and if you're an individual there's some individual in individual targeted misinformation but there's also in world individual impact for example we've seen several examples now probably see more where events have been set up for opposing groups at the same time the same place so this is somebody using this information online creator to create real world conflict and you could do this to a nation-state you can do this to a company you can do this in lots of different places so

individuals get affected communities get affected so I've done a lot of crisis mapping in my time doing data support during disasters and every crisis we see a picture of a shark in a subway or in a street same shark shark gets around a lot but it's a different it removes effort we have to go out and stop these viral things happening because it's the Sharks are funny but if it's somebody talking about a bridge that's down that isn't or shelters that are open there aren't and we've seen that too then that starts putting lives at risk so that hits communities big scale democracies so you are not hey jury still I found out about who started this

and why and whatever but what we're actually seeing is a lot of organizer pages like this so this was for a specific date it was like June so March April maybe and it was okay you're going to put out memes all the platforms so this is across all social media platforms you're gonna use these hashtags you're going to sit on trending hashtag so sometimes you see just this instruction sometimes you see suggested tweets sometimes you see images set up so French campaigns we saw a lot of image decks and this is massive scale this is country scale this is world scale so this stuff is dangerous at at scale so how big having said mentioned

that so how big the scales so smaller scale your brain that's the end target the end target is your belief systems and this diagram is unreadable diagram every single liberal spoke all the way around the outside is a cognitive bias though human has and every single one of those is a vulnerability I have some favorites I really loved the repetition ones so if you repeat information but you include not so don't look at the elephant and look at the elephant people don't remember the negatives they only remember the positive part this plays beautifully into misinformation campaigns as well because they put out the false information and anybody saying X isn't true people only remember the X

pushes it through another one is traces information traces so one thing about human brain is it takes all information in as true even though you know it isn't so there was some beautiful timing experiments on that so if I say for example um haha I crashed your car and then whoops at where I was a joke in the time it takes between me saying haha I crashed your car and saying whoops I that's a joke that is fired off a bunch of other associations like I am a dangerous driver and you remember that I am a dangerous driver even though you're told the negative immediately and beautiful mechanisms lots and lots of them lots of

them being used targeting groups this is one of the adverts that was used in the Congressional I think hearings and you can target micro target down to groups so if you target down to specific demographics you have a much more powerful campaign you can target down to them and beautiful piece of work at Tech and size out again all social sites every single social sites that has some form of user-generated content has been attacked in some way has has misinformation on it and we've been tracking no Facebook or Twitter or the big obvious ones but you're also looking at comments sections you're also looking at event sites even financial sites there there are some beautiful

misinformation campaigns around Bitcoin going on but that's site that's just where you find it what it is and scaling that trolls putting out information misinformation on sites reaches some places the way to make it really go is to make it viral by amplifying and this is one of the Russian botnet sites you can go by something called an aged botnet so about hundred accounts with emails that have been verified that are 2015 ages for example it's about hundred fifty bucks and with that you can use those BOTS and start pretending to be people start pretending to be community manager you want you want to use those carefully at the other end of the scale for BOTS know that I could personally

detect very quickly it's about six bucks per thousand so bunch of Russians unverified if I wanted to do a big campaign I would just buy a whole bunch of those by the time it was taken down by the platforms you've already got the messages through so you're scaling out to millions here so thousands are BOTS millions of people and you know that's what the problem is that's how big it is and potentially it's bigger there are other things we can do so who's doing this so if we want to counter this you need to think about who's who's behind it and one of the groups behind it is the Joker's the sharks in the street guy in Scotland

American hurricane tweets as out goes viral it's it's a mess oh it was hurricane Harvey so tend to be one-off tend to be opportunists and they're really in it for the likes they just want the attention entrepreneurs the Macedonian teens when I say Macedonian teens that's actually not the only country so there are some been some fairly big scale ones from Macedonia but they've also been ones from Japan and other countries so you seen native stuff this is worth money people will make money so again website stories push people to those stories and you get money and I work in at Tech and generally you're priced per thousand I I bought so somebody looks at a page sees

the advertising that the person the site gets paid it's worth more if they click it's worth even more if they make an action like feeling out of form so you want them to do things on your site mops Brigade Inc so extremists sometimes sometimes just doing it for the lulz sometimes you need anima style hey look this would be fun possibly behind queueing on but given the amount of Russian stuff in there possibly not and I'm a little very little uncertain here because the queuing on some of the the big botnets have actually jumped on some of these campaigns so it's hard to tell where one starts the other one ends and the big scale nation-states

so Russia is the big beasty on this but they're not alone there are other countries doing this too in fact you know pretty much anyone any country can buy a bunch now there's generate boobs very popular bloggers produce really beautiful little PC quotes that got retweeted by the BBC by Vogue by other big big sites except she didn't exist and in between these talks no these pages about celebrity bottoms and style and there were just these right-wing pro-russian things just interleaved in so you get the the the views you get the eyeballs you get people's attention through the cute stuff and then you just Chuck the propaganda into into the stream there's a few of these going on that's one of

the mechanisms for them but you know what do they want so I've talked about attention so which is the the Joker's just want what to be seen and macedonians the entrepreneurs but they're after the money they persist they generally have really good quality they be test messaging they do good work the the mobs the group's quite often they just want some form of division so this is a community splitting apart into two communities so it's a classic effect you see and the nation-states generally is geopolitical so in that in case of Russia it's probably wanting that Eastern European buffer zone seem to be doing quite well at it so things you can do so one thing is you can go find these

people or these accounts you can collect the data on them and things you can look at so this is bot Sentinel there are a set of sites who have known Charles known BOTS and they track the sites they track the associations of people on the sites they track the content so you can start seeing what's trending in bot world so see 300 added today there's a lot of botnets coming through artifacts so you can look at the content of message inks so one thing that happens a lot is you get hashtags that co-occur and then spin off on their own so walk away for example co-occurred with queueing on and then became a big campaign its own right we see these

campaigns coming correlated texts and personally I tend to try to bury my text but then I'm an AI machine learning person I can do that but you see a lot of the same phrases same text appearing it's very easy sometimes you just see a identical text and thousands of it from dots certain places so easy to find these damn thinks it's almost scary you're else so if you've got something like the people making money they need to push you to their site which means you can look for their URLs you can look at URLs that appear a lot to go find those sites once you've found them you can then look for who's pointing at them

there's a two way thing there so content artifacts but context is kind of more useful for a lot of this because given skills I expect a lot of people running botnets to start varying their cut their content even just a small change in text is going to make a difference in how you detect so one thing context is you go look at the people who are there so you go to the hashtags you already know you go to the botnets you already know you go to the trolls you already know see what they're talking about see who else is around you look at previous rumors especially the Sharks and the street-style crisis rumors they they

turn up again and again we see the same shark we see the same collapsing building we see the same bridge again and again again you look at who's friending or following them are there patterns in there if you look at the structures of botnets you see these beautiful little closed cliques if you look at it and in in graph form you look at equally the reach weeks and the likes so who is promoting this stuff are there other groups in there you can you can start to find in use and then you look at the metadata so for instance if you're looking at fake new sites quite often there are groups effect new sites and we're seeing especially in Canada at

the moment seen clusters of sites and you can go into their records and see if there are things with same owners same characteristics one thing I will talk about it that's not text to in a minute but there are a bunch of metadata tags you can start using stories so again back to are they talking about the same thing is there suddenly a story coming up and one of the places to look for these is in the fake psychic sites notes because people are reporting these things saying is this true so you tend to see story starting to emerge you can track down those and so this thing is that's not text I work for an ad tech firm an exchange and most

of the big exchange the exchange is the site that runs the auction so every webpage with advertising on is a website page and it has a set of slots in it where you can put adverts and when somebody looks at it the first thing that happens is those slots fire back to mummy say we need adverts and people who have adverts that fit your demographic bid on your eyeballs so very high frequency auction and then the adverts go back to those slots so if you want to work on our exchanges and other exchanges and usually fake new sites are very very good at building these so ads text tells you where those sites can advertise or are advertising which

exchanges are using and these are very good markers as well so there are all of these pieces you can trace so follow the money there's a way to follow the money so having talked about misinformation and sizings waste and monitor let's talk about how to push back so one of the things this report I been working with some of the smaller platforms and the bigger platforms are slowly getting their act together the the picture on the right is from yesterday a missing folk on slides that from Facebook talking about how they're going to start doing different forms of reporting but so far it's not been that great so Twitter suspended 70 million accounts most of those were in actives

which isn't really what a botnet does and we didn't see a lot of a dip a change in in activities Facebook has tried a bunch of things including buttons and that they there have a team on it so you can report to them don't expect you know hold your breath and expect much yet but maybe this just gives them more data you can report the money so one thing if you're seeing things like political advertising targeted political advertising or you see fake web pages where brand advertising on it you're being import back to groups like sleeping giants and they are putting pressure on the advertisers not to advertise on those sites also being an an ad tech

exchange there are black lists for sites so you can at least start talking to those people about moving that money flow and a lot of this is running off money so it's worth worth doing you can block at an individual level there are lots of tools already out there for dealing with harassment this is a small one it just blocks recent accounts though them as followers so you're not looking at things that are likely to be recent trolls recent recent BOTS and there are grouping so you can use you can engage the these are my favorite misinformation fighters so these are the Lithuanians so there are groups who call themselves elves because else versus trolls in the Baltic States

because they have a lot at stake here so this Lithuania Latvia Estonia and now Finland and they they're facing a wall of Russian misinformation in in their local sites and they push back so generally they push back they've learned that not things so they push back with truths and they push back with humor and they're getting some oh I'm too early they're getting they're getting a fair bit of traction from doing this there there are other groups who have done this I belong to a group called fast Faust kanima so this is one of the groups that handles data during disasters monitors social media feeds for anything new and interesting coming through and they run anti anti

misinformation deployments - we've always done it they've just done it more formally and they're doing it as this mix of humor and as this mix of asking nicely to police not pushing back I ran a deployment earlier this year on Irma last year God time flies so I've had a similar to point myself where we have redeployed Australian humour which seems to work really well there there are other ways to engage one way on the fake news sites is to use search engine optimization to sit above them you create sites too close to them saying hey these guys are fake so there's a whole bunch of interesting hacker we trick so you can use so

engagement repair these things are doing a lot of damage a lot of the point of the really big trolls isn't ever be loves Russia it's it's much more create divisions form form divisions across country make it week and there were groups like the Commons projects who are using patrol type techniques so they're using sets of bots to find people on either sides of these boundaries these fractured communities and then following up with humans to start repairing some of that damage there there's some interesting piece tech work coming on but there they also have reposts and tools you can pull and use and having talked about how to monitor and how to respond one thing if you probably end up

doing is organizing some form of campaign if there's a group of you you're gonna have to think about what you're going to be doing why and how and typically I mean there are plans so a couple laps I've been working on recently so the 2018 Canadian elections and that was mostly monitoring so we were monitoring the local trolls and they were monitoring as the non-native bots and trolls started piling onto their their messaging we watched us focused messaging going into Canada then we watched it migrated into Canadians focused messaging and so we're monitoring that we're pushing that back and saying hey guys this is a problem and the midterms 2018 mid-turn us midterms is likely to be Bottrell season

so there's gonna be a lot of groups working on this we're doing some monitoring work on that there's some coordinating going on but if you're running a plan you need to know what resources you have a need you need to know your timings are there specific events like elections if for Canada we had the Ontario elections we know we've got the New Brunswick's and the Quebec elections coming up there's also other things other timings within there so the u.s. midterms is likely to disrupt the activity that's sitting on Canada it might draw effort away so what are the timings what is your end point whatever sources and tools you need and what are you planning

to do here do you just want to monitor do you want to push back what risks are you gonna take in this so this is you need to plan you need to think through this and talking about risk the two main risks you're likely to face dock sink that's happened to the Lithuanians already that was very weird they got doxxed by the Russians they counted ducks with the lists of Russians Russian trolls and it all got messy and pages got removed but the other thing is PTSD so something we dealt with a lot during crisis deployments you're dealing with difficult material you're dealing with high stress and that difficult material combined people break so you

have to watch your people you have to at least talk to the psychologist about and read the materials about what the signs are and what you can do to manage that always pair never ever do this stuff on your own it's bad news and coordination right they were going to be so many teams are already so many teams starting to run up for the 2018 us and it'll just keep going so one thing you'll need to be able to do is to talk to the other teams that are working in this area on this campaign and one way of doing that is which I'm actually part of a group credibility coalition who we're working on standards for how you share

information about different misinformation and different types of content and context miss informations so coordination okay I hear to have given a very fast talk so we're down to the takeaways and the basic takeaways from this are you have opponents generally what you're dealing with is a set of trolls you're dealing with botnets who are amplifying you're dealing at targeted advertisements and what they're looking for I am fairies but it tends to be they're looking for attention quite often looking for power not to your political power they're often looking for money quite often as part of that power they're trying to create confusions and your defenses against that is the monitoring here's finding all of these

artifacts finding these botnet groups making those visible and then responding to that so reporting so the platform's an EdTech reporting elfing and also there's a lot of political work going on many countries now so if you can input into that political work that's that's good to just keep up the pressure on this more information if Miss infocon is a major community in this they they are at a conference right now in DC so it's worth looking at their feed at the moment I have a github repo with a bunch of links to a lot of things I've talked about I'm Boadicea cat and I've got to the end and the last thing to do is to thank

strength my mentor thank you okay if anybody has any questions please raise your hand excellent and you'll need to talk directly into this one so you said that there's different types of groups like you have the Joker's and then the political activists how often do you see one group try to leapfrog on another campaign let's say like someone gets hurricane Harvey shark trending will you see an unrelated campaign like a political campaign to start throwing that hashtag on there just to get more traffic we've seen people camping on hashtag that tends to be the brigades in the groups but the more interesting is back in 2010 we saw what was probably the first russian-backed misinformation

campaigns during a disaster so Kate starboard noticed there were very specific rumors being put out and during the BP oil spill so there's sometimes several of things at the same time which is why I say I'm it's really you can't always tell some of the nation-state stuff there they were like markers like hey these guys are really into Crimea freedom here it seems a bit weird for passion Mac but other times yeah this crosstalk this crosstalk Boog AIDS it's really hard to tell what is native and what's they stage so we have time for one more question or somebody like to jump up hi thanks for presentation I was wondering is there any takeaway for like that

break your grandmom my grandfather family to look for when things are fake news how do how do they tell are there telltale signs it could easily say from the facebook stream Twitter stream all the social cuz you know any of that asked that by my families how do I know is fake news you know I I would look in things like Snopes I would first start looking at the the sites and see if there's anything up saying this is this is fake but if you're not doing die scale data analysis is not so easy that's one of the reasons we're trying to hit at its source trying to hit at the platforms and then moving that the use of communities is

the backstop you you want to take the stuff off the platforms you want to stop it propagating and then at the end when it's already on the path already propagated out then you then this is where you go oh now people at the far end have to do something so yeah there are some specific app specific language communities depending on who your grandfather and grandmother are but yeah

hmm I have yeah I have to call it on that one everybody please give Sarah a big round of applause and if you have more questions you can find her outside yeah I'm easy find [Applause]