Phish Your Users for Free! (As In Beer!)

up next by the way I came out here from University of Missouri so the thing is that my is so good that I was still willing to introduce you okay so so what these two may have superhuman analytical and infrastructure capabilities that they will tell you about today the other one we're still trying to figure out what their real identity is because they have a very convincing doppelganger here in the Kansas City InfoSec scene but here to talk to you about phishing your users we've got Julie and thank you so my name is you gonna have the chief information security officer at the University of Kansas my name is Shane fine I'm an experienced engineer at the University of Kansas

Medical Center and you may have noticed by my bling here I'm hiring so if you especially if you are a newbie looking to get into a second first time we should yeah alright so I'm a little bit about phishing your users for free as in beer I'm the outlay attacks on this was presumably that was about five bucks so how many of you currently fish your users right now so you not everybody but close alright so I'm hoping that I can give you some ideas for how to keep doing it rather for your practice those of you who aren't yet this this talk is really aimed at helping you set up a cell fishing program

budget but maybe even more importantly how to set it up so that your users don't burn your house down if you don't mind up fire we may also be able to help keep you off the radar local law enforcement I'll talk about that a little bit so this sucks this happened in the summer of 2016 I'm driving to work and I get a call from payroll and they said we've got three or four call supporting people who said they didn't get paid on Friday and they swear and now they didn't change their direct deposit information and I'm like oh crap this is bad in the spirit of never living a good crisis go to waste

I requested permission to start at the self fishing program but a little bit more about this incident the attacker is actually targeted because we are a university because the information that we have is generally meant to be shared they targeted a list of distinguished professors we want to brag about how I distinguish the professor's with their endowed chairs and our interest in research and their longevity at the University right so we'll be a published list of them which the attackers are corrected in assuming that these are probably some of the better paid faculty in the university so there I found the fish these guys and get some their paychecks and they have about 250 people

they gotta have some sense of presence I'm a Hollywood about five painting suddenly deal so that was a drag they were also pretty crass thieving the performing it actually made the changes for direct deposit information you know they did they love them to have web access and set rules of a message to be my symptom wearing them for the direct deposit change so so this was this was a gnarly incident and because it was summer and the vast majority of these people were off-campus overseas doing research we didn't have any disability of the who clicked the link we didn't know if they were on campus we didn't have any ability to tell at a time a Bigfoot playing so as

you can imagine distinguished professors like having their password reset with no warning whatsoever we were literally chasing people through airports at the Australian outback trying to get a hold of them so we can pet him change our passwords anyway this was terrible I used as an opportunity to start at the self-sufficient program our motivations were not determined to do about you exercise this is meant to be an educational opportunity and not like a don't slap educational opportunity like we really legitimately want to train users what to do when they encounter a phishing message because believe it or not there are still plenty of people out there who are not prejudiced enough to look at a message and go this might be

as you in addition and university is imagine we have a lot of people who speak English as a second language who are coming from cultures where difference to Authority is really important and they are going to look at these messages and go well this person says they're from kayuu and they're telling me to do something if I don't do it I'll get in trouble click so we want to train them how to recognize these messages and give them give them information about what to do when they get one or reported and how they get training so they can learn more we a few goals too we felt like at the time nothing that people knew where to go and they needed help

with this so we wanted to increase our overall visibility as an office but how do we can people get a hold of us when they need to and they have question we wanted to increase G's overall security Buster we want fewer people clicking on stuff we also want to give them give users reminders and practice on spotting these types of messages whether the features they should be looking for what are the things they need to check before they click the link what kinds of emails should they expect to get from from departments like HR or markers like on TV and we just wanted to drive more people to the annual required training that we get every area so why you should

fish your users that mandatory or training you do every year assuming you're doing it all that's doing is providing metrics right all that all we know is ones or zeros yes they took it no they didn't it's occasionally annual that's once a year and if you show me a user who willingly takes that member stephan1 once a year and I've been originally with let me tell you they just don't do it plus it's like who cares you're staring a stack of PowerPoint slides and then you take a quiz gives a crap like they're not gonna remember this their girls are not going to develop any muscle memory counter problem and that's what this training is meant that's what this

fishing assessment has meant to do for them I got a fish for the gear and delete it and you'll click the link that's what we want to do for them so you need permission before you do this a wise man once told me that you can do anything you want on your last day and if you go out and do this without getting permission from the appropriate folks at your company 19's goodness you are going to best pace for every bit of political capital you've ever built up and we learned how to do that rittany if you do this without permission you will torch every bit of political can't believe you've ever you've ever built

which King says you're on the set KC slap throwing the job posting this channel I'm hoping nobody knows about that stupid stupid thing you did that permission needs to be written it needs to be in the form of an email it needs to be premiered c-suite executives or your general counsel or HR or maybe all three that you need to have permission in writing that you've been told yes go forth and fish your users you also need to be keeping people in the loop presumably you're going to be impersonating people from your company or from your organization when you do this so you're if you impersonate say your HR department without telling them first guess what you're going to blow up

your HR front desk your admin staff and you're also going to burn up goodwill that you have a chart which is probably not the department when we were at pit bull but so so this is not Lisa these aren't meant to be a double-blind study right you're not you're not going to mess up the validity of your results if you give certain people at your company and heads up that these are coming it's so you're gonna lie on one the department that you're spoofing girls are gonna want to worry your helpdesk you may win one more in your your organization's big enough to have some police force you should probably call them first little bit one half and

abets and you need to be a winner that you're gonna you're gonna piss people off and you need to have the places that people call them and pissed off prepped for this exercise so this is not your chance to exact revenge if you're going into this thinking yes I can finally get my revenge on all those jerks who would let me sit at the lunch table with them when we were in middle school you're coming at this with the wrong attitude here you're conducting an education campaign this is not something that you're you're trying to again do that double-wide study he wants to train people what to do when they encountered a situation you're presenting them with so you're

going to give them a heads up you're going to tell them we're going to issue this week here's how to spot a fish here's what to do what you spotted it the people who fall for your message they are victims they're not offenders I've seen them all in certain papers they're not idiots they're not losers they are your victims and you need to treat them as such look words that you use to describe your colleagues and these people are your colleagues are important and when they find out not if when they find out that you speak with them with contempt they're going to be a lot less likely to with you and be willing to to be your

front line of defense people are not your weakest link stop saying this you can go on Twitter right now and find some hashtag thought that are telling everybody the people are going this blended security people are your front line of defense they are not here we display and reframing that is a key part of how they gave you your users to report things to you and being a shot threaded selfie into your departments to tell you what they're seeing because there's a good chance that you are not going to be seen in all this because they do you need to give them a way to report the stuff to you and letting doing these exercises gives

them a chance to practice so they need to have an email address that they can get to you at they need to know how to reach you by phone ideally you're giving them a one press button that says here's hen report a fish tickets that that's that's one of the things that we're hoping to do because right now we get we get a lot of really innovative ways for people who are messages to us so as you start to build your campaign know that there is no new ground that you need to be writing you can access to your your abusive talent or your mail flow of your hygiene systems to get samples none of the phishing campaigns that

we've done K you have come from anywhere but our actual use account because it's just a wealth of information it's a wealth of fishes that people are seeing in the real world you also as you do this though you and this is something that I'm going to tell the tripped us up you need to understand the political environment at your organization so that you can design campaigns that are effective but not effective you have to know what your organization's hot-button topics and I'm going to talk about one of those here on this next slide it turns out parking tickets are very big deal Acadia and we understood going into this this campaign that that parking is

a hot-button issue but I don't think we fully appreciated how hot-button of an issue it is we we did we thought we did everything right we called up the parking department and I said hey we want to spook you guys and the fish is that okay they were like knock yourselves out okay so we did we gotta find them we gonna find our help this shame dopa fishing message and we were off to the races our usual click-through rate on these campaigns by the way is about 7% this parking is the seven seventy seven percent yeah so people can click today on this parking fish was 27% people were [ __ ] bricks about this

campaign among the things that we got accused of will be caused at least three panic attacks that morning we've had people calling and actually driving to Douglas County Courthouse or to the understanding they were calling the Lawrence Police Department they were calling the Kate you Public Safety Office which is our Police Department K and K you screaming at parking attendants is like a club sports so people were doing that and we we blew a local foot on local Twitter and learned their really valuable lessons about hot-button topics so here's another one I'm not this stupid but once a month like that somebody this is oh you should totally do a phishing campaign around basketball tickets no no and clearly I

need to get my graphic because we're like 14 now but you need to know what these topics are at your company so it might not be basketball tickets it might not be parking if it's going to be something else every company has the thing that everybody gets riled up about and it's a really emotional BFD issue it might be dress code it might be benefits it might be who knows what but I know that it can you basketball as a PFD I'm never gonna touch this for a couple of reasons one if I promise free basketball tickets on them don't deliver somebody's going to be somebody's gonna torch my house too this is going to

really screw up my numbers I don't want everybody to click on life issue campaigns I want this the the people who are most likely to fall for this kind of thing to self-identify this is going to catch up way too many people who might not otherwise play and to be honest most of the fishes we get argued boring your quotas or mailbox buttons over or you you know you have spam messages to unlock that kind of stuff I we are still in the low hanging fruit phrase things I'm not ready to start targeting they're really stabbing using some of our population we're just not there yet so this time of year I hope I'll get to do

include this because of time of year it is and also just because I know this crowd and I know how we think you're going to be an attempted to hear him to be tempted this to spoof IRS messages or messages from other federal agencies because you know a little bit of reaction especially in April don't do it the IRS actually specifically requests that people like us don't spoof people like that because your users or report those messages to the IRS and it wastes resources and they wind up tracking down these messages that were never actually real fish to begin with and depending on how well crafted your messages you may be breaking the law by using their names

or so just stay away from federal agencies when you're building these messages there's there's just too much too many other easy topics the fish people about to pretend to be a federal federal agency finally look to the Nigerian scammers there's there are a couple of really good papers and articles and their reference of the notes at the end about why Nigerian scammers build the messages that they go and how they built them they intentionally make those crafty how many of you know the things mentioned they make the message is proudly so great the read for the rest of you if Nigerian scammers made their messages really really good they would get way too many people responding to

them they don't know who their targets are so they need their most vulnerable target targets to sub identify so they figure if they can get you to respond to the super crappy message that is full of bad grammar weird spelling foreign English syntax there's a really good chance that they're going to be able to get you over that finish line to get you some their money so unless you are targeting a group of users who is really savvy - it's not going to fall for a standard phishing message or unless you have some kind of obligation to Fisher users other than general population I would suggest that you keep the bar of entry low especially

because you want to identify those people who need the most follow-up and who need the most help so what to collect the tool we use go fish is capable of collecting usernames and passwords we could add other forms in other forms into it if we wanted to but we decided that for our own safety that we were only going to go after usernames we think this is Ballard Berlin for the main reason that we can get a user to click a link put in their real user name they probably went in and put in their bill password too if we got through that bar we probably got them all the way we didn't want to collect

that those bad financial care for a couple of reasons why that isn't the nuclear waste of data that have sitting around or clear text for potential parents to if if we did collect that credential parent dental care is not considered compromised and because we fish about 14,000 people at the top we didn't feel like it was very fair to helped us to then immediately dump six or seven hundred people into the password reset process all the months I still have to work with that so we've said to preserve that relationship so it should be more accessible for us yes our users are scattered all over Kingdom Come at all times especially depending on the time here biggies in the summer

we have way more people it overseas and not so we make our exercises real accessible but if you choose to do that be aware Google will see this and they will eventually flag your exercise as malicious that's why we recommend the chief domains that Shane's going to talk about in a minute because you should anticipate that you will burn in domain or two per campaign along um we started out thinking yeah we're going to run this for a week as soon as mr. Joe leave that page up for a week and I'm gonna shut down well so we kicked off the exercise between 7:38 boomerang on a Monday and by Wednesday we noticed chrome was starting to flag our site as

malicious and by Wednesday afternoon Thursday morning our domain registrar had revoked her domain and we were completely different so it turns out where if you do this where people can see they will do something about it so based on some research our own you know watching the click-through rates and then going out looking at things like the horizon vapor breach investigation report we realized finding it's way too long been what business day tops release the hounds at 7:30 in the morning shut it down at about 4:30 start on that such as either an inbox of the five most of our users again to pay people apparently don't check their email and then the data breach the Verizon report

said that people will click within the first like 90 seconds to two minutes our top cook freeze happens in the first two hours so I don't know why they're so slow but they are so we just said you know I business day as money we'll cut it off after one day and the upshot of that is we haven't had to burn domain since we shortened that area so I'm going to leave it it's a shame for a technical overview yes so my domain registrar doesn't like me a couple times has been telling about a leak hacks or instituted stuff looking over a technical overview know that what we did how they set it up a little bit

into the system used so that Julie 21 today I was like yeah I think we need to run the enterprise great phishing campaign across all of campus and I was like cool Antonio you have she's like no so she'd have you her person found like 84 cents that used to come and then I had figured it out so the prereqs you don't need a whole lot machine whatever's lying around a little money because he really don't think in life history and then a willingness to appear despair and hopelessness you know most companies to perform these press assessment students turns my head but with only have to develop tears you get that nice feeling now when we ran that we were expecting

way higher numbers so and I asked some of the numbers for you it just fit stuff you can take back to your leadership and like here this allows you to do it because if ok you can do it anyone could do it guess what work for me so what do you need to buy nothing if you have the infrastructure already money isn't required for the special features if you want to demand you can use right this project Namecheap don't tell them I sent you they have a good sense you can get awesome stuff like way men and dot science I know a couple just because an external email Center if you do not want to burden your own in email

infrastructure our email administrator was weird a little worried about it at first and then anyway and it was fine so he's also expecting that we would block must give it heat most those emails that behind us so that's why you do this so the other thing you can do is just mix an internment and then you know that's priceless so a system overview an underpowered VM is required you want to do this right it was it started off Bluetooth 4.0 for 64-bit desktop 1 Giga ramble SED core than hard drive it ran despite the first time but it felt sad so I gave it some more cores and some more ram later on so it has two

cores in two gigs and we haven't had an issue even though you had some people trying to do fun stuff like run burb against it and just fine this person was the spouse of a staff member who decided to lie in from their work ID and then jump into a VPN and what they didn't realize that I was trying to do the entire time this was the parking fish by the way and it's just over I'm really upset about it these were really upset so you know this is just an example of the shenanigans that you're going to see because you know users sense that they're smarter than you and some of them are afraid

example last time we did this the first temples that were submitted was the username of penis so I looked there is no at the directory they have that users land to that I didn't wanna be rude Nick maybe that guy's name is penis I feel that it's not so and then it also has things like you're an idiot nice try a few this is BS and then one time someone tried the login is Julie so your house are pretty easier now so she pulled it out of their mailbox so the system architecture I go fish the source product I encourage you all to check it out is what you want to do it's free right because if you do it right you

should never pay more than a couple of dollars in campaign if I use a free standard account don't tell me either and then Namecheap for registration and dns because i I want to keep it as a is this legitimate as possible I don't like people look into it too much right a stabbing stabbing user could figure out that that domain had a DNS record for aku IP address but nobody did that because it's themselves so you know keep it cheap and keep it easy so and go over a little bit of go fishing than some statistics so this is the dashboard this is the old one it looks similar but they're a lot more features now I had

done all that your custom stuff but now that's unnecessary so it is wasting my time because now it's all part of the main builds it's gonna get up so you'll see the plain overview it's a single panic glass there's your marketing puzzle reach out so some of the stuff that go fish collects and some of the stuff that we collect as well so go fix collects the number of clicks number of submissions number of open emails why would you need that well it's a good way to see kind of how far your your saturation was right because you're going to have people that click you're gonna people open every email that don't load that traffic it's just a

one by one traffic fix will live it keeps track of you get ahead key address the user agent string and then some things we find we get the department a job title of every person and then we find the off campus versus on campus rates and then I normalize all the data so I get a better idea of the different size of departments of job house because that helps us so some of the statistics and this is the average of the last four for a year sixty percent of clicked emails happen to off campus so that's off the network 55 percent submitted credentials off the network 65 over the emails network and then 56 percent of people who click the land

will go in submit Kretz so Alex Melman about how you keep track of that - but if they click the link and then and then they get there there's a 56 percent put their heads anyway so the the fishing paid wasn't enough because we consider a failure when you submit credentials now clicking a link it's dangerous of course we all know that however in the spirit of the game issue if you go to the page and look scrapping out go any further then we'll give you a pass but you know you gotta keep track of that too so the most common operating system was Windows but the most common browser was Safari that's scary so the average would look

great with less than 10% released at 7 it's about about average if you subtract the whole barking thing so yeah because the k2 fish had a failure in 27 percent so our process several steps you the first thing you want to do is going to create the templates as Julie said you're gonna want just generic or the running or an error another they'll pull your stuff out of your abuse box and don't don't be too clever because you know don't assume things make an ass out of yourself you know you can see here that this is what was sent and I'm not creative or clever so I just pulled this right out of the abuse box um when it

sent it it got it it's really great you know three convenient ways to pay there's two listed and they're both labeled one at the top you get apartment fine very excited to that so the URL ml link leads to K you parking behind I win this is a good one this you know knowing our parking department they would do something anyway but so so some people didn't notice via that top line there and I got this someone sent this to me a little help from Oprah so you know that's what happens when you're going to fix your IT staff some of their going to get clutter so this is the landing page for a different pitch an HR

pay one because we felt like this one was necessary to do since we will have people those money so using HR pay page now to the naked eye it's a beautiful spray so now this is a look normal that they justice if you go to our HR page can use HR page that looks like this there's some special things I did to it to hopefully help users steer clear of it like for instance all these links up top that link to the Google search for phishing email and then these two email addresses linked to the abuse pitch so no one notice did no one told me about it so all that work was for nothing but I gave

people a chance they'll still quit I have to use something credit or if you some of the page too long because they put an HTML tag that after 45 seconds because I learned that once you click the link most users will submit second Sirleaf so some math they are math is fun so about 45 seconds if they sit too long or go ahead and redirect here anyway because I don't know what they're doing and it scares me so they get here and they can read about it what they did wrong and that it was bad so first to create the groups we do groups by status the University so they have faculty GTA series all the works and

everything they're all special so I've learned that with the go fish it's a little especially when you're running on underpowered VMs required the having the group's bigger than 3,000 people kind of struggles a little slow to pull up the records so I suggest less than a thousand so for 14,000 groups we had that 12 some because some groups were thing here's where you do that created a group that it takes us week I don't use any of that I there's an API I thought that makes it way easier so I have I have a PowerShell script that will generate the users based on groups and then have also posted does it just have a regular resume that as well that's

what I do it way easier and you can want to get through your campaigns I say five minutes per group so the standard operating procedure you want to notify the appropriate people as Julie said just gonna tell us them other people specifically what the next like your email imitators and probably number one because it's really embarrassing when your phishing emails get eaten by email I Jean you look at new so tell them give them at least a subject center maybe the URL message doesn't get even do a knock or sock they might want to know your IT Help Desk if you don't and that's SP they should know - this is specific message so they know

not to tell users to change their passwords because that's what's upsetting so I have to know all that stuff he really sounds so we sent quarterly fourteen thousand a male's and that includes everyone on campus that's that gets paid by K you regular students aren't included because that VMs and since we were sending we sent to an external SMTP it comes in its the email system we have doesn't it can handle it just fine so I worry about too much because you know systems it out of emails with their port so you gonna let it run I mean if it dies it dies so you just let go and so usually if it runs that day they usually won't kill it

until we have told all the messages back so that can take a couple hours - so I'm gonna be done until it's fire collect your spoils do it you lose your data Julie prefers to learn four charts so full man is your hydrolysed donut charts she lost all of those last things if you're low on money it's a good way to get money so I have to do is just change people direct deposits we leverage some of the statistics chained together to improve our email hygiene significantly so plus attacks do little things all right did what we do these you're gonna build these out and you're gonna have you're gonna lovingly crisis phishing campaign you're

going to tell everybody who you think needs to know and then you're going to have whoever your ashame is click that button and some of those emails and then stuffs gonna happen you need to be prepared just mentally prepare for the fact that every time you do one of these there are going to be unexpected consequences like this so so when we would meet the networking fish like I say we politest for that though they think they're all maybe maybe nice with the Kait twitter handle and you know I'm really glad that this one happened when it did because the OBEs kind of belong at one thing Twitter lately they're kind of they just surpass them they're

thousand followers I'm really glad they and they have hundred thousand followers when this happened so so but this was the this was the prime example of unexpected unexpected consequences we did not think that people would take a day off work to drive the deficit and warehouse in my parking ticket or call the police or show up at the gate we just we had no idea we know a lot better now but even then we're still going to do still going to be unexpected consequences whenever our recent fishing campaigns was just a run-of-the-mill you're you are about to exceed your mailbox quota and click here to get more mailbox so we had about 60 people we're

like wow we're going to click that but I will call a help desk and request a bigger mailbox just wait we didn't expect them to ok follow instructions but they did so well when you are we are collecting the statistics from newer fishing campaigns you should not keep them to yourself get your management reports if your your administration are your c-level folks so your folks in terms of security spam reports the statistics around off campus versus on campus clicks were directly responsible for us finally adding features to our mail hygiene that do URL rewrite and allow us to track the fruits even when users are on campus I consider that a huge magnet for this especially

since we spent of how I have developers on that so get those reports package them into statistics and package them in ways that are meaningful for non-technical people your report needs to have a concise executive summary and good storytelling because let's face it as much as we would like all the people who read our reports to read the entire thing and appreciate our well crafted prose and are confidently into qubit graphics a lot of those folks are going to read executive summary and organic glazed over because this is nervous stuff and I don't understand it so so you need to make sure you're writing for people who don't necessarily do what you do or fully understand what you do and after

three or four exercises you should start working to identify trends do you have job types or locations or departments where you have a particular uptick and people who are going to click through work of those departments or work with those supervisors to identify why is this subset of folks and in trouble of this you need to be targeted training do you need to reimagine how you're doing your and your required training do you need to do more fishing exercises of this these people and finally one thing I would say is to get get your thick skin ready I mean I think all of us that work in information security have sort of developed that over the years but I

felt like I had this step enough who never level of this because inevitably every time you run these one or two people call me up and read me the riot act about how I'm gonna get the fire and I'm the meanest person in the universe yeah yeah and I understand that this can freak people out and I I really do empathize with that but one or two people out of 14,000 I mean I'm gonna weigh that feedback accordingly and based on the benefit that this is shown I'm not going to let a couple of angry people stopping so what's next for us one of the things we're going to stop doing is notifying supervisors I had the

variety and I will have this this was my cracking idea to notify supervisors when people actually clicked credentials this didn't get us anything and then actually I think in some ways hurt us people don't want to work for us work with us when we're tackling on that so this needs to be a positive positive thing we want people to feel good about reporting things to us rather than if I interact with the Security office they're going to sell me out to my supervisor so we're not going to do that anymore um go fish is fantastic go fish go that's over a hump when we didn't have any money to spend on this kind of thing

but my next goal is to move to a commercial pool it is it is rather go vicious at this point at least rather labor-intensive for us and I feel like it's got enough moving parts that it's easy to screw it up so I would rather just move on to it's a little that lets us craft the message mash the button and collect profit at the end right and I don't wanna have to worry that I've been a blow up our mail system or or sentiment to people down the wrong rabbit hole with it with a broken tool and finally all of my ones like that if I budget for it lol is to start worrying

our users I want to be you know giving people praise for being the first to report message either overall or my departments and prizes sure but we need water to get them it's gonna have pretty come down this week and again being the first to report being the department with the lowest percentage of click throughs etc etc it's all that you know emphasizing the positive and showing everybody hey good stuff happens when don't click these messages and when you work with the Security office on recording these things so finally I wanted to share some helpful links this issue these are the things that we worked from and were built as we as we put this process together billfish is

the open source fishing framework it's fantastic go check it out I'm gonna make these slides available so you don't have to take pictures if you don't want to and and it sounds like the events the development on and even since we started using it so definitely go check that out Shayne's github a little kids have one as we negotiate to poop in it so baby yeah I don't it's to his former colleagues like don't quick change it up you can put it as mine Brad June is his life here at the University of Colorado and wrote a really great white paper on on building a fishing program for users even if you are not in academia or in the edu space

or anywhere near that I would still suggest you read this because it's it's really excellent a couple of things on the Nigerian scammers emails the first one is a fairly accessible news article about the second one is the actual study that the guy Microsoft did it's a little bit more impenetrable but slower to read and then finally measuring security where's program who's also nice and it wouldn't be the same presentation and somewhere so there this that's all we've got that's our contact information please do feel up to reach out to me or Shane on email or Twitter we're always accessible technical questions about setting at billfish and definitely go to them but if you have

other questions about the managerial side then please get in touch with me and we really appreciate the committee pronounced there's the there's to be thanks bye thank you round applause [Applause] [Music] any questions we've got about managers shy of 15 minutes or so for questions I heard you both mentioned something about at the end pulling the emails out yeah you happen to like the exchange administrators delete doesn't want so we do have any exchange of industries delete those we actually have we within the last probably three months they've built some PowerShell scripts that allow us to do that in an automated way so we just submit the sender address NIC event pulls a model we do that because the

unintended consequences issue after the help desk got there 15 percent of requests for a quota increase they said is there any way you could pull these messages because we're still getting these calls nice yeah we can do that we also realized that when we get a real fish we pull those so widely between v3 differently so we just started the in that day we kicked off the process holds all the messages and Inbox then we move on with our lives yeah so did you drill it a little personal you have to do any intentional management of people perception we're out to get them and did you have any supervisor supervisor thing a little bit the anybody say I demand a list of all

yeah so we did when when we send it up that heads-up email before we do appreciate everybody that's gonna receive the phishing message it's a heads up email that says we're going to send the phishing email here's how the spot that here's haven't reported here at rules of engagement and the rules of engagement at first said we won't tell anybody if you clicked then when we go on had an idea to actually start recording the supervisors we said if you click through and submit credentials we're gonna find your supervisor and that's when the angry phone calls released ready come Annette so we just decided it wasn't worth it and and backed it out how to behave a

like repeat offenders I imagine you guys saw the same people continuously did you yeah that might be a process of doing things we don't and that's that's an area where we're still trying to decide what's the best way to handle it because honestly the repeat offenders are a lot of times at or above my level or pure so so again the interest of continued employment we tread carefully there it's definitely an issue that I'm interested in pursuing but but it's not something that figured out the best way to handle yet that was actually related to what I was going to ask - we have you we start with the exact same thing because on one

hand like you said you don't wanna make it punitive you know there there shouldn't be people shouldn't be fearing you know getting these emails and things of that nature but at the same time if you know Joe and accounting has failed all three of their OU's you know fishing tests well obviously some sort of training needs to take place there needs to it gotta balance that knowing what your employees are doing and and we have to worry I don't know what what Kansas's work laws are but in Missouri where right-to-work state so that's especially in higher ed when everything is so segmented I know I have to worry about is the information I'm about to get to

this manager possibly going to enable them to fire an employer that I don't want that to happen yeah Kansas is right to work also when we have a kind of a very small number of represented employees most part or right to work I and I serve a pleasure my administrators why even have less ability to argue about getting cluttered most people do so yeah we just decided to tell the supervisors wasn't my head I'm coming thank you I think this question actually brings up the better question my original line is there any have you given any consideration along talking dates are about how the fisherman tests shouldn't be considered in any way any form of any form of action worthy of

like disciplinary action or a black mark of a record or anything like that you know communicated with HR in terms of defining anything like that you learn wondering if some places in some way would or would want to use the results of that test as that members will do when we started doing supervisor notifications we stressed that this is not something you need to take action on this is not something that's meant to get the employee in trouble we just want you to follow up with them and make sure they've taken their annual training but then we we fell victim to assuming that all supervisors were reasonable people and wah-wah-wah-wah we know that's not true so so we decided that it just it

wasn't worth the risk and it wasn't worth the bad feelings during the free lunch so I have a question on the other side we fixed all our new users 30 days after they are hired because they have completed their training then and the recording of the fishiness is a disappointingly low we get a 6 percent click-through to credentials when we get about 3 percent reported as a fish so we would have been food ready our annual training did you do this Bureau or as well spirit to make any changes or improvements to the training year of dividing people a little bit not much we we ruled out the annual training prior to starting the fishing exercises and we never really

made any updates based on that yet but that's that's that's in planning and the trainings going to emphasize social engineering a lot more work than it does right now because that's the big deal right but we still find that the annual order being quarterly here's how this by the fish message is it's what drives people recording things to us the strongest I don't have the numbers on but our our a recording rate of people that report fishes to us is way like way above the average I think is like 3% for higher ed and ours was somewhere in the 15% range of people who report messages to us plus we get a lot more of just the standard phishing and

spam type messages are foreign to us and yes this has also increased the number of messages that we get through life who cares like guess someone's spam you for fake viagra who cares but we feel like dealing with the cruft is worth the amount Jessica get out of that so the launch of sites you had a metric that sure in most autonomous videos and but the most common prowess yeah we're not sure of it and the second thing I was kind of wondering when you start showing the statistics for all that we've normally based on 90% of it goes and orally so so the the operating system versus the browser of Pellissippi iOS was also included

separately that's that comes from so Apple iOS kind of had a lot okay so that's why Windows was the most abused operating system the supplier was investees browser as far as normalizing data hyah I took the numbers of people in departments because we we normalize the data by department and title the other day that was not normalized so it was by in clicks so I normalize the click data by Department and by title because of course there's a disparity between people you know there are times are there's one person with a title because they're special and that other times where there's one person in a department because I to mess it up or HR nested on so that was for up to

it's hard to normalize too because you know one that one is a hundred percent how do you normalize that but I didn't like best and I tried to move people into Bruce that seems similar right so it's one of our assistant administrators to said that they don't want to be in the Department of Information Technology but of IT move aback I was also able to tap into a chars actual database where you know the acting people can't touch it I hold that information but even then people have their own departments for some reason I don't know money it's all about buckets so normalizing the data it helps helps us see where the trends lie right so we

see the high spikes in some department and then it could see kind of level off as the numbers go down but also to keep the the other side of the the table there how many people are clicking - because you want other numbers as well that's what's most important and then it's best it's know where they are - so yeah we have we have done normalize and the normalized and I include the normalized and the spreadsheets that we handed up because it is easier to read and it make more sense for our purposes attributes one to pick out the people

under the camera but I guess we're not going to avoid that well that's a lovely FaceTime video it's mine you mentioned at the beginning about being the college with people with us as a second language have you guys ever investigated that see what the differences with the efficient in things as to how many or what percentage of that is people we have and it is something I very much want to do it just have to have the pet cycles to do but it is absolutely on my list of things that I would I would like to investigate because I because back with a napkin math tells me that based on the number of GPAs GREs we have and that

they are one of our top categories of victims every time and they're aware of those people who do speak English as a second language I cannot help but think that areas of translation that you had

I was just curious in the the time period after you said about the work and I'll stay here you're gonna be giving away his campaigns do say chains and how people treat efficient residents enormous yes have you like maybe his will send one that would be like Wilson it on my neighbor to say where in the fish you guys later on this week and then every day opens though we actually sent it which is when you guys was this hey you guys I think this because you guys but and that's not only do we get accused of every phishing message they get being from us we see a huge upswing and overall phishing messages reported

because they're just diamonds and they know if they caught us or not so even though we know what work prices people get big personal satisfaction how department I caught use it so so yeah yeah one minute so yeah any other questions all right I think we'll wrap it up there no round of applause