Weathering the Storm: The Art of Crisis Communications

alright just a couple quick enough that you know we want to we want to thank our lead sponsors which we're a hacker and Fitbit and don't forget that there's a bunch of stuff going on throughout the day there's the party tonight at eight o'clock in the DNA lounge there's a hacker jeopardy here in buzz works from six to eight pm the lockpick village there's just a bunch of stuff so make sure you take part in that and now I'd like to introduce to you for this afternoon Jen Ellis and Josh Franklin okay thank you very much okay so we are roll out we are loud that's good hopefully you can hear us yes I did

start loud and the heckling has already begun goodie okay so yes we're here to talk about crisis communications also known as how to survive a catastrophe and I'm going to be a second guys just look at the picture and really appreciate the picture which does prove you can find anything on the Internet if you only search hard enough um so I'm not going to bore you guys with all the stats about the fact that bad things are going to happen you working security probably and so you probably already know that we are doomed so a crisis was going to come your way you need to know about it you need to figure out how to

handle it and typically in my experience security people will focus on the technical aspects of response which is great two thumbs up you should totally do that that's kind of your job but there is a lot more to it than that and coming out the other side depends on how you can handle the other pieces so the other pieces are what we're here to talk about and i just realized in my excitement it didn't introduce us so just briefly I'm Jen I head up communications at rapid7 this is Josh everybody give Josh a round of applause Oh Josh heads up security at rapid7 and you could also subtitle this talk Josh a nice story how we fell in love over

crises all right with that terrifying thought I'm going to move on because just runs out of here screaming okay so the basic gist is there are five main element to how you deal with the communication side of a crisis the first one is you need to make a plan the second is that you need to manage your stakeholders that's a really important one and it's really hard to do when it can make or break a situation the third is that you need to monitor the situation we're going to go into all of these in detail so don't worry this isn't it this is like oh yeah the talks done by you need to input arm your

internal team and then you need to inform those affected which is the thing that everyone focuses on so everybody focuses on that last week and there's a lot more to it than that so we're going to talk about why and how so I'm going to hand over to my glamorous assistant we started falling in love well before any crisis ease again closer to my mouth this is this is interesting yet ah so i think when i first started working in the security world I remember people telling us you need to have your incident response plan and it was like this 80 page document that you would refer to whenever something went wrong and I quickly figured out that it just

doesn't work like the 80-page incident response plan doesn't work and so I think when when Jenna and I started working together on on managing crisis's which you know happened especially when we when i first started more often than I'd like to admit it was it was disorganized at best and what we quickly figured out is by like doing a simple amount of planning by thinking about things and i forgot to reference the bobcat so i apologize but Byron by recognizing the winter is coming and planning just a little bit and socializing it with people and it will walk through a little bit how to do that you can make these processes go much much better so our hope isn't to stand

up here and say write your 80-page plan or hope is to give you a little bit of an idea as we go through each step exactly how we do it and how you can go home and try and have a little bit of an impact where you work and so this is intentionally a scary chart are this this chart represents how rapid7 responds to critical security vulnerabilities that are released in the wild and this was a very rough draft assembly was my first draft is it accurate and this importantly you guys see a bunch of letters this follows what week what's known as the racy model and it helps you figure out who's responsible for doing work who's

accountable for that work or the final approval for that work who do you need to keep in the loop right who's informed versus who do you need to see feedback and advice from and so what you realize quickly when you enter a crisis is that there's a complete lack of trust across the board everybody wants to get involved nobody knows what they're responsible for doing and the really cool thing about this and the reason we like using this is when I sent this to Jen to the first time so the people that could see me pointing at rapid7 we have to we had to separate marketing functions we had product marketing and then marketing and they were different

organizations and Jen owned corporate communications public relations and many other things that aren't relevant to the story and what we did what I did mistakingly was put social media media response as a bunch of other things under our product marketing team and we like to call this out because by spending a little bit of time taking the chart like this I mean maybe took me a couple of hours to put together and circulating it we were able to before we were in a situation where we needed to be communicating with the press or writing blog posts get ahead of who owned what because you can imagine what would have happened in crisis if a press

statement would have gone out and it wouldn't have gone through the person who owned press communications and so moral of the story is fine what model works for you but to spend a couple of hours sitting down and thinking about what crisis is could impact your organization and how you would organize your response it doesn't have to be 80 pages long thank you so I would say before you get to creating this or as you go through this process one of the great things about this is it's a forcing function for you to get to know or your team players you're going to identify the people who are going to be involved and they may be or impacts are

probably likely to be people you don't talk to you every day so you know Josh and I work in a security company and so unusually we know each other but in most companies if it's not a security company the chances of the head of comms having much of a relationship with the head of security are pretty low going through a process like this means you're going to figure out who the key players are and you're going to build a relationship with them and that relationship is something you need to invest in so like go take them out for lunch learn about who they are learn about what they care about what stresses them out what are

their triggers get to know how to work with them that's really important and that means that when you get handed a thing like this to look at an eye chart and it's wrong you're much less likely to you know eviscerates the person that sends it to you so I've got to make friends with your coworkers Jen would never eviscerate me she loves me too much so another really scary diagram so when we talk about writing process flows instead of having you know 20 pages explaining what this is this for us is a guidebook right it's not necessarily a prescriptive set of things that have to happen and have to happen in the Indus order but what it does is when when

we're in the heat of the moment we have a way that we can pull out of documents make sure we're not missing something if we've made a mistake or something that we want to improve on in a previous crisis we have a way of capturing it and and I think very importantly what it does is it lets stakeholders visualize what's going to happen right so again if you send somebody an 80 page document they're never going to read it they might tell you they read it that my read a few pages if you send them a flow diagram like this it takes them 60 seconds 120 seconds to really look look through it and gather an opinion that

way when something happens hopefully they've identified a box down here where they say something like actually this team doesn't do that you know my team does that and so instead of having that conversation in the midst of crisis you can do it via email and just just add so not only does this help you figure out what the process should be and who's doing what but actually it will make you much more productive it will make you less stress so when Josh and I started doing responses to a variety of different types of incidents when something happened we would instantly create a war room it was super disruptive everybody would drop what they were doing we'd go and hustle into

the little room together which was disruptive to the broader team because they would see us being very suspicious and it was all very tense and like we would sort of fight for control a little bit we went and loved anything this was at the beginning and and so it took a lot of time now what happens is something comes up Josh lets me know I trust him because I loved him and so he tells me hey this is what I'm going to do this is when i'll bring you in I know what the process is I know he's got it he I know that he understands what I'm going to need in the situation to

succeed and what we want to get to as an outcome for the organization and so that means it doesn't disrupt my world anymore I just carry on doing my stuff it's like I'm like all right whatever you've got it if your practitioner right Jen is one of these right and so when you think about when you're managing a crisis everybody wants to get their hands involved and so the more comfortable you can get people with what's happening and the more comfortable you can get them with knowing they're going to get involved at the right time your your job is going to get much easier oh man you stakeholders is me awesome speaking of stakeholders clearly I like

to manage stakeholders this this is I think one of the most deceptive things for people that are our practitioners the importance of keeping your stakeholder is comfortable communicating with them through the process and making sure that you are doing it in a calm and collected manner because your stakeholders are going to mimic your behavior is the single most important component of a response Jen said something a little bit earlier the monitors messing with me Jen said something a little bit earlier which we the first time we gave this talk actually kind of popped out as an underlying theme which is take your stakeholders to lunch right it sounds really silly but people get very tense

especially if they're not in crisis situations frequently there's a biological reaction that happens with people they get tense to get short and if you don't know anybody on that stakeholder team you're missing an opportunity because what's going to happen is if you you know once a quarter are getting launched with your head of communications your head of legal whoever whoever your relevant stakeholders are it easier it's going to be for you in a time of crisis to know when they're getting snappy at you it's not because they're dicks it's because there's they're under stress and that will change your your dynamic a ton so we do a lot of vulnerability research and so we disclose vulnerabilities a lot

to a lot of different size and industry organizations and in my experience will frequently talk to the security team the security team kind of understands what's under the disclosure is all about they will work with us through the process this is in the good situations where the security team understands or vulnerability disclosures about not in the myriad situations where they go moon so we work with a security team they're on board they know what the expectations are it's all good and we go public and the PR people who own the reputation of the company go what the [ __ ] just happened and I've seen it in big organizations where that went up the chain from the head of comms to the head

of business like the CEO and came down again under the head of security in a really profoundly unpleasant way and then I'm on a call going how I really sorry but we told you four months ago and so don't be in that situation those from the sense of alignment and some sense of expectation so that if you have someone come to you and say hey there's a problem with X you can give forewarning to your GC or your PR people or the product people like whoever it is build those relationships it won't just be the calm person it'll be people across the organization figure out who they are yeah you're next look at that monitoring okay so monitoring I know

right arm pictures of cats great what runs the internet and so monitoring the situation is be like deeply unsexy part of all of this like yes monitoring always sounds like it's real action piece real fun the reality though is this is critical for a number of reasons and one of those is lawsuits so there is a reputation aspect of this where you need to know what's being said you need to understand how to plan timing of any disclosure you need to be aware of if people already talking about it in the public and so monitoring the situation Morenstein social media is extremely important from that point of view but there's a secondary consideration whereby if there is any potential at all

that you are going to face legal action you need to be very much on top of what's happening in social media because anything that gets shared publicly can be cited in a law case and that's a challenge right because we had someone used to wear first and she had previously worked on a situation for a company was a doctor and storage company and they had a warehouse fire and it was a really big deal huge huge warehouse fire and when that happened people were finding pieces of paper on the side of the road or by the river they were taking pictures of them and put the math on Instagram or whatever and they weren't naming the company in the

social chair because they didn't know what had come from but nonetheless those documents that were being photographed and put online could be used as artifacts in a law case so her job was this year to figure out how to monitor social for all of those things and then basically like catalogs and get them taken down figure out what to do with them work with the lawyers on that so when you're working with your comms people bear in mind that as the security expert in the room it's your job to help them think about all of these things that they may not be aware of my experience of working with lots more becomes people a lot of them have never

worked on security situations before or disaster recovery situations and so the guidance you can give them you don't need to be the expert on how to do the social peace but bringing up to them the fact that this could be a consideration and that we should be watching that's something you can do you can help them out with that kind of thing okay I do the next one and then you do with you upset okay we're really well Brooks it's great we're great um funny cats hole yep taryn yeah yummy yeah so here's the thing people always focus on external communications and that's really important and we're going to get to that but your internal team are the people

that are in the trenches there are people who are going to get questions they're the most invested in what's happening and they are going to freak out and if you're doing that thing that I talked about earlier where you're kind of going off into rooms and having very stressed looking private conversations your team is no stupid they're going to notice that [ __ ] and they're going to be like all something's happened and the you know a team of the world is coming to an end and then they'll stop speculating and gossiping this is bad news so firstly you have to think about how you're presenting yourself to that team and you have to you know do the

duck thing of gliding on the surface and paddling like hell underneath the second thing is you have to think very carefully about when you're going to disclose the internal team and how bear in mind once he disclosed internally to the broad team like all of your colleagues that is now public you cannot just those people to keep it private it won't happen it is now public so you have to make your timing very carefully measured and you have to think about what to share with them if you share information before you've nailed the facts you will probably cause confusion that will come back and bite your mouth we've had that problem myself if you share what happens but you don't

prepare them to get questions then that's going to be difficult because their customers are going to be phoning them they're not going to know how to deal with it and they're just going to be saying things on the phone in the heat of the moment so this is where an FAQ is your best friend and you want to think about what are all the worst questions you're going to get and really push yourself to like think about those nasty questions when we write these things frequently people get offended by the questions I put in and I'm always liked it because it doesn't tell me for me this is what other people are going to ask us and we need to be prepared

with an answer like hopefully we don't get asked to be prepared and work really closely with your general counsel and your comms people to help them figure out how to answer it again they are the experts in law and communications but you're the experts and security so help them help them navigate this so finally the bit that most people care about which is the external communication there are a number of elements to this people tend to focus on press because that's the really scary thing the first step on this though is what are your legal requirements for notification there are 47 different state laws around breach notification your legal team needs to be familiar with them and

understand what the expectations are if you handle into the response in your organization you should also be familiar with them and know what the expectations are that's the first thing you have to think about how you're going to notify those affected then you can start thinking about what that means in terms of press and partners and social media and all those other pieces and again timing here is critical if you go too quickly you will do more harm than good and you'll create confusion and panic we had a situation where we didn't create confusion and panic but it was so suboptimal our DNS host with axis going back a few years ago and when we were

doing the investigation we got some information that indicated the hack may have happened via fax and people kind of saw that was funny and cool and so we tweeted it I think something like hacking like it's 1969 was first week and then as we went further we found out actually he's like faxes weren't involved and it was much more mundane how they'd done it and and then we couldn't take it back like we put that information out there and in this situation no harm was done but the reality is we put out information that's just plain inaccurate and we could have done a much better job to have waited and put something out that was right now

your comp team is going to feel a lot of pressure because they're going to have reporters I'm staring at an evil reporter right now they're going to have reporters beating down their door trying to get a comment and they're going to be writing articles like they're not going to wait they'll go write the article and lov speculation and we all know that there'll be lots of people in the community tweeting and speculating and blogging an offering comment and that's hard to deal with it's hard to say firm and you need to work with your comms team and your legal team to figure out what you can say in that situation what you can and what the right timing is and

I unfortunately cannot tell you oh the timing of three days or the timing of three hours it depends entirely on the situation and you have to help them read the situation to figure that piece out the one last thing I'll say on the public aspect is just bear in mind that there are lots of different mediums that you need to use it's not going to just be pressed it's not just going to be social you might blog you might need to think about our people going crazy on reddit I mean there's all sorts of different things that you'll need to think about is there a conference coming up and you need your booth staff to be

prepped so you need to think about all those different elements and again your comms team should be thinking about this but you can help them navigate this so you can probably tell there's a little bit of a mix of broader crisis management and security response to to anyone you face there and I think one of the things that we run into all the time both as a service provider and service consumer is that there will be situations in which we notify people that there's been an issue with their data and we're also being notified that there is an issue with our data so it's definitely cyclical one of the most important things that anybody can do in

the situation is be authentic to frequently there is pressure if you're getting on the phone with a customer to try and you know soften the language or or to do something to applicate the truth right maybe maybe if we change these words as long as they don't ask a follow-up question the damage will be the damage will be much smaller and I think in our experience at least what we've noticed is when we're interacting with people and we're disclosing you kind of a kind of fall into two camps one is unless it's a real substantive issue they're going to go oh thanks for telling us like we appreciate the transparency and the other camp is

you're going to have a group of people that are I'm going to use this as a fire drill right they're going to get worked up about it no no really like the fact that we just lost your IP addresses it with no other context doesn't pose a lot of risk it's not acceptable but so if you can start with a degree of authenticity and make all of those stakeholders really understand what happened what you did to fix it you're going to you're going to make the process much much easier because as soon as and this is for me too as soon as the students I'm dealing with the service provider and I can start to tell that

their wording things they're trying to be a little skittish around what they're saying like I'm going to go right to that thing they don't want me to ask and the worst situation you're going to be in is when you're on the phone with a customer and you're trying to work around something that's a little embarrassing and then they ask a question that you just can't lie about and you respond and it's embarrassing and then it ends up on social media that's going to be a lot more harmful for you than when they're like wow like company X just told me this and you know that sucks but they handled it really well I appreciate it much maybe like the

last pass reach was I think one of the more recent examples i can think of all centricity is really incredibly important in it and on top of that as security practitioners you need to make sure that you represent that when you're working with your comms and your legal people it's very very easy for people that haven't worked in our industry to miss out on a lot of the sub cultures that exist in our industry so you have to represent that to them because if they don't know how that works and it can go back to some of the points Jen was making on the last slide it can cause a lot more harm to your company's

reputation and then otherwise would have occurred so on top of that it was funny when we first had this slide we talked we talked a little bit about how we had to reassure the affected and then we fall we said well reassuring can certainly be condescending sometimes because if you've just lost like you know a million patient records you probably don't want to be reassuring them that everything's going to be okay this I think this ties back to authenticity and that if your focus is helping them be successful right not worrying about your own self-interest but helping helping the affected individuals of that crisis be successful your whole response is going to go much smoother right and you'll see it if

you're ever in a situation you'll absolutely see when you're in the scenario and you have and you're trying to have a conversation where you're not being truly genuine it takes a lot longer to prepare prepare for it's a lot less smooth but when you are focused on helping the people that are impacted your being authentic it actually makes the whole process dramatically easier so I feel like this cat is my spirit animal and so I just one thing I want to clarify as you know we talked about the importance of authenticity and impart that is accuracy but don't confuse accuracy for the need to share every little detail over sharing is one of the worst things you can do it will freak

people right out the number of people you talk to who have the level of knowledge that you have about security probably going to be loved if you are talking to another security team that's a different situation then they're going to ask questions they're going to want to have detailed answers but if you're talking to consumers and you over share you're going to terrify them the things to focus on when you are taking an approach that is centered around those affected is what is the effect on them and what action do they need to take to remediate those are the two questions to ask yourself constantly that's what you should communicate first and foremost every single time make it easy for them

okay so we went through a bunch of stuff in a not very long period of time so I'm going to quickly go back to what the five key learnings are across those five areas of activity that we talked about so the first thing is you have to prepare well go make friends figure out who does this in your organization Berlin love the second and I wish this wasn't true because there's no hard and fast on how to do this but timing is everything go too late and your dooms the media story will get away from you you'll look like you don't care and your unresponsive go too early and you won't have your information nails and people

will get confused and they'll panic don't be overconfident don't go out and tell people it was a fax when it was on the facts you'll just end up with egg on your face in the best scenario and a lawsuit in the worst expect that you're going to get difficult questions prepare for them arm your internal team on how to answer them whether it's the entire organization or just your key stakeholders and as usual for the last is always always always let your guiding principle be what is the best thing for those affected let that be your compass thank you very much we've been Josh and Jen [Applause]

you can take a bow if you want yeah josh and jen thanks very much your friends at fits because i want to thank you guys too so here's a gift for you guys are speakers from pizza there you go [Laughter]