Anna Bertiger - Bombs Are Easier to Detect Than Pieces of Shrapnel

So, I was told I should ask questions and give away prizes. So, maybe the first question, uh, who can tell me what an email bomb is? >> Back corner over there. It's first hand I saw.

Yep. All right. So, email bombs are uh there's a variety of motivations and one of them could be to hide some other terrible behavior. Um but they are uh the equivalent of

for your inbox. So, uh you get a lot a lot a lot faster than you can delete them. Emails uh that are going to make your inbox useless. Uh so if this happens to you or it happens to a user on a email system that you run, you're probably gonna have a sad day because uh they want to go away. Uh so what are they for? They So I guess here's what we're talk about. We're talk about email bombs, what they are, when is an email bomb happening, how do how do we know? And once we have decided an email bomb is happening, that's that's where some nerdy math will come in. Uh what do we do about that situation? So,

uh, email bombs. What are they? Uh, so this is I stole from from MITER. Uh, this is the MITER attack. Uh, uh, description of email bombs that flood an email address with an overwhelming volume of messages. Like many more than you could. Like, you can't hit the delete button fast enough for these people. Um, I mean, here it says hundreds, but like I have hund You could delete a couple hundred inbox email messages. It wouldn't make you happy, but you could do it. Uh, thousands in short spaces of time that just keep coming. You can't delete. You physically can't delete them fast enough. And so, how do you get rid of the problem and get back to business

as usual? So, emails are cover fire for something else. Uh, first of all, they could be cover fire. They could just be general misery and business disruption. And that's a thing, right? For some reason, someone wants to make your life hard, make a point, whatever. Or they could be a fake IT help desk scam. Uh, not not unheard of. So, right, first I send you an email bomb that says so, so now your inbox is inoperable and then I call you up and say, "Hi, I'm from the IT department and we see that your your uh inbox your email inbox is broken and don't you want to let me remote desktop to your computer to help you fix that

problem?" Uh, no. No, I do not. That sounds like a terrible idea. Uh, or they could hide evidence of an attack like a password reset or a credit card fraud where there's a whole bunch of terribly wrong things and you get lost worried about digging through your thousands of emails and don't notice that someone has charged 10 bajillion dollars on your credit card or that they have reset the password to your bank account and are prepared to take out a million dollars of loans in your name or whatever other horrible So, uh, in general, email bombs consist of three sort of things they could be part of that I've seen. There's spam. Goodness knows someone could sign you up

for a lot of spam. Uh, there's gray mail. So, someone signs you up for every cat video mailing list on the internet or every poorly secured cyber security mailing list on the internet or what have you. Um, and then there is a third category called that I'm calling advocacy, which is I rally a 100,000 of my internet friends to email you because we're angry about something or or wish you did something differently like we don't like that your company did X, so we would like to email you and tell you that we think you should change your behavior, which I get. I want people to change their behavior, too. But there is a volume at which that's just denial of

service to the recipient. So many email the problem the thing that makes this problem hard to solve is that many email bombs are legitimate emails that someone might actually want. So filtering them out for everyone is a bad idea, right? I want cyber security mailing list emails, but they might be part of an email bomb for my mother. Uh, so filtering them out is a bad idea for everyone, but we need to filter them for folks that are having an email bomb because they need their mailboxes to go back to work in a sensible fashion. So, when is an email? So, then we need to answer the question sort of, is this an email bomb or is this just someone who

really likes mailing lists? Um, so the answer to when is an email bomb happening is statistics. Or you could be very cool and call it AI. Um, but I'll be honest with you, it's statistics. Uh, so the idea is so, so you might say, okay, listen, if you get more than a thousand emails an hour, uh, then you're getting an email bomb. Except then you think about your support mailbox on a bad day and now it's getting an email bomb, but actually it's just like an unpleasant Tuesday. So that doesn't work. So instead, you have to compare each mailbox to itself normally. So how how do you compare to yourself on a normal day? That's where the

statistics comes in. So, we model normal for a mailbox with a distribution. How often do I see, you know, 10 emails in a half an hour? Uh, looks like the answer is slightly more than 1% of the time. Um, how often do I see 20 emails in a half an hour, etc. And then that lets us quantify how unlikely is this behavior. So, how unlikely is it that I will see maybe 48 or more emails in a half an hour? Um, and well, that's how likely is it that I will see 48 or 49 or 50 and so on. Add them all up going all the way off to the right forever and ever and ever.

So, if stat one is fresh in your mind, I have just described a right tail p value to you. If stat one is not fresh in your mind, that's fine. I just described adding up some probabilities to get the probability that it's this bigger bigger. That's also cool. So, but now that lets me say how likely is it that I'm going to get this much email, right? Or more h how surprised should I be by the current state of affairs? And that's sort of the number we want to quantify because then if this is a high volume mailbox, I'm going to be a lot less surprised by 50 emails than if this is a mailbox that

gets two emails a day. Then I'm going to be a lot more surprised. So I want to build this model separately for each inbox. So, the advantages of statistical models like like the one I just described, they're fast and cheap. And and if I want to build one for every mailbox, it better be fast and cheap or else the whoever pays the AWS bill or the Azure bill or the Google Cloud bill is going to hate my guts. Um, and I spend enough time making those people mad at me. So, um, so they're fast and cheap. All I need calculate the parameters for that statistical distribution I was just showing you are mean and variance

which we all can compute I mean if you had to do it with a pencil and a piece of paper you could and you wouldn't be happy but you could right and and databases are well set up to compute mean and variance that parallelizes nicely if there's a bunch of numbers this is like do this in SQL all day long uh inference speed it's really fast once you've counted up your numbers to say, "How surprised am I by these numbers?" There are some nice mathematical formulas. You won't even notice how long it takes your computer to calculate. The other advantage is that you can control false positive rates because if you fit your model well, whatever

threshold you pick for email bombs, whatever p value you pick is how often it's going to false positive for you, right? How often that will happen under normal behavior. So that lets you say how much am I willing to put up with in terms of false positives here and then choose accordingly. Um the other big advantage is I don't have to have any examples of an email bomb to solve this problem which is good because they don't happen that often. So, it's not like I'm gonna have a hundred thousand examples of an email bomb to use to build this model to be able to recognize when mailboxes are getting bombed and when they are not.

All right, I'm going to pause here. Questions so far? Yes.

We'll get back to that. Um, definitely unfiltering out internal emails. Like, just don't count those. Um, I'll get back to the other one. Yes. Is that a No, you're just adjusting the camera. Um, all right. So, here is uh here's a real distribution mailbox approximately. Uh, I mean, you can feel a little sad for this person. Sometimes they get a lot of email, but it's a manageable a lot of email. And then one day they got an email bomb. Uh this is the same distribution zoomed out a lot as was on the last slide. So way over here is what you were looking at before and then they get an email bomb. Way over here this many emails all in a

short period of time. So So these are like this is bad. And also I can't hit the delete button fast enough, right? Like like it becomes a multi-week project just to press delete enough. So we need to do something about this problem. Um, all right. So, your question was the answer to the other prize. Email volume alone is not enough. Um, because lots of tickets in a short window produce a lot of false positives. Uh, I I I happen to know this cuz I caught some poor soul having an incident and getting a thousand Jira tickets as I was developing this and like I felt sorry for them. They're having a really sad day, but

I probably shouldn't just take all the Jurro tickets out of their inbox and make this go away for them. Um, that that probably wouldn't be good for their continued employment. So the way to mitigate this is to model distinct senders to mitigate this problem. Uh so um there's all the Jira tickets came from Jira. They're all the same place. So that's not a bump in how many people are sending you email. But most email bombs are coming from a very wide variety of places. Also, if it's just one sender, you could probably go write some sort of rule to block that sender, send it immediately to trash, and problem solved. Um, God knows I have had folders

I've never looked at on my in my inbox for years that contain tens of thousands of messages. Um, all right. So, um, you have high confidence of an email bomb, right? you've said, "All right, I'm really sure like looked at that distribution and way out here uh is is how many messages we're getting. This is not normal. Not even a little bit. Uh now what?" And the answer is now you can get really, really, really aggressive about dealing with gray mail and spam. So, uh here's an example of an email bomb. I mean fake data, demo data, um of a whole bunch of like congratulations, you've been signed up for this mailing list. Uh aren't aren't you pleased? Uh

and now you can get really aggressive about automatically moving that to spam for the user or otherwise addressing this problem. So remediate. So, uh, our product lets you do that. Um, one of the things we we recommend is getting very aggressive about mailing lists, right? Are you getting mailing list things? Are you getting uh things that are not safe for work? Are you getting social media things all of a sudden? Shunt all of that off to quarantine, trash, spam, whatever, whatever you like. Probably the person mid email bomb is not going to be real upset that you accidentally sent their mailing list item to spam while making everything else go away. All right, we'll pause here

more. So, so fundamentally, you know, fundamental trick here. You have to know an email bomb is happening to be able to do this though because otherwise the contents of your email bomb might well be something someone wants. And and so you can't just always be aggressive. I mean I guess I guess you could always be aggressive, but that might make your users pretty upset at you. All right, pause here for more questions. >> Yeah.

Uh, so have I seen uh so no. Um, partly because of course I wouldn't catch it if it was if it looked like regular email usage. Um, because I've set up a designed to catch things that don't look normal. But also, if it looks like regular uh email usage, then uh then you can hand a user can handle that with the delete button.

Yeah. I mean, they wouldn't like it. Um, so yes. Uh, I mean, there's some what do you call a bomb question hidden in there and and that's a judgment call, I guess. Um, so, um, I I've seen things that some people say are bombs that are smaller, and then I've seen things that are that that are that no one's going to have an argument that this is an email bomb. Uh, um, and sort of where where do you draw the line? And and what we decided is sort of how much could you handle this with the delete button? If if it's obviously impossible, we need to help you a lot with it. And if it is

probably possible we want to air on the side of not accidentally deleting email you do want because that also doesn't make people very happy. All right, any other questions? >> Yes.

Maybe it depends. So, so if they have Outlook rules set up, they might not notice it or they might. It depends. Sometimes there are new, right? If if I sign you up for a bunch of new mailing lists, unless those somehow fit the Outlook filters that you you have already set up, the they will show up in your inbox. So, so like it is possible. Um, last I used look filters. They weren't good enough to be like, "Never give me any mailing list content." I just like picked out the ways to filter the mailing list that showed up all the time and sent them off to some other mailbox. But that meant if someone had signed me

up for a thousand new mailing lists, I would have gotten all welcome to my stupid mailing list uh things. So, so like yes, but no is the answer to your question. Any other questions? >> Yes. English languages. >> How does this work in non-English languages? So, you would need to change the way you look for the bad behavior for for the things that are potentially part of the bomb to shunt them out. Um, but the total numbers of things happening works just fine, right? Counting works fundamentally the same no matter what language the email is in. So, so the the the like how to find out when a bomb is happening works exactly the same. the like what to do about it

to remediate would have to be localized to the correct language. Um yeah. Yes.

>> Potentially. Yes. If I find an email bomb, there might be further consequences. So um for example, uh there might be physical security concerns uh trapped in an advocacy bomb. If people are upset at your your company's uh head for some reason, some of those people might lose their minds and make inappropriate threats and you're going to want to look at that. There's also you've just been signed up for 100,000 mail emails and it's probably you need to unsubscribe from 100,000 things. So, so that's a consequence. Um there's also the consequences of whatever this email bomb is cover fire for. So someone falls for the fake IT help desk scam uh or and allows remote access or someone's

credentials are compromised and they don't notice like whatever consequences it's cover fire for is also a potential thing. So yes potentially many but you have to figure out which one applies in this situation. Any other questions? >> Yeah. >> How do you stop?

Oh god, that's awful. Um, that's that is nefarious. So, you will notice that I learned from the past to find out how much email was normal. So, I will eventually learn that that is normal. Um however um it also doesn't fit the kinds of attacks we tend to see here because what is that cover fire for right um so so I have not seen that particular thing doesn't mean it doesn't happen uh and nor nor do I know how to solve it by learning about what's normal because eventually it becomes normal. >> Yeah. Yes.

>> Uh I don't have a good one but uh let me know if you do. Um, so, so I I I agreed to take responsibility for finding email bombs. I didn't agree agree to take responsibility for all of the long-term consequences. Um, uh, but yeah, I agree that that that if if there's I mean, some if the mailing list is on the up and up, right, there's great unsubscribed tooling. Of course, I don't see this being the mailing lists that are on the up and up. And so I don't I you know sometimes yes, sometimes no, sometimes that's the Outlook rule that got mentioned, right? >> Yes, it is very hard when it's in a

language you can't read. I I agree for sure. All right. Any other? No. All right. Well, then thanks very much. Go get in the front of the line for Chick-fil-A.