Don't Take the Bait - Online deception beyond your Inbox

[Music] [Music]

[Music]

So, good afternoon everyone. So I have a little story for all of you and but first of all I'm I'm going to have to start with my introduction. So hello my name is Andres uh and I work in info security function at a tech company. Now I'm sure many of you know that uh working for infosc is much like being a plumber for an apartment complex. Yes, we are not picking the apartment because of the plumbing. you just sort of expect it to be there. However, as it is, um, someone has to plan for it, someone has to build it, and when you have an incident and cropping crap on your kitchen floor, well, someone has

to clean it up. But you know what? It's quite difficult to do after the fact. However, um, I'm I'm getting sidetracked here. Um, what I'm going to talk about is fishing. you know, uh, nice fishing simulation emails from providers such as no before, fish bite, uh, hawk hunt, like some kind of suspicious URL, uh, weird looking mail domain, and well, you you know the usual indicators, I'm sure. Uh, however, it doesn't have to be email. It's just social engineering. However, you might be selling that, hey, unless you work for Bolt. Bolt is not exactly known for being an expert or authority on fishing. Well, Bolt is more known for something like, I don't know, scooter art and

weekly posts complaining about us on our SD subreddit. Uh however uh we do quite a bit more than scooters dangling on top of bus stations. So uh we have five core product lines. So we have taxis so you can drive a taxi or ride one. We have uh of course scooters. You're supposed to write them, not put them on top of things. Uh then we have car rentals and we also have things like delivery. So you could get warm pizza in your office or home. And we also have bulk market. So you could also get your groceries without going to the grocery store. Well, since it's Estonia, anyone here who hasn't used any of these products?

Well, at least one. Great. So I I have a quick summary of uh what it does. And if you have any friends who don't know what Balt is or what it does, well, you you will you can tell them this. Basically, you can order stuff or transport without actually having to speak to people. So, no surprise, the Bolt is built in Estonia. However, if you really have to, then you can sort of chat with uh the other party of the transaction. So you can chat with the courier that hey I'm in this apartment or I I will be at the door. You can chat with the taxi driver that hey I I'll be waiting at the junction.

You can also do VIP calls uh through the app. And you can also do phone calls. So we do uh some sort of GSM proxing as well. So you don't disclose your phone number to the other side in most countries because uh well 4G is expensive especially in Africa. Uh so how many of you have used these functionalities? Have you used the chat? Have you called the driver? Raise your hand now. Oh quite a sizable but not the majority right. Well, let's just say I know a guy who knows a guy who knows another guy. This guy and this guy has a clever idea. Very bright one. So, and well, from the topic you might

already guess what it is. It's to do some fishing. Well, actually some more fishing but unfortunately I cannot disclose to you the exact numbers. Uh but I can convey it via some other way. Since I joined in 2023, uh I believe there have been zero days without any fishing attempts on our platform. Like yep, no days without fishing. So uh but we will go in chronological order uh and I'm going to start from the beginning uh well how this uh fishing related uh research started for me. So I joined Balt in 2023 as a security engineer and well in the beginning well I I would say I had a good day right I was dealing with all the usual appst you

know configuring SAS doing task vulnerability management looking at the wheel panty submissions stuff like that until suddenly I being on Slack okay what could it possibly be a new wave of fishing in the Netherlands Can we please mitigate it somehow in the chat? And well the [ __ ] well team assemble. So we have to investigate right? Except well there is no team. There's just me and well some of the tools I could use to find out what's going on. So no specific security logs on that. I I had just had to go with whatever we had for the analytics business data and then ask around uh how everything works. However, I before we go into the investigation, I

will have to do a quick primer of how the technology that's relevant to this presentation actually works at Bolt. So, uh most likely you know about the rider app and the Bolt food app because most of you have used this. Uh but there are some other things we use on the back end side of things to actually do all the logistics and well make sure the service works. So most important one of them is B driver. So this is what taxi drivers use to actually drive around and offer their services. Then we have the support portal uh which we use to basically look at the customer complaints and try to resolve them. And there is also admin

portal which we use for making changes that are not available in application or when well there are some other issues. Um and now we also have different authentication solutions. So we do not have a unified authentication system but for the context of this presentations you will have to know how rider app authentication works. So there is the primary authentication method which is fordpo sent or SMS uh GSM basically voice phone call or WhatsApp. This is the primary login method. If you use the rider application or bolt food, this is there. This uh is always going to be there. Uh well, that's been uh there are some alternatives as well which you are free to enable but you cannot turn out off

the primary one. So you can also do OT via Meta, Google or Apple uh authentication provider. In addition, we also have driver app which has slightly different authentication logic. So as a primary one, they have password out single factor. Uh then there is email magic link login which is well the kind of flow you would usually use to reset your well email uh or some social media uh password. So magic link sent to your email go there and you are well authenticated and you can change your password but in this case we use it for login not changing password or not only changing password and uh there is also sixdigit OTP code sent via SMS so the

same as rider but it's slightly longer uh however uh there are some cool express driver accounts for example uh there is conditional UFA so if the login location is really suspicious known or exit node or country mismatch for registration between registration and actual sign in. Well, then you still have to do the either SMS code if you logged in via password or if you logged in via SMS code, then you will have to enter your password as well. And since we really don't want anyone uh to drive a taxi without having a valid driver's license which they show on registration, then we also do selfie verification checks on devices on some conditions. Anyhow, uh let's get back to our

investigation. So, uh plenty of fish in the sea. When I started looking at the chat logs, I saw about three different types of fishing lures that were quite common, almost used in a copy pasted format in a multitude of languages. So, uh, exhibit number one, hello, this is B support. The location of the client is hidden because he is an officer in the army. So, this is what you would see in the like rider or driver app in the chat. So, in this case, the target is driver. uh the driver would see that okay customer wrote to him that I'm P support and well please confirm your details another one again this is B support but

now they are dangling a reward uh so if you enter your details because this is a VIP exclusive customer El President whatever you will get $150 bonus for the ride and uh last but not least well again this this B support uh complaint has been filed against you. You are lying about your identity. Please prove it to us. Ah, nobody expects the B support. I wouldn't expect it there. However, they still had a relatively decent degree of success. Uh let me show you uh what the fraudster accounts look like. So, basically uh the rider accounts usually had some sort of uh English sounding first name. doesn't matter boy or girl Jack Chver uh Sophia Jean doesn't matter they had completely

nonsensical email address uh so something with very high entropy most likely the Gmail address in this case doesn't didn't even exist because validating it is not mandatory phone number from some random website that allows you to send or receive SMS messages to a random number and the devices. Well, here you can see only see device UID but uh mostly Samsung devices. Okay, but what do they do? Well, first they want to get the GRS for this. Please confirm your phone number. The lure you saw previously and now they ask for phone number to verify you. Well, you give them the phone number. They open the driver app on another device. Enter the phone number. Please confirm the security code because

you know your identity is in danger or this is a VIP. We really have to be sure it's you. Well, you get the code. Yay. Hacker wise, you're in. So, but why do they do it? Not really a question. To get money. So they get into the account, they open a ticket and they say, "Hey, I want to change my IP address and uh depending on how good the support agent is, they might succeed." Uh, however, this is uh the simplest of the fraudulent flows. Uh, another interesting one which we couldn't really expect beforehand was cashing out rubber. So let me explain this. Uh basically what they did is they wish for access to driver account. They

get access and after getting access they will use that access spoof their GPS location and start accepting rides in some well high paying region I don't know Paris France somewhere accept rider write to them hey this is P support you have to confirm your identity however they are fishing for uh Uber credentials of the riders so they're actually cashing out through Uber Because well let's face it we are not sharing our logs with Uber and they are not sharing their logs with us. So most likely vice versa is also going on to some extent. So now we understood that hey this is definitely an issue. So how should we mitigate it? We had uh initially about three ideas of how to

sort of quick fix it. Option number A. So let's uh run some reg access on chat messages and uh sort of do per user blacklist for participants on uh everything that looks like an OTP go second one. So this one was the one that actually was requested. So let's try to mask chat for keyword subs for rest. So I don't know let's mask everything that has bolt in it. So you cannot really talk about paltic bolt anymore right? uh or we since we know the user's credentials like phone number and we know the email address as well we can also mask these and option C. Uh let's just not send any new OTP codes while there is an active order ongoing.

Well, we decided to go with options C first and option B right after. So, uh option C, active order check. We no longer send out OTP codes to neither riders or drivers while there is an active ride going on. Uh and uh it sort of works at least initially as well because frosters were expecting a code that wasn't sent and well victims were asked for something that they didn't really have. So everyone was confused. Frosters didn't know if the victim was just messing with them and didn't believe them and victim didn't really understand what the heck was going on anyway. And now the second option is masking information. So, hey, please confirm the phone number.

Okay. Well, uh the victim saw that they shared the phone number as was asked, but what the foster saw was just well masked as the riskers. Okay. Well, and again, this also confused everyone. So, at this point, we were like, "Yeah, it works. I can finally get back to the good stuff." Doing the cool cyber security stuff. I don't know, post quantum crypto or something. Mission complete. Well, yes, but actually no. There is another season. So, here we go again. At this point, I was not even having a good day anymore. It was like boom, two messages. Oh no, not again. Okay, now what uh the support was seeing was that the frosters were moving the

scam of platform. So they were basically funneling people from the inapp chat to contact them via WhatsApp still using more or less the same pretext that hey I'm B support. Another thing they're doing is that they were sort of getting quite creative at bypassing the chat filtering we had implemented. So at this point I'm like ah it has been 200 years since Spanish prisoner fraud and somehow still we are dealing with social engineering right however it's evolution not revolution so let's dissect these vectors. So first of all, still same lure problem has been identified blah blah blah blah blah blah. However, please contact B via WhatsApp because of course we use WhatsApp for corporal communications.

Why wouldn't we? Well, and we don't really know what's going on. We don't know if they fell victim unless we see some other signs after the fact. And we really in most cases don't know what's going on on WhatsApp or Telegram. I mean they diversify it as well. It's not only WhatsApp, it's depending on what is popular at the region. Now the second bit, bypassing the chat filtering. Well, again, same lure more or less. However, aha, please put dashes between the numbers. And it became sort of an arms race between us writing better regaxes and them basically trying to bypass them and going as far as, hey, can you please like write high code that rhymes with

your phone number? the crazy creative, right? And well, you can also send email with that app symbol or if you're using Gmail, be put everything before that symbol and stuff like that. But at least now it was getting interesting. So we had an arms race going on. So uh but you remember this bad boy, right? We're no longer sending out the pig out. So everything is Gucci, right? However, if they go with the WhatsApp route, they can just sort of wait and cancel the order and then they can still get OTP out. They just have to do it through WhatsApp. However, this is not the only option they had. They could also do something funny

and I see confused faces. So, what is something funny unless? Well, uh there is email recovery flow. So if you get email address of the victim, you can sort of try to recover the account by sending an OTP code to their recovery phone number, then fish for the recovery OTP code and login via email. That way if you try to target driver accounts you have persistence through the use of email magic link login. And of course for us this is a pickle because we have no way to verify whether the legitimate user has access to the email account or is it something someone else or if we see an incident we are sure there is

illegitimate access. We don't know when has it ended and we have no authority over well security of their email account and it works with Google as well and they are basically doing the same flow recovery through phone number OTP blah blah blah or email OTP and they are then bypassing Google's uh QFA as well then basically hey we are now buying security from Google as B and can you please press the button 52 on your Samsung Galaxy blah blah blah phone. Boom. Pain. So, um, we did what everyone tells you to do in case of fishing, right? We are now adding service message to every chat that hey, you know, never share your personal information and this is

definitely not Bolt speaking with you. And we also rolled out like specific uh training for our drivers that hey you know there is this sort of scam going on and you should be aware of this and we will never contact you this way and instead of well just report this right and it I mean it sort of worked. Some people are really uh fine with training and actually follow the instructions but not everyone. And well you just want to drive taxi and earn money. you don't care about security all that much. However, meanwhile on Flosserland, they discovered something interesting with uh the rider accounts. So you can or you could sort of lock victims out of their

rider accounts. And this was also quite funny flow. So you do the usual fishing, you get access to the rider accounts. What you do then is you change the phone number connected to the rider account. You remember this was the primary means of authentication, right? You also well just to not send out any notifications on what you do you also change the email address and now the moment you change the phone number uh you confirm the OTP on the new phone number but no confirmation on the old one is necessary. Uh it basically means that the original owner is locked out. uh you are locked in and when the original owner tries to log back into their account, they can

actually get a new P account and then try to explain to our support that hey, trust me, bro, I used to have another account. Um, but it's it's no longer my first day. It's uh every day. So, season 3, the finale. So, this is well the final stage of us trying to eat this soup. Oh, oh no. Quite a lot of Slack messages, right? But they are not all that bad. So by this time we had already told everyone within the company that this is an issue, this is how it works, blah blah blah blah blah blah. So they are actually putting in some effort into planning uh accommodating resources and then coming back to our team that say

hey does this work which is great or hey we are doing a workshop on this security issues. do you want to lend us a hand and like be be the straight man in the comedy and uh also can you please talk a bit more to our support agent what is going on how to figure it out and stuff like that so actually situation is looking pretty great so but let's let's do a quick recap of where we are so mitigations uh OTP goats are no longer center during orders which is good uh we have some chat filtering which adds some friction to the fraud and flows. Uh and we have done awareness raising and guidance to

try to help people that are willing to help themselves. Uh however, we still have some remaining problems. So, there are still filtering bypasses. It really doesn't make sense to do the reg xarms race. Uh the account takeover via magic link login is still an issue and unauthorized changes are still an issue. And well to be fair you only know about the fishing incidents that you are able to detect that or that are reported. So we cannot very well size up the issue then explain okay we are losing like this many millions or this many users or like we got tens of thousands or hundreds of thousands of cases. We we don't know the actual number. We have some insights but no

hard data. So where to mitigate at this point? We could improve detection. So accurate labels. So try to get better support tickets. Try to figure out how to detect the chat uh message does have illent. We could also do more security notifications. So at least user would be able to let us know if something weird is going on. So I didn't log in from North Korea. I promise. Uh then prevention. Uh we could improve authentication. We definitely should, right? We could uh do some actions on chat already once we detect bad intent. And we could be reducing the impact. So implement some guard grails for sensitive changes, but also make sure our support procedures are actually

good. Which one to pick? So we decided to do all of these uh basically detection. These are the very nice notifications you are now receiving on new signins to your bolt account from new device and also on any changes to authentication details. This is great because it is sent a new login, it is sent on changes and uh we actually track it so we know if the notification is sent you open a ticket then there is most likely this issue and it's related and we know which login it is which is very nice and we are also trying to detect uh bad messages but this needs its own slide. So labeling chats basically uh the idea

behind the solution was that let's make potentially infinite number of bad messages into finite number of labels because this gives us aha trigger point for response. So we can already react better bad message or bad chat right this uh gives us sizing of the problem because if we can detect it better then we will have better data on the extent of the issues and since the only way to do it is with the help of AI we will get cool by points for doing it that way right woohoo future so of course we know that AI makes mistakes but 100% accuracy is not uh required if we're not banning outright banning people B detection, right? So basically this is how it would

work. You can write out in chat chip it is or copy paste a fishing message and ask it uh whether it is fishing or not. Like you basically can get an enum of hey is this fishing? Is this harassment? Is this fraud? Is this sexual harassment? So you you could get the indicators for it and it's actually quite uh good. It it has decent accuracy in our experience. So uh but we also went around to improving out. So we added support for pass keys which cool modern technology it's known to be fishing resistant. However, it's only fishing resistant if you allow to disable uh fallbacks. So the browser is never going to go for your pass keys if it can still go for

your password or OTP code, right? So we also had an optional dufa which is great because now you can have duefa however you are not forced to have one. So it's a definite improvement however it's not for everyone because you have to explicitly enable it. We cannot just roll it out because there would be mayhem with availability. Uh then we also do new device checks. Uh this is for the driver side of things. Uh and uh basically well also 3DS for riders which is the thing that credit card companies use. So you have to authorize your credit card through your bank which in Estonia is quite a nice procedure. Login through smart TV blah blah blah blah blah blah. Quite nice. So

on new device login you can no longer do credit card fraud that easily. And for drivers well uh we now have selfie checks on new device login. You can't start working until you have done selfie check with some kind of liveless test as well which I think we have built in at the moment. So and to reducing impact guard rails for sensitive changes uh which is nice. Uh so basically there is now a 48 hour grace period on new login before you can actually change the email address to phone number for either account and uh yeah profile changes are not allowed during this period. So you will get notification and you have 48 hours to respond and let us know that

hey this is there is something wrong going on. And we also went around auditing support procedures and well we we found quite a bit of uh things to improve. So root causes of tickets were not always resolved because well uh the agent had low visibility into issues. The admin view didn't really show the everything that engineering could see and uh well almost no appropriate actions available to agent. This is fine. No, no, really not. So instead we decided to sprinkle on some security. So uh we enrolled all of our agent into bespoke security training. Hey, these are the security issues our customers have and this is how to resolve them and this is now mandatory part of their on

boarding. Uh then we ask for some updates from the engineering side of things to give more actionability and visibility into for our support agents. So basically user actions especially for riders are now trackable to successful authentication. So we show some kind of like opaque session ID. So you can break this down. Uh and then we also added some actionable constraints. So for example, if you think that driver's email address is compromised, you just disable magic link login for them. It's it's a band-aid, but hey, it it stops the persistence at least. And we did some process improvements as well. So basically how and when to escalate to privacy and information security because you know all the unauthorized accesses

they actually show personal information to the fraudster which is legally a personal data breach which we have to report to data protection authority. Yay in every jurisdiction that it happens which is another headache right. And then we also well sort of trained our agents on how to identify hijacking. So if you have a newly created rider account and you think they complain about their account being hijacked then maybe check for historical accounts that had the same phone number. So you could like sort of spot it or at least escalate if you can't know it yourself, right? And we gave them a bit more freedom in trying to kick out bad actors. So the if the complaint is about

unauthorized access then you revoke all the tokens and original user can log back in. So at least there there is a bit more uh speediness to the response. So epilogue are winning son well I'm an honest guy I don't know for sure but I think we are not actively losing and that's pretty good. So the current state of things is AI detections are enabled for jet which is a great thing. It's quite effective. Uh we are quite quick to actually improve it. So it's no longer regress because AI is slightly quicker. Then we have integrated these detections with our antifra system. So we can actually unmatch uh bad rider or bad driver from the order. Uh and we can

also like if you see them doing it 10 times then maybe the human will click on the banhammer still. Then we have notifications which have very strong signal and very low amount of noise because we give some actionable information. So if you didn't log in from North Korea then you can tell us it's good and we somehow uh improved the feedback loop between support and engineering functions. So well uh support is now more up to date with what engineering is doing. So they are no longer occasionally recommending riders to change their password. There is no password but uh we we still have issues right so we have seen creative attempts at bypassing filtering even the AI one. So

they are still doing it it doesn't stop but the volume is lower. uh they're trying to use pass keys for persistence and we still have low visibility over what is going on on Telegram, WhatsApp and GSM and they have been pivoting towards using inapp voice calls for uh the fishing which we cannot run the detection on as easily because you have to transcribe it first, right? But uh I I will do something that maybe I should not be allowed to do. So I'm trying to predict the future. So fishing resistant authentication. Well, pass keys and and the like, right? UBS are quite secure. However, they're quite inconvenient. and pass keys are synced to your Google, Apple or well

password manager account which means that once we have enrolled enough users into pass keys or well enough websites and apps use pass keys for authentication the fers are going to be targeting your social accounts to get into the apps. So, however, this is a shared problem. So, smart boys at uh Google and Apple are going to have to figure about what to do with it. Now, uh in Estonia, we also have local alternatives like Smart ID, which is awfully fishable. It's not fishing resistant at all. Uh only thing that is saving us is that well, it's not scalable. You don't want to focus your criminal activities only on the three Baltic countries. still done but the

scale could be much uh higher if it was used worldwide and well ID card is inconvenient can you imagine like trying to convince on Molly that hey where's the ID card reader where's the ID card can you plug it in do you have ID card software have you updated the browser plugins well so and of course for larger companies there is no scale so there is no incentive to actually invest into this kind of authentication because this is a very small proportion of the user base. But what we all learn from this is well decision- making security people really like to decide and act. We don't put much effort into observing and orienting on our context and what has happened

before and actually explaining to other people the stakeholders the engineers the support agents the management that hey this is what is going on this is why it's going on this is what we should do. We like to block block block but it doesn't really help always. So the key takeaways are you need a feedback loop. So you might think you have solved the issue but you have to actually test it as well. It's not solved until you have tested that it works. Uh security is more than technology. Uh it's also processes. It's people and security is not intuitive to understand you as a specialist in security domain. You have to explain it to every single people

you're working with or interacting with this how it works because you might be able to grasp it but not everyone is. It's hard to understand what is going on. Understanding robbery is easy. Understanding fishing where people have been locked out is hard. And as always, prefer simple solutions first. There is no point in doing fancy fancy AI stuff if you haven't even changed admin password. And if you didn't like these takeaway points, then at least enable UFA on your B account before the end of day. And if you want to talk to me about this stuff, well, these are the bullet points I'm open to discussing after this talk. And if you liked what I talked

about, then we are actually hiring. >> Thank you. >> And if there are questions, please raise your hand. We'll pass the microphone.

Uh I talk. Thank you. Uh uh I have a question about uh this AI based uh like detection that have you uh noticed any real life uh prompt injection attacks uh to bypass this uh AI filtering? >> Uh not for this particular one but I think there have been some attempts uh for trying to bypass the AI support agent. So far we haven't detected any successful ones but maybe they are really good at it. You never know, right? >> Thank you.

Sorry sorry. >> Uh, how popular is the magic link login? I mean, if it basically means the attacker has persistence, wouldn't it just be easier to disable it alto together? >> Yeah, it's quite popular because you don't have to remember the password for it, right? you don't have to get SMS OTP code and SME doesn't actually guarantee delivery in any case. You just see that you sent the SMS but you don't know if it actually reached its intended recipient. Uh so magic link login is quite popular because you're always logged into your I don't know Gmail account on your phone so you can always use it quite reliably. So it's used in I think at least 20 to

40% of the driver login actually. But yeah, I I would personally prefer to disable it, but it's not as easy to scale out, right? >> Hello. Uh, thank you for the talk. Uh, question. Uh, fishing is a common problem across different industries. Are you competing or cooperating with other right sharing companies on solving this? >> Uh, currently I think we are doing uh neither successfully. We are doing some research uh cooperation with uh one of the universities outside of Estonia. Uh and we are well thinking about who to collaborate with because in the end I think we would be willing to do it but it's not straightforward to do without disclosing like business metrics to

competitors. So we haven't figured it out yet.

Okay, if we don't have any more questions, let's give Andreas a nice big applause. Thank you for the invitation. Thank you.