So You Are Comparing RASP and WAF?

well thanks everyone for sticking around i know it's been a long day uh this is my first impressive con in over two years so i'm pretty excited to be here right so first we gotta thank the sponsors you know obviously without them a lot of us wouldn't have jobs but we also probably wouldn't have much of a conference without them so thanks to all of them who are involved so i want to talk today about rasp and laugh and i'll get into the differences and so on and so forth but first i'm david lindner i'm a cso of contract security i spent most of my career in application security mostly consulting focused primarily in the mobile space

but did a lot of basic abstinence consulting for years and years so i want to take us on a little journey talking about our applications and our software so some not so breaking news our apps are vulnerable we have lots of them out there we have a lot of software and they're all under attack that should be uh not a surprise to any of us and i've got some stats about some of that so i want to talk about apps real quick uh application code makeup you know this is a thing that's been talked about for years i know we talk about third-party code and all this stuff but when it comes down to an application

about 20 of it is custom code that's the code our developers are writing and not pulling from somewhere else six percent of it is active open source library classes that means we're actually using that code from the libraries we're pulling in 49 is inactive open source libraries so we're pulling a bunch of in and none of it's getting used and then 25 of it is active libraries and classes that aren't even touched um you know so it's it's interesting so if you really break it down 78 of the code in our applications is active custom code you know a lot of times we're like oh well only twenty percent of this custom vocal all that's being used for the most

part you know it's the other stuff we're pulling in so you know talking about applications over time you know we've gone through this huge wave of application security and standards and movements and you know this really started way back in 1970. you know when obasa came along in the early 2000s we started doing some laughs dynamic scanning static scanning you name it but if you look at the numbers the average vulnerabilities of an app 21 years ago was about 26. today is 21. so if you think about that it's about 1 or so so it's another 100 years before we get to zero we're not making great strides and why that's important we'll get to that in a

minute so we're under attack uh the latest uh data we have is about 10 000 attacks per application every single month that these apps are seeing attacks come at them but the weird thing is only about 99.8 of those aren't even like hitting vulnerabilities and break that down that's still a lot say there's four million attacks that's about 800 or 8 000 sorry 8 000 actual exploited vulnerabilities you know so this breaks it down my vulnerability type you know you can have access to the slide deck if you want to look at it um and then there's still some likelihoods that you know based on the number of attacks we see versus if your application is actually

vulnerable to it it kind of changes the picture a little bit so you know you look at this here and you can see that the likelihood of your application having a command injection vulnerability is point three percent that's the likelihood of even having a vulnerability but all of the attacks 51 likelihood that you're going to be attacked for killing injection even though you probably are vulnerable so we have all this weird data we haven't really known what to do with it um at the same time we have this thing that i like to call a software security crisis you know we have you know specialized security staff we have very few of them uh we have our

specialized tools and there's thousands of them out there um but we're still writing more and more code it's faster faster and faster i think the last stat i saw on average the professional developer writes about 10 000 lines of brand new code every single year and there's like 30 million professional developers will start adding all that up we're becoming uh we're getting to a problem now the latest results from bsim so build security and maturity model does a lot of good work for a lot of organizations is that there's only one application security expert for every 159 developers so if you add all that up that was ten thousand times 159 and one person is

supposed to re or look at all that new code every year plus all the old stuff it's not feasible so early on we had a problem you know what what problem were we facing well the problem was we knew network security or at least we thought we did and we firewalled and you know we blocked everything uh but then web apps kind of came along we're like oh well we gotta open these big holes in our firewall uh for course 80 and 443 and it was a big thing there was a lot of infighting i remember this this was a long time ago that's aging me a bit um and it allows all traffic to those applications

right so it's just wide open holes uh in our firewall and so web application security was kind of boring um and we weren't sure what to do with it so what did we do well we knew firewalls so we built one for web we called it a web application firewall or what anyone here use them a couple most of us do and we might have you know aws and azure and all that kind of have some of that built in so we built this web application firewall with the same sort of reasoning that we have other firewalls way back in 2002 really was kind of the first lab mod security was born and we started using that to help

protect those wide open gaps in our regular network firewalls and we'll talk about that so what is a web application firewall what's that layer seven it's used to stop scanning potential attacks against that application layer okay um you can tune it but it requires a lot of tuning a lot of patching uh there's a lot of noise involved with web application firewalls um you know a lot of alerts things that we tend to ignore does anyone use their waft data for foot hunting or anything like that or they just kind of let it run that's usually the response i have we know we don't even use the data because there's just so much in it

so how does it actually work so real high level say i have an input box on my web page um and it expects some data and i'm going to submit in this case it's a name but i'm going to put something in there contrast or one equals one dash dash which is a very well known i'm just going to see if this is a sql injection endpoint and see what happens well the laugh is going to look at that it's going to run some regular expressions and it's going to score it and most likely block it say hey this is a sql injection attack it shouldn't be allowed similarly the same form maybe i'm going to change

it because that's being blocked i'm going to try command injection attack so i'm going to try to semicolon cap that's the password and see what happens similarly to the last tab contrast four one equals one the wife is going to block it however should it does it ever results in a sql query does it ever result in an actual system command on the other end the waf doesn't know that it has no context it's outside of the application so i have this analogy that is terrible but uh it's about this new thing at the the zoo there's this firewall fence on the monkey cage and so we want to tune this firewall fence to allow just bananas to go

through and block anything else right so easy peasy out of the box it blocks everything except bananas but as you can imagine we want to tune in at some point but how does it determine what's different well there's a lot of different ways it does this right but for the most part if i roll it all up what laugh does it runs a bunch of regular expressions on the data coming in you know in this case i have some here for some javascript some iframe different html entities and such um and then it will score it the scoring mechanisms are kind of all over the place you can go from like one to five or one to seven

and then it may block or made up but it's very very hard and there tends to be a lot of bypasses you know one of the ones that's commonly used is an on event javascript on events a lot of the very common ones are blocked but this changes all the time i haven't looked recently with the last time i looked there were 374 odd events depending on which browser you use so now math has to be tuned to block all the ones that matter and it doesn't really have context and it could result in a lot of false positives so you know if i want to tune it maybe i want to allow some more things because

it's blocking things it should if you imagine uh say it starts querying on the single quote with someone a leery or someone has a very a rich single quote last name and it blocks it they're going to be upset rightfully so but so now we have to tune the laugh to stop doing this because that's how actual name it needs to get through it needs to be put in the database so how do you tune in this is where it really gets difficult this is one rule for tuning your lap i'm not going to read through it because i don't even understand all of it but this is a modern security rule that that really is for command injection uh

to stop certain different command injection attacks so there's applying these rules and then there's the vendors of the laughs that will give you patches and things like that which is basically just these rules that you have to apply to your lab but the problem has always been it's very easy to bypass you know so what if it looks like an apple even though i want to allow apples should i allow you know and so you start getting into these really big problems uh with the laugh and you can see here on the right maybe uh that's all the patches that were applied in a single day by a friend of mine on their lap different cbes that that were released

that vendor said hey you need to apply all these patches so you have all this alert fatigue you have patching fatigue it's going to stop real traffic in a lot of cases but it is very helpful i mean it is really good at xss it is really good at sql injection on the front end but realize there's always going to be bypasses and there's some interesting ones um you know there's different unix commands and things where you can you know add the the question mark or the dollar sign um and it will bypass last uh you know there's there's been some with xss with the condiments i mean that's really in every bug money program

i've been involved in it's always an odd event that someone tries to use to get past a laugh or some internal mechanism to validate that input and then sql injection i've learned a lot about single injection in the recent years there are so many ways uh you know a slash star star slash is the same as a space to sequel and it will actually bypass laughs as well one recent one in the news and jared actually talked about this a couple of talks ago was the con for the most recent confluence cd the original reporters of this cbd needed to change the word eval and the single vote because it was being blocked by the

laugh of vmware where they were trying to attack so all they did is they unicode encoded it uh and instead of eval it's the unicode value of e and val and then the unicode value of single club and we're able to quickly bypass that laugh and then prove their exploit on confluence so while laughs are important we all have to realize that they do have some issues when it comes to bypassing so in comes rats or runtime application self-protection i would say this is still a very new technology you know it's been around for probably six or seven years but it's not fully mainstream yet and there's a lot of reasons for that we'll get into that

so runtime application self-protection is also a layer 7 technology used to stop those attacks on the application it doesn't require tuning for the most part just based on the way it works and i'll get into that a bit and the alerting is very low because the the way it works uh inside the application there's a very low false positive right so the alerts typically are something we need to react to so the three different things uh from a lav are it has it can see into the application it is running in the live application it instruments the code it watches data flow and it is inside that app so it has contents something the lab

does not have you know since it's in the application security can be enabled anywhere your code is running so anywhere that code is it can be enabled throwing a laugh in front of everything doesn't always work well especially if there's microservices involved in service to service interaction you're not going to throw a laugh in between all of those it's just not feasible and intelligence in modern security like we do performance you know we all use some sort of performance monitoring system uh with an agent that runs inside of our applications and gives us data where a bad performing code might be same sort of thing here and it uses instrumentation you know instrumentation has been around forever

in the real world it's used to build very complex things we don't build cars we don't build airplanes we don't build any of those sorts of things without without instrumenting them you need to know how fast you're going can you imagine driving a car with no instrumentation you'd have no clue how fast you're going how fast you should be going uh you don't know what oil changes do because you don't know how many miles you've gone um it's it'd be terrible you know and now it's getting to the point where it's like the car is yelling at you when your tire pressure is left and it's amazing you know i'm not driving down the road and having someone

point out the window like we used to and say hey your tire's low you know i had that before in high school so how does raspberry differ so rasp works on the outside it feels similar but it works differently so let's go back to the original example so i have that same form where i need to submit a name and i submit contrast or one equals one dash dash so what the rasp is going to do it's going to look at that and it's going to key on it oh wow this is something we need to look at this is something we need to watch and see where it ends up so once it traces it through the app and

that ends up in a sql query now we're asked to say oh jeez we have a problem here we need to block it so there's context the application has allowed rats to follow that data into a vulnerable what we call a sink and i'll talk about that in a second and so it says hey we can't allow this to happen it blocks it and doesn't allow that attack to function so if i do something similar with the command injection and i submit the semicolon cat etsy password this is where the big difference is since the end of this the sync is a sql query rasp is not going to do anything with it says no well maybe you want to put that

in the database it's not a big deal it's not actually running a system command so i'm not going to log in i'm not going to do anything with it because it has the context so this is how it can be very low false positive very low alert so let's dig in a little deeper so the way that a rasp works is that it starts at a source so where data comes in which is typically like a get parameter it's coming from an http request could be a header could be a cookie could be parameters you know it could be it could be any part of that request so there's a source you know so in this

case i've said okay there's a get parameter there's a source source value of user you know very very simple so now i've marked i said okay wraps we need to follow this data to sync which is the next piece so there's the source and now i need to know where it's going out like where is this data being used at the end is it committing a sql statement is it system query is it returning that value to the user allow xss and then once i understand that hey it's come in as a source it's tainted it's untrusted data and now it's going out in this sync that is running the sql query and there's nothing in between to say hey i've

parameterized it i've validated it or any of that i can officially say that there's a sql injection issue and move on from there but rasp digs in a little bit deeper so let's say we have this example here i have a username of test at example.com and or one equals one and all of that and some password the prototypical sql injection attack from years and years ago that used to work on about every website so the way that rasp looks at this uh and the way it works is it basically says okay i have this input there's some things in there that are interesting so you know or one equals one maybe maybe the semicolon dash dash so it does

a little bit of regular expression to say this is interesting i'm going to follow it i'm going to follow it to where it ends up so in this case i mark it i'm going to follow it and it ends up in sql curve so now what i can do to be extremely accurate is i can tokenize what that sql query is supposed to be you say well this sql query is supposed to have from a user id and then nothing after that there should be no more uh tokens or operations or anything after that point i should have a string and that's it so if i see something else outside of that so i've broken some

boundary the rasp would be 100 accurate and say oh geez we have a problem here and i'm going to block it so this happens even before the query is made to the back end you know so in this case here you can see that it broke it down the rasp is going to break that sql query down to all the different components of that query and realize because it knows what the query should be it already knows the different pieces and parts of that query should be and if there's more there it's going to log as simple as that so it takes that untrusted user input that sync is called of execute query we analyze that query we tokenize it and

then some alert is sent somewhere say hey we blocked this this is bad so remember how you had to tune away well tuning arrest looks like this it's either on or it's on for the most part there are cases where you can add validators and sanitizers but we're getting way into the weeds when we start talking about that but for the most part you turn it on and you turn it on now i don't want to sit here and say it's rainbows and sunshine because you know the reality is rasp is still pretty new it still has some issues it's very language and framework specific you can imagine you know i had to know that that execute

query is something i need a key on and that's very specific to java and whatever framework was being used in that case but you can imagine if i'm using spring if i'm using all these other languages and frameworks rasp has to be updated for that because it's running in the actual application you know any sort of bypass because there's no perfect security tool does usually result in a code change which is a little bit harder than applying a patch to a lap it is a little more difficult to install depending on how your system works and oftentimes which for some odd reason i've been in security long enough and i feel it when we don't see anything in security

we question don't we like if we get no results on something we're like oh geez this must not be functioning but oftentimes harass isn't going to see anything it's only going to see something but it's an actual attack that would have hit an actual vulnerable sink so a lot of times it's pretty silent which is hard for us to accept so what does it look like so let's go back to confluence so this is really it you know that confluence cd that was used uh the the attackers had to bypass the laugh with the report where they reported it to uh was just picked up by our ass easy peasy you know we had a rasp

running on confluence i said hey this is an expression language injection attack uh we're going to block it no configuration needed none of that so even though bypass the left the rats perhaps already a standard whether most of you know it or not it is 853. it's also in the most recent pci so you know it's interesting so that's some background on those two so let's compare that okay so my research team decided to kind of look at laugh and grasp and compare them a little bit and this is a high level is what we're looking at right you have your payloads you have your laugh and you have your dummy application you have your payloads and then you have your

dummy output running into it and you can quickly see where context is lost with the lab and you have full context when you're in the mass but here's the problem we were working with a bunch of companies uh for doing this and one of them returned this they said okay what we did is we stood up this application we ran some scans against a laugh against a rasp and here's the block what does this tell me does it tell you you might not be able to see that i can tell you what it tells me nothing this is just it's an odd thing there there are two technologies that we've made similar but they're different

um and results like block count are meaningless for rasp because if it's not a vulnerable sink you're not going to get a block health so if you're expecting a block count you're really expecting the rasp to behave like a laugh which it won't because it's not made to and we're going to run into problems you know so the results are all over the place like i first looked at this i'm like oh interesting the wraps pop more than the laugh but why and then i start questioning like what laugh did you use were they vulnerable what were the the scans like you know so testing methods are inconsistent um and the results like this don't

really mean a lot because it's two completely different technologies it'd be like an antivirus and an xdr i mean they're really really different technologies so we took it a little bit further we're like all right well what are these people using to scan their applications um and compare the two so we looked at you know xss radar xss scanner all sorts of different things sql map and we started just blasting through all this stuff and we realized that all of these tools are all these payload testing tools are pretty poor uh xss there's 20 percent of the xss attacks are either not valid or duplicates 27 percent on sql injection 48 command detection the patchwork was

even worse 82 of the past reversal attacks were not validated so we're using on data we're testing two different technologies in the same way expecting the same results so we're causing this really weird mentality that we shouldn't have so to me it's having different data and having that visibility through the data that's important so this is what most of us have today so the three or four hands that went up with a lab you probably have a laugh today and it's maybe throwing that data into a sim didn't seem like anyone's actually using it for anything uh because maybe we trust that it's doing its job and we're applying patches and whatever else but again we're still not sure how any

of these things would have impacted that running application now that being said there are some things that left as well uh you know being where it is it can really prevent some ddos attacks it can block some of the ip addresses and stuff much more quickly and further out than they need a wraps again and then if you look at the the actual difference here with rasp data you can throw it into a sim and then you can tie it back directly to the security model buildings uh you know so in the case of confluence you know we could have taken that data from the rasp handed it to the developers that hey we have a problem here we should probably

patch this fix this expression language issue that we have so it's just a different place to get that data out of the system so i propose we use both there are two completely different things at two different places in the network uh and as a security person we're all about control layers right you know and having different pieces of the onion and to me these are two different control layers you have the laugh on the external it's really good at certain things and you have the rest running in the application with contest and the reason the big reason why is because hackers are exploiting the land and what i mean by that is how quickly

can us as laugh providers that you know us as people who are maintaining our labs respond in a case like uh equifax which is is this here there was a cbe for apache struts with a rasp you don't really have to do anything it's already there protecting you against those sorts of attacks with a laugh i don't have to patch it and the attackers are going to exploit that in the case of equifax the lag was huge right march 7th there was a cbd released and then not till mid-may they were breached they still hadn't upgraded or fixed anything and then they finally learned of the breach at the end of july so there's a huge lag there but they

never would have had a problem if they would have been using both a laugh and rounds you know if just a laugh wouldn't have blocked these attacks same thing in the case of confluence if i was running confluence on premise which would be done but there are a lot of people that are doing it specifically government i would look at running a waf and a wrasse in that cots product because i don't have the ability to make changes to that code and i don't know exactly where the issues are and there are just some things like expression language injection that a waff is not very good at why an erasmus can so the way i look at it is this i want

to prevent and protect you know when i'm driving my car when we buy cars today they all have to they have an airbag and b has seat belts right it's two different things for two different uses a seat belt is annoying sometimes if you hit the brake a little hard isn't it just like a wax you get all these weird alerts and things like that but that airbag you're usually hitting something when those go off right and they help prevent you from getting much more damage than maybe you would that's the same thing with this you know you have the laugh for that that front gate entry system and you have the rasp to cover you

when that lap fails so to really close this out i wanted to bring up the the strengths and weaknesses of both and why i think they work very well today i know i mentioned them through this whole thing um but the weaknesses are really the alert fatigue false positive false names are always there you know it's interesting i do a lot of bug crowd and hacker one and there's been quite a few times where companies will be like all right see if you can bypass the laugh and they only do it for two days you learn a lie because it gets so many bypasses they run through money um you know patching and tuning is

required almost on a daily basis and there's no actionable results right we're not looking at the data because there's really no no action i can take i can't hand laugh data to a developer and say hey go fix your code because i don't know if it's broken i just know that someone's scanning that app with malicious intent so the rasp strength opposed to that it's very often has no results very little tuning required and the results are actually you can actually hand the results to a developer and ask them to fix the issue at hand or work with your security team be like all right hey we have rasp running we're happy we can push out that remediation six

months or whatever it is however the rast weaknesses are counteracted very strongly with the last strengths rasp does have performance events it's running in the application it's doing it's using instrumentation and the impacts are typically between 0 and 10 that's a lot of performance if you're talking bigger websites bigger applications 10 is usually hard to swallow language and framework specific it's a must you know so you i've seen every language and framework possible and it's very hard to to find a wrap so that works with all of them um and it's not always environment aware right because it's running in the app it doesn't have any context outside of that all i know is that this data came in and

i'm going to follow it through the app but the last strengths of counteracting those is performance is not an issue laughs have been around for 20 plus years they've gotten really performant there's language agnostic rules uh it doesn't matter what the language is because it doesn't it's not context there's no context it's just looking at the data and it's more environment aware so it's able to block ipv addresses it's able to prevent ddos attacks at that level rather than trying to get those sorts of things into the application so that's everything i flew through this because it's the last talk of the day and everyone's probably chopping it a bit but i'm happy to answer any questions i

can get as technical as you want yeah go for it uh

so i'm a rasp purist there have been quite a few like a signal sciences but they were required but they were more of a what they call themselves an ng laugh you know kind of in between uh contrast security where i work i don't want to throw that at you but we have what i would consider a true rap um and then there's i can't remember what there was one other that was also basically required it was more like an ng lab and it but it did run more on the system there's actually a free one if you're okay with downloading uh stuff from chinese folks uh it's very good though it's i think it's called

baidu b-a-i-d-u um they've got a lot of it's on github uh very very good but but again since it's so language and framework specific it only works on the couple you know so uh it's it's definitely an interesting technology i would say it'll probably be another i don't know three to five years before it gets the point where it's a bit more mainstream if that makes sense yeah it's it's it's got a lot of work to go i mean the performance impacts is really the biggest deal since it's running but you know memory and all that stuff is just getting bigger and better by the day too so hopefully we can get a little bit

better so uh the other question was regarding javascript [Music]

how is it doing sync analysis on javascript [Music] that's a great question um if i had my way it wouldn't touch any of it um and the reason is is because in my opinion those are better caught by a laugh than a rasp because it's so difficult to know okay i've got this data coming in it's being encoded and decoded 28 times it's going out in these 17 different places maybe it's being committed to a database so it's more of a front-end technology with wraps where it's doing mostly regular expressions for the xss types of stuff just because it's easier and i mean if if you really dig into even a laugh and a wrap and xss specifically it's

actually really easy to key in on a nexus attack um it's just hard to keep up with the changing html of javascript landscape that's the piece that's the hard part especially yes yes thankfully we're talking to jason most of the time so now it's just client-side stuff yeah what was the pci ess reference the requirement i think everything that goes like nine more uh it's i think just three one which is the the recent uh

great thanks [Applause]

thanks again for attending um got about 29 minutes