Introducing ModSecurity 3.1

right now uh well but uh let's let's talk a little bit about mod security uh just to give a little bit more details about myself i was uh i'm the lead dev for mod security for the past eight years outer for the live mod security uh i also do some research on the on on blockchain uh that's my basically my my phd uh i always try to work with multiple things at the same time so in the phd i've studied is a blockchain thing i i used to make a lot of contributions on gs gsm network insecurities but basically nowadays i'm focused on mod security and mitigate assorted kinds of webby attacks and web vulnerabilities

together with me and mod security there's a ton of people who work in a project as an open source project we receive contributions all around the world so here is the avatar of some people that make contributions in in the project uh the the guy with the beer is victor victor now lives in canada he is basically the common thing with mod security together with me i work for trustwave so mode security is a project sponsored by trustwave uh martin is also in canada he is the big picture brown picture here he helped me a lot on the development of mod security together with all those other guys from the community that also makes contributions and pets and i think

that's the main reason why mod security is the good software there is today is because i can count on the community we can counter the community to make it it's great and and well add all those features uh today i would like to talk to you a little bit more about what what the good is about version 3.1 and but before that let me give a little brief introduction about this versioning mod security and different versions and different uh families of mod security back when mod security was created more than 10 years ago 15 years likely it was only a model for apache so basically apache was the most popular website ever there and the the only one that was really

available in our distributions so what security was built to actually target apache uh later nginx became popular and other web servers like iis became popular as well so we decided to expand mod securities to support those web servers and we did that doing a little bit of a trick a hack right it's very hackish way creating a wrapper for apache and embedding apache on iis or embedding the passion in chinax and that was basically the family 2.x for mod security we had support for is we had support for nginx but mainly the code was built for apache so it was not so performance not so stable on giant x or ais uh on december 2017

we released the first version of mod security 3.0 and mod security was a game change because it was a completely rewrite of version two and with this completely rewrite we make mod security a library and this library adds a lot of possibilities like put what security work together with different web servers and yet uh make it stable and performant right so with the library create the concept of connectors so we have different connectors that put mod security to work together uh with different web servers we have now connected for apache this is still an alpha phase the connector for 3.0 uh the nginx version is well tested it's been on the market for almost five years

now and people are using in uh they are happy so far and we have connectors for tcp dump we have connected for different kind of networking processor as long as you want to inspect uh http requests you can use mod security mode security and make your own connector it's very easy i'm going to show you how you can expand mode security in different script language and see whatever you feel comfortable with to code there is this possibility and now we are about to release version 3.1.0 it has a lot of new features and the focus is on usability and how to make easy uh for writing rules for mod security um what i what i mean by that is like

mod security is just the waff web application phyro engine so it per se mod security is not capable to actually mitigate and threat it it has to have some rules and those rules are the ones who actually will use mode security engine to compute things and classify a request as as something bad or something okay uh so the rules have an important hole here as well together with the engine basically all focus here on the engine and the re one of the reasons that mod security is so popular is the fact that it's very flexible if you ever try to use a commercial app you have to have a black box installed in your network or you have to update

the dns of your web server to point your external server or do some stuff that's not so welcome let's say and sometimes it's kind of disruptive in the original architecture that you have for the network that's different from format security because you can deploy mod security as a proxy you can deploy mod security embedded in your web server you can you can also deploy mod security inside your application using script language if you will and it has tons of capabilities like it has native blue support uh there is something called transformations inside mod security that you can use for basically mitigate evasions like double encode or two poi coding or what if i have i don't know maybe utf-18 you know f-16s

that is that any bypass that could be outcoming of those then you can use the transformation to mitigate that kind of evasions uh you can also do some crazy stuff like you can we can by by each request you can execute this final process a python script a pro script and do some processing with that sternum script and give back the results to mod security to uh conclude if a certain request should be blocked disruptively or only logged or well sort of different options that we can that we can do but for 3.1 we do have this improvement of performance possibility and interoperability it's interesting to put that way you can say well what performance has to do with

security and i respect if you think about it the idea is to have the idea is to inspect the mass moon that you can in the shortest stream time frame in a way that we can do more inspections and the ad does not deliver the the results to the user so that we can deliver content to the user inspect the specs as fast as i can do a lot of inspections and everybody is happy in the end of the day because we're doing good inspection so that's why we try to make it as fast as possible that's why it's so important for this security perspective because there is no point to like a navy if you are scanning a file and

take like ages we cannot do that in the web server people wants to have the especially with http 2 latency is a key uh a key thing to deliver the results to the user for the usability so that's very important and the usability per se inward security is like we want to make easy for people to write rules if you ever try to use mod security there is their commercial rule set their open rule set one of the most famous ones the oscar rule set and if you open that there is this rules with a lot of hijacks very complicated to read and very complicated to maintain you know it's a struggle and we want to

have that easy and make it easy for the user to use another key is interoperability uh we want people to use mod security make it as flexible as possible so uh you know if just to give you an example a few days ago we had that issue open on github so i'm the project is on github i will show you my my last slide i will show you guys all the details how can you make contributions the project how can you use etc uh it will be there hold on on that but basically we had this issue on github and the user was asking if there is a way to actually pass the rules using golang and that's the

kind of internal ability that we want to have we want to to have more security go beyond the web server and actually tackle this corpse language so that if you are writing your own life or if you if you if you try to to have protections uh for a specific web server in the company that you work on and you want to have an interface so that you can expect the false positives and false negatives then you can actually use mod security uh kernel to understand that data and populate your website recording and make it easy for you to expand and implement so that's an example of what should be internal ability um as a matter of fact we don't have the

goal length uh yet uh we are willing to have so if you guys if you feel like an expert on golang and want to make some contributions on that on that issue you are more than welcome we will be supporting you to do so that's the user who actually opened the issue i think he's moving towards not to i don't think you have the efforts to actually implement that so he doesn't have the time available so we need some effort on that to give an example what the kind of thing that we want to accomplish that's the buy needs for python so if you use uh python you can actually import mod security and do load the rules like we are

loading here and with that we can we can password the rules and we can also uh apply some uh request on top of it and use python to process our request uh if you are more than a c developer than a python developer of course you can use the c implementation is straight from the library that's an example to do so here we are inspecting uh supposed connection that's originated by this iphone to this web server and the transaction was requesting this specific uri and passing the headers bothers a body response headers as a response body and that's basically what mod secure does if if there is any disruptive actions that should be taken in consequence of those inspections

uh there is a specific method that you can call in the library that will tell you exactly their code that you have to return and what information that you have to fully feel in your web server or your application to say hey that request was blocked do it is ordered that so i will say um performance is a key for do good inspection because as as performant as we go more more more uh inspection we can do and but what what is performance after all in that context well uh usually performs performance in this in the context of leads to four different topics which is latency i already talked to you about a little bit about latency

latest is how long the request takes you to be inspected or how long the request takes to be delivered to the user at the time that the user actually requests a website throughput is the maximum amount of data that your web server can can put uh for clients right so if you put a off uh sometimes it reduces the throughput because if you think only to deliver content there is no cpu being used but if you start to do processing like an analyze or decoding strings encoding strings that that will add cpu usage on top of it and that will reduce the throughput of the server we don't want that throughput to be reduced at a point that we cannot do the

inspections memory fruit print that's important to a very specific use case scenario imagine that you are working a server provided that has thousands of web servers hosted if you have like one more security brands in each of one of those it's very likely that memory will be a concern because you cannot put so many data on the memory multiplied by all those sites it will be a nightmare to manage and of course request for a second there is a basically we have to deliver the maximum amount of requests per second that we can and sometimes that's confusing with throughput and throughput and requests per second they are somehow somewhat related but not really dependent for instance

the user can be downloaded one megabyte file order can be downloaded 10k gif file and in that sense the throughput and rps requests per second will be differently because the website will deliver the gift fast and it will be able to answer for the next request for the next gift and so on so those are the distinctions of those of both in a waff we have to take into consideration those four aspects either developing the engine or developing the rules they all defend and well how it looks like when when we are developing mod security or developing rules how inspect those how we observe so that's a screenshot from our internal assistant for rules development and for the engine

development where we classify the amount of requests per second that we are having so that's basically the beginning of the development of 3.1 and now we are about to finish we are here so as you can see we have uh increased the amount of requests per second uh you know we are now in a relatively good shape so and that's what we we see we came up with um what's supposed to be a heggler a traffic for a specific website and we keep sending those in different servers and we calculate those numbers and we make sure to use the same scenario to test different versions either for the rules or for our engine we do the same thing

of course for latency and that's a each point of each point is a commit in mod security 3.1 code right we have the strong heads the maximum latency in the the the light one is the minimum and the middle is the average and of course here the minimum the better uh the lower the better and what we have here is a is a latency way lower than what's supposed to be in the beginning of the development of 3.1 that's amazing because we are doing the same amount of the expansion with less time so it's very likely that by then we'll finish the development it will be around here but yet doing so much inspection and that inspection will lead to be a

more effective protection with less latency that's the general idea if you if you pay attention to the chart you see that some commits increase a lot the latency that that's normal that's normal doing the relevant cycle of what security because sometimes we are testing sometimes we are just trying to we know that a change will be a two steps back but or a step back but we want to have that stat back to do two step forwards afterwards so that's necessary but the important thing to notice is the long long term uh long-term goal uh that's that's basically the memory footprint uh so each bar is a different commute and that's the beginning of the event that's

the end of the development so as you can see here we all have almost the same amount of rules uh same amount of memory used by the same amount of rules no big change on that area yet we are planning to have more modifications in order to have it even better uh that's that that here is is very likely to be a neighbor in the cold and mod security failed to work as expected to to work so that's why we have so low memory usage here and we do have some other problems here and here as well but that's what we saw as a matter of fact it's a it's a grafana chart like this one so we can actually select

the specific version for the giant x the connector version which rules we're using to test if it's alaska will set this if it's oscar will set the version if it's our commercial rules then we can select the version here as well and it plots all this this data for us so if you guys are i have interesting to see this chart i can show you after the talk uh via github or well it's open source so you can actually participate and and check this chart by yourselves and and do suggestions how to improve and how to make the um you know the process better uh that's that's the idea how to improve um the performance of mod security for

both rules and engine um going a little bit over the usability of the mod security uh of this new version it's possible to actually reload the rules live so the prior to that the user had to actually reload the web server or restart the website or to reload the rules that a simple example uh to show you guys how how the user can do it nowadays instead of reload the web server just by editing the rule file uh the rule will be updated and it will be loaded automatically by mod security and give you protection for the website that's very important because imagine that you are surfing on tos and you want to block a particular ip

address so before you had to restart your web server or reload web server nowadays just create a single rule saying hey block all the requests came in for that ip address and the rules will be reloaded and will be applied and the request will be blocked in a minute that you save the file so that's a big change the code is there we are done we don't know if we're make default in the installation or if there is a configuration flag to enable it or not because we feel like we didn't have so many people testing it yet so if you guys want to contribute in mod security and actually test the configurations and to see if

that feature is working it will be very good for us to receive your feedback on that matter because we are avoiding to enable feature because we feel like we we need more testing and more use basis saying hey hey this is okay it's working fine you know if you if you have that i don't know maybe maybe this world whatsoever that hp wax or any kind of weird machine in your home maybe a watch that you are not using anymore that's running linux with an arm and you want to remote security that that's a ideal case scenario for us we don't have a target platform we run what we use to say that mod security can run in

any platform even in your watch so it's still valid test case if you want to go for it go for it it'll be amazing to have feedback if it's working or not of course we have to be uh care about the configuration reload because the configuration can be a bad return and and lead to a network and here is just an example if you ever use mod security is it's very common if you're including the rules including the director twice and that's lead to a neighbor that same rule id is used twice uh not cool and sometimes it's very it was very uh annoying to find where they are or us so let me move forward here a little bit

so i can show you guys that basically the new version tells you exactly which file and which line of the file there is this clash of rule id or if the rules are misbehaving it's going to tell you hey we have an error in the specific file this specific line that was kind of a need we need to do that in order to reload the rules so that we can make a grace for a hermes age in case there is a problem if you are wondering what happens in production well if you load the rules and the rules the syntax is not well formed then it it won't reload it will keep the old version of the rule and tell you exactly

what the problem was big change in the past i'm sure that the users that use mod security a lot will be very happy to have that because uh it was very annoying to investigate where the star was used to include multiple rules from multiple directors with same ideas another thing that is very important on this usability is make use make easy for you to write rules if you are a pen tester you find out that ability you are doing a pen test you find a vulnerability on your uh on your client website and you want to have a hot fix for that a patch you don't necessarily need to patch the code of the application to have it uh

available no you you can create a multi-secured rule for that and fix uh the issue by creating this mod security rule and deploy that as fast as possible it's like a mod security reload or if you're using this reload feature just edit the file and save the hotfix will be there and but in the past was like this the user had to write a regular expression that was matching something in the request or the user has to understand how the logic of behind the application in order to write the rules now we are using leave injection and the user can only detect skli or detect cross-site script using the args is a collection that contains other

parameters to the quest for instance and what security will work on this content and verify if there is something that looks like a skli or a cross-site scripting and therefore blocking lab injections is part of mod security now but it is an independent project as a matter of fact i am nick client9 was the original author for lib injection he was he opened a company for some off products etc so he's not maintaining the project anymore so he passed he passed the project over mesa i am now the maintaining of the project uh with good friends victor uh george pereira and miguel dimora they are also working miguel's in portugal victor in canada uh judged here in in brazil

and i am in brazil as well so there is another project that needs a lot of contributions and not sure if you guys ever heard about live injection but uh there is something you can you can you can get the details on github but what lip injection does is actually try to identify payloads inside regular text so if you go over the website you can actually send some payloads to test leave injection and something new that we are actually uh deployed today is the limb injection port on telegram so if you have a telegram if you add this live injection bot user you can send text to this bot and the bot tells you if there is a

sql injection or a cross-site script on the text that you have sent that's uh that's something that we did just for for besides so uh that's something new that came up of the conference so use of course all the payloads that were using uh it's been logged for the sense that we don't want to blame you just want to make leap injection uh better so if you found a bypass if you if you send something that is a cross-site scripting or is a scale injection and limb injection doesn't feel so it tells you it's not uh report to us because we'll be very pleasant to fix or if you want to be involved in the

fix you can also report we help you to develop the things for that so it will be something cool because you can test your payload and yet you help the project to get better and all this development all these features is of course added with a security guard science in mind so we we run the fuzzer and the afl fuzzer on our code so we actually have a specific compilation for that so if you try to uh we are tackle security here in all different aspects i was talking about web security now i'm going to talk about buffer overflow here overflow and that kind of stuff so if you want to test commod security code

the afl fuzzer it's on the configuration options you can enable that and and start to test the code we can also make our framework better and and of course please send contributions for us if you find a vulnerability please do tell uh we are going to give you all the credits uh for that but uh the idea is really to show that we care about security and we have been tested our code uh a lot a specific machine that's interested it's just testing mod security and it's testing for months now it's just a screenshot the example of the day that we put the machine to work that's long ago it's already shown to us some

things that need to be fixed that nothing no serious bug yet was was was shown by that framework because it was a minor things in memory management that needs to be fixed and was fixed an array but feel free to use test and report back if you if you think that you have contributions to do and remember that what makes mod security project a cool project and a nice project is the fact that we can count on the contributions and the contributions is the key here you're thinking your different perspective on how to mitigate web threats or how to identify those threats it's very important participate in our project you are more than welcome there's a tons of

research around the world that does that and they're working together with mod security so even if you are new to pentest or if you are new to this i want to be more in the secured side than an attack side the protection side so you're more than welcome because then you have you'll be for sure you'll have the guidance from the guys from the community that is on the whole more more than than some of us and and can help us uh understand and and make our ideas more sharp let's say that's the project hold map it's all on github uh so that's the version for 3.1 we think ahead so we already have a milestone for 3.1.1 and

there are some issues so if you if you have an idea you don't know how to mock that idea feel free to go over to github open a ticket we'll classify that for you the correct version we discuss and give all the feedback that we can it will be amazing too to have you participate on that um recently as you can see by that presentation we did measures modifications and mod security along the years along the mountains specifically these last months and sometimes this changes is misunderstanding so example of the cv uh well this cv that you're seeing the screen there is an issue on github if you want to have more further information on that

the issue is that if you want to discuss your security background or your development background how we can treat that issues how can you avoid that kind of issues to happen in the in the future just to give you a little bit uh overhead here uh that issue uh we disputes that issue so we don't think it's a issue on what security but we are open-minded so if you think differently and won't participate in the discussion or if you want to corroborate with our idea that it's not an issue feel free to jump in the discussion and with that i would like to thank you for having me here it's really a pleasure like i said unfortunately i'm not there

with you guys it will be amazing here to be in chile right now uh and that's it and fork