Shift Left With DevSecOps: Scanning Every Single Code Change

Hey guys, very good evening from India. Before I start, firstly, I'd like to thank a lot B-Side to organize this conference and the event during this pandemic when all of us have actually connected. So huge shout outs to everyone who is involved in B-Side to organize this event. And also thank you for giving me this opportunity, this platform to present my talk. which is about shift left with depth of cops scanning every single code search. Before I get into talking a bit about myself, my name is Alnazh Jain. You can find me on Twitter and medium with handle as logic.com underscore one. I've been into InfoSec for past five years and so. I work with various MNCs

like Expedia, they are local Indian companies like grow first, make my trip. And currently I am working as a security engineer in a startup, which is called Cred. I'm a cybersecurity speaker, has been speaking in various conferences, meetups, talks, and on various infosects. I'm an active blogger. You can find me on medium with handle as logic bomb underscore one. I've been writing about lots of stuff that I do in my infosect journey. I have been doing a lot of bug bounty. I've been writing about a lot of vulnerabilities that I found during my journey, various topics about DevSecOps, Docker, and everything in the process. How generally in my free time, break is stuff to learn. I don't want to call it bug bounty because I started this stuff

to basically just learn and grab new learnings and new technologies. So in a few time, I spend my time to break stuff, break bypass logics, uh, break application logics and, uh, get some vulnerabilities out of them. Uh, I've been recently featured in various magazines and editorials like Forbes magazine, BBC tech range. There are various Indian newspaper like economics times, news sometimes, uh, I've been doing lots of research on, on data leakage, uh, on data beaches, uh, and finding various vulnerabilities in Google, uh, and other, other big platform. I've been acknowledged by, Google, NASA, Yahoo, European government, US and UK government as well, and also some top companies of India and outside India. So before we get into the depth of the topic, which is scanning every single

code change, that's the cops. Let's first understand what is this shift left means. The first two word of the talk, which is basically shift left, what that exactly means. So I hope you everyone must have seen red light in the traffic, which is turning red to stop the workers to close the door. Also, you have seen a rain more during rainy and sunny season. Rainbow comes when you they are seven colors and you always see a red color coming at the end of the rainbow, which is good. So that is something which always comes under. Have you ever wondered? why the red color is used to stop the cars or why only the red color is used in traffic light to stop the cars or

to stop the vectors or and why the red color always comes at the end of the rainbow so the answer is basically if i talk about scientific answer red color has the largest wavelength so anything so because red color has the largest wavelength it has a tendency to move away from the source so as you come closer to the source that is as you come towards left side, that means you're bringing something close to the source, which is what shift lab actually means. So whenever you are shifting to the right side, that is you are going away from the source. And when you are shifting to the left side, that is means you are coming

near to the source, which is what we mean by shift lab. Now let's talk about what shift lab means when we talk about DevSecOps technology

So shift left in technology basically means when you bring your process, when you bring, or in terms of security, if I say when you bring security testing, security checks close to the development process, close to the source from where the future development is getting started. So shift that basically in security, what it means when you're shifting to close to the source of development, close to the source of release of future or, or basically the discussion around features. So that what means best in terms of technology. So insist on waiting for the security at the end of the development cycle, you can include security checks or security testing within your developer workflows, but bringing it closer to your developer's code, but being as much at early in

your agency cycle. The reason why we are talking about here is that why we should bring security as close to the source because because doing security testing at the end of the cycle, doing security testing and VAPT spend testing, when the release is done, when developers have coded the release, he has given the QA, QA has done the QA testing, and then it comes to security testing. What actually happens is that when security team find any bug, they have to, again, that bug has to again go back to developers. They have to again revise their code, they have to fix the vulnerability, it again go to QA, you were against me to do Q testing and

then it against come to security testing for verifying that patch. So what actually happening here is that you are actually revising your SLC cycle again. You are actually actually repeating the steps again. You are doubling your effort and you are doubling your time. So your time and effort are getting replicated and getting again and again done. So this is the reason why we should not uh take security at the end of the cycle uh a few slides after this i will show you what impact from finance purpose and from from a business perspective this can cost to a company and also security should not be created in the afterthought because we don't want you know when release has gone live when it

has gone to production and then we are thinking about security uh someone finding someone finding report and reporting vulnerabilities vulnerabilities to the company and then they are going about to fix it the cost of fix always increases when you take anything considering after it has been done this is the graph which effectively shows how your cost gets increased when you move security to the end of the cycle or when you move security away from the source so if you're moving security away from the source that means you are bringing more of our corrective measure rather than preventive. Rather than preventing security at its source itself and do not allowing to get developed. If you are bringing security

close to the source, you're actually preventing the security from arising before they are coded. And when you are moving away from the source, that is when you are bringing security at the end of the cycle, when you're bringing security at the end of the cycle, you are actually moving away from the source and you are again and again repeating the complete cycle. The cost of the fix is getting increased because you are now spending your time and effort again to go and revise the code again, going to do a test and then again to come to security to verify the patch. So the complete process is getting, getting, you know, again, I started starting for

you and the cost of your fix is increasing as you're moving away from the source when you take security away from the source. So this is the impact and these are consequences that a security, taking security away from the source can bring into the organization. And this is the main reason why we should not treat security as an afterthought and why we should not bring security testings, VAPTs, all kind of pentesting when the release has gone live or when the release, when the developer has coded the functionality.

Now I will be talking about what exactly we are going to do in order to tackle this problem. Something which is called continuous GitHub code heading. So this is the architecture which I will be going through again after a few slides. So this is a basic architecture of the of the tool that I have deployed in my organization, which basically has brought security checks during when developer writing quotes. So instead of waiting that developers return the code and then QA testers the QA for the QA, and then it is coming for security testing. What we have done is that we have brought security checks within the developers code platform. Whenever they are raising any PR, the security sets are getting performed and

then getting, getting performed and then we want to tell us if there's any finding I found. So I will be coming to this architecture, how this architecture works and what is the salary to back end app? What are the checks that will perform for us and what exactly G-Shield works? G-Shield is the tool name that we have developed, which is GitHub Sheet. Now how GitHub Sheet is basically work is, So GitHub provides a very great feature of creating a GitHub app. So what we wanted is to basically bring security as close to the source. In order to do this well, the most logical part security can be checked are code reviews. Since at the time when we were implementing this, we didn't have a center

pipeline to, so we thought of building our own tool which can listen to any new PR which has been raised. And then that is when we came across this a feature of GitHub, which is called GitHub apps. So GitHub apps basically come with built-in webhooks where you can specify permissions. When you set up your GitHub app, you can select which repository you want it to access. Suppose you're creating a GitHub app that can write issues in a repository named octet. So only that, so that particular app can only write to only that particular repository, no other repository. So we created a GitHub app, which is known as G-Sheet. Now this G-Sheet, this GitHub app can be integrated with every repository of the company. You can even

give access as much as you want. You can give a read access to PR to this app so that this app can listen to every PR which is raised by the developer or by the engineer. You can give a common permission to the app so that this app can have a ability to comment on your PR. So you can also configure this app to listen to a pre-configured webhook URL. And this webhook URL is basically the heart of our tool, which is continuous GitHub hacking. What actually happened is that whenever any event is triggered, when I talk about even this is GitHub event. So whenever anyone raises any PR or any kind of event occurs in your GitHub, it sends an STP

post because the vehicle and this vehicle can be used to update any, any, you know, trigger any CI build or it can even deploy, go to the production server or it can update an issue tracker for you also. So this is the functionality this we have who can do. Again, we have who can update your personal issue tracker, it can trigger CI builds or it can even deploy your code to the producer. to your server. And this is where the modular approach of our code comes into the future. When I talk about modular, that means the code is the model in a way that the checks are independent of each other and they are highly customized. That means you

can add as much of checks you want in your pipeline,

in this particular tool, as much as security checks that you want to add. you can add in a very modular way so that every check works independently. If any check failed or if any check not able to perform well, other checks can perform the ability to what they are coded. Now, having understood how GitHub app basically work, now tell you the architecture that we have built by using GitHub apps, which is G Suite for us. how the architecture looks like is that now we have a G shoot, which is our GitHub app, which is continuously send the input received from the events. So when anyone in the company raises any PR, the GitHub app, listen to every PS because we get that

GitHub app, which is integrated to all the repository of the company. So when anyone in the company raises any PR, any of the repository, this GitHub sheet, which is a GitHub app, listen to them. That is the PR event

which anyone is raising. And now there's a backend app, which is in Python.

This backend app is listening to the events which are sent by GCU, which is the event which GCU received, which is a GitHub event. And then it is sent back to backend app. Now, backend app received those events. back and have as the ability to perform security checks on the top of those PRs. So suppose someone in your company, there is a PR and that PR container, the backend app, the G-Sheet, which is on GitHub, received that event. It sent that event to the backend app. Backend app will receive the event and we call backend app is highly modular and customizable. We have written custom checks where it checks whether the PR

actually contains a Docker file or not. And it checks and then perform a Docker file landing on it. It performs various other checks to find that if it is a Docker file, it performs, it checks whether, what is a base image from where the container is getting built. And then we have our own security checks like it checks the PR for hardcoded credentials. If anyone hardcoded any credential in the PR events for which we are using uh get secure aws lab it also checks whether someone has mounted sensitive volume mount points in your docker file so the the backend app is the python script which this is scan your docker file finding find whether any of the sensitive host mount point is host

is basically used in your docker file and just feel the build and feel the and basically just do not allow the engineer to merge the merge the pr if any of the any of those chats real It also perform assess which is source application security testing. We are using SonarQ for it. And now at the end Jenkins coming to the chat. So how Jenkins is basically getting cut here is that when we are receiving PR onto the backend app, we need SonarQ to run on those type of on those code. And we created a pipeline Jenkins which basically triggers SonarQ to run on those repositories, generate a report for us. report for us and send it to the to the

to the PR as in comment so what actually happens now is that anyone in the company raises any PR get up field receive that even that even goes to back an app which is a Python skip for us the back end app scan the PR on the basis of security that we have written which are which are modular and highly customizable we have written check with the format of the file enter we have written a check with checks for sensitive volume on one point we have written a check which checks whether if there any hard code and conditions in your PR we have written a check whether it really runs a sonar cube as an

sast for the on it and also we check whether the base image which being used in a docker file is actually a authorized official image and we have created a whitelist of our own Docker image and that basically compare whether the from director in the Docker file has the base image, which is into a whitelist that we have created. And if any of those checks fail, it, because the GitHub app has the ability to comment on the PR also we have. So when, when, when any of this checks fail, the GitHub app comments on a PR saying to the developer, just like all, just like we do code reviews, it basically leaves a comment saying that Your checks is failed and you need to fix it before you get

it much The github app is also has the ability to do not allow the The engineer to merge the PR until unless all the checks are passed So the so for now because we don't want to we know create a hassle for developer what we started with and not we are not blocking the PR but we are commenting on the PRs saying that if your checks are failed you need to fix it before you proceed ahead and So this is how the architecture looks like. And then we have a salary queue, which is, which basically handles your multiple PR events that you're coming to Gsheet. Of course, you have a hundred of developers in a company, you're raising dozens of PR in a day, in an hour.

So salary queue basically is a queuing system that queues your PR and then back end app picks it one by one and parallely as much work that you have given to salary queue. So this is how, this G-sheared work for us.

I already spoken about it. So what are the capabilities that GitHub core has is that we perform a SAS scanning, which is static application and security testing using Synarchy. It can also perform DAS. For DAS, you can use WebSAP, OpenSource tool, or you can build your own to perform a DAS scan. It can also perform dependency scan. Again, for dependency scanning, you can use WebSAP dependency scanner to find if there's any external dependencies in your PR event. And you can compare whether the versions are outdated, whether it contains any CV or whether there's any high vulnerabilities in the dependencies that are used in your PR or in your core repository. We are also written a continuous scanning check, where it is scanning your Docker images for vulnerabilities. So

you can use an core for it to clear to run Docker scanning on the images. And then there's additional capabilities which we have, you know, using the company where it detects any secret which are getting leaked in the PR. It performed Dockerfile linter. We are using HydroLint, which is an oposal tool to perform Dockerfile linting. And then it also check whether your Docker, which is getting built, is building from a secure, trusted, official base maze that we consider as safe. So these are the capabilities that this continuous data code hacking can bring to you. And this is how, this is a sample how it looks like for us. So when anyone raises NPR, it triggers the GSheet. GSheet sends it to backend app,

backend app returns all the checks. And because GitHub app has the functionality to come in, it basically come in on the PR just like every human does during code reviews. So if you can see, the PR doesn't have any secret in the PR. So it commented that there's no secret leak. It also gives you a full report of SonarQ, which basically says scanning tool. And again, it perform a Docker linting on the Docker file. And then it comments that what are the checks your Docker file has failed. So this is the sample how the GitHub, so how basically when anyone raised any PR in the company, how it looks like for them. What are the improvements? So it basically brings a lot of

improvement for us and also anyone who want to use it. Now, because now we have brought down security checks and the building when any developer or when any engineer is in the PR. So now instead of waiting, the PR is raised and then it is matched to dev branch and then the code is deployed and then the security team is coming to the picture. Now what we have done is that we have brought down security scanning as checks within, you know, within the code review. So whenever anyone raising any PR, your security checks are also getting performed as and when. Then the result is posted on PR as a commentary view. It is highly flexible.

I've been saying that it is very modular. That means you can add as many checks as you want in your, in your, in your backend app. So we wrote multiple four, five checks to run whenever any PR is, and you can even add more checks as you want going ahead. And it is completely automated. So till now there's no manual work for security. Everything happens automatically. When anyone in the company raises any PR, the GitHub app runs, which is issued for us, it triggers the backend app and backend app runs all the security checks. and perform and find the vulnerabilities and comment on the PR as we find any vulnerabilities in the PR. So the complete cycle is completely automated for us. So

as a whole, what we have done is that we have brought security checks within the code reviews, which is basically we have shifted to the left side and we are scanning every code change. That means we are scanning every PR which has been raised, every code change that has been going into the production system. And instead of waiting at the end of the cycle, instead of waiting at the end when the future has gone live, when the future has been deployed to production, we actually brought security close to the source where we are doing security testing as and when the code is done. So for us, we are not following the philosophy of security, creating security as an afterthought. so we have brought down security

checks within your close to the source and doing everything uh doing the code reviews itself so these are the improvements which uh which you know this particular app and particular tool brings to you uh this there are early checks uh you can perform security checks within the code reviews these are highly flexible highly modular that means you can write as many as you want and they will run independently of each other and it is completely completely automated you don't need to wait for someone to raise the PR, then the future goes to production, and then you are doing the security testing. So everything is basically automated.

So this is about the GitHub app. And because the theme of the Bistray Charlotte is years of improvement, so I will take a couple of minutes to talk about what being, you know, how has been the year for me so far? What are the things that I learned and what are the things that personally I achieved during this COVID, during the time when pandemic hit us. So again, to keep aside everything which is going around the world, to keep aside the negativity, the pandemic which, you know, going around. I've been doing a lot of work on my personal growth. I've been researching about a patient data privacy. So I recently, a couple of weeks ago, I

reported a bug to Indian government where they were leaking cold patients data, data, data details. So I helped them to catch it. And it is also covered on various, on various newspapers. One of them is the Convice Times, which is India's newspaper. During the pandemic, started learning car driving because I think because being being away from your workplace, you are actually getting more time to spend with your family or to spend on your own stuff. So I whenever I find time I started, you know, I said I go for a long drive. That is how I know learning car driving. And I've been fortunate enough to spend good time in the family. I've been away from home for last

nine years or so. And, you know, It is like a blasphemy disguise that I'm getting so much time to spend and enjoy with my family. So this is how I've been here going for me. I hope you're also making full use of working from home, you know, that area. And also you're keeping yourself safe and learning new things and learning new habits, learning good habits to stay away from all of the things that is going around. So yeah, this is about our talk. And this is about the years and more for me. Thank you for being a patient listener. If you have any doubts, you can reach out to me on Twitter. My handle is logic form underscore one. You

can just drop me a DM, which is open. So you can drop your email. You can also email me if you want to shoot out a mail. My email ID is OUSD. And again, You can if you want to read about various infosec topics and also depth of ops, continuous security, data breaches, I've been actively writing on medium which has been published on various various platform like BBC and Forbes and others so you can obviously go to medium and check out them. So this is from my side. Thanks a lot guys and I hope you will have a fantastic talks ahead in Visha Charlotte. So here I say goodbye. Thanks a lot guys. Bye bye.