BSidesATL 2018 - Value Change Maps for Open Source Ecosystems (Chris Corriere)

size' who interact Leakey okay so I'm Chris for year the best way to explain me is sentient electrical mathematician in March so I've got a background in mathematics and I did a lot of work design and system design game theory kind of stuff so there's only the services a disclaimer right I do hang out with the security crowell up and I'm barely in the DevOps community but that's a very high level view in the mathematics lets me branch out other stuff pretty easily so I'm gonna we're gonna give this neat stuff today I'm also the coordinator for DevOps tazed Atlanta I'm gonna organize our for DevOps ATL I hope that would be inclusive collaboration movement and

started out of England they're actually trying to raise awareness around diversity in tech so I'm not autistic they don't have Asperger's I'm the complex or a eat so now I'm during the version I'm not very typical seeing lunch or some very nice people and someone said you're definitely not normally do that it's also math Canon which is organized by assignment wordly England inwardly mapping but help them with that that's also out of London I think we're trying to do it in Atlanta next year but come join my the DevOps Atlanta community like we love security people you all are more than welcome to come hang out with us key takeaways here so we're gonna augment

humans with tank instead of replacing them but the key focus of this talk we're going to spend time together community and build trust so we're associated Technical Systems just looking at the tech part of the system you're working on you're missing the big team in the factory social piece right systems are born without people in them they don't do anything you've got to remember we've got real human beings in the system working those teams with mutual goal is a lot of people here on the job hunt if you tell me they're looking for a security position like you know so much about security you've got all these certifications but you're running into these ridiculous job

openings they want 15 years of experience gogoat back in Java and B security the best friend on that team they were okay but we need that more diversely more Center our practice diffused through an organization unless siloed activity right so security can't just be in security anymore gotta permeate the rest of you minimize your threat service so I'm not a security expert I'm not even gonna try to argue that and you begin some dunning-kruger stuff but this this is always true this is all these advice I followed if we're minimizing threat surface we're probably going to have a better name so Amanda I do want feedback on all this video that's wrong your security experts

please don't tell me this is what we're doing in the DevOps community right now around security so I'm going to drop you a lot of information really quick some of this is like two or three days worth of workshops we could do instead I'm gonna cover this in like 40 minutes so we're gonna cover some max I'm gonna give you the security of them sort of software ecosystems and then we're going to get into some action kind of stuff so some apps so I do need to do the intro to DevOps so the first way is system syncing with the system level view the Phoenix project by June abandoned FEMEN bearer and some other folks was a good reference for that the

second way is amplifying feedback leaders we want information back to people who can make changes to correct it you don't just want to document this stuff in story we want to actually know the feedback right um continues pelipper you digest humble they Farley that's another big publication they talk a lot about how many feedback loops in the third way is a culture and continual experimentation and learning that's the understanding this connection between failure and learning that we have fill safe organizations where it's okay to make a mistake and learn you're going to be more adaptive and live longer than a rigid work where you're blame center and it's not okay to make mistakes because then you can Quentin so this is sort of

definition in elevators employee playing some link building we've got inclusion complexity and empathy culture automation lean learning and a lot of this comes out of Six Sigma community [Music] measurement and sharing right this is a broader definition of DevOps from a holistic perspective but this is a general introduction I almost didn't go over these lines because I about this a lot but someone said this might be new to y'all so quick intervention the DevOps and on page three thousand wants to remind you what's cooler than being cool size columns so the pesticide acronym easier to remember easy DevOps 101 stuff if someone wants to pull you into the weeds with a culture of sharing motivation

inclusion complexity and empathy so we put the second upset box to explicitly my daughter the party it's it's understood it's implied in the general term but if security people are thinking little bit office isn't for me we're actually the whole agility thing and being able to pivot quicker as an adaptive organization we're more secure if we can change quicker right so we might got your help with this we're not experts on it but we're doing the best we can with the tools we have the dilemma is the fact that the invitation isn't applied so this is everybody right everybody's in the club who's familiar name is present so this is ultimately a bad game board the probably one that run

into a lot is it who's going to go to jail the only way you can not go to jail in this game or it with this payoff matrix is if you throw your partner under the bus so this is right from this estates for competition right terrible game this is a little bit that anymore this is a stag hunt so we would get two points if we cooperate until a stag we each get one point if we kill a rabbit but in this exclusive work situation these are rabbits and is only starting to understand by himself somebody ends up going hungry so on the flip side of this you have passion equilibrium game states that are

naturally going towards competition right so this is an environmental problem more than the game itself right but it spread out to the environment and we're learning that or design and changing culture within that the right is a big part of getting these saved game States this isn't deductive an inventive but a deductive logic which means it's not like top level zooming in we're not trying to deduce that's true but we're not trying to induce it from a bottom-up like in a recursive step this is more gravitational about what's probably going to happen just if we let the chip stall right or wrong what slightly so this is a tracker in a equilibria this is a board I came up with if you can see

here we've got commensalism has holds up the most spots on this board so this is a numerator of prominent torrents by expanding our board options and our choices where we can just become insular and sharing people together in respect that we need developers and security in the building at the same time we're going to get the competition about some things we're going to step on each other sometimes which is where the mentalistic behavior is sometimes people will prey upon each other and someone's trying to help put them in a position to take advantage of it DevOps management says hydrostatic here but that exposes us to some risk as you all know and sometimes they're going

to get pulled into this competition about doing in access rights and is that will help with the board but if we can always come back and recenter on this commensal state nobody goes to jail fair enough so yeah that's a cop so this is a peach chestnut great the automation does a lot of good stuff but sometimes it leaves messes for security cleanup right these rainbows shooting unicorns thank you all day long so anybody feel like they're dealing with this in their work like there's a DevOps get a couple minutes right so this is the only one this is this is competition not cooperation right definitely is incremental and this is because we're dealing in a complex domain so this is a

kinetic ontological sense making framework by Dave soda and that's a mouthful right we like especially in Western culture we like these linear best practices don't put your post-it note on a keyboard right because we're gonna find it and get into your system but then we get into this good practice where it's a complicated domain where you've got experts say yes we're going to change the passwords of every 60 days they have to be unique and rotate them and then we drift them in this complex domain where we've zoomed out a little bit more have people in the equation it's like well if I'm changing my password all the time then I forget it and all of a sudden I'm

writing on a post-it note again and that's how we dive into this chaotic state where things are invented everything's fine dog right the building is effectively on fire either because we're having trouble logging into our system because we're maintaining compliance or we bond and over this way we provide it and we're still breaking that that simple best practice rule are not writing our password down somewhere so this is where the people come into it socio-technical system we like this linear space we like to refer to experts but it's definitely emerging practice based off of more factors than you're going to be able to calculate so this is what we call fat-tailed distributions just how we compute things with calculus

in central limit theorem out the door we're finding out that the dependency chain and mo socio-technical systems leads to fat-tailed distributions which means you're mean doesn't mean as much anymore you've got wider variance and the tails and those Black Swan events are more likely to occur as the dependencies line up so again those environmental if the dependencies are there for something to catch on fire it's probably going to catch on fire so this makes a Newton cry because it's calculus is sort of like shrunk all of a sudden it isn't as impactful and statistics doesn't mean as much anymore because by Camille distributions are rarer than we would like them to be we don't see them as

often as we should say Thomas waits in his toaster project does anybody know about this this guy tried to invent a toaster but was like this is a simple thing I can buy one for like less than 20 bucks obviously I can just reverse engineer the same take it apart in reverse and toaster that's what he ended up with he's got a book on it and he ended up he plugged it in for about five seconds and then physically trying to toasted some toast before it slipped out but he was like trying to smell his own iron in 40 years of them plastic he was like getting like work off of rubber trees like nothing off the shelf he was trying

to build it everything in-house and he just couldn't do it it's not possible to take it for granted the supply chain in front of us right especially in open source community nobody is like 40 feet like 14 copper for the x86 bus you didn't build your processor either great your kernel this notion that I'm gonna write on the drop of myself and it's gonna be awesome and I don't just want to download the thing off the shelf and use that that's had a lot of very well-qualified people coding on it for a very long time it is ridiculous so I'm gonna spend a little bit more time talking about Thomas and birthday max and value changes so with a value chain

traditional value chain and this one's for a technical need customer need which is highly visible which could be I want to make toast you get your requirements for that and then you source your dependencies right so if I'm doing something I get my requirements I'm gonna write some source code I'm definitely going lean on some open source component or at least some third-party library users or somewhere and I really like things as a service I can provide care provider you just provide that access that functionality to me and I can put it I'll take that option right let them specialize here's a picture of me sign in in kind of are we stop them last year we did this thing

with math camp in this event in Scotland where we were flying back and forth between London and Edinburgh all week but what Simon did with the worth named agnus he brought this out into these domains right so Genesis is this idea exists in the world that we know don't know how to build autonomous vehicles or in Genesis right now they're not that complex domain where it's not quite emergent it's crossed that threshold until we're trying to manifest the same we just haven't done it yet once you move that at Genesis you get the custom-built where certain organizations will be able to build this thing in house they've seen one exist in the wild but it's not a product you can buy off

the market like off the street get after we get done with custom built this diffuses further into products you can just buy it off the rack somewhere there's also a rental functionality and it comes with that and once we get out of project it comes in to commodity where I might be able to summon an autonomous vehicle from the kerb with my phone was like lyft or uber or something like that and this is the game play Amazon made because they took computing turned it into a commodity function it's a utility you can spin up compute like you can turn on a tap for water or you can flick a switch or electricity right which

means I don't have to build data centers anymore I don't have to deal with that complexity in-house I can outsource it so here's the wordly map for developer feedback this is somewhere what we're doing within automation within the deficit cops this automated feedback is this commodity thing where we really want that flying out like it's off of a tap we like my de plugins that go straight to a product level and get that feedback to a developer as far left in the as we can so it's in their face and then the position where they can take corrective action with where they commit their gooded once they've done a poor request in this automated don't has run

and they've got all green lights from the automated system you get rid of all the dumb noise like hard-coded password changes that we don't want to waste a security professionals time with and we can get to the more articulate peer review that's going to be custom-built because we really want that to be a face to face conversation with a real person or maybe even a group of people this ties in the source control again the development environment itself is back in private domain go ahead and shoot intestine back in house in this should line with the many visible development mean that's coming from a business a customer might be a business customer might be next from right where we are

injecting some of this feedback I don't like preschool that much they were using it at the shop this this is real work this was an app I was using internally a couple jobs ago they're using crucible for code review but they moved to get full or if you follow porter class before that gets merged in we run a number of automated checks and if it doesn't pass you get a notification you got red and yellow lights you need to clean your code up and make another commit before we're even really going to consider merging games trunk right center on can be used for static analysis Longwood makes its lifecycle Nexus lifecycle does third-party heat dependency management there is no mouse

plug-in that does it as not as strong with the tool but the lost one is free and fairly hearing this automated feedback you've got build pipelines which is going to be Jenkins or circle CI or go see there's a number of them unit testing center on cube Skinner a lot top ten we've got the point integration tests things like thread stack HP fortify on demand is still popular it's not my favorite tool but it's better than nothing they're not skating and I also like those attack proxy so we spin that up automatically during an automated build it's not a full authentic pin test it does not replace that but it does get us some quick feedback even just the spider off

of that to hammer our URL and tell us what codes not getting executed what links are showing up dead it's still a lot of good feedback and they brought an understanding of risk profile right and I have talked to a lot of security folks at the prefer bourbon sweet but again bread Suites not a free tool on a desk cost a little bit of money and it doesn't lend itself to the automation quite the same ways that those right now if you know more about that so security even open-source II ecosystems what does this mean for ASHRAE don't build a house just gonna get it from somebody else but the Box critter here at all so back to

the skin so we ended up having a big break with back in JavaScript because all of a sudden front end and back end code ended up looking very similar it allowed a developer to ramp up and learn how to code full stack a lot quicker and then we actually had a shout out to dentist and in a let that happen what's the date that I have on there was last year so this wasn't a vulnerability right what this was is this is left that was just a little dependency within the tool chain right so if you're running a repository which could be artifactory or Sun inside Nexus is another one and you had left pad cached on friend you

had it on your local network you were fighting that day but if you're resolving everything from in hand in real time and depending on that external dependency this broke production you could not build a ship code if you were running anything that had had this dependency in its chain right so this isn't the security vulnerability but it's still a vulnerability in some exposed risk that could have been mitigated by running a an on-site repo right and this dependency thing chain dependency chain plays out where yeah it shakes it open source trust already so 39 miles does packages an NPM undetected for two meats and I feel personally that the JavaScript community left a lot of

lessons learned at the door that Java had been through a lot of this stuff and knew how the pendency management work better and has a stronger ecosystem but it's just like that's too complicated we wouldn't could ingest a lot of that one went out the window with it right so relearning a lot of lessons in the space 52% of all JavaScript Indian packages could have been hacked being weak credentials so this was just really not changing login password around dependency management with the NPM repo so malicious code would have been skinned and discovered these these issues before the attack could have gained published access to almost 70,000 NPM packages which is 13 percent of the

entire JavaScript fu ecosystem that's a lot right that's the odds of you pulling down something hot have gone up a lot but if you pull the dependencies along with that it jumps to 52% malicious data so maybe it's not the thing you're using but if the thing you're using depends on something that I've got an exploit in and you pull that in are you scanning CVS for all the dependencies of your dependency right this tertiary dependency situation 52% of the NPM ecosystem that's that's not all of it but from a security perspective that's enough from kind of like ok that's pushing everything at this point and this one was just a couple days ago and

this might be a rehash and it's the same offer but this just happened again where they found a deprecated package that wasn't in use of you mark and it kind of fluttered away and someone just wrote out a vulnerability and an exploit in this package and sure enough there are enough people using deprecated packages that pulled it in they didn't even need this it was just bad housekeeping it was left in the dependency chain they had an upgraded and it was an opportunity to sneak in somebody realized it wouldn't publish a threat they came in they're gonna pull this they don't know what there so what can you use socio-technical systems depend on nine fundamental human

needs this is Manfred max-neef I'm not going to read out all nine of these but they do have four existential categories for each of the nine needs this turns into like a 36 L matrix so here's just two for the show that you need subsistence this means being physically and mentally healthy which means having food shelter and employment which means eating drinking sleeping working and interacting in your home in place of employment so all these break out that way right so participation affection and this is what I want ology it's not the hierarchy we're used to saying and it's it's a way this this is getting into work-life balance a little bit right you can't

tell me someone who's very rich but it is very alone is going to be happier than someone's poured that has a extremely strong community relationship right so this is your IT theory which again this is a flat on ecology and existence needs plans or relatedness needs playing the growth needs but we moved back and forth this from a flatter pilot is less hierarchies a flatter ontology these two things together have replaced Maslow's hierarchy need rate and the reason this is important is this is getting back to the human centered design aspect what does this have to do with coal mining right so this is a paper that got published in 1951 about long long versus short pummel short of

all coal mining so you used to do this collaborative tool routine with which a short wall where you went in with your your picker and it was on a contract basis they got paid together it was right or not the miner had the the haulers back a lot of communication and collaboration and the very very much a sense of team and belonging but they introduces one wall this is a filler say he's in charge of getting him coal out of the time they went automated this stuff which means they have to put these long belts in which turned into long wall mining but what that did is it broke apart the people in the system and

put them straight to take into the tech they were dealing with hardware and study humans and communication with them so this is out of that paper but you can see you have two Porter's together two companies are cutting the core when they found it governors who were sending out these the braces and all the conveyor belts that were shipping this stuff out brain fitters who would put down that one after they were done cutting and the builders would build it back up and then you have these 20s dollars on our runways and they're not talking to each other right and what ended up happening is these folks it's the fillers weren't getting go out of mind nobody got paid

so this hierarchy formed where these folks had a little bit higher status were closer together could talk around behind these people's backs and all that flow downhill where all these fillers ended up getting blamed if coal wasn't shipping so sickly went out people stopped showing up for work they had a higher turnover rate people weren't making as much money because they did not Center the humans around that automation when they put the automation so being together with technology security weren't they nice we want to augment the human needs in tech instead of replacing them spend time together build trucks working diverse teams neutral goals and minimize your threat surface well I hadn't done the dog

soccer dog ate my homework so we even sonarqube for a lot of this I have a couple docker containers up what the demo was doing is it was pulling up Jenkins and a docker container I was building web GUI is used web built before number of people so it's an intentionally intentionally flawed program to help you learn how to do security better so there's a velocity pendency check plugin for Jenkins which isn't as good as a Senate I product but it is free which will give you CVD data about third-party charge you're pulling in from Haven central it might be hot after that Milken completes we run a certain latitude scan which pulls out a

loss top ten some CWD stuff and in other things like code complexity and duplication so with respect to mitigating Aurora's removing duplicated code and dropping this complexity where you're put into more maintainable more accessible that's an area and your threat surface even if that code doesn't have a CD in an explicitly if it is it's not a vulnerability I'm still looking at can I shrink my my footprint overall and part of this is the container game so we've got containers the idea isn't the gradual virtual machine and sign into the darker container and put that into production you want to run the leanest you can so like Amazon's got help right now it's a really small footprint unit

kernels is more of a theoretical thing right now where it's just the code it's like okay make straight into the chrome process so there's not even really an operating system in that container but even in the course of doing my demo the Jenkins latest docker container are pulled down had a bunch of the security security vulnerabilities in it so I had to take that off the shelf component and put it back but I had to go build an updated container on Alpine with the the internal bash from the bun - which is lighter weight but have to make maven into it so the complexity automatically increased just because the stuff you're pulling out the darker in automated in

central is probably update if you're keeping maintenance up it's almost a dedicated role in a security container talk we heard earlier today believe it to some of that where it's just a level of rigor in detail I'm making sure I've got clean images to work with I understand all this and I want to help but maybe that means I need to home security to catch up with darker and kubernetes and cloud stuff I want them to focus on the security tips right so how can we communicate with this and collaborate better and so yeah Jenkins bill is going to get the dependency check center on to you for static code analysis then we would deploy that to an

environment and hit it was that which is an automated pen test I run a coverage agent on those so it just tells me what code is being executed again it's an opportunity to pull dead could have the system and lower a threat surface human believes benefit if I spider your site we've run all our automated testing and I don't see that code think that by code executing anywhere in that system probably safe to believe it go ahead and clean it up so I ran out at time start with a map try this there's a book by Elizabeth show should call it a social practice mapping and there's a process for popping these out and directly attacking it once you've got

this practice established it important this cuts down communication what would be a three hour PowerPoint meeting with a lot of people bidding on stuff you can hand them a Mac and say when we move that over here and there's like oh that would be better if we can make it work and automate as much as possible we'll see to respond with agility if heart leads a good example the strength owner ability and the nickel effects got hit with is another one but if you've got to go through an internal review board and deploy a patch through SSH manually on ten thousand ohms it's too late right the idea what you really want to be able

to do is if you've got infrastructure is coded together the second component building comes out you could be clean in production today we get a zero day out if I've got our automated I can drop this I can patch my image or my container drop those 10,000 nodes and reprovision them and you know what not even an hour it shouldn't take an hour to do and we saw a lot of work to be able to respond like that but it's because they it was failsafe there's high trust and high communication it wasn't this rigid blame driven process centric behavior we see in a lot of places diversity mitigate renesis gets into hash B's law requisite variety the

complexity your system has to match the complexity of your environment our environment is getting more complex as it gets more connected that gets back into the Black Swan events from the fat-tailed distributions but that's what the tools perspectives in people you need multiple roles they were kind of people on your team will pull feedback from different places to be able to break in an effective mahakal trip that's all I got today sorry negative the demo I will give some code up on github so y'all could run that on your own and yeah thanks you're listening I'm very open to feedback so you can tell me how DevOps could be doing security better let me know we'll get it back to

you