Bsides Orlando - Owen Redwood - Lessons From Training Ninjas

so you're actually proof that junk food is good for here my car is still in one piece yeah it's so that's kind of like indicative of the luck that I have um in a recent interview I was asked whether or not a series of questions of whether not I'm an it's a good thing to find out before you're hiring someone and so he's asking the questions all these weird ways I'm like look let me cut to the Chase look I've been stabbed I've had a number of guns pointed at me in weird situations and I've accidentally wound up in a Minefield and turkey still drunk from the night before it takes a lot to get under my

skin so I'm just a guy who works really really hard and every now and then has crazy good and bad luck at the same time and so this talk is going to be basically about with a little bit of luck and a lot of hard work how you can do something similar to what I've done so are we rolling good to go yeah okay so the to this talk is lessons from training ninjas or basically how to revamp your security curriculum um the target audience for this talk is really Educators perhaps professors or instructors um I myself I am a graduate student um and I founded uh the cyber security club at my University and as at

the club meetings people would just show up and they'd just be like no one had any initiative to you know propose to do things they just kind of showed up and expected to be like a classroom so I started teaching things to get everyone to a level where they basically confident that they could start doing things and so it turned into a class basically the club itself and so someone suggested why don't you just take what you're doing and teach an actual class and so this is basically what has come of this so the purpose of this talk is to provide you with wisdom insights and tricks so you could do the same because uh the reason I'm here is because I was

invited to talk and the class was I guess really impressive to the organizers so at some point I was appointed as the Director of a research lab on my University and that basically means I'm the unpaid system administrator who helps the professors get their stuff done um I founded the C the CTF group called n pointer um I primarily come from assist administrator and computer science background and perhaps the most important thing on here is I have a lot of experience being a student and that's that really comes uh into play when you're actually teaching a class being able to relate to the students especially at this level um and I'm here today because I'm teaching this

class so most security classes and major universities focus a lot on crypto in our General survey of topics at a high level um sometimes it'll make you actually do a lot of crypto and that's Hands-On in-depth stuff but you don't get exposed to stuff like intrusion detection signature writing um writing uh SQL injection and stuff like that and so what I noticed in my own University is people were being graduated with degrees sometimes in infos SEC and I found them to be not as high level as I should as I would like for my fellow classmates um and so and this isn't this isn't just specific my University a lot of universities Pro students who usually

can't do security training and that's why we have all these servs so this talk is actually an Ode to uh Dan Guido's 2009 talk at source Boston so you want to train an army of vengers and he uh took over a class that was an existing class that had fallen apart called application security and bbilly Analysis and the website for it is pent test. crypto city.net it's a fantastic class he videotaped every single lecture the class was taught by himself and five basically industry Professor professionals like they had a guy who was really good at re guy who was really good at malware analysis guy that's really good at web application security and uh they came in and they taught what

they are really really good at and uh it is kind of um for open- Source classes it is whenever you talk to people it always comes up in a conversation and it's quite successful and so my talk title is lessons from training an Army of Ninjas is simply an not to off so if you want to see the website instead of typing in you can use this QR code um I have a bigger one on the next slide so in in my class I decided to raise the bar for the students and uh we went in a in a very aggressive Direction I went handson and in depth on x86 RSE engineering exploit development Network

hacking web application hacking and then at the end I taught them the tools I dragged them Kicking and Screaming through all the fundamentals because I didn't want to train script kitties so the the the perspective I taught is teaching all this stuff from a very red team heavy perspective because I believe that if you understand how to attack a system well you're much better equipped at defending it um and so if you want to see website I don't see anyone with their phone out so the result is some of the students in my class started off as news and now they're actually very very good um I would consider them ninjs and so I didn't do it alone um I don't have five

industry professional friends that I could have brought in I had the help of one and then my professor low me to do it and then a fellow PhD student and I didn't get their permission to use their face so I just use the appropriate images and so why am I here today why am I talking because there some really interesting results came from this class um we had some really good work on uh improving Security in the Android current especially addressing the non-existent permission solution for Android um and that comes from uh students work to basically port Droid uh from foyo up to the most recent Android which is a very difficult task in itself um and these

are all individual term projects that students chose to do uh some uh actually chose to do extension of volatility and a lot of these ideas I got from talking to my friends in uh the industry asking if you had an intern and you had 10 weeks for them to do whatever you wanted what would be a really badass class project for them to do that they could listen on the resume it would help anyone out in if you're an incident responder or malare analyst uh and they could they could get it done it would be a realistic uh project for them in 10 weeks and they could have something to really be proud of because the majority

of term projects in classes are BS things that are just easy for the professor to grade granted these are open-ended projects so it's very difficult for me to grade them um and it's especially for people to grade if if someone else were to take what I'm doing and do it and someone follows a project that's not within any of the realm of expertise of the professor so there's some difficulty in approaching this in itself but uh there are some really good uh projects like um students writing custom encoders for uh polymorphic payloads um and which brings me to the challenges of teaching this class I didn't take an existing class and one that was decaying

and revamp it I created a brand new one from scratch and that has a whole new set of difficulties um and if you're going about this route usually one of the main hurdles to teaching a new class is you have to get your faculty members or your chair of your department to care and the way to do that easy is to speak money so there's a lot of ways to do this um there's a lot of funding sources the NSA centers of excellence centers of academic Excellence they have cyber Ops now that's brand new infos research so FSU already had uh infos research and we were going for cyber Ops and this class was a huge help to getting that however

I don't know if the NSA really knows a Clear Vision of what they're going for cyos because we in academic Community feel clueless because they keep changing the bar that we have to jump to there's also NSF grant money and then there's extra funding for SFS which has now been renamed to cyber Corp um and these are all wonderful things however I should note this NSA cyber of academic Excellence is going to be uh completely changed and they're wiping the Slate clean in like two years they're taking away everyone's status and redoing it all um so this will probably change between now and then next challenge is you have to organize the topics that you want to

talk about and this is not an intuitive challenge in itself um and lastly you have to bridge the knowledge gaps this was one of the more challenging things I assumed the students had a basic competency in a norm um I assumed that they had talked about Security in their software development classes and a lot of things that they didn't know surprised me and we had to as with any class deal with them so in essence we want to produce students with a ability to actually do all the material that we're teaching um but in order to do that there's basically you have to address the Swiss cheese knowledge Foundation that they have and brid all those gaps I found perhaps the

best way to do that is to not actually spend time okay we're going to spend this week reviewing this setting up virtual machines and stuff like that um I actually inspired them by hacking stuff regularly doing demos like hey I just popped a shell this is how I did it showing people these things actually inspires them to hey actually go that's fascinating I'm going to go do this on my own and look it up um and so before I just start rambling I'm going to follow the rest of my slides so that line kind of basically how to plan a new class and see it through uh is basically you have to I think address some eight key things and

then uh Dan Guido presented a very good how-to and I'm going to basically touch on what he presented and how I took it and made it work for me who knows this yes if I was if I was going to stick around if I could I I'd buy you be um so you have to seriously determine how many core areas do you want to cover um do you want to stick to a textbook or do you want to go your own way crypto is always the big dog in the room when talking about security every security class teaches it and they teach it for a reason it works however teaching it takes weeks if you leave out crypto in a

class like this you could cover more other material um but if you do teach crypto you have to keep it in every single thing it touches everything um so the reason I left it out in my class uh is that the rest of security classes in my University all were extremely crypto heavy and they felt like copies of each other so I decided to completely leave it out because the students will get a very strong crypto uh background from just taking the other classes so other core areas you might want to consider is teaching reverse engineering code auditing is also extremely important um for any offensive or defensive security um do you want to teach actual exploit development how

about malware analysis and so on and so on and for for all the core areas that you choose you have to identify some Advanced topics that you want to shoot for for instance I wanted to shoot for explaining R to students and making sure they come away with an understanding of that um and then executable security mitigations aslr D NX I was shocked that students were being produced uh with the degree and they had no idea what these things were and I also chose to uh go uh let's see in depth on a number of other things but they came along as the class went um so I kind of set a loose plan at

first you have to establish what you expect the students to already know um that way you can have basically a better way of keeping track how the students are failing your expectations or how they are meeting them um and then if you can perhaps establish a prerequisite for the class I found out at the last second that I couldn't because the class was brand new it had to be a elective so told students the first day if you haven't taken the security class before you will fail this class you will get an f and you will probably cry um and it was good that I said that because like five people dropped um and then you have to find out what's

you have to come to terms with what you think is the best way to assess the Grasshoppers results and their progress um I say homeworks and in my homeworks they're basically ctfs I participate in a lot of ctfs and this entire semester every weekend that I could I was doing ctfs and some I working on problems all on my own and I'd solve them and then I'd take them and I'd use them against my students I'd obate them so they couldn't find out what it you know look up a write up or anything and there a number of times that there was no ride up and I was I and my friends were the only ones solved it and so I reused it

against the class and perhaps extra credit projects and presentations are kind of long-term things unless if you do presentations frequently uh Dan Guido's class does uh presentations they I believe every every class um student is basically pick to find something out present for 5 minutes and the reason I think presentations are extremely important is if you produce hackers who cannot communicate they're pretty worthless and the best way to assess their abil their knowledge of the skills and their ability to communicate to make them do a presentation it's pretty easy to determine whether or not they're bullshitting you just by having them talk through a subject um when I started teaching this class I definitely didn't know 100% of the

material I was probably at 50% I'm not going to lie about that um but I'm very efficient at teaching myself things and the ability to do the material I know I knew I could do about 75% of all of it um but I just had to figure it out first um no one can know everything your ability to do things is probably your your probably larger than your knowledge base um the human brain can only store so much information each day so about the instructor if you have a group of Industry professionals to teach this with you that's great um but there has to be a primary instructor and its main purpose is basically to guide the

class to organize all the topics make sure it flows well make sure people see the big picture so the set the the Sensei must be current um it's it's important to have uh presence in the community attend conferences um read daily security news and keep students involved in that um get them involved in basically the pace get them to have a a good perspective on the pace of the security Community um and another way to be current is to participate in CTS and there was a talk yesterday inur CTS way more resources he provided than just this one L to uh get in get into CDs so these are just some of all the books

that I use to teach this class I have probably six more um but another note is that I'm not a good public speaker um and I hadn't actually taught a class before and you don't have to be a great public speaker to teach a Hands-On you know fast-paced class like this um but you should have a teaching method that's appropriate for the aggressiveness of what you want to set out and do if you want to you know have an in-depth fastped class on exploit development and everything like that um and you planned everything out the Socratic method may not work although it's extremely useful teaching method and it helps people think things through if you need if you have a lot of

material to get through it may be better to have a lot of demos and to videotape them so they can go and see what's happening over and over and over till step so the setup for my class is I heavily rely on VM virtual machines um the university provides msdnaa is now called dreamspark I believe and you can get free versions of Windows through there and that's great for the students when I introduce them to Windows Concepts uh working with malare and windows and stuff like that um and uh there's all these wonderful dros out there backtrack is now just replaced by Cali and I haven't toed around with that um and then this is a

great textbook because it comes with live CD it just it's simple it has aslr disabled by default and if you try to explain how to disable aslr when you're starting the class students are just going to be like what are you talking about so it's nice to have a live CD with just disabled by default and it contains all the source code for the textbook and textbooks coding samples are packed with Vernal programs and exploit code and you can sit there and do them all day long and students really learned a lot from this and so my class I taught about 30 students and we only had we didn't have a lab I wish this class could have been

a lab um and so I had to fit all the material into 75 minute lectures twice a week and so what I did is I talked very very fast went through the material very very fast and I videotaped every lecture when possible um and a lot of people have asked me looking at my website how I did it um I use this software called Microsoft expression encoder I know free Microsoft product that works um because it works they are discontinuing it I think so uh I'm not sure how available that is anymore so uh I'm just going to pull up the class website this is the the the lecture slide website I have flash block

on right now but I have all the lecture videos on YouTube along with the required reading overview of the lecture uh uh the lecture slides any related exercises for that lecture along with any extra reading material so sometimes for like related resources I just point students to deathcon talks to get them exposed to the wonderful things that go on at death and when I was talking about uh SSL and the state of certificate authorities their required reading was Moxy Marlin spikes talk and the related reading was Moxy Marlin bike and Whitfield Diffy fireside chat you know Defcon with whiskey and everything and so um I wish myself that more professors more instructors would use these

wonderful resources these wonderful talks that are recorded videotaped and put online by the security community in the everyday classroom so that's what I try to do myself so the main tactics I use day to day tactics are short-term strategies and then I have long-term strategies I talk about next so I use frequent demos to reinforce what I'm talking about with the suits you can talk all day about hacking stuff but once you show them how to do it and show it right there in front of them they're like wow that's awesome it really sets in another useful thing is have a mold to get feedback um unfortunately I have like seven friends in my class and and they all tell me how it's

going some days they tell me you're really crushing the undergraduates I'm like well that's kind of good but I'll lay off but uh it also is kind of hard grading seven of your friends um because it's like man you should know that you're idiot another tactic I use is in order to give them perspective on real world security I usually spend like the five first five minutes of every lecture talking about some important recent development so right when AP1 report came out I talked about that um right when Java zero days come out I talk about those and the importance of you know watching CVS well yeah Java zero days it's like days since last Java

zero days usually less than two um so there was one lecture I was looking at the uh the cve list and the first four pages were all [Music] Java so another tactic that I use and so I have one student in the class right now and he's probably angry at me right now I made things do earlier than I expected them to do to be done this way when I gave extensions frequently they're like oh my God he's the best student he's the best teacher ever he cares about us no I'm just I'm manipulating you and it actually works well so the other thing that I got a lot of great feedback on is Hands-On

workshops setting these up were hard because I didn't have a lab I didn't have you know 30 computers that I could have students do their own thing I had to actually uh work with them weeks in advance to get them to have laptops for this week and have you know everything set up all the virtual set up right and what we did is week three we had a guest lecture come in and we dive right into the deep end of x86 reverse engineering like Hands-On looking at binaries in Ida and everything like that um and so this was one of the most successful weeks and if you have the opportunity to do a lab there are tons of resources out there

that you don't have to that you can use so you don't have to put it together a lab every single time by yourself I strongly urge reusing Labs from existing resources practical malare analysis has tons of great Labs it's wonderful book it's un it's widely regarded as the def facto standard on malare reversing and reverse engineering um there's Labs from pentest crypticity Donnet itself there's another good uh resource here uh open security training. info the guy that teaches that is strictly Windows side um and he has some great lectures on the entire life of a Windows binary um also uh cor land. be is wonderful and if you go through one the expoy tutorials that would make a great lab in

itself um and so also all the these traditional level based CTF exercises also make great Labs smash the stack I would love to have seen that in a a lab setting when I was a student um taking classes Expo exercises is also is much newer they have uh three uh exercise virtual machines uh nebula proar and fusion I believe and these are great resources to save the instructor a lot of time and they're all set up basically with instructions on how to solve them um

so I guess this should be more strategy but you have to determine whether or not you want to have a term project and if you want it to be individual work only I am a strong uh proponent of individual work only term projects because I've been way too many team projects and always had to deal with The Free Loaders um and so individual work is a good way to get rid of the Slackers and to also make everyone work on challenging stuff um and prove their own skill and so I went with open-ended term projects and allowed them to work on projects that would help the industry would produce tools and plugins like plugins to volatility that will take a

take the output from proc M dump and actually rebuild the IAT table so you can import it into Ida without going through uh inre or inre light if you can actually ever get that to work and saves anyone doing instant response a lot of time um you can directly pump uh any thing from Mal findind to Ida and all the all the symbols are there and uh so that that was a great result from the class um the class is still going on semester is not done we're in the middle of student presentations week two weeks from now is the final um so I don't have a lot of the other results ready to brag about

yet um but if you do open-ended you have to provide sample projects like that you can't just say go solve security 10 weeks Chop Chop uh it's not going to work um and I found that with open-ended projects you really have to provide detailed Rubiks so students know how to get the a and so I provided that all on my website and you're free to copy it I've already talked about presentations distinguishing the bsers from the students who know their stuff and so the long-term strategy is I wanted to cover a lot of advanced topics and I saw the best way to do it is just to Blitz cre through the material um so week one and two was kind of like

fluffy overview of Windows and Linux and root kiss and code auditing and then this difficulty of class is pretty much like this and I warned them that it was going to be extremely difficult very soon and I could see the eyes glaze over on some of the undergraduates very quickly and um but they stuck through it and actually some of the some of the undergraduates who aren't even computer science like criminology they're like this class is so difficult but I love it so much which I I really liked hearing and so when you're going through material aggressively it's very important to remember to come up for air cover the big picture because I frequently offer them like five point

bonus on their homework how do you feel about the class so far what do you want understand well what don't you understand and I commonly got I don't understand the big picture of how this all ties together and I appreciate that I can't count the number of times I've been in a class I'm like okay why am I learning this why is this important and the teacher never explains it unless they're specifically asked another strategy is because I wanted to teach a lot I continued teaching them on assignments on the homeworks I gave them something new that I didn't cover in class basically 20% of every homework was making them do new stuff but applying skills and techniques

and fundamentals that I uh explained in class and then lastly one of the most important strategies is to keep them inspired if you keep them inspired they will solve that knowledge Gap problem on their own they'll go out and do the research they'll do the Googling um and they'll come to class having done the reading it's like wow I I can't remember the last time I was in class and you know everyone had done the reading so the campaign um I I play D andd on the dungeon master not by choice because no one else wanted to be it and I was the only one capable and creative enough and so for the basically flow of the class I

always uh made sure to every now and then discuss ethics bring up the disclosure debate um also aside from making sure all the topics flow well and making sure people understand the big picture provide anyone who wants to learn more about subject with external tutorials extra help homework solutions and uh I had to coordinate the guest lectures and the travel which is more complicated than it should be and what the goals of the guest lecturers are and what their expectations are um you have to make sure you have a clear understanding of any guest lecturer's expectations of the students and uh what they want to teach in order to uh be very successful so in 15 weeks

I delivered uh with help 23 high content fast-paced lectures all of them are videotaped almost and they're all on YouTube right now I had actually was worried about if I was videotaping everything and providing them to the students right away if people would just stop sh off class I had a very very High attendance rate the entire semester which actually I was a little bit surprised by um all almost all the homeworks are CTF style and uh I had two homeworks and I painstakingly took noobs and made them leave and uh like I said I didn't teach them the tools till the very end so I didn't make any script kitties at least we'll find out by the end when I grade

everything um so the advanced topics that I set out to teach were Rob um memory analysis of running exploits um and understanding how things work so understanding why things are why payloads are polymorphic and once they're actually running what they actually look like after they decode and stuff like that um we were going to have a UAV drone hacking Workshop uh but the parts never came in from China I think someone in China saw my lecture on cyber warfare and decided to cancel the shipment I'm not sure and we had a a workshop on lockpicking and so yes I taught lockpicking in a university class um and so grading was pretty straightforward homeworks were one of

the biggest things about the uh the grade makeup and I really didn't care about the lateness as long as they had the work turned in before uh I presented the solutions and that's because you know attackers really don't have deadlines bad guys breaking at the weirdest times and so the results from the campaign were actually pretty great students have frequently been asking for an offensive security 2 class or Advanced EXP ation and some of my students have uh that are graduating have jobs at really high-profile incident response teams which I was pretty happy to have a part in influencing um the students are now totally excited about Devcon and uh well some more were going to be here at

bsides but I think they're being crushed by this uh project in be in the semester so I used a lot of CTF questions I actually reuse a zero day against students I found a zero day in a CTF it may have been an end day that they disclosed the vendors like I'm good crap um it was anti- uh reversing anti- debugging zero day that if you loaded the binary up in GDB or Ida they would crash right away so I'm like here's the problem get the key just complete CPS style and so they had to Binary patch it out and it was a great way uh to expose students to these topics because they can put on the

resume that I understand what zero days are I had to deal with one um so students actually love the homework I had a student that uh she said she spent eight hours straight in the library and she loved every minute of it I'm not sure if she's sucking up but she seems to generally me that um so blah blah blah blah blah I did too much work main lesson here is get a TA um other results is I hear weird rumors about myself like I can wear sandals on the moon um and now I have a big stick to beat up the other faculty um because I definitely have something to say how how they can teach stuff a

little better um so the overall lessons from this is if you think you can do this and you're inclined to do this I say totally go for it follow Dan Guido's outlines follow mine my lessons learned um it's hugely rewarding I have guaranteed jobs for life just teaching this uh and get a TA de grade find guest lecturers to teach your weak areas use existing material steal from me is all all Creative Commons I just want credit if you reuse it take it build upon it it's out there uh it's out there because I understand the best way to be successful is to get back to the community and so that's what I was trying to do here the other lesson is

keep the bar high and Inspire the students and they will surprise you they will fill that knowledge Gap and they will solve problems uh on their own and lastly you will actually if you do this you will learn a lot from your students and from teaching if you haven't taught before um last last big things is I strongly urge you to videotape your lectures and a lot of people are cautious about this because they fear immortalizing their mistakes um if you preface something you're un sure about with I not sure about this but I think it works like that that's a fine way to go about being wrong and if you correct it by the next

class you're fine and I make a a lot of mistakes I teach taught this all on my own I didn't have you know five professionals doing it um so if you vide tape it really helps to put them online quickly students universally said that videotapes were extremely useful for doing the homeworks um it allows you to teach a much faster paced agenda um and so anyways at the end of it all really I just need some sleep and so it was fun teaching this class and if you if you want to learn these subjects um you can sit at home on a weekend and watch the videos give the reading do the homework and that's any

questions no all right thanks oh you got a question what's your question I just wanted to tell you that I have something you what you have something she hacked Sheed she hacked something what you did you hack the Gibson the Gibson that's [Applause] awesome said about not being able to do everything and you hey everyone I just got a quick announcement um our speaker over in the lemon room has gone aall so talking here like9 sorry

was