Whose Idea Was That? Comparing Security Curriculum

[Music] alright ladies and gentlemen we're gonna start a little bit quickly because the last talk went over my a little bit so let us just jump straight into it good morning there we go that's the cutters boss I went here for curriculum talk so today we're gonna be presenting whose idea was that and this is gonna be an overview a comparative overview of security curriculums and also accreditation is compared to industry needs my name is Tim Sanders for those of you who don't know me and that my co-presenter over here is Rob Olsen and this isn't even our final form yet yeah as we keep going we're gonna talk to you today about computing security education

something that really enthralls and invigorates most people now the first question you should have here although many of you know us is why are you guys at all qualified to be talking about this type of stuff so both of us are professors here at RIT if you're not quite sure where you are right now this is the Rochester Institute of Technology and interestingly enough we both kind of socially engineered our way into computing security education Rob came originally from computer science and I came from industry this is a little bit of a strange situation because they've pretty much given us the keys to the kingdom here where two of us sit on the four person computer computing security

curriculum department committee and what we do on this committee is we steer curriculum this is pretty much a self-explanatory name they came up with it it's pretty good now we represent two of the people the other two are Bill stackable who chairs the committee and then Alan Kaminski who is the computing science representative on the committee and this is gonna be kind of an interesting situation now the one thing that we want to note is that the only Association we have is with RIT it means that we're not associated with things that we'll be talking about like a bet or ACM or I Triple E or the NSA or CIA or the DoD any of these things we're not

really associated with and we're gonna continue to talk about them in spite of that so we had a couple of questions that we came up with while we were doing this research they pretty straightforward the first is probably the most obvious why do we care about computing security education anyway why does this matter to us we have some ideas on that the second and third come kind of go together if you were to start a program where do you put computing security where does it go does it go directly into computing science does it have its own degree and are you even going to teach it right you could be a smaller school and say this

is too much if you do teach it what topics are you gonna cover what how do you know what you're gonna teach and who who dictates that does anyone dictate that are there standards here that seems like a pretty good question and of course the last question is kind of the most interesting question based on what people are teaching can we say something about the jobs that those students are best equipped for and are assertions going to be yes essentially if we were to tell the artists the question is what was the last time you've thought about who's behind a computing security curriculum and what does that mean to you is NaN let's say students or at least

non professors from the most part oh well even for students right yeah because the kind of program you're in is really going to sort of influence where you had an industry so the first question we have is why do we care about security and and we have this kind of like classic notion which is just like there's this huge law of computing security experts for the amount of jobs we need and interestingly enough we're gonna see that there's like no data to back that up there's just like this assertion which is kind of cool so this data is from the Department of Bureau of Labor and Statistics and you should note that this is a probably not

going to be right it says there are 90,000 information security analysts now this is probably due to the fact that this is a very specific job title right this doesn't include anything else the definition of security is a little bit tenuous we have no clear definition as hey is that system administrator also doing security we don't know so this becomes kind of an interesting data the other interesting data we have here is obviously that it's growing regardless of what data you see we're gonna see that there are more jobs to be had so there's kind of some questions you have here and the questions go around the fact that if there's 90,000 people who are doing computing

that seems low right they can't possibly be right that would mean that one out of every four point-five people in this room attended DEFCON last year that's probably not right and that's not even accounting for people who came from outside of the US they went to DEFCON then 20,000 people who are in the only attend so are there other numbers sources and it turns out yes there are but they're extremely sporadic at best so we have for instance the currently employed us it goes between 780 thousand and 1.7 million now that's a pretty large discrepancy there and including when we took it take a look at currently employed we have between four million globally and six million little blade

that's almost as large as the discrepancy of the amount of illegal voters as Donald Trump said that there was in the u.s. election yeah yeah cool so we have a kind of a question here clearly there's a lot of jobs right we don't really know how many jobs but can we make some assumptions about that and if we can does it matter how many students we have so then we have to have a question about how many students there are no data available for that however since we work at RIT we at least have our eyeties data probably the most troubling fact that you'll run into is that academia doesn't publish any of this data zero not so you don't know how

many students there are how many programs there are even in computing security and we're gonna talk about accreditation and whether or not that exists so we can definitely say that there's one thing that we have going for security academia tends to do what we call market focused education and that means in other words it seems like those people are going to be able to get jobs afterwards and if they can get jobs afterwards I can put that statistic on my website so we should definitely do this which means that there's a lot of computing security departments now we've highlighted some here but the problem is we have no basis for this right there's no comparative criteria for the quality

of computing security programs these are just people who did well in CCDC which is not even really a particularly effective metric of compute security so we have this kind of interesting question where this data comes from who's the most reliable and how many students there are so what we can do is we can take a top data point right we can put a boundary on this and we can say it is unrealistic that every school has let's say 120 people per year graduating with a computing security degree I think we could all agree about that if we did do that we have this nice equivalent so we say three out of four people which actually is much higher

than I anticipate graduate every year if they've gone in so one three-fourths of people who go into a program finish it that's a pretty good rate that seems really nice so we have take that number and we assume that there are 137 schools this is kind of the minimum number that we can get here this is from the CAE CD we're going to be talking about what this alphabet soup means in a little bit soon so if we multiply that all together we get that there are 90 students per year who are going to be going to 137 different schools at most and that's gonna give us at most twelve thousand two hundred twelve thousand three

hundred thirty which is a wholly unrealistic number but it's an upper bound that seems pretty good based on our numbers though how many students do we need well we have this kind of interesting case the lowest number we had of total jobs right now in the US was one million one hundred and twenty eight thousand three hundred seventy seven very specific number so if we assume that people work for 40 years give or take you know who knows we can divide this by 40 right and then we could get the current amount of people that you're going to need if you started today to fill all those jobs which essentially means although it's not a

really realistic number it's probably front-loaded there's probably more security analyst positions than there are CIS OS we can use this information if we took this extremely liberal approach this would mean that we need 28,000 209 students per year to be graduating all right so we're clearly not fulfilling this goal even with this lowball number unrealistic class size and the lowest estimate of computing security experts in the field that's kind of nice in fact we can verify that this is some of the case because is si square it only has 73,000 members in the u.s. so we can kind of say that we're probably pushing at least a good amount of these per year through this this is maybe right but we

don't really have any data to prove this this is the best we have these estimates come from various different sources so the ones that you saw here right here come from these different sources they have different criteria for their inclusion but almost all of them do you include the DoD as part of their kind of yeah lifestyle but not the US military that's the largest employer on planet earth at 5 million people cool so we went through all this data which was really nice and then we kind of came up to this how do we evaluate these security problems how do we evaluate our security curriculums well this is actually an interesting situation we have kind of the academic approach and

our one of our arguments is that this is a major driver of we're computing security workers come from is academia I don't think this is a crazy argument to make we could certainly say there are other sources no one's disagreeing with that training on the job certifications maybe military training all of these factored in but we're saying that this is a large factor and if it's such a large factor we should really really focus on those 12,000 or so that we actually made how do we assure ourselves that we give them the best security education we have because we're clearly already short on the number of students we have so with that in mind we're going

to jump into each of these sections where we would fit computing side computing security in computer science information technology information system we're also going to discuss some of the current trends within academia accreditations now this is to say that generally majors have an accrediting body someone who says this is the minimum curriculum that you're going to need in order to offer this course and get our frugal sometimes people even ask like we'll come in and say who accredits you in computing security there is no accrediting body at this point we're going to discuss the possible contenders for this location or this title and and then we're going to go into some proposed solutions and our

recommendations on the matter so it's also worth thinking about how this accreditation process goes so in terms of the accreditation where does the accreditation actually get its information from to accredit people on well as things stand right now that's coming from the largely from the ACM curricula so the ACM the Association for Computing Machinery publishes these curriculum guidelines and abet I don't recall what that stands for off the top my head comes in and says yes you're meeting those roughly with a little twist here and there or no you're not meeting those so a bit of credits now we're going to talk about NSA NSA is kind of playing in the same space a little bit there not been accreditation

during designation which has some different some implications for colleges because you can't technically say your program is accredited it's designated instead so yeah so let's jump right into this by taking a look at a cm curriculum guidelines so for a cm curriculum guidelines in computer science this is what we end up with now this is ranked in credit hours so we have two kinds of credit hours for I'm sorry not in credit hours in instructional hours we have Tier one instructional hours and tier two instructional hours Tier one instructional hours labeled T one are mandatory hours you have to you have to be in a room with a professor either doing either in lecture a lab or some

other similar setting for these Tier one hours here tier two hours are hours that you should ideally have but ACM doesn't actually expect you to meet those okay so some relatively high percentage 70 to 80 percent ballpark is kind of what's expected there notice this makes sense for computer science so software development fundamentals is actually pretty high up on the list some things that don't make us so much sense for security which is a place that will see a sea where so we'll see that the sea s is a place where security often ends up three instructional hours okay so that's three lectures it from the ACMs perspective in order to be a computer science person you have to have

three lectures in college on security three hour-long lectures okay ideally you'll have a total of nine hour-long lectures but that that's sort of it in fact if we take a look at where this ranks overall this is one of the lowest knowledge areas for ACM computer science standards okay and here we have a breakdown of those hours so this is largely geared towards developing secure software now if we think about it as security people all of the things that we need to know about developing secure software you know everything about all of us everything about how web applications stuff gets exploited everything involving buffer oh but for overflow exploitation we're supposed to cover that realistically in two hours

okay then we also have a little a few extra things thrown in you should know some knowledge of each have some knowledge of threats and attacks that's one optional hour you're supposed to learn everything you need to know about crypto in one hour yeah yeah right so they can't possibly be right okay so to put this from a perspective you know to sort of toss our experience as instructors into the mix a little bit so we have as I said this is measured an instructional hours that means we're sitting in a room with people for about well so we're in a room when we teach a class for about 45 hours a semester with students okay from

security we're saying that there are so this usually has up to three instructional hours for 15 weeks some CS topics we kind of hit that right so the software development fundamentals gets kids kind of close to that at 43 hours we already sort of Jareau from this point about about three to nine hours on security which means really three to nine one-hour lectures that's all you get as a computer science student of security in your curriculum that means one to three weeks and one to three assignments maybe one project now in terms of how this ends up being translated most CS programs to satisfy these requirements have a course to kayuu mapping so when you have something like software

development fundamentals okay we'll say that's 43 hours well that's almost an entire semester so we'll say that's going to be a course okay so when we have mandatory hot half knowledge units with a high number of required credit hours that ends up being a mandatory course okay so software development fundamentals that's your computer science one class okay or your computer science to class or it might be split up a little bit okay so those end up as required courses when we have low qantarah low number of credit hours or apps are a low number required our instructional hours those end up as elective courses okay so you might have a security class as an elective that you won't be required to

take and it doesn't have to be too intense because they're not ex-ac em isn't expecting you to come out knowing that much about security anyway this gets a little bit interesting when we actually look at some other places so in terms of how we break this down okay how we actually break this down we do see so that what we have here are the security are the content of a suggested security curriculum now notice we're this kind of lives right in terms of instructional hours per semester for for an elective security class it's very heavily geared towards application security in blue okay what we see less of is IT security in purple so we end up with about maybe nine hours

about nine hours or so there eight hours in in IT security and much more the semester ends up being in Webster in application security okay now I teased a little bit different so ACM has a separate designation for program for IT degrees they have a separate curriculum recommendation for item it's actually very different right one thing that's nice to see is that information security and assurance is has a fairly high number of required instructional hours all right so this is a positive thing there's also some security security adjacent topics here in a grade of programming there's usually a lot of there's usually a lot of security covered in that since as we all know this is one area where things where a

lot of security problems arise social professional issues that's also a place where security and security adjacent topics get talked about a lot that's your ethics your societal implications of technology etc lots of those are security focus these days so on all this seems like a much more security focused curriculum okay and if we take a look at a vis curriculum so we hit what we have here our marked our classes that are marked where there is some security content in the class now they have for the is curriculum it doesn't break it doesn't break the curriculum down into into the number of required and optional instructional hours in the same way that CS and IT have but it does we do we can

look at syllabi and figure out how much time in these courses is supposed to be dedicated roughly speaking how much time in these courses is supposed to be dedicated to security now we see security in a lot of places in the syllabi which is nice because it does mean there's gonna be a lot more reinforcements okay we don't necessarily know quite how much time each one of these is going to cover we would have to make some s some guesses but this is nice a nice reinforcement of security even though it's a non technical security or less technical security I should say because it's an Information Systems degree you're gonna have less hands-on more things like auditing more

larger discussions of policy that sort of thing so what we actually see here I've got a couple of sample courses so we have IT security this very much tracks what we would expect in certain introductory security class we also have a second class here IT audit and controls that hits a lot of the same notes so again lots of reinforcement going on and for someone who's going to be in a policy position or a non technical analyst position this makes sense right but in terms of how EBIT actually does its accreditation so this is where we see a little bit of a disconnect so a bet is a bet there's a loose association between a bet and a CEO okay

in that a bet as I said they kind of they kind of follow the ACM curriculum most of the time in their accreditation guidelines you'll see a lot of the concepts are very very similar to what to what ACM to what the ACM recommendations were notice that in the computer science section of the abet requirements there's no security security is not mentioned students never have to take a security class and these are people who are going to go out and become developers right why do we see I mean if we take a look some of the substitutes recently been going on why do we see the same bugs coming up over and over again this is how we get bugs

in terms of their actual learning outcomes so the way that a vet works is they have their curriculum guidelines and those are supposed to map towards learning outcomes things that students are supposed to be able to do when they graduate so here we have a general abet learning outcome now this applies to all abet programs okay every single abet program has this particular one this particular learning outcome which does mention security okay but you have to have an understanding of security all right that's very broad okay and that understanding of security means something that's a mean something very different to an is person though it does to a CS person or an IT person okay if

we look at what else CS has to do mathematics algorithms etc finance degree and then also you have to be able to apply it you have to be able to apply design and development principles so it would be nice if this has if there was some learning outcome here related security for CS but what this is really saying is that any pet doesn't actually care if a CS person can apply security in any way and in terms of learning outcomes these often pop up for colleges in what we call assessment and that's how a program manages measures its success or failure okay periodically we have to go through and see you know look at how students are performing on

particular assignments particular tests etc and those assignments or tests map to these learning outcomes for C us there is no learning outcome for security which means no one's assessing it which time is back to the point I made earlier maybe this is why we don't have any data on it for IT oh this is a little bit better okay so here we have the the actual required curriculum for an IT degree for abet to accredit the there has to be some coverage of information information assurance or security all right so this is good all right this is probably a class an IT assurance IT security something like that okay so IT definitely a step above

see us and has prioritizing security at least at least the courses they have to take include a security class if we look at their learning outcomes tall so we have again something about applying current technology okay we have something about analyzing user needs which made sense if you're gonna be an IT you should be able to think from a user's perspective we have one about integrating ok we do have one requiring sort of discussing best practices and standards if we're gonna measure security if we're gonna have a learning outcome the measured security this one probably makes the most sense but at the same time it is tangentially related to security it doesn't explicitly say that

students coming out out of an IT degree have to be able to do security instead they just have have to do best practice hopefully best practices are connected to security but that's not always the case and then assisted increase creation of project - a little bit of project management there is again is like IT references and explicit references security is a specific area that we see things covered ok however it is it's one of many and certainly an is degree is going to be less technical than a CS degree or an IT degree ok so here we have a discuss each discussion to policy discussions of auditing compliance issues arise those types of issues which

is not bad it is what we're preparing you know it is a different mindset ok but that's not a problem I mean that's that's an important part to a functioning security security ecosystem in a company as much as some of us might not be might not find that quite as interesting and then when we look at the learning outcomes for those again no mention of security right so we're expecting students to take security to take a class to cover security content but we're never measuring it in any way when we actually look at who does what okay when we actually look at who does what the breakdown of who is covering security in different places we see that

this is sort of interesting most students are getting security in CS right now we're tracking this by looking at the numbers for NSA designations largely okay and for schools in CI a-- and for schools competing NC cdc but most of the schools providing teeka either getting NSA designated or going to these kinds of competitions are coming out of computer science the place where we just said no one has to ever take security really okay that's an interesting trend okay some of the places where we would expect to see it if we take a look at IT you know IT is actually fairly low on this list it's similarly CSEC it's nice to see CSEC is a big chunk of the pie but

there's some weird results too Computer Engineering we don't necessarily expect to see computer engineering as producing a large portion of security security people but that's actually what we do see when we look at who's getting who's getting these designations

yeah so again this kind of drives home if we're thinking about who's getting security and where they're getting it it looks like most people are trying to get their seat their computing science programs to be their security programs which kind of again drives on this point of what are these good at people going to do out in the field when they actually have to be doing security so so in the past slide we actually just saw breakdown that included all the si CAE all the NSA Co all the NSA CR and all the CCDC schools that's a lot of letters what are those letters mean that's the real thing that we care about today so

one of the most interesting things in the most widely-used accreditation is actually this NSA designation and it is an NSA designation for a variety of different potential areas so when we start talking about CAC D this is the most common one there are a hundred and forty schools that have a designation of CAE C D in its previous form not the current gratified form but the form that almost all of these schools have that requires that there is a program that can be made from the entire institution that fulfills the requirements that we're going to talk about in a couple seconds really at its previous form it's an extremely high level security program there's no

requirement that people are able to even physically take these classes just that they exist in the current 2016 iteration of this you have to be able to actually take this we're gonna talk about how that might be done however as you'll see the topics are still fairly high level see a CEO is the newest accreditation that you could possibly office our newest designation that you could possibly get from the NSA and this is actually heavily focused at least in our opinion which is why the stars are there on state-sponsored offense and we'll see why we have that feeling as we go forward the last one is the CA er and interestingly here this has no

correlation to any academic classes this is all about whether or not you can actually publish research you have a research institution that has PhDs so on and so forth now there are times that you'll overlap these so for instance 39 institutions have both a CAE CD and a CAE our there's a hundred and seventy institutions that have at least some of one of these but more the most popular one by far is DCA ECB and there's only 67 that has the CA are and there's only 13 that have a CEO right now that's going to be really interesting when we started talking about what these curriculum requirements are so they have some specific needs in

order to take some ku's like we saw before knowledge units but for the most part you can either be a two-year designation or four-year we're only going to consider the four-year for right now and there's the the four-year has the requirement of both the two-year schools and the four-year schools additional with some additional knowledge units so what we have here are the requirements for for the CAA CD ok these are requirements required ology and it's not optional knowledge it's for the four-year five for the two-year program they have to take everything on sort of the left side of this graph the things that we would expect to see data analysis and scripting some networking on the right side we see the

differentiation between a two-year a two-year program that's accredited with CD all rights are designated with CD and a 4-year program that's designated with CD more focus on defense more of a focus on theory including operating systems which we're going to come back to probability databases weirdly I guess that doesn't hurt but it's a little bit of a strange choice now these are the required knowledge units everybody who graduates for in the current iteration everybody who graduates from ACE CA a CD program has to has to hit these marks okay that was not previously the case previously these knowledge units had to be offered somewhere in the institution okay which means that in computer science class that your IT stood or a

computer science class and operating systems that your IT students could never even take because they didn't meet the prerequisites for would count towards the previous iterations of CA a CD it's a strange view right you you can have this class and that's all that matters actually care if any students ever take it now in thinking about CA ACD and what it's geared for too we also have to look at the knowledge out of the optional audience because these required knowledge units give you a nice baseline okay the optional analogy ins for this are crazy broad okay and now students graduating with a CAE CD have to be able to take any five of these okay that is the school has to

have five of these somewhere in their curriculum okay that fits every basically every program out there we see things like fraud prevention that might make sense for an is program we see things like software security analysis secure secure programming OS hardening operating systems theory they make more sense for computer science program and certainly IT we even see things in here such as Hardware firmware security that make more sense for computer engineering program we start to see why there might be more computer engineering okay and when we move into CAE CD when we actually look at what their learning outcomes are remember that these are the things that are supposed to be measured there's nothing too crazy for C a CD

okay here we have the requirements for it in the required learning outcomes for an entry for an intro to cryptography class that an undergraduate student is supposed to be reasonably as supposed to be able to take this is their takeaways from that intro intro to crypto class so for example students will be able to describe which cryptographic protocols killzone techniques are appropriate makes sense that's not crazy seems like something a for your students should be able to do okay and then yeah that's this time so the newest member of the NSA designation team is the CAE co which stands for cyber operations if we recall now these are some interesting requirements here so there are ten

required K use within this space and at least some of them seem very highly geared towards shall we call them government in the for instance how many of you needed to learn about the Hager Hager Geneva Conventions to get your job in security off the ground probably not many of you actually you know engage in war level efforts there's also something interesting missing from this core group does anyone see where the ethics is seems in a purpose seems maybe an appropriate choice for for a government level agency perhaps now there are some optional ones as well and the interesting point about the optional ones is they're very specific to things that you might not necessarily need now

some of you who are in the professional world like to reverse engineering of hardware on a day to day basis but for the large majority of people who are going to leave school you probably aren't going to need to be able to do low-level microcontroller design as part of your day to day now this kind of drives an interesting conversation and a theory about why some of these might be in here it is my opinion although it's not necessarily one that can be condemned by everyone that some of these courses were put in here for specific schools to be able to achieve this accreditation this is an extremely strict definition of what these things are as my dear friend Rob will tell you

and a couple second and just also drive home for that for that last slide schools have to be able to get 10 of those 17 areas okay so learning outcomes for CAE Ciel there are some things that make sense right so if we think if we think back to the crypto one that we saw a minute ago that makes sense for a two-year or a four-year undergraduate student okay if we look at what CAE CEO has we have some things that also make sense for a four-year student an understanding of an understanding of operating systems okay a thorough understanding of our operating systems work the theory behind them etc students should be able to implement significant architectural

changes to an existing OS okay I guess I guess that's make sense that's not a bad thing for students to know making architectural changes I'm not sure how many people do that on a day to day basis but they should come out of college and knowing how to do it that if they needed to if they got into that position they should be capable okay so this makes sense I think as a learning outcome for the operating systems class although it does make this feel more CSE if you will or Computer Sciences there are some that are a little a little strange a little specific ok so here we have one of the learning outcomes for the mobile

knowledge unit now the requirement where the students know how to its analogy and its cellular and mobile technologies so students must basically the students need to be able to trace a pack end-to-end you know from start to finish in a mobile network ok there's a lot of protocols in there that many people aren't going to be using on a day to day basis particularly if we think about mobile integrates with plain old telephone system ok but this certainly wouldn't be bad for students to know a student who could do this should be pretty read be pretty good to move into a networking position ok and then there are some for lack of a better word that

are batshit crazy so here we have the assembly language and low-level programming iWork knowledge unit so if we read through this what it says students need to be able to implement a standalone program a standalone networking program such as telnet in assembly and hold on wait for it no external libraries ok a four-year student needs to be able to write telnet in assembly with no libraries that means no drivers no windows utilities none of that who would do this who would need to do this on the day to day basic on a day to day basic basis except ta oh right what security analysts needs to do this on a day to day basis as a part of their

regular job particularly at an intro level right this this only makes sense for NSA so when we look at how things are changing ok when we look at how things are changing so NSA is kind of the dominant player in the accreditation space even though they don't actually offer accreditation they offer designations which is a little different when you talk to Dean's and price presidents and so on and so forth okay a cm and a bed and our Triple E are kind of looking to move into this space so they started this thing called the computers cybersecurity engineering project to look into the problem it's formed by a group of people multiple institutions and ACM is basically using

this group's work to create a joint task force for studying cybersecurity education which is about the most academic thing you can say okay we're using your work to create a joint task force to analyze whether or not we need to do more work what this is going to do is it's going to end up being the basis of this of what's probably going to be an abet accreditation for cybersecurity called C 2nd 2017 this by the way is the ACM named for that's not the abet name for this we don't know what the abet name for this is yet a bet is has their computer accreditation committee looking into this they're basically going to end

up implementing implementing probably implementing the CSEC 2017 that we'll talk about in a minute but they are going back in revising the previous that the stuff that we previously showed you so they're in the process of revising those accreditation our requirements for CSIT and is interestingly also yeah so interestingly the engineering accreditation committee is looking into this as well but they're not publishing any of their work so the engineering the the group that at that the group with an event that certifies engineering programs is also trying to move into the cyber security space but their stuffs not available yet so it'll be interesting to see how this goes there's one reference to one reference in 1/2

paper or two-page presentation description that says EA C is also looking into this so the avorite revisions so largely speaking what we see is that EBIT is actually pulling security out even in our town okay so the within CS for example so within leech within the general curriculum requirements we don't see that it references security anymore but we do say

you

that it seems like we don't have a very good representation of yet additionally we're seeing accreditation finally move more towards or we at least assume that it is moving more towards industry based offense which is kind of a nice area that we've been lacking our recommendations for their space are kind of very interesting in that sense so first off there's no metrics anywhere that would be a really nice thing to make available so that we can know how we're doing or whether or not these are effective or you know anything at all that would be cool certifications should be used now when we say certifications this is an academic term and it can get a

certificate in a particular degree and in that space we say things like I have a certificate in information assurance this will become important for the CAE CD now that you can't just have it anywhere in the institution what they're gonna say is this is the path that you can take in order to get this certificate probably no one will still take it or very few people will but they'll at least have this capability as a result of this will probably also see a slight shrinking in the amount of CA ACB accredited schools I would assume interestingly there's a couple other options here that are kind of nice when we start talking about CAE C D probably

with the new requirements RIT and RI are CSEC are probably the only curriculars that will actually have the requirements so for instance if you're mandatorily needing to take a system administration class and a networking class CS is probably not going to cut it for you that's not usually in CS curricula now the other kind of caveat here is this master's versus minors business the minor makes a lot of sense however there's a problem with a minor and why we see it's so little minors need to go through state certifications don't colleges can just do whatever they want and just say it's a certification and it's fantastic now we see a lot of masters actually and there's a really

good reason why we see a lot of masters or at least a really good conjecture first off masters are fantastic because you can just say we have a master's program come see us that's great another aspect that many of you may or may not be familiar with is that masters cost more money and as a result schools really like masters that's a really ideal opportunity for the they also don't require you to have as many prereqs an across-the-board general education and they only take two years or a year depending on your situation it's a really great solution at least from the school's perspective and I think we'll see it more regardless of whether or not it's the best solution

for the student now when we still start talking about the NSA CA Co it actually based on its really deep dive into operating system concepts and some of the other concepts we see best fits a computer science or computer engineering degree and it almost certainly doesn't fit in information sciences degree now yesterday and of course we we kind of see this business where we see kind of often slowly picking up but we are going to predict that just because the NSA CAA CEO doesn't necessarily meet the accreditation needs or the needs of industry that we'll see at least some other type of accreditation or a kind of a designation emerge so with that in mind I think we're going to transition

to some questions if we have a couple minutes which we have maybe a few minutes are there any questions

so let me speak to that one so usually see that kind of innovation done in the form of elective classes or special topics that's largely where that comes in now sometimes those end up becoming permanent classes sometimes they don't so usually this starts off as a special 5x class if it runs well for a couple of semesters it will turn into a permanent class that probably will be an elective unless unless it gets rolled into one of these accrediting accreditation guidelines and of course we've seen non accredited programs information security is a great example so if someone feels like they can make money off of it the college they'll pull it out and make it

a non accredited program and it really has no bearing on the real world whether or not it's accredited yeah it may be a question that someone asks the available house he might seek an SI designation but so right now I would go with the NSA designation and the only reason I would do that is because that's what everybody has additionally the accreditation aren't currently available and even when they are available I highly doubt that people are going to go that route because it costs additional money and there's one other really good reason to go for that too is that it opens up grants for the college to apply to there's actually a positive versus just

getting mine to get yeah final really for a bet stuff defeat your market of course people say you're a bet accredited are the parents of students and when doing open houses I have never had a single person ask me if a program is a both accredited I've heard it once and the answer was well we're NSA doesn't need it they're like alright that's good yeah in fact that probably has higher higher recognize recognition yeah any other questions excellent well thank you guys very much

[Applause] [Music]