BSidesCharm - 2018 - Nolan Hedglin - Counting Down to Skynet

wow

all right ladies and gentlemen thank you so much for coming uh today i'm going to be talking about terminator no i'm just kidding today i'm going to be talking about predicting the threats of the future in 10 years using a new form of analysis that i've done in order to bring some sort of objective lens into that idea so to begin with i want us to all kind of put ourselves in the context of 2028 we're in tibet now you are sick you are a citizen of a small town in tibet and you need to go to the doctor the issue is there's only one doctor in the entire town so the doctor has to

rely heavily on artificial intelligence in order to diagnose and recommend medication for a lot of their patients so they continue to get everyone through now something goes wrong the ai that actually recommends this medication is recommending the wrong stuff and it's not only actively not alleviating the problem that you have but it's also making it worse now the question you're asking yourself is well who's responsible for this how could we even predict something like this from happening and that's the question that we try to answer so to begin with i'll talk a little bit about myself i'm currently cadet at the united states military academy where i major in math whose year-long thesis you're about to

see and physics after the academy i will spend two years at mit studying quantum information science afterwards i will also branch into army cyber uh where i will hopefully conduct further analysis on how we can do threat prediction and strategic foresight for the military as you can see here i like competing in cyber policy competitions put on by the atlantic council and also i like to play rugby uh this is i think two years ago we beat navy doesn't matter we've beaten navy every single year for the past four years yeah the real streaks right here honestly so um let's begin today i'm going to talk to you about something called threat casting this was developed in

conjunction with the army severance 2 and aci then i'll also get the idea of how we can use threat casting for strategic foresight and conduct further objective analysis on it and then we'll go through an example and i'll predict i'll present what future work could look like for this and take any questions you guys might have so to begin with as you all know technology can be abused for nefarious purposes much like geeks like yourselves and current methods of assessing what those abuse methods look like right now relies heavily on social science expertise and subject matter expertise in the field of cyber security now my issue with this is right as you can see from the bottom line

there's a gap in the quantitative and qualitative analysis for strategic foresight so the threat casting lab was developed in conjunction with arizona state university's future of innovation and society as well as the army cyber institute headquartered at west point and as you can see from their mission statements to serve as the premier resource for strategic insight teaching materials and exceptional subject matter expertise to envision possible threats 10 years in the future so right now they're looking at specifically 2028 and what those futures look like in this case what the threat crashing process looks like is this what they do is they look at social technical and uh specific trends within society in order to form data

with opinion as they bring in subject matter experts twice a year in order to create these futures from there they turn the science fiction prototype into science fact through analysis and they look at how they can get recover from these events from occurring in the future so what you see normally is a 170 page technical report this past year they did one on artificial intelligence the year before that it was about automation in society and typically these reports contain about 100 pages worth of soft data analysis and expert opinion from leaders of industry government and academia now i have one problem with this focus is too heavy on subjective analysis and this some kind of makes

sense we have a lot of leaders in artificial intelligence right now that say one it'll either elevate our society to new heights never seen before or skynet's gonna happen and really it all depends on what it all depends on what your experience has been in the field of artificial intelligence and that's something that i want to be able to bridge that gap between those two understandings so that we can find what the actual verifiable threat is so the goal we want to objectify objectively analyze the futures developed by the threat casting lab so what's the solution here models so agent-based models in social simulation is a very unique and powerful programming software that can be used

to create entities autonomous entities that act given a certain set of rules that you program into the simulation they all interact within a specific environment and they have specific characterization tools that you can use to create very very intricate systems that would normally not be developed in the real world for instance computing species systems are you really going to put like a group of mice and a group of hawks in the same room and watch all the hawks kill the mice probably not i mean that's cool and all but maybe you could just simulate that or humans versus zombies i think we all know we all want to know what that looks like in the future

i don't think we really want a zombie outbreak i mean maybe some people do they're weird so you might be asking yourself but why agent-based models well the qu the answer to this lies in the fact that one they're adaptable asian-based models are specifically designed so that you can create any type of scenario that you want using a very interesting programming language called groovy two they're visual there's a lot of visual capabilities that you can leverage with asian based social simulations so that you can look at what the interactions look like in that model itself and three they're free i think we can all agree that free things are better i've seen a lot of you

at the booths so so there are three main considerations that went into developing this model so the first consideration is how do we uniquely characterize the environment that these simulations happen and this is a very interesting question that goes a lot into kind of the idea of what does cyber security what does the cyber security field look like what do the technology threats look like and how can we kind of bridge those two in order to determine what the agents are going to do when they're actually placed in that environment two who plays a role in this scenario what are the agents specifically that are going to be in this scenario and finally three how can we best create

this model so that is actually reflective of what we see in cyber security and society as a whole so we've gone set out for the past year so of determining three main factors that go into this model first are the technology factors and essentially what we're doing is we want to model and analyze what the threat posed by emerging technologies looks like in in the future so we've developed these 10 10 maybe i can count specific technology factors that we believe characterize technology and society some of these might be very intuitive to you such as implementation costs i think we can all recognize that some of these might be a little bit more difficult such as

what does ability to unify mean or what does ability secure mean so what you do is you take these models and you'd apply them to a specific domain as you can see here the domains that we have listed of where these technologies would apply is something like the critical infrastructure as listed by presidential policy directive 21. or would you traditionally consider space cyber air and land not c because the navy doesn't matter um and then finally we have four specific agents that actually go into this model the first is what you consider your traditional malicious malicious actor the attacker next you have the defender geeks like yourselves that try to actually create systems that are robust so that

they cannot be attacked and finally you have the user of the technology everyday people maybe the government maybe a hospital and then you have what we consider almost a a pseudo node or a pseudo agent called a tech node and the tech node is the agent by which every single other agent in the simulation interacts with one another this is kind of what you'd see in regular society right you need some sort of physical medium in order to conduct an attack on any other system so the tech node serves as that proxy to conduct the attack so i'm going to go through a little bit of an example developed by the threat casting lab last year

and from there we're going to apply this business process in order to see what this type of analysis looks like the simulation is called two days after tuesday hopefully it scares the pants off you so we have bill bill is an employee of red hook is an employee at the red hook port in new york bill's responsible for the shipping of autumn of goods to and from locations in new york city however there's way too many goods that they're trying to ship out in the year 2028 because it's a consumer world so bill has to rely heavily on his artificial intelligence scanners in order to conduct security scans at the port for things that are coming in

however these scanners suck they constantly break down these scanners don't know the shipping parts always have to come in multiple times because of this bill complains about them quite a bit now what bill doesn't know is underneath the surface of all this happening there's a phishing attack that occurred with one of his employees at the port because of this the malicious actor that has now gained access into the communications network of the red hook port can see when bill is complaining about the stupid scanners and in addition to that they have an artificially an ai assisted web crawler in order to determine what type of vulnerabilities might exist on this system so they see the word ai

or they see the word scanner and they think ah here's the window of opportunity so what do they do they pass a command to the automated resource movement system to order milk and bananas now milk and bananas are a perishable good so they are prioritized over all other items that would go normally through the red hook port so they order milk and bananas for every single household in new york city now this takes up a lot of the reports resources and now suddenly because of this you can't get the artificial intelligence scanning parts that you want in order to conduct further security scans on your port so they let something sneak by this is the perfect window of

opportunity for kinetic attack to occur what you have here is you have a radioactive substance known as a dirty bomb passing through the port because the ai scanners were not able to pick it up because they had to resort to manual checks on these items because manual checks take a long time they're not going to check every single thing that goes through that port so the bomb detonates in new york city chaos rains stock markets stock markets plummet global war on terror number two is declared as you can see this is a terrifying scenario millions are dead we don't want this to happen the end now now you guys are sufficiently terrified of all dirty bombs

and artificial intelligence let's go through an analysis of what this situation actually looks like from an objective frame so to begin we'll start with the three variables here you have the ai security scanners the automated resource movement tech node and then the communications network used by bill and his employees what you want to look at is the variable specifically these levels for each different tech node so i'm going to look at is the levels and the effect that this variable has on the simulation the levels here are going to be recorded on a scale of one to ten sort of akin to a likert scale this is where we bring the subjective analysis from subject matter expertise

in the cyber security field into the objective framework so we'll start with the i security scanners we have a dependency level of seven which is what causes these security issues to pop up in the first place is because they're so dependent on this technology in order to conduct these scans at the port in addition to that the upkeep cost is huge for a technology like this level 10. because of this the parts keep breaking down as you can see this creates a window of opportunity to sneak something kinetic into the port now you have the automated resource movement node as you can see here there are three main high levels that affect this simulation you have the level 10 dependency

everything depends on the automated resource movement in order to continue which is why it can be used as a tool of manipulation to give access or move commands through the system then you also have the unification factor of the automated resource movement which is essentially what allows the attacker to have access to every single other tech node such as the security scanner and the comms network uh and what is what allows them to gain this sort of autonomy to do what they want in the system and then finally you have the communications network itself used by the employees now the dependency on comms i mean i would be surprised if we moved to a society that just never talked to each

other but there's a high level of dependency on communications network which is what allows for that surveillance in order to determine where that vulnerability is in addition to this there's a medium level of user savviness which is what allows the attackers to essentially conduct the phishing attack because as much as we'd like to think the general population knows when to spot a fishing attack it's probably not the case um and then you have the unification of the cons which is what allows to unify the automated resource movement which is not just regular communication between employees but also between systems and creating a smart automated system in order to ship goods out but in addition to that this is what

allows for that monitoring to occur of note there are no defenders in this future there's nobody here to actually secure the system at least not in this scenario right now and anything that you see here that says n a just means it didn't have an effect on the simulation for that specific tech node so the attacker itself is what we characterize as state-sponsored terrorism they're more patient they have better capabilities and their end goal is destabilization in this case so now let's go through what the interaction diagram would look like in this simulation so to begin with we'll start with the tech nodes so as you can see here we have the artificial inter intelligence communication scanner used

by the attacker so that they can crawl and look for the different vulnerabilities that they're trying to seek then you have the ai security scanners used by the port in order to determine whether or not something is radioactive etc and then you have the automated resource movement tech note itself which is what ships goods to and from every location so nested in between the automated resource movement system and the ai com scanner is oh sorry first you also have the communications network which is the compromised technology in this case as well as the security sensor up here which is a part of the ai security system as a whole nested in between these two is the

state-sponsored terrorist itself and in addition to this you also have the two other users that are involved in this scenario in this case you have the red hook port which are the employees and you also have the new york smart appliances that are the ones ordering this milk and bananas so let's start with the connection that red hook has with the other two tech nodes the ah security and the automated resource movement because of the unification factor of the arm there's a high level of unification between each different tech node so there's a link between them in addition to that the arm is also going to be linked to the new york smart appliances because that's what's

communicating with them to ship these goods now here comes the malicious actor they conduct a phishing attack they get access into the communications network which inadvertently gives them access as well into the arm and the ai security this is what allows them to conduct surveillance on the red hook port communications network which then in turn they find a vulnerability and they pass a command through what this command does is it inadvertently pauses the ai security scanners from getting new parts so you might be asking yourself all right well this sucks where can we place a defender where would you guys place the defender does anyone have any ideas maybe somewhere like here or here or

here or here yep right here so that's an interesting thought previous times people have answered this question they've given something like this personally i thought we'll put our superstar defender right there because what if we were allowed to put our superstar defender so that monitored traffic between the smart appliances as well as the automated resource system in order to see whether or not a command such as milk and bananas at every single home in new york is really a legitimate command so this is something to kind of think about is that the power of this type of objective analysis is like as you said you could put it here or you could put it here

what's nice is that we now have in a framework to look at it and look at maybe different avenues of approach of how to mitigate these problems so my vision for this project is that we want to develop this model into a business process i believe this is something that strategic foresight firms can use in order to determine how they want to look at threats in the future so they're not bogged down by groupthink or other types of problems to that would only give them one avenue of approach in addition to this i hope that we could improve the military's ability to adapt to future warfare challenges uh the threat plane right now is widening as

you can see it does not take a lot for you to gain access into a system or to cause damage into that system as it did before in addition to that we have a lot of non-traditional actors that have come onto the scene that are causing a lot of havoc and my goal is that we could develop new tactics techniques and procedures there's your military buzzword in cyberspace uh based on the type of analysis that we can do here now if you have any thoughts about where you think this project could go or maybe about the scenario itself and maybe how we could develop the scenario so it's better reflective of society i would love to hear them my email is

right down there it's just first name dot last name or last name dot first name so finally i'd like to acknowledge uh dr charles mccall and dr ozick at argonne national lab they helped me with developing this model as well as looking at the protocols that you could do for this model dr pulley blank at the united states military academy and bride david johnson at arizona state university he heads the thread threat casting west workshop over there and of course finally our benevolent overlords besides charm so i'll now take any questions that you guys might have yep well so you have to get the dirty bomb somehow into the united states in the first place

i mean that would be very impressive if they could find the special nuclear material in order to actually harvest that and develop it in the u.s that would be i mean the question was so why don't they just drive the truck from somewhere native to the us into the port or somewhere into the uh city and my my answer to that would be it's much more difficult to develop a dirty bomb in the u.s than it is to develop it outside and ship it into the port this is just one specific scenario of something that might happen any more questions yep yeah yeah

um you're absolutely right no

right so that actually gets into the other part of this analysis this is more the brunt of the work that i've been doing for this year-long project is other considerations that we have to look at which are technical and strategic considerations so yes there are politics that play into this and other financial movements that actually play into this type of scenario and that's something that we're trying to explore is what does this look like and one of the questions we actually tackled when you talk about the generalizations is how specific should each node be what's the granularity that we want on these tech nodes because i mean if you think about it say we have a vulnerability in the way that

microsoft word starts up each time was that really something that we want to model in this case for this simulation that's very specific to microsoft word so how would we be able to adapt that to strategic foresight as a whole i'll take one i'll go come back to you

later

right yeah so um the the intent of this project was to actually remove personal bias from subject matter experts by developing this framework for them to use so in this case you're right my personal bias does come into this but the idea is with all with all methodologies of bringing soft science into the hard science and using numbers there's going to be some sort of level of subject matter expertise that you need to be able to quantify which is why we're trying to minimize that as much as possible by doing something like this that is a very important concern you have another question okay yep

yeah absolutely thank you

yeah no no that's thank you for asking those so uh the point of this year-long project was actually to address not just ai obviously we used a specific case study here of ai and you're right there is something important about the supply chain that could potentially mitigate this risk but it's to look at all technologies in the future such as quantum computing or any of the other technologies that you might see out there that could potentially affect society as a whole that's when we apply the framework to look at well what did this look like in the simulation itself and as to your supply chain common is that that's important right like did the visuals did

the visualization tool help you recognize that maybe there are some supply and chain mitigation techniques that you can use in order to prevent something like this from happening and if it did then yes that is very good and we've done our job here

uh and i asked that because the human intelligence from that would help fill in any um absolutely thank you for asking that um so my advisor colonel nadal venata sitting right back there she developed this specific scenario she did talk to doc workers um but you're right this is part of the processes if we're going to do this we need to actually talk about people talk with people on the ground in order to see what their expertise in it is

so it's a little bit of a mix of both so yes we're bringing in leaders of industry government and academia in order to see what's feasibly could happen 10 years down in the in the future and then from there that's when we're conducting our mitigation techniques and trying to figure out how we can solve those issues so there is a prediction aspect of this which is what we're doing we're predicting 10 years in the future what these scenarios could look like maybe there's a feasibility analysis that we're conducting on that and a likelihood analysis based on politics and whatnot and then from there we're doing that mitigation