Let That Think In: Thought Experiments And Their Application To Cyber Security

thanks very much Craig hi everyone uh so this has lit that thinking thought experiments in our application to cyber security so my name is Matt wixey I'm a threat researcher and writer at Sophos I've been in infosec for about 12 years so I work with John who did the keynote just now um although this particular talk is in a personal capacity so the views and opinions aren't necessarily those of my employer uh before doing threat research I worked in pen testing research and development and threat intelligence at PWC and before that worked in law enforcement I'm also a PhD student at UCL and usually if you've seen me talk before I usually talk about very Niche

weird and wonderful things like light and sound attribution worms criminal marketplaces that kind of thing this is quite a different talk um and also as you can tell from the opening slide my ability to keep up with memes ended in about 2017 so there's going to be some really crappy memes uh in this presentation so what I'm going to try and do in this talk is explain what a thought experiment is and how it works try and apply it to cyber security and then give you a guide to creating and using thought experiments as well so let's start with a brief primer on thoughts experiments so it's actually quite hard to Define and there's no consensus in literature

as to what a thought experiment is it's probably easier to say what they're not so here are some examples of things that people are mostly thought leaders have said our thought experiments mostly on LinkedIn so a hypothetical scenario based in reality what if we had a billion pounds to spend on security a tabletop role-playing exercise so you know the kind of thing you're the head of a sock and you've been hit by ransomware or think like an attacker that kind of thing or analogy so the one you hear a lot is leaving RDP exposed to the internet is like having your your front door open that kind of thing uh that's not to apply that these can't be

helpful um but they're not thought experiments in the way that they're commonly understood in other disciplines I even read when I was kind of compiling this list I read about one thought leader in cyber who talked about doing Hands-On thought experiments whatever they are so why are these things not thought experiments well as I mentioned there's no consensus on definition um but there are some common elements to the proposed definitions so it's an imaginary scenario which for some reason is unrealizable in reality either ethically it would be unethical or illegal to actually do it or it's just completely impractical to do it and crucially they have a goal uh whether that's to destroy an existing Paradigm

or support an emerging one it's not just exploring possibilities and sort of wondering what if and what might happen um and it's not something that's done mentally just because it's more convenient to do it mentally there's also an element of rigor in the construction of a thought experiment and in the answering of it so I thought experiments typically come in two parts you have the experiment itself the scenario and then you have the argument that follows it but it's all worth also mentioning this is a controversial Topic in science and philosophy so some people disagree thought experiments can actually be useful some say that empirical experiments are more valuable and provide more uh more use

so a really brief overview of thought experiments they began probably with Socrates and Plato City uh allegory of the cave is probably the most famous One although lots of others um anyone familiar with the allegory of the cave Plato's allegory of the cave so it kind of um uh a number of later thought experiments built on that one so you had things like uh they cause demon brain in the vat if you've heard of that and then ultimately things like the Matrix and stuff as well uh they weren't really formalized until 1897 by a German scientist called Ernst muck who at the time said they were a necessary prerequisite to doing empirical experiments although that's no

longer considered true um they've obviously been used extensively in philosophy and also in physics so in physics uh they uh there are some really famous examples there's Galileo Newton Schrodinger Einstein and you might be familiar with some of those more recently in Information Systems research and in politics and in law as well and then even more recently our Ai and privacy so probably starting in 1940 with the Turing test uh with regards to Ai and then a lot of others since then as well thank you so I mentioned empirical experiments and people sometimes think that these two things empirical experiments and thought experiments are in opposition to each other um they're not really or they

shouldn't be um both of them are manipulating independent variables applying some sort of control and then measuring the outcome on a dependent variable in order to test some sort of hypothesis or to test assumptions or to make inferences about the world or all of them uh there are people that say that thought experiments are more of a rhetorical tool um so they're an argument um or a paradox or whatever but ultimately it's really just the data and then the interpretation on the discussion of that data is then used to support a refuted position um there is a little bit of a difference between philosophy and physics in that regard um so with philosophical thought experiments the philosophers do tend to

have an argument in mind because philosophers love to argue whereas with physicists it's more of a model like a mental model that they're using to explore something so let's um talk about a few examples of thought experiments these are all in philosophy uh the first one is Mary's room anyone familiar with this one before I never heard of this one so I'll just kind of read out the thought experiment and then I'll talk about the argument that underpins it so Mary is a scientist and she has lived her entire life in a black and white room so everything in the room is black and white the furniture her clothes all the equipment in it whatever

uh there are no windows um there's a door but she can't see through it uh the only contact she has with the outside world is with a monitor uh attached to a camera but it's a black and white monitor so Mary has never seen color but I mentioned she's a scientist and she specializes in neurophysiology so she knows everything there is to know about color she knows what happens in the brain when we see a red tomato or a blue sky she knows the specific wavelength combinations that stimulate the retina for specific colors and how this leads to us associating that stimulation with a word like red or blue so one day Mary is released from the

room and sees color for the first time does she learn anything that's the thought experiment so uh quick show of hands who thinks that Mary does learn something when she goes out the room for the first time yeah it's quite intuitive thing isn't it so um this thought experiment is an argument against physicalism which is the argument that everything in the universe including Consciousness is physical it has some sort of physical form and it basically argues that there are non-physical things that can only be obtained through experience so when Mary is released from the room she uh experiences color for the first time as opposed to just knowing everything about it so it's a really long debate I'm not

going to go into physicalism and sort of the ins and outs of this one um but this is very famous Lord experiment that um uh stimulated a lot of debate so we can see something interesting about the structure of thought experiments from Mary's room this isn't applicable to all but it is to many so we have kind of four steps we have an element of World building so Mary is a scientist in a black and white room we have some rules uh so Mary can't just walk out of the room there are no windows and so on we then have a challenge or a conflict so one day Mary is released and then finally we have a question does Mary

learn anything uh let's talk about another one day cars demon which I mentioned earlier uh anyone familiar with this one no okay uh so um it's quite simple just imagine that there's an all-powerful demon which presents a complete illusion of the physical world so that the sky air Earth colors shapes sounds all external things are merely delusions which this demon has created so you don't actually have hands eyes Flesh Blood or senses but you believe you do because of the demon so this is this was descar's way of exploring uh what he called systematic doubt how do I know that anything in front of me is actually real This was later expanded on with the

brain in a bath thought experiment in 1968 how do we know that we're not just brains floating in a vat somewhere and everything we see is a hallucination so again similar structure to Mary's room this one doesn't have a particular challenge or conflict but other than that it's the same and then the experience machine uh by nozick in 1974 so this one is um based on a premise that there's a machine which can give you any pleasurable experience you want so if you're plugged into it it's indistinguishable from reality but you'll stay there until you die and you can choose the pleasurable experience you have you'll be attached to an intravenous drip which provides you with

nutrition do you have any reason not to plug into that machine so this was nozick's argument against Hedonism um he was basically saying that pleasure is not the only valuable thing in life that most people wouldn't choose to be plugged into this machine we want to do certain things not just have the experience of doing them we want to be a certain sort of person uh if we did plug into this machine we never see our real friends and family ever again and so on so Brown in 1991 suggested three types of thought experiment uh destructive thought experiments which destroy or challenge an existing Theory constructive thought experiments which support a theory and there are three

subtypes of those mediative where you facilitate a conclusion from a theory conjectural where you just have an explanation but not necessarily a theory and then direct where you don't start with a theory but you end up with one and then platonic thought experiments as well which are both destructive and constructive uh some other formats um that are very common is something called a reductio ad absurdum argument this is basically where you establish a claim by showing that the opposite position leads to absurdity so uh you'll be familiar with this if you had an upbringing like I had if you do something bad and you tell your mum I did it because my mate did it

and your mum says well if your friend jumped off a cliff would you do that as well that's a reductive ad absurdum argument um along the same lines like if you ever got told off of picking flowers or something and then someone said to you if everyone picked flowers there'd be no flowers left that's that's a reductor I had absurdum argument uh Sometimes they come in the form of paradoxes as well so uh there are lots of examples of these but like Achilles and a tortoise is a really famous One by Zeno uh Zeno's Arrow as well um and sometimes really strange forms as well so uh fiction has been argued as being a thought experiment or at least

some examples of fiction so is the Matrix thought experiment is a good example of that and then particularly in cyber security there's a whole load of um fiction that could be really interesting inspiration for Thought experiments things like cyberpunk sci-fi Tech horror so things like Brave New World 1984 Black Mirror that kind of thing uh and then co-ons are um a really fascinating one anyone familiar with coans ever heard of the term before so they originate in uh Zen Buddhism um and they're like sort of riddles or puzzles that are given to Zen novices by Zen Masters um and sometimes the novices had to think about these for for years in in silent meditation um Carl Jung said that they aim for the

complete destruction of the rational intellect so they run completely counter to Western dualism and rationalism and that kind of thing um they're intended to like provoke what in Buddhism is called the Great doubt so um and then eventually lead to Enlightenment but basically they disintegrate logic or they're intended to so there's lots of examples one you've probably heard before is what is the sound of one hand clapping that's a Cohen another one is does a dog have Buddha nature um so a student asks this to a Master a lot of Koons have an answer which involves a concept of moo mu which is really hard to Define but it basically means like nothingness or unasking the

question or just rejecting the premise of the question entirely um I won't go on about cars for too long um but they're really interesting so about 2 000 of them survived um from when they were first built there have been no new ones written for centuries but they're online if you want to have a look I wouldn't binge them because your mind will melt but uh yeah um definitely have a look at a few um and one really interesting application of thought experiments is to address something called a coonian crisis named after Thomas um so Coon's Theory or a coonian crisis is um when you have a paradigm or a theory that explains a certain phenomenon

over time that theory will result in more and more anomalies which at first are either ignored or attributed to measurement error or that sort of thing but at some point at some point you just can't ignore them anymore and at that point you have a coonian crisis and thought experiments have in various Fields destroyed that existing Paradigm completely kind of torn it down and then constructed a new one which both explains the pre-existing phenomena and also the anomalies so that's kind of one really interesting application of thought experiments some criticisms against thought experiments um so some people have argued that the conclusions from them are sometimes due to biases so if we take the example of

the experience machine which I talked about um nozick said that most people would say no to being plugged into the experience machine but a lot of critics have argued that people wouldn't say no because they're rejecting Hedonism they're saying no because of status quo bias so they just prefer things the way they are um so some critics proposed a reversal test so if you knew that you were already plugged into the machine would you choose to disconnect from it for example Norton who's a very vocal critic of thought experiments has said that they are just picturesque arguments that they don't provide us with any new data that they simply reorganize what we already know and just make it explicit

um and then there are epistemological problems associated with thought experiments as well potentially so the question that gets asked is can we genuinely learn new things about reality just by thinking and if so where does that knowledge come from which in itself is a kind of thought experiment I think the Counterpoint to it is a thought experiments aren't necessarily aimed at generating new knowledge but just to evaluate and justify theories um some of the scenarios in thought experiments may be uh too far removed from reality to actually answer the question or to focus on it so for example there's a thought experiment by a guy called Derek parfit um called amoebas which involves humans who split

down the middle and that's all he says about it so you're trying to kind of conceptualize where they split at the waist or are they split in half and can they still function as humans and does it hurt and that kind of thing um so you just really kind of end up being distracted from the main point at hand they almost may also just have really bad logical flaws in them so lucretius is Javelin is a good example of that lucretius's Javelin is a reducto ad absurdum argument that the universe is infinite so he asked us to imagine a man at the edge of the universe throwing a javelin um and he says that the javelin can't go

forward because the man's at the edge of the universe and it can't rebound because what is it rebounding off of that's absurd therefore the universe is infinite but he kind of doesn't take into account the fact that a surface can be finite and have no Edge so a sphere for example and then finally thought experiments can involve idealization so we say what we think an ideally rational calm person would say in response to a thought experiment and therefore may not get uh realistic answers okay that's the boring stuff out the way uh we'll talk about thought experiments and cyber security so why do we need them in cyber security um will they allow us to test

assumptions from practical scenarios um as I mentioned they can address cunying crises which I think we have a number of in cyber security so an example would be humans are the weakest link right which um personally I think is quite an outdated way of thinking um and you could argue that our anomalies associated with that belief um and we also need more thinking in cyber security we often prioritize the accumulation of knowledge not unreasonably but um kind of thinking about issues in cyber security can also be really helpful and finally they can be an alternative to or supplement uh throw away hot takes which you've probably seen a lot of on social media um that's not to say that hot takes are

bad they often contain really interesting fresh insights but too often they're just posted without much thought really quickly and just as quickly forgotten whereas thought experiments could put some rigor and some consideration around them as well so I'll talk about some thought experiments in in related fields starting with AI um the first one is the kitten and the Cockroach um this uh I've tried to look for I've read this once on the internet and I've never been able to find it again so if you know who did this let me know and I will credit them um but it's basically this so imagine I have a box on either side of me one on my left one on my right cardboard boxes

and the one on the left I open it up and there's a kitten inside so it's about three months old really fluffy really friendly um who would like to hold the kitten quick show of hands okay cool and then uh on the I'll take the kitten back from you I put it back in the Box on the Box on my right I open that and I have a glass jar and inside the jar is a cockroach and the offer is that I will unscrew the jar and I'll tip the Cockroach into your hands so you can give it a cuddle uh who would like to hold the cockroach okay now imagine that I have the ability

with the click of my fingers to transpose the brains of these two creatures so I can put the cockroach's brain inside the kitten and the kitten's brain inside the Cockroach so I snap my fingers and that's done I now open the box on my left again and I take out the kitten so the kitten is no longer thinking cat thoughts it's thinking cockroach thoughts it wants to find somewhere dark and warm to hide away it wants to lay eggs it wants to feed it thinks it's got six legs instead of four uh and if you held it it would probably try and wriggle up your sleeve who wants to hold the kitten now some strange people over there

um so that's the thought experiment and the idea behind this is a thought experiment which you well it's anyone about Hazard a guess or what that thought experiment is about you want to shout out

kind of yeah so it's basically about um why we shouldn't trust AI um it is saying that no matter how friendly and anthropomorphic and cute and cuddly uh some form of AI might appear so whether it's a chatbot whether it's like Sophie the robot or it's dressed up to look like you know Megan uh in the film the recent film something like that the brain that's inside that thing is alien to us um it's it's as dissimilar to our brains as a kitten's brain is to a cockroach uh so next we've got roko's basilisk I'm imagining people have heard about this quick show of hands if you've heard of roko's Basilisk okay I was expecting a few more than

that um so Rocco's basilisk uh was in 2010 uh it got a lot of attention when it first came out um because I so Rocco's Basics is probably the silliest thought experiment ever um but it got loads and loads of hype when it was first released um it it's basically about why um you should give all your money and time to developing a super intelligent AI to avoid that super intelligent AI torturing and killing you in the future um if you look it up if you haven't heard of it you won't understand it because you're not supposed to understand it because it is based on some really really torturous logic which only people in a very very strange

corner of the internet actually believe um but the reason it got so much attention is that those people who do buy into that logic were so traumatized by roku's basilisk that it caused them real genuine psychological distress so it's worth looking up just just for that reason um there's the Turing test which you'll be familiar with so um by Alan Turing in 1940 so originally this was involved three humans so you would have a human questioner and then you would have a man and a woman answering the questions in like closed rooms and the questioner had to decide who is the man who is the woman and then it was changed to include a human questioner and then you have a

human answering the questions and a computer answering the questions and a questioner has to determine based on the responses who's the human and who's the computer so the Turing test statue has been passed by a number of implementations of AI including Eliza um although uh John so a philosopher criticized it in a thought experiment called the Chinese room um where he basically said that the ability to manipulate symbols doesn't necessarily make a computer intelligent um so if we can if we look at something like chat gbt for example is a good example of that so chat gbt uh in its simplest form predicts what the most likely next word in the sentence is um it doesn't mean that it understands

what it's saying it doesn't make it intelligent so the Chinese room experiment is if you imagine a computer takes Chinese text as input through a slot in a door and processes it according to some instructions and then produce other Chinese characters input and put it out through a slot in another door if it passes the Turing test it doesn't mean that it understands Chinese it simulates the ability to understand Chinese so by the same token if you had a human in that room who takes in Chinese characters through a a slot in the wall processes them according to the computer's instructions and then passes them out it doesn't mean that the human understands Chinese and then finally have the trolley

problem which you I imagine most people would be familiar with so you have a train on the track if it carries on straight it kills to kill one person if it goes on straight and then if you flip a switch or no it's gonna kill five people if it carries on straight or you can flick a switch and it will kill one person something like that so it wasn't originally intended for AI but it's been used for AI ethics with things like self-driving cars and that kind of stuff then it's privacy as well so this is a thought experiment by a guy called James Moore so this one's pretty simple if you consider someone who has their whole

life under surveillance and it's really really intrusive so all their internet activity or their phone activity um everything that they do outside the house or their followed pictures are taken of them videos um everyone they meet everyone they talk to say there are cameras and microphones all over their house whatever however this person that's under surveillance doesn't know anything about it and it doesn't interfere with their life at all um their life is absolutely no different to how it was before this intrusive failure started the only thing that's changed is that this person has no privacy so the question this one poses is is the person morally wronged by this so quick show of hands if you think they are

yeah okay feel worried about the people who said no um but the argument here uh by James Moore is that um privacy has an intrinsic justification even though no harm comes to you um it is still intrinsically Justified assassination politics this is um quite a dark one anyone familiar with assassination politics our hands no one oh okay this is okay so this is a really interesting one um so assassination politics is a thought experiment created by a guy called Jim Bell uh in 1997 Jim Bell was a member of the cypherpunks a group of uh uh philosophy philosophers um cryptographers researchers um who uh banded in the the kind of early 90s and they had something really

famous called the cypherpunk's mailing list where they talked about some of their ideas um it was the foundation of a lot of uh stuff that we use today uh Julian Assange was the cypherpunk um pgp came from a Cypher punk from Phil Zimmerman a lot of the principles underpinning Bitcoin and cryptocurrency came from the cypherpunks as well so Jim Bell was a member of this group and assassination politics is bench basically an incentivized Deadpool so imagine that there's a secure Anonymous way to place a bet online on the date and time of a politician's death when that pool is large enough there's sufficient incentive for someone to place their own bet and then assassinate the politician to make sure that their

bet wins and then they collect the pool so at the time Bill proposed this it got a lot of attention from federal authorities as you might imagine but the tools to actually do it weren't available at the time so the cypherpunks really wanted like Anonymous digital cash but the the things they were using all had kind of flaws didn't quite work that kind of thing but by the time we get to 2013 we have tour hidden Services we have Bitcoin and other derivatives um and someone actually made this a reality there was a hidden service on tour um which um had one of these incentivized Deadpools and there were pools worth 75 000 on Barack Obama Ben

Bernanke and Keith Alexander now obviously uh none of those gentlemen were actually assassinated um and really amusingly in 2018 someone stole all of those Bitcoins um sorry it's probably a scam but in 2018 uh someone created a new one based on a protocol called auger which is a decentralized prediction protocol and that was used to set up new assassination markets not just on the death of politicians but also on mass shootings and terrorist attacks are really disturbingly so this is an example of a thought experiment which actually uh came true um although you know touchwood no one's been assassinated yet uh and then there's the uh ship of Theseus Mark II anyone familiar with the ship of Theseus

oh cool okay more people here are people familiar with trigger's New Broom as well yeah okay cool um so if you're not um it's from Only Fools and Horses so trigger uh won an award for having the same room for like 20 years or something and it had 14 new heads and 17 new handles so this is essentially the the same thought experiment as a ship of Theseus um so if you have a wooden ship and you take away old planks as they Decay and replace them to the point where you've replaced all the planks is it still the same ship or not so um This was later adapted um into a thought experiment called

neurat's boat where Sailors have to reconstruct their ship while they're at sea but they can never start afresh because they're actually sailing it so instead they have to replace it bit by bit with the rest of the shipping support so that eventually it's a new boat but only gradually and then in 2005 a philosopher called stanovic adapted new Earth's boat and actually applied it to the process of um uh dealing with things like disinformation and conspiracy theories and irrational beliefs and that kind of thing his kind of concept was that you can revise your beliefs recursively so if you identify certain tenets of belief that um are not rotten that you can actually stand on and use to support you

then you can stand on those and then replace the planks that are rotten I.E your kind of belief in uh you know conspiracy theories or whatever so uh that's the uh experiments in related fields I'll move on to some thought experiments that I've created um these are drafts um and they are not very good um and I will be more than happy if you want to adapt them borrow them steal them um change them whatever you like but I've got three of them and we'll run through all of them and see what you think okay so the first one's called Cookie Monsters so um imagine that every time you interact with a computer you leave a trace behind

a unique encrypted identifier which contains an encrypted form of your name and it's literally in everything you do so every packet that gets sent from your machines every command you type every message you send on social media or email or whatever it contains this tag it's not visible unless you know it's there and you're looking for it and it can be decrypted and someone can get back your name you don't know this tag exists until one day you do what's changed so um you could actually use this thought experiment for several arguments you could use one from an attribution perspective so if you think about it this tag sort of already exists um in behavioral signatures so when

we're trying to attribute threat activity for example we're trying to find out kind of who a threat actor is or where they're based we tend to only focus on discrete identifiers like IP addresses emails domain names that kind of thing we might look at like correlating office hours with time zones with attack activity and that sort of thing but we don't tend to look at behavioral signatures which all human beings leave behind when they interact with something else so when you write for example you have a unique writing style you have a particular typing speed you will use commands with certain switches and you'll tend to use those switches in the same order and that kind

of thing you could also look at this from a privacy perspective so by knowing that those unique behavioral traits exist which are of a great deal of interest to a number of people who might be monitoring activity you could try to change them and then finally you could look at it from a uh a kind of I guess a wider ethical perspective so do we act differently when we know that we're being observed and monitored and tracked if we did know exactly how much of our data is tracked and analyzed would it change our online behaviors if this thing was real what would it say about you know what would happen with trolling for example

um if we knew that all our online activity was traceable and anonymity was impossible uh would that be good or bad so that's that thought experiment uh this is an interesting one so this is um called keyboard Warriors or if you like snow crash 2 so imagine there's a new kind of malware and this new kind of malware can directly harm people once it's on an infected system it causes their monitors to flash at a certain frequency which induces a seizure so nobody has died from this yet but several people have been injured and in the future many people may die and everyone's vulnerable to it so the question is is this a kinetic weapon is

it a weapon and how should we respond to it so this one's a bit of a Trojan uh horse thought experiment this is based on some of my PhD research and actually attacks like this have already happened uh in 2008 um some uh some attackers uh uploaded flashing gifts to an epilepsy sufferers support Forum causing several people to have seizures uh there was also an individual uh in I think it was 2019 2018 2019 who suffered from epilepsy and people tagged him in tweets containing flashing gifts um in an attempt to get them to have seizures um there have also been uh as you're probably aware uh vulnerabilities that have been demonstrated in things like

pacemakers insulin pumps cars medical implants OT iot um to the point of causing explosions or the release of mercury and that sort of stuff so this particular thought experiment is an argument for more Street or would lead to an argument around more stringent legislation and a redefinition of the term cyber weapon um the the kind of old Paradigm here is that cyber weapons are abstract that it's a hyped up threat um not helped by journalists in kind of the the 2000s talking about things like digital Pearl harbors and cyber 911 and that sort of stuff but the old Paradigm is that cyber weapons can't cause as much damage as kinetic weapons that they could even be used to wage more moral

Warfare or more bloodless Warfare and that they're only really applicable to things like Espionage and sabotage um but the the fact that there is now uh there are now attacks which can cause kinetic harm uh would suggest that there's maybe a cuning crisis there are there are anomalies here and then finally our proxy wars um so imagine that uh both through actors and Defenders so both red and blue increasingly leveraged increasingly powerful AI until eventually all attacks and all defenses are completely designed planner created by Machine learning models and there's no human involvement or interaction has anything really changed other than the fact that we could just chill out uh and just watch or should we watch

so I'll leave that one with you um so benefits for applying thought experiments to cyber security uh so they don't cost anything um other than time which is a good thing um even a bad experiment like the three examples I've just shown you there could generate some discussion um but at their best and as in other disciplines they've replaced entire paradigms and completely changed ways of thinking they can just be used to reframe arguments and make them more persuasive uh encourage more scientific thinking and they're also just fun it's more imaginative or refreshing than just a plain argument uh at least I think so anyway some drawbacks for them in cyber security it can be quite hard to come up

with them um for a field as specific as cyber security um security topics tend to be more Niche and less Universal than saying philosophy or physics and let's be honest like normally the problems we're dealing with in cyber security tend to be lower Stakes not always but most of the time compared to you know what is personal identity and is consciousness real and that sort of thing uh there may be flaws in construction kind of logical flaws particularly if we start using them and we're sort of new to doing it in this discipline we might make mistakes and as you can tell from those two examples you often need to make uh heavy use of analogy and metaphor which

isn't necessarily a bad thing but it's a possibly a bad thing so uh sorry so let's talk about designing a thought experiment so um I'll run through a really quick guide on kind of how to do it so to start with you don't necessarily have to come up with a thought experiment from scratch you can just repurpose an existing one so here are three that I've just repurposed these are very much off the top of my head but we could do Schrodinger's breach so what if we acted as though we're simultaneously in a state of being compromised and not being compromised uh the veil of ignorance so originally this was about kind of how we deal with

issues around sort of equality and social class and and kind of um Prejudice and that sort of thing um but imagine that we have to design the entire cyber security industry from scratch but we don't know who we're going to be in that industry so you don't know if you're going to be red or blue you don't know what role grade level of Education or knowledge you're going to have um and then whatever industry you design you have to work in it what choices would you make and then finally the security machine so this is sort of a a take on the experience machine so imagine that AI takes over all security research produces really Superior results to

humans would we could still continue doing it anyway would we still do vulnerability research will we still look for vulnerabilities will we still try and find new ways to bypass EDR and AV that kind of thing um would there be any point in it and it's more about kind of uh thinking about why we do these things so do we do it for logic ethical a larger ethical purpose do we do it because it pays well because we enjoy the intellectual challenge of doing it and so on so uh five steps to coming up with your own thought experiments uh firstly what is the argument so are you trying to destroy an existing Theory or create a

new one um or both and how can you actually test that premise and explore it step two World building um so you want some rigor in creating a model if you have kind of people or systems in that thought experiment make sure they behave in a particular and consistent way um you don't have to limit yourself to reality in fact you shouldn't you shouldn't have to make it realistic or practical but consistency is the key thing foreign so you want to patch up plot holes Define the behaviors seal off The Logical floors and Escape Clauses that are in your thought experiments and remember that while the scenario itself might be implausible the rules should be

conceivable and then the optional step of introducing a challenge or a conflict um so what changes about the scenario that you've presented um to kind of kick off this thought um you can kind of think of this as like the inciting incident in a screenplay or something we should build on the scenario rules it's not always necessary you don't necessarily have to have it and then finally the question so what question does the thought experiment provoke um you know so this is things like what has changed why or why not is this possible and so on so that's kind of the five steps to producing a thought experiment and then you have a last step which is the actual

output itself which will be an argument uh which is based on that thought we experiment um tries to answer the question explores its own implications and then evaluates the argument for errors as well we made it to the end um so this was an introduction to the world of thought experiments very much an overview there's obviously been tons written about them so I've tried to jam as much as I can into about 40 minutes um so thought experiments are low cost but potentially very powerful they've had really dramatic impacts in philosophy and physics and in a lot of other fields as well so potentially it's something that we could make use of in cyber security a bit more

there is more research needed on it so if you're looking for a research topic this could be a good one and it's potentially a really interesting mashup of philosophy and cyber security and philosophy of science and that kind of thing as well so I'm ending with a call to action um which is uh think about some assumptions or theories in your daily work in your particular corner of cyber security and how you might be able to apply thought experiments to it or even wider an industry level with things you've seen remember that the whole point is either to support dominant or existing theories or to challenge them so you could do both um or either one

and then I've also got a little bit of a challenge so if you'd like to come up with your own security related thought experiment um so 30 days from now you've got to send them to me at that email address uh the one I like best in my really unqualified opinion gets a very very small prize um and I'll give you bonus points if it's a co-own as well because it's been kind of a few hundred years since anybody's actually come up with a new Cohen so if you're able to come up with one and it's based in cyber security uh you won't get a very small prize you'll get a small prize so Next Step Up

so if you're interested lots of references here um so these contain kind of all the the thought experiments that I've um talked about during this presentation lots of discussions around thought experiments generally um and that kind of thing um and that's me so thank you very much for listening I hope that was interesting and or useful um I think we have quite a bit of time left so if anybody's got any questions uh let me know thank you

[Applause] Yeah question back

that says

how do you manage the expectations

yeah absolutely you'd have kind of a number of choices I think if you were doing it I think it's a really good idea um definitely so one thing you could do is you could make the thought experiment an argument but you could kind of almost come up with two opposing thought experiments and sort of split the room into two um send them away for a few minutes have them you know come up with an argument based on a thought experiment and then come back and present to the other half and then kind of talk about the pros and cons of each one um but yeah absolutely kind of I think even if you were just doing one through

experiment sort of putting it out there um getting people to to kind of think about the the proposition that's behind that thought experiment and then either sort of thinking about the internally or sharing it with the group could be really useful yeah yes

it's like who's ultimate judge before his parents any good because you said you might say good I might say it's bad yeah yeah it's a really good question I mean so um even with uh full-time professional philosophers who come up with thought experiments so Mary's room for example um was done by a guy called Frank Jackson who's kind of a quite a well-known professor of philosopher of science people have said that that thought experiment contains flaws um and it doesn't take into account this and that and that so I guess there's not really kind of an ultimate way to tell it's it's certainly in philosophy um criticism the thought experiments are based on sort of logical flaws in them

um where you kind of maybe make an assumption before which there's no basis for example um I don't think sort of kind of repurposing thought experiments for like particular disciplines is necessarily bad I mean you kind of want to make sure that you caveat it maybe if you wanted to say this is only applicable to physical security or this is only applicable to cyber security you could kind of caveat with that um but yeah I guess like I said kind of the the main thing you would want to do to try and make it as robust as possible is to try and seal off like uh logical flaws plot holes that kind of thing and

to make sure you're not making sort of deductive leaps in either the thought experiment itself or on the argument that it uses

um it it depends what you what the goal of it is um so if the goal of it is just to get people to think about something that they hadn't previously thought of then yes um if your goal is to use it to support an argument which says X is the case and Y is not the case then if you have a logical flaw in it that's almost sort of a back door that could be yeah exactly um so yeah it depends what the goal is really uh any more for any more nope okay thank you very much