Keynote, Day 2: Homicideware

all right without further Ado The Fabulous Andrea Matan not sure about the fabulous but thank you very much the check is in the mail as long as you're sure as long as you're sure about the Andrea matu part we're okay most days okay can't be 100% most days yeah so thank you very much for the invitation to speak today thank you all for showing up at 9:30 at a hacker con morning that is hard I know um so uh I will share with you some thoughts um some of them will be intentionally a little bit provocative hopefully and I look forward to engaging with all of you on the topics that I mention in my slides also I have quite a

few slides so if we get to the 45 minute point and I'm still doing this someone Josh Audie someone please pick up the St uh du uh uh hand thank you okay so this talk is titled homicide wear um for reasons that are probably uh somewhat obvious to this crowd because increasingly we are starting to be worried about the Confluence of lifethreatening events that endanger both life itself and physical safety of human bodies so before before I launch on my thoughts on this topic just to give you a brief sense of who I am where I've been I've been doing this a while um and for better or worse it is my 25th Law School reunion this year not

sure how I feel about that I'm speaking at it also not sure how I'm feeling about that one um so done some government stuff been in this community it's been a great pleasure since 2003 um and I look at things maybe a little differently because my background is a little different than um most of you in this community My Views are mine please do not attribute anything that I say to the nice people in government that I work with I'm not their fault I'm me um also this is original academic research so please discuss this to your heart's content but please also do remember where you heard it because it's kind of Soul crushing to have you know 20 years

worth of work summarized in a one paragraph blog post that may or may not be quite right without attribution um so please thank you okay away we go in case you need to take a little nap at any point during this talk I'm going to summarize everything on two slides security is the Lynch pin of Technology safety but we have generally not been talking about it this way in policy and legal circles we have instead kind of essentialized exceptional lied the questions of Technology as they relate to issues of physical safety but also economic safety emotional safety Financial safety all of these are bound up in the issues that this room thinks about regularly but the policy and legal

discourse conversation hasn't really mapped well so confidentiality Integrity availability issues are regularly now threatening public safety questions both in terms of infrastructure but also on an individual level but here's what's shifting now and I've very um kind of mixed feelings on this one the normies are starting to get scared the kinds of questions that I'm getting from my non-te friends now are shifting that include and they include things like oh this is what you've been trying to warn me about for the last 25 years so that is a positive Evolution but it also means that they're having that initial um moment of trepidation fear that some of us uh may have experienced when we first realized that all we've

got to defend us is us so the way that we tell the story to policy folks to normies to ourselves about the last 25 years of what we've been doing in security is going to shape the next 25 years of security this is a great point to pause do a little bit of self-reflection on the way that we've been talking about these questions and what whether the way that we discuss them in community are the best way to talk about them with people outside the community and figuring out how those things fit together so I'm calling this conversation a Goldilocks problems situation which you'll see in a minute uh hopefully with a bit of humor but

there are some hard truths that I think we haven't quite discussed amply about Insider attacks the uh perhaps inadequate threat modeling for Insider attacks the internal controls inadequacies that are rampant inside our companies that are very sophisticated in some ways and kind of let stuff slide through the cracks and other ways and the language of internal controls is the legal language so some of these terms that you know what I'm talking about but they look a little weird it's because they're the way that the legal framing would engage with this as we enter the age of AI security which you had a wonderful keynote about yesterday things are going to be more complicated not necessarily new but more

complicated and even more about tech safety for reasons that we'll briefly mention and increasingly the line as we all know between civilian safety questions and national security safety questions is functionally gone because of the revenue models of some of the uh threat actors that we know exist in other parts of the world so the public relations spin about harming innovation in Tech creation it's just not going to hold water going forward it's not going to stop liability because the Frameworks that will be applied are not those from technology spaces they're from law spaces and the way that legal uh F finders of fact we would say judges juries uh Regulators think about these questions are going to be very harm

focused they're going to look at the bad thing and work backwards they're not going to start with the tech question so how should we think about this if you just hang your hat on be it hopefully a white or gray hat um on the idea of context harm and intent knowledge and I'll will unpack each of those as we talk those are legal terms of art that will help you get a handle on the way that a judge would look at these questions the way that a regulator will look at these questions so the context that you're operating in the kind of harm that's POS possible and who knows what and controls what or who should

know what and control what and we'll walk through the example of homicide homicide you generally think about first-degree murder you think about the attacker who premeditates something for months and executes it but that's not how the law thinks about homicide there are lots of very spur ofth the- moment situations that are very serious felony level of prosecutions and so homicide with a computer is still homicide and we'll walk through that example to talk through things so I'm not going to leave you with all gloom and doom we'll also talk about what we can actually do and this is a position that I would not have adopted 25 years ago this is fresh and new to me because of my work in

government frankly and seeing the tech economy shift in some ways that um I find concerning and that have um made me fight to remember the joy that I felt opening my first browser and telnet and those bygone days of the green and black um and I think helping to share that history with the youngans would um instill a little bit of Hope in the way they were dealing with these questions and remind ourselves that there is still hope so I think we need a new regulator I think we need a technology regulator of Last Resort not to change things dramatically just to fill in gaps to coordinate to create alignment as the techies would say um on various matters

of policy and law and to help stand up interesting new Gap filling initiatives in the most critical sectors like healthare where the security problems are not easily handled by small hospitals on a local level they need help um and they need coordinated help so Tech safety is a different framing kinging on context that should be context sorry harm and intent and beware the uh exceptional iing of AI the last point which is just a teaser for the workshop that will happen immediately after this we'll be talking about professional Society emergence and I know this is anathema but it is a part of every mature industry sector's evolution where you get to reclaim your story and

tell it the way you want it told and so this is about storytelling and this is about building the next 25 years in a better way so that we don't feel like each year we're just banging our heads against the wall a little harder so please come to that Workshop immediately after and um there will be stickers so many stickers um so even if you only stay for a little bit come for the stickers okay let's start the Story Once Upon a Time an attacker compromised a physical perimeter of a physical plant the confidentiality of the information the Integrity of the internal manufacturing processes and the availability of critical infrastructure resources once upon a time a potential

data science entrepreneur leveraged gorilla collection tactics to generate assessment metrics for residential infrastructure supply chain inputs manufacturing processes and Qi QA using trial and error and high reliability low latency sensing technology once upon a time a Lost Child sought refuge in a structure and survived on foraged food until rescue once upon a time a trespasser entered a bear habitat ate the day's rations broke the UR sign enrichment materials damaged the living structures and potentially germed up the den with human pathogens dangerous to the Cub's health these are all the same story they're all potentially simultaneously true so what does this mean it means that we're living in a complex situation where narratives matter Goldilocks is simultaneously a

criminal Intruder a budding data scientist from the Bear's perspective a destroyer of habitat and the Bears would view themselves as the center of the story not necessarily the human child this is also a story about the intersection between lawyers hackers and bears they are all dangerous when they're hungry and let me assure you that at this point the lawyers are getting hungry and what will make the future roll out of uh the crowd strike litigation that's already been filed and will be filed um very uh educational interesting from a legal Observer perspective is to see how that plays out in terms of the claim allegations the settlement rates Etc and the insurance Dynamics which are going

to be messy so some facts about context harm and intent are going to be disputed so what do we do with this well in Goldilocks and the Three Bears there are actually many versions of the story where even the main character changes so these kinds of tellings and retellings are something that we are all familiar with in the time we're kids but what does change is whose story it is what the perspective is from which it's told whose progress is centered in the evolution of the story so in the last 25 years of security there's sort of been a default Assumption of a free ride on the part of many companies of a default of no

liability and the conversation in legal and policy circles has been very s uh sort of focus circling around questions of breach remediation and compliance which is a word that I am not a fan of um as my meme there demonstrates um in the public sector we've had a national security Focus that has been I would argue too narrowly focused on the cyber what in this community jokingly we say pew pew right a little bit too much on the Cyber pew pew aspect of it and on the enforcement of confidentiality failures of systems rather than looking at for example how serious the availability harms and availability failures can be in terms of Simply the maintenance of everyday life for

increasingly millions of people around the world and the rates at which particularly body embedded devices are being adopted in medical but sort of um medical contexts where there are non-invasive Alternatives available it's something that really gives me pause as to whether we're building more than just additional medical infrastructure so there's really no more bright line between National Security concerns and civilian safety concerns when we're looking at the way that threat actors and Insider attacks can happen and on the ground impact human bodily safety Financial safety infrastructure safety and the future of our own economy and Country and democracy and this is going to get progressively more problematic in AI context so let's start to think about

how we can retell this Story Once Upon a Time very long ago it was 1999 and there was a Y2K crisis that was ending and this story and if any of you were at my uh Defcon policy talk last year you know that um I'm really bullish on retelling the story of Y2K because I think that the people who were embroiled in it at the time suffered post y 2K traumatic stress trauma and didn't um uh want to talk about it then as much but maybe now enough time has passed that they can talk about it um there was a successful hole of government respon with hearings and plans and two statutes were passed and

public private sector cooperation worked there was actually very little disruption in our infrastructures and in the economy there was litigation anyway there was a lot of litigation but it really could have ended differently and that's the part of the story that's getting lost why 2K is too often a punchline it wasn't a punchline it was a success story and we need to reclaim that story and retell it every time someone says that response doesn't work security is isn't worth investing in this is one of those case studies that you can point to concretely but something else I've been thinking a lot about lately is the first generation of viruses and worms and they weren't motivated by intention to steal but they

still had huge availability impacts and so when we look at the way that we thought about these two scenarios in 1999 Maybe by putting ourselves in in the time machine a little bit we can reconnect with earlier versions of ourselves and start over reclaim the next 25 years because really I think it's our last chance in terms of making sure that we don't suffer an e economy-wide catastrophic security related event that isn't as easily recoverable as the uh availability incidents that we've been recently dealing with so with the Melissa virus stuff there's one um prosay case where someone accused a private party of talking smack that he was spreading the Melissa virus no big deal but with the Y2K cases despite the

fact that we had two statutes there was substantial litigation but it got resolved and through a lawyer's eyes that's just normal there's going to be litigation people breathe there's litigation that's why you have lawyers but here's what we should think about in particular when you look at the Y2K act even though it was tailored narrowly about a very particular event with a commonality of interest that was narrow and we cannot pick it up and wholesale apply it to our current complex context of security Dynamics we should learn from the way that they thought through some of these issues and they may give us Colonels to chew on and to think about Colonels in the corn sense not in

the computer sense to think about how to uh move forward in the way that we talk about our current security issues so they Define material defect they explicitly Exempted any claims for physical injury that's big they created a special contract right and for most of you this may not be particularly interesting but from a lawyer's perspective this is huge this is what I saw this reminded that myself that this existed it was a little bit of a mind-blown situation we can talk about that in contract nerd study hall later um they were conscious of small business concerns they were conscious of the Dynamics of various control entities they were conscious to take out Securities litigation and enforcement by

other agencies and here's the one that I think May resonate most with this community they developed a process for including true expertise for the most complicated cases so there was a way for a judge to appoint a special Master to appoint someone to help unpack the toughest questions so here are the lessons context really matters if you focus on each of the issues that's understood and approachable you can generate cooperation and you can build a process that includes experts input harm comes often from disruptions and there will be potentially deadly effects unless mitigated and so the key question that we're facing now is still are the the problems at hand the incidents at hand disrupting the

confidentiality integrity and availability of systems and will those impacts be felt by particular groups of people who may be particularly vulnerable and may experience particularly severe harm physical economic Etc and finally intent and knowledge calculus in law is nuanced and we'll we'll unpack this but this is I think the main thing that I hear Executives not understand that intent is not just what you do know or what your lawyers told you and and can be proven with documentation they told told you but it's also what in light of your role you should know and what other factors around you should have informed you if you were paying attention that is the best course of conduct that a reasonable

person in your situation would engage in all right so back to that talk last year I kind of uh set out the the warning Beacon that liability is already here liability is coming that non-liability actually I would argue a form of legal Tech at that we can't keep writing on and that it's going to take various forms private litigant civil regulatory action and potentially criminal harm and then I wrote this post in January about boot loops and security and I made some proposals including the one about the Bureau of Technology safety and um the importance of messaging the importance of having those interventions the importance of a new agency and did not anticipate that uh we would be

facing an availability uh incident U that penetrated the consciousness of everyday folks so increasingly when we talk about security I would encourage us to think about security as the lynchpin of Technology safety and that the whole Enterprise is about keeping people and things safe that is a framing that resonates with policy makers and with everyday folks it will resonate with your grandma it will resonate with your aunt it will resonate with your 9-year-old and the best test for whether we're messaging well and whether we're talking about our own life choices and careers well is whether we can explain what we're doing to a nine-year-old they are a very tough audience as someone who just spent brunch with a 9-year-old a

couple weeks ago earning her attention was a trick so telling things in straightforward ways that make sense is um a skill that um I think it would be important for certainly myself but I think the more we can all do it it's helpful so the push back that I always hear is oh but what if it's just a defect what if it's not a malicious attack what if it's just an Insider error context harm and intent will capture all of that because ultimately the focus is still on the potential harm it's still a safe incident even if it was an accidental safety incident so for example the one that's top of mind we're all familiar with the

recent incident that uh resulted from um an update that was pushed um by crowd strike um and so $5.4 billion of damage is one estimate 25% of the Fortune 500 disrupted according to this estimate um and 100% of Transportation sector disrupted allegedly according to this estimate and the banking impact was one that particularly concerned me but we also had 911 systems disrupted we had hospitals disrupted flights disrupted we had again with the banks disruption um and this lasted for um most of the day at at least in some places um and this was a blood supply problem for example in the case of New York blood center so uh assuming all this reporting is correct there were

many different kinds of entities that were impacted and the mayor of Portland announced a city-wide state of emergency that's bad in terms of potential impact when a city feels then this is not a debatable technical thing this is how the city feels about its own State of Affairs the city felt that it was in a state of emergency when normies feel they are in a state of emergency that's definitely a safety issue regardless of how we frame the technical discussion that underpins that safety issue and indeed um sisa um and kudos to them released um a set of uh reports and updates about the incident including po incident review but we also saw that other kinds

of uh unexpected follow-on activity was visible including some scammers using this as an opportunity to push out new forms of malware and to exploit this uh public concern in ways that were advantageous to them as criminals and frauders and again damaging to safety of the public so of course the social media conversation in this community was um aggressive shall we say and this meme pointed out the experienced reality of these availability incidents and this is how normies perceived it too they didn't they didn't necessarily understand what was going on at first um they were worried there was this was some sort of large scale attack um and it got to the point where um AARP was sending out

emergency bullettin to the seniors to tell them it's it's okay it's going to be okay right so when you have panicked senior citizens it's a safety issue okay so I found this interesting from a legal standpoint the characterization of this update incident as not a security incident but as a defect so defect is a an an interesting legal word there um now of course the commenters you all among them no doubt we're quick to point out that an availability problem is one of the categories of the CIA TR Triad right so there was debate about this characterization naturally but there were consequences for the entity that again to a lawyer's eyes immediately trigger the likelihood of certain kinds

of litigation whenever you have an 11% Plunge in value uh you start to anticipate the likelihood of suits from shareholders because that is commonly viewed through lawyer eyes as potentially enough above the threshold of what would constitute a material change for Securities laws purposes so you're dealing with a public company there's that layer of Securities Law and likely a corporate law follow on but I'm getting too much into the weights but here's something that I just want to share as a little signal that I saw this headline and I was like oo David boy is a very expensive lawyer so when I saw that Delta was investing in David boy that was a signal to me that they were very serious about

this litigation and so we also have other pieces of the legal ecosystem that are starting to flourish around looking for opportunities with potential plaintiffs so the lawyers have arrived in ways that they hadn't 5 years ago even so here's just a quick sketch and I didn't get a chance to run the search again last night to see what was new but we do in fact have a shareholder suit that's been filed uh in um connection with the crowd strike incident um Delta has of course signaled uh their interest ensuing or settling um and we're likely to see tort meaning civil wrong suits and lots of contract litigation potentially some infrastructure disruption issues potentially some regulatory forcement

and potentially some physical harm suits so um sadly I know of one case personally where a friend's teenager who's um uh addressing um successfully happily addressing lymphoma could not reach the hospital and had a time sensitive medical issue going on so happily that case worked out but it makes me think that maybe there's the potential particularly in light of the hospital disruption of some cases that perhaps um will be brought in connection with harm so suits will be filed under like under various theories of Law and this isn't just about the crowd strike litigation at this point I'm talking generally about this kind of scenario so this is not me trying to harsh on crowdstrike um

there will be various theories of law that will be filed that was the point of the last Slide the litigation will go on for years and there will be appeals and that will go on for years this is a multi-year thing that will be very very very expensive for everyone and very disruptive to business ounce of prevention pound of cure really that's the thing that just keeps sticking in my in my head um Whenever there is an incident um that involves something that is preventable um and we will see to what extent that is true here as the cases progress now um some claims will be delayed because law world does not move fast on a typical contract suit in

most cases under most state law because contract law is state law that means lots of different state laws but generally you get up to two years to file for a contract soon that means there can be a dribble of various litigation happening for quite a while um and some settlements will be paid out to end threats of litigation in these kinds of situations and that's a prudent choice in many cases depending on the particular facts and circumstances the context the harm and the intent knowledge are always going to be those three magic variables so um sometimes you end up in a renegotiation or a contract termination and the parties go their separate ways and also we might have

some regulatory action there is at least one enforcement action for from a financial regulator cfpb that focused on availability issues so we might see these kinds of enforcement actions uh get brought in terms of the various Regulators um scope of um Authority under their enabling statutes so um this will be an exciting time we're entering into in terms of litigation and lawyers so when I talk about context what I mean it varies by place and jurisdiction it varies by the community and how they've defined harm in statutes and among themselves it varies in the emerging effects that happen sometimes you can't necessarily anticipate fully what the context will hold for you it varies across time law evolves and so

does technology and so do the ways that people interact with them so the question is one of suitability of the design design and the internal controls for that context so let's come back to some bears we're all familiar with these hacker Bears uh for better or worse but you might not be familiar with this hacker bear who let himself out of a zoo multiple times and needed to be moved to a different Zoo design with a moat you might not be familiar with actually this one was pretty widely known but nevertheless you have to respect the bear you might be familiar with this bear who has uh a dedication to pursuit of lasagna who led himself

into a Connecticut home opened the freezer pulled out the lasagna and left without incident mission accomplished you might be familiar with this bear who just likes to hang out on someone's picnic table in their backyard different context different threat profile different possible harm you might have heard about this Alaska bear that decided to just hang out on a roof until the Skylight collapsed so that he could fortuitously get access to cup cakes different context you might have heard about this Canadian bear that broke into a c guzzled 69 cans of pop that's a very different threat profile on that bear than the one that walked 200 kilometers for Revenge so in Ontario there was a

bear that broke into a set of cars not once but three times they took him away the first time he came back back and they are sure it was the same bear because they tagged him the first time so this bear walked 200 kilometers for Revenge that is a very different context a very different threat actor and a different response and a kind of Amplified harm because I think they're down four cars now so context matters and the the owner of the car cars in this case said normally they're no problem but honestly this one was known to be bad reputation matters too so harm so I've been doing some research looking at historical catastrophes broadly defined

and this is still research in progress but I wanted to preview some of it because as I'm going through this increasingly there's resonance with the types of issues that we think about in security so if you know about a local catastrophe where resulted in connection with an engineering safety choice or lack of choice and please do share that incident with me because the resources are not complete they are kind of scattered and not necessarily curated in ways that are tailored to this framing but I think this is an interesting framing for stimulating discussion for us so these are catastrophes where death resulted and there's an engineering Choice design choice that was involved so far I have

over 120 in my sample it's going to keep going up um and so basically this is a traditional uh social science methodology I qualitatively coded each of the cases in the sample using um in in my case uh a basically a binary a yes no question around whether there was an uh an engineering safety problem that arose um leaving in the ones that that did and what through my eyes a court would determine to be the cause the butt for causation the underlying cause they're just legal footnote then in um Tor law in particular in civil wrongs there are two kinds of causation but for cause cause in fact and proximate cause proximate cause varies a lot across jurisdictions

and people fight that one out C but the butt for causation less controversial usually and so what this started to crystallize for me is at least 10 lessons that I think this research might um be useful for stimulating discussion in this community so I kept seeing a repeating pattern of failures to remediate technical debt failures to finish in complete projects we have Bridge collapses with cracking cement partially because of uh things falling through the cracks no pun intended in terms of addressing incomplete parts of the design there are warnings that are ignored repeatedly users testers employees uh even the actions of neighborhood children sometimes point out that there is a problem and and management has actual

knowledge of the problem but chooses not to act for a panoply of reasons there's also a recurring trend of failures to usability test for human operational error so the interactions that are going to happen on the ground and even when there is a human in the loop they're frequently set up to fail in terms of the way that the controls are set up or the way that they are likely to get distracted by something in their environment sometimes you even have fail safes that are in place but they get turned off and so we see train derailments we see oil platform explosions that fall into these categories um we see a recurring pattern of failing to threat model to anticipate

failure scenarios in foreseeable contexts of deployment and in lack of a planned response so grain silos and molasses making operations frequent ly apparently have exploded um and so if you were choosing to engage in a line of business that has a history of these kinds of events depending on how you design your plants and Technologies um that gives you a heightened need for thinking through in your threat modeling how you're going to address those things regularly I'm seeing things that are built too fast and unsafely for financial and profit reasons there's a motivator that's an external deadline that has nothing to do with engineering quality nothing to do with engineering safety but there is some sort of

financial Target of opportunity they're trying to hit and people choose to look the other way on safety to hit the financial Target there's a failure to test appropriate tools and materials so supply chain issues and we see this happening with Construction crane collapses regularly which is still a problem um sadly um that we're we're working through design choices sometimes conflict directly with safety and those are sometimes done for uh aesthetic reasons something looks nicer in a certain way um or the person who is making the call is not up to dat on what is considered best practice at the time so they might use for example in building contexts a kind of plan layout or um

uh choice in the build in in basically putting up the building that is faster but is known to have serious safety implications um so that's a design choice there's also a regularity in these incidents of deviations from plans so even when the plans were made made correctly in the first place something happens on the fly in the course of deployment that causes someone to cut a corner or to not point out or correct uh a defect in fact as it's happening because it's kind of annoying to fix things and so convenience sometimes leads people to cut Corners leading to catastrophe building specs um are one type of recurring problem here failures to maintain or respond respond

to incidents adequately Bridge collapses have regularly happened due to neglect and people just forget about checking out if the bridge is still okay because that's a maintenance thing not a splashy headline thing that's a going to the dentist kind of thing but you know we all need the dentist and finally a failure to adjust to emergent changes environmental changes or interactions with the Technologies themselves it's how we had the key bridge problem the barges kept getting bigger and the Bridges stayed the same across time and so there was just less and less of a space for uh forgiveness in the driving all right I will I will uh give you a quick example of a counterintuitive case

that um I found particularly interesting 1919 had 21 people die from a molasses explosion in Boston The Operators of the plant blamed the incident on Italian anarchists however locals had observed structural issues with the Molasses tank in fact we know that there was a wall of molasses that was powerful enough to destroy a fire station and it was in January and so 21 people ultimately died some from the initial 50 foot wave of molasses which was not slow and then The Rescuers got injured and some of The Rescuers died because of these circumstances so there was a supply chain issue in that the Molasses had been or at least the inputs to the Molasses had been transferred without

being allowed to adequately cool the tank itself was known to be too thin in retrospect but even at the time it was not to Speck according to the engineering experts who reviewed this the workers in the plant told their bosses hey we've got a leak the neighborhood children keep coming around to steal molasses from the holes in the vet that's a sign but because this was in connection with a speedy window of production for the War uh era they were so focused on meeting those deadlines that they painted the Vats instead to hide the problems with the cracks so there was also a legal angle this ended up in litigation the company was ultimately found liable and one of

the quirks was that it had been licensed the Vats have been licensed as receptacles not as the standard full permitting process for a building and so this caused a shift in the way that the engineering profession and Architects did their work going forward it's a tributal mostly to this particular incident uh in the way that the engineering historians re it and created a sense of more of a professional duty to safety and independent reporting of of incidents but we haven't solved the Molasses problem fully but we have made it less deadly generally so 2013 we had a pipeline issue in molasses in Hawaii and there were no deaths in that case but there was no human deaths but there

was mass death of Wildlife and some pollution from it so 100 plus years later we'll still we're still struggling with molasses safety but the interaction of technology today threatens to unwind some of the progress that we've made in the past because those tank issues might end up being over trusted to a piece of technology that itself is flawed and of course as we all know part of the challenge is that it's a two-way street the maintenance is ongoing the maintenance continues and a pushed update can change everything even if you have the original set of circumstances under control so um back to the intent question because I think this will interest uh you all in this audience

there's a slippery line that can't necessarily be placed well at the time of an incident as you all know between an act of an external attacker and potentially an act of an internal sabur or someone who had a bad day so figuring out where the reality of this particular context this particular set of harms and this particular case that is going on at the moment whatever it may be that assessment of what the intent drivers were what people were thinking what they were doing in their hearts it's not going to happen on the Fly you're focused on remediating the immediate issue and the incident response but retrospectively that's what courts are going to look at they're

going to look at who are you what is the role of your business in the economy what is the expected knowledge for someone in that role you chose that line of work you chose that business model you chose those employees whom do you control what do you control what choices did you control and for individual liability questions and there have been some cases recently that I know have been top of mine in this community with person personal liability issues you look at yourself your own professional history what knowledge you have been exposed to what knowledge someone similarly situated would have and should have and then you deduce a third party finder of fact objectively looking at the situation from afar

whether if that's a jury or a judge or an enforcer says hm based on what I see objectively do I think this person made a certain kind of choice and then that determination about the state of mind for that choice guides the determination of what legal consequences follow so in civil contexts you don't have to foresee the full extent of your harm and this is one of the biggest misunderstandings you just have to intend to engage in some conduct whatever harm follows is on you you made the choice to take that step what happens after is attributable to that step so that's why it matters whether you had knowledge whether you used care and what you promised people

in criminal contexts there are specific levels of knowledge that are listed in statutes and it's one of the things that legislators fight about and lawyers fight about whether you've met that level of knowledge and they're also tort meaning civil corollaries to most of the criminal causes of action so here's as promised a walk through homicide we have 50 plus versions of homicide statutes potentially this is a general intent crime what does that mean it means that the actual intent to do some act as I was just saying is the operative question not necessarily whether you chose the result so there is murder which is by degrees first second third that varies by jurisdiction what that means exactly

and what's entailed and which ones they have there's felony murder which means someone dies in connection with your committing a different felony there is also something called depraved heart indifference which means that you functionally take control over a situation where you know the person is not going to be able to defend themselves and then they die that is a different category that falls into this murder Spectrum there's an intentionality about it but intention can be expressed or implied the magic language of intent in criminal context in this murder context is malice of forethought it's just the way we talk about intent there's some common defenses but there's also a category of manslaughter which means that you knew you were doing something

but the full ex the the full nature of what you were doing was perhaps not as easily anticipated by a reasonable person so there's voluntary there's involuntary there's vehicular there are other special kinds and this is all specified by Statute this has come up recently because of Boeing so as we all know there was a sensor issue with the Boeing planes and there was software implicated and the families of the deceased filed wrongful death claims there was a lot of litigation Boeing settled nearly of it so we don't have a lot of case law on it but we have reporting about just in the one flight there were 171 people's interests represented in the lawsuits that were filed and 140 of

the of 150 claims in the northern district of Illinois were partially or fully settled so that's just one jurisdiction we have a chief executive who had committed under oath that he had knowledge of whistleblower uh complaints and potential retaliation um I watched that testimony the question that I had uh was um why that particular CEO wasn't more curious about the past history of whistleblower complaints um so we also have testimony from um the FAA about expertise deficits in inspection and so this is all connected so just to finish up here won't this kill Innovation she's talking about liability she talking about new steps with regulatory agencies won't this kill Innovation it's time to Define that term

because that's a magic slippy word and when we look at what happens in other Industries things have worked out so the question is who's progress are we advancing is it just novelty for extractive profit that we're building our Tech ecosystem or is it a case where we're building something that is truly making humans lives better whose story is this whose progress are we centering and you don't need to take my word for the risks here take President Eisenhower's he warned us about the emergence of a technology Elite that would present problems for continued governance and stability of our economy and Country this is the second part of that military-industrial complex set of comments that's often oted but rarely do

you see this piece of that speech quoted and I think this is the piece that we need to uh perhaps Ponder in our present moment look at other Industries they have targeted agencies they have enforcement they have ongoing commitments they have various different licensing regimes the level of oversight with many eyes looking at a problem is just completely different from what we have in the tuck ecosystem and the other challenge is that of course as every company becomes a tech company companies that are regulated by these standards will have their own ability to successfully manage their own internal controls face challenges in light of technologies that are not necessarily in their control in the same way but that

they are perhaps naively over trusting with access AI of course will make things even worse you had a wonderful keynote on this yesterday so I'm not going to go through these issues but here are some questions from this so won't these problems solve themselves is the market perfect why don't you trust the market Andrea well um I know a lot of humans and I've been a lawyer for 25 years the Market's not going to fix this and 25 years of security and watching this community and the catastrophe research that I'm doing tells me the market is not going to fix this also bears bears tell me the the Market's not going to fix this if you have not read

this book I cannot recommend it highly enough it is the story of a bunch of folks who shared uh aggressively libertarian views who tried to set up a Utopia and things were going relatively fine until bears bears showed up and destroyed everything and so emergent effects and problems matter and arise and individuals won't be in a position to address them and always for better or worse there's always that one guy who let's say finds a dead baby bear CB drives it to New York stages a scene where it was allegedly hit by a bicycle and then runs away or walks away maybe maybe he walked away drove away I don't know leaves it for someone else to

clean up and so in fact that's what happens and that is why the idea of having coordinating points in government is sometimes necessary um and so with that I I will return to my suggested approaches um the law fair post I would love it if you would read and comment to me about what I get wrong I always want to hear that um so in particular what I think we need apart from those interventions is an agency focused on the biggest actors in our economy so um am using the heart Scott rudino standard for those of you who are really in the weeds of competition law so it's the biggest companies and uh my ideal

situation would would be a bureau of Technology safety with three branches an enforcement division a policy coordination technology Futures tracking Division and a pilot projects Division and that is it thank you for your attention [Applause]