Security, Politics, Neutrality, and Protecting Users

hello so yes you're all welcome to heckle me on twitter my twitter handle is USS joined it's also at the bottom corner of every slide however since I'm in the IMAX theatre that means that those in the first five rows can't see it but it won't change from slide to slide I promise so I want to talk to you all about politics neutrality and the work we do as security professionals we have one job as security people and sometimes we tell ourselves this is it right I fight for the users which is actually a misquote the original quote in the original movie was that guy over there he fights for the users but this isn't actually our job I mean super cool

to see yourself as Tron guy acting as though eager to duel to the death with a giant panda and a fancy bear and a grumpy NSA but this isn't actually Fight Club super cool cyberpunks effects edition now with more lasers that go pew pew pew pew the problem with I fight for the users is that it focuses on how we're not the users how we're cool and we're separate and we're laser enabled and we're cyber whatever that means I think that's an issue but it seems really appealing at first to many of us the ones of us who grew up reading science fiction focused on how one day we'll get to fly away and upload our

consciousness into this cloud or go seek out new life and new civilizations among the stars technologists have been told for decades that we're the ones who get to save the universe and while most of us don't exactly believe that like not consciously the meme that were special and different and above all else separate from the normies has persisted for many of us and it affects our work and our relationships with others and it affects how we see ourselves as a profession this separation from the users is what we as technologists and particularly as security people pretend to makes us special somehow when we refer to users is the weakest link and when we hate on people who can't be

trusted not to click on what we're really doing the setting ourselves apart because after all we all know not to click on things oh I've never successfully clicked on a fish and if you believe that don't talk to any purple teams that companies I've worked at so when we're different when we tell ourselves we don't need to get involved we're above the fray we're above the pesky concerns of mere mortals and we're neutral in other words so this talk is gonna discuss this meme in the detail what neutrality and separation really mean a bit about the history in tech what this meme causes for our relationship to society and what we need to do now and

this isn't gonna be a real good talk to give a real big disclaimer for so here's mine I do not speak for my current employer or any of my former employers my clients the staff 1962 World's Fair held in Seattle that built you the fine thing guru is the space Neil and also a pointless monorail or the cast of the movie dr. Strangelove or how I learned to stop worrying and love the bomb I'm also not your lawyer which is just a generically a good thing to say this talk is not about legal advice but please if you see parts of this talk and think their legal advice to you seek professional help immediately so neutrality as a concept is actually

tied up with another concept from law of sovereignty to be in the sole control of your destiny so we're gonna start by discussing how sovereignty came about and then we'll zoom in on how neutrality ties into sovereignty so the eighty Years War in the 30 Years War occupied most of the 17th century along with all the side Wars that surrounded them and they're collectively some of the bloodiest wars in European history including modern works we think are super bad like world war ii so in the 30 Years War alone more than eight million people were killed and understand that the population of Europe was a lot closer to eight million than it was now they were were there were several things

the right of religious self-determination being a really important one with both Protestants and Catholics invading each other to protect what they saw as the rights of their religious adherents so Catholics were invading Protestant countries to protect the rights of ethnic Catholics in those countries and if that seems like a really familiar excuse to you you're right it's also exactly the same excuse to Vladimir Putin use when he wanted to invade Crimea no I don't just want a couple more deepwater ports it's really important to me to protect the rights of ethnic Russians in Crimea and if you believe that I have this wonderful bridge to sell you because these wars and all these side conflicts were so much of an everyone

vs. everyone war it took three separate treaties to end them these three treaties are collectively referred to as the Peace of Westphalia and dated to more or less 1685 but there's some new on on that and they they comprise the piece of Muenster the Treaty of Munster and the Treaty of on his Brooke generally what happened under these treaties is the Catholics and the Protestants split into nations but where they didn't the Catholics were required to protect the rights of Protestants in Catholic territories and vice versa so these three treaties were remarkable in a few different ways one was because they weren't negotiated by Kings previously treaties had been negotiated by Kings standing on a bloody battlefield Sun

surrounded by the dismembered corpses of their various armies instead for this thunder-like that seemed a little tawdry so instead let's appoint representatives of the various sovereigns and we're gonna send them to diplomatic conferences at various cities across Europe this process of negotiation and the entire way in which it was negotiated formed the basis of the modern international diplomatic system the other thing though most important way for this talk is that they recognized and incorporated a new concept that all Kings were equally kingly that is that they were equally sovereign and so within their territories a king had internal sovereignty they could do whatever they wanted within their territory outside of their territory they weren't the king

full-stop as opposed to they weren't the king until they raised a new army and invade all your stuff which had previously been the system under which everyone in Europe lived so why does this matter it matters because it enables things like this exact photo to take place if sovereigns are equally sovereign that we can have diplomatic relations between powers that respect internal differences while recognizing countries generalized equality on the world stage and that's true whether it's an enormous rich country or a power backwater and it's true when the formerly poor backwater becomes one of the richest countries in the entire world and when the former richest country in the entire world decides to cut itself off from all of

its trading partners inexplicably in order to make Britain great again and then suffers a famine in six months they're still like Queen Elizabeth is still going to be a queen she's still going to be the sovereign of the United Kingdom whatever is left of it and this also leads to another thing this is why countries can agree to give up some of their sovereign Authority if they want to so for instance almost every Western country has signed a thing called the United Nations Convention on the Rights of the Child which prohibits things like child soldiers some countries the United States chief among them have refused to sign this treaty because we really love signing sixteen

year olds up to join the army we call it JROTC all of Europe thinks this is horrible and there's nothing they can do about it because we're sovereign inside the territories these sovereigns though people when entities have to follow the law of wherever they are so if you're present in the kingdom of the plastics and they mandate that on Wednesday we wear pink then you have to wear pink on Wednesdays when you're doing business in their kingdom just how like now if you do business in the United States you have to follow us laws but you don't have to follow say the laws of Khattar unless you're doing business in Qatar which seems obvious but it becomes

really important later so remember that so that's what sovereignty is so now let's talk about neutrality it's a real simple addition because neutrality is incredibly tied up with sovereignty your travel needs a statement of just two things one we are sovereign which means we are fully separate from you for all values of you so you have no authority over us and secondly we're not going to get involved with your crap it doesn't make sense to say you're neutral unless you're sovereign if you're not then your allegiances are already written for you so we're talking about tech sovereignty and tech neutrality well is tech sovereign well they just said no it's not because there's no land of tech

utopia where only technology rules despite the most insane ravings of Peter Thiel and the sea stutters by the way Peter Thiel in the sea stutters is my Beach Boys cover band that only plays I honor and inspired rewrites that said it's not like tech is sovereign hasn't ever been a topic of discussion John Perry Barlow is one of the people who claim to have founded the Electronic Frontier Foundation said back in 1996 that tech was naturally independent of all governments and they had no moral two rules and you have no sovereignty where we gather this was somewhat cheeky it was also pretty ironic considering he said it just a couple weeks after the close of the World Economic Forum in

Davos and he said it in Davos it's hard to see a larger gathering of sovereignty and wealth and power than Davos but okay cool you do you unfortunately that idea didn't quite die 20 years later so now in 2016 Brad Smith the general counsel and president of Microsoft said the same thing the tech and especially Microsoft should think of itself as a digital Switzerland where the largest companies just off top my head Microsoft would be immune from those pesky within our borders laws and get to define what countries processes they would comply with even if the expense of their host countries laws this is an interesting thing to say Microsoft was trying to sell this concept because they wanted to

hold themselves out as the lone staunch defender of privacy rights what they were saying was well you can trust Microsoft to not comply with say authoritarian Chinese demands for your data because Microsoft is so big that we shouldn't have to comply with Chinese laws ok cool beans Brad but the problem is you're also saying we should be actually sovereign so we don't have to comply with pesky US laws like minimum wage laws or labor standards or OSHA or whatever but let's focus on the separation Microsoft's at the tech in general and Mike are supposed to soft specifically could and should be fully separate from lost politics in society so that it shouldn't get involved with

the concerns of mortal this proposal was of course roundly mocked when Brad Smith made it in 2016 and yet the future is here apparently because last month Microsoft appointed its own representative to the United Nations which should really scare you if it doesn't it's just because you decided the UN has failed and like I hope that's not true even though the United Nations does have a pesky problem where its host country ie us breached the host country treaty which is a little weird but okay so Brad Smith thinks of himself as a un statesman and on a broader no neutrality require sovereignty as this persistent recurring meme at both the kind of crazy hippy level of the FF and the deeply and

terrifyingly powerful level of Microsoft's general counsel that text should be neutral and therefore sovereign so what like what's the harm really in fulfilling Brad Smith boyhood fantasy of getting the United Nations seat for a giant paperclip with googly eyes well let's go back to the mean we discussed in the beginning the separateness to neutrality of Technology this neutrality means that Tech almost always considers itself above politics the theory is that tech occupies a perfect amoral space and stop me if you've heard this one before technology responds only to what is possible and therefore the uses of technology that seem that are not technology's fault but societies and of course since we respond only to what is

possible we're therefore the natural kings and queens of civilization so we should have everything including giant slides run into the middle of our office we can certainly see this idea in a story from a month ago from NK jemisin who's a fantastic science-fiction author if you haven't read her books you should so she was invited a couple months ago to a big tech company to speak on one condition that nothing she said made anyone there uncomfortable and she didn't talk about politics in any way what an amazingly infantilizing statement towards all the engineers and people who work at whichever large tech company this was I don't know which one it is by its management these

technologists and their management are demanding the world in all its complexity use their technology but they're so fragile about that demand that we can't make them uncomfortable in any way by pointing out say that their technology has political implications that might make them not enjoy the slide as much and where would they be there and I don't know which tech company it was she didn't say and to be honest it could have been any of the usual suspects but this meme is really pervasive and we can certainly see it in this quote from Google's management so Meredith Whittaker was the head of artificial intelligence ethics for Google and she went to Google and said hey it seems really bad we're building a

massive persistent surveillance space and then selling it to authoritarian regimes and gules management said hey don't worry about it technology is like a hammer it can be used for good things or for and things and as she went on to point out that's true but maybe we should start asking questions about the political implications of selling hammers the government Department of hitting people with hammers or I'm sorry that was unfair I should really use their full legal formal name the government Department of locking minorities and cages and then hitting them with hammers to make America great again so some companies are directly abating human directly aiding human rights abuses by choosing where to sell their software

but I think there's a lot of harm being caused by more self-consciously neutral actions the ones who really honestly believe were not taken aside so I'm gonna use github as an example of this next issue because this came up a couple months ago with regard to github but I want to be clear that this is just realist reach and github isn't alone in taking this self-consciously neutral approach so many technology companies get up among them will comply with any valid court order issued in any country not just United States ones because they feel they should be neutral now most of the time these companies are not actually bound to follow court orders from other companies because they're not

present in the country github is a slightly weird case on this because of Microsoft and there's a lot of questions as to how separate github really is from Microsoft but they've had this policy for years and Microsoft only owned them for like 14 months so let's set that issue aside for now so there was an encrypted messaging app called tsunami Democratic in the fall which was set up to help protesters in Barcelona organized coordinated protests against the Spanish government the reason they were protesting is because the Spanish government literally locked up all the leaders of a political movement all of them because they didn't like them so they charged some old terrorism and threw them all in jail so there was this

website and there was this app and a Spanish Court instantly ruled you have to take down this website and you have to take down this app Google immediately took the app off of the Google Play Store but tsunami Democratic was like that's okay we're young youthful tech protest movement we can teach people how to sideload apps and you can download it off of github so a court-ordered github to take down the app and github immediately complied even though again they're not in Spain because that was their stated policy and because they don't believe that they're in a position to be able to decide among differing places we should be neutral which would be the politics and to be clear github does

one very right thing by publishing government order to take down to the public get repository you could literally see both of the orders that you have received on this issue it's at github calm slash github uh slash gov takedown and slash twenty-nine Tre slash Spain slash 2019 it's a new country then date so that's super cool and I don't want to make you think this is just about yet hub though I kind of obviously disagree with this decision of github and to recap remember companies in Spain have to comply with the laws of Spain now I might prefer that they take the civilly disobedient action of ignoring a clearly illegal court order to suppress a democratic protest but

today's talk is not on the duty of civil disobedience in response to an authoritarian government that's a different talk but company's not in Spain get to laugh and ignore clearly illegal court orders like this so this behavior on the part of most tech companies and most tech companies do this and most tech companies are US for now at least is entirely voluntary they're volunteering to help suppress protest movements under the guise of neutrality they're volunteering to help take down controversial content less they upset the Chinese government and so on so the companies that really fundamentally believe in this neutrality and a lot of companies do it's not just business experience this is a sad problem they might even write a tweet

about how they hope that the Democratic situation in Spain will improve itself without any intervention on their part of course but apparently it's more important to them to be separate from real people's concerns and we hear this kind of response continuously from tech company is confronted with ethical concerns over how their tech is being used from any vision scraping every photo on the web or every photo that's ever been uploaded to the entire web so they can make a massive facial surveillance database and then sell it for pennies to random cops without informing them of like the accuracy rate of it which seems you know weird to NSO and dark-matter who openly sell attack tooling to repressive governments to use

against their local activists the geo group who just profits from internment hey if you don't like internment of a minority group it's society's problem if you want to change our behavior we're just following the laws you've got to change society it's never our problem of course there are also companies that move beyond simple apathy or pseudo apathy towards direct harm so we have companies like Palantir in uber who already ignore laws in every country uber quite famously has a big red button that enables you to destroy all the evidence as the cops show up in any foreign country and Palantir of course who in addition to their CEO having said a lot of really concerning things about

Western supremacy on video interviews in the last couple years also named themselves after a fictional mass surveillance system without noticing that in the books the system irredeemably corrupted the souls of everyone who ever touched the mass surveillance system I felt like Palantir should be be read a book all the way to the end sometime I might suggest that they start with IBM and the Holocaust but that to me so to sum up this is the behavior of the tech industry as stated okay not stated this is a completely made-up quote but it's a great song and you all listen to it by Tom Lehrer once the Rockets go up who cares where they come down that's not my

department says vandervon broad but I will state that this is not acceptable behavior for those of us who work in security because we have to care where the effects of our actions land so we've talked about the meme of neutrality intact we've talked about why it's both miss founded and worst causes quite a lot of harm so what do we do I only give you part of the NK jemisin the story before so here's the key bit she said there aren't politically neutral spaces people are involved with technology so technology spaces are inherently political and I would add to this that again pretending there are these politically neutral spaces because we're above the fray directly infantilizes us

as technologists and harms us as members of a civilization that really needs to spend some solid time coming to grips with what we've done what we're still doing and what we plan to do next so how many of you by a quick show of hands in this theater can actually see some of you have heard the phrase it's more important to have a seat at the table when your company is just a fine doing something you disagreed with anybody yeah forty percent of the people in this audience which is maybe a little self-selected let's be honest so this often comes out is well we don't agree with all the government or enormous corporation or whatever's actions or

policies but we believe that by having a seat at the table we'll be able to effect change from within so okay seems fine on face but pinboard who's wonderful they should follow him on Twitter play method the quiet push from change from within is never-ending an insufficient the problem is the people who say this kind of thing always want to protect that seat at the table as it lets them be with the cool kids they never want to actually take the stand but they never really make a red line where they're saying this far and no farther so instead they continue to enable enable the behavior and never change it Eva galperin who is chief

security officer and chief a bunch of other things I can ever remember but anyway huge badass Electronic Frontier Foundation delivered this quote the choices you make in your job matter if you're a technologist and she was talking about engineers in an electronic medical record system called practice Fusion who designed opioid pushing into the electronic medical record so every time you doctor entered something that might cause pain like a broken bone into the EMR would say hey maybe you should prescribe them opioids and the reason they did that is because this feature had been paid for its implementation by a major opioid manufacturer so then lots more people got addicted to opioids when they got cut off lots more people got

addicted to heroin and that has certain negative consequences Joe Redman is a famous computer vision researcher and he designed the Yolo or you only look to computer vision algorithm which made a lot of mobile facial recognition faster and cooler and he became real world famous enough to get his own full conference TED talk and then nerd famous enough to get Google to pay for an entire PhD for him to study computer vision at the University of Washington which as of a few days ago he's announced he's stopping forever because he's like it's all military applications there are no good applications of facial recognition and computer vision left it's only harmful ones and he specifically thinks it's

impossible to say you shouldn't look at the broader societal impact which was the debate this was having on the internet I said before they don't like I fight for the users of statement for security professionals this is what I prefer the whole that's not my department idea isn't acceptable because our role is security is to be the guardians of trust we're not just hammers and our work isn't just hammering things we're the sword that protects the world and our goal is to make the world suck a little bit less every day not to enable harm to our you they put their trust in us and indeed we're all users of these same technologies so we as a community put

our trust in ourselves we fight as the users and but we also have a larger goal and a larger role of protecting everyone so everyone in this room has the power it's the stunningly talented technologists you are to do better to care and to engage honestly with the work you personally are doing and then to make these decisions based on giving a damn even as the meme says a tiny potato can make a Gordon change so here's my call to action and it's very simple I want you all to figure out what the effects of technology that you're building are and then decide are you okay with those effects or not if you're not I want you

to change the effects change the technology or even can simply refuse to continue working on it but please stop pretending you're above whatever the effect of your technology are I want you to engage with your work as though it has moral implications because it does and then I want you to decide if you are morally okay with what you're doing now I have certain political leanings you may have been able to come across in this talk but if you don't come out where I stand on these issues that's okay too the important thing is that you recognize that there are moral implications to our choices and then our security professionals our highest duty is to protect the trust that users place

in technology and look it's hard a politics is hard worrying about implications is very very hard and requires a lot of talking which is scary and talking to people which isn't we're not exactly like us which is even scarier and change is slow and not inevitable I some days I have to admit I wish that Barlow stream or Brad Smith's adolescent fantasy were workable I wish that we could all focus on simply the beauty of the baud and all the things we've studied and practiced for years to just hone our craft as security professionals but it's not possible to do that and it's not right to do that the truth of the matter is the

technology is not separate from society is an essential component of the fabric of society now more than ever before and when we make technology decisions as though we're somehow above politics we reinforce power at the expense of justice we enable surveillance radicalization destructive misinformation and the Poli a part of our hard-won civilization it is critical that as security professionals in particular we stop defending what tears of the fabric of our society and we move to defend what we can justify to ourselves if no one else as work that causes an actual good to happen in the world we're all in this together and it's well past time we started acting like it thank you I don't

know if we have time for slide out questions I'm kind of guessing we don't but you couldn't again all hit me up on Twitter at USS join maybe we have a slight a question what were they no slight up like thumbs yeah we have time for probably maybe one slide Oh question if anyone's got one the room again is besides SF or you can just yell it out just yell if there's not on the line so the question was what is the role of collective action in deciding what you work on and having a broader impact I think that's it's very important I personally would love to see more tech unions because I am that kind of a nerd

I think the collective action thing is another step beyond this and so I'm emphasizing trying to take the first step of just believing you have ethical choices but yeah I absolutely believe that recognizing that we have collective power as well as individual power is important to making sure that what we work on is valid and defensible as well as making sure that our colleagues aren't viciously exploited see also every article about the horrors that Facebook and Google reviewers have to deal with awesome thanks a lot Brendon