PW - Can a password management service safely learn about users' passwords?

today we have with us jeffrey goldberg from one password uh i'm gonna say you you're becoming a permanent resident of password scon more or less um and jeff always have interesting talks in my opinion uh he's a great guy to talk to as well i know he loves getting questions so prepare for every questions that you have for him after the talk or also initially in in the breaks afterwards as well and uh he's here to talk about can a password management service safely learn about users passwords take it away jeff hello okay uh you can all hear me um so first of all it's really great to be here it's great to see pear here it's

been a while since you've been in las vegas at these conferences um and as he said i've actually you know been part of passwords khan one way or another for a long time and i remember my first talk was in 2013 over the golden nugget and i was the first talk on the program other than pair's introduction and you know i mean i'm a dilettante academic that's really what i am i'm a phd dropout in linguistics from a long time ago and things went to sleep okay i hope the things that went to sleep are not you or at least not yet um do i need to plug this into power for this there we go okay so i'll just

i'll just do stuffs okay so i'll fiddle here awesome that's not what i wanted okay i won't do that um you know so so i came in i had no idea what the audience was going to be like or i had an idea and i was wrong so the first talk i ever gave at a passwords con was basically math from beginning to end and so this time i'm talking about something that could involve a lot of math and i've left all of it out so uh so uh anyway so that's it um now one of the things that pair just said i said that people are often reluctant to give honest answers to certain sorts of questions on a

survey um this is uh in some fields called the social desirability effect on you know in survey design and purdue and uh as it happens i'm going to be talking a lot about soliciting honest answers to very sensitive questions okay so let me just set the scene um i am uh jeffrey goldberg i've been working at one password for uh 12 years i can do subtraction um i really want to know everything and there are certain things that i absolutely do not want to know

and we'll get to that now one password is a password manager um we have client software that people run on their own devices and a service um used largely for synchronization but for other things as well the client software is where most of the action is this is all just background i should have a kitten up there i don't um but just imagine a kitten if you know all of this stuff um uh the unlocked client knows certain things like the names of your vaults and the titles of your items and those sorts of things the server doesn't um and the unlocked client knows the strengths of your various passwords to the extent that password strength can be

measured we've talked about that elsewhere and have other conferences um uh client you know so all of the all of the usual stuff and uh the overall design and architecture of the system is so that we know as little about user secrets as possible but we also think that we know a thing or two about password behavior you know maybe by relying on shitty surveys um or maybe not but but we we we certainly have opinions and we certainly believe that we know something about passive behavior um and uh broadly one of the things i'm interested in is what can we learn about a one password user's behavior without putting them at any risk and what technologies for doing so are

within our reach and so i'm going to actually present a sample research question because it it it gives an extreme case uh if we can understand and research this and look at this then we can pretty much do anything safely so something that i've said at previous password cons is that people who enable 2fa for their password manager will tend to use weaker account passwords than they otherwise would and i've given a couple presentations about that and why i believe that and other things about misunderstandings of what multi-factor authentication does and how those get implied and how that impacts the case of of using it for password manager it doesn't matter whether you believe me or not

doesn't matter whether i believe me or not um i would like to know whether that's true so in more research terms is there a negative correlation between use of 2fa for one password itself and the strength of the account password now i'm going to tell you i'm not answering this question today we don't know i'm it's what i am actually going into about fine that's a few slides later um uh we do know whether people have whether a user has 2fa for one password enabled because that is something that is done server-side um we do not know the password strength of anybody's account password um and we really don't ever want to know the password strength of anybody's

account password uh we don't want to put anybody or anything in a higher risk of cracking against uh against their password uh so i don't really think i need to list the reasons and that's also largely because i was putting the slides together on an airplane and i decided to skip this one okay so um uh so there are things that we don't want to know but we also want to make it absolutely clear that we don't know and that we don't have the capacity to know so not knowing is not enough people need to know that we're ignorant of these things okay so i've been thinking about this problem and this example and what you're going to get is a

more or less what's a preliminary progress report on my thinking with no actual answers to the questions and for those of you who are familiar with differential privacy and embedded in those communities everything i say here is going to be most of what i say here is going to be a review of what you already know um and so i won't be insulted if you get up and walk away but uh hopefully even for you there might be something new or interesting here or at least you can correct me when i'm wrong about stuff okay so we have clients that have sensitive data when they're unlocked their sensitive data is decrypted within the clients only

and we want to do some data analysis um you know across a broad number of users and so the general is that the client would do something with the sensitive data and then report it up to some service outside of the users and clients control that's the structure of how gathering such data would work and you know everybody's first correct reaction is well you collect the data but you keep it anonymous you remove all identifiers um and you know it you can remove all identifiers that the client sends you still have ip addresses unless you go to other measures to not have ip addresses so potentially something like this could be done through tor or something similar

again you know it's saying is that yes we could promise to remove ip addresses but we want this to be transparent to the user in the security community that uh that this these privacy guarantees are there de-anonymization is a real thing um so anonymization is hard in the first place the common practice of simply hashing the identifying information is bad breakable reversible for reasons i'm not going to go into here it's really common it's just not the way to do such things but they're always doing uh you should you should design your system under the assumption that your data will leak again this has been a principle that we've used at one password since the

beginning um uh but uh anonymized data even properly anonymized data when combined with other data can be de-anonymized in very important and surprising ways and depending on time i'll run through an example of that at the end very famous case with netflix and i with netflix but uh it really is the case and that other public data that things get combined with is data that comes from other places not from you you have zero control of it you've got zero control of the publication of that other data and furthermore you have to design your system to resist de-anonymization um when the anonymization could be against public data that doesn't even exist yet uh so uh

uh so de-anonymization is a thing which leads to this major point is that good anonymization and data protection is absolutely necessary but they are far from sufficient so uh one approach is to simply have the client be answer vaguely instead of giving you the precise password strength to the extent that the password knows it it could lump it into let's say one of three bins you know a low medium high you know so so you add some vagueness to it

and but for additionally there's the notion of what is called the random response technique which is actually in a sense a precursor to differential privacy um and it goes back to the 1960s so this example i cite here is from a paper in the 70s um so subjects were given a questionnaire uh but told to roll a die before answering each question and if they rolled one through four they were supposed to answer honestly if they were to roll to five they were supposed to answer yes and if they rolled a six they were supposed to answer no and uh and this increased response rates and through other analyses by comparing with other methods and and statistical um

cleverness uh you could see that it actually increased honest response rates uh so so again the in a sense the randomness of this uh was transparent to the user and uh and made them feel safe answering honestly um uh another study conducted in mexico in 2001 where abortion was highly stigmatized and illegal used another form of random response and uh people were given a red and green question and they pulled a red or green thing out of a sack they knew that there was just one red and green thing in the sack and if they drew red and of course the researcher the interviewer did not see what people drew um and this was again transparent to the

subject they answered uh did you ever interrupt a pregnancy or green were you born in april now from census data they knew how many people in the women in this age group were were born in april and we're able to do some statistics on this and um uh i've got in my speaker notes which you're not displaying here but i think it was around 16 of women in this range

in a sense yeah i mean after anyway see so they estimated for the population given the way they sampled that 16 percent of women um had had an abortion okay uh and so various statistical techniques here various statistical techniques here have demonstrated that this stuff is effective at at getting more honest responses um and in enough that in it and it does that enough to easily make up for the statistical noise introduced by this uh and in these cases um went the wrong way um uh so let's suppose we were to set up a system that had answered honestly about password strength fifty percent of the time um uh and otherwise pick let's say the

three bins randomly are uh if our data leaks and is de-anonymized is the user sufficiently protected and the answer is i don't know yet um but that's for later because i haven't looked through the order of my slides okay so these techniques add statistical noise um they reduce the statistical power of the tests and the consequence of this is that you either need a larger sample or you end up with or you end up with wider confidence intervals okay now where differential privacy comes in differential privacy it's really really hard to explain what it is without the math um so i'm going to try to explain what it attempts to do and it is about limiting the possibility

of de-anonymization it's an anti-de-anonymization technique um and it provides a common mathematical notion of of the protections of some data set in this in this respect but across a wide variety of techniques and uses it does add statistical noise in ways that are similar to the vague and random response stuff and the noise it adds is designed to have certain statistical properties that make it easy to combine easy to reason about when you combine it with other things um okay it is not a single technique it is not something that is only done at this stage of some data analysis and collection and it is not particularly easy

so when i said that it can be applied at different places or implied that it could be applied at different places it can be done at data collection time it can be done to transform a data set that you have that does not have privacy guarantees into a data set that does have differential privacy guarantees and it can be done at query time you've got a data with sensitive data that is at risk of de-anonymization but it's also differential privacy can be applied as what analyses you do on the data and adding its magic noise to those queries uh so that the reports you get out of it could be safely published um now there are

really really nice tools that are coming to be available and usable by people who don't have phds in math for for the third one of those and to a lesser extent the first the second one but in our case we never want to have the data in the first place so we need these things to be applied and done by the client um what would what would be required uh if we're to go beyond the random response stuff would be uh something called homomorphic encryption it's a fancy scary name uh for actually magic i'm sorry it's just magic um so the paper that first introduced it without this terminology was in 1982 and it set up the question

two millionaires wish to know who is richer however they do not want to find out inadvertently and they do not want to find out inadvertently any additional information about each other's wealth and i should also add that they do not want any third party to know either uh how can they carry out such a conversation so when you look at a problem like this you would say they use a trusted third party the trusted third party will learn the wealth of each also millionaires seem to be have been different back then um now they would be boasting about their anyway never mind um i need to stay on track here okay so um uh you know so the

normal way of thinking about this is that you have a trusted third party you do various things to ensure your trustability of that third party but the third party gets both numbers and then simply reports back he reports back uh who's richard um oh and i keep on doing that okay uh well the particular paper there uh developed a technique to solve this problem and a bunch of similar problems uh without the need of a trusted third party and yeah no math magic it's done by magic um but it's done by magic in polynomial time and space and that is very important uh so and in general for uh it was proven in subsequent work for this paper

uh for any protocol that can run in polynomial time using a trusted third party there's a protocol that can produce the same result without the trusted third party in polynomial time and so it allows multiple parties to compute things over their individual secrets without revealing those individual secrets to each other or to a third party and so this is this is generally what homomorphic encryption is about uh multi-party computation um so the practicalities of of this stuff uh i still think it needs a phd to make real use of it um the libraries and tools are getting better but yeah um polynomial time in space doesn't actually mean fast and small and so even though

the theory of this stuff was developed 40 years ago the reason that we're only seeing these techniques used in the last five maybe 10 years is because it actually takes a lot of computation um and the protocols used actually have a lot of back and forth they're complicated protocols so i'm going to say that that technique is still beyond our reach when i started investigating i was hoping otherwise but um but we're still there now i do want to make an aside on the adding noise so sorry i know you're all passwords people and here i am talking about you know stuff but again focusing on the example of of anyone who's doing research into user's

behavior of passwords you know this is this all this stuff is relevant even if general to many other cases uh one of the objections that see so there have been cases where people have provided differential privacy techniques as a real clear viable usable alternative to what uh to what systems are doing and their clients you know the people that they were developing this for or trying to get to use it uh would okay i got ahead of myself uh would complain about the statistical noise added these techniques add noise um but they had a quantifiable and well understood amount of noise and so you either need a larger sample or you accept wider confidence intervals

but i've heard cases usually from other people who've been trying to get this to work where they run up against people saying we can't accept anything with statistical error or confidence intervals well if you're doing data analysis with or without this stuff you have statistical noise error terms confidence intervals that is simply a part of working of working with data and doing any analysis of it and uh so my big mean rant um uh is this if you can't handle statistical error or you don't know that you always have some you should not call yourself a data analyst um and i am being far less polite than the people i know who've encountered these objections

they couldn't say it i can um partially because nobody's hitting me with the oh we can't do statistical noise stuff if people were hitting me with that i probably couldn't get away with saying this um okay and that's also an indication of why i don't have friends so uh conclusions is there a combination of of of there should be an oven there a combination of vagueness random response and data protections on the acquired data that would offer sufficient guarantees for our users and allow us to answer the 2fa strength question with sufficient confidence i still don't know this is a work in progress i thought that i would be able to figure out what how strong of an effect

there need to be in the real world about those things combined with these techniques to then estimate what sample sizes were needed but my math isn't quite up to the task that i thought it was when i started the project okay so i actually finished um oh netflix case right okay so i actually finished the guts of my talk much much earlier because i only noticed a couple days ago that i had 55 minutes instead of instead of 25 but i think it's useful to go over the netflix case of de-anonymization that i talked about again i'm sorry it's not really passwords but it's about preserving user secrets so way back when ah good i have it on the slide in

uh 2006 netflix really wanted people to really wanted an algorithm or to improve their algorithms at predicting what things to recommend to people and so they publicly released an anonymized data set of about 100 million movie ratings from nearly half a million subscribers uh and all customer identifying information has been removed all that remains are ratings and dates now people might have dozens or maybe a hundred different netflix ratings at the time or maybe more but people also rated um things publicly on imdb now they you know they will have for the imdb stuff they knew that that was public and so they were okay letting the world know that they watched that if they also watched

um you know uh if they watched again and again and rated highly nickelback's world tour um they that's something that you know you wouldn't want anybody to know about you know and there are other things that people might watch that uh um you know that that they don't want the world to know that they watched but they rated them with netflix but not with imdb so uh a paper in 2008 showed that even with just a tiny number of imdb ratings you could de-anonymize almost all of um you could de-anonymize almost all of the netflix data and so this is this is why i wanted to emphasize that de-anonymization is a real thing and it is against data that may not even

exist yet when you are designing your system and uh and so differential privacy is designed in a sense to add a certain amount of noise to let's say the data that netflix published in ways that would prevent this or any other kind of de-anonymization um okay and uh there are other examples there have been many many more cases of cases of de-anonymization but that one is really dramatic and much simpler to explain uh so um uh so i don't have an answer for those who for those running password managers who want to study their users passwords behaviors but it's not fully out of reach it's just not quite within reach yet but these are the kinds of things you

need to look out for when designing such a system and with that i will take questions

thanks jeff um i'll i'll ignore the homomorphic thing because i don't have the math either but when you talk about adding noise to a measurement in order to uh in order to [Music] add a little bit of uncertainty about the the answer to the question in the case of um the you know maybe the the the the master password for your password manager the thing that the reason that you might be sensitive about the answer really has to do with whether you know em am i an easy target to hack or not uh if you're adding noise to it aren't you if it's additive noise aren't don't you still end up with just a larger

probability that you're easy to hack

the noise can have a negative sign if that answers your question no okay in that case i've misunderstood the question um uh you know so you know so the noise can add or subtract values but obviously i've misunderstood the question right well let's say that you have a a scale of 0 to 10 for the strength of your master password where 0 is you use pairs license plate and 10 is you use something that was really really good and long probably um if you're adding a certain amount of noise to it and you end up with an answer that is 8.5 it's still likely that that 8.5 came from the fact that the the original number was 10. on the other

hand if it if you end up with a it did maybe it depends on the distribution of the noise right so uh two things one yeah okay uh so yes um you know you can if the date is de-anonymized um then you can make broad guesses of the true value based on the reported value um uh so you combine this and and in a sense you have to be able to do that for there to be any data analysis possibility whatsoever so uh so that's why by um my question of whether something like the random response thing is with the data protections with the anonymization is enough and it's why my answer to that question is i don't know

you use wide bins so you wouldn't use you wouldn't ever have a report like 8.5 on a scale of 10 i was thinking much more of a one two or three and the integer values um so you so you have a vague response then additionally the noise and but yes the question is is are you still leaking if this data leaks too much information and i don't know and i do you know it's it's hard to quantify the notion of leaking too much there are ways to do it in differential privacy terms where you can where you can at least know how much you're leaking and so we can analyze this to know how much we're leaking

[Music] but in the end it's a whole big risk analysis thing so maybe you know with what you say is what i'm trying to do here is impossible i hope not but i recognize the possibility that what i'm attempting to do here is impossible yeah so i have a question myself i was just wondering um thank you for the presentation by the way it was great um i'm curious from your comments regarding it's being impossible but close reach are you currently researching this and if so where are you with your current research

so when you say researching this it's like uh this top the topic that you're referring to regarding deadly change whether or not to analyze a user's passwords so this is a work in progress research report where progress is just beginning but i'm trying to at least put up stuff and right now it's just like placeholder documents uh but this is uh uh this repo here again it's mostly placeholder documents and the slides for this talk once i merge them into main um uh on privacy preserving analytics so yes we're researching it um but it's slow it's preliminary it's just at the beginnings uh jeff great topic very intriguing so thank you very much uh let's say to answer that question we

compute the entropy on client side for the password and then we encrypt the entropy answer whether it is high or low let's say we don't have three values but just two high or low using uh homomorphic encryption uh especially the one that is used for e-voting is called pilear homomorphic encryption pi layer homomorphic encryption and then send it to the server side um on server side we don't have to decrypt the ciphertext to know the value whether it is high or low just by applying the same e-boating verification scheme we can find out whether the password has low entropy or high entropy and then correlate with the 2fa okay so so i was not aware aware of that technique

but if i understand correctly there are tools and known techniques for collecting binary data and so if we just take this as a high or low binary then there are tools you can actually use um thank you very much uh you have my contact information please send me the the actual link any more questions for jeff no okay so thank you once again jeff