An Unlikely Friendship: Why Security Engineers and Product Managers Should Be Working Together

hello there we go all right everybody we're going to get ready for our next talk uh our next speakers are leaf drizzler and rachel landers leif is a information security professional with almost a decade of experience we've joined segment now part of twilio in 2017 and currently manages a team of software engineers focused on building security features rachel is a product manager for segment now part of twilio based in san francisco california today together today they will be telling us about an unlikely friendship why security engineers and product managers should be working together thank you thanks everyone [Applause] great uh thanks for the intro uh i'm leaf this is rachel and we're here at b-sides and we're stoked to be here and

we're stoked that all of you are here and we're gonna tell you about uh some of the stuff that we've been working on so just a high level of what we're going to cover today first we're going to talk about some of the common pain points felt by enterprise product managers and security engineers then we're going to talk about our unlikely friendship between product and security we're going to talk about a few of some of our earlier projects that we worked on and then we're going to actually walk you through building one of the enterprise security features that we tackled together then we'll wrap it up and leave some time for questions if there's anything in this uh that you

think you want to reference later i uploaded the slides this morning to the sketch link so if you just go to our uh like b-sides page the pdf of the slides is there so yeah that's easy to find i think context is important so a little bit about segment we're a b2b software company and we build a customer data platform this cdp is used by folks in marketing product engineering to create a single view of their customers which we call end users and we process over a trillion events for months so this is a very data-heavy company we have a very modern tech stack which is great primarily built using typescript and go and it's all in the

cloud as of about a year and a half ago we were acquired by twilio so if uh seems like stuff you might want to work on we're hiring for a bunch of different things inside security outside security you might be thinking what does a single view of customers actually mean i think in the security industry the single pane of glass gets a lot of so this is not quite that this is just something custom that you build you can create tracking events for anything that you want and then you can send that data through segment to things like salesforce google analytics you know whatever your your company uses we'll send it there it's worth noting that this is only

first party data so we don't share any data between our customers our customers have to build their profile about the end users individually now you have context on segment as a business and a product and now we're going to introduce ourselves and talk a little bit about some of the the problems that we were facing in our roles at segment um so part one the woe of the enterprise pm aka me uh my name is rachel landers i'm actually currently doing a product strategy at a company called patch i've been in the it's a climate tech company that builds apis to fight climate change i've been in the climate space for about two months and if you are contemplating joining the

climate tech space now is the time to do it it's really cool it's really awesome um couldn't recommend it more before patch i was a product lead at segment where i was focusing on everything betw from ranging from pricing and packaging self-service and enterprise cx which is basically like the customer experience of the largest and most complicated customers i'm going to be mostly focusing on that particular role as we talk about the project that leif and i worked at uh before a segment i worked a company called microstrategy which if you're tuned into the cryptocurrency world you might have heard of it michael saylor is big in the the bitcoin game that all kind of happened after i left but now

it's a funny logo to have on my resume um this is my first besides i actually texted leaf yesterday asking what i should wear and if i wore a blazer would i look like an arc uh so i hope you appreciate my product manager presence in my my blazer uh anyways um so i was an enterprise pm for the the bulk of my time at segment and i kind of want to talk through some of the the woes that i experienced and i think it's actually going to be a lot of things that you've heard before if you build software or work adjacent to people who are building software for enterprise customers enterprise customers are notoriously

noisy and demanding they have really really high bars really really high expectations they also spend a lot of money on software and so oftentimes you have to meet those demands they also have disparate table stakes that they expect because they're so large and complex and um we'll get into a little bit but spoiler alert uh a lot of those table stakes are actually product security features um and last but not least they're incredibly complex you'll have more users in an enterprise instance of a sas application than maybe a startup because there's more people to use it and so they'll often find edge cases that other customers might not find and now the woes of the security

engineer uh these will prob oh uh intro first i guess uh yeah i'm leaf uh i'm the engine manager for the securities features team at segment twilio i'm also a conference organizer for locomocosec which is happening later this month so if anyone wants to go to a really fun and really great conference in hawaii there are still tickets we're also looking for sponsors and before this i was a sales engineer at bug crowd before that i was a security consultant at redspin uh probably one of the few people that knows what that company is right here jason haddocks and i've also been pretty involved in the security community as a chapter leader for the bay area owasp and a

conference organizer for the apps at california conference fun fact b-side san francisco is actually the first conference i spoke at in 2015 so and this is my first time being back as a speaker so very happy to be here so the woes of the security engineer i think these are ones that a lot of people in this room have probably experienced firsthand the first one is like maybe you're seeing features in other software that you think your customers should have you know maybe your company doesn't offer sso or maybe you don't offer mfa or maybe there's something else that you think is relevant to your business that they don't have the next one is you feel like you're

disconnected from the customer and from other teams at the company and maybe at a worst case they actually don't want to work with you not only disconnected it's like intentionally disconnected because they perceive you as somebody that's going to slow them down or block them from doing things i really do think like that sentiment is changing i think netflix and github should get a lot of credit for promoting the idea of security being a business enabler which we'll talk a little bit more later i think another thing that i hear a lot of security teams struggling with is creating meaningful metrics and this is really important because this is a way that you can speak to

other parts of your business in a way that they're already used to understanding i think it's easy to talk to somebody else on a security team from another company and be like hey i did x y and z and they're like oh yeah that's really valuable that makes a lot of sense but to another part of your business that has no idea what any of that stuff is it's a lot harder for them to see the value security engineering should be a team that the rest of the company wants to work with this is going to make your life easier as a security engineer it's going to make it a more secure experience for your

customers which is going to benefit your company and i think one way to do this is learning more about the teams that you're supposed to be working closely with and this goes for anything it's like if you're in grc maybe that's working with legal if that's in if you're in sales maybe that's working with the marketing team uh the more you know about the teams near you the easier it's gonna be for you to collaborate on things and this is really important for security and engineering to have a good relationship because security should just be a component of building good software the same way that you're thinking about reliability or being able to scale efficiently

your software should be secure so let's talk a little bit about how we can actually bring security engineering closer to the rest of the organization in a way that empowers the business i think security as a business enabler is something that's been pretty popular over the last like five-ish years and i think it's something that a lot of people kind of just you know think is a good idea at this point um but similar to shifting left a few years ago i think there's still maybe some ambiguity about like how you actually do this and i think the most important thing is being aligned with your company's goals and helping other teams execute on things with the appropriate level of

risk not every company needs to have the same security posture and so it's about figuring out what level of security posture is acceptable for the you know business that you're in your customers et cetera and helping your teams move quickly if your competitors are moving faster than you because you know you're blocking them from doing you're blocking your team from doing things that's going to hurt your business and if your business isn't making money you're not going to be making money either because this business probably won't exist anymore i think the next thing is ideally you're helping make things faster easier and more secure that's not always possible but i think netflix should get a ton of

credit for promoting the paved path idea which is like let's build a road let's make that road easy to use and let's just sneak some security in there maybe people don't even realize that that's the best honestly it's like if people don't even know they're being secure that's perfect um and then i think the rest is just use every opportunity you can to meet people and educate them so that they can just do security on their own because engineers are going to have a much bigger impact on the security of your product than the security team there are so many more engineers that are doing stuff it's like if those people are all advocates of security you don't need to

worry nearly as much and then this is a great oh it's like totally cut off that's so weird um well if you download the slides there's a oh there we go perfect thanks i was like why do i see on my screen that's so weird um here this is a great talk from absolute california from a couple of years ago um that i think is is really inspirational yes pause for photos um so let's recall the org chart briefly uh this is you know where security engineering is we're going to zoom in on security a little bit obviously there's many other parts of security other than just seconds and grc but we'll get into that in a second

unfortunately a lot of the parts of your security program are something that customers won't see probably the majority of your security program are things that your customer won't see this is things like enterprise security physical security app set cloud sec a lot of these are just things that your customers don't directly interact with they benefit greatly from them but they don't really like see how they work unfortunately probably the most important part of what a customer thinks about your security is public comps during a security incident hopefully you don't have any of those but i think that those can really like shape in either way what people perceive your security program is like under the hood

i think the next one is actually security features and so this is things like single sign-on role-based access control security events auditing that kind of thing and i think this is actually a reasonable metric because i find it very hard to believe that a company would invest a lot in security features when they're under investing in security as a whole i think if you have a strong security program this is just a component of that and it's a reasonable reflection that's like hey this company is probably doing a pretty good job in all these areas that i can't see there's also security certifications and security questionnaires i put these slightly below the security features just because there's so many

fewer people at your customers that actually review these maybe that's just people on their grc team and the third party risk team but most of the uh you know people using your platform probably aren't actually looking at these all right so to recap uh security is a big ass iceberg there's a lot going on under the surface that a lot of people can't see and is often really siloed away from teams so now it's time for us to maybe tell a little story about how leaf and i came together and broke that silo down such that we could work together and build together towards a common goal so if you recall our woes when we were

reading them out loud they maybe seemed kind of related as you were listening to them but if you take a closer look they're actually um pretty complementary towards each other um there's a lot of overlap so i said that enterprise customers are really noisy and demanding uh leif mentioned that he didn't really the security team didn't really understand what customers wanted or needed so that's an opportunity to share the the wealth of the demands of customers and actually help synthesize those requests to other teams so that you can have a diversity of opinion on how you can actually solve them or start to tackle them um i also mentioned that enterprise customers had pretty disparate and sometimes weird to

me the product manager that doesn't know anything about security table stakes that's an opportunity to bring those requests and those table stakes features that you see in rfps from enterprise customers to the security teams and start talking about how you can actually solve for them and then last is security mentioned that they wanted to be able to move metrics a little bit more meaningfully that's the product manager's job so if you can start building features together then it's a way to get everybody bought into the the business metrics and so everybody can start moving the needle a little bit more uh we wanted to take a picture of us shaking hands but due to covid we

couldn't really get together so we asked an artist to to animate something for us um i think it's pretty lifelike um but before we kind of talk about the the big project that we we put together together and built this presentation around there's a couple low-hanging fruit projects that the security team actually tackled um sometimes with product help sometimes without but we really just wanted to demonstrate that there is low-hanging fruit all around no matter what company you work at i mean you just have to like look around and start to pick it um and leaf's going to walk us through some of those features cool yeah so the first one that we worked on was a password strength meter

this was a great place to get started because it was mostly front end changes uh it was something that was easy for customers to understand it has a simple like four panel ranking system we give them feedback if their passwords been part of a an unrelated breach and really this was pretty easy to do because we were able to use the dropbox zxcvbn password strength library as well as troy hunt's have i been pwned api that allows you to meet the nist password guidelines and everything other than that is just like using your company's front-end framework uh in the case of segment it's something called evergreen which is open source and it's really easy to use and really

great documentation even a bad front end engineer like me was able to get something working pretty easily the next project which was slightly bigger was bringing multi-factor authentication to our customers this was an early attempt at a security and engineering partnership and we actually learned a lot about how to make this more successful in the future the project was very successful in the sense that it shipped on time there haven't been any major problems with it you know a lot of the code is still the same as it was when we when we built it um but i think we learned a lot about kind of the the partnership process and i was in no way excluded from the team

that uh we were working with but it really was more like a relay race where i built the back end and then somebody else kind of just took it from there and and did the rest of it in the future we made a much more intentional effort to actually make the person joining another team feel part of that team and so uh we would have one-on-ones with their managers we'd attend the like weekly team meetings uh we'd even attend like fun social events i got invited to a go-kart thing and actually won first place and haven't been invited to another one since but uh yeah it does work um if you want to hear a little bit more

about some of this stuff these are some things that i've blogged and spoken about in the past the youtube link is to a presentation from last con last year all right it's time to talk about the the big security feature that we talked to that we built together that we've been talking about uh for the past 20 minutes um like i said earlier i've said it a couple times enterprise customers are pretty needy and complex especially in the way that they manage users in sas applications um at segment and i'll talk about a little bit what this looked like but some of our largest customers had over a thousand users that they were managing like disparate

role-based access control across manually in the ui it wasn't scalable it wasn't sustainable and it created a bunch of security gaps and holes that we didn't love our customers didn't love and we needed to fix i saw this as a problem that i had no idea how to fix but buzz lightyear aka leaf came in and introduced me to something called scim which is what we ultimately built together spoiler alert so as i mentioned managing users in the segment app was really confusing you had two options the first was to as i said go in the the segment app ui and toggle on or off different uh roles for different users and there were like

hundreds if not thousands of permutations of roles that any individual user could have and segment as leif mentioned was processing user data so there was also a pii component to who had access to what and so it was ultra important that our customers could get this right and provide the principle of least privilege with our application the second option was like kind of a non-option we had an api that allowed uh customers to build tooling to manage users but the api was kind of janky and not didn't really work very well it's no longer janky we fixed it we built a completely new api but it was this option that we provided for customers that was programmatic and scalable but

actually didn't work very well and so it just forced everybody back into that ui which is almost worse than only having the ui in the first place but again we fixed it hindsight 2020 always learn from your mistakes so we've said skim a bunch of times and you might be asking what is skim uh it is the system for cross domain identity management which i am the first person i've ever heard actually say the full name of everyone just calls it skim but skim is amazing because it allows your identity provider like octa or azure active directory or whatever to create users put those users in a group you know maybe move them to another

group and then at some point when they leave the company or they switch roles or whatever you can de-provision them this is amazing for customers that use identity providers because these customers are typically managing hundreds of sas apps each one is its own provisioning nightmare and segment enterprise customers have as rach mentioned hundreds or thousands of users so multiply like you know thousands of users by hundreds or maybe even thousands of apps and that this is a very very tough problem this is a great tool because it's both secure and easier for enterprise customers to manage where people go within your app skim is really cool if you're a if you work at a b2b app that doesn't offer

skim you should definitely look into it um it would make the job of enterprise security and i t people so much easier if everything supported skim it would still be a very tough job but just like not dealing with like did this person get removed from this thing when they quit two weeks ago that is a huge problem so i'm the product manager at the security conference and i'm about to talk about process most product management teams have a product development life cycle and it's basically like the the flow of the river like it's the way that we have to go it's it's there's nothing else if it doesn't follow the pltc then it's not going to happen and so i think

something that leif and i have learned and reflected on as we were building this presentation talking about um how we came to work together is it's really important to meet somebody that's in a different part of the organization where they're at and so if you're working with a product manager if you want to work with a product manager to get a security feature prioritized in a product roadmap it's really important to understand what the the process is for a product team or a product organization at the company that you work at so i'm just going to do a quick overview of the pdlc throughout the presentation but the first step of any sort of prioritization

framework or process is actually need finding or identifying a business use case and so there's a couple different aspects to this that are important to note um we need to build something that is meaningful for customers meaningful for customers right now like why is right now the time to build something and how is it going to achieve some broader goal that we as a team or an organization want to achieve and as you're building a business case you are often product managers or even anybody in any org that wants to get something prioritized will write some sort of product requirements document or prt this is kind of the grounding document that everybody could can rally

around to understand what we want to build why we want to build it and why is it important right now and it also becomes this artifact that is passed through the rest of the process all the way through to the build phase where engineers designers leadership marketing sales can all go back to this document and understand exactly like what we're doing and why and so in this presentation we're going to use the prd framework to talk about why and what we build for the skim solution at segment i think i mentioned this before but why something is the right time to build at that time is one of the most important facets of prioritizing a

feature so just to talk through why it was the right time to build skim for us at the time that we built it we were basically just jumping on a moving train um we were rebuilding our apis as i mentioned the old apis were a little janky we needed to fix it and so since we were already rebuilding the apis we identified that as an opportunity to build skim on top of the unreleased api endpoints for user management and get something out the door to customers before we were ready to release the entirely new api and so it was just kind of a right time right place and it's really important to kind of look around look at what trains are

moving and and hop on the moving train that's going in the direction that you want to go so the first section of a prd that i write and i think most product managers write is the vision and with the vision you're really trying to paint a picture of the world that you want to build with whatever feature that you're building um what's the tldr also since this is at the beginning of the document this is going to be the section that all of the leadership team reads and nothing else and so this is the time to really nail in what you want to build why it's important so that you can get there by and not waste too much time

so in the case of skim the vision that we wanted to paint was that customers could leverage their identity provider for a seamless and scalable solution for programmatically managing users and the the end goal for the user here was to reduce friction and managing users and so that there was no friction point such that no additional users could be added to an enterprise application because we were actually finding and i'll get into the problem statements a little bit later but we were finding that it was so difficult to manage users for these enterprise it teams that they just weren't letting people onboard onto our product which completely stunted any sort of growth or expansion of the the product use case

which is not good i feel like i don't have to tell you all that this next section that you really want to focus on is painting the problems um why does this particular part of the process suck what is the experience like for the customer and just really nail down the different issues that exist that you want to software so that you can have the full scope of the problem identified so when it's time to actually spec out the solution you have full coverage of all the problems that you wanted to solve versus like a 50 coverage or something like that um so at segment the user and group management was really challenging at scale the the companies weren't granted

access to the right things there's a little screenshot at the bottom we provided just in time provisioning for users that had single sign-on or for organizations that had single sign-on enabled um but when they were provisioned they had access to nothing so we had users onboarding to our platform and then just dumped into this abyss if you don't have access to anything because there was no scalable way to grant them access to something when they were provisioned and so it was not only a security issue if people wanted to just like give access to everybody for everything it also was a user experience issue because they didn't have access to anything and so they would come into a

tool with a particular goal and they had no idea how to to solve for it because they would just hit a brick wall a proverbial brick wall um and then last but not least segment is a customer of segment and so we were actually uh everybody at the company was originally provisioned into the segment workspace so that we could use it for our own uh product releases but it became so hard to manage that everybody got kicked out and so then only like a select few had access to the segment platform and we had to go through them to to get events tracked for for our product releases and so we ourselves were definitely feeling this pain and

could empathize with the customers that we're feeling it too so you probably bought in to skim already because rachel's done a great job of selling it um but the goal of this isn't to convince you to build skim this is really to show you some insight into the process uh so that you can maybe replicate this where you work and part of it should be like why is the technology that you're backing the right one and maybe at the point of the prd you're undecided that's totally fine you might have a couple different options where you're like maybe we should go with this vendor maybe we should build this thing ourselves you know maybe we

should build this other thing and that's totally fine if you're outlining the pros and cons that's great but for us skim was the obvious choice because this is already something that's leveraged by it teams at enterprise companies there's a standard that defines it there's multiple rfcs that tell you what you need to build how you need to build it and as rachel mentioned earlier we were already in the process of building our user and group management api and so we had the underlying functionality we just needed to translate it in a way that skim could understand so as mentioned earlier we actually benefit a ton from skim ourselves we have this internal tool called access service um somebody on my

team andy lee wrote a blog about it um you can read a ton about access service but basically what access service does is it's an internal app you go to access service you say hey i need access to this thing somebody who is an approver for that thing gives you access for a set period of time this works with octa apps and groups and octa groups are really amazing because you can map a group in octa to a group using skim into a service provider like segment and so you go to access service you get approved the api puts you into a group in octa and then octa puts you into the corresponding group within segment and so now you have the

exact uh you know group access that you need to do your job okay so we decided to build skim we pitched it we got buy-in but now we need to actually narrow down what specifically we need to build so we got the y and the when good to go but what was a little bit unsure so we looked into the data um and just a note to anybody who doesn't know are that stands for annual uh i don't actually know what it stands for annual recurring revenue oh man saturday sorry all right and i'm recurring revenue um and so we actually used two different like slices of data to understand which particular um idp to build for so the

number of customers is important just to see kind of like the scale of usage across an idp but also the our share was really important to us because uh those were the customers that we wanted to keep happy because it's recurring revenue and so if we keep them happy then perhaps the recurring revenue will stay the same or even increase you may see the also the emojis mean the juice was worth the squeeze i thought it was kind of cute but it's probably a little bit convoluted so just wanted to call that out but you may be wondering why we decided to go with the azure active directory which is the third line here but if it's only representing two

percent of ar and 12 customers this is a really important lesson that i learned at this time and also have kind of continued on with other projects i've worked on is the importance of marrying qualitative data and quantitative data so the quantitative data the first two home run need to build them this third one though we actually had a couple customers that were using active directory that demanded skim like they needed it or they were going to walk they they used it for all of the other sas apps that they were using segment was the only exception and they needed it and so in working with our customer success team that was on the ground with the customers hearing

them understanding what they actually needed we were able to understand that we could actually save that two percent of error which two percent is a low number but that's still a fair amount of money um and also try to get a little bit more buy-in from other customers that could be using it as well so always really important to marry the quantitative and the qualitative data together [Music] another aspect of the prd just to kind of walk through the the product manager process is to build user stories which user stories i think are really important and helpful and are used by many product managers because it helps us to build empathy towards the end user

by putting yourself in their shoes in the statements that you make about what they need what they want and some of the problems that they're facing so as an example for scim as an i.t manager i want to connect my company's idp to segment so that i can automatically provision users based on their team and trust i think trust is a really really important aspect of building any sort of security feature and trust that the right folks have access to the right data and nothing more so now we're going to move to the next phase in the process which is writing the sdd this is the software design dock and this should be really heavily

inspired by your prd and you should definitely steal a lot of content the goal of this doc is really to convince people that you're building something in a sensible way and invite them to make suggestions about how things could be improved the std should be very complementary to the prd in the sense that the pr that all the requirements listed in the prd are things that should be part of the proposed scope for the sdd a great place to start with your sdd is just links to other documentation whether that's something that's internal or external defining terms things like that you really do want to meet people where they're at currently when you're writing the std you've probably thought about

this problem more than almost anybody else at your company and you need to make it so they're not lost when and like looking up every other thing linking to other internal docs is really helpful too because people might be like oh yeah i was actually part of that project like now i understand how this thing that you're building is connected to this other thing that like i actually already know a lot about those are great people to invite to review your sdd because they probably know stuff about these systems that you don't know and they're going to bring up really good pieces of feedback here's an example of some of that feedback from our co-worker drew

i don't know if this you know is something that like maybe that i knew at the time but it's a great call out by drew and you should definitely consider feedback from your co-workers as a gift the next section that you're going to work on is the context and background section similar to rach's comment during the prd like there's probably a lot of people that are just going to read this context and background section and that's it and so you my guidance for this section is write this for a level of understanding that's somewhere in between like your manager and your manager's manager people that are familiar with like what you're doing on a day-to-day but not actually

involved like too heavily in that process you don't need to define every single thing like just kind of assume that people generally know like what is happening at your company but write it at a level of depth that you know they don't like have to be working on that thing and as part of this like never pass up an opportunity to do a little bit of selling uh always include a little bit of a pitch for like why this is important maybe you throw in some high-level customer stats people should generally be on board from the prd section but it doesn't hurt to convince people again that this is a good idea always be selling

okay you may or may not be thinking why the hell are y'all writing so many documents like when are we going to actually build this thing i think it's really important to maybe delineate the the engineering technical process with the the product discovery business need finding etc process and i had a mentor one time that broke it down in a really clear and concise way which is how i broke it down here where product owns what are we building and why and then engineering owns uh how are we going to build it like what system should we use what technology should we use and when not necessarily when is the right time to build it but

how long is it going to take and when can product managers go back to the customer facing teams and say this is when it's going to be ready and i think there's obviously a lot of different pieces and parts of handoff throughout the entire process but there's a very clear handoff point between the prd and the std that is important to share the sense of ownership of the feature and kind of pass that ownership over when it's time to start some of the technical implementation i actually love writing sdds because i love the phrase weeks of programming can save you days of planning or something along those lines and i find that if you actually write a lot of this

stuff out you're going to come up with a plan that's a lot better and it's just going to save you time like nobody likes building something halfway and then realizing you totally botched the design so now that we've talked about determining business need you've written your prd you've written the sdd everyone's like super excited about whatever you're trying to build um let's actually talk about building skim a little bit so as a quick refresher at the end of the day skim is really just user and group crud it's provisioning users de-provisioning users creating groups you know deleting groups updating things like that and going back to our original business use case kind of from a more technical

slant the goal was to create parity with the segment ui so on the far right we have our control plane service which is our modern monolith that powers much of segments web app and api the back end of segments app is the app gateway section and then on the far left you have segments web app what we are trying to do is create parity for identity providers to call the api endpoints to be able to do that same user and group provisioning that you could do with hundreds of clicks in the application so zooming in slightly as rachel mentioned earlier we are really just jumping on the moving train at the end of the day the user

management endpoints that are called by both our public api and the skim endpoints are the same those are part of control plane service skim actually lives inside of our public api which has non-skim user management endpoints like if a customer wanted to do this you know using their own system for some reason they could they could write this but identity providers are calling the skim endpoints so this part's going to feel a little bit like drawing the owl because uh you know we're not going to actually fill in the whole owl um this is going to be very specific to your business like i can't build skim for you like i built it at segment

because i knew what segment does but i don't know what your company does but i will give you a couple of tricks if you're like hey you know leaf skim's going to change my company's life i got to build this thing definitely start with the rfcs the rfcs are pretty dry but they are very helpful and they really define like what you need to do i'd highly recommend using a library for the query filtering skim supports a bunch of like queries that you don't want to parse yourself and then really for a minimum viable skim implementation you just need to support get delete post and patch for users and groups there's tons of other stuff that the

skim spec supports it's actually very massive but in practice the identity providers don't really care about a lot of it and then some tricks really read the onboarding docs and understand the onboarding process for the different identity providers that you're planning to support here's a couple of my faves and how we avoided them first octa requires that you support marking a user as inactive segment doesn't actually have a concept of an inactive user you're either in a workspace or you're not in a workspace so that's what we do when octa wants to mark somebody as inactive we just kick them out of the workspace when they want to make them active again we just invite

them back the next one is azure really wants you to be able to retr return a group without its members they just want to know the metadata sometimes and they won't let you onboard your app unless you support this filtering so we just put in an if statement that's like hey if you want to filter this one thing we don't return it we don't support any other filtering because none of the other identity providers cared about filtering anything and then the last one one login really wants to do a put during user provisioning to align the properties of the user within one login to your app we actually support very few group or user attributes it's just email and

display name we don't let identity providers update those so we just respond to their put with a get and we just return the user so this is like you know pretty specific to skim but this is just to illustrate like it's worth it to read the identity provider docs so you can avoid some of these like weird little quirks we've talked a lot about identity providers if you build skim please take the time to build the integrations with the identity providers same thing goes with sso this is going to cut down on a ton of support requests it's going to make customers way more likely to use it because it's easy the worst thing you

can do is build sso and then like require your customer success team to like do a bunch of stuff to like get somebody on board that is nobody likes doing that but if you've promised a customer that this is going to be done by a certain time make sure you build this into your timelines because the identity providers are not very fast i actually give octa a lot of props they were able to get our app onboarded in like three to four weeks which was pretty good considering some of that was like waiting for me to do things uh azure active directory was like three to four months and a decent amount of back and forth

and unfortunately one login took almost a year to onboard so if you have a noisy customer that is a shared customer with one of those have them be annoying to the identity provider because i think you could probably jump the queue but yeah if you don't have that like definitely build in a lot of time so now you've built it everything's great your customers love it there were probably some things that went great during the process there are probably some things that you learned i think doing a retro at like a project level is really common in engineering land may be slightly less common in security land but i would highly recommend that you do this because it's a great way to

reflect it's also a great way for other people on your team that weren't even involved in the project to learn a little bit about what you're up to and they can maybe avoid some of the mistakes that you made and in the spirit of reflection we would love to share some of the lessons that we learned and takeaways of working on this project working together collaborating etc so recall the woes um customers will always be noisy and demanding specifically enterprise customers but we talked about a number of ways to kind of leverage the power of the demands and build something really really cool um i think the all of these was actually really manifested as opportunities to

work with leaf and the product security team but specifically around the table stakes which i wasn't really that big of a surprise in the way that it was probably framed at the beginning but some key takeaways enterprise software is secure software and they can't exist without one another like enterprise customers just aren't going to use your software if it's not secure and so if you're not building product security or weaving product security into your road map uh you're going to fall behind security is your friend not your foe i think there's probably like a meme in the product community of like security being kind of noisy with all of the vulnerabilities and all the extra stuff

to do but like it's all going to make a better stronger more secure product which again if you want to have better price customers which pay the most money uh you want to have a secure product um rachel wasn't paid to say that i promise i'm here at my own free will um and also many of these enterprise problems these table stake problems have already been solved um so you don't need to boil the ocean and you should definitely work uh smarter and not harder and work with your security team because they probably know a thing or two about existing security features that you can build into your product and every pm knows this but i think it's

really important to kind of drive home in every facet of an organization is that you should use the data that you have access to and that you should track metrics and build everything that you you are doing at work towards the metrics that you're trying to move and so just always grounding on a north star or a particular goal is paramount to a successful workplace and then tying things back to the woes of the security engineer i think one of the biggest benefits of building stuff collaboratively with engineering is you learn a ton about your company's architecture the people on my team all have a very good understanding of the like parts of segment that we interact

with regularly and this makes it a lot easier for them to propose new designs for things if they want to build something like they actually know you know roughly how to do that it's also great for if you encounter vulnerability you can probably figure out like literally the specific line and like maybe even a recommended fix and when you can do that it's just way easier for engineering teams to actually like go in and fix stuff you're going to feel a lot more connected with these teams because you're literally like working directly with them and when these teams are customer facing that's going to bring you a step closer to your customers with metrics we learned a lot from working with

rachel and our our product team and metrics are now every are part of everything that my team builds even our most simple services should have at least one product metric as well as health metrics this makes things like on-call lot easier because things break less often they break more predictably and it's easier to fix them and the presentation wouldn't be complete without talking about some metrics related to skim so roughly a third of our customers that use sso which i i consider to be a prerequisite to using skim use skim which is pretty cool because sso is actually built like three or four years earlier than skim and so there's probably a lot of customers that just

like haven't really revisited their sso config and so the fact that a lot of them are using it is pretty cool it's also associated with just over one-fifth of segments arr which is pretty cool for something that was only built a couple of years ago and uh part of uh you know doing this work in collaboration with another team is you should be able to do this independently uh moving forward just like we expect engineers to be able to do security tasks autonomously we can do a lot of this stuff that we were previously reliant on other parts of our business part of shifting left is about learning from engineering product and design so the most important part of

building this in is actually just making time for it part of our sdd or our prd is actually defining the metrics that we want to collect and maybe you'll decide later that some of those are too difficult or you know you won't be able to do them but like it at least gets you thinking about what you should be doing and it's also part of our work breakdown structure if you don't make time for this at the end of the project you're inevitably going to get pulled to like whatever is next and you shouldn't consider the project done until you have at least done part of this this makes it a lot easier to quantify

the success this goes back to one of the things i said way earlier in the presentation about being able to define things in a way that other parts of the business can understand and it's easy you know for a security person to say hey you know i built this thing um but a sales person they're like i don't know what skim is but when you tell them like hey this is used by 21 of our ar they're like oh that's cool um it's also a great thing to put in people's performance reviews if somebody on your team works on something that's really awesome being able to tie that directly to um you know metrics is

really helpful to like get them promoted or get them a better score during performance time it also makes it a lot easier to plan future work because people are bought into your previous work they're like oh that team is good they built cool stuff they have metrics like we trust them to keep making good decisions which makes it easier to get future projects funded especially if it's something maintenance related because you can say hey this thing's slow or this thing's expensive or this thing is used by a lot of people so i want to do one final example that's not related to skim just to prove that we're able to do this stuff on our own

without the help of rachel this is actually a project that our intern did last summer the goal was to create a new part of our web app that encouraged workspace owners to turn on mfa at the workspace level this is really important because once it's turned on at the workspace level everybody in the workspace has to use it this is a great summer intern project because it's a combination of greenfield development as well as you know based on existing work and they were able to create a nice little dashboard uh to show off as part of their summer intern demos so you'll see here when the workspace dashboard or the nudge was introduced uh it was just under 10 of active

workspaces we're using mfa we turned the modal on and by default and we saw a huge spike in the number of workspaces that were using mfa which is great you'll notice that the number goes down which i'll talk about on the next slide but that kind of this shows the importance of defaults which any product person can tell you defaults matter a ton and you might be think asking like why did we turn the default from on to off when it was so successful well we realized it was actually annoying users because they would click continue they weren't paying attention they were just like get me into the workspace and then they had to turn on

mfa themselves which they didn't want to do but they had to to get into their workspace to turn off mfa and then turn it off for their account and so we made this change to only turn on mfa for the workspace as a default on this modal if the user already had mfa turned on for themselves because if they have it on for themselves then they can still get right into the workspace which you know isn't annoying for them so if you want to hear more about what twilio's up to my manager eric is speaking at 1 30 and jeevan who's in the audience right here is at 3 30 so they have some really

great uh talks that you should definitely attend and that is it for us we are on twitter if you think we'd be cool people to work with we have uh job postings and again the slides are part of the b-side schedule so if there's anything you want to revisit later it is all up there thank you

i think it's lunchtime so if you guys have any questions we're here but if you're hungry then go get some food thank you