← All talks

Real Time Bidding, or, Why the Internet is Free, or, Why Does That Creepy Ad Keep Following Me?

BSides Peru40:5595 viewsPublished 2017-06Watch on YouTube ↗
About this talk
Real Time Bidding, or, Why the Internet is Free, or, Why Does That Creepy Ad Keep Following Me? - Sid Faber This talk will center around the online world of real-time bidding for Internet advertising. After presenting some of the structure necessary to support current trends in advertising, we will look at some concrete examples, explore trends in cookie policies as separate from privacy policies. If time permits, we will also explore typical cases for malicious advertising. By the end of the talk attendees should understand why ads tend to follow you through your travels on the Internet, and enhance techniques for traffic analysis.
Show transcript [en]

all right our first speaker is his favorite vidiq you probably already knows this paper he has fought his community built upon his face so if you know this isn't good or do we look like the part down there he's also I need to work with miss earth and he's parent lady and I have to read this one the director of information too small my eyes beakers years dark of information of security as federal agents investors I am your juice [Applause]

all right so going to echo some comments and you can you hear me without the mic I tend to wander and lock good alright alright so I want to say thanks a lot like everyone to the sponsor just this event and having the local audience here guys that I can reach out to you can reach out to me is invaluable in our community or community built on trust so thanks long intricated the only criticism I ask is if this event sells out faster than a Jimmy Buffett concert on Saturday at Starling or what you can be starving so it's pretty common I'm here today anyone politics about online advertising and I got to say first and

foremost them leads and I don't see any tinfoil hats in the audience and this is a no tinfoil hat allowed type of presentation I'm going to give you some background about what's going on with online advertising I kind of fell into it by accident a little bit about that and this this world is absolutely amazing again I don't see a lot of awareness about it in our community so I want to start going to lay a little bit of groundwork about how things are built on the internet and how advertising is kind of foundational for that hopefully kind of change your mind - Don Londoner Linnell online advertising really means to you because that then let me go back and figure out

what that means as you protect your network as you figure out what that means for privacy what you have to do when your next-door neighbor ask you about that creepy ad that keeps following them all over the Internet they'll be having in this creepy yet yeah so this is the southern belly fat week want everybody back right so and what I mean by I don't want to be paranoid about this is that if you know even going back to the earliest days of newspaper curiosity can you get a newspaper that doesn't have ads is they don't exist the magazine that doesn't have at it just doesn't exist no matter where you go you're going to run

into advertising advertising time you know it's just a part of our society and so as part of that I want to recognize that that all not all ads are evil in fact I want to propose to you that ads are instant improving jumping so we don't necessarily have to block them all let's see and try and understand a little bit more about what's going on here and this is again this is just an immensely complex kind of issue there's a lot of stuff if you climb this entry's Galasso online and the community is very open about what they're doing with online advertising all you have to do is ask you could videos wonderful videos out there because there are all the

marketing guides that are building these videos and they want you to understand what they're so there's lovely places where you can go to find out more the genesis of this talk is it's kind of like if the work that I did in this area started first there's some questions I'm an FBI about what's real-time bidding advertising start looking into it and then I went to work and I got a request across my desk where marketing said hey we want to put a pixel on our website okay that feels okay right it's just a pixel okay so this is essentially what we're being asked to approve from a security standpoint are you okay with this making

any problems with this you put this on your website if you did just like everybody else in the world all right these are everywhere all right it needs your tracking pick we used to think that they were evil because your ice cream X police and so on but now they're actually heavily used in the marketing industry but this not you're thinking what am I actually doing so let's take a look real quick at what it means an online advertising guy I know you probably can't see a lot as well kind of explain to you what these pieces are up but when you're on the internet you go out and you request content from a Content publisher pretty

standard you go to cnn.com and you ask for a webpage and once you ask for a webpage you get your content back but also embed it in there is your ad so your browser then goes out and request the ad that's pretty straightforward and then the ad comes back right and you're redirected to go get the content from an ad server that's usually somebody else in this might both be from CNN two different servers at CNN and then that has sort of where it's actually going to redirect you out to some sort of ad agency server may be a double-click server or something like that and then you build your page and you're done right you're familiar with that right if

you've seen that good down here right well the interesting thing is it's really not that simple if I were to number this in the simplest version there's actually like about a dozen steps before you actually get that ad back to your browser now that's what I kind of want to show you you missed apply some of that what actually happens between the time when you request an ad and you get the ad is that that real estate for you as an individual actually is sent out to bid in real time somebody said oh you know I have this type of person I have this type of ad campaign I put them together and I deliver you your

personal ID that this all falls under the ground will expand its programmatic bidding so if you want to look into this more you can start with programmatic bidding which cost a lot about cannot what is this automatic advertising involved back in here we have kind of four different quadrants the ones that we care about here's is in your work there's a fixed price for that ad that that's reserved for you so if you got a banner ad and you know who's going to drop their ad in there that guy sits in here and this is our traditional realm and that still exists today for some cases in some cases you might have a fixed price bag that goes down through a

number of different people that might choose whether or not to the advertiser to buy that has but where it gets really interesting is up here when that ad space is put out for auction and that what's known as real time business but we'll dig into this area a little bit more all right in order to do that I think you can't do this - well we're working down here about you know your area I want to go separate over here talk a little bit about the marketing area for men there's a number of folks that wants to sell you product matter of fact just about everybody wants to sell you something right so these are folks that have a brand and

when you have a brand and you want to sell your product you work with the marketing agency right so marketing agency gives you that overall how we're going to produce our brand what our statistics are you know who we want to get getting you that whole thing about how I Drive my business forward you may choose if you have a brand you may choose to go directly to an ad agency it just says you know I want to advertise my brand or you may do a whole marketing campaign somewhere along the way you deal with these folks you may actually just go directly to an online ad network as well but there's no match of business that happens over here and

this is a huge huge business right this online marketing business if we have a guess on how big this is in terms of GDP this is 6% 6% of the u.s. GDP is involved in this whole marketing and digital marketing effort it's extraordinary with the size and the crepitus this stuff this whole market all right so let's go back we've got this just ad campaign being set up brands that want to market to you that one advertised view that's going on over here let's go back to where we were with you visiting the internet and going out to your absolute so the first thing that actually happens when when this ad is is offered up by a CNN or something like

that is that it goes to what's known as a supply-side platform right a supply-side platform is where you set up your your view as a publisher a publisher can set up an offer real estate out for bid so this is where somebody will make money online so CNN wants to make money off of presenting you news they can offer up this real estate through a supply-side platinum right now set up rules and what like a real estate they have and what's like it can't be with such a face is that they can offer them what the good audience for that but it's all about them being on the supply side and offerings offering up real estate for a user

that's me drop an ad right so the thing to the supply-side platform might work on and things like if I ever seen this person before and if I clean them before but like the things are they interested what do I know about this person it's kind of individually talking about one website so maybe they could new things that you you view or so on that do I have any data also that's provided by a third party we'll come back to this but essentially trying to create a persona about you the visitor to publishers ads like that they need to make some revenues they can continue publishing content right so after that goes to supply-side platform and this is all

actually on the sell side so this is on on if you consider this an exchange come back that this is the sell side of the exchange whereas this is the buy side of the exchange right so these folks are all selling real estate so the next thing that happens after the publisher sends it to the supply side platform lifelike platform ads in and aggregates and stuff about you as an individual about what the publisher wants to do then that goes off to an exchange right and this exchange actually does work like a real goodness exchange where this is a bit alright so the real estate that in your browser is being presented of being built that is sent out for bid

it's embraced with additional information from the supplies my plan right the exchange may have some pre cash bid and may be able to answer this off really quick but sometimes it doesn't so it wants to send that out for bids it wants to offer up that real estate to other folks and that's where we get into the demand side platform somewhere before all this trades actually happen the exact agencies have actually worked with the demand side the buy side folks to say this is the type of thing we want to buy you want to buy people that are interested in travel vacation or interested in that thing that you looked at and I'm going to Home

Depot add or something like that that's all set up in our demand side platform I set up yep the algorithms or whatever so then when the exchange all first is now for bid the multiple demand-side platforms and to take the other exchanges another online ad networks now the demand side platform can offer up their bid just as how much I'm going to pay for that ad that's being rendered in your realm the big sense of for you follow me my go too fast good alright did you ever know this is going on okay so you see a little bit of it but it's crazy right so they offer up a bid and that the exchange does

actually a you know what you won the bid so so they will notify the winner that they've won the bid and then the winner actually will provide instructions back to the exchange or what's the display in your website and how the display right so that bit goes through and that actually gets passed all the way back through this system back to the publisher ad server this bid happens within believe it's 10 milliseconds all right so this is fairly fast transaction this old cycle should happen in less than a second preferably less than a half a second while you're loading up your weapons and then finally you get directed to the agencies ad server and you're set on your merry way to go and

deliver that any questions about this general club okay so you get the idea of the self side versus the by side setting up marketing campaigns and defining this whole project here and that finally you getting directed to an end so that's fine but that still didn't answer the question about what is it why do I draw a pixel on my website because my website is not involved in this I am NOT publishing content I'm just doing a marketing campaign so where does that pixel come in and that deals with these folks that are known as the DMP or the data providers right and this is where it gets a little bit more interesting so in the data management platform the

purpose of the data management platform is to coordinate across multiple sites across multiple days and to kind of understand a little bit more about you so I can actually target things that you're interested so again this is our flow here right and the question is why did this guy actually wins that bit we understood about the technology to kind of put this whole in place but why did he actually win that bid and that's what has to do with the data management data management platform actually deals with marketing agencies ad agencies on the on the buy side and then on the sell side they also work with publishers publisher content servers to get information about

the visitors right and then link all that information together so now when I'm at the publishers ad server going to work with the supply-side platform I can tell a little bit more about who's visiting based on data that's provided by this data management plan okay so this is passing a persona up here to match it over here to make a better bid make a better offer to get a higher base okay things that a management platform actually does this is this is where it gets a little bit interesting okay so some of the data sources that they do that they have or obviously you know your inputs and your visits to your site you also have email databases think

about your email things about your de place think about other campaigns that are going on all kinds of first party and third party data so things they know about you directly and also things that they might be able to purchase and link to your online identity so big information about you go ahead and knowledge about the computer consumer that the supply side providers and add the beef offer before that offer goes up to the exchange and they collect the data collect bugs data and do the analysis these are the big data counts and then their output exactly to work with online advertising and marketing campaigns and not just the online campaign but the overall marketing

campaign so they might be able to go in a data and this data management platform they may actually coordinate with a print campaign or something along this line it doesn't have to be jumped online it can be the overall marketing campaign so going back to that pixel now if you're working in corporate you have a corporate website your ads to drop a pixel on to your website so here we have you as a visitor coming to a website and dropping up pixel epics will will come back in to request and actually send you out in direct or indirect out to the data management platform so now if you're providing one in corporate website if somebody has had an intro

in my website so they visited my website the maybe are often we have going to a retail site and they'll drop a pixel on a retail place let them know a little bit about what you're interested in and the next time you visit somebody else in this ecosystem that that a management platform will understand and you're kind of interested in taking a vacation buying a new fence for the backyard or dealing with sellers belly fat right so the data management platform that has its pedicles if you will has its connection into all kinds of places within this ad infrastructure in particular when we talk about going to multiple content publishers the data management platform can have a one

platform can add LinkedIn to multiple publishers multiple ad servers and then the service providers these tracking pixels that show up on sites that are not ones that are actually advertising their services that your sister offer whether that's you're trying to okay you okay with that question so forth all right all right so one thing I didn't want to until I said this the simple hats but there's a little bit of evilness in the air when it comes to malware sizing so understanding how malware typing fits into this whole realm you know if we have this kind of setup here great you have probably seen from time to time when your users will get redirected out to a malicious site

through a tag okay and this is particularly challenging because you're down here you can see your user getting directed to hear you hopefully will be able to see that you know they ask for an ad and gotten the ad got redirected and redirected them to an evil place but the problem is you don't know where in this whole cycle that redirect happens right so that redirect may have happened because you have an ad agency that injected itself into this whole thing that it's less than reputable or makes just simply may not double-check things that are sent out for advertising and maybe if somebody on the demand side platform actually paid a premium for the user on your network it may be that

somebody on the supply side platform did something as well so this kind of cycle is subject to a little bit of frog makes it very difficult to track down what the root cause we've had a couple of really interesting events here where the left hooks don't go visit local sites right one of my favorites was he was a local newspaper tonight and our users were getting redirected to Pittsburgh mom calm right you've heard of Fitchburg dead if I heard of Pittsburgh dead okay then they heard of Fitchburg mom yeah no I never heard of it it was categorized as malicious apparently at one point years ago something was clop there right and so for some reason Pittsburgh mom was

you know it was a ad source a destination and we went back and we could figure out that users to a newspaper local newspaper get redirected out the Pittsburgh mom but who was it that was actually doing it my brother this those we've opened a lot of interesting questions about how you could actually target your user population because we can target you as a consumer so well arguably we fetch a target with malicious codes all right so you're okay with this general structure on how this is all set up all right let's talk a little bit about some of the players this is where for me it got kind of really surprising first of all publishers write compact

publishers do you recognize these okay okay is Facebook free sure it's free right again not really right if you think about what the space will know about you well not you guys cuz you guys security console hasta bug that's what the space will know about your kids and your parents all right and then put that into the data management platform right because that's how these guys make their money LinkedIn until they make their money all right that's how they provide this service is with the advertising right it's just like subsidizing your newspaper you have to get an ad in your newspaper in order your newspaper to cost beyond what it costs so it's part of the cost

all right but it's not just that right let's just share with everybody all right and if you think about it like all these social media sites they're all tags and you have to share this tag embedded all over the Internet right because that actually does a phone home to these content providers which puts you in the mix so now when you're visiting a site to have to share this ad and there's some knowledge that it's you that's visiting this there's an interesting kind of potential mix there right that's from the publisher side just walking through there's a number of different kind of faces around here so for instance you'll see a lot of this in

this industry where on the supply side platform Facebook does not have its own supply side platform but basal block library and library is a significant player all right so and you see a lot of these publishers down here that are making a lot of money that acquire different parts of this holy it's also interesting these ad exchange ad exchanges app connected to the large one open X is one but also Microsoft operates an ad exchange and we go to Microsoft and you'll see a lot of this where you have terms the clay on all sides of this ecosystem those on the demand side and the supply side usually they should be separated out to make the

bit fair but yeah if players on all sides this one you might find interesting as well who wants double-click how's that ok Google right and Google's your friend Kaku all the tons of stuff for you for free right yeah so Google has global click so Google nutzy that not in foil hat right I'm just talking organizational relationships I don't know how this whole piece is out but Google on the puzzle content side has double-click for publishers double-click based manager as a supplied by platform double-click operates an ad exchange double-click also has digital marketing and so they spit this whole entire structure right and Google now double-click can all feed into this gonna man this one also is pretty surprising to me

look at one of the largest data management platforms I don't think of Salesforce as being in the online advertising business all right but they actually are white a large data management platform along with Adobe all right along with work so really interesting kind of like food some of the players are in this market space if you tease it all out can you read this in the back okay because this is this is like the complexity and this is why this is 6% of the GDP right it's such a large kind of infrastructure of all kinds of things going on so I had a number of additional thoughts just things that are interesting when you work in this realm

there are some publishers that you can avoid so you might say again if you're being very privacy concern you might say I just don't want to deal with tracking pixels or I want to turn them off right well it can be as not you as as you looking up your airline reservation her doing your online check-in for your airline reservation if you look at the cookie policy of your online airline reservation they have third party cookies and we use this for marketing materials do they do it I don't know do they have the ability to do it certainly it's in the ability so when you think about something as innocuous getting your checking in for an airline ticket

if you feed yourself into this whole ecosystem you just can't avoid it all right and this is to my point of recognize what it is so then you can think about where is the security of privacy in fact I could track you by email and feed you into this whole system if you allow the embedded images once again you're feeding yourself into this whole ecosystem also mention that there's a trend in the industry to separate out cookie policies from privacy home they might here know if they're firm has a cookie a separate cookie policy so nobody okay so there are a number of and there's a lot of movement in the industry to separate out

your cookie policy and your privacy Falls alright and if you look at some of these two links I have in here if you look particularly at the Google cookie policy very well read very open about what's going on very clear about what a cookie is and there's also a lot of separation here between the United States maybe you alright so privacy means something very different in that you used to a lot of times there's be a very different privacy policy or you'll be forced to acknowledge of cookies because of the privacy implications okay have you heard about the ABS locking and coming up in chrome it's a pretty cool stuff right all right as an after built

into chrome sounds good right you know what the Chrome ad blocker is what's up filter it's a filter well guess other than Google yes yeah it's a it's a HIPAA and so Google is right all over this so and that's what I mean that's best that that's the point to Google being all over this and Microsoft being on and all these key players being olives you can't fight the momentum like you're stuck in this ecosystem so make it work for you make it secure for you on the Google standpoint the Google upcoming has Locker and CRO that's built in is an enforcement tool and if you look at this there's a lot of movement in here to to

good ads right because every going to like the ad that when you click on the webpage it starts talking okay that's a bad ad in the Google Chrome ad blocker will block do that because they don't conform to the Amster standards how about that one that shows off and covers your page it says your past is looking at this for 30 seconds and then you can click Next right badass and this whole industry is driving towards good ass another interesting feature about ad blockers you can say I'm let's turn on an ad blocker and get rid of all of this stuff and maintain my privacy there's Price Waterhouse Cooper did some good research behind this and they

believe based on their studies that we can get you to turn off your ad blocker two-thirds of the time all right and this is everyone all right so you included it two-thirds of this time I convinced can convince you to turn off your ad blocker I think about it you brought across it hey I'll let you read a part of this article but you got to turn on that before you get read the little thing or you can only watch the first 30 seconds of this video right so that's all built into that and and it's to make the intent is to make revenue while also making the internet a pleasing experience a lot of interesting

facets built into this whole thing all right so with that I'm going to leave it open for questions I don't know how much I can answer because this is a huge area but I just want to make sure that you're aware of this entire ecosystem of online advertising so that you can reason about what it means to you for the sake of security what it means for your most importantly for online privacy what you can and can't do any questions yes it's really the only effective way to break that name to lock and I think a server there somewhere a long way trying to fight it so two questions there can I break the chain and how can

I fight in West militias all right so can I break the chain I don't think you can break this chain I don't think you can break it any more than you can go shopping at the grocery store without having to step on an advertisement as you go to get your service all right it's built in all right so as far as malicious campaign when you have a malicious advert ads that's being delivered to you I think content management here works very well in reputation and I think also working with third parties that can actually clean this out it is in this whole ecosystem best interest to keep this clean and not allowing malicious act so these are very

open to reporting malicious ads and malicious creators obviously this content blocking only happens after somebody else got top you know and they told them that with malicious injury or like so first time user I don't know but I would say that's like everything I've read everything I've heard the whole ecosystem is very supportive up and keeping things clean thank you that looks like a good opportunity for spear fishing I mean you who've been insist to nose up there you can start causing individuals so this was like a good opportunity for spear fishing so think about it right think about where we went with the credit card industry where we started out with those that would pop credit

cards here and there you know and then they would pop like a large organization and then for a while there we saw folks pop credit card Clearing House you know and then we got these full sales millions and millions of credit cards got pops right so let's take that analogy you put it in in infrastructure all right fishing we see fishing down here right now against individual organization potentially we feeds a service provider fishing you know the stealing data for fishing so that we looked at DocuSign had a debt issue within the last few months and they're kind of like a general service provider what happens when these guys get off what happens if they lose their

data all right so I'm asking your question with a question all right is that something that we have to prepare for what does it mean when these guys your pump and recognize like what are their reporting relationships to let you know and how will I see efficient camping and that mystery bears out there's a good chance that they may already be for all we done it will do kids for today you can't track what he does today you can't track where your data is going and very challenging you can you can review again all these everybody in this myth is very transparent about what they collect from you so if you want to read privacy policies and so on you can

understand what is being collected but it's challenged yes sir so on I see a business opportunity here for what monsters that'll do operation

so George question looks like there's a business opportunity here right so and there's some funds that are taken advantage of it I think that's exactly the case but I think our traditional way of saying I'm just going to block ads won't work I think if you really wanted to do the tinfoil hat exercise what you would do potentially is presented different persona each time all right and and be a little bit smarter not just block because if you block ads your Internet experience is going to be miserable all right but if you can do something slightly different now maybe I don't know well I think there's something else here I think just simply go no-go

one ad is pretty much not going to work because you guys are smart enough that they're working around and they can convince you to turn ads on but to do something different like maybe George has there's 100 different personas in his browser all right and then and then privacy isn't so much of a concern because it's really not you it's kind of you know your doppelganger and the question yes so how does how do I identify a meat selling data about their users so how do I add feed selling data about their users fit into all that that's a great question that's related to the recent change by by the administration and allows ISPs to kind of get into this mix

I don't know all right I think it's kind of scary actually in putting this together I've worked a lot with our marketing group non-technical folks currently digital marketing group who on one hand really love this infrastructure because they can really target their advertising and make very effective advertising decisions and on the other hand they're the people that always add the up surf the internet with private mode on turn off all their cookies delete it because it's just plain creepy all right there's a people that first told me about the administration to change the balance ISPs being able to sell advertising so if you think about it and again to my point of not being able to separate

yourself if your ISP now sells or you know allow or share some information into the infrastructure about your address about where you're coming from that feels a little bit more creepy and marketing even hoping that yeah that feels kind kind of creepy right and that is that's very different from how it's handled into Europe in Europe as well I think it may be because we just start paying them for this much attention part of my point of thought to do today yes

so once I said I kind of stay in departments that you really can't break this flex can't you run some polls that actually block at so on we actually work a lot of this I like to think that that with our we like so at a corporate level we do content management I think most firms do that nowadays to prevent content and you can use your corporate content management the block ads as a specific category we actually have a lot of business problems when we block ads there's a lot of legitimate content that people want to do research on that they can't get to when we've blocked them at the corporate level so although you can

turn off a lot of this at the corporate level then you're also going to impact our ability to get the published content but that's the challenge that's the put poll that happens that can you do it yeah do you want to do it no potentially you could all right but you're not going to get as rich of content because that's of supporting this whole ecosystem yes what like the

without that so it feels like in order to do this you can go to the native management platform yeah baby good luck right and again this is this is a whole huge monetary ecosystem this is advertising but you are takes down what you have to do is makes levels that make it less effective and that's what maybe maybe there's a new improved ad blocker that lets down there that makes that less effective for us but not for others longer interesting John I don't have solutions for this right I just want to make this evidence to you and also I want to state that that please don't take my statements as authoritative take them as informative

and it takes them away and do some some looking into this area look at it from a privacy standpoint look at it from a security standpoint again this is very very transparent they're really letting us know what they're doing I think we're not paying a lot of attention to all right so I'd encourage you to take a look at this and take it home and see if you can build a better ad blocker build a better content management platform feel better privacy logic into the way that you present your Julie right with that I believe my time is up so I unfortunately have to get back to the office [Applause]