Sprinkling Wireguard With Vault And Marshmallows

good afternoon thank you for coming it would be kind of weird it was just me and myself and okay so sprinkling my regards with vaulting marshmallows it's the name of our presentation and we have split the presentation with within kind of six chapters we will speak a bit about s state of affairs that way it is to this simple storyline just to explain the concept then we have a story to kind of illustrate what we did in then there's the backstage and future work so about us hi my name is Marcel you can see I've been software developer in the past I failed kidding I turn site reliability engineer and it's what I do in the

current times and so my name is Erskine I'm was network engineers software developers engineer engineer so state of affairs how we got here so we were working in an organization that as many has some form or another remote access and in our current EPM solution kind of answers the company main requirements but it has some kind of constraints it's not bad it's not good but this is the job however there's one small problem with our solution is that users hate it and the user experience and so can the potential for insecure work around since really big and usually today especially it's easy to kind of pop a new instance in the internet and try to access

resources that should be private and that then somehow publicly available in the Internet this is a big problem for us so one of the complaints was connection instability we work in a company that it's really remote friendly and this means that users change networks a lot and this leads to kind of instability problems within the network's themselves and by moving around the VPN connections change to kind of fall and people need to reopen ticket in this part of the authentication is another one of the problems because we currently have kind of a quirky two-factor authentication solution having a two-factor authentication solution is a kind of good thing but our solution is not that great and the authentication process

really takes forever this leads to kind of developers losing a lot of time just trying to reconnect so this was kind of problem so we started asking for help good for another option and we learned about your recent system at the time that would be probably included in the Linux kernel and what's like a kernel module and it had lots of technologies and crypto shenanigans that we thought it were really interesting and there can work out this is kind of the marketing lingo from the website if you want to check it out simple and easy to use preferably sound etc so seem promising however there's one problem that's usually when trying to operationalize this kind of stuff which is key

management and this is spread out across all the technology called parts of maintaining an infrastructure securely so we kept asking for help and then after asking for Kenneth what exactly the word the pains of the users and not trying to reinvent the wheel and using kind of stuff that we already were using within our own infrastructure and we wanted something to be easily integrated so that involved and this is I don't know if anyone or anyone does not know how but it's kind of a product from a jacquard and this is these are the kind of the features the main features do it secrets management that the encryption identity based access auditable and it's easy customizable by plug-in

architecture which is something that we'll explain later on so so many promises ok let's do it and this is the sort of I'm sorry yes the storyline is system then first should make it easier for to explain what we did and how its kind of created the problem or created explain the problem that we have so there's a caste and because there's kind of continental members there's Winston Sharon the optional manager there is another part of the team which are the curators the administrator curator adjudicator then there's the Sakura also known as the high table composed by ego binky fellows and insensate and there's mr. obviously will be spotty and there's the problem the patent so this is kind of a set we

have the users in these splits in two parts one of them is provisioning the keys for the users that part can be then kind of without being connected to the Internet and you can provision in all the keys involved and the other part is for the axis itself and for us to access these particular resources so the puppy and the high table you need to access the accessor or the VPN which is connected to the Internet that one so talking about the storyline we start by having a user that wants to reach the protected resource and for that he needs to require the blessed account to do that someone needs to provide that blessed account and then the user can can

actually access the protected resource later on the key is it's considered compromised and will be revoked so the treatment the generic individual it means the the one that's entire access to the VPN cannot detect the VPN service because it's and cannot impersonate any member the member of continuous module is a simple client can be with all sorry it's a like an auditor guide that can view the access logs and can view who is using that case decorator sees the typical client is he can view that flight a cocaine provision new members and can change public keys from his members he his success is detected and the changes and everything may fail to revoke that he's the member of the

seclorum is the administrators he can view the access of a table provisioning keys changing keys of other members and revoke access he also can change the server configurations of course all of these previous operations are logged and can be detected so getting the puppy mr. Ricca as a puppy and taken by the member side table console and the populace in a special chamber protected by a cyber cell Arden postponed the mechanism of that stuff controlled by the a table and at this time it seems to auto manager to talk mr. week how may I be of service I need access to protected resource under all sort I need access to resource on the protection of a table you need a

blessed account I understand where I can acquire one it is not something that you can acquire very well who has one well mr. Benson mr. week uses his special charm to earn the right to own an account and the way the system is set up requires a new key to be registered mr. week Wow politely asks for mr. Vincey to provision his account his new key and then this fellow because she's the second day because internet fell because yes that means it happens it's not a real them win favor just give me a minute sorry the internet will return in a moment

hopefully yes connected so back into the movie work but this time the guy that needs to access the resource needs to provision a key for someone that can put this key on the on the server that means mr. week needs to send this key to an administrator or something like that this is what happens here this is the public is concerned to open Internet like an email or other mechanism like copy pasting that web what is happening here at this time mr. Vincey gets the key of mr. week and put it on vault it's allocated an IP for mr. week and the configuration is updated without compromising the current connections on the Vivian this is what what you can see

here here mr. Vincey we'll start by logging into vault third secure password as you can see and you can see that it's a guy that has privileges we're gonna see this later on this policy he also you can see we have a token duration for eight hours so this means that with this token we can do all the operations on vault during eight hours starting now and then using the previous token you're gonna put the mystery key on the server this will generate some configurations on on what quite server and returns templated configuration to be used by the clients as you can see we have the IP that client will assume it's 1 7 to

16 1 2 and we have expert time for that key on server that will be on the pressed it's already open and also we have a template of configuration that client can copy placing and use to connect to the server so this is what happens next mister we get that configuration by put it on his computer and access to the protected resource

copy-pasted completing parts updating the client he needs to replace the private key fields with this sound private key that generated the public key that is sent to mr. Vincey and after that we'll be able to connect the server [Music] type over and then at this point whatever it is configured and is able to reach from client to the to the server

and reach this puppy that's what he wants as you can see we have connection to the server where is the puppy so moving forwards what damage that's usual so moving forwards while mr. will weekly moves forward with his endeavor the Nullification of mr. Vincey does not global and notice and so the house the high table housekeeper reads the accessor logs and there it is mr. Vincey with a brand new public he this was unprecedented mr. bings he was emotionally attached to that key the housekeeper didn't understands who may be behind this behavioral change and he's another point in sirs mr. wick therefore asks for the presence of the most capable height civil servants and well mr. wick has valid arguments his

puppy is now safe behind him a pile of corpses lie on the tapestry that welcomes newcomers to the high table and the high table declares mr. wick excommunicate oh yes again and revokes all of mr. pings his excesses actually this - and this is what happened here someone from the the administrator part Socorro revokes the access of mr. Vincey he will able to lease the current apply of T's on vault that is also deployed on the server via volt by listing all the clients we only have one for this proof of concept but we can add multiple ones as you can see was a key that was provided for mr. week by mr. Vincey and

then revoke the key or delete a key from vault that will be deleted off on the server what about certain we could do it faster so no keys on server right now and shaking on the server there's no more tears to be connected well also we can check that cleanse is not longer able to connect the server to reach this puppy or the pup is no longer there so since these access is revoked truly it doesn't matter because we show it doesn't care so you can have the best of breed security system but you cannot get between mr. week and his party as we all know so regarding backstage backstage is we are going to explain

each part of this story which is the fun part yeah ok starting by the vault as we previous mentioned we don't we are ready using vault to do some secured parts on our network by generating tokens database credentials and other part fun stuff that vault provides but why you are using fault because of the goals that is being designed for for receive so needs to be a single source of Secrets his these secrets need to be programmatically easily accessible and needs to feed some of that on a multi data center architecture that directs us to the multi-valve trending world that we are living today this this means the does not need only to be compatible with

multi cloud providers but needs to provide a secure way to transmit that between those those data centers is that we can occur in a secure manner and also needs to provide access control and provides also practical security above all the system so regarding the features of vault he's he provides dynamic seems that evil mentions that like PK certificates SSH certificates database secrets and so on he also has ability to write reach HCL programmatically that means that some resources only can access specific parts on vault and others the same and this is very important because if one of application is compromised only I think I use compromised that application is only able to reach the secrets that that

application has able to to read this this is very good because as many secrets you know it's better and it also offers flexible authentication and auditing overall regarding the secret to secure the your secret storage although that on vault is encrypted by meaning by thing although that is the data that it's at rest and in transit using Tilia soon 1.24 communications for transport layer between the clients and vault and use encrypt all the things on on a with 256 bits using AES in gawad control mode it is quite faster than usual cipher block chaining and this also provides authentication by by default volt also use algorithm College summer secret sharing to build a master key that it includes all the data this

it depends on the configuration but for generating those that master key you don't need all the parts or all the shards of these this sheet secret sharing but some parts only to reveal that secret key also the very important part the living and renewable reservation all the secrets on vault in token says as at least time that can be renewable or revoke it before the the lease actuate also the auditing vault locks everything locks all the requests in the response but does not start the secrets unlocks some point in time well we already seen applications that do that but this is not good as we know and all the secrets are at our action this

means that they are searchable but cannot be reversible the decrypted also we can put as many weekends we want because this is very good because we can also have a backup backup of the auditing but we can also compare two logs to see if one of them has been tampered with or not it also provides safety available this means that if vault cannot out it or log something it won't return the response field which will fail before and finally regarding the authentication photo vault offers multiple backends for authentication like tokens open Ivy certificates user password to be very easy to use by machine audiences like applications or we user-friendly

the kind of the motto is to be an extremely simple yet fast in modern VPN that Jesus state of the art chromatography and this these were kind of the things that allowed us to think that was really neat solution although it states that for the time being is not production ready but the design principles are close to our heart let's say so it's easily auditable it's supposed to be simplicity of interface the static fix flying feathers static guarded state allocations in guarded states stealth solid crypto and abuse resistance and we'll explain a little bit about each one of those so it's implemented in kernel space so you don't have kind of the problem of being an

application in user space that has a lot of contact switching which makes it also very fast it's highly opinionated this means that for example Open VPN or IPSec can have a lot of configuration for cipher suites etc and not wire guard where I got kind of really restricts your options they are basically men what you get is what it's out of the box and this has obviously advantages and disadvantages but statistically speaking I think the advantages overcome this advantages it's a real three only solution and this has also advantages like we will see a little bit later it has less than 4,000 lines of code and this enters in the parts that I was speaking for about being auditable that

is the kind of the footprint of enough codes are few so it's easier to kind of audit the code itself and it is operationally coherent with common networking tools reducing operational levels easily what does this mean this means that this is kind of one of the principles when this component was built and what this allows us to is to have that part there of simplicity of interface and so the persons that are currently kind of managing the servers are managing the network don't really need to understand thoroughly for example for all the options that IPSec provides you to or even OpenVPN about certain configurations because it's given to you out of the box also regarding crypto where God really takes

advantage of the noise framework which is used also another stuff like whatsapp and signal etc and this allows it to be it's kind of the one round trip for key exchange and it uses curve 25:19 for the points for building the the public keys using elliptic curve diffie-hellman and it uses for the symmetric crypto parts Shasha 20 with poly 1305 through the encapsulation of encryption of the UDP packets that are encapsulated wire that uses UDP and it uses like two flashing and public keys it's themselves because it kind of builds a routing table around those public keys with the targets use a hash table that takes advantage of the function of sip hash to for Mike this

also isn't intends to be really fast and so not vulnerable also to collision attacks and it has created a new crypto key place mitigation for do s attacks it's and this is not kind of a new idea but they creates kind of a different protocol for building this schedule cookie and one of the properties interesting in this particular protocol is that wire guards deals with both endpoints the initiator in the responder they both can interchange roles and that means that for example the way the crypto cookie is built is in order for the not the clients or the initiator cannot do s the responder and the response cannot us initiate it also has on particular property which is no

resources are allocated for an unauthenticated packets this is goes into that also part of stealth so for example if you have a network scanner and you don't sense authenticated or valid packets to the two-wire guard it won't it will simply drop your packets in it won't to answer you indeed during the it has kind of a two minutes session that it rotates the the session keys for therefore the VPN session itself and it provides DFS perfect secrecy this has one very important property that it takes advantage of which is crypto key routing and it connects kind of the the public key will be IP address the one that we were seeing before and this is

kind of tightly coupled cryptographic website wake up and so this is an interesting property that it takes advantage later on like we will also show one advantage of this this part of the routing enables that solving the the problem that we mentioned at the beginning of unstable connections because it has the capability of roaming the connection between similar to much that's why it's expander in this means that for example if we have a VPN connection now at this moment we can cause this laptop imagining that it has a Wi-Fi connection here we go home open the laptop again and we would still have the same at the same not the same VPN connection but it would automatically be

established with new session keys so dealing with mobile users and then for the office to be possible and then we develop a deployment that integrates fault with vari parts for that we also do that we established some goals that planes need to to be able to solve us the those goals are needs to be needs to simplify the white guard configuration even more my simplified by simplifying the configuration means for the server and for the clients and they also needs to need to revoke the access and auditing all all the connections so the the features of the plug-in we want to want to add peers dynamically on server stablishing establishing TTL for those keys and build a simple date DHCP server

that we can match the IP with a public key and get that matching information we also want to provide the configuration for the clients so we can easily connect to the server so we define four endpoints for the plane the main part that we'll provide them in straighter parts like deleting keys or or how do you think the kids that are deployed on the server the client part that is for the client interaction to send public keys for halt and after that those keys be used on server the configuration part that is for configuring the plug-in like establishing the site and walk off the meds that were required we will use the TTL off the keys and the public key for the

server for example the private key in the public the private key is stored on another part but the public key and the server part that will be used by the server to read the keys and create interfaces on the server the white card server we also we use called console template to read those values from from the vault and deployed the server configuration so the process is clients rights to vault console templates reads from vault and the configuration whatever then we lost it and finally the user can also can can access whatever this is the process where we installed the plug-in on vault and enable it to do that we use another super secret token to do that remember

this is only a proof-of-concept [Music] locates the plugin and for this case we are under under vault plugins and install it by providing the 256 of the the binary to be registered on on fault this is very useful because if those this binary is tempered vault ignore its after that and this this is good because we cannot do changes on binary to do some bad stuff this is the expect of the help page for the plugin with the the endpoints more detail and we talk more detailed about the more detailed about the endpoints the two ones are the admin parts the first one is for listing the keys and the segment's for reading the details of one specific key the client

already told what it is is for the client right this probably key to server the configuration is to write the confirmation server and the server and is for leasing in the last one is for reading the key on vault we also installed some some policies that disable or deny some clients for for reading the some possumcods sorry I mean denied but vault denies by default and only allows on this specific policy this is user why I mean that means the can do everything on the the plug-in path that since we get up with VG this is the client one that is able to write right he's only public key because it's only havin a lot of parameters to write to

the client the client path and this is a read-only part the rhythm of the part is for example this is an error right because it's not should we read this is only this it should be answered but is only to read it's like an audit guy that comes to halt and shake what is the configurations and the clients that are deployed at that time and WG server this is where server used used for with the public keys and provide the provisioned the configuration this is an example of the right configuration where we establish you the time to leave of the keys the sidewalk of the the network that what word we will use and this

Caesar is used for the the DHCP client generate new IPS for clients we also provide the endpoint that needs that will be used by the the endpoint and the public key that will be used by the clients for the template part this is a video with the public key I think I'm gonna pass because you are running out of time and this is a client example where the client send this public key to the server to be used and template is on the server and this is an example of the return response when the clients send this public key to the server as you can see we have the expert time of that key the IP address that will be use and a

template of configuration that client could use later on and put on is on each side to connect to the server we also can pipe to some pipe magic and generates dynamically the our private configuration by generating a key piping to vault by drop by generating these public our public key Doucette after that and use the generative response on the cost on the server side we are using console template as we mentioned previous measurement to generate the configuration and this configuration will list all the secrets and read everything to put on on on the server this is an example of the configuration generated that leads will have all the peers connected to the server and its

associated public key so one of the the parts that we wanted to ensure was that she we had accountability by the usage of all this solution and so what it was an important part of it and so we wanted to keep a detailed blog of every interaction with walls and with some specific exceptions like myself mentioned before all strings are run through an edge map and this means that when it's written into the other talks you really end unless you know the secrets you do not understand what's in the other table and this is provided by default by fault however you can still search for something some string that you know for example if you know the

public key there's an endpoint also Indian volts that returns you the Ravel or the the calculation of that HMA with the secret that it's with involved and so you can use then that image to search into for example the example that we did was we placed everything in elastic search and access it by equivalent and in this particular case only the deletes appears there with the public key but the play and creates etc you would need to do that part of the default that is to calculate the edgemax of the republic and then search for it then this is part of the problem so this is the kind of the part of the the provisioning of the keys in themselves

but then you can get the access to the VPN and like like we explained before wire guard is a kind of layer three only solution so this enables kind of other stuff and since we have kind of this strong cryptographic mapping between the client IP and the public key we can use common IP monitoring tools like any other tool that's usually people used for networking stuff being these are kind of two examples night flow or IP tables connection tracking and we since it was easier for us we just leave an IP tables with logging of these particular connections that's everything that goes through that virtual interface network interface of gg0 created by a wire guard

we can simply log it and understand which clients connected to where at what time and this is again this is only for the example is there's only one really one connection or one client one server so this is kind of boring but it's just making a statement that it's completely possible and easy to do again it was using heuristic search and given it to reach red cards wait sorry and there we go again Wow amazing future it's so regarding regarding the future work we want to any instrumentation by writing godox on on the plugin because this is writen on gold I think we didn't mention that all the planes on go for halt all right and go because well

record provides an awesome SEK with with all the abstraction for that we also want to open start supplying so community can know using it and build a let me stick on top of avi plots we also want to improve the test coverage and later on we want to enable that only a specific user will be able to produce news on key this is a bug that we have right now and where are the marshmallows like we promised there and this is the video of the full process no he won't play okay questions no questions question over there because going to is going to make the question I think that's one question hi dears do you support private key stored on some

cryptographic tokens like on smart cards or tpms and and stuff like that no for the time being unless I'll do that we not do this those thing on the plugin yeah that's given to you by involved itself because Holtz has a interface for HSM for example using pkcs 11 so if you want to you can just connect that to hold but what we thought was interesting for us it please because we use what was that you can take advantages of those properties their votes out of the box so but this is for the part of the the private key of the server yeah I was I was asking regarding the wire guard regarding the VPN

connections like yeah from from the client-side whatever you want like you said you can store the the private key in the smart card or the the important part when kind of the the creator of wire guard what he wanted was to make kind of parallelism with OpenSSH and so whatever solution you have for OpenSSH you could use for okay so you're just using public private keepers without wrapping it into a certificate it's x.509 no x.509 okay and itself that's one of the things at the beginning that's it uses the the curve 25:5 online and for for taking the the points from that curve suit so that you can have a 16 bytes public key but not with x.509

because that would mean a lot different problems and you know like OpenSSH what you want is kind of something very similar simple and that's the purpose okay thanks okay any more questions one more here this one first places yeah hello so how do you deal with the high availability of vote the vote no sorry [Music] in terms of the vault implementation they use enterprise version or no no the question is for the time being we are using only the open-source version we deal with the AI availability we are dealing with console or for the backend storage okay okay thanks here hi so I saw a lot of ipv4 addresses what's your ipv6 readiness it doesn't

matter because it's a layer 3 so whatever you put in ipv4 it's the same variety v6 it really doesn't matter yes our pot is only on that before it all yes only our proof of concept is in ipv4 because we are old-school we want the Internet to burn but that's only its it has the capability of simply creating virtual network interfaces on the Box on the operating system but it's not connected to all ipv4 itself it doesn't matter whatever the operating system gives you it's what it can use right and another question the key generation and then the putting the the public key in the vault that looked a little bit like a certificate signing request in public

infrastructure setting in the example that I give you so this problem is exactly the same problem with anything else about public is like OpenSSH we are talking of for example example right you're talking about this any any example about putting their public key right yeah yeah so public keys yeah so at the beginning sorry about this for anyone with problems about spinning stuff so what if you can do whatever you want hmm but here for example what it's trying to kind of illustrate is that you can use kind of a trusted environment to store or to provision the public key of the user okay but the other other possibilities for example if I already trust another public key that you have

you can send me kind of a message encrypted with your PGP key and I can open that because you can send me because we have a trust relationship right so the important part is to have a kind of a trust environment where you can share those keys but this is a public key if this map is only only the armor of the but private key can use that that connection for questions no so thank you guys now there's Cimmerian