A Retrospective: Security Incidents and Lessons from 2018

good afternoon everyone so I'm gonna be talking about 2018 interesting events that happened over the past year and lessons that we can learn from that just a quick introduction my name is jarred no dear I'm a cloud architecture software engineer specializing in security and cryptography I'm a certified AWS Solutions Architect and DevOps Ingenium my research interests include privacy and security working on a number of other certifications and my twitter handle is at jerod Nadia I work for a company called synthesis we do specialist software developments and cloud consulting for the financial services industry so we work with banks and other financial services institutions to build out their digital channels online banking asset management systems credit

management systems markets and payments integrations regulatory and compliance reporting as well as security solutions we're also the first advanced AWS consult important in Africa and the Middle East and we are hoping to be premiere next year and we also got our financial services and DevOps competency this year my day job is essentially working with banks and enterprises to help them architect build secure and operationalize their cloud environments so in this talk we're gonna cover a number of topics first from a global scale and then towards the end of the presentation we'll get it will become more relevant to South Africa so as we kick off with acquisitions and trends over the past year we saw a lot

of security vendors basically improving their threat detection through the use of machine learning Big Data and AI both on anti viruses as well as EDR products as well as the some tools then we saw a number of improvements for getting aggregated views across environments to make defend the life of defenders easier as well as automated Incident Response through security orchestration and automation so that when you actually do get alerts you can actually respond to that then they were in a number of acquisitions so Splunk acquired Victor ops for $20,000,000 bomb co acquired beyond trust as well as mom cause the quieter Tata and then we had a local success story way if secure bought MW are not

all of the acquisitions however were actually positive went earlier boats Ingrid for two billion dollars analysts really didn't like that and that's in the tank stocking you know tank tanking about half percent and when Microsoft bought gets up for seven point five billion dollars again analysts were very concerned about the future of github as well as people in the open-source community and when IBM bought redhead for thirty four billion dollars as a play in the public cloud space we saw a number of Linux distributions issuing statements in supports of the acquisition as well as other Linux distros issuing statements in the disapproval of the acquisition so I think in 2018 we can say that it will

certainly the year of supply chain problems and supply chain attacks and we cannot have a retrospective without speaking about probably one of the biggest stories of the year which is the Bloomberg story about super macro in which Britain burgled ages 13-17 sources that the Chinese government what through the use of bribery and corruption forced super macro motherboard and silver manufacturer to insert hard way backdoors into the motherboard during the manufacturing process one of the key elements of the story was a company called elemental technologies which was acquired by Amazon in 2015 and has contracts with the CRA as well as other US intelligence agencies now it makes sense that the Chinese intelligence agencies would want to gain access to

the systems so this alleged chip is a 6-pin SOR seawater gray in color and at least on a theoretical level depending on the bus that it was placed on the motherboard it could basically pose a lot of authentication and could mess with other systems that it was connected to after the story came out SuperMario stock tanked by 40% and both Amazon and Apple issued very strongly worded statements denying that this was actually true and it's really important to understand that these statements are regulated under the SEC em bar sarbanes-oxley so if it turned out that these statements were Inc were incorrect or false they the boards of those companies could actually be held criminally liable for defrauding their

investors which definitely added to a lot of the haze that was around the story and as Tom went on a number of holes started appearing in the story firstly Bloomberg sought 17 anonymous sources as part of their story including named sources like Joe Fitzpatrick who is a hardware security researcher and when Bloomberg approached him to comment on the story they're awesome you know how would such an attack work and he came up with a number of possibilities and it is very interesting that every single possibility that he came up with was actually confirmed by the sources which is a major red flag in addition to that the sources behind the story or the journalists rather they have a very bad

track record of kind of bending the truth and they have in the post created fictitious articles that have been widely do not in the security community and then is also the question about the practicality of this attack because if you wanted to be able to break into systems on the Mond it would probably be more practical to backdoor the firmware rather than trying to have a hard way backdoor implant what was interesting after the story came out there was this narrative in the US that we can't trust China and basically everybody should buy us a gear which is very interesting because at roughly around the same time period there was a ban on war way and

ZTE equipment inside of America Canada Australia and New Zealand and so three years ago our published a paper looking at the Edward Snowden documents and I basically made an argument in their paper that we in the security community can't look at intelligence agencies as you know friends we should view them as adversaries and what we also saw in the Edward Snowden documents is that the NSA has their own catalog for odd way back to implants we also know that the NSA routinely intercepts Hardware so in this case they intercept Cisco devices where they put their own hardware backdoors into those devices during the shipping process so it is completely possible their Chinese intelligence is doing the same thing

however in conclusion we can say that the story as reported by Bloomberg is a load of rubbish there's a lot of facts that have been disputed by many sources including the Department of Homeland Security and UK intelligence however I do think that there is more to the story than what is currently public the in staying in supply chain attacks Ticketmaster was breached earlier this year a number of customers in Europe had fraud on the Accord and Monza who is a start-up Bank in the UK they had customers that also had fraud on their court and during the investigation process they ran a bunch of statistical analysis on these transactions and they found that Ticketmaster was most likely

the cause of the breach court information and they've been contacted Ticketmaster telling them that you know they probably had been breached and ticketmasters response was basically while there hasn't been a breach as opposed to we haven't found it yet and then Monza found the smoking gun which was they had one customer that only had one ecommerce transaction and when it was used on Ticketmaster the transaction was declined because the expiry date was incorrect and then the fraudsters tried to make a transaction which was also declined with the incorrect expiry date and that effectively was is no other plausible explanation other than Ticketmaster was the source of the information and the Ticketmaster breach was caused by a third party that they

use which was breached so this third party provides a third party chat support and they were breached and in this the attackers edit JavaScript to harvest these chords and the group that was behind us is mage caught they hit a number of other companies including British Airways Newegg and Fatah far including thousands of other smaller ecommerce sites we also saw the browsealoud breach which is the third party servers for accessibility so people with hearing and visual impairments can use their software to basically read websites and it is used by over 4,200 websites including a lot of governments including the u.s. the UK Australia and New Zealand including many others the question then arises how do

we defend against these attacks and they are basically three ways that we can do that we can use sub resource integrity where we create a show of the JavaScript and we add that as a attribute in our script tags so that when the browser load to the JavaScript it will compare its hash to the hash that you have specified if you do this it's very important that you version your JavaScript because if you have some kind of automated process to create your JavaScript this might actually break things the other thing that you can do is you can use content security policies where you restrict which domains you load JavaScript from what you can also do is you can restrict which content

types are loaded in addition to that you can then actually send these violations to your own reporting infrastructure or just to make things easy you can use the service court report URI that is actually very economical and it makes it easy to actually roll these things out then with the Roz in pub in the popularity of games like fortnight in pub G over the past year there has been an explosion in online game cheating so online game cheating is not really new about ten years ago there were a number of black hats sorry def con talks you know tracing how you can actually cheat in online games but by manipulating the execution stack memory pointers and registries

you can do things like create bullets that appear from no way you can have unlimited health ammo and resources you can jump around in the map you can see through walls you can shoot through walls and you can do those types of things now because a lot of the processing that's happening on the client side it's actually very difficult to defend against this because a lot of online games they the servers just basically act as traffic control for what's going on so what what a lot of game companies have done is they've created very intrusive anti-cheating capabilities on their clients which have their own sets of privacy problems and people in China specifically are actually monetizing online game cheating

by they will join you in a match so if you are playing pub G or fortnight they will make sure that your team wins and when you have games like Tom Clancy's the division where you have a leveling up system they will join you in game and then help you level up foster then in May we had the enforcement of GDP or and basically a massive amounts of hysteria around it kind of everybody lost their minds including a lot of publishers a number of websites closed down and after we were bombarded sorry after we were bombarded with email about companies updating their privacy policies a number of people outside the EU were basically sorry we're basically searching how we can

block EE uses from our websites so there were a number of ways one was using cloud flee workers the other one is basically using SAS as share services which is basically JavaScript that you run on your website and it will block European citizens from visiting your site what was also interesting is that a lot of websites including the USA Today when they removed all of their tracking scripts to be compliant of gdpr it actually reduced the amount of bandwidth that you needed to load the website so that actually resulted in a better experience for everybody but a lot of this hysteria was misplaced because if you actually go read the GDP or legislation it's not meant to be

applied in a draconian way it's actually somewhat forgiving and it's only on your third strike is the maximum penalty actually do so a lot of the sites that closed down while some of them were not compatible with GDP or completely due to how they handled data a lot of them closed down without any you know real reason or valid reason I should say so as we move on to vulnerabilities you know 2018 had a lot of interesting vulnerabilities I've just selected two that were personally interesting to me the first one was a vulnerability that Oracle published in October of last year which was a vulnerability in WebLogic in December of 2017 a Chinese security researcher published the exploits to

exploit this vulnerability and then once it was released it was basically turned into a mechanism to install crypto manners on vulnerable systems and one of the most vulnerable or most impacted systems was systems running PeopleSoft which is a H or middleware application and the attackers are taking advantage of vulnerable people soft systems men - man over 250,000 dollars in Manero and basically how the vulnerability worked is attackers would exploit a system kill any other miners that were running on the system and then creates a cron job to constantly download the mana and run it then the other interesting malware was the VP in Fulton malware which took advantage of a firm way owner ability running in a SUSE d-link huawei

ubiquity and a number of others millions of devices were actually affected by this and the malware was a multi-stage payload so the first stage created persistence which is actually unique because a lot of malware that affects IOT devices when the device is rebooted the malware is not persistent the second stage then went to go download the actual payload that would intercept your HTTP connections to get credentials once it had those credentials that were then sent there to a command-and-control server and then the third stage basically added additional functionality including the communication of a tour as well as a number of other features and when this happened I went to go set up my own set of honey pots to look at what

was going on I saw a lot of traffic from Russia the Ukraine Brazil in Egypt as well as some other countries and what was most interesting is that in on my honey pots that I was running I found three unique groups of attackers and on a random basis but between 3 and 10 minutes another attacker would exploit the vulnerability wipe the malware of the previous attacker and then install the own version and this went on the whole day and basically the whole week that I was running these honey pots then as we move on to HTTP and encryption so earlier this year we had a new version of PCI DSS which mandated the use of TLS 1.1 by the

30th of June as well as additional requirements for multi-factor authentication we also saw a massive rise in HTTP adoption where over 70% of the Internet today is actually encrypted verses 30 to 40 percent three or four years ago we also had new tools to visualize this year ecosystems cloud flay developed a tool called Merkel town which is based on the certificate transparency logs and it can actually show us what's actually going on in the CI industry we know that about 700,000 certificates are being issued per day the majority of their comes from let encrypt and elliptic curve cryptography certificates are still in the minority I think primarily because a lot of services especially cloud services do

not support elliptic curve yet we also saw the death of public key pinning so if you're not familiar with public key pinning it is basically a mechanism for you to pin your public key so that when you have a returning visitor they should expect to see the same public key the problem with us is is that on systems where it is deployed or even systems where it is not deployed if an attacker gets into your system they can turn on this feature or actually delete your certificates and create a new one basically creating a massive denial of service attack and a number of companies were actually affected by this and both Mozilla and Google kind of had to you

know bail them out through an emergency patch and so this is actually going to be deprecated or has been deprecated in a lot of the major browsers we also saw distrust for Symantec coming into force this year so due to immense semantics bad behavior over the past year a few years Google decided to distrust them Symantec they insult the CA business to digi cert for a billion dollars and that then led into the next incident which was the trustee Co incident so trustee ko is a reseller of as certificates or Symantec certificates which was now earned by DG cert and they wanted to move all of their customers away from Symantec over to Komodo and in

a pure money grab the CEO then emailed a juicer to revoke all of these certificates and digi cert refused because customers should revoke their own certificates and then the CEO email 23,000 private keys to digit and under the sea a forum rules digit had to revoke them which they did in four hours like good job guys and this and a lot of people actually had outages due to this because you can imagine you know if your certificate has been revoked and you don't know about it and your customers try and connect and they are verifying whether your certificate has been revoked or not that won't work it was also then came out that trustee ko was generating all of the customer

certificates and storing the private keys on the infrastructure with no protection no encryption no encrypted data stores no use of hardware security modules nothing like that they were also vulnerable to both sequel and command injection on their website so you could actually just go through their websites and get all of those private keys and we in the security community cannot allow such reckless and irresponsible behavior we in the community need to hold these top resellers and CAS accountable for the actions and this speaks to larger issues in the CA community so Komodo CA was acquired by Francesca partners which is a private equity firm which also owns the in is a malware group so the NSM

alwah group basically creates malware for governments and I think it's very problematic that a private equity company earns both a company that does who is a certificate authority as well as a company that creates malware then with the increasing developed of quantum computers a nest created a call for papers basically asking people to submit papers for new quantum resistant algorithms and in this in 2018 we entered the analysis phase so these algorithms have been created and now they are being assessed to see if they have any weaknesses what is very interesting though is that is the strain that has emerged to mix strong AES encryption with quantum resistant algorithms and one of the companies that

are doing that is a company called senators which is an Australian company that makes layer 2 line encrypted and I think the trend of mixing a is with quantum resistance cryptography algorithms is actually it's quite cool then TLS 1.3 was approved by the internet Engineering Task Force and it replaces TLS 1.2 and a number of features were removed from TLS 1.3 basically all of the bad stuff or C 4 is such one md5 renegotiation and then a number of features were added to TLS 1.3 including downgrade protection and perfect forward secrecy we also get a massive speed improvement so TLS 1.2 has two round-trips in the key negotiation process whereas with TLS 103 we only

have one round trip which speeds key negotiation up quite dramatically TLS also has a new feature called zero resumption which allows a client that has already connected to your website to connect again without having to negotiate keys now there is a problem with us and that is that if an attacker manages to capture the session information they can basically create a replay attack against the infrastructure so this feature isn't enabled by default for obvious reasons but if you do use it for get requests that should be quite sufficient so one of the big controversies of TLS 1.3 is the metal box problem so that's where slowed balances and proxies so a client connects to a website they go

through a proxy or any other metal box and then that would respond you know I support TLS 1.2 and then in an additional security header it will say hey do you support TLS 1.3 and then if the client supports TLS 103 will open in new TLS 1.3 connection and because most proxies don't support that it will actually break now because proxies cannot intercept TLS 103 because it has perfect forward secrecy this becomes a very big problem for enterprises that rely on TLS interception devices so the only way you can make this work with a proxy is basically you have to terminate the connection on the proxy and then creates a new connection from the proxy

to whatever saw that you are going to go to but that has drastic performance problems for the proxies that Enterprise is currently have the argument was also made that we should move away from a world where our protections are on the network parameter and we should move those controls on to the endpoint kind of building out a zero trust environment and I also predict in the next 18 months we're actually going to see a massive security incident related to TLS 103 because it is so different from the other versions we also know from research that proxies are actually not good for security in many cases due to miss configurations they actually weaken security because they will accept

self-signed certificates certificates that are expired as well as they can they are often vulnerable to downgrade attacks so that the connection between the clients and the proxy is actually more secure than the connection between the proxy and the actual site that you're trans ago to then moving along to BGP hi checking so bgp hijacking is not something that is new you know we saw the first incident in 1997 but over the past year we've seen over 17,000 incidents relating to problems with bgp the two major ones was when Russian eyes be hacked the ERP space of Visa and MasterCard as well as when somebody in America hijacked or an attacker in America object Amazon's DNS to steal

cryptocurrency now the week after I presented this at hexagon in Joburg a Nigerian I speed through a borough fat fingering basically caused Google's traffic to flow through Russia and China and the fact that this is even possible is actually a major concern for us on the internet the fact that because the Internet is built on trust that any announcements are kind of trusted so what essentially happened while this wasn't malicious main one in Nigeria announced incorrect routes which then forced their traffic to go through China and Russia and to kind of combat this a little bit CloudFlare has created or PKI which basically takes the PKR that we use for certificates authorities and kind of merges it onto bgp announcements

now this is a band-aid solution because this is you know kind of trying to put a patch on a very big problem but it also requires that a lot of people actually adopts it in order for it to be successful now how this would work is when network creates a BGP announcement that has vents and and in the regional internet registries like ethnic ethnic and rap they will actually validate that the announcements is actually correct now as we move to as we focus on South Africa if we look at our own space a lot of cloud providers coming to South Africa and a lot of cess providers coming to South Africa and a lot of them

are creating points of presence inside of terakoya when we actually look at the BGP routes for South Africa we actually see a number of providers there that you know are quite censored and are quite critical and if you actually go look at a lot a lot of the major banks they all use common providers and in terrico they also host a number of smaller ISPs which typically don't have the best security so it is completely possible that an attacker could attack a small ISB create a fraudulent BGP announcement and then basically intercept traffic going to banks or any other of their targets the question then becomes how do we defend against this and unfortunately other

than the RPK I think that I just mentioned which is kind of a band-aid solution it isn't really a proper way to defend against this what we can do however is we can deploy DNS ik which will validate the DNS both that your browser goes through one of the other things that we can do is we can add certificate of certificate authority authorization records which is a DNS entry that you make to say which CAS are allowed to issue certificates for your domain so if if you are a victim of a BGP hijacking someone can't go issue certificates because they're on your public space and basically perform a man-in-the-middle attack so if if your IP space is actually hijacked it is

possible for an attacker to basically remove these DNS records but you were just making it a lot harder for them so as we move to containers cloud and DevOps order flops so a number of banks and enterprises in South Africa have adopted a multi cloud strategy including both private and public cloud in addition to quite a vast array of cess services the question then comes in how do we as defenders actually defend against all of these things and it's actually really really difficult and I think we are seeing a lot of the banks and enterprises really struggle with us because in the conventional world we would put a whole bunch of agents on our bolts but when we start looking at

containers it starts to become quite complicated because do you put the agents on the host or in the container or on both of them and shouldn't containers be lightweight so ideally you shouldn't be putting those agents in the containers however you do have you know people that kind of treat containers as virtual machines and you need to basically protect the rest of your organization and when we have things like kubernetes which is becoming increasingly popular this picture even gets even more muddled because if you have kubernetes do you have one stack for your entire environment or do you section off the business units or per function and we expect the popularity of kubernetes to grow quite dramatically

we've in 2016 we had two clients that were using kubernetes this year we have seven clients that are using kubernetes and we expect that to double mixture so if we look at DevOps and agile we know from research their teams that use waterfall take three to four months to respond to a security in ability versus agile teams that take two to three weeks however enterprises and organizations that are actually using DevOps today are actually less secure than the the organizations that are not using it and the question comes in why is this the case and fundamentally it comes down to foundational problems organizations are not putting in the foundations of their DevOps tool incorrectly and they are

baking in security as afterthoughts a lot of them are practicing rajole which is basically a gel that's basically waterfall so they have stand up meetings with Kanban board and now they're doing agile but they really are not doing agile and then a lot of them a lot of operation teams in organizations they're basically the firefighters for the organization so they constantly have fires burning in their organization that they deal with instead of actually spending time and actually putting in fire prevention you know as a metaphor one of the other problems is a problem recording enterprise architecture so there is often a big debate in organizations as to who should own the DevOps tooling inside our organization

so in closing a lot of organizations they want to use cloud they want to use DevOps and all of the good things that it can bring but a lot of them don't think of the foundations that they need to put in place as well as the journey that they need to take the organization's through and on that note that is the end of my presentation thank you very much

any questions awesome presentation was a lot to take in it will the slides be available you matter and if you mention it tweet them out light is not oh yes

so that is certainly possible in my experience I haven't found that to be the problem in my experience I've found that often when they put those tools in their coffin operationalize those tools that is actually the kind of bigger problem that are found personally no one else you quiet

um so in the past we used to be very you know we used to be very open to ideas I right now because we are basically working with all of the major banks we've taken a very different approach where we are actually very prescriptive because we found that out that a lot of organizations actually don't know what their wants so we basically tell them what they should have and then they can disagree with us and then we'll deal with it but I think a set of best practices as well as real-world experience of implementing that is the correct way to go and you really do need the correct consulting partners as well as architects in your

own business in order to make it successful that's what we have found with working with our various clients

so I think it's a very good idea cloud technology which is we are mostly spend most of my time that makes it very easy to do that I've seen a number of banks in South Africa these three banks that are trying to do zero trust at the moment one of the problems that they are finding is that because of all of their legacy environments it's actually difficult to know what talks to everything and in order to do your trust properly you actually need a number of capabilities which they don't have and fundamentally a lot of the problems actually comes down to you have political or politics in organizations they basically block the roll out of

this because of the fear that it might break stuff and what we've also found and again the operationalization of of Syria trust because you know if you isolate everything through network access control you you know you might break stuff and then you need a team that actually deals with those incidents and basically restores functionality so what's very complicated is especially at banks on the core banking systems they systems may only talk to other systems once every three months or once every six months so even if you have monitoring you might not be a hundred percent certain that you know your data and traffic flows so it's quite a it's a difficult problem yes I can't can't

comment now seriously we we know names

yeah I've seen this forest it's difficult let me answer your question the Savannah understand I'm not actually on through your question so one of the problems with cloud is that core banking applications are not going to be moving to cloud anytime soon right is always going to need be a need for low latency hardware on-premise in addition to the fact that right now banks comput the payments HSM is inside of cloud at the moment so payments HSMs will always be something at least for the foreseeable future will be on-premise so in terms of security I have seen a lot of thinking that is correct but unfortunately we've also seen organizations basically falling for the same problems so one of the new

banks we've seen that they are falling into the same politics bad communication you know aspects that larger enterprises have yeah I know sorry yeah

so so so so my feeling about a post is that I actually think it sends the wrong messages to enterprises I can tell you that there are five bank you know sorry three banks in South Africa that are using Azure stack and all of them hate it because the the architecture when you run stuff on premise is very different to how you would do it on actually on Azure one of the big problems is is when you do when you have automation that joins instances to a domain and depending how you architect as your stack that actually becomes very very very difficult so I guess they will probably do that but I'm not a fan of

that and it also understand that they may be enterprises that need stuff on-premise and maybe that is the whole with the view that they should move to like proper public cloud in the future

I actually don't think they give it any thought one thing I have seen though is that a number of banks are actually building out security tools for their CRC deeper plants and one of the things that those tools do is they will actually scan what libraries you are using to actually tell you whether it has vulnerabilities and whether the library itself is actually trustworthy or not that's that that isn't quantitative it's very like oh yeah it's like very very opinionated um there are some vendors that they like but the isn't like formal I haven't seen formal like - about that bothers yeah okay last question

so they don't so the so a lot of the licensing problems actually get solved in cloud if you use on durmond because Microsoft and Oracle and some of the other vendors they've actually blocked you from taking your on-premise licenses to cloud a Microsoft with sequel servers the most notable noticeable one of that I guess what the other people are doing is they're actually using SAS software like flex zero and putting agents on all of the infrastructure to basically reports all of that

[Applause]