BSIDESKyiv 2018 - HELEN TABUNSHCHYK GUARDING THE INTERNET FROM BOTNETS

I want to go through the theory first, so that no one has empty seats, so that we all speak the same language. I apologize for someone for this repetition, but just in case. One of my favorite photos is the phone exchange in Manireale, 1895. DDoS attack was easy enough to do, in order to shut down all this, but it was very difficult to remain anonymous. In 1973, ARPANET map about Internet parents could be placed on just A4 format sheet. It's all about peering, all hops, how they connect, USA, everything is very small. There is such a wonderful project called OptE. They make a map of Internet peering. They take dumps of BGP protocol, and draw such things. This is the picture generated in

2015. Can you imagine how much has been added in three years? I'm sorry. 7 levels of OSI model. Today we will talk about the third level - network IP protocol, transport, TCP, UDP, TLS and application level. Additional data is added to the Internet package on each protocol. it becomes more and more and then it is deployed back to the place of receiving. Two types of DNS servers: recursive server and authoritative server. Authoritative knows the table, roughly speaking, the table, domain name and IP address, by which you can find this domain name. Recursive, it is a resolver, it asks these authoritative servers until it finds out who owns this information, who knows the domain name and IP address. Two types

of routing. Unicast and Anycast. Unicast is a single-signal routing from the client to the server. Anycast is when we choose one of the available servers. So, what is DOS Attack? DOS attack is a channel, both the Internet channel and the server resources, that is not valid for traffic, which does not allow valid users to get the resources they need and get what they came for. Why is it done? Obvious reasons. Ideology. Both personal hatred and some companies or some anti-state groups. Ransom is for buying, to attract attention and non-profit competitors are engaged in it. In fact, DosAtaco can be done simply. On the black market, you can put about 150 dollars in a small company. Recently, a survey was conducted. 70% of the companies

that are investigated are subjected to DOS attacks 30-50 times a month. And these DOS attacks cost an average of $50,000. So, what do you think is the first thing you need to do a DOS attack? No, it's not a cat, it's a botnet. Traffic, well, yes, traffic generated by a botnet. You need resources to generate all this. There are four main topologies of these botnets. CNC is Control and Command Center. Where all the instructions are issued, how the infection occurs, all the control. One point from which it all comes, and a bunch of bots. High availability of infection, when several servers. hierarchical structure, when they can pass commands to each other, and peer-to-peer, the most harmful and hard to

eradicate. Just a few days ago I found a wonderful study from Fastly, which is also a provider of DDoS mitigation services. They conducted an IT study. On average, an IT device is infected and tries to launch a DOS attack for 6 minutes as soon as it connects to the Internet. In fact, I'm scared of it. They are checked 800 times per hour to see if they are broken or not. 400 login attempts on one device. 66% of them are successful. What does it mean? That no one changes credentials by default. A good example. In 2016-2017 there was an attack of Mirai's bootnet. From January 2016 to February 2017. Mirai was the biggest DDoS attack at the time.

1 TB/s. 600,000 IT devices DDoSed resources. They used HTTP, UDP, TCP flooding. How did the infection happen? Scan, report to the server, the server responds and infects the device. What conclusions can be made from this? Default credentials are a huge evil, they always need to be changed instantly. Autopatching must be done immediately, because it is all discovered quite quickly, patched quite quickly, it must be applied. You need to implement rate limiting, limit the number of requests to your server, to your current traffic. Ingress and egress filtering, that is, to look at the requests, we will talk about it today. And only Trusted resources should be executed on your devices. This is not the worst yet.

Two months ago, Botnet was discovered on MikroTik routers. How was it used? Passport brute forcing, USB-based SMB overflow, And HTTP post requests, there was an incorrect validation, which led to the possibility of executing the executable code. Link to this study. Someone smiled at the panda. What types of attacks do they exist? The most primitive thing is the attack of the protocol level. Everyone has heard of SYNFLUT. TCP protocol works like this: client sends SYN request, server receives it, if it is able to process, it answers SYN_ACK and this request, which is the initial SYN, adds its SYN_ACK queue and waits for the client's response. And the client must answer SYN_ACK, that "yes, I agree, let's install the connection". This is

how it works. Client sends SYN packet, Server receives it, writes down its queue, says "Give me a SYNAC message", and the client never answers. Thus, the queue of the server is filled with these SYNAC messages and there is no place for valid traffic. This problem is partially solved by SYNCOOKs. It's a Linux mechanism for all of this. There is no more SYN/ACK queue, the exchange takes place with the help of cookies. But there is a very cruel drawback, the size of the TCP package is very limited, therefore, SYN/COOKIES are only enabled during DDoS attack, that is, by default, the traffic should be fine through three-stage communication. Quick. Have you heard about Quick? Who has heard

about Quick? I heard about QUIC QUIC is a new protocol that is in the standardization stage It is designed to replace TCP, partially HTTP, because it will work on UDP with TLS encryption Everyone knows that TCP is not an efficient protocol and it has its own shortcomings and this issue needs to be solved I'm interested in the attack levels of the protocol on QUIC It's gonna be fun. Volumetric attack, or amplification attack, or... I forgot the third one. How does it work? Bot sends a message, most often DNS resolver is subjected to it. It sends a message to DNS resolver, substitutes source IP address in the package, puts not its source IP address, but victim of the site,

the victim. And DNS resolver doesn't know that this request is invalid, and it answers to this victim with an answer. The nuance of the DNS protocol is that the request is much smaller than the answer. Thus, we achieve amplification, that is, increasing the number of sent bytes to the received bytes. Two months ago there was an attack, which our company Cloudflare called memcrash, and it somehow survived. But the point is that they have a protocol with UDP on their servers. And few people close it. And in the end of February, there were amplification attacks on memcached. What was the nuance there? The same thing: the IP address is replaced by the victim's address. You send

one byte and get 50 000. Thank you. There is a website Shodan.io, if you don't know it, I highly recommend you to visit it. It monitors the state of the Internet, the state of devices at the moment. And then, at the end of February, there were about 90,000 open memcached servers that could do this. GitHub was then placed for a few minutes, DDoS attack at 1.35 terabits per second. It is still the largest DDoS attack. Again, it's UDP's fault. There is no control over IP spoofing. IP spoofing control should be done at the level of providers, at the level of Internet Exchange. For example, a provider in Kyiv, some local network, they should check the package, and they should see that our

IP range, the addresses are available only here. Where did some other left IP address come from? This package should be discarded. But very few Internet exchanges do this verification. That's why we have all these amplification attacks. This is the most popular attack. Amplification attack. It is about 66% of all attacks. Attack of the application level. DNS protocol is used, HTTP protocol is used often. When a request is sent to the server, attackers look at what your server thinks about. For example, you have a request from a database with a bunch of join tables. Attackers are interested in constantly pulling this request so that your server lies under this load. and generalization of all attack types. I really like the name of the attack protocol level

- ping of death. Unfortunately, it was already patched. This was such a vulnerability in the implementation of the CMP protocol, when if you send a deformed ping package, then there is a stack overflow and the system is stalled. The easiest way to make an attack of the protocol level is to reflect it. It is also not difficult to make a volumetric attack. It is already more difficult to reflect it due to the fact that it is difficult to understand the IP spoofing of the final origin servers, because they do not know which IP addresses are available. The attack of the application level is the most difficult to do, but it is also the most difficult to track. And I want to tell you

how we fight attacks on specific examples. A few words about who we are. We are a reverse proxy service. We are between a website or IP or some other service and clients. All requests go through us. The original IP address, the origin of the website, is available only to us. No one else knows about it. It is closed for everyone. If it opens, then the server is exposed. Sorry, it's not CSS, but it still crawled. We serve more web traffic than Twitter, Amazon, Apple, Instagram, Bing, Wikipedia. 200 million users see if we have broken something. Every day 15 thousand new customers subscribe to us. And even if you didn't hear about us, you still used us, because an ordinary Internet user touches us

500 times a week. We have 151 data centers in 74 countries, more than 7 million clients, we process 10% of all internet requests, on average we have 7.4 million requests per second, 10 million in peak, 1.2 million DNS requests. We don't have statistics for DNS resolver yet, because we just launched our DNS resolver, 1.1.1.1 ad, subscribe. Almost 3 billion people are being served by us. The biggest DDoS attack we've seen two weeks ago was SynFood. The attack is 942 Gbps. But it's still hard to put our network on the table, because our network is 15 terabits. So far, the numbers are incalculable. And Trusted By. We have a link to the case studies, if you

are interested in what we do, how we solve problems of other people, I recommend you to read it. So, how do we have mitigation? I really like this chart. This chart is Synflood Attacks. Every day we receive some amount of attack from Synfloat. And in this moment, in September 2017, we announced that all plans, even our free users receive free protection from DDoS attacks. And it's not interesting for attackers to attack us. The frequency has dropped a lot. Only sometimes someone is warming up and that's it. I think this is a great way to mitigate. We use Anycast and ECMP routing within our data centers. There are 120 data centers on Globus, 30 less. We haven't drawn a new one

yet. It is clear that our globe is covered and it is quite difficult to put it. We have smart routing. Not always the fastest route, the closest route. So we collect data, intelligence on the current network state and send requests on the least congested route. We reduce latency by 35% and errors by 27%. I think it's a very good result. What tools do we use? What kind of DDoS companies use tools to do this? Level 3, level 4, they usually write together. First of all, it's Linux firewall, at the moment it's IP tables. But IP tables is a slow thing, a little bit inconvenient. Therefore, the next one is the Berkeley Packet Filter. This is, roughly speaking, IP tables rules,

but in bytecode. They are executed faster and more efficiently. There is such a thing as Kernel Bypass. Linux stack network is quite slow and unrotative in terms of processing large mass of traffic. Therefore, if the company works with loaded servers, it must be implemented either Kernel Bypass or XDP. Kernel Bypass is processing TCP packages not at the core level, Directly from the network card to the user space. You know the package type, you can optimize it somehow. And it works faster. And the last thing that appeared is XDP. It is gaining a lot of popularity now. Express Data Path. This is like a kernel bypass, only without a kernel bypass. We do not use Linux network stack. But we continue

to work in the core. In the same way, packets directly from the network card go to XDP and are processed. There, too, this whole thing is compiled into bytecode and works through bytecode. And I found a study that XDP is more than twice as fast as the usual IP tables of the rule. How to mitigate level 7 using Linux tools? Again, IP tables. We haven't left this place yet. And modules for IP tables such as contract, conlimit, hashlimit, ipset. You can find other modules, but these are the main ones. How do we do it specifically? We have a development called GetBot, which looks at our traffic, samples the periodic traffic, and looks at the patterns, at the anomalies

in this traffic, and it blocks these anomalies. It blocks with the help of IP tables at level 4. It also sends notifications to our engineers. In principle, it notifies everyone that something is happening. We have a Flood Gate. This is our kernel bypass. It is also a little custom. It is enabled during DDoS attack. We don't use it by default. Scattering is when we change IP addresses of servers. We see that for free users we have a lot of clients sitting on one IP address. During the attack, we want to move these clients from this IP address, because usually some IP address is attacked. Thus, we distribute the load a little. We have rate limiting. Rate limiting can be implemented in different ways. The most

straightforward way is to use IP addresses. We draw Gilbert curves and see what trends we have in source IP addresses. And we start from here. Now, if someone knows the IP address of your website, we will not be able to return it, because we will be able to attack directly. Of course, we have a product that solves this problem. Origin website is completely closed, all ports are closed. We have encrypted TLS connection, only we can communicate. On this TLS connection, of course, you can weigh all sorts of load balancing and health checks. And, I think, a week ago we announced that we also protect the remaining 65,533 ports. In principle, somewhere along the same scheme. Now the most interesting thing is level 7. First of

all, I think that our main advantage is 7 million clients. If one client attacks, the rest of the 7 million can use the conclusions we made from this attack. Again, a straightforward way to mitigate L7 attacks is to turn off HTTP Keep Alive. This, of course, slows down the work for valid clients. but it's a very effective way to fight DDoS attacks. We use Honeypots. What is Honeypot? If you have a website, you host a link somewhere in an invisible place, so a valid user will never find this link. But if a request comes from a crawler who does not understand where this link is, he just parses the page, sees the link and goes through it. We collect data about

who opened this link. We collect fingerprinting, look at some parameters, for example, a user agent came to us, or used a cipher suite, or SSL hello looked like this. And then we use it for mitigation. We also have a very cool web application firewall, 12% of which is WASP rules, 2% is user rules, 86% is our work. WordPress Drupal vulnerability, everything is closed, it is no longer interesting to attack, you can no longer capture this server. Also, for attack level 7, JavaScript challenges are very often used. The simplest way is to ask the browser to calculate something, to perform something. The more difficult way is to collect some browser characteristics. For example, there is some extension. or use some encryption with this

browser. And so, according to these characteristics, you can look at valid and invalid browsers. Again, the obvious way is Capcha, but no one likes Capcha, because Capcha starts to be shown to valid customers, they need to click and choose these pictures. It's not very pleasant. After all, it's easier to do JavaScript Challenge without your participation, so that the browser can figure it out by itself. Fingerprinting, as I said, some patterns in these requests. Fingerprinting is not only done in application packages, but also in network packages, because network packages can also talk about some trends. Of course, we use our gatebot, it monitors all of this and automatically mitigates it. And again, rate limiting. We have an IP reputation and we have a list of

bad IPs, relatively speaking, this is the most straightforward solution. And at a certain level of threat, we prohibit access from certain IPs. But again, IP can be spoofed, It's obvious and simple solution. A little bit more interesting things about Cloudflare. We have a project called Galileo. If you have a website that deals with freedom of speech, some kind of human rights movements, something like that, and someone tries to block you, then please write to us and enterprise level protection will be provided to you absolutely free of charge. Especially in North Korea, Pakistan and so on.

In Russia, DigitalOcean was banned. We have a question about it. Yes, it is our partner. There is an interesting situation with Russia. As I said, we have free clients. Many of them are hosted on the same IP address. Russian Comprehensive Supervisory Service bans one client, the entire IP address is blocked, the rest of the clients suffer. Russian Comprehensive Supervisory Service, to put it mildly, is disliked for this. We also have Athenian project, which is the same as Galileo project, but for elections. If you host elections, we can protect you completely. And a minute of advertising. We are having fun. Come, catch me, write. I will be glad to at least try to answer your questions. And

I finished.

Are you working with other solutions like Spam House? I mean, do you take it from them and build your own rules based on their solutions, for example, to block, for example, spam that currently works, a spam campaign, for example? I think it's already working. I can hear you. Not all the information is available to me, because the company is quite large, but we work on our data. Once again, we have 7 million customers, we have a lot of this data, and we are interested in processing it. First of all, it is very often unknown to whom DDoS attack is produced. DDoS attack is produced on our customer or on us. To be honest, I don't know about spam, but I can tell you

that we use the project RIP Atlas for the current marketing and peering. I can't say for sure about spam, but I think that it's not. Tell me, please, what are the ways to protect REST API from DDoS attacks? There is no kook, no html, nothing like that. Thank you. The protection will be mainly fingerprinting. No company will tell you what the principle of fingerprinting is. They look at the patterns. Such a request comes from a valid client. For example, such a client with such signs usually decides the captcha. If the captcha is decided, then we know that this person is the most important. For the API, again, the signs are collected.

Yes, Argo is... I don't remember whether it's enterprise or not, because I don't care about plans. But it's definitely a paid solution for a separate money. It's not for free clients. We use... Yes. If you are a free client, you use Synflood, which is very easy to define, then it is clear that you need to close all ports, etc. There is also such a thing as fingerprinting packages in terms of uniqueness, so that there is no reflection on the tag, so that there is no repetition of the package. There is such a test. Some unique ID is assigned, this ID is recorded and it is compared. There was also a continuation of the question. Yes, about API. Is there any

possibility to collect behavior Standard client? Yes, we collect the behavior of the clients. What kind of requests does he make, in what order? Yes, in what order he does it, what encryption he uses. How much test data do you need for this, so that we do not sift the right clients? A lot. But usually what happens in our lifetime, we take sampling, usually 1%. And we look at the trend by 1%. Because even 1% is enough to set the trend. And what is highlighted will be malicious attack. Okay, so how much do we need to train CloudFlower to raise the server normally and be sure that DDoS will be rejected? Specifically, the client does not

need to do anything. This is a problem for the company. And the company, as I said, we share. intelligence among all clients, that is, one is attacked, we transfer it all. Specifically, we usually have a reaction within minutes. We have a gatebot, this automated tool that looks at traffic, looks at trends and automatically mitigates it within minutes. How many requests do you need? How many requests? Well, yes, in some dimensions. How much do you need to train Cloudflare and how to train it so that it recognizes attacks correctly? Once again, you don't train, that is, machine learning is not available to the client, that is, this is what we do inside, we look at our trend of requests. Once again, we have

7 million requests per second, if we take sampling 1%, here is 1% of 7 million, we can already show a trend. I may be misunderstanding the question. So we just need to turn on the site, hide it for CloudFlare and learn for some time. I'm just talking about the numbers, how long do I need to wait for it to learn. No, no, no, you don't teach it. You don't teach it. I don't teach it. I need to know how much time I need to spend to learn CloudFlare and recognize it correctly. We can recognize it correctly. And we catch new trends within minutes, because these are some anomalies from our traffic. I see. Thank you for the paraphrasing. Within a minute. First of all, we will

immediately turn on the rate limiting and look at our internal data. We know what requests are valid and invalid, and we will immediately start applying them. Within a minute, it will be transferred to all our data centers. Because it takes some time to transfer all these rules and synchronize data centers. Why do you have your own group? If they follow you, it will be the same.

No, he doesn't need to transfer the infrastructure, he just needs to close access to himself and that's it, and only we can open it. Yes. Can you please put the microphone on the microphone so that those people who are listening to the broadcast can hear it too? Yes. Let me repeat the question. How much time do you need to... if there is an API or a site under DOS Attack, how much time do you need for all this to be applied? Once again, we close all access to our server, we only open access to Cloudflare. That's it. Thank you. I wanted to add something. The person said that when a site is translated to you,

the DNS is switched. Yes. The IP address. The point is that even if the IP table is closed, and the connection is allowed only by your IP address, if the attack is very strong, the channel can simply be blocked. It's like a firewall. It will be blocked, it will not last. And this is a serious case and a good best practice, because there is a service called DomainTools.com, they are the ones who are running the history of all DNS records and Whois records in all domains. So, as often as professional DDoSers can do, they don't look at the site if it's closed by CloudFlare, they go to DomainHistory, look at the previous IPs, see the previous IP, the direct

IP, and start DDoSing it. As a recommendation to everyone, if you come across some external firewall, like CloudFlare, AWS, they also have firewalls, it doesn't matter. It is advisable to change the IP address, because it is already in history, it is already saved on the Internet, and only after that, you can immediately switch it to this. Yes, you are absolutely right, it's all right. As far as I know, a lot of clients subscribe to us during DDoS attacks. And when they are DDoSed, they call us and say: "Protect us". Of course, for some time, it will all be turned on, for some time, it will all change. Usually, minutes, hours, but it's individual. Yes, it takes some time, it's obvious. Can you tell us a little bit

about SSH and SFTP protection? You mentioned that you are now taking protection. A little bit more in detail. Is it commercial or not? And how is the implementation for the client? I work with a server with equipment. I set up a session. I set up a session, I knock on it directly, on the IP. How will it knock here? Will we knock on some kind of… Yes, your server is supported by an extension that supports the connection with our data centers, our daemon. And it proxies all requests, it also processes them additionally and proxies them to your service. I understand. I mean, the request from me should look like this. Is it like your IP

and some extension that identifies me as a client? Yes. I'm interested in the development of the software for this. I understand your question. I won't tell you for sure, I haven't been working on this product. But logic suggests that there should be a unique identifier, a practicing demon. The question is, will your requests go through the practicing demon as answers? I will not say. I think it should be configured logically. Another question?

I have a question about Argo Tunnel. I blocked all IP addresses except CloudFlare. How will it differ from Argo Tunnel? So, do I have the right to do so if Argo Tunnel is a paid service? No, you absolutely have the right to do so. It's a logical decision to block everything and leave one port for certain IP addresses. We are one of the first to implement TLS 1.3. We have additional pluses, like an additional health check. This is additional functionality that can be useful to you. Again, since the Argo Tunnel is a paid thing, it is discussed individually for each specific use case. We have special engineers who deal with client problems. You come and say that you have such and such situation, such and such features.

And even if we don't know how to do it, you just describe your problem to us and we implement it. Because you pay us for it. I answered? Yes. And another question. About the sick. If Roskomnadzor blocks another 15 million addresses, will Cloudflare find a way to route traffic? They are constantly blocking us. Yes, they block us every day. Our support and SRDs call someone Russian-speaking to confirm what the Russian Ministry of Internal Affairs wrote. Again, it takes some minutes to an hour to transfer everything, to do scattering, transfer new addresses, transfer routing, update our BGP tables, etc. Yes, they do it all the time, and we are constantly working on it. Hello, I have a question.

Is your load shared on the server or distributed? Let's say you have 100,000 clients, and they all start with some power. Do your servers lag because of it? If everyone starts attacking at the same time? No. Distribution is inside. Yes, we distribute and constantly collect intelligence. We collect data about the state of servers, routes, not about the state of data centers, but about the state of specific servers, that is, what is happening in our country. And we redirect these requests. For example, if you are from Kiev, we have a data center in Kiev, and if you make a request, it absolutely does not mean that your request will go through the Kiev data center. The Kiev data center can be occupied and you can be sent there via Frankfurt.

But in silence, for free clients, once again, routing is not so smart. I showed there was an argot routing, again, this is a paid feature, because we pay for services that collect data on the state of peeing at the moment. It is more effective, but it is paid. If clients around the world, for example, If the US, Europe and Ukraine are attacked simultaneously, and distribution will be everywhere, then the load percentage will be low. Once again, the biggest attack on today was the attack of memcached - 1.35 terabits. We have 15 terabits. Even if we don't mitigate, no one has reached 15 terabits yet. We already have a very good knowledge base, we know the type of attack, we know the principles.

Usually attacks are quite primitive. For example, Synflood is still a very popular attack, although it is very easy to do and very easy to recognize, but it is still done. It is done, we mitigate it, and that's it. Yes, about the Argo of your tunnel. So you say that your infrastructure is distributed to different devices in case of a very heavy attack. Then you, as a client, at the level of my devices, will need to allow access to all your infrastructure, which you have. It will not change dynamically. We do not give you an IP address directly, we give you a configuration. In the case of Argo, you also have a demon that collects all this. It's not you, but

we put a demon that monitors all this.

- A question about the bandwidth, you said 15 terabits, is it free load for DDoS or the bandwidth? If it's the bandwidth, what's the percentage of the load that's taken by the legitimate white traffic? This is the common capacity of all our data centers. We work every day to install new peering with new Internet exchanges. We sign something every few days. The percentage of white traffic is a very difficult question, because different sites have different patterns. I understand, but it sounds like this: we have 15 terabits, so we are not afraid of 1 terabit DDoS, but I don't exclude that almost 50% of the traffic is occupied by white traffic. So, is it enough to have 7 terabits DDoS to put the whole network? As far

as I remember, our servers are not loaded more than 20%. We expect that we have a very large overhead. And in fact, the advantage of this is that we do not use it 100%, it is stupid to use it 100%, it also depends on the time of day. Usually in America, when it is active, we have a night here, and therefore we can redirect traffic to the Kiev data center, which is now generally built.

As for machine learning, you have some kind of trained artificial intelligence. For example, I deploy a site, hide it behind Cloudflare. And what happens next? Is this model pre-trained for a specific site? Is it improved? Or how does it happen? It's individual. Again, the question is how much money you pay for it. If you pay us well, we will do everything for you. And quite quickly. If you need to implement something, it is implemented instantly, the release is carried out outside the schedule. But this is quite rare. Our knowledge base is enough. I say, attacks are not really that smart. Before that, there was Mirai, a botnet that was quite interesting, and it was caught

for a year, and only a year later they found out who did it, and only a year later the consequences were not visible. And even memcached, which is the biggest DDoS attack, it disappeared in a month. Because everyone has already learned and closed all these memcached servers, already connected the UDP protocol, everyone started to review UDP policies, not only memcached servers, but also services. That's all? Any more questions? I have a question. Do you have your own Kyiv data center or do you rent a place in some commercial estates? That's a very good question. We rent a place in a data center, in a stable. And the network channel. Of course, we can't extend it, but all the hardware is always ours. If we open a new data center,

we always send the hardware to a specific data center. We have contractors who do this, of course, our engineers don't go to the place every time. For example, we had a story in Brazil. In Brazil, they asked for a lot of bribes to host our Metals, our servers directly in their data center. And we had to go through several places before we could place our servers in Brazil. Did you collect money?

You know, I don't know the exact answer to this question, but knowing Cloudflare, it seems to me that they were not collecting money, but looking for an honest option. Because we recently defeated patent trolls. Patent trolls attacked us, they had 50 thousand patents collected, including Cloudflare. And Cloudflare, instead of paying this ransom, which would be 100 thousand dollars, which is not quite for the company, Cloudflare decided to fight and went to court. They were in court for two years and won recently. They, of course, got into an appeal. One more clarification. You rent a facility and rent channels. Does the data center itself fully comply with the standards that you have in your company? What is the question? Does the data center you

rent in Kyiv meet your needs? Yes, it does. It meets our needs. Well enough. Thank you. Was it a question like "Is Kyiv a good data center or do we have low standards?" I think it's good. I can't ask you directly which data center you are located in, so I'm just asking. Yes, it's a secret, I can't tell you which data center. But again, the logic tells me that Cloudflare is very interested in building data centers in Kyiv, because we have an interesting neighbor here, who constantly blocks everything. And the traffic from there is still there, they have not yet been banned on the Internet. Do you have a staff in Kyiv that supports you? No, we don't have a staff in Kyiv.

We hire contractors for specific jobs. If we need to open a data center, we hire contractors for half a year, and then we just reduce their number and leave it for support. And for the route, we have network engineers, 3 offices with technical staff, on-call, SRE, network engineers, support, etc. in London and Singapore. All 24 hours are covered with normal shifts from 9 to 5. Great, thank you. Any more questions? Thank you again.