2038 Isn't A Future Issue — Get That Through Your Thick Skull

again. It's good to be home. Uh hired. Okay. Hello. Uh welcome to the afternoon. I hope we don't bore you today. Um quick note, the first time I presented at Besides Lisbon was some years ago and I was presenting with my good friend David so uh it was the last time I ever presented in a duet on stage. So, it's a great pleasure to be here with with Trey presenting something again in a duet fashion. Trey has been thinking about the topic we are going to present for a long time. That should be here, but it's still not. >> Perfect. >> Hang on. Sorry. >> Uh, so it's been very interesting to work on this forest over here. has trees.

>> When you do that again, >> I don't see wildlife. Maybe because >> No chipmunks. >> No chipmunks. >> Perfect. Okay, there we go. >> There you go. Um, so has been thinking this uh uh about this problem for for a lot of years. Uh I I just started like one year ago and I I'll get to that. But it has been a very interesting research partnership. uh he has a very special mind and way to address problems. Definitely doesn't fit into Jen's law of triviality when he's thinking about stuff. He's thinking about everything everywhere at the same time. So um let's begin. >> Thanks Pedro and good to be here in Lisbon. It's my first time here and I

can't believe it's my first. It won't be my last. So a Rube Goldberg machine is a pretty good model for our modern infrastructure. Lots of moving parts, but not all designed for today's load. If we bought this particular Rube Goldberg machine at IKEA, somebody definitely threw out the assembly instructions. >> Yeah, they probably use the leftover screws to build some LM or some badge, I guess. >> Oh, yes. So, our talk today is about time integrity, the glue holding this whole distributed system we call the internet together. >> We're not talking about NTP patch levels here. We're talking about trust in time itself. If the clock lies, everything that's built on top of time starts to

wobble. That's right. So, before we dive in, a bit of quick history, so we're all aligned. Time on the net was first improvised, then standardized. NTP was syncing machines back in the early 80s, perhaps before ARP had even moved to TCPIP. It used a 32-bit unsigned counter starting in 1900. This yielded 136 years rolling over in 2036. In the 1980s, that felt infinite. No one imagined the same protocol still driving the global internet four decades later. NTP was the life's work of David Mills. He passed in 2024, leaving us that gift. But even the most elegant protocols have finite horizons. So in 7 February 2036, NTP's counter rolls over and many dependent systems will suddenly think it's 1900. Think of

your TLS certificate change, your logs, your forensics, instantly trying to parse the Victorian era. This happens 2 years before the 2038 bug, which you probably came here to hear about, which is in fact the signed integer version of this story. See, computers mostly rack in time by counting seconds since 1 January 1970 with a 32-bit signed time truct that yields about 2.1 billion seconds or about 68 years. At 3407 UTC on 19th January 2038, this counter maxes out. The sign bit flips and time jumps back to December 1901, which is interestingly the Eduwardian era, not the Victorian era. So anything touching C inherits this timet strct operating systems databases protocols firmware. It's the sort of damocles hanging over

billions of devices. The Linux kernels patched maybe, but userland and tool chains are not, nor is our installed base. Now this looks like CWE 190, but it's bigger. See, you're not wrong. It is a classic integer overflow, but you're also not right. When the shared clock overflows across billions of systems all at once, it stops being just a bug. Now it's a vulnerability class. So let's name that class clearly. Y2K affected maybe 100 million systems, 50 to 100. Today it's on the order of tens of billions. Routers, industrial controllers, telecoms, medical power, all ticking toward the same rollover. And because authentication and coordination rely on time, a single epoch lip could lock millions of CIS

admins out or desync entire networks. Now, I tried setting a 2038 reminder in my Proton calendar because I was going to say that you should all set a reminder in your calendar. And I said, "Let me not be a hypocrite. I'll go do it myself." But it turns out that Proton calendar, which launched in 2022, only supports events between 1970 and 2037. If even the best, most security-minded teams on planet Earth are brushing up against these hard-coded limits today, then legacy stuff is doomed. There's almost no coordination or awareness today. You see, bugs you patch, but vulnerability classes require coordination. Where did this come from? Unix time started as a pragmatic hack back in the

1970s. Just count seconds on a PDP11, a 32bit sign long made sense. Marshall Kirk Mchusseek, who was sharing an office with Bill Joy and Berkeley back in the 80s, told how at the time the compilers didn't even have options for 64-bit math yet. So originally this was a 16-bit number counting minutes. This proved unworkable and they killed that idea fast. Around Bell Labs version four, maybe five, they moved to a 32-bit system, counting seconds. This gave us 68 years. Back then, 68 years feel felt like forever. As I approach my 48th birthday, I can assure you time moves fast. Nobody expected a research project to become planetary infrastructure. That disco era hack got baked into Pix. Pragmatism

oified. The design pattern proliferated. And here we are today. But 2038 is just a pivot in a century of rollover risks. Here's a mental map for you. See, these aren't random bugs. They're a pattern baked into binary arithmetic. They cluster on the bitwidth boundaries. 8 16 32 64 128 whenever a counter runs out of room. Once you see that, you can't unsee it. So 36 and 38 are just the opening shocks in a century long cascade. If we don't fix the pattern now, we'll surely meet it again in ' 06. And in whatever protocol hides the next ticking bomb, now try selling that complexity to your board. Y2K had a 10-second elevator pitch. 20,000 looks like 1900. Everybody

got it. 2038 needs a crash course in signed integer math and ebook rollovers. 5 minutes in and your board's already tuned out on their phone. It's not about competence. It's about cognitive load. See, 2038 is not an elevator pitch. Now, to the question of prior art. I first spoke publicly about this space back in 2016, the opening keynote at O'Reilly security the morning that the world all woke up to the first Trump presidency. Since then, I spent almost a decade chasing this particular question quietly because it seemed tenfoil hat. But I was sure that there must be some working group somewhere in the world. And so I joined the Belgian National Cyber Security Agency, the Center for

Cyber Security Belgium in their early days. My goal was to use that network and access to try to figure out who the heck was working on this. Then I left there. I went to Accenture. I ran for the first board of directors where I got to travel all around the world meeting people from Tokyo to Montreal and spending weeks at a time hosted by national cyber security agencies around the globe. And what I found was that there was no working group tracking this. I could not find it for all the effort of 10 years. What I found was silence and polite confusion and nervous laughter. So to everyone hearing me out on this topic of 2038, I thank you for

listening. But to Pedro, thank you for hearing me. So when you spot a systemic failure, you might well ask, "Sorry, I went too fast there." Is there a historical precedent sometime when we hardcoded 32 bits into a thing and said, "Whoops, that's too small." Yes, IPv4 to IPv6. So let's look at this. Let's hold up a mirror. One protocol, one 32-bit field, which we outgrew. The first RFC to standardize IPv6 was back in 95. Over 30 years later, countless industry working groups, vendor coordination efforts, national mandates, billions spent, and where are we now? Hovering around 50% adoption on the scannable side of the firewall. Most enterprises still depend on IPv4 because of deep technical debt.

IPv4 exhaustion was gradual. 2038 is a cliff. There's no market push to fix it. But I suggest to you if one 32-bit address field took us three decades to halfway replace. 32-bit time may not be easier. Meanwhile, standardization exploded, we've gone from about 2 and a half thousand RFC's circa 1999 to nearly 12,000 project by 2038. These define everything in our stack. Networking, timing, data exchange. Each new RFC links to dozens of others. Change one and the ripple spreads globally. The spec stack grows dense and with AI tools drafting fresh text, the churn will only accelerate. Five-fold growth means compounding complexity. And complexity breeds fragility. Interlocking standards like a root ball, binding the foundations together, but impossible to

untangle. And it's not just the IETF. See, they get the spotlight, but they're just one stall in a global marketplace. There's ISO, ITU, 3GPP, ITLE E, GSMA, and countless closed standards bodies. We don't even see change one field in one spec, and this ripple spread through dozens of others. Coordination across technical committees normally takes years, but we have multiple rollovers converging on us in barely a decade. By 2038, we'll be juggling tens of thousands of standards, each a part of our global Rub Goldberg machine, and each a potential fault line. Scale has also exploded. Back in 99, it's hard to ballpark it because this was the early days of the internet, and so measurement hadn't matured. So, we'll say 50 to 100

million devices. Today, we're past 30 billions, at least a 5 to 600x jump purely numerically since Y2K. The defining factor in the 2038 equation is the multiple orders of magnitude growth of our installed base. This has exploded. This superlinear growth is the fastest technological expansion in human history. Unlike 2038, far 2038, unlike Y2K, we don't even have a denominator. We're scaling blind. And so we measure what we can see. People assume NTP is fixed. 2036 fixes were rolled out in NTP4 back around 2010. So we scanned about 200,000 internetf facing NTP servers and we found roughly one in three failed the 2036 test. This yields about 62,000 vulnerable NTP servers. That's just the visible tip of the

iceberg. The line of visibility is refracted by the firewall. But what we do see proves the iceberg is real. Now some nations treat time like critical infrastructure. Take Sweden for example. They treat it like a national utility. They have a system that's airgapped with atomicbacked redundant stratum one servers NTS security a new protocol that they funded and developed GPS independence and a sustainable public funding model that involves tagging the revenue of all the ISPs for I think it's something like a fraction of 1% to pay for this. If Sweden can do it, why can't we? They invest. They own their clocks. So, here's a pivotal question. If expanding 132-bit field took decades and billions, what happens when tens of

thousands of protocols must be revisited within a single decade? We're at tminus 49 quarters.

Yeah, as I gaze through your faces, uh, some of you might be thinking there's a lot of FUD in this stuff, right? Uh, I I I clearly remember one year ago I was heck heckl, I was watching Trey trying to do the elevator pitch for Y2K38 uh, in a lightning talk. So that that was hard. Well done. Uh, but I was thinking about this. Is this just, you know, fear, uncertainty, and doubt? And at the time, you might remember, I was trying to blow up some gas stations. So, I I I was testing some stuff. And I went and test those devices, and the devices I tested, they they failed on the rollover and I thought, "Okay, that

that's funny." Then I looked around and I tested my watch, my wristwatch, and it failed, too. And I thought it was a coincidence. Then I go and I run around the house testing all my TVs. and half of my TVs also uh had issues and then I got concerned then I called Trey and say hey buddy we you want to start like working on this because I I'm concerned so I state that I don't think it's FUD please entertain my formula here I state that uh FUD is actually uncertainty plus doubt right but in this very special case doubt is zero we know this is going to happen it has a fixed date it's 2038

so there's zero doubt and if there's zero doubt But all it remains is uncertainty, right? Uncertainty about what? About what will fail? How hard will those things fail? Where are all those things and devices and software pieces that might fail? We need to solve for uncertainty so we can gauge fear. Um what will fail is anything that uses Unix timet 32bit anywhere in its stack. So I'm not talking about old or only about old 32-bit kernels here. Uh, I'm also talking about newer 64-bit devices that ship 32-bit user land lipy. I'm talking about drivers. I'm talking about SDKs. I'm talking about database fuels, file format fuels, uh, timestamps in protocols that use a 32-bit timestamp. This can be

everywhere. So, no, I don't think this is a hype for epoch rollovers, FUD. It's just simplified. We have no doubt it will happen. We have a date. We only have uncertainty. So, how can we solve for uncertainty? I call it the discovery challenge. Um, so we talk about we need to figure out what's vulnerable, where it lives, and how to fix it. That's easier said than done. Uh, what's vulnerable, it's kind of 32-bit stuff mostly, and what's not. Uh, it's 64-bit stuff sometimes. So, this there's a lot of uncertainty here, right? So I started to think about okay what approaches and methods can we use to search for vulnerable devices. I I work for uh

bitsite. It's a company that we scan the internet you know every day. So it was kind of natural for me thinking about okay let's use our scan data. We have scans for stuff. So if we think about scanning and fingerprinting maybe we can see something there. We can also you know think about client side stuff. There are stuff connected to the internet that's not easily reachable but they are still using the internet. So there's endpoint metrics, maybe statistics, maybe sinkhole data and you know guess who has one of the largest sinkhole operations in the world too this guy. So I was also you know uh looking at our sinkhole data and trying to figure out more or where could we

find this um vulnerable devices. We can look at software software repositories sbomb lists firmware. There are companies that are you know downloading firmware at scale and deriving the sbombs from it. It's very interesting data you know has you know that data at hand. Uh I don't but if if you know please reach out because it's very interesting information for us to use too. Uh you know hardware go read the hardware specifications go read protocols or RFC's check dumps if the RFCs are not public. There's a lot of approaches and ways to study this. And in the next slides I will share with you some of our you know explorations into this data sets. Now first case is kind

of obvious if you ever did pen testing back I don't know 20 years ago you search for SNP SNMP SNMP had a lot of information. It still does if you have you know the proper username and password. It used to be easier back in the days. Nevertheless there are still a lot of SNMP exposed online. And we look at our scan data. there's around 15 million IP samples uh last month alone that we can derive some sort of information about the underlying architecture and and sometimes we can even get the Linux kernel version. So with those pieces of information we can kind of derive high quality probes to understand if they're vulnerable or not. So for example of the 600,000

systems we we we we saw 400,000 we were able to uh properly identify precisely the the architecture if it's 32-bit or 64-bit of those half were a little more than half 32-bit that so that's a big telltale sign if you're running a 32-bit system probably will crash and then from from those around 20,000 of them actually had the full Linux code version. Now we know what which year the Linux kernel corrected this patch. So we have additional information to kind of state well there's a very high probability that this systems will definitely crash around 14,000 of those systems. The maps you see below is just a way to show that this issue really impacts everybody everywhere and it will

happen all at the same time. Moving into another server side example NTPD. NTPD Trey was was showing the data that uh around one-third of those NTPD servers were vulnerable to 2036 rollover but interestingly enough more than two or at least almost twothirds are actually running 32-bit architecture. So will they have been they have been have they've been patched for 2036 rollover just to fail a couple of years later? That's an interesting question. We have to go, we have to check and we have to understand if that's going to happen or not. It's the only way. So, Portugal has a lot of red dots there. Red is red is bad. You don't want to Okay, moving

away. So, I was I was telling I was trying to blow up gas stations last year. So, it's a research about ATG. If you don't know what an ATG is, is an ICST system that's used to control fuel in big fuel tanks. uh you think about gas stations because that's the common use case, but there's all all sorts of places where ATGs can uh be deployed, namely places that have big backup generators. So, think about military facilities, think about hospitals, think about airports, think about your friendly data center. So, from the ATGs that are exposed, there's around 10,000 of them online exposed. But there's over 1 million gas stations in the world today and they have to have some sort of

controller, an ATG controller. Okay? And that's just gas stations. We don't even know where the other like backup generators are. So are this device vulnerable? Will will a rollover happen and how bad it will be. Well, in what I found interesting in in this research is that I tested different devices, different code bases, brands, models and they both failed in the same way. And if you forget about a second, this is an ATG and think about an ICOT system in general that has a a web interface, right? The remote administrator administration started to fail. Why? Because the user could log in. It it verified the username and password, but then generated an authentication token that was always expired. It was back in

1901. Doesn't work. You're always locked out. Both devices failed in the same way. Um there's the le in this case there's leak detection freezes because the time stamps were all messed up. So you couldn't detect if there's a leak or not. Uh the logs were nonoperational. So in essence, these devices cannot be used or remotely administrated. So that's concerning. Imagine a synchronized colonial pipeline event at worldwide scale all at the same time. Doesn't look good. Moving away from that that example, moving to client side, I really like the the the smart TV example because a smart TV represents something that we are surrounded with in our day-to-day. It's a computer that doesn't look like a computer. Okay, there's a

computer inside of it. It's running and it's supporting all kinds of systems across all sectors of our society. A smart TV and nowadays everything is smart. You can't even buy a toaster. That's not smart. So, a TV is used to display information. It's not just to watch the game. So, it could be uh in your hospital. It could be in, you know, your airport, in your subway, it could be everywhere. And essentially what we well well not we that uh state strategy analytics predicts by the end of next week year there will be over 1 billion smart TVs in the world working 1 billion with a B that's that's almost billions and billions like my like car like Carl Sean

would have said so how can we fingerprint these smart TVs if they are not directly reachable and online well some of some of them I I learned that are which is also bad news But anyway, uh you can, you know, the vendors have the brand and models that connect to their firmware update servers and things like that. So they have that telemetry, but so do ad companies sometimes because they serve ads when you go watch YouTube on your TV. They grab some information about your TV. But funnily enough, also malware because some TVs come pre-installed with malware. Uh, and I was looking at our sinkhole data and we have some TVs there and I I just did a a

quick search in in a week I found 127,000 TVs reinfected with malware communicating with just our sink holes uh that just see a small part of what's going on in the world. But 6% were vulnerable to the rollover. How do I know that? It was the branded model of my TVs back home. So I tested those ones. Are there more? Are there less? We don't know. we have to go and and check but I'm not saying 6% of 1 billion is vulnerable but any percentage of 1 billion is a very large number okay the examples could go on and on these are just some explorations uh there's a group chat I see some of you on your

laptop and if you are please open your laptop and run this terminal this command on your terminal fine you modify the time some date over 2039 and just raise your hand or just yell if it fails I know you want to be the first to get uh okay measuring this at scale it's very hard okay we have to accept that at some point we are going to be unable to map out every single system with high confidence so to have a more accurate insight about the scale of the issue we we can correlate different data points in some uristic fashion we can take educated guesses to to have a way to look at this problem at a global scale

for example uh the OS vendors have statistics about you know the different computers that install their own uh systems and luckily Debian shares the statistics it's in a package called popularity context funny name um so we can kind of use that as a proxy to understand uh a Debian system per year how likely it is to be a 32-bit system or not we have the statistics imagine you you scan the internet you find a bunch of different you know uh fingerprint and you find that FTP server over there with uh it was built in 2003. So built in 2003 and still deployed and online probably, you know, likely it's running on a 32-bit system. Is it 100% accurate?

Well, it's not, but it's it's a start. So with that in mind, I start to explore more and more all those architectures that Debian was sharing and trying to figure out which one were 32-bit, which one were 6 64-bit and then plotted over time. And if you're a mathematician, you should be very excited right now because these are our perfect curve adoption curves and declining curves. And it's they kind of show that sometime around 2012 64-bit systems became more popular than 32-bit systems. But 32-bit systems didn't go away. They are still being used and reported back today. Um, is this a universal proxy for every single Linux system? Well, it's not. There's operating system bias in this kind of

predictions. there's distribution bias even like I read that enterprise Linux will probably have longer adoption cycles because they they last longer probably Ubuntu systems they cycle faster so there's more newer systems coming into the the the market but it's still a useful proxy for Debian it's it's a perfect snapshot we can any Debian system we can use this to try to assess per by year which architecture is behind that system and for you know Linux Devon derivatives it's also a good approximation um it's a way to godge the weather so to speak anyway my last chart I promise um SSH is extremely widespread across many systems I don't have to tell you that uh it makes a very interesting target to

study it does not report the architecture when being scanned well usually it reports its implementation right its version sometimes even the underlying ing operating system we can see. So I looked into two distant implement implementations to see how widespread they are but they are very different implementations. That's on purpose. I wanted to understand what is the probability of a version given its release here being deployed on a 32-bit system. So that's chart a so the two different versions are drop bear and open ssh. Drop bear has a completely different probability of being deployed on a 32-bit system than open ssh. makes sense because Dropberry is specifically designed for low resource and historically, you know, 32-bit systems.

Many times they are still 32-bit systems despite the hardware is 64-bit because running a 32-bit system consumes less memory. It's faster. So, you have a 64-bit processor, but still a 32-bit system behind it. Open SSH is, you know, full feature standard SSH server for all mainstream servers, desktop clubs and so forth. So I wanted to understand how likely it is for a 32-bit system to also be vulnerable to Y2K38 given the release here and that's chart B. So if if you are running a 32-bit system, how likely it is to be vulnerable? Well, before the patches, before anyone fixes anything, you're 100% vulnerable almost, except if you're running OpenBSD, which has been non uh vulnerable for a long time, or

some Windows versions. But you can kind of measure this uh at scale. Um, if you multi multiply the two together, you end up creating a rough vulnerability of the probability that will tell you how likely it is that a random SSH server as long as it in this case drop air or open SSH random one uh is vulnerable to Y2K38. So looking at the data set of 12.5 million SSH fingerprints that had like the exact full version information like CP8 2.3 then we can access using this method methods that around 2.5 million are probably vulnerable to Y2K38. Now read all these charts with a grain of salt. Okay, they are approximations. There are estimatives. They are good

guesses giving our research and I do believe that the results are not exactly like this but they are in the same order of magnitude. Um interesting to note about scanning for this unlike CV scanning if you do like vulnerability research and trying to identify vulnerable versions unlike that um the false positives are usually low because package backports usually cannot fix this issue you cannot you cannot backport uh uh let's say an open s age version and fix it for to use 64-bit time because the operating system hasn't been updated yet. You have to update the whole thing. So, in this sense, it's easier to less to to have less false positives. But the main problem is something else. It's the

false negatives. So the 64-bit operating system can be safe and it can be running still a legacy 32-bit you know compatible application and that application can be vulnerable and it will crash or corrupt some data in 2038 even to even though it's on a safe 64-bit system. So a remote scanner will have zero visibility into the architecture of the actual processes that running that are running behind that system. only an authenticated scanner could do that and probe all the processes running. So is everything that I'm speaking about the last couple of slides useful at all? I I think it is. I think probabilistic like remote scanning is not the answer, but I believe it's a good triage tool to help

solve for this discovery channel challenge not. So when dealing with a mass number of systems, a remote scan helps you sort out that unknown list into buckets that you can prioritize, start looking, start fix, start moving. But then then we have to test. Okay? And for a 100% accurate assessment, you need to test each system. But testing means two different specific things. You can't just check the operating system and be done. This is the central problem of Y2K38. You have to check both the systems and the applications. And for any complex system, it has to be in CTO. Why? Because simulations and lab tests, they make assumptions. You cannot fully, you know, replicate the chaos and things

that happen in a live test. And this isn't like theoretical. If you think about if you if you know one of the most f famous is overflow failures in history is Aryan 5 rocket explosion it happened because of this gap okay a 64-bit number which is the rocket's horizontal velocity was being converted into a 16 bit number so the rocket was much faster than this predecessor and when it you know surpasses that speed after 37 seconds uh the velocity got just too high. The six 16 bit number overflowed and the navigation system crashed. So this wasn't a time based overflow, but it's it's it's the same thing. There's a number that overflows and in this case

the device self-destructed. So why didn't simulation catch it? The code that crashed reused the Aryan 4 code didn't replicate in a perfect way. So here's some examples of things that you can test if if you own like a nuclear submarine. you can start test your own nuclear submarine but uh really these things have to be tested satellites missile systems telecoms I won't read the entire list part of the list is in red because I already test you know small subsample of those and I found issues in every single one of them cars included so remember this time is a root of trust and it's surrounded by assumptions like it can only go forward we we are seeing

that that's not True. Think about it like certificates, cryptography, processes, logs, consensus, every aspect of our digital world is time anchored. TLS will break. It's easy to understand what that means in your browser. What that means is you have a pop-up saying, "Oh, it's expired." But if it's your operating system, it will probably won't update. Maybe you have a warning, but maybe you don't get a warning. If it's machine-to-achine communication, it will just silently fail. You get no warning. So what I'm here to tell you today is this. All these 2038 rollover vulnerabilities, they can happen today. Don't have to wait 13 years. You can make them happen today. And that's a problem. NTP is not secure. GPS

is not sovereign. Time manipulation is fairly easy and inexpensive like GPS spoofing, NTP injection, file format fields, first protocol time stamps. This is real. This is lab tested. Systems will fail sometimes silently. We don't even know where most of them are. Just little over a month, China publicly accused the NSA for penetrating their national time service center networks between 2022 and 24 and they were allegedly messing up with time, disynchronizing time by a very small fraction. So this kind of attacks are happening today. Now imagine this little snail over here is my well not my jet GPT depiction of subtle snail. Subtle snail is an Iranian group that's targeting telecoms here in Europe. They were uncovered like some

weeks ago. Imagine if they go into an ISP and start pushing down spoofed NTP packets at scale to everyone. What will crash? We don't know. What an easy, cheap, and potentially disruptive attack. Remember who controls the clock controls the system. Now if after all of this you if you're feeling like this urge or this itch okay I'm going to go back home and I'll start changing the date on everything. Uh remember you know two major things. One is setting the clock is like picking a lock. You don't do it if it's not yours and you don't do it if you depend on it of course. And this is a question for you before I leave. If a device won't

let you set the date while you're testing, if a device won't let you set the date past January 19, 2038, ask yourself why Trey, >> I think we should give Pedro some round of applause. That was magnificent.

So now I'm going to try to take us on out to the close here. So hit me baby one more time. I didn't really actually like Britney's fears until I started doing this research and then I found that she's kind of cool. Anyway, in the 90s pop and panic both went global. Y2K did not sneak up on us. Peter de Jogger warned us back in 1993 and the world listened. We had UN coordination with over 120 countries. National task forces, ZARs were appointed. In 1999, the US and Russia, still barely speaking after Kosovo, opened a joint missile warning center so that a glitch would not accidentally start Y2 uh World War II over the Y2K rollover. That's how

seriously we all took it back then. And it wasn't just code. It was coordination and investment. The dotcom boom did not start with venture capital. It started with Y2K remediation. Y2K forced a global audit. It built the safety net that made the digital economy possible. When Y2K passed, we stopped investing and the first dot bubble popped soon thereafter. The digital age was built on a successful failure avoidance campaign. Society saw that technology had made technology safe for society and we believed the risks were behind us. If we act together now, we're not just preventing disaster, we're laying the foundations for the next era. We're at T-minus 49 quarters. So what did Y2K really cost us? Adjusted for today's

dollars on the order of 1 to2 trillion. And what did we get for that? Nothing happened. That's because it worked. See, this is the paradox of Y2K. The success of the remediation effort made the return on investment invisible. And so now no one sees the business case for 2038 and 2038 will be harder. So, it's worth asking, is this a problem which is solvable, or should we all just go get drunk? We wouldn't be here talking to you if we thought it was a case for despair. There is much room for hope if we start to act soon. Like, can we fix this? So, well, coordination is failing us right now, but technical fixes do exist. The challenge is

deploying them across billions of devices. IPv6, remember, took us over 30 years, and we're just about halfway done. 36 and 38 converge in one decade with embedded systems, bad incentives, and no coordination. If we face that reality together early, we can still blunt the worst. That's why we need to remember that not all failures hit us at the same speed. Triage by tempo is the only sane strategy. And may I say I think I invented the Britney blackouts light effect here. So what we don't want there are cascading blackouts. First life safety. If medical or aviation systems desync people die. Next command and control. A false time stamp in deterrence or live ops can escalate a

crisis. Then critical infrastructure, power, water, transportation, where time desynchronization has the potential to cascade into black swan events. Last, the economy. Our markets unravel in slower motion. Money can wait. Lives cannot. So, we triage by tempo. Our reality will do it for us. And respect to Britney. Now, let's talk about the 800 pound gorilla in the room.

We've known about 2038 since the '9s. But admitting this means confessing that our foundations were brittle. See, there's Amelia Heheart beside the gorilla. We pretend it's not there. We see it, but we pretend we don't. Y 2K had prophecy and panic. Governments had to be seen acting, and so they did. 2038 just has burnout and fatigue. We're at the apex of an AI tech boom now. Smart money expects the bubble to pop any minute. And as it deflates, the veterans who built our last era will be entering retirement. What's left is a generation of cloud engineers who've never been taught what an RFC even is, just as the rollover comes to. This timing is brutal.

We'll need to invest trillions in the long live stuff we can't just reboot. Power grids, sensors, satellites, industrial controls, medical systems. Now, convincing governments, financial markets, and the public to fund a remediation at scale campaign for invisible systems during an economic contraction will be one of the hardest challenges of our era. We converged our entire planetary communications infrastructure onto internet protocols. If there's still a red line between DC, Moscow, and Beijing, it surely runs over internet protocols or relies on them. When the internet's clocks drift, there's no fallback communications plane. And so remediation is not optional. Climate change is glacially slow by comparison, local and gradual. 2038 is instantaneous. Everyone, everywhere, all at once. 3:1407

UTC19 January 2038. Set a reminder. Put it in your calendar if you're able to because when that second tick's over, we'll learn whether we did enough. So, politics rewards fearful silence. But physics sadly does not negotiate. And the supply chain gorilla is on the roof. 70 to 90% of critical technology depends on Asian production, but the dependencies run in all directions. Taiwan, China, the US, and Europe are all interdependent. We have different install bases, but the same vulnerability class affects us all. Pedro's data proves it. Everyone has this problem. It just manifests differently depending on what you run. Resilience isn't just about protocols. It's also about parts. You can't stockpile your way out. You can't compete your way out. And no one

is going to willingly accept monopoly solutions. Trying to take advantage of each other's vulnerabilities simply guarantees we fail. Kongs on everyone's roof and that monkey won't climb down by itself. Either we all coordinate or we all fall together. Now let's bring this home to where it actually means life and death. Dump technology today becomes systemic failure tomorrow. 2038 won't hit the world equally. Obsolete gear is already being dumped into emerging markets with unsafe firmware today. Hospitals, utilities, and transport and underserved regions will inherit the failures which richer nations refuse to fix. That's not technical debt. That's moral debt. The weakest prepared will take the hardest impact. Whose children will live with the failures we leave behind? Justice

means exporting resilience, not vulnerabilities. And that inequity doesn't stop at economics. It reaches to defense. Oh no, that is so funny. I even used to dedicate a laptop so there would be none of that. Hang on a second. Ah, so sorry. So this is Macau's big gorgeous head. We might wish we were home alone, but we're not. Deterrence chains are brittle. A rollover during peak tension could escalate before humans even notice. Nuclear subs can't just rep boot for patches. Every recall means a blind spot. Defense avionics misreading a time stamp could look like an attack. Even remediation has risk. Patch windows open use it or lose it logic. So I would submit that based on what

Pedro described about the type of experimental let's say alleged attacks in this domain, time synchronization is a layer that most of us have never pentested or even seen a conference talk about. And so perhaps we should not be monkeying around here at a time when the world is on the brink of wider conflict. But with less than 49 quarters remaining, it's important to remember that under the best of circumstances, we're already running late for 2038. We need to fix this first in coordination or we fall together. This is a competition unless and until we decide to cooperate. It's hard to think when you're trying not to sink. And so we do our thinking now before we hit the iceberg. Here's

what to do technically. Think Sweden. They treat time as critical infrastructure. Maybe we should too. First test silent failovers don't self-report only test beds catch them early. Second know your assets. Third harden in layers NTS on premise clocks and ribidium or cesium backup so GPS's loss isn't fatal. Remember this is about buying yourself time to fix the problem because the problem is exploitable today if your time upstream is manipulable. Harden in layers. Isolate your control planes. Fourth, critical infrastructure should not be syncing time over the open internet. And fifth, clean your NTP hygiene. The net is polluted with junk Stratamax servers. Don't let bad time poison good systems. Treat time with as much care as you treat DNS.

So, one minute on policy and procurement. I'm very sorry for everyone who's playing the badge. We've talked about risks, but money still talks. And this is where some of our agency lies at the national level. This is a TLP red conversation with your vendors. Procurement is leveraged. Demand post 2038 certification and make your vendors prove it. Push for a temporal bill of materials at the regional level. This is a TLP amber conversation. See telecoms and grids and skate across borders. You can't just trust. You need to audit with your neighbors. Sunlight is leverage. At the international level, this is a TLP green conversation. See, these are in many cases going to be weaponizable. zero days that we're trying to

remediate. So share wisely. Don't arm nihilists. No nation fixes this alone. Talk to your adversaries. Own your clocks. That's sovereignity now. So demand proof, audit, and coordinate. Sorry. Resilience isn't a patch for 2037. It's having a backbone in an upcoming meeting in Q1 of 2026. Here's one more tool. Obsolescent by design. When we procure obsolescent by design systems, every deployment becomes a countdown. Every upgrade an amnesia ritual.

Story time and then we'll have questions. In pilot training, they said we wouldn't do spin training because it was too dangerous. More people died doing spin training than getting into spins. A spin is when the airplane does this down to the ground. So my dad, who is a former US Air Force instructor pilot, said, "No son of mine gets a pilot license without doing spin training." I was like, "Holy [ __ ] no." But up we went. So here's a key thing to know about spins. There's a concept called minimum safety altitude, and it varies from aircraft type to aircraft type, but it's baked into the physics of the aerodynamics of the specific airframe. If you do everything

right, it still is going to keep spinning for a while before you recover. And there's a minimum safe altitude to start the procedure before otherwise if you do it below the minimum safe altitude, you better hope your parachute has been service lately. That's not something you want to be thinking about while you're in a spin. Another key thing to know about spins is they are [ __ ] terrifying. The ground gets bigger so fast and it's coming at you so fast and you've never seen it spinning like that and your dials and your instruments are spinning and your controls aren't responding and all you want to do is [ __ ] your pants. But the only path to survival is a

counterintuitive procedure. Power idle ailerons neutral. That guy didn't pull up in time. So, it's like a specific procedure that's five or six steps and you have to wait in between and then when you do it, you still have to wait for a certain number of rotations before you know if you did it right. Back in 1912, a guy named Wilford Park discovered this and people swore he made a deal with the devil. It's so unintuitive. Every primal animal instinct within you cries out to fight it. But the only path to survival is discipline sequence and steady nerves. I once asked my dad, who is also a former airline pilot, how it was that he would go to a day job that involved

carrying the moral weight of all those lives in the back and all the families who would be touched and impacted if something went wrong. And he said to me, "Son, it's easy. You're in the front of the plane and wherever it goes, you get there first. So, you focus on seeing your own family again." Now, we're not the pilots of the internet. that the internet has no cockpit. It's a distributed system. This is by design. But we are its stewards who flew here to Lisbon. So those of you who made the trip by plane, I want you to think about the stewards whose job it is to make sure that everyone is in a dignified brace

position if there's danger. In terms of 2038, we are above minimum safety altitude barely, but the airframe is already locked into the spin. We may do our jobs so well that the public doesn't notice in 2038, but we will notice. Stewardship means acting decisively with the information we have been given. When we normalize expert within our critical infrastructure, we convert time into debt. When that throwaway logic infects the invisible computational substrate that upholds civilization, it stops being a business model and becomes existential risk. The aircraft is spinning. Remediation is not optional and there is still time. 2038 isn't just a technical deadline, it's a moral one. See around Y2K, Dennis Richie, co-creator of C and Unix, gave a talk at

Bell Labs where he was asked about Y2K visa v Unix. And he said, "Unix has no issue until 2038. And I don't care because I'll be dead by then." And he was right. Dennis died in 2011. May he rest in peace. This is not a criticism of Dennis. It's a reminder like many of our software design patterns, Dennis too was mortal. People keep saying to me, Trey, you're just too early. And I say maybe, but I just keep seeing these kids and students on trains and thinking about what they're going to inherit. Remediation isn't heroism. It's being an adult in the room. And getting home to your own family in this frame means realizing the impact this problem space

will have on your life and that of your loved ones. You may say, "But I'll be long retired by then. This is someone else's problem to fix." But I ask you, do you want to retire as a hunter gatherer? Don't wait to be warned by your national authorities. You have been warned today. ITD inventory test demand proof. Write that down. There's your practical audience takeaway. And so with that, we'll move to Q&A. I'll simply say this. Every resilient system begins with a courageous human who refuses to shut up in a meeting. >> Remember guys, setting a clock is like picking a lock. >> Thank you. >> Good job.

>> Any questions? >> We do have a few minutes for questions.

It is hard with these lights. >> First of all, I wanted to say thank you for this talk. For a person that's inherently stressed, this has made me think about so many things. So the first question that I have is what do you think it's the best thing that we can do like as a developer that has the best return on investment uh when I go into work tomorrow or Monday? And the second question is more of a curiosity. Due to the all the embedded systems that don't have a way to be upgraded or patched, do you think we will have a problem with e-waste and managing all the the things that will be thrown out? Uh, and those

are my questions. >> I I sometimes think about that and cry while I'm washing the dishes, but I didn't want to put that in the talk. You're right. This is going to produce immense amounts of e-waste. One of the problems is we need to just stop making the problem worse today. People aren't testing for this. We're importing devices. We're deploying devices. We're embedding devices. Think about how many computers are in a car. Like on average, I hear about 200 in like a Tesla rolling off the line right now. There's a lot of embedded systems in there and they're not testing for that. What are you going to do with all that? How are you going to rebuild all

that? You know, there will be tremendous amounts of waste. But if we could just start testing and stop making the scale of the problem bigger today, that would already be a start. >> Yeah. And I don't want to go into geopolitical that's what you do. But uh indeed there's this uh temptation of just getting rid of those systems to uh countries or nations that have less acquiring capabilities. So you can get rid of your old stuff, buy new stuff and just re allocate the problem to someone else. As for your stress levels, yoga helps. It helped me. Um, as an engineer, just being aware of this is an issue. It's the first step I I guess. But then

you have to start making questions. Where does time touches all my processes? Where does time touches uh the systems that I'm building? And how c can it fail if it's something related to time? Because you might not be able to control the system where your software is running. You you might be producing perfectly apparently safe software, but then it's running on a unsecured operating system. So it's not easy. Has to be a coordination effort throughout the entire software development life cycle. >> Yeah. Thank you so much once again for this talk.

If you're asking a question, we can't see you because of lights. So, just talk. Otherwise, clap >> so we know what to do. >> Hello. Hi, Pedro. Hi, Darly. Um are there currently any scanners for this that would s uh sit on the system to check if for example libraries are using u 32-bit uh counters things like that so anything that would allow I know that there are systems that are closed and for which this can't be done but at least for systems that are open and where we can run stuff that will help us identify by protocols, libraries, etc. that have these problems, any existing ones are being developed that you know of. >> Uh that's an excellent question. Uh

first of all, uh uh running locally means you have much more uh access, right? So you can see more stuff and that's kind of the only way to be sure is uh to do what you just described scanning all the binaries in the system understand what what time libraries are they running in what's the operating system version and architecture um I'm not aware of any particular piece of software it doesn't mean it doesn't exist it just means I'm not aware of it >> I I will say this that's a great idea but also that we're not the best self-promoters. This is a big talk and so we tend to run out of time and so

we've skipped this part. We're trying to spearhead coordinate a broader effort. So apocalypse project.org is got a signup but also we're setting up this uh time security special interest group inside of first.org work which Pedro and I are going to co-lead with to try to coordinate stuff like volunteer efforts to create the scanners to put them on GitHub to you know this is work that has to be done. >> Yeah, thanks. Maybe we should work on that.

Any more questions?

Well, thank you for coming after lunch. >> It looks like we're out of time.