Unlocking macOS Internals: A Beginner's Guide to Apple's Open Source Code

Next up on the stage, we've got Olivia Duchcci, who is a senior security engineer at Sakun Fra. >> Yeah. >> Okay, just wait. I'm sure I will go ahead and botch something with my tongue yet. Uh, she's also a personal trainer. She's been around for a while. She's spoken at many places. Um, how many people here have MacBooks? >> Yay. How many people wish they understood what was inside a MacBook? >> Yay. >> How many people here want Chucky to just shut up and get on with it all? >> All right. Here's Olivia for you with the whole thing on unlocking Mac OS internals. >> Hi. All right. Um hopefully everyone can hear me. So, hi everyone. Thank you so

much for being here. My name is Olivia Galuchcci and I'm a security engineer at Secure Infra which is a consulting MDR and research firm in Germany. And today I'm going to work walk you guys through the foundations of Mac OS internals and how to start exploiting them uh using Apple's open source code. So what are Mac OS internals? Mac OS internals are the underlying components that make the system work. the X and U kernel, process and memory management, system libraries, sys calls, device drivers, basically everything that runs under the graphical user interface. It's the difference between what you click and what's actually happening when you click something. Mac OS internals matter in so many contexts like security

research, performance tuning, or debugging weird system behavior. It lets you analyze vulnerabilities, understand potential undocumented features, and even patch behaviors. And there's a bonus for iOS researchers, which is that iOS shares its core with Mac OS. Now, I want to address the elephant in the room, which is Apple secrecy. And its secrecy is problematic for a number of reasons, but it's a myth that 100% of Apple is closed source. A large chunk of Mac OS and iOS is actually open source. Apple publishes the X and U kernel, system libraries like Libby C and Libs Dispatch, Damons like LaunchD, and various other things. Anyone can go online and read the code that runs on a Mac or iPhone. Of course, Apple doesn't

release everything. Some frameworks like Core Foundation and UI Kit are closed, but what is public is incredibly useful, and knowing how to work around those gaps is really helpful in internals research. Now, let's explore the intersection between computer architecture and Apple's open source code. To understand Apple's operating system, we've got to talk about Darwin. Darwin is Apple's open- source Unix like operating system. It's the foundation for both Mac OS and iOS. And at the heart of Darwin is the XNU kernel, short for X is not Unix. It's a hybrid of the Mac micro kernel and BSD. Mac handles task scheduling, memory, and interprocess communication or IPC. BSD adds the POSIX APIs, networking, and file systems. And IKit, which is written

in uh C++, manages the hardware drivers. There's there's some other things in this mix too, but we're not really going to get into those. And so when Apple builds Mac OS and iOS, they're layering proprietary features like the guey and frameworks on top of Darwin and XNU. So yes, Mac OS and iOS literally run on the same base OS. Here's just a cool diagram of how Ubuntu and Mac OS compare. Um, it doesn't really have to do with much other than I thought that people might find it cool. So let's explore this shared foundation. Here is a table comparing some system dammons and services. You'll notice that LaunchD, MDNS Responder, the preferences Damon, PowerD, and others run on both

platforms. Sometimes they're even identical. Because of this, vulnerabilities found in Mac OS Dammons can also exist in iOS 2. And as you can guess, it's an advantage for researchers. You can debug and reverse Mac OS binaries, which are easier to work with, in my opinion. and then apply that knowledge directly to iOS. Another important note is that in general iOS is considered to have more restrictions because it has stricter sandboxing entitlements and application isolation. But underneath those constraints, the code is often the same. So what is actually available? Apple publishes a range of open source components like the X and U kernel, core libraries like lib C, uh lib system, lib dispatch, and some other dammons like

launchd. If you go on to their open source page, you'll notice it's like over 400 repositories online. These are all downloadable, inspectable, and sometimes even buildable. But they don't publish everything. Missing or partially available components include core foundation, CF network, launchd in recent releases, iOS specific drivers, the full dynamic linker, and their crypto library, which is known as core uh crypto. And it even has like legal protections like a 90-day and then delete policy. uh which you can find online. Um, additionally, its source drops are often incomplete or outdated. You'll see headers referencing files that don't exist or tarballs with build instructions completely missing. So, while Apple's open source code is a gold mine, I think it's important to approach

it knowing you'll hit some walls. The big idea here is that Mac OS and iOS share the same core. Apple open source is more than most people think, but not everything, and navigating this code is essential for internals research. Now, we'll talk about how to actually find and use this open source code. So, Apple's open source a surprising amount of code as I've already stated, but the the company from from an outsers's perspective is also kind of redacted or explicitly redacted and has really moved around a lot of these resources it resources and as a result depending on how you're going about accessing this code, it can be really difficult. For example, a tutorial written four years

ago might include links to exclusively 404s if that tutorial links to Apple's downloads or source dumps before 2024. So then where do you find this code? Historically, Apple published uh their source code as gzipped tarballs on its open source website. These archives were organized by Mac OS version. So things like XNU8792.4.9.tar.gz and so on, right? And that worked until it didn't. In March of 2024, Apple began retiring these direct downloads. Now, the primary home is its GitHub organization called Apple OSS Distributions. Each component now has its own repository tagged by version. So, instead of downloading from Apple's website, you're grabbing release sips or browsing repos on GitHub. Now, how do I navigate GitHub? Well, as always, each

repository follows a basic format starting with a name like XNU, lib dispatch, or lib C and then a set of tags like XNU8792.41.9 which maps to a specific OS version and then a release field again if if if you're lucky that tells you what Mac OS or iOS version it matches. But here's the catch. Apple just doesn't always uh explain these mappings clearly. And this is extremely important because you can't often explore things properly if you're not using the version of Xcode used to develop that specific component. For example, XNU version uh 8792.41.9 corresponds to Mac OS 13.4. But Apple doesn't just tell you that. You have to infer it by checking the release pages or using

communitymaintained lists like Wikipedia's version history. Unfortunately, there's many challenges with Apple's open- source releases, and I'll cover the three main ones here. First is delays. Code usually appears weeks or months, maybe even years after the OS ships. There's no warning, no schedule, just coming soon. So, like sometimes even six plus months, the release pages will still have a coming soon notice. And then next is incompleteness. Some tarballs are missing files. Headers reference code that just is not there. And then entire components like core foundation or newer launch db buildilds are MIA. And then lastly is uh tag confusion. One second. So tags like you know XNU 8796.121.2 mean nothing unless you know how to decode them. And that's why learning how

to use git tags, learning you know general project structures and knowing how to use git and github in general is really useful in this process. So, I want to go into a few uh more tips and tricks I use. First, I check the release field. If it's available uh in the repository metadata, and if it ever says something explicitly, if you can find something explicitly like Mac OS 13.4, use that. It's usually accurate. I also use Apple's distribution manifest to cross reference components per release. Lastly, I look for clues in the binaries themselves, like uh version strings or source file names via strings or what programs to figure out which project a binary might have came from.

And what I've learned is that finding the source source code source code source code is doable but messy. And again, Apple doesn't make this easy, but with GitHub, a few sleuththing tricks and some trial and error, you can usually track down the source for many major system components with the caveat being it might be a much older version of that component. So, yeah, now I will show you how I actually use the source code, especially when it's incomplete, to do reversing and security research. Okay, there we go. Um, specifically, we will be looking at how to use Apple's open source code to examine system binaries. Even though the code is often incomplete, it can still guide your

research. The key is to treat it like a map. You might not have the full te ter territory, but the landmarks are real. And what I do and what many other security researchers do is use Apple's open source code as a reference while working with the binaries. You take a binary like launchd, sislog d, a kernel extension or whatever you want and use the available source code to do various things. For example, I might try to match known functions. This helps identify familiar code and binaries, making it easier to understand behavior and verify functionality against Apple's open source implementations. I might also try to annotate call patterns, which reveal how functions interact and in what sequence,

allowing me to infer program logic and detect deviations from expected behavior. Lastly, I might try to reverse unknown logic. Although difficult, attempts at this enable me to uncover the purpose and or vulnerabilities of soft software that lacks source code using structure and context to reconstruct what it does. Essentially, even when symbol names are skipped, you can use structure, string constants, or known SIS call usage to map back to open source code. And here are four tools I use uh to do this. And if you've already been in the reversing space, these tools probably aren't new to you. Um, of course, there's strings, which dumps human readable strings. This sometimes includes debug messages, file paths, and

format strings. Then there is what, which extracts version and project information from binaries. There is nm, which lists symbol names, especially in unstripped binaries or frameworks. There is also uh class dump which extracts Objective C class and method definitions and it's great for finding things like interface information. And through using all these tools I can figure out which open source package a binary might have uh came from or have like some sort of like ancestry in or what APIs it's using even if I don't have the the source code or the exact version. And then after this I'm usually at a point where I want to go a bit deeper. Here I would generally use some sort of

binary diffing which is the process of com uh comparing two compiled binaries typically before and after a patch to identify changes which helps uh pinpoint patches uh or changes and uh vulnerabilities within that source code. tools like bindiff and diaphra the the person who made made it says like diaphora but I'm gonna call it diaphra um they let me compare two binaries again uh like before and after a patch and maybe spot those changed functions to give a theoretical example uh if Apple silently fixes a vulnerability in sislog d I could try to diff the old and new binaries and immediately see which function changed then I could gp the open source code for that function reconstruct the patch

logic and and then reverse the vulnerability from there. If you prefer Gedra, the kernel cache plugin lets you load X and U function signatures into the kernel cache. I I really like this uh plugin and I did a workshop on it once and it was great. So uh yeah, it auto labels all like the known kernel functions using headers and it this is a really big uh timesaver and in the academic sense we can actually look at the example I just described. In 2016, a researcher named Ox OSX Reverser used uh diaphra to diff syslogd from Mac OS 101.2 and 101.3. Only one function had changed. And as you can see in the code on screen, in

syslogd's add lockdown session function, the call to realloc plus4 was changed to realloc* 4 + 4, which was to allocate enough space for each session and prevent a heap overflow. All other functions uh remained identical. Same thing with the the call graphs. And inside this he found a string that read read uh sorry it read add lockdown session realloclock failed. And this led him to uh the vulnerable code within Apple's open source syslog d. And this kind of surgical diffing is how some vulnerabilities are reversed after a patch. This is especially important because security fixes often need to be applied across multiple files and locations. For instance, suppose you patch identical code in files one and two on

Mac OS, but neglect to update files four and five, which is which is very common because these operating systems that just have tons and tons of files. And in that case, the unpatched files four and five still contain the known vulnerability, leaving your system at risk. It's also worth noting that updates can be platform specific. If the iOS versions of file one is patched but it's Mac OS equivalent is not Mac OS remains vulnerable unless again there's other uh like security protections in place which often times there is the same actually goes for kernel extensions by dipping kx across versions researchers have found changes to external method tables surfacing newer patched entry points and IO kit drivers

thus even partial source is powerful combine it with binary analysis and you can reverse Apple systems with surprising accuracy. And here you can see you're not really flying blind. You're flying with half of a map, a flashlight, and some really good tools. So, you might be wondering, can I just like build Mac OS? I too get ambitious with my findings and I'm like, time to compile Mac OS from scratch. Yeah, I have since learned that compiling these components yourself, or at least for me, is only sometimes possible possible and almost always painful. I'm sitting there with Apple source code downloaded and I'm ready to compile like whatever I've downloaded. And after hours of frustration and

dreams of achieving some new like mental research PR, I embrace this truth of you or maybe just me can't just build Mac OS. not completely, not cleanly, and definitely not without frustration. So what makes compilation hard? It's hard really because of three main reasons. First is incomplete source code. Apple regularly emits proprietary or device specific code. For example, there's instances of all ARM specific assembly being stripped from iOS X andU missing power management subsystems like X uh CPM and to this date some important drivers and frameworks have never and probably will never be published. Second are environment mismatches. Apple builds like all companies with their own internal tools and exact Xcode versions. If you're off even slightly, you'll hit

missing headers, undefined macros, compiler flag mismatches, and linker errors. And lastly is sparse documentation. Although I doubt this is a new problem for anyone who's into, you know, Apple security research, right? You probably won't even get build instructions. There's community guides, but most of it is trial and error. And at least for me, I always end up having this constant feeling of excitement and confusion. who spent hours resolving dependencies that Apple never documented in patching quirks like what you see on screen. And this isn't a joke, by the way. This is a real fix from a Mojave X and new build. So, if you can't fully build the OS, why even try? Well, you still get a huge

value from, you know, reading the build scripts and headers, tracing the structure and the logic, compiling parts like lib dispatch, and testing patches in controlled environments. And even if said patches are kind of janked on some OS versions from 10 years ago, I think of the build as a reference model, not a product. One tip I have is to use the iOS simulator on Xcode, which allows me to obtain working binaries with symbols and less cache extraction pain. It includes standalone DIB frameworks, works with public symbols, ARM 64 slices and versions very similar to real iOS uh like version usage and you can load these into IDA or gedra and reverse cleanly and you don't require any sort

of jailbroken device. It's an amazing shortcut for when source code is missing but you still want visibility into system behavior. And all this is to say that compiling Apple's open source software is hard and incomplete but worth it. Every build error in my experience teaches me or has taught me something about how Mac OS is stitched together. I try to use this process as a learning tool, not a goal. And all of that said, what do we learn? Mac OS and iOS both run on Darwin. It's Apple's open source Unix like foundation. That means that what you learn on one usually applies to the other and vice versa. They share the same XNU kernel. Many system dammons and

tools are crossplatform and much of the good stuff like launch D, libc etc. is available in source form. Apple might be secretive but its open source code is one of your best research tools. You just have to know how to use it. So then what's like the strategy here, right? First use Apple's or sorry first use the open source code that Apple does release. Study the headers, read the build scripts and match symbols and you know functions and whatnot. Second when things are missing try to reverse those binaries. Use tools like strings nm class dump uh bind diff gedra. Combine static and dynamic analysis diff updates. F follow any sort of breadcrumbs you can find.

And then whether it's, you know, Mac OS or iOS, the internals are usually more open than you think. And even the gaps are reverse engineerable with the right tools. There's also whole communities built to talk about this stuff online, which helps a lot too. And so right here, this talk hopefully should give you a starting point for everything you need to start hacking these machines. And as you can see, it's not really just about compiling code or reading docs. It's about building the confidence to explore Apple's GitHub repositories or some random dial you found. And if you're curious, methodolical, um, and really just persistent, you can really do amazing work in the the security research space. And in terms of

the content of this presentation, you know, thank you so much for your time and happy reversing, but I have some uh terms of where you can go after this. So, one, all citations and resources are included in the footer of these slides and images and whatnot. And if you want to get involved with the Mac OS community IRL, there's two amazing conferences, uh, Objective by the Sea and Objective for the Wii. This one's going to be in London and that one's going to be in Spain this year. I've both benefited a lot from these conferences and I absolutely love them and full disclosure, I have strong ties to both of them. Um, there's also a Mac

uh, DevOps conference which is in Vancouver. I met the organizers last week. They're great. And then if you're interested in my other work, I have some a couple presentations coming up next week on Mac OS internals. And uh yeah, all my written work is available online. And I recently wrote a blog post on Mac OS internals for detection engineers if you're someone who focuses more on the defensive side of things. And yeah, uh thank you so much for your time. I hope you all enjoyed my presentation. And if there's time for questions, cool. [Applause]

Hi. >> Um, so between the x86 and ARM 64 binaries, what did you prefer? >> ARM because um that I recently graduated from college, so the amount of time that I've been like competent in this space has been limited. And the time that I have been competent in this space, it has all been ARM. >> So ARM. >> Okay. And the the second question, so you mentioned the like a strict binaries and unstricted binaries. Do we have access to the like a debug versions of whatever libraries they compile with the unstricted binaries? Because I would assume everything is like released strict. >> Yeah. So part of it. So, there's whole tutorials like about how you can do this

in an organized fashion online, but usually there's some pre-processing in place that you need to do to get the the unstripped binaries because they're they're not always like available. I think um I think Chabal Fitzell actually was the person who who published a piece on this. It was either Chabal Fitzell or also OSX Reverser. Um, one of them was like, "Hey, here's how you can can organize these things so that you can actually like understand what you're looking at." Um, and the same thing with the tools as well. One of them actually had like a full tutorial on how you can actually get usable information from uh from the the binaries. >> Okay. So, the there's like a service

from Apple that you can download them somehow. >> Oh, okay. So, you mean not or Okay. Yes. There's a there's a website and then there's also the GitHub. you can get you can you should be able to get them uh some of them from both and they're also organized by system component as well >> because back in the days like in Windows times you can download like PGBS like basically debug binaries from Windows and I'm curious what's available >> so um I think I had a little like gift like going through their website but if you go to it's I think it's like open s don't don't quote me on this. Just search like open source Apple and then

you'll find that website. But um on that website, you'll go into developer tools um on the OS page and there should be ones for Mac, iOS, I think. Oh, I think iPad. Um and then there's a developer tools section. >> So that that that's where I go. Any more questions? Is there any enforcement for any of those repos that requires you to 90 days? >> Oh, so the the the core crypto one I know for a fact. Yeah. But I think it's the usage usage of it itself because it they you have to like part partner with them in a way like you have to get like signed things from them and also there's uh Oh yes. Can I take a selfie with

everyone? >> Sure. >> Yes.

Okay, I got like six. Um, but yeah, sorry. What was your question? I saw I Yeah. Um, okay. Yeah, the Okay, cool. But yeah, some of them do have special special policies, but it's usually not the open source ones because the the license is always attached to that. So, that would be the policy. anything with like special things you usually have to get through an Apple portal that will make you like sign in and whatnot. I think I think I'm done. Yeah. [Applause]