Inside Apple Intelligence and Xcode: Special Presentation Welcome, everyone. Wow, this is amazing. It is genuinely great to have you here. WWDC is one of our favorite moments of the year when we get to share what we’ve built for you, and then watch what you build with it. Now quick show of hands. How many of you are in the Steve Jobs Theater for the first time? Okay, yeah, I see you. I hear you. Yes. Today we’re going to be going deep on Intelligence. One of the biggest things that we’ve been working on this year.

Across the industry, the AI story is often complicated, with different components coming from different companies and providers. And then you as the developer, are left to stitch it all together. It’s a lot of complexity before you get to the good part, building something that people love. And that’s not the story we want for you.

At Apple, AI isn’t a layer you bolt on. It’s integrated into our platforms. The silicon that runs it, the OS that integrates it, the frameworks and models that enable it, the tools that help you create with it. And this year, it reaches all new levels so your ideas can go further.

Now we know that everyone is at different places with AI right now. Some of you are just beginning to ask, what could this mean for my app? Others are deep in model development, pushing Apple Silicon to its limits. And most of you are probably somewhere in between. Curious, digging in with agentic coding and looking for the right place to start.

Apple’s AI platform has something for all of you. Like those of you who want to build faster. You got into this because you love creating things. You have ideas, you have users, and you want to move. And that happens when AI becomes part of how you actually work.

Xcode 27 brings agentic coding, not as an isolated chat interface, but as a collaborator that sees your whole project. It thinks in Swift and speaks in Apple frameworks. Whether you’re sketching out a new feature or asking it to write and validate a test, it meets you where you are accelerating the work without ever taking control of the wheel away from you. Think of it as a development multiplier, the same craft with dramatically more momentum.

Then, for those of you who want to integrate your app throughout the system, you want your app to feel like it’s connected and the way that people actually think and work throughout the day. App Intents is how you get there. Connecting your app to Apple Intelligence, and more. When you express your app’s content and its actions through them, something powerful happens.

Your app becomes part of the fabric of the system itself, woven into how people discover things, get things done, and move throughout their day. It’s searchable in Spotlight, available through natural language and in Siri. Then integrated in Shortcuts and the action button. That’s AI that helps you surface your app in more places across the system. Your app becomes more findable, more useful, and more present in people’s lives.

Then, for those of you who want to ship intelligent experiences in your app, you’re ready to build features that are powered by language models. The Foundation Models framework gives you a single native Swift API to choose the right model for the right moment. You can use the on device Apple Foundation Model for tasks that need to be fast, private, and available offline.

And many developers in the App Store Small Business program can use the Apple Foundation Model on Private Cloud Compute for deeper reasoning. This is frontier-level Intelligence with unparalleled privacy protections, all at no cloud API cost. Or you can connect to models from companies like Anthropic and Google and more for specialized use cases. The Foundation Models framework enables language understanding, visual reasoning, and agentic workflows directly in the experiences that you ship. This is the best way to build intelligent experiences with any language model, which gives you the flexibility you need to swap the right one in as your needs evolve.

Then, for those of you with custom models, sometimes you might have a specific model in mind that’s trained for your domain or optimized for your use case, and you want it to run entirely on-device, with no external dependencies and no compromises. Core AI is built for exactly that.

It’s a brand new framework, integrated into our platforms, and is designed to be the best way to bring and run models on-device in your apps. It delivers uncompromising performance through a modern Swift API with extensive tuning capabilities, taking full advantage of Apple Silicon. No cloud calls. No third party services. Your models will perform efficiently across all devices, and because Core AI is built right into the platform, your apps automatically benefit from the latest fixes and enhancements with no extra work from you.

Then, if you’re an AI enthusiast, MLX is an open source framework that lets you experiment with, train, research, or fine tune generative models, or you can run a local inference server. It’s a flexible, high performance framework that scales from numerical experiments to running the largest frontier models. And with distributed training and inference, you can build a local compute cluster that delivers the bandwidth and latency that serious ML workloads demand. This is AI for exploration and scaling. Open source, optimized and built for wherever your curiosity takes you.

What makes Apple’s AI platform distinct isn’t any single piece, it’s that every layer reinforces the others. The silicon that powers Core AI and MLX is the same silicon that makes the on-device Apple Foundation Model practical. The privacy architecture protecting Apple Foundation Models on-device and on Private Cloud Compute is the same one protecting App Intents. The Swift APIs that surface intelligence in your apps are the same ones that Xcode agents use to help you build it.

This isn’t a collection of features. It’s a coherent system designed from the ground up so that wherever you enter, you’re working with something that makes AI faster, safer, and more capable at every level. So whether you come to this as a builder, an integrator, a product engineer or a researcher, there is a place for you here.

The only question is where do you want to start? Today we’re going to help you answer that by going into each of these areas in depth. starting off with Xcode 27 and the new agentic workflows that will change how you build. Here’s Ken and Jerome! [applause] Hey, good morning!

Hey, I’m Ken. Hi, I’m Jerome. And today we’re going to show you what agentic coding looks like in Xcode 27. Now, for decades, writing software meant telling a computer exactly what to do. One keystroke at a time. But we built tools to help, like autocomplete, predictive code completion, and more recently LLMs that can edit single files. Now each one of those shifted what a computer could do.

What’s happening right now with agentic coding that is a much bigger shift. From a single prompt, Xcode 27 works across your entire codebase. It plans. It applies changes across files. It uses tools like previews, playgrounds, device integration, and more to bring your ideas to life. You’re no longer limited by how fast you can type or how much of an API you can remember. So today we’re going to build something fun, live, on stage together. But first, Ken will give you a quick tour of Xcode 27. All right!

Here is Xcode 27, which takes the next big step in agentic coding. And you work with agents through conversations. You can start one right up here in the toolbar. and each conversation now gets the full main editor area. And right next to each conversation you get a dedicated space for everything that Xcode produces. So plans, files, previews all together in one spot.

And you can have as many conversations going as you want. It makes it really easy to learn and experiment with new ideas. Okay, Jerome, let’s build something. All right! WWDC it’s such an incredible moment each year. You get to learn about new technologies, meet developers from around the world, and get the pins.

[laughter] Every year. [applause] Yeah. Every year there’s a new design. People trade them, people collect them, people put them on their jacket. You know, I got a lot of those. And I always wanted to build an app to track my collection. Well, luckily for me, my colleague Summer, she gave me all the pins artwork. I put them in a folder right here, organized by year. And also have a subfolder “collected” for the pins I already have.

I also drew a sketch of what I have in mind. It’s a grid of pins organized by year. The ones I have are in full color with a checkmark, and the ones I don’t have are grayed out. When I tap the pins, it opens a brand new view.

The ones I have, I see in full color, and the ones that I don’t have yet, I can mark them as “collected.” So I’m going to use that sketch as, the beginning of our app here. I’m going to copy it. I’m going to start Xcode. Start a new conversation.

Add the sketch and ask for what I want. The “Pins” directory in my project folder contains my collection of pin images. They are grouped by year, and each year has a “Collected” subdirectory, matching the current state of my collection. And I like to create an app that can be used to track my collection based on the attached design.

And now Xcode jumps into plan mode. Now plan mode is one of my favorite features of Xcode 27. When working with coding agents, the best result comes from collaborating on the implementation and design first. Right now, Xcode is parsing the prompt and exploring the pins folder pulling out anything that needs clarification before it starts writing any code.

Oh, now we have a few questions I need to answer, and we built this great Q&A experience directly in Xcode, so you can stay in the flow of your conversations. All right, let’s see what we have. First questions. How should we handle the pin images? I’m going to use an Asset Catalog for that. This sounds like a good idea.

How should we persist which pins are collected? I think I could use user default. It’s a small app. I can always implement SwiftData a little bit later. And… Should user be able to uncollect a pin? Why would I want that! I have no plan to give away my collection so I’m going to say no to that one. All right. And I’m going to send it.

Now Xcode is working with the agent to build a plan using the answers I just gave. Think of this as Xcode and the agent doing a white-boarding session. And this is just exploration phase, no code is being written yet. So I can always add more details, change my mind on the answers I just gave, and more. Let’s wait for the plan to finish.

And here it is. We have a plan to create our app! Rendering in markdown. Fully render markdown, right next to the conversation! All right let’s have a look. So we’re going to build a SwiftUI app Perfect. It’s using my sketch as a starting point. That’s great. I’m going to have filter at the top all “Collected” and “Missing.”

I love that. It’s going to use pin images from the Pins folder right next to our project. And look, you know, it’s adding all the decisions we make, with the questions before. That sounds like a good plan. Let’s kick it off. And it’s going to take a few minutes.

All right. While Jerome keeps an eye on that, let me tell you about what Xcode does behind the scenes when you’re working with an agent. Xcode leverages the full power of the best models and agents in Xcode so you can get the best possible results. And we worked with Anthropic, OpenAI, Google to bring their agents right into Xcode, so you can choose the model that works best for you. All right.

[applause] And with the ACPs so that’s the Agent Client Protocol you can bring in any agent, any model, Including ones that run locally right on your Mac. Now when you type a prompt, Xcode adds a lot of details and it gives agents the ability to use different tools. So that you get the best possible results. Now, the first thing Xcode does it layers in everything the model needs to know about your project. So how it’s configured. How it’s organized. Your build settings, and more.

Then Xcode adds in context about itself. So what tabs are open. Current selection. Your most visited files. So when you say something like “fix this”, the word “this” well it actually means something. And then finally Xcode gives agents a set of tools like building and testing, rendering previews with variants, interacting with the simulator localizing your app, debugging and more.

Now, for example, when you’re using a specific API, the agent can use the documentation tool. We created a version of the documentation that’s optimized for model use with vector embeddings for fast access. Yeah. [applause] Works really, really well. And that gives the models more context when there using Apple APIs and frameworks.

Some of these tools are things that you could type into Terminal yourself like copy, move or delete. They need to be used carefully. So Xcode implements a new security approach that leverages new features in macOS 27. There’s a new setting in Xcode that enables file access at the kernel level.

So when a sub-process like an agent wants to read or write a file, the kernel asks Xcode first. Xcode can apply policies to those requests. And it can trust your project and all of its files. And the agent can run, but it can only touch what Xcode allows it to. And when something does need your approval, Xcode asks. The work keeps moving you always stay in control.

So you’ve always got the control that you need. All right, Jerome, how are we looking over there? You know, Ken, we’ve been done for a while. But I really didn’t want to interrupt you. That was super interesting. I do appreciate- I appreciate that, Jerome. Thank you. All right. Our app is ready. Let’s just run it.

And here it is! Right. [applause] Pretty good already. It’s already seeded with my collection the pins I have in full color, and the ones I unfortunately don’t have yet are still greyed out. I can tap on a pin. I see it in full color. Now, this looks really great.

123 pins, zero set up, one prompt, and we already have a pretty good app, right? And that’s really what agentic coding unlocks from an idea to a working app. You describe what you want, you make the decisions, and the agent does the heavy lifting. But it doesn’t stop here. Once you have that solid foundation, you can keep going, keep exploring, keep creating.

Now I have really fun ideas to make our app even better! I think that, when I collect a pin it should burst into color. It should scale up, It should spin. It should- Hey, Jerome? I’ve got an idea. Do you think we could add some sparkles? Ken- I love sparkles. It’s WWDC. Of course we have sparkles, right? I love it. And when I touch a pin I want to see a really cool holographic 3D effect. Right? So let’s just build that.

We’re going to add an animation when marking a pin as collected. I want the pin to gain a color, scale and spin rapidly. Also, add sparkles. And add a holographic 3D effect when dragging a pin on screen. Actually I have a second idea. I’m opening a new conversation here and this is what I will ask.

I want to be able to see the pins I collected on a corkboard. That’s how I used to collect pins when I was a kid. So we’ll just do that. And right now, this is what I ask. I want to see my current collection on a realistic corkboard background. The pins should be scattered randomly without overlapping. That would actually make a great wallpaper.

And now both conversations are working in parallel. Here in the Navigator, I can track the progress of each of them. And if an agent requires my attention for example to allow specific tools to be used, I get a status message right here. I can switch back and forth to track the progress of each.

And I use this a lot to read documentation about new APIs, or get some explanation on the code that’s just been generated. I think it’s a great way to build and learn at the same time. All right. Apparently the corkboard is building… already done. And, the animation, one is building, too. I’m excited to see the result of that. All right. Looks like we’re ready. Let’s just run it.

All right. There’s a new icon right here. I’m going to click on it. And here it is. [applause] That looks really great! Right? Lots of great memories right here. And then I’m going to tap a pin like, you know, the new Dogcow from this year. Mark it as collected, and here they are. Nice- the sparkles. Love it. [applause] Now watch this When I drag the pin around, I get this really cool 3D holographic effect on the pin. How cool is that.

[applause] So agentic coding together with Apple platforms, frameworks and tools, helps bring ideas to life. And you can build those ideas for yourself, for your friends, your communities. Agents in Xcode can help with all kinds of things like adopting new APIs, making your app more accessible, and more.

For example, when using an app, there is nothing that makes it feel more personal than when it’s localized in your own language. Let’s talk about how agentic coding can help you with that. Localizing an app is way more than just translating strings word for word. It’s about understanding how each string is used in the context of your app. For example, the same word can mean many different things depending on how it’s used.

And when you ask the agent in Xcode to localize your app, there’s three stages that it goes through to ensure maximum accuracy. So first, Xcode adds the language to your project settings. Then it builds your project so that it can discover all the localizable strings. It creates a strings catalog if you didn’t already have one, and then it adds slots for each and every string.

Then the agent does the actual translation. And to get that done as fast as possible it creates subagents to do that translation in parallel. Xcode provides a lot of context about the strings themselves. Things like the surrounding code or the UI that they appear in. And some strings can even vary by plural or device. So Xcode makes that info available to the agents as well, so you get the correct plural and device variations. Once the translation is complete, the primary agent goes back, checks everything that the subagents did.

And then pulls it all back into Xcode where you can review and refine all of those strings. You know, Ken I would love to localize the app for all the developers in the audience but we have developers who have traveled from 65 different countries, right? It would probably take a while.

So I’ll do that later. But for today, how about I add a feature that let’s me collect pins from a picture. Oh, I love that idea. That’s a great idea. All right. I’m going to do that. So here’s what I simply ask. Allow me to collect a new pin by taking a photo.

All right. While we let that run, let me tell you about how you can extend Xcode’s capabilities. Now, everything we’ve shown you so far is built right into Xcode. But you can also bring your own skills, your own tools, and your own agents through plugins. Now, that’s a format created by Anthropic. Already widely adopted by the community. Now, a plugin can contain skills which are just simple markdown files and those capture how to do a particular task. It can contain MCP tools, for connecting to anything from a database to a design system.

And then we added something really cool on top of that With the Agent Client Protocol, a plugin can also bring in the agent of your choice. Installing one? Easy. Paste a GitHub URL Xcode clones it, installs it, manages it. And partners like Figma and GitHub already have shipped their own plugins. One click to set it up, super simple.

And like I mentioned before, you are always in control. You decide what each plugin can see, everything else needs your approval. Okay. How are we looking over there, Jerome? It’s still building. And, you know, it was a simple prompt, but actually, there’s a lot of things behind it, right?

Think of this as, we’re going to create an experience with the camera, right? And then we’re going to have a vision model that will compare the image we’re taking with all the images from my assets catalog. Right. And then we’re going to create the entire experience around it. In the app we already built. So that single prompt is actually a very complex features that leverages lots of technology and platforms.

So let’s wait for it to finish. Now it’s updating the info pin list to use the camera. That’s perfect. And now it’s building the project. All right. Build succeeded. I’m excited. We’re going to run that on a real device, so we can use the camera. So I’m going to select my phone here, and just build it. All right. I’m going to lunch it on my phone. Perfect. And we’re going to tap the camera icon. And here I have this Finder pin from 2025. I love this design. I’m going to snap a picture of it. Here’s a picture. And here it is. Recognized immediately.

[applause] I think I’m going to keep that one. I’m going to add it to my collection. And here they are. The sparkles. Love it. There’s so many more ideas I have for this app, but unfortunately that’s all the time we have here today. Hey, Jerome, before we leave today, I do have a little surprise for you. Oh.

I have… a never… before seen special edition WWDC26 pin for your collection! I think I still have room for it on my jacket right here. Okay. That’s pretty good That looks amazing. Is anybody else interested in one? [cheers] I thought so. Great news, we’ve got one for everyone here. [applause] All right.

Well, 20 minutes ago, we started with a folder of images, and an idea. Now we have a real app. Pin collection by year. Tap one, colors, drag it, holographic. Snap a picture of a new pin, it gets recognized. All built live, in the time that it took to explain this. And best of all, it even included a few sparkles Next, here’s Michael to talk about Siri and App Intents!

[applause] Welcome, everyone, and thanks so much for coming out today! My name is Michael Gorbach and I run the App Intents “core team” here at Apple. Today, I’m super excited to show you how you can bring your app’s content and actions to Siri. This year, Siri is taking a huge step forward. It’s more capable, more contextual, and more personal than ever before. Siri can now take action inside your app, driven entirely by natural language. It’s got a rich understanding of people’s personal context, allowing it to search and reason over your app’s content in a safe and private way.

And Siri’s aware of what’s on screen when you interact with it, so people naturally can refer to what’s right in front of them. You can make your app a part of all of this by adopting APIs from the App Intents framework. That’s the foundation for integrating your app with Siri and Apple Intelligence. It provides a structured way to describe what your app can do and the content within it.

By adopting these APIs, you’re really modeling your app’s data and capabilities so they become first-class citizens in the system. Once your app’s plugged in, Siri can handle the heavy lifting finding content, taking action with natural language, and understanding context to let people do amazing things with your software and their devices. Let’s start with a demo.

I’ve been working on an app called HotTickety that handles alarms, timers, and stopwatches. I do like to eat, but sometimes I get caught up in things and kind of need a reminder. Let me show you how I can interact with HotTickety through natural language with Siri. If I open the Siri app, let’s start a new conversation. I can say something like “2 p.m. alarm for panini lunch in HotTickety.

And Siri would do that for me. Notice that Siri is able to take action on my behalf within this app. If I tap on that alarm, Siri will open it for me in HotTickety so I can see the details. Cool. If I go back to Siri we can try something like, “Let me know when 30 seconds are up.”

And Siri will start a timer in HotTickety for me. Notice it’s running up there at the top. Also notice I’m just using very natural phrasing. HotTickety didn’t have to do anything special to handle all the myriad different ways someone could want to start a timer. And if I type something like, “Cancel it.” Siri will go ahead and do that for me as well. Siri has context on the conversation and remembers what just happened. Let’s see if the can find some information from HotTickety for me. I can ask it something like, “Check my alarms in HotTickety.”

And it will search for them and find all of my alarms. One last thing. If I go back to the app, I’m looking at the schedule and I think I’m a little bit still too full from breakfast. So I’m going to put off lunch. “Delete the fourth one.” Siri knows what’s on screen, will confirm and delete it for me. And that is how I use HotTickety with natural language through Siri.

[applause] So today I’m going to walk you through how all the features you just saw break down into straightforward adoptions of powerful APIs that you can try in your app. We’ll start by talking about entities, intents, and schemas which are the basic building blocks that you use when you adopt App Intents to build a Siri experience. Then we’re going to cover how you can use entity annotations to augment Siri’s on screen awareness. Like you just saw. And finally, we’ll talk about interaction donations, and how they allow Siri to learn from user behavior.

Let’s start with entity schemas. Say you’re cooking dinner, and you ask Siri to start a 12 minute pasta timer. The noun “timer”, as I use it here, does have a direct representation in the App Intents framework. It is an app entity. When you adopt App Intents, your app will have its own key types of content that will become your entities. An event if you’ve got a calendar app. A mail message for your mail app, Or an album or photo if you work on a photos app.

Entities have an identifier that the system knows them by, plus a set of properties. For example, HotTickety’s timer entity includes a duration, and a label. It’s also got an enum that describe its state, showing that it’s currently running. In code, there easy to create. It’s just a Swift struct, and each property is marked with @Property.

Okay, now that we have an entity, let’s talk about how Siri can work with it. That’s where App Schemas come in. App Schemas give Siri an understanding of common concepts, like messages, albums, events, or timers. Groups of schemas are organized into domains like this clock domain, which covers alarms, timers, and stopwatches. Super useful for HotTickety and also great for your app if it includes similar features.

HotTickety’s timer entity adopts the timer AppEntity schema, which lets Siri recognize it as a timer, and defines the properties that it needs to have. Apple’s Clock app does too, and so can your app, if you have a timer feature. When I add the schema at the top of that entity struct, Xcode helps me get the properties just right. I can even delete the ‘@Property’ wrappers because the schema is taking care of them for me.

Okay, let’s try another phrase. Responding to this one means Siri needs to figure out the right timer entity to answer this question. The good news is that when receiving a request like this Siri actually already knows what’s on your device. Because Apple Intelligence has a datastore for entities: the semantic index. Siri can search not just based on keywords, but based on the meanings of words, or the values of specific properties. And your entities can be included in it too.

Adding HotTickety’s timers to the semantic index is simple. I start by adopting the ‘IndexedEntity’ protocol. And then I call ‘indexAppEntities’ whenever my timers are created, renamed, or deleted, so Siri can know about them and be ready to take action. This works super well for an app like HotTickety where all the data is local. For apps that have huge catalogs of data, or data stored on servers, Siri can also retrieve entities from them directly, through queries. You can learn more about those in our sessions and documentation.

Schemas provide a common structure for entities representing the same kind of content. The cool thing here is that since HotTickety’s timer entry conforms to the schema, each timers are first class citizens in the semantic index. They’re found and shown together with timers from the Clock app, or other apps on the system. Now that we’ve created some entities and we can find them, it’s time to take action.

Looks like I made a different brand of spaghetti this time. The verb “change” here also has a direct representation in the App Intents APIs. It is an app intent. Intents are the functions that your app performs. They may open your app when they run or work completely in the background.

Here’s one that updates a timer. Intent to take specific inputs like in this case, the timer to change, the new duration, and the label. They produce a specific output, like the newly updated timer entity. In code, an intent is just another ‘struct’. The inputs are declared with ‘@Parameter’ and you can see the output type shown in the return value of the perform method that’s doing the update.

Like it does for entities, the App Intents framework also defines schemas for every intent that Siri can understand, including what the expected inputs and outputs are. The update timer intent schema specifies the parameters like the duration, the label, and the timer to change as well as the result: In this case, an entity conforming to that timer schema. HotTickety adopts this schema, and again, so does Apple’s Clock app. Doing this makes their intents available to Siri.

The adoption is again one line of code at the top of our intent struct. And I can skip writing the ‘@Parameter’, because the schema is handling it for me. A couple more super cool things about schemas. First, they automatically configure your Spotlight keys for you so the semantic index can search over granular information about your entities with minimal adoption from you.

Second, they allow Siri to automatically handle lots of privacy and safety features for you, like confirming actions with people when it’s appropriate, to do so, and intelligently handling authentication and redaction for locked devices. If you adopt schemas, your app will also automatically benefit from future Siri updates, as Siri’s understanding improves to handle new languages or regional dialects.

Okay, now let’s talk Entity Annotations. Everything you say to Siri is interpreted in context. This context is another area where the App Intents platform, and your adoption of it, can really help empower Siri and the people that use it. The App Intents framework has a rich set of entity annotation APIs that you can use to augment Siri’s understanding of what’s on screen. Let me show you an example.

What if you say something like this while looking at HotTickety in the foreground. Handling this one requires Siri to understand what’s on screen on your device right now, and be able to take action on it. App Intents includes simple APIs to let you tag your app’s, views, or user activities with the specific entities that correspond to them. So Siri knows exactly what’s going on, right at the point where people make a request.

It’s really easy to add annotations to your views. In HotTickety, I added a single SwiftUI modifier to specify the ID of the associated entity for each of my alarm rows. And there are other flavors of this API to handle collections of views with a selection or custom drawn canvases, and even support for UIKit and AppKit.

But Siri’s understanding goes a bit beyond the views in your app’s UI. There’s a whole set of APIs that allow you to add entities to your adoption of core system functionality, like notifications, now playing sessions, and AlarmKit alarms. Adopting these can let Siri read your app’s messages to folks using AirPods, or take action on an audio track someone’s listening to.

In HotTickety, I added an entity annotation to my AlarmKit alarms, in just a single line of code. And that means Siri can cancel a firing timer, or snooze an alarm, even if my app isn’t visible onscreen. Lastly, let’s cover how Siri is more intelligent and more delightful because it can learn from previous interactions, and how you can contribute through Interaction Donations.

You might have noticed that in my demo, I sometimes wrote out the name of the app: HotTickety and sometimes I didn’t. People often use language that doesn’t specify the app like that. In fact, people often use language that doesn’t specify a lot of things. If you ask to message your work colleague, you might not even specify that you want to do that on Slack. It’s just implied. You expect Siri to figure it out.

Siri can automatically learn from previous interactions a person has had with it. Each time you say “Start a Timer in HotTickety” Siri saves a receipt of that action into a temporary transcript. Eventually, if you use HotTickety with Siri enough like I did during my development of it, it’ll start to default to that app automatically because it starts to know that’s the one you prefer.

But Siri and Apple Intelligence do need your help to see the interactions that happen within your apps, so they can learn from those too. You can teach Siri how people actually use your apps to make Siri even more personal for them. To do so, App Intents has a set of “Interaction Donation” APIs that let you add to this transcript based on what actually happened in your app’s UI. Siri can use these donations to do a better job routing user requests to just the right app based on the parameters, like always contacting that work colleague over Slack.

Interaction donations also help Siri improve its knowledge of what entities are relevant on the device. If I’ve started a stopwatch in HotTickety with my voice, Siri can understand there’s a stopwatch now running in the app. And when I go to HotTickety and tap, to reset that stopwatch… HotTickety donates that interaction and reading it, Siri can know the stopwatch has stopped running.

Donating an interaction is as simple as creating the app intent equivalent of what the user did in-app. In this case resetting the stopwatch and then calling ‘IntentDonationManager’’s donate method with the intent. OK, let’s pull all of this together. The way to make your apps work with Siri is through the App Intents framework which powers a lot more than just intents!

With all these APIs we covered today, you can model your content with entities, and tap into a whole set of powerful capabilities like entity annotations, interactions donations, and search. Getting started with App Intents is actually pretty easy! Schemas can let you do a lot less work. And you can quickly start getting functioning features, even before you’ve adopted everything.

Now, I only had a little bit of time today, and there’s so much more to this platform that I could not get a chance to cover. For example, there’s a Transferable API that lets Siri translate entities into the different formats needed for cross-app actions. There are all kinds of ways to customize dialog and visuals to show off your app’s unique style, and the ability to mark your entities as syncable with the Cloud so conversations just work across all of a person’s devices. There’s even a super cool new framework called ‘AppIntentsTesting’ to help you get all of your adoption working, and keep it working. We have a ton of talks, documentation, and sample code this year that you can check out to learn about all of this.

Every year you folks blow our minds by building things with App Intents that we couldn’t even imagine. That is what gets us up and out of bed every morning. This year is a big one, and we are setting our alarms for release day to check out all the great apps you’re going to build. And now I’ll hand you over to Louis.

[applause] Thank you Michael. Hi everyone I’m Louis. Now Michael showed you how you can bring your app’s, content and actions to Siri, and make your app work seamlessly with Apple Intelligence. Now I’ll show you how to integrate AI into your apps. So let’s talk about the Foundation Models framework. It gives you direct access to a large language model built right into the system all through a powerful native Swift API.

At its heart, the framework is very straightforward. Your app sends a prompt to the large language model. The LLM then reasons about your prompt and generates a response. For example, you can ask it to create a three day itinerary for visiting Paris, and the model will respond with a natural language plan, very detailed.

But Foundation Models goes far beyond basic text generation. It also lets you control exactly how the model behaves. For instance, using Guided Generation, you can guarantee perfectly structured output, simply by providing a Swift struct. And You can also use Tool Calling to extend the model’s capabilities, allowing it to retrieve real-time or dynamic data from your apps.

And you can guide the model’s tone from custom Instructions and it can provide real-time UI updates by Streaming the model’s output. There is so much more. When you put these pieces together, the possibilities become endless. You can build features that analyze complex data, synthesize information into concise summaries, or create entirely new conversational interfaces.

But to build the next generation of truly agentic experiences, you need more scale, more context, and more control. So today, we are bringing five big updates to the Foundation Models framework. We’ll cover Image Input, scaling up with Private Cloud Compute, integrating third party models, new APIs for letting you build agentic experiences in your apps, and testing it all with our new Evaluations framework. So let’s start with my favorite new feature, Image Input.

You can now use both text and images in your prompts. And this opens up so many possibilities. Your app can now be multimodal. If you combine that with the Generable macro, you can easily ask the LLM to get structured data from an image. For tasks like this, it used to be that you had to train a custom ML model… but now this is all possible with the built in model. And yes, I can tell you, he really is a good boy.

[laughter] So that’s Image Input. Now let’s talk about Private Cloud Compute. The on-device model we were just looking at is incredibly useful for so many tasks, especially with the image input that we’re adding this year. But we know that sometimes you might want a larger model for deeper reasoning and more complex tasks. So this year we’re also giving you access to a new server model, which runs on Private Cloud Compute.

Now, developers in the App Store Small Business Program with less than 2 million downloads can get access to this new server model. And just like with the on-device model, access to Private Cloud Compute is integrated in the OS. So there’s no account setup, no authentication, and no API keys!

And even better, there are no token cost to you, the developer. There is a daily usage limit per user. So this lets you scale your number of users, without worrying about the total cost to you, the developer. And users can upgrade to iCloud+ to get even higher limits if they want.

So let’s look at the code. With the on-device LLM, it takes just a few lines of code to get started. Create a session and give it a prompt. And now with just one line of code, you can switch to the new server model running on Private Cloud Compute. It’s that easy!

[applause] One line of code. Now of course Private Cloud Compute is private. It’s architected in a way that guarantees that user data is never stored. Your data is only used for your requests. And it comes with a verifiable privacy promise. So that’s Private Cloud Compute. Now let’s talk about 3rd-party models. There are a lot of LLMs out there. Maybe even some new ones by the time we’re done here.

And now any LLM provider can create a Swift package that conforms to the new LanguageModel protocol. And that lets you run virtually any LLM with the Foundation Models framework. And this year, Apple’s on-device AI inference is getting a big upgrade with the new Core AI framework and upgraded MLX. And both of these now have an open source package, letting you run a variety of on device models, all with our unified Swift API in the Foundation Models framework. And here is Matt to show you this in action.

[applause] Thanks, Louis. Hi everyone. I’d like to start by looking at some of the APIs we’ve all come to love and write since last year. So I have a Playground open here where I’ve created a instance of the SystemLanguageModel. I then pass that model to a LanguageModelSession, and finally, I prompt the model via the session. In this case, my prompt is tell me about the history of origami.

And on the right I can see the model’s response. So tell me about the origin of origami and its cultural significance. Now we LOVE the SystemLanguageModel and seeing the capabilities it’s added to your apps. But wouldn’t it be great if I could use other general-purpose models like Qwen?

So I actually have a version of Qwen here in my project. and I’ve also added the Core AI models package that has an implementation that conforms to the LanguageModel protocol. This allows for the Foundation Models framework to reach for models that are ready for Core AI. We’ll talk more about Core AI later. But for now, let’s see what it takes to bring this model into my project.

All I have to do is add the import for the package. Change out the SystemLanguageModel for the Core AI language model, And have it target my Qwen model. And that’s it. I don’t have to change any other code and I don’t have to change my prompt That’s the power of the LanguageModel protocol.

It lets you plug in any model you’d like directly into the Foundation Models framework. And on the right I can see the new model’s response. It’s that easy. We are so excited about the LanguageModel protocol and what it allows you to do. We can’t wait to see the models you’ll bring to the FoundationModels framework this year. Thanks. Back to you, Louis.

[applause] Awesome, thank you, Matt! And beyond on-device models, as we announced yesterday at the Platforms State of the Union, Google and Anthropic are also extending the Foundation Models framework with Swift packages of their own, making state-of-the-art Gemini and Claude models available to all Swift developers. Now, with all these models just a few lines of code away, the AI experiences you can build are more capable than ever.

And to enable you to build agentic experiences, we’re also giving you new APIs! So imagine you’re building a travel app. You could use the on-device model as an offline planner, especially useful when you don’t have internet. Like in an airport. And in the same app, you might want to use a server model, like the one on PCC, for more complex tasks like rebooking hotels.

With agentic features using multiple models, you typically want to share that context in some way. Because here’s Session B with that server model can’t see what time my flight is. Now sharing that context is typically complicated. And hard to manage- like my two kids at home. [laughter] and I wish I was kidding.

So this year we’re introducing new APIs, letting you manage multiple sessions with shared context. Now, If you had to write this would imperative code, you have to write all this boilerplate! And nobody wants that. But with our new DynamicProfile API, you get a high level Swift builder syntax to easily use models together. And this reduces the overhead of managing state, and lets you focus on the specific details of your features.

And we know this space moves fast. So to give you even more APIs, we’re also releasing an open source utilities package! This contains custom skills. To let a model dynamically discover tools in your app. And we have other utilities like context management. And you can clone this package from Github today.

And to help you profile and debug, we’ve updated Instruments this year. You can now see everything the models see at runtime. Each prompt, tool call model output can be inspected in detail. But what if a model isn’t always doing what you want it to do? We’ve all been there, right? Well, that brings us to evaluating your apps. We’re also making that easy for you this year. We’ve created a new Swift framework, that’s integrated right in Xcode. And here’s Rob to tell you all about that.

[applause] Thanks, Louis. Hello, everyone. If you’ve spent any time building features with Language Models, you already know the biggest challenge: their output can be unpredictable. Traditional code does exactly what you tell it to do. Language models not so much. Let me show you what I mean. I’m building a book tracker app that uses a model. And I wrote a simple prompt asking the model to act as a librarian and generate descriptive tags based on the users book review.

When I test it myself, it looks great. It added tags to this book and it handled a few other books with no issues. But then I shared the app with a friend. They wrote a review for Dracula, and the model tagged it as I mean, cuddly? Family friendly. I don’t know what that is about kittens. [laughter] Clearly these are problems.

And these are the kind of problems that Evaluations can solve for you. The Evaluations framework provides a flexible system to measure the quality of your model output. You can define metrics, automatically grade outputs using language models, and analyze statistical trends across large data sets. And you can tailor it to your app so you ensure your intelligent features perform reliably.

The framework lets you scale your evaluation over time, and across a variety of samples. If you need help creating more samples the Evaluations framework has your back too. It can synthesize samples for you using language models. Evaluations are easy to run, integrating directly into your test targets. We have a new trait for Swift Testing, called ‘evaluates’ that will automatically set up your evaluation harness, run it, and provide the results inside of your test. And our metric aggregation makes it easy to assert against trends in your evaluation.

We’ve added a new Test report in Xcode 27 for evaluation results. It’s a great way to dive into the details and analyze your results further. Let’s look at this report in more detail. This report lets you analyze every metric in your evaluation. Each metric is measured by an evaluator for each data sample. Let’s consider the “tag count” metric.

And this is the code for the evaluator. It’s a simple swift comparison of the tags array returned from the model for a given sample. Evaluators can be any function of the model response and sample passed to the model. Here I return a passing measurement if the tag count is within my desired range. But Metrics don’t have to be pass fail. They could also return a Double value, using the scoring measurement. Here I return the raw number of tags generated for each sample.

I can combine both metrics to determine if the model created the right number of tags, and observe the distribution of tags. This aggregation looks across all data samples in the evaluation. You can use aggregate metrics in your tests to assert against statistical patterns in your evaluation results. These aggregates are particularly useful when evaluating a large data set.

But not all metrics are calculated in code. Oftentimes, you want an assessment, using a qualitative scale. Instead of a hard-coded measurement, I use a ScoreDimension here to define one or more scoring scales for a model judge. Model judges are powerful evaluators, which use language models to approximate human judgement. This model judge determines if the generated tags are relevant to the book, describing the book instead of repeating personal reactions from my review.

My model judge uses to ScoreDimensions, relevance and usefulness, to describe the quality of the generated tags. When aggregated, this gives me a top-level understanding of my feature’s quality. This helps me understand how well the model performs at tasks my app needs. And with this knowledge, I can engineer my system instructions and prompts to maximize performance.

I’ve been using my evaluation for a few days to help me refine the tagging service so it consistently behaves as I expect. You can see the effect of these iterations on the instructions passed through the LanguageModelSession inside of my tagging feature. If you remember, this is what I started with in the beginning. And here’s my instructions now, after several evaluation runs and tweaks. Here you can see each expectation of the system I verified next to the instructions I added to the system based on evaluating those expectations.

This process of iterative prompt engineering, based on evaluation results is called “hill-climbing”. And when you integrate a hill-climbing feedback loop into your app’s development process, we call it Evaluation-driven Development. Developing with evaluations helps you determine when you’re ready to ship. It helps you maintain quality over every future release. And that’s the Evaluations framework. Now back to Louis.

[applause] Amazing, right. Thank you, Rob. So the Foundation Models framework now lets you plug in a variety of models. And as you just saw the evaluations framework lets you measure your features, no matter which model you’re using. So putting this all together this year we’re giving you all the building blocks to create great AI features! But we didn’t stop there. The Foundation Models framework is going to open source later this summer.

[applause] So that means you’ll be able to use it on other platforms, like on Linux Swift on Server. And that will make the Foundation Models framework THE Swift API for any LLM on any platform. Now the team got so excited, that we did even more. So new in macOS 27, there is the ‘fm’ command line tool, which lets you easily access both on-device and the new server model on PCC. Now this is so great for prototyping and scripting. And there’s even more that we didn’t have time for. Like the CoreSpotlight RAG tool, We have Vision tools, dynamic instructions API, and a lot more. But that’s all the time we have right now. So now, back to Rachel.

[applause] Thanks, Louis. All right. How’s everyone doing? Great. Great. Okay. Are you excited to try some of this stuff out? Amazing. Okay. So far, you’ve seen how you could build faster with Xcode, how you can integrate your apps content and capabilities across the system, including through natural language in Siri And how you can ship intelligent features in your app through the Foundation Models framework.

Then finally, we get to test some of these features through the Evaluations framework. But we’re not done yet. Many of you might want to take your own models, or take custom generative models and run them on-device. For that, we have two more exciting technologies starting with Core AI. Here’s Raziel to tell you more.

[applause] Thank you, Rachel. Hi everyone, my name is Raziel. I lead Core AI. Today Geppy and I will give you an overview of the technology. But let’s start with some context and hopefully something that is not controversial at this point. Is that the world of AI is vast and growing. Just look at this list. There is a large amount of models and a large amount of potential applications for each one of them.

It’s things like image captioning. Voice cancellation. Gesture control. Background removal. So as the AI landscape becomes more complex and your ideas grow more specialized we want to make sure that the paths to bring them to production remain as smooth as possible. And that’s why we created Core AI. It marks the next evolution of on-device AI execution across Apple platforms.

It lets you bring cutting-edge AI into your apps. Enabling some truly magical experiences. And of course, this comes with the benefits of local execution: Your data stays private on-device. Your features are always available because there is no reliance on connectivity, and there is no inference cost to you, or your users.

Core AI is built from the ground up with modern workloads in mind. And it provides the fast inference that you need to build advanced AI features. And it does this by taking advantage of all of the Apple Silicon. Across the CPU, the GPU, and the Neural Engine. Now, Core AI is not just the framework that we build into the operating systems, but a comprehensive suite of technologies and tools that expand the full model deployment lifecycle. Starting with model creation, optimization, and inference fine tuning, all the way to app integration and debugging. We designed these to enable the fast iterations that you need to build great AI features.

Now, to give you a better sense of how all these pieces click together, let’s go a little bit deeper into four key areas of Core AI. Starting with a foundational one which is our OS Framework. Core AI delivers great performance through a clean, expressive, and modern Swift API. This API is designed to progressively disclose its capabilities. What this means is that we want to make it very easy for you to get things up and running, while still providing the layers of flexibility that are necessary to support performance critical applications.

And it’s also designed with modern Swift in mind. And this allows us to give capabilities like memory-safety without sacrificing performance. In its simplest form, the API is very straightforward. It always starts with an AI model. This AI model gets loaded into your app, into the device, and it’s specialized. The model contains one or more functions and each function receives inputs and produces output in the form of NDArray objects.

This is even simple to to understand when you look at the code. There is just one line to instantiate your model, another line to select the function that you want to load and run, and then another line to just simply run it. That’s it. We take care of all the complexity underneath to make this fast and efficient.

Now, the second area is a comprehensive suite of Python Tools, that enable preparing your models for a framework and runtime. These tools integrate seamlessly into familiar PyTorch workflows, providing rich APIs to prepare your model. What this means is that it’s very easy to go from a trained PyTorch model into a Core AI model that is optimized for efficient inference in Apple Silicon.

Let’s look at the code. It’s also very simple. We start first producing what we call an exported version of the PyTorch model. In order to do this, we just use standard PyTorch APIs. Then we use the Core AI Torch package. In this case, the TorchConverter, to produce a Core AI representation of the model. And finally, we take this representation of the Core AI model and we save it into an asset. This is the asset that you will load into your application and you will run on device.

Now we understand that each application needs are unique, so Core AI offers extensive Customization Capabilities. There is APIs to control different aspects across model creation, optimization, and execution. For example, on the author inside we just show you the very simple conversion workflow. But the Python package is much richer than that.

For example, it allows you to directly author your model using these APIs. We also provide a comprehensive suite of techniques and tools that allow you to optimize your model for Apple Silicon. And you can go even all the way and provide your own custom kernels to optimize your model according to your specific needs.

On the execution side, the Core AI framework allows you the control that you need to optimize your application as you want it. You can go as deep as you want. For example, you can control how cached management works, when to purchase resources, you can also share cached models across different app groups. You can also query the formats of inputs and outputs to device-specific layouts. So there is no there is no cost of copies or conversions and inference time.

Similarly, we allow you to pre-allocate the values of states and outputs and have functions write directly into them and this is critical for optimizing Large Language Models. And we have a very flexible asynchronous API. So you can create function pipelines and take advantage of parallel execution. Again you can go as deep as you want depending on your needs and the goals that you have for your particular application.

Finally, all this is tied together by our new Developer Toolchain. We’ve built deep integration in the into the tools that you use every day while still providing some cool new features. So let me show you a couple. First, with Core AI, now we are enabling you to pre-compile the models ahead of time producing a pre-compiled asset. When this asset gets loaded into the device, it will still be optimized, but this will be very fast. So this allows you to work with very large models. Again, very critical these days.

And to complete that seamless loop between modeling and development, We’re introducing the new Core AI Debugger application. This allows you to visualize computation graphs, inspect tensor values, and even trace those values from the Core AI model back into the original Python source code. Making everything very actionable. With these four pillars, Core AI now lets you deploy powerful AI to your users at scale. And talking about a different type of scalability, Core AI is engineered to seamlessly scale with the compute available on the device.

This means that you can run a highly responsive vision model on an iPhone for doing something like real time camera capture, inquiries, or push the limits with a multi-billion-parameter model running on a Mac Studio for doing some complex agentic workflow. Now, to give you a concrete example of what it’s possible, here is Geppy.

[applause] Thanks, Raziel. Hello, my name is Geppy Parziale and I’m a creative technologist here at Apple. Today I want to show you an iPad app I’ve built. It’s an ideation creative tool for working with images and generative AI. Let me show you. The workflow I’m going to show you is style re-targeting. You bring in an image, write a prompt, and generate multiple variations of the source image. So let me load an image. I’ll pick up this one. Let me add a prompt thatI have prepared.

So “Create variations from this image across different camera angles and lighting conditions.” I hit generate, and it starts to work. This is running fully on device. No cloud APIs, no external services. All the inference runs locally using Core AI. Now what’s interesting is what’s, what’s happening under the hood. Let me show you.

So first, I use a Visual Language Model that analyzed the image and combines it with a user prompt, and an internal art direction prompt to produce a creative brief. That brief feeds into FLUX, and image generation model, that generates multiple variations. I converted both FLUX and the VLM from PyTorch and wired them in with just a few lines of code.

Both models are run using Core AI completely on device. This ensures to preserve the privacy of my users, since the assets do not leave the device. In addition, since I want to fine tune FLUX, I need control of the full stack. Core AI gives me both privacy and control. And here is the result. Beautiful.

[applause] So multiple variations, fully on device. So what’s next, I wanted to scale this up to use larger, more capable models for higher resolution and more variations. What I did was simple. I took the iPad codebase, I built a macOS app, I’m showing you here, Swapping in larger variants of FLUX and VLM. Same code, different models. Core AI made that swap completely straightforward. Now, back to Raziel.

[applause] Thank you, Geppy. This is just an example of the flexibility and power of Core AI in action. And there is no better proof of this flexibility and power than how we’re using it ourselves. This is why now Core AI powers Apple Intelligence and the all new Siri. With Core AI, every developer can now tap into the very same inference framework that powers our most advanced AI features. One that is deeply optimized to unlock the full potential of Apple Silicon.

With Core AI, we’re giving you the tools to remove the friction from AI deployment, and we cannot wait to see what amazing new experiences you create. Now, this is not all that we have to tell you about AI frameworks. So next, let me hand off to Ronan, who’s going to tell you what’s new on MLX. Thanks.

[applause] Thanks, Raziel. And hello everyone. I’m, Ronan from the MLX team. And today, I’m going to talk about MLX, a Machine Learning and Numerical Computing framework designed for Apple Silicon. And with MLX, we are advancing modern science on Apple Silicon. The framework is built on four core pillars. First is Numerical Computing, at the foundation of modern science. MLX brings a flexible numerical computing library to the Mac for fast linear algebra and vectorized array operations.

Second is Automatic Differentiation, which handles gradient computation for you. This is often used for model training in machine learning. Thirdly, Distributed Computing lets you scale massive workloads across multiple Macs. And finally, Machine Learning. With MLX you can run powerful vision, speech, or language models right there on your Mac. And MLX is Apple Silicon first! It’s built to take advantage of the unified memory architecture. It’s GPU accelerated directly leveraging Metal, and it supports the latest advances in Apple Silicon, including the GPU Neural Accelerator from the M5.

With MLX, we bring the same numerical computing interface to several languages. First, in Python, as Python is the most common programming language of the Machine Learning community. Second is Swift, because Swift is the language of choice when it comes to Apple devices. And for people who are into low-level programming, we also have C and C++.

And from there, MLX brings also support for Machine Learning and AI, with several higher-level packages in particular, MLX makes it easy to train, fine tune, or simply run language models locally on your Mac. In fact, here is all the Python code it takes to use a state of the art model on your Mac. One line to fetch a model and one line to generate text according to a prompt. It is that simple. And beyond Python, we also provide a command line interface for ease of use and a native Swift frontend as well.

In fact the whole MLX stack is open-source. From low-level kernels to these high-level APIs. MLX is available on GitHub, with a thriving community and a lot of contributors. MLX has also a great presence on HuggingFace. HuggingFace is a good place to find the latest and greatest language models from the machine learning community. And there are more than 10,000 models from there supported out of the box by MLX LM.

Today, some of the most popular tools for using language models on your Mac are now powered by MLX. This includes Ollama, LM Studio, or vLLM, to name just a few. So If you have been using one of these tools on your Mac, chances are you have been running on MLX already.

But the community keeps building more. There are tons of open-source 3rd-party packages, available out there, like for training models from scratch, fine tuning large language models, support for speech, image or video generation, exploratory research work, or even Large Language Model servers, all that running locally on your Mac!

[applause] On that note, I’m going to hand it over to Angelos who is going to show you what you can do with MLX on the MacBook Pro M5 Max. [applause] Thank you, Ronan. Hello everybody. Today I’m going to show you a macOS application that allows you to load the book, a whole book and have a conversation about it with a language model. Okay, let’s load a book. I’m going to kick The Great Gatsby.

And off we go. Here we are, feeding the whole content of the book to the KV cache of a language model. And because MLX takes great advantage of the Neural Accelerator on this Mac, on this M5 Max actually, we are processing almost 2000 words per second which is actually impressive because we’ll end up needing only a little bit more than 30 seconds to process the whole book. Almost 200 page and almost 100,000 tokens. And after we processed it, our answers are going to be instant because we don’t need to process it again and again.

So let’s give it one more second. And here we go. So just to make sure that the model has indeed read the book, let’s ask a simple question. I’m going to ask it to “List the top then characters in the book”. That looks correct. And off we go. Indeed, indeed. Starting with, Jay Gatsby and, the narrator, Nick Carraway. And it looks like a pretty good list to me, but that’s a pretty simple question. Let’s ask you to make something more interesting. How about to make a cover image for the book?

So here, we’re actually running a 12 billion parameter diffusion model also on the Mac. And we- in mere seconds, we have a very nice cover very much based on the context of the book. [applause] Thank you very much. And with that, I’ll give it back to Ronan. [applause] Thanks Angelos. This was blazing fast! But how can we make the experience even faster?

Well, instead of using only one MacBook Pro M5 Max, let’s use two MacBook Pro M5 Max. And here we go. Angelos demo is now processing over 3000 words per second. So how did we make that happen? Well, we leveraged distributed computing. Distributed computing is about parallelizing operations over multiple Macs By doing so, you scale up not only compute, but also memory capacity and memory bandwidth. But distributed computing requires machines to be efficiently interconnected And that’s where Thunderbolt 5 comes into play. Last year we added Thunderbolt 5 RDMA support into MLX. Thunderbolt 5 enables high bandwidth and low latency communication, which are both critical for machine learning applications.

Beyond speed, another good reason to scale is to run even larger language models. To solve the hardest problems on your Mac, you may need the largest language model available out there! One of this language model is Kimi 2.6 It comes with 1 trillion parameters. It won’t fit on a single machine, but it will fit on several Macs. And here I’m going to hand it over to a very special guest. I’m going to invite Yagil from LM Studio on stage. He is going to demonstrate distributed inference on a cluster of Macs.

[applause] Thank you. Hi, everyone, I’m Yagil, founder of LM Studio, an app for running AI models locally on your Mac. Today I’m going to show you an upcoming feature in LM Studio, that leverages MLX Distributed to run much larger models on your local hardware. Over here I have four Mac Studios. They are all connected together with Thunderbolt 5 to form an ultra-low-latency communication network using RDMA-over-Thunderbolt. With this setup in LM Studio, I’m able to load and run one of the largest models available today Kimi K2.6 a frontier open weight model with 1 Trillion parameters.

So this is LM Studio and as you can see, I have Kimi loaded on the cluster, distributed across these four Macs over here. And I’m actually going to be using it remotely from this MacBook using a feature in LM Studio called LM Link. LM Link allows you to use your compute power remotely over an end-to-end encrypted connection.

So we have the model loaded 1 trillion parameters right here on these four Macs. And I’m going to demonstrate live using it. And we’ll see how it goes. So I’ll paste the prompt and let it start, let it start working. While I explain to you what it’s doing.

All right. So, I’ve taken The Great Gatsby and I’ve given it to the model so this is the entire book in the model’s context. And I asked it to read through the whole book. And when it’s ready to tell me that, it’s ready. And it did it. And, we’re using a giant model. It’s extremely capable. And when you’re using a capable model you can expect to give it a very specific prompt. You can expect to give it tools, and you want to see it following exactly what you ask it to do.

So in this example we’re giving it, The Great Gatsby. And I’ve, I’ve told the model that I’m a ninth grade schoolteacher and I’m asking it to help me generate a quiz for my students. And I’m very specific about what I want. I want so and so many questions, and so and so many questions and when it’s done, I would like it to be, an HTML page. And then I’ve also asked it to open the page when it’s done. It was obviously too quick. Here it is. So this is the quiz it was able to generate, here live on stage, running on this cluster. Pretty cool.

[applause] Cool. So this was done totally locally, right? So, the model is running locally. Nothing went to the cloud while it did it. Now, when you’re using a very capable model, you can also expect to give it follow up questions and follow up tasks like this one. So we asked it to make a quiz for the ninth grade.

But, maybe we want to make a quiz that’s suitable for university setting or something like that. So I’ve gone ahead, and, asked it to, to do something additional. But while it’s working, I’ll just explain to you a little bit how this works. So this is leveraging MLX Distributed.

And this allows you to run extremely, large models. And for some tasks you need frontier intelligence, But the really incredible thing about this thing is that, there’s no token cost. You can run this thing 24/7, and no data goes out of the cloud. So anything that I’ve showed here today, nothing went out, to the cloud.

And this is suitable for, using sensitive data, at work, at school or at home. MLX Distributed support in LM Studio is coming later this year, and you’ll even be able to use your compute power remotely from your other devices, using LM Link. Thank you. And back to Ronan.

[applause] Thanks Yagil. Totally inspiring, like a frontier model right at your fingertips. So, wrapping up with MLX we are advancing modern science on Apple Silicon. And it’s all open source. So whether you want to contribute, back, build on it, or simply experiment with the largest language models out there, well, come and check it out on MLXframework.org. Thank you. And now back to Rachel. [applause] Thanks, Ronan!

Across our technologies, no matter where you start from tools to frameworks to operating systems and hardware you’re working with something that makes AI faster, safer, and more capable at every level. And while we went deep in these areas today, there’s so much more to explore with over 100 sessions to dive into everything that we’ve covered today and all of our other announcements.

You can find them on the Apple Developer app and website, YouTube, and on a new bilibili channel. You can also join us online throughout the week in Group Labs hosted by Apple engineers and designers so drop in, bring your questions, or just follow along. Anyone can watch live or on demand. Apple experts are also on the Apple Developer Developer Forums all week long and beyond. And you can use the new AI powered search on the developer website, which brings together documentation, sample code, sessions, group labs, and more.

But right now we’d love to continue this moment, connecting with all of you throughout the afternoon at the Apple Developer Center across the street. You can join us for lunch from a variety of food trucks, and connect with Apple engineers You can explore the Developer Center and sit in on a live viewing of our Swift Group Lab. Or you could vibe with our Apple experts in lounges for Icon Composer, Xcode, and Reality Composer Pro. We hope to see all of you there! Have a great WWDC! [applause]