Foundation Models

RSS for tag

Discuss the Foundation Models framework which provides access to Apple’s on-device large language model that powers Apple Intelligence to help you perform intelligent tasks specific to your app.

Foundation Models Documentation

Posts under Foundation Models subtopic

Post

Replies

Boosts

Views

Activity

Is Small Business Program enrollment (incl. banking/tax info) really required just to evaluate Private Cloud Compute? Extra concerned as a non-US (Japan-based) company
Hi all, We're currently evaluating Private Cloud Compute (PCC) for a technical accuracy assessment. No production release or monetization is planned at this stage — our only goal is to test/evaluate the model. Per the official documentation, PCC access requires: Enrollment in the App Store Small Business Program Fewer than 2 million first-time downloads The Private Cloud Compute entitlement assigned to the account To complete Small Business Program enrollment, our Paid Applications Agreement is currently stuck at "User Information Pending", and we're being asked to submit a bank account and U.S. tax forms (Certificate of Foreign Status of Beneficial Owner / Substitute Form W-8BEN-E) before the agreement can go Active. Honestly, we're having a hard time accepting that submitting banking and revenue-related tax documentation is required when we have no intention of selling a paid app at all. The Small Business Program itself is meant to be a reduced-commission program for developers earning revenue through paid apps/IAP — using it as a gate for free AI model evaluation feels like a mismatch. On top of that, we're a Japan-based company, which raises the bar further. The required tax forms (W-8BEN-E etc.) are aimed at non-US entities and require pulling in our legal/finance teams just to prepare — a fair amount of overhead for what is, on our end, purely a technical evaluation. Before we go through the internal process of preparing this documentation, I wanted to confirm: Is there any path to obtain the PCC entitlement / Small Business Program status for evaluation purposes only, without completing the full Paid Applications Agreement (banking + tax forms)? Is submitting real banking and tax information a hard technical requirement of the Small Business Program itself (i.e., the Paid Apps Agreement cannot go Active without it), or can an account remain PCC-eligible while the agreement is "pending"? Is there an official Apple document (beyond the general Small Business Program / PCC pages) that explicitly confirms banking/tax submission is mandatory before PCC entitlement can be granted? We need something citable for internal approval. For non-US companies (e.g. Japan-based), has anyone gone through this purely for evaluation purposes? Is there any simplified path for foreign entities, or is the full W-8BEN-E process unavoidable? Any pointers to official documentation, or confirmation from anyone who has been through this, would be greatly appreciated — we need a clear, citable answer to justify preparing this documentation internally. Thanks in advance.
1
0
347
Aug ’26
Foundation Model tool calling giving system error in iOS27 beta 5
After updating my iOS and xcode to latest iOS 27 beta5 and xcode 27 beta5 all the system language model session calls with tool calls inclusion throwing Unrecognized system-instruction prefix ID: com.apple.fm_api.tool_calls_override error. The same code was working perfectly in iOS27 beta 4. Even the apple sample project OrigamiCraftingADynamicTutorialForAppleIntelligence failing with the same error when tool calls invoked. Anybody else facing similar issue or any workaround for this issue? sample code: struct GetRecordNotesTool: Tool { let name = "getRecordNotes" let description = "Fetches internal notes and returns Note_Title and Note_Content for up to 10 notes." @Generable struct Arguments { @Guide(description: "The API name of the module, e.g. Companies or Contacts") var module_api_name: String @Guide(description: "The unique record ID to fetch notes for") var record_id: String } func call(arguments: Arguments) async throws -> String { return "Fetched content" } }
1
0
870
Aug ’26
Advice on Referencing Previous Prompts / Responses
When using Private Cloud Compute, I want to be able to submit more than one prompt per LanguageModelSession, ideally using the prompt and response from the first interaction to inform a second interaction. How can I reference this first prompt and response when making a subsequent prompt in a session? I have tried plain language like "current data" and "previous prompt" but it does not seem to understand.
1
0
398
Aug ’26
How are you iterating on Foundation Models prompts before building the app workflow?
While building with Apple's Foundation Models, I kept running into a workflow problem before the app code itself. The hard part was not only calling LanguageModelSession. It was figuring out the shape of the interaction: What should be in the system prompt? What should stay in the user input? What output is actually usable by the app? How much instruction is too much? How do I test the same prompt repeatedly without creating another small Xcode project? I ended up building a small macOS tool for myself, LocalLM Lab, mainly to speed up that loop. The first use case was a Prompt Playground: system prompt, user input, model output, and a repeatable way to compare results before moving the workflow into app code. The current version also experiments with connector-style context, such as system clock, weather, reminders/calendar, contacts, and a scoped filesystem folder. That has made the prompt design problem more interesting, because the question becomes: what context should the model see, and how should the app frame that context so the output is useful? I am curious how other developers are handling this while building with Foundation Models. Are you mostly iterating inside Xcode playgrounds? Are you building small internal test harnesses? Are you separating system prompts and user inputs during testing? How are you evaluating whether the output is reliable enough for the app workflow? For reference, this is the tool I have been using for my own experiments: https://thisbrain.ai/locallm I would be especially interested in any patterns people have found for designing and testing prompts before committing them to app code.
4
0
730
Aug ’26
Foundation Models are broken in iOS 27 Beta
Hi guys, I'm testing the Foundation Models Framework with the on-device model in iOS 27 (beta 4) and macOS 27 (beta 4) and is completely failing to respond. There are many errors. For starters, the model doesn't respond to prompts directly, you need to specify instructions, otherwise it refuses to provide an answer. It is always looking for tools, even when no tool has been provided, and returns an error saying that it couldn't find the tool. Then, when it produces a response, it shows all the thinking process first, which completely ruins the response. Most of the time, the response begins with all the JSON code. And when I try to have a long conversation, it just says "I cannot write content or generate text." I wonder if someone is experiencing the same issues or maybe the way to implement this model changed and I'm missing something? Here is a screenshot of one of my interactions when I asked the model to describe a unicorn. It tried to access a tool that doesn't exist. (the app just prints the value of the content property) Here is the code. It is performing a simple request. struct ContentView: View { @State private var response = "" var body: some View { VStack { Button("Send") { let prompt = "Write a paragraph describing a unicorn" let session = LanguageModelSession { "Respond to the user's request. Never acknowledge the request, add preamble, or comment on what you are about to write." } if !session.isResponding { Task { do { let answer = try await session.respond(to: prompt) response = answer.content } catch { response = "Error accessing the model: \(error)" } } } } .buttonStyle(.borderedProminent) Text(response) .font(Font.system(size: 18)) .padding() Spacer() } .padding() } }
5
0
1.7k
Aug ’26
Use of SpotlightSearchTool() returns "Model Catalog error: Error Domain=com.apple.UnifiedAssetFramework Code=5000" , although model is available
On macOS Golden Gate Developer Beta 4 the following code: import CoreSpotlight import FoundationModels let tool = SpotlightSearchTool() let session = LanguageModelSession(tools: [tool]) let response = try await session.respond(to: "What hikes have I gone on?") , returns the following error: Model Catalog error: Error Domain=com.apple.UnifiedAssetFramework Code=5000 "There are no underlying assets (neither atomic instance nor asset roots) for consistency token for asset set com.apple.modelcatalog" UserInfo={NSLocalizedFailureReason=There are no underlying assets (neither atomic instance nor asset roots) for consistency token for asset set com.apple.modelcatalog} , although the model is available in general and can return responses without using the tool. The code: print(SystemLanguageModel.default.availability) returns 'available'. What am I doing wrong?
9
0
1.2k
Aug ’26
Generation Error
So I'm having an issue with the FoundationModels framework but idk if this is just my feeling or not, the issue comes up after I updated my Mac into 26.6 the code was very simple actually: #Playground { let model = SystemLanguageModel.default let session = LanguageModelSession(model: model) print(model.availability) var query = "How to hide button" Task { do { let response = try await session.respond(to: query) print(response.content) } catch { print("\(error)") } } } the code works before I updated the version, but then after I updated the version it says: Error Domain=FoundationModels.LanguageModelSession.GenerationError Code=-1 "The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.)" UserInfo={NSMultipleUnderlyingErrorsKey=( "Error Domain=FoundationModels.LanguageModelError Code=-1 "(null)" UserInfo={NSMultipleUnderlyingErrorsKey=(\n "Error Domain=ModelManagerServices.ModelManagerError Code=1026 \"(null)\" UserInfo={NSMultipleUnderlyingErrorsKey=(\n)}"\n)}" ), NSLocalizedDescription=The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.)} this is runned in Xcode 26.6, additional information I have also coder 27 beta 4 installed in my Mac, is this problem occurring because the Xcode 26.6 and Xcode 27 beta 4?? can u guys help me
2
0
841
Aug ’26
Foundation Models, image input and locating things within an image
I'm trying to use Foundation Models to identify the things in an image. That part is easy and is working well. I'd like to also know where in the image the things are. This is where I'm hitting a wall. For example, if there's an image of a horse and a cow, I'd like to be told (even approximate) coordinates of where in the image the horse is and where the cow is. Bounding boxes are fine for my needs. (Any coordinate system will work because it's easy enough to convert from one to another) The LanguageModelSession consistently lists the items in the image and gives me bounding boxes for their location that are reasonable approximations of where the images are in relation to one another, but it will (usually, not always) completely fail at explaining where the objects are in relation to the image as a whole, which is what I need. What's more, the failures are not consistent. Sometimes it will tell me that all the images are in the top half of the image. Other times, it will blow up the location of one or more objects in the image to multiples of their actual size. I've tried asking the LanguageModelSession to output the locations in various coordinate systems: raw pixel numbers normalized position (0 ... 1) integer percent position (0% ... 100%) a few different attempts at "soft location" systems where I just ask the LLM to tell me if the objects are in the top left corner or in the center for instance Of these, the "soft location" gives more consistent answers, but nothing that is complete enough to be usable. Asking for raw pixels gives answers that are ALMOST usable, but the position rectangles it gives are often off by one or two times the width or height of the object or suffer from the issue of "bunching" all the rectangles into the top of the image. I believe that part of the problem I'm having is that FoundationModels must downsample the image before processing. It appears that it's downsampling to 896px for the longest dimension of the image. Even accounting for this, though, I get strange output. Yes, I have considered using VisionKit's GenerateObjectnessBasedSaliencyImageRequest. It works well for another part of my project, but it doesn't fit exactly the particular need that I have here. It gives me locations of objects but not what they are. FoundationModels gives me what objects are in the image but not their locations. It may be that FoundationModels just isn't going to give me an accurate enough location for the objects in the image. It's a LLM, not a ML model, after all. If that's the case, I'd appreciate if someone would verify that so I can stop barking up this tree. It just seems like it should be possible, and I keep getting results that are almost accurate enough to be useful to me. Any help at all would be appreciated. Below are the instructions and prompt I'm using. let session = LanguageModelSession( instructions: """ You describe images to help another AI model identify and label distinct objects. Identify the distinct foreground subjects — objects, animals, people, or things that stand out as individual items someone would point to and name. Be specific (e.g. "a black and white cow", "a red coffee mug", "a wooden chair"). For each subject, provide a tight bounding box as pixel coordinates: - topLeft: upper-left corner of the box (x from left edge, y from top edge) - bottomRight: lower-right corner (x and y must be larger than topLeft's) - (0, 0) is the top-left pixel; x increases rightward, y increases downward - the exact pixel dimensions of each image are stated in the prompt you receive Also note background objects — items visible in the scene but not the main focus. Describe the setting — the background environment (surface, room, landscape, or space). Do not merge subjects and setting. A cow standing in a field has the cow as a subject and the field as the setting — not both as subjects. """ ) let prompt = Prompt { "Describe this \(imageWidth)×\(imageHeight) image. Bounding box coordinates are in pixels: (0,0) is top-left, (\(imageWidth),\(imageHeight)) is bottom-right." Attachment(modelImage.cgImage, orientation: modelImage.orientation) }
1
0
366
Jul ’26
Issue: Inflexible API Versioning Logic in Foundation Models framework utilities
In the Foundation Models framework utilities package, the private method buildURLRequest in ChatCompletionsLanguageModel handles the construction of OpenAI-compatible API URLs: private func buildURLRequest(for request: ChatCompletionRequest) throws -> URLRequest { let isVersioned = baseURL.pathComponents.contains("v1") let endpoint = isVersioned ? "/chat/completions" : "/v1/chat/completions" let url = baseURL.appendingPathComponent(endpoint) ... } Problem The current implementation hardcodes "v1" to determine if the baseURL already includes a version. This limits compatibility with API providers using alternative versioning schemes. For instance, Volcengine Ark uses "v3" in its Base URL, making it difficult to seamlessly integrate their services. #Playground { let baseURL = URL(string: "https://ark.cn-beijing.volces.com/api/v3")! let modelName = "doubao-seed-2-0-mini-260428" let headers: [String : String] = [ "Authorization" : "Bearer \(apiKey)" ] let model = ChatCompletionsLanguageModel(name: modelName, url: baseURL, additionalHeaders: headers) let session = LanguageModelSession(model: model) do { let result = try await session.respond(to: "Hello").content } catch { print(error.localizedDescription) // HTTP error with status code 404: } } #Playground { let baseURL = URL(string: "https://ark.cn-beijing.volces.com/api/v3/responses")! let modelName = "doubao-seed-2-0-mini-260428" let headers: [String : String] = [ "Authorization" : "Bearer \(apiKey)" ] let model = ChatCompletionsLanguageModel(name: modelName, url: baseURL, additionalHeaders: headers) let session = LanguageModelSession(model: model) do { let result = try await session.respond(to: "Hello").content } catch { print(error.localizedDescription) /* HTTP error with status code 404: {"error":{"code":"InvalidAction","message":"The specified action is invalid: /api/v3/responses/v1/chat/completions Request id: 021784381168842fdfd2e3c33d5b6eddad55ac385080e727cab08","param":"","type":"NotFound"}} */ } } Suggested Solution To better accommodate different versioning conventions (e.g., v2, v3), we can leverage Swift's modern Regex (#/v\d+/#) to dynamically detect the version pattern in the path components. Here is a recommended update for the isVersioned check: let isVersioned = baseURL.pathComponents.contains { component in component.wholeMatch(of: #/v\d+/#) != nil }
2
0
370
Jul ’26
More Detailed Quota Usage for PCC
Unless I'm missing something, it seems like the quota usage information for the Private Cloud Compute model is too limited. You can tell if you've reached your quota or are below it. If you are below your quota, you can tell if you're approaching the limit, but what does this actually mean? Am I over 50%, 90%, 99%? It would be nice to have actual numbers in the quota. For example, I can see my token usage for a session. If an app could keep track of that versus the quota, you could come up with something way more useful for the user. Example: You have 100,000 tokens per month, this app has made 4 requests, that used a total of 5,000 tokens. If the user has used on 95,000 tokens of their quota so far, they know they can maybe make ~4 more requests from the app before the limit is reached, so they know to be careful with their usage. If they've only used 10,000 tokens of their quota so far, they know that have some breathing room and can use the feature more freely. The way the current system is designed, you have no idea at all. Adding real numbers (even percentages – if we can get usage percentages for the app as well), would really help in giving useful feedback to the user on their usage of PCC. Right now, everything is too vague.
2
0
535
Jul ’26
Accessing Private Cloud Compute
Hello, I recently learned about Private Cloud Compute (PCC): https://developer.apple.com/private-cloud-compute/ I am currently using a standard Developer Program account, and it seems that I cannot apply for the program directly. Is there an alternative? Also, is there any additional fee for using this service? If I want to call PCC in the app, for example, using the following code: let session = LanguageModelSession( model: PrivateCloudComputeLanguageModel() ) Do I need to apply for a specific plan to ensure that my App is successfully published on the App Store and available to users? Thank you!
1
0
524
Jul ’26
Sensitive Content Error When Using Foundation Models
I am using the following code in my iOS application. #Playground { let session = LanguageModelSession() let response = try await session.respond(to: "List all states of USA.") print(response.content) } And I get the following error: The operation couldn’t be completed. (com.apple.SensitiveContentAnalysisML error 15.) I have turned off Apple Intelligence and turned on again. No use. I am using Xcode 27 beta 2. any ideas?
2
0
486
Jul ’26
Adapter Problem - compatibleAdapterNotFound
Hello. I have a problem with the FoundationModels adapter and the Apple-hosted managed asset pack via TestFlight. I have created an adapter that works fine locally by creating a model via (fileURL: URL) on a real device, but I cannot create a model using background assets by downloading the adapter via TestFlight. Every time I try to get an adapter, the creation of the adapter is interrupted by the compatibleAdapterNotFound error. The aar. archive i created using a special command - xcrun ba-package foundation-models package --adapter-path aurelius1.fmadapter --asset-pack-id fmadapter-aurelius1-9799725 --output-path ./aurelius1.aar --platforms iOS --on-demand\ after that, I replaced "OnDemand": null with "OnDemand": {} in the manifest so that the Transporter could send my archive to the App Store Connect. I followed all the recommendations in this topic - https://origin-devforums.apple.com/forums/thread/823148 ...but unfortunately unsuccessfully I would appreciate any help in solving this problem. here is the code that I use in my app -
6
0
718
Jul ’26
Recommended App Store distribution strategy for apps that require Foundation Models
Hello, I'm evaluating Foundation Models announced at WWDC 2026 and have a question regarding App Store distribution. My understanding is that Foundation Models are only available on supported devices and operating system versions. For apps that rely on Foundation Models as their primary functionality (rather than offering AI as an optional feature), I'm trying to understand the recommended distribution strategy. Currently, iOS provides Required Device Capabilities to prevent users from installing apps that require hardware features such as GPS, ARKit, or NFC. However, I couldn't find an equivalent Required Device Capability for Foundation Models. I also couldn't find a way to limit App Store availability by supported device models. My questions are: What is the recommended way to distribute an app whose primary functionality depends on Foundation Models? Is there currently any supported mechanism to prevent users with unsupported devices from downloading such an app? Is Apple planning to introduce a Required Device Capability (or a similar App Store filtering mechanism) for Foundation Models before public release? Without such a mechanism, users may be able to install the app successfully but then discover that its primary functionality is unavailable on their device. I'd appreciate any guidance on the recommended approach. Thank you.
5
0
574
Jul ’26
SpotlightSearchTool Not Invoked, Console Error
I'm following along with the WWDC video on SpotlightSearchTool and hitting an error - looking for some guidance. I've configured SpotlightSearchTool and I'm sending it to the session. let session = LanguageModelSession(tools: [tool]) { spotlightSearchInstructions } let response = try await session.respond(to: prompt, options: GenerationOptions(toolCallingMode: .required)) I set the tool calling mode to required as a test - without it I don't get errors but the logging makes it seem like it's not calling the search tool and the responses would seem to confirm that (they're not grounded in my data). So, I figured I'd try forcing it to use the tool. When I do that, I get this console error: InferenceError::hostFailed::InferenceError::inferenceFailed::TokenGenerationCore.GuidedGenerationError.invalidConfiguration(errorMessage: "Tool Choice requires tools") in response to ExecuteRequest Error during session.respond. description="The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.)" Returning empty Spotlight result. elapsedMs=3254 toolReplies=0 totalSearchItems=0 uniqueSearchItems=0 What does that mean? I'm passing in a tool, everything compiles correctly, etc. Not sure how to debug - any advice appreciated! Testing this via the Simulator on beta 3.
6
0
740
Jul ’26
Foundation Models: Model-level refusal regression on iOS 27 beta for health app prompts (not guardrailViolation)
I have a health app on the App Store that uses Foundation Models to generate brief narrative summaries from the user's own glucose and menstrual cycle data. No medical advice, just supportive summaries of their own numbers. This has been working reliably on iOS 26.x since early 2026. After updating to iOS 27 beta 2, every prompt is refused. The error is LanguageModelError ("The model refused to answer" / "May contain sensitive content"), not GenerationError.guardrailViolation. I've confirmed: Same device, same code, same prompts. Worked on iOS 26.x, fails on iOS 27 beta 2. Two independent features with different prompt structures and different service architectures are both affected. Using SystemLanguageModel(guardrails: .permissiveContentTransformations) does not help. The classifier passes. The model itself refuses. The prompts contain terms like "luteal phase," "progesterone," "glucose," "time in range," and "diabetes" in the system instructions. This appears to be a model-level sensitivity change in the iOS 27 on-device model that broadly blocks health/medical terminology, even when the use case is summarizing the user's own data. Filed as FB23513774 with the full prompt text, instructions, and source file attached. Is anyone else seeing model-level refusals (not guardrailViolation) on iOS 27 beta for health or medical content? Related threads from iOS 26 betas: Model Guardrails Too Restrictive? Model w/ Guardrails Disabled Still Refusing Using Past Versions of Foundation Models As They Progress
1
0
534
Jul ’26
Can any Apple Watch running WatchOS 27 access PCC via Foundation Models?
Apologies, if I've missed the answer already here, I've searched around but can't find it. Foundation models and Private Cloud Compute require Apple Intelligence to be enabled in Settings as mentioned here. At the same time it says that Foundation Models PCC calls are supported on all Apple Watch models that run WatchOS 27. So, will there be a seperate Apple Intelligence setting in WatchOS 27 for those devices? Otherwise if a user has an Apple Watch Series 11 (which does support Apple Intelligence) paired with an iPhone 15 (which doesn't support Apple Intelligence), will they be unable to use the Foundation Models PCC calls from WatchOS in my app? Despite the fact the iPhone isn't involved in these queries anyway?
1
1
724
Jul ’26
Is Small Business Program enrollment (incl. banking/tax info) really required just to evaluate Private Cloud Compute? Extra concerned as a non-US (Japan-based) company
Hi all, We're currently evaluating Private Cloud Compute (PCC) for a technical accuracy assessment. No production release or monetization is planned at this stage — our only goal is to test/evaluate the model. Per the official documentation, PCC access requires: Enrollment in the App Store Small Business Program Fewer than 2 million first-time downloads The Private Cloud Compute entitlement assigned to the account To complete Small Business Program enrollment, our Paid Applications Agreement is currently stuck at "User Information Pending", and we're being asked to submit a bank account and U.S. tax forms (Certificate of Foreign Status of Beneficial Owner / Substitute Form W-8BEN-E) before the agreement can go Active. Honestly, we're having a hard time accepting that submitting banking and revenue-related tax documentation is required when we have no intention of selling a paid app at all. The Small Business Program itself is meant to be a reduced-commission program for developers earning revenue through paid apps/IAP — using it as a gate for free AI model evaluation feels like a mismatch. On top of that, we're a Japan-based company, which raises the bar further. The required tax forms (W-8BEN-E etc.) are aimed at non-US entities and require pulling in our legal/finance teams just to prepare — a fair amount of overhead for what is, on our end, purely a technical evaluation. Before we go through the internal process of preparing this documentation, I wanted to confirm: Is there any path to obtain the PCC entitlement / Small Business Program status for evaluation purposes only, without completing the full Paid Applications Agreement (banking + tax forms)? Is submitting real banking and tax information a hard technical requirement of the Small Business Program itself (i.e., the Paid Apps Agreement cannot go Active without it), or can an account remain PCC-eligible while the agreement is "pending"? Is there an official Apple document (beyond the general Small Business Program / PCC pages) that explicitly confirms banking/tax submission is mandatory before PCC entitlement can be granted? We need something citable for internal approval. For non-US companies (e.g. Japan-based), has anyone gone through this purely for evaluation purposes? Is there any simplified path for foreign entities, or is the full W-8BEN-E process unavoidable? Any pointers to official documentation, or confirmation from anyone who has been through this, would be greatly appreciated — we need a clear, citable answer to justify preparing this documentation internally. Thanks in advance.
Replies
1
Boosts
0
Views
347
Activity
Aug ’26
Foundation Model tool calling giving system error in iOS27 beta 5
After updating my iOS and xcode to latest iOS 27 beta5 and xcode 27 beta5 all the system language model session calls with tool calls inclusion throwing Unrecognized system-instruction prefix ID: com.apple.fm_api.tool_calls_override error. The same code was working perfectly in iOS27 beta 4. Even the apple sample project OrigamiCraftingADynamicTutorialForAppleIntelligence failing with the same error when tool calls invoked. Anybody else facing similar issue or any workaround for this issue? sample code: struct GetRecordNotesTool: Tool { let name = "getRecordNotes" let description = "Fetches internal notes and returns Note_Title and Note_Content for up to 10 notes." @Generable struct Arguments { @Guide(description: "The API name of the module, e.g. Companies or Contacts") var module_api_name: String @Guide(description: "The unique record ID to fetch notes for") var record_id: String } func call(arguments: Arguments) async throws -> String { return "Fetched content" } }
Replies
1
Boosts
0
Views
870
Activity
Aug ’26
Advice on Referencing Previous Prompts / Responses
When using Private Cloud Compute, I want to be able to submit more than one prompt per LanguageModelSession, ideally using the prompt and response from the first interaction to inform a second interaction. How can I reference this first prompt and response when making a subsequent prompt in a session? I have tried plain language like "current data" and "previous prompt" but it does not seem to understand.
Replies
1
Boosts
0
Views
398
Activity
Aug ’26
How are you iterating on Foundation Models prompts before building the app workflow?
While building with Apple's Foundation Models, I kept running into a workflow problem before the app code itself. The hard part was not only calling LanguageModelSession. It was figuring out the shape of the interaction: What should be in the system prompt? What should stay in the user input? What output is actually usable by the app? How much instruction is too much? How do I test the same prompt repeatedly without creating another small Xcode project? I ended up building a small macOS tool for myself, LocalLM Lab, mainly to speed up that loop. The first use case was a Prompt Playground: system prompt, user input, model output, and a repeatable way to compare results before moving the workflow into app code. The current version also experiments with connector-style context, such as system clock, weather, reminders/calendar, contacts, and a scoped filesystem folder. That has made the prompt design problem more interesting, because the question becomes: what context should the model see, and how should the app frame that context so the output is useful? I am curious how other developers are handling this while building with Foundation Models. Are you mostly iterating inside Xcode playgrounds? Are you building small internal test harnesses? Are you separating system prompts and user inputs during testing? How are you evaluating whether the output is reliable enough for the app workflow? For reference, this is the tool I have been using for my own experiments: https://thisbrain.ai/locallm I would be especially interested in any patterns people have found for designing and testing prompts before committing them to app code.
Replies
4
Boosts
0
Views
730
Activity
Aug ’26
Foundation Models are broken in iOS 27 Beta
Hi guys, I'm testing the Foundation Models Framework with the on-device model in iOS 27 (beta 4) and macOS 27 (beta 4) and is completely failing to respond. There are many errors. For starters, the model doesn't respond to prompts directly, you need to specify instructions, otherwise it refuses to provide an answer. It is always looking for tools, even when no tool has been provided, and returns an error saying that it couldn't find the tool. Then, when it produces a response, it shows all the thinking process first, which completely ruins the response. Most of the time, the response begins with all the JSON code. And when I try to have a long conversation, it just says "I cannot write content or generate text." I wonder if someone is experiencing the same issues or maybe the way to implement this model changed and I'm missing something? Here is a screenshot of one of my interactions when I asked the model to describe a unicorn. It tried to access a tool that doesn't exist. (the app just prints the value of the content property) Here is the code. It is performing a simple request. struct ContentView: View { @State private var response = "" var body: some View { VStack { Button("Send") { let prompt = "Write a paragraph describing a unicorn" let session = LanguageModelSession { "Respond to the user's request. Never acknowledge the request, add preamble, or comment on what you are about to write." } if !session.isResponding { Task { do { let answer = try await session.respond(to: prompt) response = answer.content } catch { response = "Error accessing the model: \(error)" } } } } .buttonStyle(.borderedProminent) Text(response) .font(Font.system(size: 18)) .padding() Spacer() } .padding() } }
Replies
5
Boosts
0
Views
1.7k
Activity
Aug ’26
Use of SpotlightSearchTool() returns "Model Catalog error: Error Domain=com.apple.UnifiedAssetFramework Code=5000" , although model is available
On macOS Golden Gate Developer Beta 4 the following code: import CoreSpotlight import FoundationModels let tool = SpotlightSearchTool() let session = LanguageModelSession(tools: [tool]) let response = try await session.respond(to: "What hikes have I gone on?") , returns the following error: Model Catalog error: Error Domain=com.apple.UnifiedAssetFramework Code=5000 "There are no underlying assets (neither atomic instance nor asset roots) for consistency token for asset set com.apple.modelcatalog" UserInfo={NSLocalizedFailureReason=There are no underlying assets (neither atomic instance nor asset roots) for consistency token for asset set com.apple.modelcatalog} , although the model is available in general and can return responses without using the tool. The code: print(SystemLanguageModel.default.availability) returns 'available'. What am I doing wrong?
Replies
9
Boosts
0
Views
1.2k
Activity
Aug ’26
Generation Error
So I'm having an issue with the FoundationModels framework but idk if this is just my feeling or not, the issue comes up after I updated my Mac into 26.6 the code was very simple actually: #Playground { let model = SystemLanguageModel.default let session = LanguageModelSession(model: model) print(model.availability) var query = "How to hide button" Task { do { let response = try await session.respond(to: query) print(response.content) } catch { print("\(error)") } } } the code works before I updated the version, but then after I updated the version it says: Error Domain=FoundationModels.LanguageModelSession.GenerationError Code=-1 "The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.)" UserInfo={NSMultipleUnderlyingErrorsKey=( "Error Domain=FoundationModels.LanguageModelError Code=-1 "(null)" UserInfo={NSMultipleUnderlyingErrorsKey=(\n "Error Domain=ModelManagerServices.ModelManagerError Code=1026 \"(null)\" UserInfo={NSMultipleUnderlyingErrorsKey=(\n)}"\n)}" ), NSLocalizedDescription=The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.)} this is runned in Xcode 26.6, additional information I have also coder 27 beta 4 installed in my Mac, is this problem occurring because the Xcode 26.6 and Xcode 27 beta 4?? can u guys help me
Replies
2
Boosts
0
Views
841
Activity
Aug ’26
Foundation Models, image input and locating things within an image
I'm trying to use Foundation Models to identify the things in an image. That part is easy and is working well. I'd like to also know where in the image the things are. This is where I'm hitting a wall. For example, if there's an image of a horse and a cow, I'd like to be told (even approximate) coordinates of where in the image the horse is and where the cow is. Bounding boxes are fine for my needs. (Any coordinate system will work because it's easy enough to convert from one to another) The LanguageModelSession consistently lists the items in the image and gives me bounding boxes for their location that are reasonable approximations of where the images are in relation to one another, but it will (usually, not always) completely fail at explaining where the objects are in relation to the image as a whole, which is what I need. What's more, the failures are not consistent. Sometimes it will tell me that all the images are in the top half of the image. Other times, it will blow up the location of one or more objects in the image to multiples of their actual size. I've tried asking the LanguageModelSession to output the locations in various coordinate systems: raw pixel numbers normalized position (0 ... 1) integer percent position (0% ... 100%) a few different attempts at "soft location" systems where I just ask the LLM to tell me if the objects are in the top left corner or in the center for instance Of these, the "soft location" gives more consistent answers, but nothing that is complete enough to be usable. Asking for raw pixels gives answers that are ALMOST usable, but the position rectangles it gives are often off by one or two times the width or height of the object or suffer from the issue of "bunching" all the rectangles into the top of the image. I believe that part of the problem I'm having is that FoundationModels must downsample the image before processing. It appears that it's downsampling to 896px for the longest dimension of the image. Even accounting for this, though, I get strange output. Yes, I have considered using VisionKit's GenerateObjectnessBasedSaliencyImageRequest. It works well for another part of my project, but it doesn't fit exactly the particular need that I have here. It gives me locations of objects but not what they are. FoundationModels gives me what objects are in the image but not their locations. It may be that FoundationModels just isn't going to give me an accurate enough location for the objects in the image. It's a LLM, not a ML model, after all. If that's the case, I'd appreciate if someone would verify that so I can stop barking up this tree. It just seems like it should be possible, and I keep getting results that are almost accurate enough to be useful to me. Any help at all would be appreciated. Below are the instructions and prompt I'm using. let session = LanguageModelSession( instructions: """ You describe images to help another AI model identify and label distinct objects. Identify the distinct foreground subjects — objects, animals, people, or things that stand out as individual items someone would point to and name. Be specific (e.g. "a black and white cow", "a red coffee mug", "a wooden chair"). For each subject, provide a tight bounding box as pixel coordinates: - topLeft: upper-left corner of the box (x from left edge, y from top edge) - bottomRight: lower-right corner (x and y must be larger than topLeft's) - (0, 0) is the top-left pixel; x increases rightward, y increases downward - the exact pixel dimensions of each image are stated in the prompt you receive Also note background objects — items visible in the scene but not the main focus. Describe the setting — the background environment (surface, room, landscape, or space). Do not merge subjects and setting. A cow standing in a field has the cow as a subject and the field as the setting — not both as subjects. """ ) let prompt = Prompt { "Describe this \(imageWidth)×\(imageHeight) image. Bounding box coordinates are in pixels: (0,0) is top-left, (\(imageWidth),\(imageHeight)) is bottom-right." Attachment(modelImage.cgImage, orientation: modelImage.orientation) }
Replies
1
Boosts
0
Views
366
Activity
Jul ’26
Issue: Inflexible API Versioning Logic in Foundation Models framework utilities
In the Foundation Models framework utilities package, the private method buildURLRequest in ChatCompletionsLanguageModel handles the construction of OpenAI-compatible API URLs: private func buildURLRequest(for request: ChatCompletionRequest) throws -> URLRequest { let isVersioned = baseURL.pathComponents.contains("v1") let endpoint = isVersioned ? "/chat/completions" : "/v1/chat/completions" let url = baseURL.appendingPathComponent(endpoint) ... } Problem The current implementation hardcodes "v1" to determine if the baseURL already includes a version. This limits compatibility with API providers using alternative versioning schemes. For instance, Volcengine Ark uses "v3" in its Base URL, making it difficult to seamlessly integrate their services. #Playground { let baseURL = URL(string: "https://ark.cn-beijing.volces.com/api/v3")! let modelName = "doubao-seed-2-0-mini-260428" let headers: [String : String] = [ "Authorization" : "Bearer \(apiKey)" ] let model = ChatCompletionsLanguageModel(name: modelName, url: baseURL, additionalHeaders: headers) let session = LanguageModelSession(model: model) do { let result = try await session.respond(to: "Hello").content } catch { print(error.localizedDescription) // HTTP error with status code 404: } } #Playground { let baseURL = URL(string: "https://ark.cn-beijing.volces.com/api/v3/responses")! let modelName = "doubao-seed-2-0-mini-260428" let headers: [String : String] = [ "Authorization" : "Bearer \(apiKey)" ] let model = ChatCompletionsLanguageModel(name: modelName, url: baseURL, additionalHeaders: headers) let session = LanguageModelSession(model: model) do { let result = try await session.respond(to: "Hello").content } catch { print(error.localizedDescription) /* HTTP error with status code 404: {"error":{"code":"InvalidAction","message":"The specified action is invalid: /api/v3/responses/v1/chat/completions Request id: 021784381168842fdfd2e3c33d5b6eddad55ac385080e727cab08","param":"","type":"NotFound"}} */ } } Suggested Solution To better accommodate different versioning conventions (e.g., v2, v3), we can leverage Swift's modern Regex (#/v\d+/#) to dynamically detect the version pattern in the path components. Here is a recommended update for the isVersioned check: let isVersioned = baseURL.pathComponents.contains { component in component.wholeMatch(of: #/v\d+/#) != nil }
Replies
2
Boosts
0
Views
370
Activity
Jul ’26
More Detailed Quota Usage for PCC
Unless I'm missing something, it seems like the quota usage information for the Private Cloud Compute model is too limited. You can tell if you've reached your quota or are below it. If you are below your quota, you can tell if you're approaching the limit, but what does this actually mean? Am I over 50%, 90%, 99%? It would be nice to have actual numbers in the quota. For example, I can see my token usage for a session. If an app could keep track of that versus the quota, you could come up with something way more useful for the user. Example: You have 100,000 tokens per month, this app has made 4 requests, that used a total of 5,000 tokens. If the user has used on 95,000 tokens of their quota so far, they know they can maybe make ~4 more requests from the app before the limit is reached, so they know to be careful with their usage. If they've only used 10,000 tokens of their quota so far, they know that have some breathing room and can use the feature more freely. The way the current system is designed, you have no idea at all. Adding real numbers (even percentages – if we can get usage percentages for the app as well), would really help in giving useful feedback to the user on their usage of PCC. Right now, everything is too vague.
Replies
2
Boosts
0
Views
535
Activity
Jul ’26
I did well on iOS a decade ago. So - no foundation models for me?
I had a great run in the first decade of iOS development. Not so much since. I had 180k downloaded units in the last year - but I'm excluded from foundation models because I did well before 2015. That seems like an odd policy. Apart from anything else - it explicitly punishes long-term accounts... Lifetime downloads...
Replies
5
Boosts
0
Views
656
Activity
Jul ’26
TTS Advanced Speech Generation: Expressive voices
During WWDC26 Keynote a second generation on-device model was announced with better speech generation capabilities. Is there a new API available for developers to generate speech?
Replies
1
Boosts
1
Views
535
Activity
Jul ’26
Accessing Private Cloud Compute
Hello, I recently learned about Private Cloud Compute (PCC): https://developer.apple.com/private-cloud-compute/ I am currently using a standard Developer Program account, and it seems that I cannot apply for the program directly. Is there an alternative? Also, is there any additional fee for using this service? If I want to call PCC in the app, for example, using the following code: let session = LanguageModelSession( model: PrivateCloudComputeLanguageModel() ) Do I need to apply for a specific plan to ensure that my App is successfully published on the App Store and available to users? Thank you!
Replies
1
Boosts
0
Views
524
Activity
Jul ’26
Sensitive Content Error When Using Foundation Models
I am using the following code in my iOS application. #Playground { let session = LanguageModelSession() let response = try await session.respond(to: "List all states of USA.") print(response.content) } And I get the following error: The operation couldn’t be completed. (com.apple.SensitiveContentAnalysisML error 15.) I have turned off Apple Intelligence and turned on again. No use. I am using Xcode 27 beta 2. any ideas?
Replies
2
Boosts
0
Views
486
Activity
Jul ’26
Adapter Problem - compatibleAdapterNotFound
Hello. I have a problem with the FoundationModels adapter and the Apple-hosted managed asset pack via TestFlight. I have created an adapter that works fine locally by creating a model via (fileURL: URL) on a real device, but I cannot create a model using background assets by downloading the adapter via TestFlight. Every time I try to get an adapter, the creation of the adapter is interrupted by the compatibleAdapterNotFound error. The aar. archive i created using a special command - xcrun ba-package foundation-models package --adapter-path aurelius1.fmadapter --asset-pack-id fmadapter-aurelius1-9799725 --output-path ./aurelius1.aar --platforms iOS --on-demand\ after that, I replaced "OnDemand": null with "OnDemand": {} in the manifest so that the Transporter could send my archive to the App Store Connect. I followed all the recommendations in this topic - https://origin-devforums.apple.com/forums/thread/823148 ...but unfortunately unsuccessfully I would appreciate any help in solving this problem. here is the code that I use in my app -
Replies
6
Boosts
0
Views
718
Activity
Jul ’26
Recommended App Store distribution strategy for apps that require Foundation Models
Hello, I'm evaluating Foundation Models announced at WWDC 2026 and have a question regarding App Store distribution. My understanding is that Foundation Models are only available on supported devices and operating system versions. For apps that rely on Foundation Models as their primary functionality (rather than offering AI as an optional feature), I'm trying to understand the recommended distribution strategy. Currently, iOS provides Required Device Capabilities to prevent users from installing apps that require hardware features such as GPS, ARKit, or NFC. However, I couldn't find an equivalent Required Device Capability for Foundation Models. I also couldn't find a way to limit App Store availability by supported device models. My questions are: What is the recommended way to distribute an app whose primary functionality depends on Foundation Models? Is there currently any supported mechanism to prevent users with unsupported devices from downloading such an app? Is Apple planning to introduce a Required Device Capability (or a similar App Store filtering mechanism) for Foundation Models before public release? Without such a mechanism, users may be able to install the app successfully but then discover that its primary functionality is unavailable on their device. I'd appreciate any guidance on the recommended approach. Thank you.
Replies
5
Boosts
0
Views
574
Activity
Jul ’26
SpotlightSearchTool Not Invoked, Console Error
I'm following along with the WWDC video on SpotlightSearchTool and hitting an error - looking for some guidance. I've configured SpotlightSearchTool and I'm sending it to the session. let session = LanguageModelSession(tools: [tool]) { spotlightSearchInstructions } let response = try await session.respond(to: prompt, options: GenerationOptions(toolCallingMode: .required)) I set the tool calling mode to required as a test - without it I don't get errors but the logging makes it seem like it's not calling the search tool and the responses would seem to confirm that (they're not grounded in my data). So, I figured I'd try forcing it to use the tool. When I do that, I get this console error: InferenceError::hostFailed::InferenceError::inferenceFailed::TokenGenerationCore.GuidedGenerationError.invalidConfiguration(errorMessage: "Tool Choice requires tools") in response to ExecuteRequest Error during session.respond. description="The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.)" Returning empty Spotlight result. elapsedMs=3254 toolReplies=0 totalSearchItems=0 uniqueSearchItems=0 What does that mean? I'm passing in a tool, everything compiles correctly, etc. Not sure how to debug - any advice appreciated! Testing this via the Simulator on beta 3.
Replies
6
Boosts
0
Views
740
Activity
Jul ’26
Foundation models tied to Siri in Mac OS beta 2
Since beta 2 I think, it seems Foundation models are not accessible if Siri AI is not enabled. I'm on Mac OS, and not sure how it works on iOS, but does that mean that Foundation Models will not be usable if Siri AI is not enabled (Europe)?
Replies
1
Boosts
0
Views
451
Activity
Jul ’26
Foundation Models: Model-level refusal regression on iOS 27 beta for health app prompts (not guardrailViolation)
I have a health app on the App Store that uses Foundation Models to generate brief narrative summaries from the user's own glucose and menstrual cycle data. No medical advice, just supportive summaries of their own numbers. This has been working reliably on iOS 26.x since early 2026. After updating to iOS 27 beta 2, every prompt is refused. The error is LanguageModelError ("The model refused to answer" / "May contain sensitive content"), not GenerationError.guardrailViolation. I've confirmed: Same device, same code, same prompts. Worked on iOS 26.x, fails on iOS 27 beta 2. Two independent features with different prompt structures and different service architectures are both affected. Using SystemLanguageModel(guardrails: .permissiveContentTransformations) does not help. The classifier passes. The model itself refuses. The prompts contain terms like "luteal phase," "progesterone," "glucose," "time in range," and "diabetes" in the system instructions. This appears to be a model-level sensitivity change in the iOS 27 on-device model that broadly blocks health/medical terminology, even when the use case is summarizing the user's own data. Filed as FB23513774 with the full prompt text, instructions, and source file attached. Is anyone else seeing model-level refusals (not guardrailViolation) on iOS 27 beta for health or medical content? Related threads from iOS 26 betas: Model Guardrails Too Restrictive? Model w/ Guardrails Disabled Still Refusing Using Past Versions of Foundation Models As They Progress
Replies
1
Boosts
0
Views
534
Activity
Jul ’26
Can any Apple Watch running WatchOS 27 access PCC via Foundation Models?
Apologies, if I've missed the answer already here, I've searched around but can't find it. Foundation models and Private Cloud Compute require Apple Intelligence to be enabled in Settings as mentioned here. At the same time it says that Foundation Models PCC calls are supported on all Apple Watch models that run WatchOS 27. So, will there be a seperate Apple Intelligence setting in WatchOS 27 for those devices? Otherwise if a user has an Apple Watch Series 11 (which does support Apple Intelligence) paired with an iPhone 15 (which doesn't support Apple Intelligence), will they be unable to use the Foundation Models PCC calls from WatchOS in my app? Despite the fact the iPhone isn't involved in these queries anyway?
Replies
1
Boosts
1
Views
724
Activity
Jul ’26