Any guidance for migrating legacy AppIntents that are nearly identical in functionality to AppSchema app intents? I want to avoid duplicate intents (old vs new) showing up in Shortcuts.

Hello, I'm still working on my @addToAlbum schema implementation, and I'm exploring how multiple entities could be "destinations" to the intent. I considered using a @UnionValue for this, but I'm running into compiler difficulties trying to get a @UnionValue to conform to @AppEntity(schema: .photos.album) Am I out of luck on a Unionized "target" for the add-to-album intent?

During the keynote, we saw an amazing example of Siri using Visual Intelligence to identify items in the user's physical space and make inferences based on their size. Do 3rd party apps have the ability to perform this same, or similar actions? For example: User loads a photo of an item or product and clicks a button that says 'Find Item In My Space'. Apple Intelligence is then used to analyze the user's surroundings, and notify the user if the item is present or not present, along with some positional or physical context. Response is shown on the user interface as text, "This item is in your room, 1 meter to your right." Goal: Developers currently can not access the Passthrough Camera on Apple Vision Pro to run AI/ML vision processing models on, for privacy reasons. If Apple Intelligence can look through the camera for the developer, in a privacy-preserving, isolated black box, without providing the image texture to the developer in any way, the user can make use Visual Intelligence features based on their physical surroundings without sacrificing their privacy. Purpose: Visual Intelligence is a key feature for that exemplifies the benefits of Spatial Computing, and examples like the one shown in the Keynote are a perfect use-case for the medium. Since Siri now has this capability, users will come to expect that all apps across VisionOS will be able to perform the same kinds of actions. Developers don't generally want or need direct access to the images of a user's surroundings, and having a local/private method of processing these requests is ideal both for developers concerned with data privacy management and users concerned with developers having too much access to their surroundings. Wearable devices with cameras are a foundational accelerator to users adopting AI in useful ways for their daily life. It is the most natural way to communicate with AI about what is relevant to you at any given time, removes the friction/difficulty of manually scanning good data for AI inferencing, and brings purpose to wearing this class of device every day. As these devices become more common and capable, data privacy becomes even more important. Users will need reassurance that the devices they choose to wear will only have access to observe their surroundings when they choose to allow it, while retaining the capability to use the powerful features that make them worthwhile. Accessibility: Using Visual Intelligence is an extremely powerful accessibility tool (for example; for individuals who have low vision), and can meaningfully improve quality of life. Various applications beyond Siri AI can be designed by developers with very specific inferencing capabilities powered by AI. The future of Visually Intelligent apps should have intentional, unique purposes that users can choose to incorporate in their lives. This will not be a one-size-fits-all Visual Intelligence approach, and will require specific design, training and development to create meaningful capabilities. If this is already possible, amazing! Any resources to learn more would be greatly appreciated. If this is not yet possible, please let us know what we can do to encourage Apple to consider it. Thank you.

Does the App Intents framework expose the user's active spoken Siri locale (e.g., ja-JP, fr-FR) directly within the perform() context, or must the extension rely on the system's global locale setting? If a user switches Siri's language dynamically, how is that locale string propagated to the intent execution block?

When using the .photos.asset schema for an AppEntity conforming to IndexedEntity, is there a way to trigger the rich grid layout in Spotlight results that native apps like Photos and Messages currently utilize?

When implementing @AppIntent(schema: .photos.createAssets) in my app I can accept files to import as assets into my app. Therefore I would expect be able to use this in Shortcuts and pass in a selected file to my createAssets intent. For example using the Files app Select Files action and pass the selected files on to my photos.createAssets action. However doesn't seem to be possible in any way. The only thing I can do in Shortcuts is select a file for the create assets action at the time I make the shortcut. Which removes all flexibility. Is this a bug or a missing feature? Or am I missing something how to set this up correctly? I've been testing this with the new "Integrating your photo app with Apple Intelligence" Apple Sample and it's not working with that in any way. FB23066510

Can App Shortcuts (and their trigger phrases) be generated or translated dynamically at runtime based on user-defined configurations (e.g. download-on-demand language models or custom voice aliases), or must all voice shortcuts and translations be statically declared in the application's compiled String Catalogs (.xcstrings)?

How does the App Intents framework support streaming spoken voice (TTS) output for long-form text responses? Is there an API (such as an asynchronous sequence or buffer stream) that allows Siri to begin speaking a response while the underlying AI engine is still generating the remainder of the text?

We are designing a macOS utility that runs a local background agent via launchd (managing a local SQLite database and Unix socket). We want to expose controls (start/stop, status checks, CLI command invocation) to Siri via the App Intents framework. Since App Intents typically execute within a sandboxed App Extension or a sandboxed App wrapper container: What is the recommended IPC mechanism (e.g., Unix domain sockets, local HTTP/TCP ports, XPC) to securely communicate between a sandboxed App Intent extension and a non-sandboxed launchd helper daemon on macOS? Are there specific Entitlements (com.apple.security.temporary-exception.files.absolute-path.read-write or network exceptions) required to allow App Intents to talk to local UNIX sockets or loopback interfaces (127.0.0.1) without triggering sandbox violations? Can an App Intent directly invoke a helper command-line tool or launch a plist-configured service without bringing up the main application UI?

Under the App Schemas list, some of the entries are listed under "Shortcut-specific domains" (like Books and Journaling). What's the difference? Will those domains not be used with Siri AI?

I'm an app developer with an interest in nutrition data. While watching the Keynote segment on Monday, there was a brief section about Siri AI and iOS 27 Vision framework being used to scan a meal with an iPhone camera and display the meal nutritional content. I have two questions: Do the displayed data include GI (glycemic index) and/or GL (glycemic load)? and Are these nutrition data surfaced anywhere, especially in HealthKit, or — if not — will Apple make a public API available so developers can use these data directly in iOS27 apps?

Hi all, This is a simplification of an AppIntent where I need my phrases to recognize three "regions" and a specific set of numbers associated with those regions. I am having trouble defining the AppEntity and associated AppShortcut phrases that works well with Siri, which is having trouble extracting numbers for some reason... Now the example: Imagine my app provides information surrounding Starbucks establishments on three major thoroughfares in NYC, such as: Broadway & 26, 40, 43, 54, 63, 75, 86, 95 Madison & 44, 50, 84, 96 Lexington & 43, 48, 55, 63, 67, 77, 85, 96 I need AppEntities, AppShortcuts, and friends that best work with Siri when launched with phrases such as: "Show 'example' for Madison and 50th in \(.applicationName)", "Show 'example' on 43rd and Broadway in \(.applicationName)", Long story short, Siri is picking up my phrases well, but then asking me to answer whether the value is 44, 50, 84, or 96, instead of just hearing that I said: "Madison and 50th" I've defined streets like this: public let nycStreetDisplayRepresentations: [DisplayRepresentation] = [ DisplayRepresentation(title: "26", synonyms: ["twenty six", "twenty sixth"]), DisplayRepresentation(title: "40", synonyms: ["forty", "fortieth"]), DisplayRepresentation(title: "43", synonyms: ["forty three", "forty third"]), DisplayRepresentation(title: "48", synonyms: ["forty eight", "forty eighth"]), DisplayRepresentation(title: "50", synonyms: ["fifty", "fiftieth"]), DisplayRepresentation(title: "54", synonyms: ["fifty four", "fifty fourth"]), DisplayRepresentation(title: "55", synonyms: ["fifty five", "fifty fifth", "five five"]), DisplayRepresentation(title: "63", synonyms: ["sixty three", "sixty third"]), DisplayRepresentation(title: "67", synonyms: ["sixty seven", "sixty seventh"]), DisplayRepresentation(title: "75", synonyms: ["seventy five", "seventy fifth"]), DisplayRepresentation(title: "77", synonyms: ["seventy seven", "seventy seventh", "seven seven"]), DisplayRepresentation(title: "84", synonyms: ["eighty four", "eighty fourth"]), DisplayRepresentation(title: "85", synonyms: ["eighty five", "eighty fifth"]), DisplayRepresentation(title: "86", synonyms: ["eighty six", "eighty sixth"]), DisplayRepresentation(title: "95", synonyms: ["ninety five", "ninety fifth"]), DisplayRepresentation(title: "96", synonyms: ["ninety six", "ninety sixth"]) ] And an entity like this: public struct MadisonThings: AppEntity, Identifiable { public let id: Int public var displayRepresentation: DisplayRepresentation { nycStreetDisplayRepresentations[id-1] } public static var typeDisplayRepresentation = TypeDisplayRepresentation( name: "Madison" , synonyms: ["Madison Ave"] ) public static var defaultQuery = MadisonTHingsQuery() } An EntityQuery: public struct MadisonThingsQuery: EntityQuery, EntityStringQuery { ...attempts to find synonyms in the phrase } Intent: public struct ShowMadisonThingsIntent: AppIntent { { @Parameter( title: "Street Number", description: "The street number (e.g., 5, 12, 44)" ) var streetNumber: MadisonThings // I have also tried just plain 'int' to no avail } And finally the shortcuts: static var appShortcuts: [AppShortcut] { return [ AppShortcut( intent: ShowMadisonThingsIntent(), phrases: [ "Show example for Madison and \(\.$streetNumber) in \(.applicationName)", "Show example on \(\.$streetNumber) in \(.applicationName)", ... This should be very simple, I think, and I've tried multiple ways of extracting numbers from my speech, but for some reason this is not working well at all. Thanks for your help!

I'm trying to create an App Intent that takes in an image and copies it to a destination within my app. Sure, this can be done with shortcuts, but that's lame, I want siri to grab an image off the screen and invoke the action! Without an app schema, it doesn't seem like siri can tell what's possible in the app, so I'm trying to find a schema that fits? I've started trying to do this with the photos.addAssetsToAlbum App Intent schema, but it doesn't seem like Siri is onboard with the concept. It's replied many times that it is unable to add things to my App and that I should try via other means like opening my app and importing directly. Should be I be trying this with a different intent schema? I was thinking about files.open, but documentation says that Schema is not available to Siri.

I have an app that controls speakers. Among others it has an action to change various settings. The intent has three parameters, setting, value (dependent on the setting selected) and speaker. Is it possible for Siri to understand phrases such as 'Turn On Night Mode in Bedroom' or 'Set Bass to 2 in Bedroom'?

How does next-gen Siri handle semantic or semantic-adjacent parameter resolution when using EntityQuery? For example, if a user says 'Siri, show memory tags related to auth errors' and the exact tag is auth-handler-failure: Does the App Intents framework support passing the raw query string (from natural language) directly to our search/recall logic within the intent, or does it try to map it strictly to pre-registered AppEntity lists? How do we configure an AppEntity to support synonyms or regex-like string match resolution directly within Siri's offline/online parsing?

Is there any new functionality in WebKit for bridging WebViews with the AppIntents framework? For example, is there a simple way to create an IndexedEntity whose content is provided by a WKWebView? Also, if I use appEntityIdentifer in a View that contains a WKWebView, will Siri see the web view’s content?

I'm trying to get my head around adopting AppEntity schemas. Is it OK to shoehorn a closely-related concept from our apps into AppEntity schemas, even if they'r not a perfect fit? For e.g. I have 'tasks' in my app, that are related to a 'person' entity. Would it work to adopt a schema like @AppEntity(schema: .reminders.reminder) struct TaskEntity: IndexedEntity { var personForTask: PersonEntity ... fill in other required fields } Most of the fields are optional or arrays, so it could work, but one of the required fields seems to be var list: ListEntity, so I would have to conform to that in some way. Similarly, I would want to adopt the .reminders.createReminder @AppIntent for my app, even though I won't be able to support all the parameters that it can support. Would that be OK as well?

Is there a hard limitation on the size or total number of entities that can be indexed by a single application? Additionally, are there any recommended performance thresholds or best practices when dealing with a large volume of entities (e.g., thousands to ~30K)?

The Keynote/SOTU mentioned that phone hardware wise, only iPhone Air and 17 Pro are getting the highest quality local models. Does this only impact 1st party system features, or does that impact us as well? In other words, will our schema conforming App Intents behave differently on this subset of devices. Or is testing on any Apple Intelligence supported hardware sufficient?

Hi all. I'm building a chat assistant app and want to bring in the new Siri. I can adopt App Intents, but I'm trying to understand how much of the new Siri's functionality, tools, and system access I actually get as a third-party developer. Is App Intents the full extent of what's available to third-party apps, or is more of the new Siri's capabilities and system access exposed to us? Thanks.