Posts under Spatial Computing topic

Post

Replies

Boosts

Views

Activity

What does `ARWorldTrackingTechnique: resource constraints [33]` mean?
We are building an ARKit application targeting iPhone Pro devices on iOS 26.4+. During testing, we consistently observe the following log message at a specific point in our AR session (approximately 90° into a walking arc around a vehicle): ARWorldTrackingTechnique: resource constraints [33] What specifically does resource constraint code [33] represent? Is it a VIO constraint type, a sensor fusion budget limit, or something else entirely? How does it differ from code [1] (which appears to be a general CPU starvation signal for the ARKit sensor fusion thread)? Is there a public API to detect or respond to this specific constraint condition, short of waiting for a full ARCamera.TrackingState downgrade? Was this constraint type introduced or changed in iOS 26.4?
1
0
1.3k
Jun ’26
Creating hexagonal triplanar materials
I'm trying to create a hexagonal triplanar material in Reality Composer Pro 3 using Shader Graph. Could you provide guidance, or an example node setup, for creating a seamless hex-tile texture using triplanar projection? My goal is to apply a hex pattern to 3D objects without visible seams or texture stretching. Are there any sample projects, Shader Graph examples, or recommended workflows for achieving this in Reality Composer Pro? Thanks
1
2
269
Jun ’26
MeshInstancesComponent vs. manual merging
For Vision Pro's tiled GPU, how does MeshInstancesComponent compare to manually merging meshes for draw-call reduction? Does it collapse draw calls similarly while still allowing per-instance transform and material variation? Deciding whether to re-architect a heavy interior scene around it instead of hand-merging in Maya.
1
0
255
Jun ’26
Real Animation Graph with Curves and Keyframes
RCP 3's Animation Graph covers state machines and blending of imported clips. As an animator, is there now any in-editor support for authoring or editing animation curves and keyframes directly — a graph editor or dope sheet — or must all keyframe animation still be authored in a DCC (Maya/Blender) and imported as clips?
1
1
187
Jun ’26
RCP3 Tutorials
RCP 3 adds a lot of artist-facing surface: lightmaps, Animation Graph, Script Graph. Are step-by-step, artist-oriented tutorials planned beyond the API reference docs? A guided onboarding path would help 3D artists adopt these features much faster.
2
2
217
Jun ’26
ARKit object tracking performance
I'm trying to track identified objects in realtime with bounding rects, with no 3D integration, but still has poor update performance. I'm trying to understand how to optimize frame updates. (I'm a new programmer) Using: Foundation, AVFoundation, ARKit, CoreVideo, Vision, CoreImage, CoreVideo with YOLOE-11s object detection currently throttled to 2fps. (target iOS, testing on 16pro) YOLOE-11S CoreML model detects objects with class labels + bounding boxes Labels are matched against ObjectCatalog.json for relevance Matched objects are promoted from blue (detected) to green (identified) Log warnings: ARSession <0x110afdb80>: The delegate of ARSession is retaining 13 ARFrames. The camera will stop delivering camera images if the delegate keeps holding on to too many ARFrames. This could be a threading or memory management issue in the delegate and should be fixed. Skipping integration due to poor slam at time: 619447.208339 vio_initialized(1) map_size(0) tracking_state_is_nominal(0) is_3dof(0) reinitialize_attempts(6) slam_mode(RegularSLAM)
2
0
708
Jun ’26
Draw order between model sort groups
I’m familiar with using ModelSortGroupComponent to control draw order, but how does RealityKit determine the draw order between model sort groups? How can I enforce a specific draw order between two model sort groups that require different depth pass modes and therefore can't be combined into a single model sort group? Are there model bounding box tricks I can safely rely on?
1
0
251
Jun ’26
First-party recording of the composited AR view
Film crews record "takes" in my app: camera feed plus virtual AR content with timecode. On SceneKit I rely on a third-party Metal-layer hook (SCNRecorder), because ReplayKit captures overlay UI and adds a permission prompt — and RealityKit appears to have no capture API at all, making recording a regression in any SceneKit→RealityKit migration. Is there a recommended first-party way to record an AR view to a movie file, or should I file feedback?
1
0
488
Jun ’26
Camera/Player Start
On visionOS the user's viewpoint isn't directly movable, so we currently invert world movement to simulate locomotion — translating the scene root rather than the camera. With RCP 3 / RealityKit this year, is there a supported way to define or animate a starting viewpoint — a "player start" transform — for an immersive scene, or is world-relative transform still the only sanctioned approach?
2
1
163
Jun ’26
How can an in-ImmersiveSpace menu behave like a regular 2D window in visionOS?
I’m building a visionOS app with an ImmersiveSpace, and I want to show a menu or control panel inside that immersive space. The menu would be created as part of the app’s immersive content, for example as a SwiftUI attachment in a RealityView, or as a custom RealityKit entity with UI-like content. What I would like is for this in-space menu to behave more like a regular visionOS 2D window: The user can move the menu naturally. While the menu is being moved, it automatically adjusts its orientation to face the user. It maintains a comfortable apparent size or distance while being repositioned. It avoids awkward angles or unreadable placement. It feels similar to the system-managed behavior of regular 2D windows. My question is: is there a supported way to give an in-ImmersiveSpace menu the same placement and movement behavior as a normal 2D window? More specifically: Is there a built-in component or API that provides window-like movement, billboard-facing behavior, comfortable distance handling, or automatic scaling for custom panels inside an immersive space? If not, is the recommended approach to implement this behavior manually in RealityKit, for example by tracking the user’s head position and updating the panel’s transform? If manual implementation is required, are there recommended comfort guidelines for menu distance, scale, rotation limits, and movement behavior in immersive spaces? Alternatively, is the recommended design to use a regular 2D window or utility panel outside the immersive content, rather than trying to recreate window behavior inside the ImmersiveSpace?
1
0
161
Jun ’26
Runtime import of non-USD formats into RealityKit
My users import their own 3D assets at runtime — OBJ, FBX, DAE, STL, PLY, often with skeletal animation — which we convert in-app to SceneKit scenes via Assimp. RealityKit only loads USD/.reality at runtime. Is the recommended path to hand-build MeshResource, skeletons, and animations from parsed geometry, or is there any supported on-device conversion to USD? This is one of my biggest blockers to leaving SceneKit.
1
2
572
Jun ’26
SceneKit / ARSCNView support
My app is an 8-year-old film-previsualization tool built on ARSCNView, with SceneKit rendering over an ARKit world-tracking session. Given SceneKit's deprecation with critical-bug-only maintenance: does that commitment, and the "ample notice" promise, explicitly cover ARSCNView? Should I plan on existing SceneKit apps continuing to work on new hardware and OS releases for several more years? I have a few blockers stopping me from migrating to RealityKit and I've noticed the lack of big updates to ARKit over the recent years.
1
1
412
Jun ’26
Colocating iPhone and Vision Pro in one shared scene
On iOS I colocate multiple iPhones/iPads by sharing ARWorldMap over MultipeerConnectivity. visionOS 26 added the conceptual equivalent (shared coordinate spaces via SharedCoordinateSpaceProvider) but as far as I can tell it's Vision Pro-to-Vision Pro only, and visionOS has no ARWorldMap, so there's no bridge. Is there any supported way today to put an iPhone and a Vision Pro into one shared coordinate space or should I file feedback for cross-platform colocation? It would be useful to my users, for example, to have someone controlling an AR scene layout on a Vision Pro while using iOS devices to preview and record the scene from various specific angles.
3
1
409
Jun ’26
Simultaneous ProRes Log 422 Capture and ARKit VIO Tracking
Is simultaneous ProRes Log 422 capture and ARKit VIO tracking supported on iPhone? If not today, is this a capability Apple is considering for future SDK releases? I’m interested in capturing high-quality ProRes Log footage and, at the same time, using ARKit for real-time camera pose tracking for synchronization and post-processing workflows.
1
0
370
Jun ’26
Ultra-wide camera in world tracking
My app previews cinema lenses by cropping the AR feed, so I can only simulate lenses tighter than the main camera's ~24mm-equivalent, meaning wider cinema lenses are impossible to accurately preview. ARConfiguration.VideoFormat exposes captureDeviceType, but world-tracking formats only ever list the wide-angle module. Is running world tracking on (or alongside) the ultra-wide camera fundamentally infeasible? Or just a case of not enough Feedback Assistant requests?
2
0
417
Jun ’26
ALS patient accessbility app, camera access request
Hello, I’m developing an Apple Vision Pro app designed to help people with mobility and speech challenges communicate more effectively with their caregivers. The app requires camera access in order to function as intended. I previously submitted a request for the required camera access entitlement by email and received an initial response indicating that the request would be reviewed with a colleague, but I have not received any follow-up since then. My Apple Developer account is registered under an LLC, and I currently have other visionOS apps available on the App Store. Additional documentation for this app is available here: https://revivrstudios.com/stareandshare.html My company is focused on developing Apple Vision Pro apps that improve quality of life for people living with mobility and speech challenges. I believe this app has the potential to provide meaningful support for those users and their caregivers. Could someone advise on the best way to follow up on the camera access request, or confirm whether there is an additional process I should complete? Thank you.
1
0
339
Jun ’26
Recommended visionOS architecture for opt-in nearby exchange triggered by a physical gesture
I’m exploring a visionOS interaction pattern where nearby Apple Vision Pro users have opted in ahead of time, such as when entering a shared venue, and a physical gesture like a handshake triggers a lightweight exchange of user-approved information between their devices. I’m not asking about identifying strangers, accessing raw camera data, or tracking another person without consent. I’m trying to understand the most automatic Apple-supported architecture for this kind of privacy-preserving nearby interaction. What is the recommended approach on visionOS? Specifically: Can a visionOS app use ARKit hand tracking to detect the current user’s own gesture, then combine that with nearby peer discovery or ranging through Nearby Interaction, Network framework, Multipeer Connectivity, Bluetooth, Wi-Fi, or another supported API? Is there a supported way for two nearby Vision Pro devices to exchange a small user-approved payload after prior opt-in, without requiring users to manually start SharePlay or confirm every individual exchange? If SharePlay or Group Activities is the recommended path for shared spatial context, is there a supported alternative for venue-scale or multi-user interactions that should not be limited to a small active SharePlay group? What are the privacy and App Review boundaries for this pattern? Should developers assume the app cannot identify nearby people, observe another person’s body or hands, or trigger an exchange unless both users have explicit opt-in and clear awareness? If a mostly passive gesture-triggered exchange is not supported today, what is the closest Apple-recommended design pattern?
2
0
763
Jun ’26
What does `ARWorldTrackingTechnique: resource constraints [33]` mean?
We are building an ARKit application targeting iPhone Pro devices on iOS 26.4+. During testing, we consistently observe the following log message at a specific point in our AR session (approximately 90° into a walking arc around a vehicle): ARWorldTrackingTechnique: resource constraints [33] What specifically does resource constraint code [33] represent? Is it a VIO constraint type, a sensor fusion budget limit, or something else entirely? How does it differ from code [1] (which appears to be a general CPU starvation signal for the ARKit sensor fusion thread)? Is there a public API to detect or respond to this specific constraint condition, short of waiting for a full ARCamera.TrackingState downgrade? Was this constraint type introduced or changed in iOS 26.4?
Replies
1
Boosts
0
Views
1.3k
Activity
Jun ’26
Creating hexagonal triplanar materials
I'm trying to create a hexagonal triplanar material in Reality Composer Pro 3 using Shader Graph. Could you provide guidance, or an example node setup, for creating a seamless hex-tile texture using triplanar projection? My goal is to apply a hex pattern to 3D objects without visible seams or texture stretching. Are there any sample projects, Shader Graph examples, or recommended workflows for achieving this in Reality Composer Pro? Thanks
Replies
1
Boosts
2
Views
269
Activity
Jun ’26
MeshInstancesComponent vs. manual merging
For Vision Pro's tiled GPU, how does MeshInstancesComponent compare to manually merging meshes for draw-call reduction? Does it collapse draw calls similarly while still allowing per-instance transform and material variation? Deciding whether to re-architect a heavy interior scene around it instead of hand-merging in Maya.
Replies
1
Boosts
0
Views
255
Activity
Jun ’26
Real Animation Graph with Curves and Keyframes
RCP 3's Animation Graph covers state machines and blending of imported clips. As an animator, is there now any in-editor support for authoring or editing animation curves and keyframes directly — a graph editor or dope sheet — or must all keyframe animation still be authored in a DCC (Maya/Blender) and imported as clips?
Replies
1
Boosts
1
Views
187
Activity
Jun ’26
RCP3 Tutorials
RCP 3 adds a lot of artist-facing surface: lightmaps, Animation Graph, Script Graph. Are step-by-step, artist-oriented tutorials planned beyond the API reference docs? A guided onboarding path would help 3D artists adopt these features much faster.
Replies
2
Boosts
2
Views
217
Activity
Jun ’26
ARKit object tracking performance
I'm trying to track identified objects in realtime with bounding rects, with no 3D integration, but still has poor update performance. I'm trying to understand how to optimize frame updates. (I'm a new programmer) Using: Foundation, AVFoundation, ARKit, CoreVideo, Vision, CoreImage, CoreVideo with YOLOE-11s object detection currently throttled to 2fps. (target iOS, testing on 16pro) YOLOE-11S CoreML model detects objects with class labels + bounding boxes Labels are matched against ObjectCatalog.json for relevance Matched objects are promoted from blue (detected) to green (identified) Log warnings: ARSession <0x110afdb80>: The delegate of ARSession is retaining 13 ARFrames. The camera will stop delivering camera images if the delegate keeps holding on to too many ARFrames. This could be a threading or memory management issue in the delegate and should be fixed. Skipping integration due to poor slam at time: 619447.208339 vio_initialized(1) map_size(0) tracking_state_is_nominal(0) is_3dof(0) reinitialize_attempts(6) slam_mode(RegularSLAM)
Replies
2
Boosts
0
Views
708
Activity
Jun ’26
Draw order between model sort groups
I’m familiar with using ModelSortGroupComponent to control draw order, but how does RealityKit determine the draw order between model sort groups? How can I enforce a specific draw order between two model sort groups that require different depth pass modes and therefore can't be combined into a single model sort group? Are there model bounding box tricks I can safely rely on?
Replies
1
Boosts
0
Views
251
Activity
Jun ’26
How can I export from RCP3beta for use with AR Quick Look?
There is no "export" for a scene or project like in RCP2. I tried to select root|world but none of the 3 file formats export components, therefore no interaction is possible. It works inside RCP3 with PLAY. I tried the exported .reality asset in macOS27 and on iPadOS27. No interactions.
Replies
2
Boosts
1
Views
123
Activity
Jun ’26
First-party recording of the composited AR view
Film crews record "takes" in my app: camera feed plus virtual AR content with timecode. On SceneKit I rely on a third-party Metal-layer hook (SCNRecorder), because ReplayKit captures overlay UI and adds a permission prompt — and RealityKit appears to have no capture API at all, making recording a regression in any SceneKit→RealityKit migration. Is there a recommended first-party way to record an AR view to a movie file, or should I file feedback?
Replies
1
Boosts
0
Views
488
Activity
Jun ’26
Camera/Player Start
On visionOS the user's viewpoint isn't directly movable, so we currently invert world movement to simulate locomotion — translating the scene root rather than the camera. With RCP 3 / RealityKit this year, is there a supported way to define or animate a starting viewpoint — a "player start" transform — for an immersive scene, or is world-relative transform still the only sanctioned approach?
Replies
2
Boosts
1
Views
163
Activity
Jun ’26
How can an in-ImmersiveSpace menu behave like a regular 2D window in visionOS?
I’m building a visionOS app with an ImmersiveSpace, and I want to show a menu or control panel inside that immersive space. The menu would be created as part of the app’s immersive content, for example as a SwiftUI attachment in a RealityView, or as a custom RealityKit entity with UI-like content. What I would like is for this in-space menu to behave more like a regular visionOS 2D window: The user can move the menu naturally. While the menu is being moved, it automatically adjusts its orientation to face the user. It maintains a comfortable apparent size or distance while being repositioned. It avoids awkward angles or unreadable placement. It feels similar to the system-managed behavior of regular 2D windows. My question is: is there a supported way to give an in-ImmersiveSpace menu the same placement and movement behavior as a normal 2D window? More specifically: Is there a built-in component or API that provides window-like movement, billboard-facing behavior, comfortable distance handling, or automatic scaling for custom panels inside an immersive space? If not, is the recommended approach to implement this behavior manually in RealityKit, for example by tracking the user’s head position and updating the panel’s transform? If manual implementation is required, are there recommended comfort guidelines for menu distance, scale, rotation limits, and movement behavior in immersive spaces? Alternatively, is the recommended design to use a regular 2D window or utility panel outside the immersive content, rather than trying to recreate window behavior inside the ImmersiveSpace?
Replies
1
Boosts
0
Views
161
Activity
Jun ’26
Runtime import of non-USD formats into RealityKit
My users import their own 3D assets at runtime — OBJ, FBX, DAE, STL, PLY, often with skeletal animation — which we convert in-app to SceneKit scenes via Assimp. RealityKit only loads USD/.reality at runtime. Is the recommended path to hand-build MeshResource, skeletons, and animations from parsed geometry, or is there any supported on-device conversion to USD? This is one of my biggest blockers to leaving SceneKit.
Replies
1
Boosts
2
Views
572
Activity
Jun ’26
SceneKit / ARSCNView support
My app is an 8-year-old film-previsualization tool built on ARSCNView, with SceneKit rendering over an ARKit world-tracking session. Given SceneKit's deprecation with critical-bug-only maintenance: does that commitment, and the "ample notice" promise, explicitly cover ARSCNView? Should I plan on existing SceneKit apps continuing to work on new hardware and OS releases for several more years? I have a few blockers stopping me from migrating to RealityKit and I've noticed the lack of big updates to ARKit over the recent years.
Replies
1
Boosts
1
Views
412
Activity
Jun ’26
DCC round-trip preserving authored data
When I iterate geometry in Maya/Blender after assigning components, prototype overrides, and lightmaps in RCP 3, does re-importing the updated USD preserve those bindings, or do they need reauthoring on each import? What's the recommended round-trip for art that's still changing?
Replies
2
Boosts
2
Views
148
Activity
Jun ’26
Colocating iPhone and Vision Pro in one shared scene
On iOS I colocate multiple iPhones/iPads by sharing ARWorldMap over MultipeerConnectivity. visionOS 26 added the conceptual equivalent (shared coordinate spaces via SharedCoordinateSpaceProvider) but as far as I can tell it's Vision Pro-to-Vision Pro only, and visionOS has no ARWorldMap, so there's no bridge. Is there any supported way today to put an iPhone and a Vision Pro into one shared coordinate space or should I file feedback for cross-platform colocation? It would be useful to my users, for example, to have someone controlling an AR scene layout on a Vision Pro while using iOS devices to preview and record the scene from various specific angles.
Replies
3
Boosts
1
Views
409
Activity
Jun ’26
Simultaneous ProRes Log 422 Capture and ARKit VIO Tracking
Is simultaneous ProRes Log 422 capture and ARKit VIO tracking supported on iPhone? If not today, is this a capability Apple is considering for future SDK releases? I’m interested in capturing high-quality ProRes Log footage and, at the same time, using ARKit for real-time camera pose tracking for synchronization and post-processing workflows.
Replies
1
Boosts
0
Views
370
Activity
Jun ’26
Ultra-wide camera in world tracking
My app previews cinema lenses by cropping the AR feed, so I can only simulate lenses tighter than the main camera's ~24mm-equivalent, meaning wider cinema lenses are impossible to accurately preview. ARConfiguration.VideoFormat exposes captureDeviceType, but world-tracking formats only ever list the wide-angle module. Is running world tracking on (or alongside) the ultra-wide camera fundamentally infeasible? Or just a case of not enough Feedback Assistant requests?
Replies
2
Boosts
0
Views
417
Activity
Jun ’26
Spatial scenes API on iOS
Does iOS 27 support the spatial scenes API (ImagePresentationComponent) so apps like our Art Authority Museum can let users “lean into” scenes? (Feedback FB21882981 submitted last year)
Replies
1
Boosts
0
Views
158
Activity
Jun ’26
ALS patient accessbility app, camera access request
Hello, I’m developing an Apple Vision Pro app designed to help people with mobility and speech challenges communicate more effectively with their caregivers. The app requires camera access in order to function as intended. I previously submitted a request for the required camera access entitlement by email and received an initial response indicating that the request would be reviewed with a colleague, but I have not received any follow-up since then. My Apple Developer account is registered under an LLC, and I currently have other visionOS apps available on the App Store. Additional documentation for this app is available here: https://revivrstudios.com/stareandshare.html My company is focused on developing Apple Vision Pro apps that improve quality of life for people living with mobility and speech challenges. I believe this app has the potential to provide meaningful support for those users and their caregivers. Could someone advise on the best way to follow up on the camera access request, or confirm whether there is an additional process I should complete? Thank you.
Replies
1
Boosts
0
Views
339
Activity
Jun ’26
Recommended visionOS architecture for opt-in nearby exchange triggered by a physical gesture
I’m exploring a visionOS interaction pattern where nearby Apple Vision Pro users have opted in ahead of time, such as when entering a shared venue, and a physical gesture like a handshake triggers a lightweight exchange of user-approved information between their devices. I’m not asking about identifying strangers, accessing raw camera data, or tracking another person without consent. I’m trying to understand the most automatic Apple-supported architecture for this kind of privacy-preserving nearby interaction. What is the recommended approach on visionOS? Specifically: Can a visionOS app use ARKit hand tracking to detect the current user’s own gesture, then combine that with nearby peer discovery or ranging through Nearby Interaction, Network framework, Multipeer Connectivity, Bluetooth, Wi-Fi, or another supported API? Is there a supported way for two nearby Vision Pro devices to exchange a small user-approved payload after prior opt-in, without requiring users to manually start SharePlay or confirm every individual exchange? If SharePlay or Group Activities is the recommended path for shared spatial context, is there a supported alternative for venue-scale or multi-user interactions that should not be limited to a small active SharePlay group? What are the privacy and App Review boundaries for this pattern? Should developers assume the app cannot identify nearby people, observe another person’s body or hands, or trigger an exchange unless both users have explicit opt-in and clear awareness? If a mostly passive gesture-triggered exchange is not supported today, what is the closest Apple-recommended design pattern?
Replies
2
Boosts
0
Views
763
Activity
Jun ’26