Posts under Spatial Computing topic

Post

Replies

Boosts

Views

Activity

WorldAnchor instantly removed when SpatialTrackingSession and ARKitSession run together
Bug: When SpatialTrackingSession and ARKitSession + WorldTrackingProvider are running concurrently, any WorldAnchor added via WorldTrackingProvider.addAnchor() triggers .added followed immediately by .removed—without any user call to removeAnchor(). The anchor never persists in allAnchors. import SwiftUI import RealityKit import ARKit struct ImmersiveView: View { @State private var worldTracking: WorldTrackingProvider? @State private var arSession: ARKitSession? @State private var processWorldTrackingUpdatesTask: Task<Void, Never>? var body: some View { RealityView { content in let configuration = SpatialTrackingSession.Configuration(tracking: [.world]) if let unavailableCapabilities = await SpatialTrackingSession().run(configuration) { if unavailableCapabilities.anchor.contains(.world) { fatalError("World tracking is not available on this device.") } } let worldTracking = WorldTrackingProvider() let arSession = ARKitSession() self.arSession = arSession try! await arSession.run([worldTracking]) self.worldTracking = worldTracking processWorldTrackingUpdatesTask = Task { @MainActor [weak worldTracking] in guard let worldTracking else { return } for await update in worldTracking.anchorUpdates { let worldAnchor = update.anchor switch update.event { case .added: print("Anchor added: \(worldAnchor.id)") case .updated: print("Anchor updated: \(worldAnchor.id)") case .removed: fatalError("Anchor removed unexpectedly — this should not happen in this demo scenario.") } } } } .task { try? await Task.sleep(for: .seconds(3)) guard let worldTracking else { return } do { try await worldTracking.addAnchor(.init(originFromAnchorTransform: Transform.identity.matrix)) } catch { print("Error adding anchor: \(error)") } } } } Expected: Anchors persist until explicitly removed or out of range. Actual: SpatialTrackingSession interferes with WorldTrackingProvider's anchor lifecycle, causing immediate removal. This was originally reported in 2025 (https://developer.apple.com/forums/thread/773351) , but remains unfixed in visionOS 27.0 beta. I've re-filed as FB23420195.
4
1
80
5h
Gaussian Splat demo changed, supports USDZ?
Today, I picked up on my work on the Gaussian Splats demo from Apple I was working with last weekend, and it didn't work (I've updated everything to beta 2 this week). I tried downloading a fresh copy of the demo (in case I screwed mine up), and not only does it work, it can load Gaussian Splats data from a USDZ file. Am I mis-remembering what the demo was like last week vs. this week? This feels like major new functionality I haven't heard mentioned anywhere. Is there any documentation/guidance on how to package up Gaussian Splat data (e.g., originally from a PLY file) into a USDZ file?
1
0
41
15h
Allow External Apps to Remain Visible in an Immersive Space
Many apps utilize ImmersiveSpace simply to render large-scale, custom geometry (such as curved screens, wide-canvas media players, or specialized data visualizations) that cannot be achieved within a standard volumetric window limitations. However, users often still want to multitask—check Messages, reference Safari, access YouTube, etc. Forcing the user to exit the immersive space just to see their other apps breaks their workflow and creates a disjointed user experience. Proposed Solution: I propose adding a property to ImmersionStyle (or a similar API configuration) such as allowSharedSpaceVisibility. When enabled, this would: Allow the ImmersiveSpace to render its custom RealityKit content as usual and prevent the system from automatically hiding the user's existing Shared Space windows. Do me a solid here VisionOS team and please consider this feedback request. 🙏 FB: FB22973973 FB: FB21630340
1
0
61
23h
Backwards Compatibility
No technical justification is strong enough to erase months of developer work through incompatible project migrations without a reliable upgrade path. Reality Composer Pro has evolved quickly, but projects built around timelines, behaviors, transforms, spin, orbit, hide actions, and synchronized audio represent hundreds of hours of creative work. When those projects import with missing actions or stripped timelines, the cost isn’t just technical—it is developer time, money, and eduction that cannot be recovered. Please prioritize migration tools or compatibility layers that preserve existing timeline projects. Backward compatibility is not just about code; it is about respecting the investment developers make in your platform. Xcode 27 Beta 2 Reality Composer Pro 3 Import reports: Missing Transform To Missing Transform By Missing Spin Missing Orbit Missing Hide Missing timelines and behavior Audio synchronization no longer present 60 others. Basically any movement or action. For a new developer, this isn’t the loss of a feature. It’s the loss of two years of blindly learning, invested in a workflow that no longer exists. That work now survives only in memories of an app that never had the chance to be approved. Can i revert to the previous version? Thank you for any replies and the time taken.
2
0
72
1d
How do you get continuous eye-gaze coordinates on visionOS? (accessibility AAC app — all hover APIs return zero events)
Building Dasher (eye-gaze text-entry system for people with motor impairments — ALS, cerebral palsy, etc.) for visionOS. See https://github.com/dasher-project/Dasher-Apple The app needs continuous (x, y) gaze coordinates to steer a zooming alphabet interface. Vision Pro has incredible eye tracking hardware built in — this should be the perfect platform — but I can't get gaze coordinates through any API I've tried. What I've tested (all deliver zero events in a windowed app): UIHoverGestureRecognizer — Apple's docs say it "detects when the user is looking at a view" on visionOS. Added it to a UIView inside UIViewRepresentable. Looked at the view for 2+ minutes on real hardware. Console.app shows zero began/changed/ended callbacks. .onContinuousHover (SwiftUI) — zero .active phases delivered, whether attached to the canvas view or the ZStack root. UIPointerInteraction with a delegate — regionFor callback never fires. The system clearly is tracking gaze — I can see beginScrollingWithRegion events in Console.app with the correct window rect, and the visual hover highlight appears on buttons. But the coordinates are routed to scroll handling, not exposed as a usable pointer position. What does work: pinch-and-drag follows the hand position (where the pinch is in 3D space), not gaze. That's fine for general apps but doesn't help users who need eye-only control. What I'm wondering: Has anyone successfully received continuous gaze coordinates in a windowed (non-immersive) visionOS app? If so, which API? Is UIHoverGestureRecognizer supposed to work on UIView inside UIViewRepresentable, or does it require a pure UIKit view hierarchy? Is there an ARKit provider for eye tracking that I'm missing? I see HandTrackingProvider but no EyeTrackingProvider in the capability list. Could the issue be that my SwiftUI overlay (toolbar VStack) is intercepting hover events before they reach the canvas underneath? If so, how do you make a SwiftUI overlay pass-through for hover? Dasher is a well-established accessibility tool that lets people type using only their eyes. Its the fastest text entry method for eyegaze and head controlled systems. On every other platform (iOS with external trackers, Windows with Tobii, etc.) it reads continuous gaze coordinates and zooms toward wherever the user is looking. Vision Pro should be the best hardware for this — the tracking is already there, we just can't read it. If there's no public API for this, I'd love to know so I can scrap my feedback assistant entry!. If there IS a way and I've missed it, I'd be grateful for a pointer. Thanks!
2
0
54
2d
Production LiDAR capture app affected by ARKit world-tracking drift on iOS 26.4+ (re: thread 827240) mitigation guidance and fix timeline?
Hello, and thank you for taking questions this week. We develop DeepWalk, a production App Store application that captures LiDAR depth, RGB, and ARKit camera poses while field technicians walk continuous outdoor routes (sidewalk and infrastructure assessment). Recorded ARCamera.transform values are used downstream for 3D reconstruction, so world-tracking accuracy directly determines our output quality. Problem: Beginning in early April 2026 and coinciding with the iOS 26.4 rollout across our device fleet, we observe directional, accumulating pose drift in recorded trajectories during normal walking motion. The behavior matches the regression reported in forum thread 827240 (world-tracking drift on LiDAR devices on iOS 26.4+, absent on non-LiDAR devices running identical code). Link to existing post here: https://developer.apple.com/forums/thread/827240 Our configuration is minimal: a default ARWorldTrackingConfiguration with frameSemantics = [.sceneDepth, .smoothedSceneDepth] and no other options set, running on a bare ARSession with a Metal renderer (no SceneKit/RealityKit). We measure drift directly from per-frame ARCamera.transform values, so the displacement appears in ARKit's pose output itself rather than in any rendering layer. The drift accumulates with walking distance and does not occur when the device is stationary. We've observed this across a large variety of devices (iPhone 15/16/17 Pro/Pro Max) on all iOS versions 26.4 or greater. We've also observed this on iPad Pros with updated firmware. Thread 827240 additionally reports that disabling autofocus, HDR, and plane detection and varying scene reconstruction and world alignment did not resolve it. Question Because depth capture is our core function, we cannot avoid engaging the LiDAR scanner, and we're not aware of a public API that adjusts its contribution to world tracking. We'd be grateful for guidance on three questions: Is this a known issue in iOS 26.4 and later? Did the LiDAR contribution to world tracking change in that release? For apps that require sceneDepth, is there any recommended session configuration, video format, or capture-side practice that reduces the drift in the interim? Is a fix planned, and if so, should we expect it in an iOS 26.x update or the iOS 27 cycle? This affects how we manage our deployed device fleet. We're happy to provide trajectory datasets, additional sysdiagnoses, or anything else useful. We have quantified drift-per-distance data across OS versions.
4
1
376
3d
Vision Pro operating limits and battery in 45°C heat
Hello everyone, ​I’m an R&D software engineer researching the Vision Pro for on-site CAD visualization and quality assurance. This requires using the headset outside in the open air in KSA, where ambient daytime temperatures easily hit 45°C (113°F). ​Before I try deploying this for field work, I have a few specific questions: ​Operating Limits: Will the headset function in 45°C ambient heat, or will it immediately trigger a thermal shutdown? ​Throttling: How aggressively does visionOS throttle performance or dim the displays when running demanding spatial apps in this heat? ​Battery: Will the external battery stop supplying power or refuse to charge under these conditions? ​Workarounds: Are there any known best practices or shading accessories for using the Vision Pro outdoors in extreme climates? ​I know the official operating limit is 30°C, so I'm looking for real-world insights from anyone who has pushed these environmental limits. ​Thank you!
0
0
73
5d
Can a RealityKit ShaderGraph read a custom-named USD primvar?
In Reality Composer Pro I try to read it with a Primvar Reader / Geometric Property node (type float), setting the name field to AttributeTest and also to primvars:AttributeTest. In both cases the node only returns its fallback value never the actual per-vertex data. No error or warning. If I instead author the data as a vertex Color Attribute so it exports as primvars:displayColor, the Geometry Color node reads it correctly. So the data pipeline works for the color slot, but not for a custom-named primvar. I actually verified in RCP 3 too i have the same issue, but at least you can see in RCP 3 that the attribute does actually live on the mesh when selecting it. Thanks.
0
0
78
5d
Reality Composer Pro should expose Assistant and project automation through MCP or a CLI
Reality Composer Pro 3 Assistant appears to have useful project-aware tool categories for working with scenes, assets, materials, animation, behavior trees, navigation, skeletons, audio, planning, documentation, and ShaderGraph material creation. It would be very helpful if a supported subset of these capabilities could be exposed to external developer tools through an Apple-provided MCP server, command-line tool, or similar local automation interface. Similar to what's possible with Xcode: https://developer.apple.com/documentation/Xcode/giving-external-agents-access-to-xcode Possible exposed capabilities could include: Querying and modifying scene hierarchy, entities, and components. Inspecting and managing project assets and structure. Working with materials and ShaderGraph/SSL material creation. Reading and editing animation libraries, timelines, skeletons, and rigs. Inspecting behavior trees and navigation data. Reading audio components and playback settings. Querying Reality Composer Pro documentation through the Assistant. Returning structured JSON responses for external tools. Letting external coding agents request project-aware plans or safe editor actions. Expected Result: Reality Composer Pro provides a documented, permission-aware MCP server, CLI, or similar interface that allows external developer tools to safely use supported Assistant/project capabilities. Actual Result: Reality Composer Pro Assistant is available only inside the app UI. External coding agents such as Codex or Claude Code cannot reliably inspect or automate Reality Composer Pro projects except through unsupported package/file manipulation. Why This Matters: Many developers now use coding agents alongside Xcode and Reality Composer Pro. A supported interface would make app development smoother by allowing those tools to coordinate Swift code, RealityKit assets, scene state, materials, animations, and Assistant workflows through an official integration point. Feedback ticket: FB23276072
0
2
112
1w
Reality Composer Pro Assistant should support current provider APIs and latest frontier models
Reality Composer Pro Assistant currently appears limited to a fixed OpenAI model list and the legacy /v1/chat/completions endpoint. On Reality Composer Pro 3.0 (75.0.14.500.1), the OpenAI provider exposes only a small set of models such as gpt-4.1, gpt-4o, gpt-4o-mini, o1, and o3-mini. Newer models available through OpenAI’s current APIs are not usable. It would be nice to add support for current provider APIs, including OpenAI’s /v1/responses endpoint, and allow dynamic model discovery or manual custom model IDs. Feedback ticket: FB23274502
0
0
95
1w
Reality Composer Pro exports to 'reality' file
I had planned on using the new version of Reality Composer Pro to export USDZ files for interactivity, but I noticed it can only export selected objects, and they export to a new '.reality' file type. I can't open these on MacOS26.6 or MacOS27. On the old version of Reality Composer Pro you could export the entire project as a USZ file, which was really cool since it was kind of universal and maintained many of the interactions. is this something that just hasn't been integrated yet?
4
0
260
1w
How complicated should a tap-to-open animation for a box be in RCP3?
Before WWDC I had built a fairly simple tap-to-open suitcase where you could tap on an object inside to activate it. (Play audio, no are, etc) all of this was managed in the timeline, which no longer exists. This was really nice because the usdz file was easily shared as a storytelling tool. in the new version… do I really need a model with a closed state, an open state and an opening and closing animations to do something like this? Is there a transform I can perform on an object with a tap instead of creating a specific animation separately? And then using script graphing and animation graphing? I appreciate the in-depth tutorials to showcase the awesomefeatures, but I’m finding it hard to take the information for something as basic like this out of them. even a simple description of how to tap and open a door would make things easier to understand.
5
1
188
1w
Reality Composer Pro 3 is unable to properly import my .usdz file
Hi, I’m running into an issue when importing a .usdz asset into Reality Composer Pro 3. The exact same asset was working perfectly in Reality Composer Pro 2, and it also works correctly when previewed in Xcode 27.0 beta or loaded directly in RealityKit from code. However, in Reality Composer Pro 3, the asset does not import correctly: The material appears completely broken and looks different from the original asset. The animations are not working. The same .usdz works as expected in Reality Composer Pro 2, including materials and animations. The same .usdz also works as expected in Xcode / RealityKit. I also noticed that the Preview app does not seem to import the asset correctly either. In Preview, the asset has the wrong orientation and it looks like the skeleton and animations may not be recognized properly. Here are a few screenshots showing how the asset appears in each app: Reality Composer Pro 2: asset works correctly, including materials and animations. Xcode 27.0 beta preview: asset works correctly, including materials and animations. Reality Composer Pro 3: material is broken and animations are not working. Preview app: wrong orientation, possible skeleton / animation import issue. I filed a ticket FB23214120 that contains an example Xcode project and RCP2 & RCP3 projects. Is there anything specific that changed in the USDZ import pipeline in Composer Pro 3 that could explain this behavior? Thanks!
2
2
204
1w
visionOS nearby SharePlay blocks local microphone capture
I created a minimal sample project that reproduces a microphone issue with nearby SharePlay / Window Sharing on Apple Vision Pro. Sample project: https://github.com/JerryNee/mic-demo Screen recording: https://youtu.be/aMj_A_leJWU Issue I first implemented SharePlay through FaceTime and found that local microphone capture was unavailable. That seemed understandable because FaceTime itself uses the microphone. I then moved the app to a nearby SharePlay / Window Sharing session specifically to avoid a FaceTime call, but local microphone capture still fails. Local microphone capture works before starting nearby SharePlay. After starting a nearby SharePlay / Window Sharing session with GroupActivities, the same app fails at: try audioEngine.start() This does not require a FaceTime call. It happens with an in-room nearby SharePlay / Window Sharing session. The screen recording also goes silent when the Share Window UI appears. I checked the MP4 audio track with ffmpeg silencedetect; audio becomes silent at about 00:00:22.60, matching the moment nearby sharing starts: silence_start: 22.600612 silence_end: 55.051927 | silence_duration: 32.451315 Environment Two nearby Apple Vision Pro devices Xcode device list: visionOS 27.0 and visionOS 26.5 Xcode 27.0, build 27A5194q Frameworks: GroupActivities, AVFoundation, Speech Entitlement: com.apple.developer.group-session Microphone and speech recognition permissions granted Steps Run the sample on Apple Vision Pro. Tap Start Listening and speak. Transcription works. Tap Stop Listening. Tap Start Nearby Session. Share with another nearby Apple Vision Pro. Tap Start Listening again while the nearby session is active. audioEngine.start() fails. Ending the nearby session makes microphone capture work again. Minimal microphone code let recognizer = SFSpeechRecognizer() let audioEngine = AVAudioEngine() let request = SFSpeechAudioBufferRecognitionRequest() request.shouldReportPartialResults = true let session = AVAudioSession.sharedInstance() try session.setCategory(.record, mode: .measurement, options: [.duckOthers]) try session.setActive(true) let input = audioEngine.inputNode let format = input.outputFormat(forBus: 0) input.installTap(onBus: 0, bufferSize: 1024, format: format) { buffer, _ in request.append(buffer) } audioEngine.prepare() try audioEngine.start() Minimal nearby SharePlay code struct MicDemoActivity: Codable, GroupActivity, Transferable { static let activityIdentifier = "com.example.mic-demo.nearby-mic-repro" static var transferRepresentation: some TransferRepresentation { CodableRepresentation(contentType: .micDemoActivity) } var metadata: GroupActivityMetadata { var metadata = GroupActivityMetadata() metadata.type = .generic metadata.title = "Mic Demo Nearby Session" return metadata } } for await session in MicDemoActivity.sessions() { session.join() } _ = try await MicDemoActivity().activate() Questions Is local microphone capture expected to be unavailable during a nearby SharePlay / Window Sharing session? Is screen recording audio expected to become silent during nearby sharing? What is the recommended way to support nearby shared UI/state while still allowing local voice input?
1
0
95
1w
re-entering website environment fails
I have been working with visionOS 27's website environment features, and I have found when entering a website environment, leaving the website environment, waiting 10 seconds, and then trying to re-enter the website environment, the attempt fails. The thrown error reports: AbortError: Immersive request was superseded by another request Strangely, if I try to immediately re-enter the website environment, I succeed. I have to wait several seconds before trying to re-enter the environment to experience the failure. The workaround is to reload the page. Anyone know why this is occurring or how to avoid it? Anyone know a more graceful workaround than asking the user to reload the page? I have filed a feedback on this: FB23208345
1
0
97
1w
WorldAnchor instantly removed when SpatialTrackingSession and ARKitSession run together
Bug: When SpatialTrackingSession and ARKitSession + WorldTrackingProvider are running concurrently, any WorldAnchor added via WorldTrackingProvider.addAnchor() triggers .added followed immediately by .removed—without any user call to removeAnchor(). The anchor never persists in allAnchors. import SwiftUI import RealityKit import ARKit struct ImmersiveView: View { @State private var worldTracking: WorldTrackingProvider? @State private var arSession: ARKitSession? @State private var processWorldTrackingUpdatesTask: Task<Void, Never>? var body: some View { RealityView { content in let configuration = SpatialTrackingSession.Configuration(tracking: [.world]) if let unavailableCapabilities = await SpatialTrackingSession().run(configuration) { if unavailableCapabilities.anchor.contains(.world) { fatalError("World tracking is not available on this device.") } } let worldTracking = WorldTrackingProvider() let arSession = ARKitSession() self.arSession = arSession try! await arSession.run([worldTracking]) self.worldTracking = worldTracking processWorldTrackingUpdatesTask = Task { @MainActor [weak worldTracking] in guard let worldTracking else { return } for await update in worldTracking.anchorUpdates { let worldAnchor = update.anchor switch update.event { case .added: print("Anchor added: \(worldAnchor.id)") case .updated: print("Anchor updated: \(worldAnchor.id)") case .removed: fatalError("Anchor removed unexpectedly — this should not happen in this demo scenario.") } } } } .task { try? await Task.sleep(for: .seconds(3)) guard let worldTracking else { return } do { try await worldTracking.addAnchor(.init(originFromAnchorTransform: Transform.identity.matrix)) } catch { print("Error adding anchor: \(error)") } } } } Expected: Anchors persist until explicitly removed or out of range. Actual: SpatialTrackingSession interferes with WorldTrackingProvider's anchor lifecycle, causing immediate removal. This was originally reported in 2025 (https://developer.apple.com/forums/thread/773351) , but remains unfixed in visionOS 27.0 beta. I've re-filed as FB23420195.
Replies
4
Boosts
1
Views
80
Activity
5h
Gaussian Splat demo changed, supports USDZ?
Today, I picked up on my work on the Gaussian Splats demo from Apple I was working with last weekend, and it didn't work (I've updated everything to beta 2 this week). I tried downloading a fresh copy of the demo (in case I screwed mine up), and not only does it work, it can load Gaussian Splats data from a USDZ file. Am I mis-remembering what the demo was like last week vs. this week? This feels like major new functionality I haven't heard mentioned anywhere. Is there any documentation/guidance on how to package up Gaussian Splat data (e.g., originally from a PLY file) into a USDZ file?
Replies
1
Boosts
0
Views
41
Activity
15h
Allow External Apps to Remain Visible in an Immersive Space
Many apps utilize ImmersiveSpace simply to render large-scale, custom geometry (such as curved screens, wide-canvas media players, or specialized data visualizations) that cannot be achieved within a standard volumetric window limitations. However, users often still want to multitask—check Messages, reference Safari, access YouTube, etc. Forcing the user to exit the immersive space just to see their other apps breaks their workflow and creates a disjointed user experience. Proposed Solution: I propose adding a property to ImmersionStyle (or a similar API configuration) such as allowSharedSpaceVisibility. When enabled, this would: Allow the ImmersiveSpace to render its custom RealityKit content as usual and prevent the system from automatically hiding the user's existing Shared Space windows. Do me a solid here VisionOS team and please consider this feedback request. 🙏 FB: FB22973973 FB: FB21630340
Replies
1
Boosts
0
Views
61
Activity
23h
Backwards Compatibility
No technical justification is strong enough to erase months of developer work through incompatible project migrations without a reliable upgrade path. Reality Composer Pro has evolved quickly, but projects built around timelines, behaviors, transforms, spin, orbit, hide actions, and synchronized audio represent hundreds of hours of creative work. When those projects import with missing actions or stripped timelines, the cost isn’t just technical—it is developer time, money, and eduction that cannot be recovered. Please prioritize migration tools or compatibility layers that preserve existing timeline projects. Backward compatibility is not just about code; it is about respecting the investment developers make in your platform. Xcode 27 Beta 2 Reality Composer Pro 3 Import reports: Missing Transform To Missing Transform By Missing Spin Missing Orbit Missing Hide Missing timelines and behavior Audio synchronization no longer present 60 others. Basically any movement or action. For a new developer, this isn’t the loss of a feature. It’s the loss of two years of blindly learning, invested in a workflow that no longer exists. That work now survives only in memories of an app that never had the chance to be approved. Can i revert to the previous version? Thank you for any replies and the time taken.
Replies
2
Boosts
0
Views
72
Activity
1d
Will the Chaparral Village sample code be available for download?
Hi, several WWDC26 sessions this year feature the Chaparral Village project. However, I can't find a download for it anywhere. Will the Chaparral Village project be published as downloadable sample code? Thanks!
Replies
2
Boosts
0
Views
223
Activity
2d
How do you get continuous eye-gaze coordinates on visionOS? (accessibility AAC app — all hover APIs return zero events)
Building Dasher (eye-gaze text-entry system for people with motor impairments — ALS, cerebral palsy, etc.) for visionOS. See https://github.com/dasher-project/Dasher-Apple The app needs continuous (x, y) gaze coordinates to steer a zooming alphabet interface. Vision Pro has incredible eye tracking hardware built in — this should be the perfect platform — but I can't get gaze coordinates through any API I've tried. What I've tested (all deliver zero events in a windowed app): UIHoverGestureRecognizer — Apple's docs say it "detects when the user is looking at a view" on visionOS. Added it to a UIView inside UIViewRepresentable. Looked at the view for 2+ minutes on real hardware. Console.app shows zero began/changed/ended callbacks. .onContinuousHover (SwiftUI) — zero .active phases delivered, whether attached to the canvas view or the ZStack root. UIPointerInteraction with a delegate — regionFor callback never fires. The system clearly is tracking gaze — I can see beginScrollingWithRegion events in Console.app with the correct window rect, and the visual hover highlight appears on buttons. But the coordinates are routed to scroll handling, not exposed as a usable pointer position. What does work: pinch-and-drag follows the hand position (where the pinch is in 3D space), not gaze. That's fine for general apps but doesn't help users who need eye-only control. What I'm wondering: Has anyone successfully received continuous gaze coordinates in a windowed (non-immersive) visionOS app? If so, which API? Is UIHoverGestureRecognizer supposed to work on UIView inside UIViewRepresentable, or does it require a pure UIKit view hierarchy? Is there an ARKit provider for eye tracking that I'm missing? I see HandTrackingProvider but no EyeTrackingProvider in the capability list. Could the issue be that my SwiftUI overlay (toolbar VStack) is intercepting hover events before they reach the canvas underneath? If so, how do you make a SwiftUI overlay pass-through for hover? Dasher is a well-established accessibility tool that lets people type using only their eyes. Its the fastest text entry method for eyegaze and head controlled systems. On every other platform (iOS with external trackers, Windows with Tobii, etc.) it reads continuous gaze coordinates and zooms toward wherever the user is looking. Vision Pro should be the best hardware for this — the tracking is already there, we just can't read it. If there's no public API for this, I'd love to know so I can scrap my feedback assistant entry!. If there IS a way and I've missed it, I'd be grateful for a pointer. Thanks!
Replies
2
Boosts
0
Views
54
Activity
2d
Work with Reality Composer Pro content in Xcode
May I ask if there is a complete source code project for this instructional video that needs to be learned. Work with Reality Composer Pro content in Xcode
Replies
4
Boosts
0
Views
468
Activity
2d
Production LiDAR capture app affected by ARKit world-tracking drift on iOS 26.4+ (re: thread 827240) mitigation guidance and fix timeline?
Hello, and thank you for taking questions this week. We develop DeepWalk, a production App Store application that captures LiDAR depth, RGB, and ARKit camera poses while field technicians walk continuous outdoor routes (sidewalk and infrastructure assessment). Recorded ARCamera.transform values are used downstream for 3D reconstruction, so world-tracking accuracy directly determines our output quality. Problem: Beginning in early April 2026 and coinciding with the iOS 26.4 rollout across our device fleet, we observe directional, accumulating pose drift in recorded trajectories during normal walking motion. The behavior matches the regression reported in forum thread 827240 (world-tracking drift on LiDAR devices on iOS 26.4+, absent on non-LiDAR devices running identical code). Link to existing post here: https://developer.apple.com/forums/thread/827240 Our configuration is minimal: a default ARWorldTrackingConfiguration with frameSemantics = [.sceneDepth, .smoothedSceneDepth] and no other options set, running on a bare ARSession with a Metal renderer (no SceneKit/RealityKit). We measure drift directly from per-frame ARCamera.transform values, so the displacement appears in ARKit's pose output itself rather than in any rendering layer. The drift accumulates with walking distance and does not occur when the device is stationary. We've observed this across a large variety of devices (iPhone 15/16/17 Pro/Pro Max) on all iOS versions 26.4 or greater. We've also observed this on iPad Pros with updated firmware. Thread 827240 additionally reports that disabling autofocus, HDR, and plane detection and varying scene reconstruction and world alignment did not resolve it. Question Because depth capture is our core function, we cannot avoid engaging the LiDAR scanner, and we're not aware of a public API that adjusts its contribution to world tracking. We'd be grateful for guidance on three questions: Is this a known issue in iOS 26.4 and later? Did the LiDAR contribution to world tracking change in that release? For apps that require sceneDepth, is there any recommended session configuration, video format, or capture-side practice that reduces the drift in the interim? Is a fix planned, and if so, should we expect it in an iOS 26.x update or the iOS 27 cycle? This affects how we manage our deployed device fleet. We're happy to provide trajectory datasets, additional sysdiagnoses, or anything else useful. We have quantified drift-per-distance data across OS versions.
Replies
4
Boosts
1
Views
376
Activity
3d
WebXR test for AVPro - Fly anywhere in the world ( using Google 3D Tiles, and headset orientation )
Hi there, I'd appreciate it if someone could test this on their AVPro, and give feedback etc. https://face-flight.com I'm hoping to add WebXR support for as many headsets as possible.
Replies
0
Boosts
0
Views
37
Activity
3d
Surface Snapping in visionOS Simulator
How do I snap a window to a surface in the visionOS Simulator?
Replies
2
Boosts
0
Views
99
Activity
5d
Vision Pro operating limits and battery in 45°C heat
Hello everyone, ​I’m an R&D software engineer researching the Vision Pro for on-site CAD visualization and quality assurance. This requires using the headset outside in the open air in KSA, where ambient daytime temperatures easily hit 45°C (113°F). ​Before I try deploying this for field work, I have a few specific questions: ​Operating Limits: Will the headset function in 45°C ambient heat, or will it immediately trigger a thermal shutdown? ​Throttling: How aggressively does visionOS throttle performance or dim the displays when running demanding spatial apps in this heat? ​Battery: Will the external battery stop supplying power or refuse to charge under these conditions? ​Workarounds: Are there any known best practices or shading accessories for using the Vision Pro outdoors in extreme climates? ​I know the official operating limit is 30°C, so I'm looking for real-world insights from anyone who has pushed these environmental limits. ​Thank you!
Replies
0
Boosts
0
Views
73
Activity
5d
Can a RealityKit ShaderGraph read a custom-named USD primvar?
In Reality Composer Pro I try to read it with a Primvar Reader / Geometric Property node (type float), setting the name field to AttributeTest and also to primvars:AttributeTest. In both cases the node only returns its fallback value never the actual per-vertex data. No error or warning. If I instead author the data as a vertex Color Attribute so it exports as primvars:displayColor, the Geometry Color node reads it correctly. So the data pipeline works for the color slot, but not for a custom-named primvar. I actually verified in RCP 3 too i have the same issue, but at least you can see in RCP 3 that the attribute does actually live on the mesh when selecting it. Thanks.
Replies
0
Boosts
0
Views
78
Activity
5d
Reality Composer Pro should expose Assistant and project automation through MCP or a CLI
Reality Composer Pro 3 Assistant appears to have useful project-aware tool categories for working with scenes, assets, materials, animation, behavior trees, navigation, skeletons, audio, planning, documentation, and ShaderGraph material creation. It would be very helpful if a supported subset of these capabilities could be exposed to external developer tools through an Apple-provided MCP server, command-line tool, or similar local automation interface. Similar to what's possible with Xcode: https://developer.apple.com/documentation/Xcode/giving-external-agents-access-to-xcode Possible exposed capabilities could include: Querying and modifying scene hierarchy, entities, and components. Inspecting and managing project assets and structure. Working with materials and ShaderGraph/SSL material creation. Reading and editing animation libraries, timelines, skeletons, and rigs. Inspecting behavior trees and navigation data. Reading audio components and playback settings. Querying Reality Composer Pro documentation through the Assistant. Returning structured JSON responses for external tools. Letting external coding agents request project-aware plans or safe editor actions. Expected Result: Reality Composer Pro provides a documented, permission-aware MCP server, CLI, or similar interface that allows external developer tools to safely use supported Assistant/project capabilities. Actual Result: Reality Composer Pro Assistant is available only inside the app UI. External coding agents such as Codex or Claude Code cannot reliably inspect or automate Reality Composer Pro projects except through unsupported package/file manipulation. Why This Matters: Many developers now use coding agents alongside Xcode and Reality Composer Pro. A supported interface would make app development smoother by allowing those tools to coordinate Swift code, RealityKit assets, scene state, materials, animations, and Assistant workflows through an official integration point. Feedback ticket: FB23276072
Replies
0
Boosts
2
Views
112
Activity
1w
Reality Composer Pro Assistant should support current provider APIs and latest frontier models
Reality Composer Pro Assistant currently appears limited to a fixed OpenAI model list and the legacy /v1/chat/completions endpoint. On Reality Composer Pro 3.0 (75.0.14.500.1), the OpenAI provider exposes only a small set of models such as gpt-4.1, gpt-4o, gpt-4o-mini, o1, and o3-mini. Newer models available through OpenAI’s current APIs are not usable. It would be nice to add support for current provider APIs, including OpenAI’s /v1/responses endpoint, and allow dynamic model discovery or manual custom model IDs. Feedback ticket: FB23274502
Replies
0
Boosts
0
Views
95
Activity
1w
Reality Composer Pro exports to 'reality' file
I had planned on using the new version of Reality Composer Pro to export USDZ files for interactivity, but I noticed it can only export selected objects, and they export to a new '.reality' file type. I can't open these on MacOS26.6 or MacOS27. On the old version of Reality Composer Pro you could export the entire project as a USZ file, which was really cool since it was kind of universal and maintained many of the interactions. is this something that just hasn't been integrated yet?
Replies
4
Boosts
0
Views
260
Activity
1w
How complicated should a tap-to-open animation for a box be in RCP3?
Before WWDC I had built a fairly simple tap-to-open suitcase where you could tap on an object inside to activate it. (Play audio, no are, etc) all of this was managed in the timeline, which no longer exists. This was really nice because the usdz file was easily shared as a storytelling tool. in the new version… do I really need a model with a closed state, an open state and an opening and closing animations to do something like this? Is there a transform I can perform on an object with a tap instead of creating a specific animation separately? And then using script graphing and animation graphing? I appreciate the in-depth tutorials to showcase the awesomefeatures, but I’m finding it hard to take the information for something as basic like this out of them. even a simple description of how to tap and open a door would make things easier to understand.
Replies
5
Boosts
1
Views
188
Activity
1w
Reality Composer Pro 3 is unable to properly import my .usdz file
Hi, I’m running into an issue when importing a .usdz asset into Reality Composer Pro 3. The exact same asset was working perfectly in Reality Composer Pro 2, and it also works correctly when previewed in Xcode 27.0 beta or loaded directly in RealityKit from code. However, in Reality Composer Pro 3, the asset does not import correctly: The material appears completely broken and looks different from the original asset. The animations are not working. The same .usdz works as expected in Reality Composer Pro 2, including materials and animations. The same .usdz also works as expected in Xcode / RealityKit. I also noticed that the Preview app does not seem to import the asset correctly either. In Preview, the asset has the wrong orientation and it looks like the skeleton and animations may not be recognized properly. Here are a few screenshots showing how the asset appears in each app: Reality Composer Pro 2: asset works correctly, including materials and animations. Xcode 27.0 beta preview: asset works correctly, including materials and animations. Reality Composer Pro 3: material is broken and animations are not working. Preview app: wrong orientation, possible skeleton / animation import issue. I filed a ticket FB23214120 that contains an example Xcode project and RCP2 & RCP3 projects. Is there anything specific that changed in the USDZ import pipeline in Composer Pro 3 that could explain this behavior? Thanks!
Replies
2
Boosts
2
Views
204
Activity
1w
visionOS nearby SharePlay blocks local microphone capture
I created a minimal sample project that reproduces a microphone issue with nearby SharePlay / Window Sharing on Apple Vision Pro. Sample project: https://github.com/JerryNee/mic-demo Screen recording: https://youtu.be/aMj_A_leJWU Issue I first implemented SharePlay through FaceTime and found that local microphone capture was unavailable. That seemed understandable because FaceTime itself uses the microphone. I then moved the app to a nearby SharePlay / Window Sharing session specifically to avoid a FaceTime call, but local microphone capture still fails. Local microphone capture works before starting nearby SharePlay. After starting a nearby SharePlay / Window Sharing session with GroupActivities, the same app fails at: try audioEngine.start() This does not require a FaceTime call. It happens with an in-room nearby SharePlay / Window Sharing session. The screen recording also goes silent when the Share Window UI appears. I checked the MP4 audio track with ffmpeg silencedetect; audio becomes silent at about 00:00:22.60, matching the moment nearby sharing starts: silence_start: 22.600612 silence_end: 55.051927 | silence_duration: 32.451315 Environment Two nearby Apple Vision Pro devices Xcode device list: visionOS 27.0 and visionOS 26.5 Xcode 27.0, build 27A5194q Frameworks: GroupActivities, AVFoundation, Speech Entitlement: com.apple.developer.group-session Microphone and speech recognition permissions granted Steps Run the sample on Apple Vision Pro. Tap Start Listening and speak. Transcription works. Tap Stop Listening. Tap Start Nearby Session. Share with another nearby Apple Vision Pro. Tap Start Listening again while the nearby session is active. audioEngine.start() fails. Ending the nearby session makes microphone capture work again. Minimal microphone code let recognizer = SFSpeechRecognizer() let audioEngine = AVAudioEngine() let request = SFSpeechAudioBufferRecognitionRequest() request.shouldReportPartialResults = true let session = AVAudioSession.sharedInstance() try session.setCategory(.record, mode: .measurement, options: [.duckOthers]) try session.setActive(true) let input = audioEngine.inputNode let format = input.outputFormat(forBus: 0) input.installTap(onBus: 0, bufferSize: 1024, format: format) { buffer, _ in request.append(buffer) } audioEngine.prepare() try audioEngine.start() Minimal nearby SharePlay code struct MicDemoActivity: Codable, GroupActivity, Transferable { static let activityIdentifier = "com.example.mic-demo.nearby-mic-repro" static var transferRepresentation: some TransferRepresentation { CodableRepresentation(contentType: .micDemoActivity) } var metadata: GroupActivityMetadata { var metadata = GroupActivityMetadata() metadata.type = .generic metadata.title = "Mic Demo Nearby Session" return metadata } } for await session in MicDemoActivity.sessions() { session.join() } _ = try await MicDemoActivity().activate() Questions Is local microphone capture expected to be unavailable during a nearby SharePlay / Window Sharing session? Is screen recording audio expected to become silent during nearby sharing? What is the recommended way to support nearby shared UI/state while still allowing local voice input?
Replies
1
Boosts
0
Views
95
Activity
1w
How to make RCP start playing Timeline from scratch when receiving OnNotification.
How to make RCP start playing Timeline from scratch when receiving OnNotification. At present, I have only played the timeline once after receiving the OnNotification, and cannot replay it again after receiving the OnNotification again.
Replies
1
Boosts
0
Views
108
Activity
1w
re-entering website environment fails
I have been working with visionOS 27's website environment features, and I have found when entering a website environment, leaving the website environment, waiting 10 seconds, and then trying to re-enter the website environment, the attempt fails. The thrown error reports: AbortError: Immersive request was superseded by another request Strangely, if I try to immediately re-enter the website environment, I succeed. I have to wait several seconds before trying to re-enter the environment to experience the failure. The workaround is to reload the page. Anyone know why this is occurring or how to avoid it? Anyone know a more graceful workaround than asking the user to reload the page? I have filed a feedback on this: FB23208345
Replies
1
Boosts
0
Views
97
Activity
1w