RealityKit

RSS for tag

Simulate and render 3D content for use in your augmented reality apps using RealityKit.

Posts under RealityKit tag

200 Posts

Post

Replies

Boosts

Views

Activity

WorldAnchor instantly removed when SpatialTrackingSession and ARKitSession run together
Bug: When SpatialTrackingSession and ARKitSession + WorldTrackingProvider are running concurrently, any WorldAnchor added via WorldTrackingProvider.addAnchor() triggers .added followed immediately by .removed—without any user call to removeAnchor(). The anchor never persists in allAnchors. import SwiftUI import RealityKit import ARKit struct ImmersiveView: View { @State private var worldTracking: WorldTrackingProvider? @State private var arSession: ARKitSession? @State private var processWorldTrackingUpdatesTask: Task<Void, Never>? var body: some View { RealityView { content in let configuration = SpatialTrackingSession.Configuration(tracking: [.world]) if let unavailableCapabilities = await SpatialTrackingSession().run(configuration) { if unavailableCapabilities.anchor.contains(.world) { fatalError("World tracking is not available on this device.") } } let worldTracking = WorldTrackingProvider() let arSession = ARKitSession() self.arSession = arSession try! await arSession.run([worldTracking]) self.worldTracking = worldTracking processWorldTrackingUpdatesTask = Task { @MainActor [weak worldTracking] in guard let worldTracking else { return } for await update in worldTracking.anchorUpdates { let worldAnchor = update.anchor switch update.event { case .added: print("Anchor added: \(worldAnchor.id)") case .updated: print("Anchor updated: \(worldAnchor.id)") case .removed: fatalError("Anchor removed unexpectedly — this should not happen in this demo scenario.") } } } } .task { try? await Task.sleep(for: .seconds(3)) guard let worldTracking else { return } do { try await worldTracking.addAnchor(.init(originFromAnchorTransform: Transform.identity.matrix)) } catch { print("Error adding anchor: \(error)") } } } } Expected: Anchors persist until explicitly removed or out of range. Actual: SpatialTrackingSession interferes with WorldTrackingProvider's anchor lifecycle, causing immediate removal. This was originally reported in 2025 (https://developer.apple.com/forums/thread/773351) , but remains unfixed in visionOS 27.0 beta. I've re-filed as FB23420195.
4
1
80
4h
Allow External Apps to Remain Visible in an Immersive Space
Many apps utilize ImmersiveSpace simply to render large-scale, custom geometry (such as curved screens, wide-canvas media players, or specialized data visualizations) that cannot be achieved within a standard volumetric window limitations. However, users often still want to multitask—check Messages, reference Safari, access YouTube, etc. Forcing the user to exit the immersive space just to see their other apps breaks their workflow and creates a disjointed user experience. Proposed Solution: I propose adding a property to ImmersionStyle (or a similar API configuration) such as allowSharedSpaceVisibility. When enabled, this would: Allow the ImmersiveSpace to render its custom RealityKit content as usual and prevent the system from automatically hiding the user's existing Shared Space windows. Do me a solid here VisionOS team and please consider this feedback request. 🙏 FB: FB22973973 FB: FB21630340
1
0
61
23h
Reality Kit 3 large scale practices - what is under the hood
Hello guys I want to use Reality Kit 3 for none AR games. Coming from unreal (7y) I have a few technical questions if i may please: (all questions are assuming my game will handle 50-500 enemies on screen) How does RTK3 handle everything under the hood? its easy to create a struct that hold multiple floats as a data blocks for the system to handle (like hp, dmg, ect), but, what happen when i want generic ASSET data (materials, meshes) ? What if i want to construct a material and hand a cheap id to the processor (system) ? while make sure i packed it nicely to the cpu's cache line . There is not much in the documentation to explain the under the hood architecture of the system for me to make educated decision abut my code. i try to avoid reference bloat in my game, and make sure the system does not coupled new material or new mesh every time it want to perform a generic command on the entity (in 60 fps) For example, in unreal mass, as bloated and over complicated it is - its pretty clear to me what is construction phase, what is the recommended way to generate my data blocks (called fragments there) and how exactly to use them in the system so its all tightly packed At this moment my only assumption is to create a 3 layers custom system to make sure its all running properly: The builder of the pool - it will do the " let shinyMetal = SimpleMaterial(color: ... ect" and save it on some dictionary map the data block - just a simple struct that hold a thin reference to the pool that can be use as a component in the ECS the system - The RTK3 built is system real time . can use the struct as a component and even swap the id in real time is this safe consider how RTK3 build under the hood? or is this even slower ? i have very little to go by. if reality kit engineer can reply and help me here it would be awesome :) Thank you so much for reading guys ! was kinda long. cheers
0
1
74
2d
MaterialX definitions download link is broken in ShaderGraph documentation
The link to download “MaterialX definitions” is broken on the main page of ShaderGraph documentation in the Interoperability section. See here: https://developer.apple.com/documentation/ShaderGraph#Interoperability Link in the page is https://developer.apple.com/augmented-reality/realitykit/files/MaterialX-definitions.zip but leads nowhere now. Since Reality Composer Pro 3 has been released, it would be good to have these definitions available to download again and to know if it’s still MaterialX 1.38 being used in the updated RCP & RealityKit. Opened a ticket about this: FB23258836
1
0
150
1w
Reality Composer Pro 3 is unable to properly import my .usdz file
Hi, I’m running into an issue when importing a .usdz asset into Reality Composer Pro 3. The exact same asset was working perfectly in Reality Composer Pro 2, and it also works correctly when previewed in Xcode 27.0 beta or loaded directly in RealityKit from code. However, in Reality Composer Pro 3, the asset does not import correctly: The material appears completely broken and looks different from the original asset. The animations are not working. The same .usdz works as expected in Reality Composer Pro 2, including materials and animations. The same .usdz also works as expected in Xcode / RealityKit. I also noticed that the Preview app does not seem to import the asset correctly either. In Preview, the asset has the wrong orientation and it looks like the skeleton and animations may not be recognized properly. Here are a few screenshots showing how the asset appears in each app: Reality Composer Pro 2: asset works correctly, including materials and animations. Xcode 27.0 beta preview: asset works correctly, including materials and animations. Reality Composer Pro 3: material is broken and animations are not working. Preview app: wrong orientation, possible skeleton / animation import issue. I filed a ticket FB23214120 that contains an example Xcode project and RCP2 & RCP3 projects. Is there anything specific that changed in the USDZ import pipeline in Composer Pro 3 that could explain this behavior? Thanks!
2
2
203
1w
EnvironmentBlendingComponent(.occluded(by: .surroundings)) culling entities entirely?
Hi there! While building my visionOS app, I’ve encountered some strange behaviour with EnvironmentBlendingComponent. Entities using .occluded(by: .surroundings) are culled entirely whenever a Mac Virtual Display or Apple Immersive Environment is behind them, regardless of the entities’ actual depth, and even when the environment is set to coexist. This feels like it could be a bug, although I’m fairly new to visionOS development, so I may be missing something! In a .mixed immersive space with plain passthrough, the component behaves as I would expect: an entity is occluded accurately when a real-world object is placed in front of it. However, when either of the following is visible behind the entity: a Mac Virtual Display window; or an Apple Immersive Environment enabled using the Digital Crown, the entity carrying EnvironmentBlendingComponent disappears completely. This happens even when the entity is physically closer to the viewer than the virtual surface. An otherwise identical entity without the component remains visible. Moving the virtual surface out of the line of sight, disabling the Immersive Environment, or removing the component immediately makes the entity visible again. Is it intended that .occluded(by: .surroundings) treats Mac Virtual Display and Apple Immersive Environments as occluders? The documentation describes this mode as depth-based occlusion against static “real-world objects”, so it isn’t clear whether these system-rendered virtual surfaces should participate. Even if they are intended to participate, the observed behaviour appears depth-independent: the entire entity is hidden when the virtual surface is behind it, rather than only when the surface is closer to the viewer. That seems inconsistent with the documented depth-based behaviour. I can reproduce this on an M5 Apple Vision Pro running visionOS 26.5 using the following minimal example: import SwiftUI import RealityKit import UIKit @main struct OcclusionReproApp: App { var body: some SwiftUI.Scene { WindowGroup { OcclusionReproLauncher() } ImmersiveSpace(id: "immersive") { RealityView { content in // Control: opaque, no component — proves the bare setup renders. let control = ModelEntity( mesh: .generateSphere(radius: 0.12), materials: [SimpleMaterial(color: .green, isMetallic: false)] ) control.position = [-0.25, 1.2, -1.3] // ~1.3 m ahead, eye height // Under test: identical opaque sphere plus the occlusion component. let occluded = ModelEntity( mesh: .generateSphere(radius: 0.12), materials: [SimpleMaterial(color: .red, isMetallic: false)] ) occluded.position = [0.25, 1.2, -1.3] occluded.components.set( EnvironmentBlendingComponent(preferredBlendingMode: .occluded(by: .surroundings)) ) content.add(control) content.add(occluded) } } .immersionStyle(selection: .constant(.mixed), in: .mixed) // Let the app's mixed space coexist with an Apple Immersive Environment, // as visionOS suppresses Environments while an immersive space is open otherwise. // The same bug can be reproduced with just a Mac Virtual Display, but // this provides a second way to reproduce the same behaviour: .immersiveEnvironmentBehavior(.coexist) } } struct OcclusionReproLauncher: View { @Environment(\.openImmersiveSpace) private var openImmersiveSpace var body: some View { Button("Open immersive space") { Task { await openImmersiveSpace(id: "immersive") } } .padding() } } To reproduce (on a Vision Pro, as the Simulator won't provide passthrough): Open the immersive space in plain passthrough. Both spheres should be visible. Place a real object in front of the red sphere. It should be occluded correctly according to depth. Open a Mac Virtual Display or enable an Apple Immersive Environment, with its visible surface behind the red sphere. The red sphere disappears entirely, despite being in front of that surface. The green control sphere remains visible. Move the virtual surface out of view or disable it, and the red sphere reappears. Please let me know if this is expected behaviour, or if I'm doing something wrong - thanks! Jack
3
0
125
1w
Is there a way at all to create a shadow catcher object in RealityKit?
As the title says. I am developing an app for IOS 18, which involves adding a building relatively far away from the user's place (around 10-20m.), and I need to building to show contact shadows with the floor for added realism. Now, I know that RealityKit has automatic contact shadows using GroundingShadowComponent, but this only works if the object is placed in a plane that has been detected by ARKit... and ARKIt doesn't detect floors so far away, so I need to add my own shadows: add an invisible plane acting as a floor below my building, and have it receive shadows and show only the shadows. The problem is that, from what I see, RealityKit has no materials that can do this: UnlitMaterial ignores all lighting, including shadows. SimpleMaterial does display the shadows, but the floor is displayed too. I can make the material almost transparent (setting the opacity to 0.01 or something), but the floor is still visible. OcclusionMaterial with receiveDynamicLighting is a solution, but it doesn't work either. If I declare it using OcclusionMaterial(receivesDynamicLighting: true), all I get is an invisible plane with no contact shadows in it. What do other people do for this? Do people just bake in the contact shadows in the 3D model?
2
0
143
1w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 I have a created a feedback/bug report with ID FB22823733 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
5
0
396
1w
Reconstruction with Object Mask in PhotogrammetrySession Fails .
Reconstruction with Object Mask in PhotogrammetrySession Fails and doesn't complete because of alignment and object masks with thing segmentation even after increasing the mask region . Also the PhotogrammetrySample hangs with EXC_BREAKPOINTwhen I usePhotogrammetrySession(input: [PhotogrammetrySample])withobjectMask set and the Apple Developer Forums report crashes (“HelloPhotogrammetry” / sample+mask), and (EXC_BREAKPOINT` was being observed at ~4 GB) with freezing the execution and Xcode interface
4
0
682
2w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Xcode hangs when I call PhotogrammetrySession(input: [PhotogrammetrySample]) with objectMask set and traps on some devices, see the attached screenshot, it gets to the function and hangs. Even the folder reconstruction also doesn't complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. In this case it failed while object masking was ON, so RealityKit could not find enough consistent feature tracks inside the masked pixels across the image set. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding
4
0
1.7k
2w
Memory leak on processing stereoscopic video frame, makeMutablePixelBuffer()
Hi, I downloaded and ran https://developer.apple.com/documentation/realitykit/rendering-stereoscopic-video-with-realitykit and noticed that memory usage grows linearly. I replaced the sample video with a different 8k side by side video, and the app crashed almost immediately due to memory leak. it looks like the culprit is from makeMutablePixelBuffer() function and the allocated pixelBuffers are not recycled after being used. screenshot is from a physical device.
1
0
524
2w
Lack of documentation on RealityKit
SceneKit was perfect to import 3D assets directly into our app scene. Since it's soft deprecation and the coming out of RealityKit, tutorials or documentation on how to bridge old possibilities are missing, everything seems to be about AR. More specifically, I am not able to find any resources on how to create a scene, load 3D assets, add a light and a camera, and everything within the app itself. No augmented reality. Is it intended to add some support about this? Any guidance would be appreciated.
2
1
195
2w
Reality Composer 3 iOS versions support
I'm building an iOS game with RealityKit + Reality Composer Pro and was hoping to migrate to Reality Composer Pro 3. I created a new project in RCP and used the "link a new xcode project" flow. It built and rendered correctly on iOS 27. However, I encountered two issues when trying to build for iOS 26. The new RealityKitScripting package threw an error indicating <iOS 27 is not supported. I removed the related code, as I do not intend on using any Script Graphs. The iOS 27 scene still loaded correctly after removing this package. Are there any other consequences to removing this package other than not being able to use Script Graphs? Once the scene loading was working on iOS 26, I noticed that no materials built with the Shader Graph would render. Unlit and Physically Based materials render correctly, but Shader Graph materials do not. Even a simple Shader Graph with one Unlit material node input would not work. Is this expected behavior? My main question is, should I not expect to be able to use Reality Composer Pro 3 if I'm going to target an OS version <27?
0
0
204
2w
Handling Animations in Reality Kit
Hello, In the past my apps using animations were unable to use multiple animations. Following online guides, I put together sequences of animations and then used timestamps to playback the animation I wanted. Is there any intention for realitykit to support better selection of animations within our entities? Its somewhat of a limiting factor in what I am willing to do within reality kit. Thanks for the improvements to realitykit
1
0
152
2w
Script Graph runtime JS API
RCP 3 Script Graphs in 27.0 appear to be JavaScript-backed and hot-reloadable. SceneKit historically exposed its scene graph to JavaScript via JavaScriptCore, so Apple has bridged a 3D framework to JS before. Questions about the runtime surface: Is there a public runtime API to evaluate JavaScript against a RealityKit scene, or to drive Script Graph behavior programmatically at runtime — or is the JS layer private SPI used only by RCP-authored, baked graphs? If there's a runtime JS layer, what object model is exposed — which entity/component types are bridged, and is that surface documented or intended to become public? What's the hot-reload mechanism, and is it reachable outside RCP's Preview on Device? Can a script graph be constructed or parameterized at runtime from a string, or is authoring strictly RCP-side with baked output loaded as an asset? We ship JavaScript scripting via JavaScriptCore ourselves, so a public RealityKit-to-JS binding would be a natural way to extend it — trying to understand what's public vs. private and the intended direction.
2
0
130
2w
New Native Gaussian splats vanish up close
I'm building a MR teleoperation app for the Apple Vision Pro. Instead of 2D camera feeds, I render the robot's remote workspace as 3D Gaussian splats, so the operator can perceive depth and inspect objects from any angle by moving their head. That means the operator needs to lean in close to an object, and sometimes move the viewpoint right up to or through it, without the object dropping out of view. Right now when I use the visionOS 27 splat sample out of the box, the splats disappear when the operator gets too close. As the viewpoint approaches a splatted object the entire splat will vanish. This can be solved partially with chunking the splats, but the operator still can't move inside the splat itself. The entire splat vanishing, and chunking helping, makes me think the splat entity is culled once the camera enters its bounds, rather than individual Gaussians being clipped at the near plane. I don't know the renderer's internals, so I may be wrong about the cause. I believe the sample renders in an ImmersiveSpace, not a bounded volume. Does GaussianSplatComponent expose any control over how splats are culled up close (near plane, clipping, or entity bounds), so the splat stays visible as the viewer moves close to or inside it? If there's no such control today, is there a recommended way to let the viewer move through a splat without it disappearing?
2
2
159
2w
How can we occlude a thin metal needle (held in the hand) with virtual content on visionOS?
Hi, We're building a mixed-immersive visionOS app in which the user holds a thin metal tool — a needle — near virtual content. We need the real needle to correctly occlude virtual content: when the needle (and especially its tip) is physically in front of a virtual object, the virtual object must not draw over it. We are solution-agnostic. Whether the right answer is LiDAR / scene-depth based, object-tracking based, hand-tracking based, or something else entirely — we just need needle occlusion to work. Here is what we've ruled out so far: ObjectTrackingProvider — blocked at enrolment. We can't even produce a usable reference object: the object-capture / scanning step doesn't pick up the needle at all (≈1 mm shaft, specular/metallic, almost no surface features). So object tracking never gets off the ground — it fails before runtime. SceneReconstructionProvider + OcclusionMaterial — only usable for large planar surfaces. Occlusion against a wall or floor is clean. But the mesh is too coarse for anything with finer geometry: a chair already gives very rough, ragged occlusion edges, and the needle is far below the mesh resolution — its shaft never appears in the mesh, so it never occludes at all. So this isn't only a needle problem: it's a mesh resolution problem that goes from rough (chair) to nonexistent (needle). Automatic hand occlusion — doesn't cover the tip. The hand is composited in front correctly, but the needle tip protrudes a few centimetres beyond the fingers, and virtual content draws over exactly that part. The needle tip is the part that most needs to occlude correctly, so this is blocking for us. The full Xcode sample project and the screen recording are in this GitHub repo: https://github.com/JerryNee/occlusion-demo (the video is at media/occlusion-demo.mp4). The screenshots below are stills from that recording, which has four stages, all with the same virtual cube: Occlusion off — baseline; the cube draws over everything in the room. Occlusion on, against a wall — the cube is occluded cleanly. Occlusion on, against a chair — the cube is occluded, but with rough, ragged edges. Occlusion on, with a needle — the needle does not occlude the cube at all. The same occlusion path therefore degrades from clean (wall) → rough (chair) → nonexistent (needle) as the real geometry gets finer. Questions / avenues we'd welcome guidance on (any one would unblock us): What is the recommended way to occlude an object this thin on visionOS — and is it possible at all today? Is there app-accessible per-pixel scene depth (analogous to ARFrame.sceneDepth on iOS) that we could use to occlude geometry the reconstruction mesh misses? Can the system's automatic hand / upper-limb occlusion be extended to include a thin instrument held in the hand — i.e. treat a held tool as part of the hand region? The tool has known, rigid geometry, so we considered attaching an OcclusionMaterial proxy mesh and positioning it from hand-tracking joints — but the needle's orientation within the grip is not determined by the hand pose (the same grip can hold the needle at many different angles), so a hand-joint proxy would be misaligned. Is there any supported way to recover the actual 6-DoF pose of a thin tool held in the hand (i.e. something that senses the tool itself, since the hand pose doesn't constrain it)? Is there any way to make object capture / scanning succeed for thin, specular objects, or an alternative enrolment path? Environment: Apple Vision Pro, visionOS 26.2. Full Xcode sample + screen recording: https://github.com/JerryNee/occlusion-demo Even a "not currently possible / known limitation" answer would help us plan our roadmap. Thank you!
2
0
225
2w
PhotogrammetrySession(input: [PhotogrammetrySample])
Please I would really appreciate a timely response to this: Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. Please I would really appreciate a timely response to this
1
0
239
2w
Object Capture broken on iPhone 17 Pro (iOS 26.x): ObjectCaptureSession sensorFailed + PhotogrammetrySession cv3dapi 4004
The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)
1
0
363
3w
WorldAnchor instantly removed when SpatialTrackingSession and ARKitSession run together
Bug: When SpatialTrackingSession and ARKitSession + WorldTrackingProvider are running concurrently, any WorldAnchor added via WorldTrackingProvider.addAnchor() triggers .added followed immediately by .removed—without any user call to removeAnchor(). The anchor never persists in allAnchors. import SwiftUI import RealityKit import ARKit struct ImmersiveView: View { @State private var worldTracking: WorldTrackingProvider? @State private var arSession: ARKitSession? @State private var processWorldTrackingUpdatesTask: Task<Void, Never>? var body: some View { RealityView { content in let configuration = SpatialTrackingSession.Configuration(tracking: [.world]) if let unavailableCapabilities = await SpatialTrackingSession().run(configuration) { if unavailableCapabilities.anchor.contains(.world) { fatalError("World tracking is not available on this device.") } } let worldTracking = WorldTrackingProvider() let arSession = ARKitSession() self.arSession = arSession try! await arSession.run([worldTracking]) self.worldTracking = worldTracking processWorldTrackingUpdatesTask = Task { @MainActor [weak worldTracking] in guard let worldTracking else { return } for await update in worldTracking.anchorUpdates { let worldAnchor = update.anchor switch update.event { case .added: print("Anchor added: \(worldAnchor.id)") case .updated: print("Anchor updated: \(worldAnchor.id)") case .removed: fatalError("Anchor removed unexpectedly — this should not happen in this demo scenario.") } } } } .task { try? await Task.sleep(for: .seconds(3)) guard let worldTracking else { return } do { try await worldTracking.addAnchor(.init(originFromAnchorTransform: Transform.identity.matrix)) } catch { print("Error adding anchor: \(error)") } } } } Expected: Anchors persist until explicitly removed or out of range. Actual: SpatialTrackingSession interferes with WorldTrackingProvider's anchor lifecycle, causing immediate removal. This was originally reported in 2025 (https://developer.apple.com/forums/thread/773351) , but remains unfixed in visionOS 27.0 beta. I've re-filed as FB23420195.
Replies
4
Boosts
1
Views
80
Activity
4h
Allow External Apps to Remain Visible in an Immersive Space
Many apps utilize ImmersiveSpace simply to render large-scale, custom geometry (such as curved screens, wide-canvas media players, or specialized data visualizations) that cannot be achieved within a standard volumetric window limitations. However, users often still want to multitask—check Messages, reference Safari, access YouTube, etc. Forcing the user to exit the immersive space just to see their other apps breaks their workflow and creates a disjointed user experience. Proposed Solution: I propose adding a property to ImmersionStyle (or a similar API configuration) such as allowSharedSpaceVisibility. When enabled, this would: Allow the ImmersiveSpace to render its custom RealityKit content as usual and prevent the system from automatically hiding the user's existing Shared Space windows. Do me a solid here VisionOS team and please consider this feedback request. 🙏 FB: FB22973973 FB: FB21630340
Replies
1
Boosts
0
Views
61
Activity
23h
Reality Kit 3 large scale practices - what is under the hood
Hello guys I want to use Reality Kit 3 for none AR games. Coming from unreal (7y) I have a few technical questions if i may please: (all questions are assuming my game will handle 50-500 enemies on screen) How does RTK3 handle everything under the hood? its easy to create a struct that hold multiple floats as a data blocks for the system to handle (like hp, dmg, ect), but, what happen when i want generic ASSET data (materials, meshes) ? What if i want to construct a material and hand a cheap id to the processor (system) ? while make sure i packed it nicely to the cpu's cache line . There is not much in the documentation to explain the under the hood architecture of the system for me to make educated decision abut my code. i try to avoid reference bloat in my game, and make sure the system does not coupled new material or new mesh every time it want to perform a generic command on the entity (in 60 fps) For example, in unreal mass, as bloated and over complicated it is - its pretty clear to me what is construction phase, what is the recommended way to generate my data blocks (called fragments there) and how exactly to use them in the system so its all tightly packed At this moment my only assumption is to create a 3 layers custom system to make sure its all running properly: The builder of the pool - it will do the " let shinyMetal = SimpleMaterial(color: ... ect" and save it on some dictionary map the data block - just a simple struct that hold a thin reference to the pool that can be use as a component in the ECS the system - The RTK3 built is system real time . can use the struct as a component and even swap the id in real time is this safe consider how RTK3 build under the hood? or is this even slower ? i have very little to go by. if reality kit engineer can reply and help me here it would be awesome :) Thank you so much for reading guys ! was kinda long. cheers
Replies
0
Boosts
1
Views
74
Activity
2d
MaterialX definitions download link is broken in ShaderGraph documentation
The link to download “MaterialX definitions” is broken on the main page of ShaderGraph documentation in the Interoperability section. See here: https://developer.apple.com/documentation/ShaderGraph#Interoperability Link in the page is https://developer.apple.com/augmented-reality/realitykit/files/MaterialX-definitions.zip but leads nowhere now. Since Reality Composer Pro 3 has been released, it would be good to have these definitions available to download again and to know if it’s still MaterialX 1.38 being used in the updated RCP & RealityKit. Opened a ticket about this: FB23258836
Replies
1
Boosts
0
Views
150
Activity
1w
Reality Composer Pro 3 is unable to properly import my .usdz file
Hi, I’m running into an issue when importing a .usdz asset into Reality Composer Pro 3. The exact same asset was working perfectly in Reality Composer Pro 2, and it also works correctly when previewed in Xcode 27.0 beta or loaded directly in RealityKit from code. However, in Reality Composer Pro 3, the asset does not import correctly: The material appears completely broken and looks different from the original asset. The animations are not working. The same .usdz works as expected in Reality Composer Pro 2, including materials and animations. The same .usdz also works as expected in Xcode / RealityKit. I also noticed that the Preview app does not seem to import the asset correctly either. In Preview, the asset has the wrong orientation and it looks like the skeleton and animations may not be recognized properly. Here are a few screenshots showing how the asset appears in each app: Reality Composer Pro 2: asset works correctly, including materials and animations. Xcode 27.0 beta preview: asset works correctly, including materials and animations. Reality Composer Pro 3: material is broken and animations are not working. Preview app: wrong orientation, possible skeleton / animation import issue. I filed a ticket FB23214120 that contains an example Xcode project and RCP2 & RCP3 projects. Is there anything specific that changed in the USDZ import pipeline in Composer Pro 3 that could explain this behavior? Thanks!
Replies
2
Boosts
2
Views
203
Activity
1w
EnvironmentBlendingComponent(.occluded(by: .surroundings)) culling entities entirely?
Hi there! While building my visionOS app, I’ve encountered some strange behaviour with EnvironmentBlendingComponent. Entities using .occluded(by: .surroundings) are culled entirely whenever a Mac Virtual Display or Apple Immersive Environment is behind them, regardless of the entities’ actual depth, and even when the environment is set to coexist. This feels like it could be a bug, although I’m fairly new to visionOS development, so I may be missing something! In a .mixed immersive space with plain passthrough, the component behaves as I would expect: an entity is occluded accurately when a real-world object is placed in front of it. However, when either of the following is visible behind the entity: a Mac Virtual Display window; or an Apple Immersive Environment enabled using the Digital Crown, the entity carrying EnvironmentBlendingComponent disappears completely. This happens even when the entity is physically closer to the viewer than the virtual surface. An otherwise identical entity without the component remains visible. Moving the virtual surface out of the line of sight, disabling the Immersive Environment, or removing the component immediately makes the entity visible again. Is it intended that .occluded(by: .surroundings) treats Mac Virtual Display and Apple Immersive Environments as occluders? The documentation describes this mode as depth-based occlusion against static “real-world objects”, so it isn’t clear whether these system-rendered virtual surfaces should participate. Even if they are intended to participate, the observed behaviour appears depth-independent: the entire entity is hidden when the virtual surface is behind it, rather than only when the surface is closer to the viewer. That seems inconsistent with the documented depth-based behaviour. I can reproduce this on an M5 Apple Vision Pro running visionOS 26.5 using the following minimal example: import SwiftUI import RealityKit import UIKit @main struct OcclusionReproApp: App { var body: some SwiftUI.Scene { WindowGroup { OcclusionReproLauncher() } ImmersiveSpace(id: "immersive") { RealityView { content in // Control: opaque, no component — proves the bare setup renders. let control = ModelEntity( mesh: .generateSphere(radius: 0.12), materials: [SimpleMaterial(color: .green, isMetallic: false)] ) control.position = [-0.25, 1.2, -1.3] // ~1.3 m ahead, eye height // Under test: identical opaque sphere plus the occlusion component. let occluded = ModelEntity( mesh: .generateSphere(radius: 0.12), materials: [SimpleMaterial(color: .red, isMetallic: false)] ) occluded.position = [0.25, 1.2, -1.3] occluded.components.set( EnvironmentBlendingComponent(preferredBlendingMode: .occluded(by: .surroundings)) ) content.add(control) content.add(occluded) } } .immersionStyle(selection: .constant(.mixed), in: .mixed) // Let the app's mixed space coexist with an Apple Immersive Environment, // as visionOS suppresses Environments while an immersive space is open otherwise. // The same bug can be reproduced with just a Mac Virtual Display, but // this provides a second way to reproduce the same behaviour: .immersiveEnvironmentBehavior(.coexist) } } struct OcclusionReproLauncher: View { @Environment(\.openImmersiveSpace) private var openImmersiveSpace var body: some View { Button("Open immersive space") { Task { await openImmersiveSpace(id: "immersive") } } .padding() } } To reproduce (on a Vision Pro, as the Simulator won't provide passthrough): Open the immersive space in plain passthrough. Both spheres should be visible. Place a real object in front of the red sphere. It should be occluded correctly according to depth. Open a Mac Virtual Display or enable an Apple Immersive Environment, with its visible surface behind the red sphere. The red sphere disappears entirely, despite being in front of that surface. The green control sphere remains visible. Move the virtual surface out of view or disable it, and the red sphere reappears. Please let me know if this is expected behaviour, or if I'm doing something wrong - thanks! Jack
Replies
3
Boosts
0
Views
125
Activity
1w
WindowGrop How to customize bending styles?
How should I set the window of WindowGrop to resemble a curved screen style?
Replies
3
Boosts
0
Views
768
Activity
1w
Is there a way at all to create a shadow catcher object in RealityKit?
As the title says. I am developing an app for IOS 18, which involves adding a building relatively far away from the user's place (around 10-20m.), and I need to building to show contact shadows with the floor for added realism. Now, I know that RealityKit has automatic contact shadows using GroundingShadowComponent, but this only works if the object is placed in a plane that has been detected by ARKit... and ARKIt doesn't detect floors so far away, so I need to add my own shadows: add an invisible plane acting as a floor below my building, and have it receive shadows and show only the shadows. The problem is that, from what I see, RealityKit has no materials that can do this: UnlitMaterial ignores all lighting, including shadows. SimpleMaterial does display the shadows, but the floor is displayed too. I can make the material almost transparent (setting the opacity to 0.01 or something), but the floor is still visible. OcclusionMaterial with receiveDynamicLighting is a solution, but it doesn't work either. If I declare it using OcclusionMaterial(receivesDynamicLighting: true), all I get is an invisible plane with no contact shadows in it. What do other people do for this? Do people just bake in the contact shadows in the 3D model?
Replies
2
Boosts
0
Views
143
Activity
1w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 I have a created a feedback/bug report with ID FB22823733 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
Replies
5
Boosts
0
Views
396
Activity
1w
Reconstruction with Object Mask in PhotogrammetrySession Fails .
Reconstruction with Object Mask in PhotogrammetrySession Fails and doesn't complete because of alignment and object masks with thing segmentation even after increasing the mask region . Also the PhotogrammetrySample hangs with EXC_BREAKPOINTwhen I usePhotogrammetrySession(input: [PhotogrammetrySample])withobjectMask set and the Apple Developer Forums report crashes (“HelloPhotogrammetry” / sample+mask), and (EXC_BREAKPOINT` was being observed at ~4 GB) with freezing the execution and Xcode interface
Replies
4
Boosts
0
Views
682
Activity
2w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Xcode hangs when I call PhotogrammetrySession(input: [PhotogrammetrySample]) with objectMask set and traps on some devices, see the attached screenshot, it gets to the function and hangs. Even the folder reconstruction also doesn't complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. In this case it failed while object masking was ON, so RealityKit could not find enough consistent feature tracks inside the masked pixels across the image set. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding
Replies
4
Boosts
0
Views
1.7k
Activity
2w
Memory leak on processing stereoscopic video frame, makeMutablePixelBuffer()
Hi, I downloaded and ran https://developer.apple.com/documentation/realitykit/rendering-stereoscopic-video-with-realitykit and noticed that memory usage grows linearly. I replaced the sample video with a different 8k side by side video, and the app crashed almost immediately due to memory leak. it looks like the culprit is from makeMutablePixelBuffer() function and the allocated pixelBuffers are not recycled after being used. screenshot is from a physical device.
Replies
1
Boosts
0
Views
524
Activity
2w
Lack of documentation on RealityKit
SceneKit was perfect to import 3D assets directly into our app scene. Since it's soft deprecation and the coming out of RealityKit, tutorials or documentation on how to bridge old possibilities are missing, everything seems to be about AR. More specifically, I am not able to find any resources on how to create a scene, load 3D assets, add a light and a camera, and everything within the app itself. No augmented reality. Is it intended to add some support about this? Any guidance would be appreciated.
Replies
2
Boosts
1
Views
195
Activity
2w
Reality Composer 3 iOS versions support
I'm building an iOS game with RealityKit + Reality Composer Pro and was hoping to migrate to Reality Composer Pro 3. I created a new project in RCP and used the "link a new xcode project" flow. It built and rendered correctly on iOS 27. However, I encountered two issues when trying to build for iOS 26. The new RealityKitScripting package threw an error indicating <iOS 27 is not supported. I removed the related code, as I do not intend on using any Script Graphs. The iOS 27 scene still loaded correctly after removing this package. Are there any other consequences to removing this package other than not being able to use Script Graphs? Once the scene loading was working on iOS 26, I noticed that no materials built with the Shader Graph would render. Unlit and Physically Based materials render correctly, but Shader Graph materials do not. Even a simple Shader Graph with one Unlit material node input would not work. Is this expected behavior? My main question is, should I not expect to be able to use Reality Composer Pro 3 if I'm going to target an OS version <27?
Replies
0
Boosts
0
Views
204
Activity
2w
Handling Animations in Reality Kit
Hello, In the past my apps using animations were unable to use multiple animations. Following online guides, I put together sequences of animations and then used timestamps to playback the animation I wanted. Is there any intention for realitykit to support better selection of animations within our entities? Its somewhat of a limiting factor in what I am willing to do within reality kit. Thanks for the improvements to realitykit
Replies
1
Boosts
0
Views
152
Activity
2w
Script Graph runtime JS API
RCP 3 Script Graphs in 27.0 appear to be JavaScript-backed and hot-reloadable. SceneKit historically exposed its scene graph to JavaScript via JavaScriptCore, so Apple has bridged a 3D framework to JS before. Questions about the runtime surface: Is there a public runtime API to evaluate JavaScript against a RealityKit scene, or to drive Script Graph behavior programmatically at runtime — or is the JS layer private SPI used only by RCP-authored, baked graphs? If there's a runtime JS layer, what object model is exposed — which entity/component types are bridged, and is that surface documented or intended to become public? What's the hot-reload mechanism, and is it reachable outside RCP's Preview on Device? Can a script graph be constructed or parameterized at runtime from a string, or is authoring strictly RCP-side with baked output loaded as an asset? We ship JavaScript scripting via JavaScriptCore ourselves, so a public RealityKit-to-JS binding would be a natural way to extend it — trying to understand what's public vs. private and the intended direction.
Replies
2
Boosts
0
Views
130
Activity
2w
New Native Gaussian splats vanish up close
I'm building a MR teleoperation app for the Apple Vision Pro. Instead of 2D camera feeds, I render the robot's remote workspace as 3D Gaussian splats, so the operator can perceive depth and inspect objects from any angle by moving their head. That means the operator needs to lean in close to an object, and sometimes move the viewpoint right up to or through it, without the object dropping out of view. Right now when I use the visionOS 27 splat sample out of the box, the splats disappear when the operator gets too close. As the viewpoint approaches a splatted object the entire splat will vanish. This can be solved partially with chunking the splats, but the operator still can't move inside the splat itself. The entire splat vanishing, and chunking helping, makes me think the splat entity is culled once the camera enters its bounds, rather than individual Gaussians being clipped at the near plane. I don't know the renderer's internals, so I may be wrong about the cause. I believe the sample renders in an ImmersiveSpace, not a bounded volume. Does GaussianSplatComponent expose any control over how splats are culled up close (near plane, clipping, or entity bounds), so the splat stays visible as the viewer moves close to or inside it? If there's no such control today, is there a recommended way to let the viewer move through a splat without it disappearing?
Replies
2
Boosts
2
Views
159
Activity
2w
How can we occlude a thin metal needle (held in the hand) with virtual content on visionOS?
Hi, We're building a mixed-immersive visionOS app in which the user holds a thin metal tool — a needle — near virtual content. We need the real needle to correctly occlude virtual content: when the needle (and especially its tip) is physically in front of a virtual object, the virtual object must not draw over it. We are solution-agnostic. Whether the right answer is LiDAR / scene-depth based, object-tracking based, hand-tracking based, or something else entirely — we just need needle occlusion to work. Here is what we've ruled out so far: ObjectTrackingProvider — blocked at enrolment. We can't even produce a usable reference object: the object-capture / scanning step doesn't pick up the needle at all (≈1 mm shaft, specular/metallic, almost no surface features). So object tracking never gets off the ground — it fails before runtime. SceneReconstructionProvider + OcclusionMaterial — only usable for large planar surfaces. Occlusion against a wall or floor is clean. But the mesh is too coarse for anything with finer geometry: a chair already gives very rough, ragged occlusion edges, and the needle is far below the mesh resolution — its shaft never appears in the mesh, so it never occludes at all. So this isn't only a needle problem: it's a mesh resolution problem that goes from rough (chair) to nonexistent (needle). Automatic hand occlusion — doesn't cover the tip. The hand is composited in front correctly, but the needle tip protrudes a few centimetres beyond the fingers, and virtual content draws over exactly that part. The needle tip is the part that most needs to occlude correctly, so this is blocking for us. The full Xcode sample project and the screen recording are in this GitHub repo: https://github.com/JerryNee/occlusion-demo (the video is at media/occlusion-demo.mp4). The screenshots below are stills from that recording, which has four stages, all with the same virtual cube: Occlusion off — baseline; the cube draws over everything in the room. Occlusion on, against a wall — the cube is occluded cleanly. Occlusion on, against a chair — the cube is occluded, but with rough, ragged edges. Occlusion on, with a needle — the needle does not occlude the cube at all. The same occlusion path therefore degrades from clean (wall) → rough (chair) → nonexistent (needle) as the real geometry gets finer. Questions / avenues we'd welcome guidance on (any one would unblock us): What is the recommended way to occlude an object this thin on visionOS — and is it possible at all today? Is there app-accessible per-pixel scene depth (analogous to ARFrame.sceneDepth on iOS) that we could use to occlude geometry the reconstruction mesh misses? Can the system's automatic hand / upper-limb occlusion be extended to include a thin instrument held in the hand — i.e. treat a held tool as part of the hand region? The tool has known, rigid geometry, so we considered attaching an OcclusionMaterial proxy mesh and positioning it from hand-tracking joints — but the needle's orientation within the grip is not determined by the hand pose (the same grip can hold the needle at many different angles), so a hand-joint proxy would be misaligned. Is there any supported way to recover the actual 6-DoF pose of a thin tool held in the hand (i.e. something that senses the tool itself, since the hand pose doesn't constrain it)? Is there any way to make object capture / scanning succeed for thin, specular objects, or an alternative enrolment path? Environment: Apple Vision Pro, visionOS 26.2. Full Xcode sample + screen recording: https://github.com/JerryNee/occlusion-demo Even a "not currently possible / known limitation" answer would help us plan our roadmap. Thank you!
Replies
2
Boosts
0
Views
225
Activity
2w
PhotogrammetrySession(input: [PhotogrammetrySample])
Please I would really appreciate a timely response to this: Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. Please I would really appreciate a timely response to this
Replies
1
Boosts
0
Views
239
Activity
2w
Object Capture broken on iPhone 17 Pro (iOS 26.x): ObjectCaptureSession sensorFailed + PhotogrammetrySession cv3dapi 4004
The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)
Replies
1
Boosts
0
Views
363
Activity
3w