I’m exploring a visionOS interaction pattern where nearby Apple Vision Pro users have opted in ahead of time, such as when entering a shared venue, and a physical gesture like a handshake triggers a lightweight exchange of user-approved information between their devices. I’m not asking about identifying strangers, accessing raw camera data, or tracking another person without consent. I’m trying to understand the most automatic Apple-supported architecture for this kind of privacy-preserving nearby interaction. What is the recommended approach on visionOS? Specifically: Can a visionOS app use ARKit hand tracking to detect the current user’s own gesture, then combine that with nearby peer discovery or ranging through Nearby Interaction, Network framework, Multipeer Connectivity, Bluetooth, Wi-Fi, or another supported API? Is there a supported way for two nearby Vision Pro devices to exchange a small user-approved payload after prior opt-in, without requiring users to manually start SharePlay or confirm every individual exchange? If SharePlay or Group Activities is the recommended path for shared spatial context, is there a supported alternative for venue-scale or multi-user interactions that should not be limited to a small active SharePlay group? What are the privacy and App Review boundaries for this pattern? Should developers assume the app cannot identify nearby people, observe another person’s body or hands, or trigger an exchange unless both users have explicit opt-in and clear awareness? If a mostly passive gesture-triggered exchange is not supported today, what is the closest Apple-recommended design pattern?

Hi, We're building a mixed-immersive visionOS app in which the user holds a thin metal tool — a needle — near virtual content. We need the real needle to correctly occlude virtual content: when the needle (and especially its tip) is physically in front of a virtual object, the virtual object must not draw over it. We are solution-agnostic. Whether the right answer is LiDAR / scene-depth based, object-tracking based, hand-tracking based, or something else entirely — we just need needle occlusion to work. Here is what we've ruled out so far: ObjectTrackingProvider — blocked at enrolment. We can't even produce a usable reference object: the object-capture / scanning step doesn't pick up the needle at all (≈1 mm shaft, specular/metallic, almost no surface features). So object tracking never gets off the ground — it fails before runtime. SceneReconstructionProvider + OcclusionMaterial — only usable for large planar surfaces. Occlusion against a wall or floor is clean. But the mesh is too coarse for anything with finer geometry: a chair already gives very rough, ragged occlusion edges, and the needle is far below the mesh resolution — its shaft never appears in the mesh, so it never occludes at all. So this isn't only a needle problem: it's a mesh resolution problem that goes from rough (chair) to nonexistent (needle). Automatic hand occlusion — doesn't cover the tip. The hand is composited in front correctly, but the needle tip protrudes a few centimetres beyond the fingers, and virtual content draws over exactly that part. The needle tip is the part that most needs to occlude correctly, so this is blocking for us. The full Xcode sample project and the screen recording are in this GitHub repo: https://github.com/JerryNee/occlusion-demo (the video is at media/occlusion-demo.mp4). The screenshots below are stills from that recording, which has four stages, all with the same virtual cube: Occlusion off — baseline; the cube draws over everything in the room. Occlusion on, against a wall — the cube is occluded cleanly. Occlusion on, against a chair — the cube is occluded, but with rough, ragged edges. Occlusion on, with a needle — the needle does not occlude the cube at all. The same occlusion path therefore degrades from clean (wall) → rough (chair) → nonexistent (needle) as the real geometry gets finer. Questions / avenues we'd welcome guidance on (any one would unblock us): What is the recommended way to occlude an object this thin on visionOS — and is it possible at all today? Is there app-accessible per-pixel scene depth (analogous to ARFrame.sceneDepth on iOS) that we could use to occlude geometry the reconstruction mesh misses? Can the system's automatic hand / upper-limb occlusion be extended to include a thin instrument held in the hand — i.e. treat a held tool as part of the hand region? The tool has known, rigid geometry, so we considered attaching an OcclusionMaterial proxy mesh and positioning it from hand-tracking joints — but the needle's orientation within the grip is not determined by the hand pose (the same grip can hold the needle at many different angles), so a hand-joint proxy would be misaligned. Is there any supported way to recover the actual 6-DoF pose of a thin tool held in the hand (i.e. something that senses the tool itself, since the hand pose doesn't constrain it)? Is there any way to make object capture / scanning succeed for thin, specular objects, or an alternative enrolment path? Environment: Apple Vision Pro, visionOS 26.2. Full Xcode sample + screen recording: https://github.com/JerryNee/occlusion-demo Even a "not currently possible / known limitation" answer would help us plan our roadmap. Thank you!

Hi all: Novice iOS programmer here. I am developing an iPhone app (target: iOS 18) where I need to place a large building a bit far away from the user: around 10-20 meters away. Now, the problem I am having is that ARKit's floor detection usually doesn't reach that far. I solved it by using "existingPlaneInfinite" in the raycast: let raycastResults = arView.session.raycast( ARRaycastQuery(origin: rayOrigin, direction: rayDirection, allowing: .existingPlaneInfinite, alignment: .horizontal) But the problem is that the building appears... but without grounding shadows. I tried enabling GroundingShadowComponent, but without success. After some more testing, I found out that if I place an object in a plane that ARKit is actually detecting, then it will have grounding shadows... but if I place it further away, in the part of the floor that ARKit doesn't detect, then it won't add shadows. It will still place the building at the right height (because it extends the detected plane to the infinite), but it won't add contact shadows with the floor. I can solve this problem using the usual tricks (adding an invisible plane as a shadow catcher, etc.), but I just wanted some confirmation that this is indeed the correct behaviour. Also, are there any changes in this regard in IOS 26?

Hello! Is it possible to turn on hand pass-through (hand cutout, not sure what is the correct name, real hands from the camera) in WebXR when in immersive-vr and hand tracking feature is enabled? I only see my hands when I disable hand tracking feature in WebXR, but then I don't get the joints positions and orientations.

Hello, does anyone have experience using Apple Vision Pro in countries where it has not yet been officially released? I work for a company in Austria, and we are interested in developing internal XR applications for Vision Pro. Since the device is not officially available in Austria, we are considering purchasing it in Germany. My main question is whether it is possible to develop and test Vision Pro apps using an Austrian Apple ID / developer account, or if there are any regional restrictions we should be aware of (e.g., related to App Store access, provisioning, or device functionality). Apple Support was unfortunately unable to provide a definitive answer and recommended asking here. Any insights or experiences would be greatly appreciated. Best regards, Don Appelonie

With the recent announcement that VisionOS 27 would support custom actively tracked tools, via arrays of embedded LEDs, I would just like to confirm if the demo given in this video here: https://developer.apple.com/videos/play/wwdc2026/283/ at minute 1:00 is indeed an actively tracked tool, or if it is passive IR. if it's passive IR (reflectance), are there any examples of how this could be achieved, i.e. can we access the onboard IR cameras directly?

The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)

When switching from RealityView’s .spatialTracking camera mode to .virtual camera mode, the camera’s orientation relative to the scene is preserved permanently with no way to reset to default World-Up orientation. Since .spatialTracking’s camera mode will always have a non-default orientation, switching to .virtual camera mode ensures that the cameras’s ‘UP’ direction will never match the device display’s ‘UP’ direction as is default. This is especially noticeable when using .orbit camera controls, as the orbit’s UP direction matches the scene, not camera, and all rotation directions give unexpected results. Expected: When setting virtual camera mode after using spatialTracking camera mode, either 1. The Virtual Camera orientation returns to default (world up). Or 2. A 'content.camera.resetOrientation()' call is made available which resets the RealityView camera to default orientation. Reality: Switching from .spatialTracking -> .virtual camera mode permanently locks the .virtual camera’s orientation the final frame of the .spatialTracking camera’s rotation (relative to the RealityView content scene). One imperfect workaround is to reset / rebuild the entire RealityView after changing modes (by resetting .id() or otherwise. This is not ideal as it causes everything inside the make closure to rerun, which not only is a performance & time cost, visually incurs a flicker and can also be problematic with managing increasingly complicated views. Another imperfect alternative is to use more than one RealityView - which is not ideal as it incurs double the base ram usage, significantly increases code, and seemingly goes against the intent of being able to change the camera .virtual/.spatatialTracking mode at will. Code Sample: import SwiftUI import RealityKit struct RKSpatialVirtualToggle: View { @State var showAR: Bool = false var body: some View { RealityView { content in let cube = ModelEntity(mesh: .generateBox(size: 0.25), materials: [SimpleMaterial()]) cube.position.z = -1 content.add(cube) content.camera = showAR ? .spatialTracking : .virtual content.cameraTarget = cube } update: { content in content.camera = showAR ? .spatialTracking : .virtual } .realityViewCameraControls(.orbit) VStack{ Spacer() Button("Toggle AR"){ showAR.toggle() } .buttonStyle(.borderedProminent) } } } Xcode Version: Version 26.0 (17A324) iOS Version: iOS 26.5 (23F75) Tested on devices, iPhone 12 Pro, iPhone 15 Pro

Hello, we are evaluating Apple Vision Pro for an AR calibration experiment. Can BarcodeDetectionProvider on visionOS 2.0+ detect a QR code/barcode and provide its 3D position and orientation, such as a transform, so that we can align virtual content with a real-world marker? Does the BarcodeAnchor update continuously when the headset or the barcode moves? Thank you.

When I click on the download link for reality converter, it just sends me here: https://developer.apple.com/augmented-reality/tools/ Is there something I am missing?

attachments: { Attachment(id: "UI_Book_01") { Button { print("Tap...") } label: { Text("Tap me") .padding() .background(.ultraThinMaterial) }.hoverEffect { effect, isActive, _ in if isActive { print("hover...") //Unable to trigger } return effect } } }

Hello Apple Developer Team, We are currently developing an enterprise medical navigation application for Apple Vision Pro and would like to request clarification regarding the currently available visionOS Enterprise APIs related to camera access. Our application scenario involves real-time medical/surgical navigation and instrument tracking in a professional enterprise environment. We would like to better understand the following: How many cameras on Apple Vision Pro are currently accessible through the Enterprise APIs? Which specific cameras are accessible? For example: Main RGB cameras Passthrough cameras Tracking cameras Front-facing cameras Depth sensors LiDAR or structured-light related sensors Are simultaneous multi-camera streams supported? Does the Enterprise API provide: Real-time image frames Camera intrinsic/extrinsic parameters Stereo camera data Depth information Low-latency tracking-related data Are there any restrictions regarding the use of Vision Pro cameras for: Medical navigation Surgical guidance Instrument tracking Enterprise healthcare software Is Apple Vision Pro currently permitted or recommended for medical enterprise spatial-navigation workflows under the Enterprise APIs? We would greatly appreciate any official clarification regarding the current capabilities and limitations of camera access on Apple Vision Pro for enterprise medical applications.

Hello, I am doing a personal research project around ARKit camera frames and camera calibration data. For each accepted ARFrame, I export: ARFrame.timestamp ARFrame.camera.imageResolution ARFrame.camera.intrinsics ARFrame.capturedImage The target video mode is 4K / 30 FPS. In my test captures, I noticed two things: ARFrame.camera.intrinsics is not constant across the recording. Some neighboring frames from ARFrame.capturedImage have noticeably different sharpness: one frame can look sharp, while the next frame can look blurred. Regarding intrinsics, I observe small per-frame changes in fx/fy/cx/cy. For example, in one session: fx/fy changed by approximately 36 px cx changed by approximately 1.5 px cy changed by approximately 2.3 px I understand the standard meaning of the intrinsic matrix: fx / fy = focal length in pixels cx / cy = principal point in pixels relative to the image/reference frame My main question is not about the definition of these values, but about how to correctly interpret their changes over time. In particular, I am interested in cx/cy, because the principal point is important for measurements. I would like to understand whether a change in cx/cy corresponds to a meaningful update of the camera model for the current frame, and what physical or camera-pipeline process may cause this change. Questions about ARFrame.camera.intrinsics: Are ARFrame.camera.intrinsics values expected to change between frames during a single ARWorldTrackingConfiguration session? If fx/fy changes over time, can this reflect autofocus / focus breathing, internal camera calibration updates, digital crop/scaling, stabilization-related mapping, or other camera pipeline changes? If cx/cy changes over time, should I interpret this as an updated principal point for the current ARFrame image/reference frame? Does ARKit update cx/cy to account for any internal crop, scaling, stabilization, virtual camera behavior, lens movement, or calibration changes? Is there a known physical interpretation for small cx/cy changes reported by ARFrame.camera.intrinsics, or should these values simply be treated as ARKit’s best current camera model for that frame? In an ARKit-first pipeline where focus/exposure/stabilization are managed by ARKit/camera pipeline, is the recommended approach to store ARFrame.camera.intrinsics per frame together with ARFrame.timestamp and ARFrame.camera.imageResolution? Is there any public API that exposes the reason for intrinsics changes, such as focus position, stabilization transform, crop transform, active camera constituent, lens movement, or per-frame calibration update reason? I also have a related question about ARFrame.capturedImage frame quality. In some recordings, neighboring frames can have noticeably different sharpness. For example, one frame can look sharp, while the next frame can be visibly blurred, even though the scene and camera movement are continuous. Questions about frame sharpness / camera pipeline behavior: Is frame-to-frame sharpness variation expected when using ARFrame.capturedImage as the video source? Can ARKit change focus, exposure duration, ISO, white balance, crop/scaling, or other camera pipeline parameters between frames during an ARWorldTrackingConfiguration session? Is there any public API to inspect per-frame exposure duration, ISO, focus position, lens position, stabilization state, crop/scaling transform, or other capture parameters for ARFrame.capturedImage? If some frames are sharp and neighboring frames are blurred, should this usually be interpreted as motion blur / exposure behavior / autofocus behavior, or can ARKit internal processing also affect this? Is there any recommended way in an ARKit-first pipeline to reduce frame-to-frame sharpness inconsistency, or is the only reliable approach to use an AVFoundation-first capture pipeline with locked focus/exposure/ISO/white balance? The main goal of my research is to understand how to correctly interpret ARKit-provided intrinsics over time and how much of the camera pipeline behavior is observable through public ARKit APIs. Since fx/fy/cx/cy are important for measurements, I want to know whether treating ARFrame.camera.intrinsics as the Apple-delivered per-frame camera model is the safest approach. Thank you.

FB22720952 (ObjectCaptureSession fails with ARError.Code.sensorFailed on iOS 26.4.2 while RoomPlan and Measure continue to work). 17 pro Max

I have a visionOS app that plays audio using AVAudioEngine and presents both a window and an immersive space. If I close the window, the audio session gets interrupted and attempting to restart the session and audio engine has no effect. I need to dismiss the app, then reopen it, which reopens the main window, in order for audio to start playing again. This is in all visionOS 2 betas. Note that I have background audio enabled for my app.

Hi everyone, I’m currently using the RoomPlan API, which has been working reliably until recently. However, I’ve started encountering an intermittent error and I’m trying to understand what might be causing it. The error is triggered in the ARSession observer method: session(_ session: ARSession, didFailWithError error: Error) It has occurred on at least two devices: iPhone 14 Pro iPhone 17 Pro Here’s the full error message: ARSession failed domain=com.apple.arkit.error code=102 desc=Required sensor failed. userInfo=["NSLocalizedFailureReason": A sensor failed to deliver the required input., "NSUnderlyingError": Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action" UserInfo={NSLocalizedDescription=Cannot Complete Action, NSLocalizedRecoverySuggestion=Try again later.}, "NSLocalizedDescription": Required sensor failed.] This seems to indicate that a required sensor (likely LiDAR or camera) failed to provide input, but I’m not sure what’s causing it or why it happens only occasionally. Has anyone experienced something similar or has insight into possible causes or fixes? Thanks in advance!

I'm developing an app requiring data entry across several devices. My SwiftUI app runs on iOS and iPadOS but I also want to run it on visionOS. I'm using the visionOS simulator. When I enter data in one of my views I use a Form within a .sheet and this works perfectly well on iOS and iPadOS and I can dismiss the sheet by simply tapping the view behind the sheet. On visionOS I click my + button, the sheet appears, I enter the data as usual but after that there is no gesture in the app I can perform with keyboard or mouse that will make the sheet disappear! Do I have to add a "Close" button for visionOS or is there a way to enable the same interaction that works on iPadOS?

I'm trying to find where to buy the Vision Pro Developer Strap Gen 2. I've looked all around this site and cannot find it. Help

Hi. I want to know what's the difference between VirtualEnvironmentProbeComponent and ImageBasedLightComponent? It seems they both can achieve the same light and reflection effect of environment.

Hello, does anyone have experience using Apple Vision Pro in countries where it has not yet been officially released? I work for a company in Austria, and we are interested in developing internal XR applications for Vision Pro. Since the device is not officially available in Austria, we are considering purchasing it in Germany. My main question is whether it is possible to develop and test Vision Pro apps using an Austrian Apple ID / developer account, or if there are any regional restrictions we should be aware of (e.g., related to App Store access, provisioning, or device functionality). Apple Support was unfortunately unable to provide a definitive answer and recommended asking here. Any insights or experiences would be greatly appreciated. Best regards, Don Appelonie