Object Capture

RSS for tag

Turn photos from your iPhone or iPad into high‑quality 3D models that are optimized for AR using the new Object Capture API on macOS Monterey.

Posts under Object Capture tag

105 Posts

Post

Replies

Boosts

Views

Activity

Reconstruction with Object Mask in PhotogrammetrySession Fails .
Reconstruction with Object Mask in PhotogrammetrySession Fails and doesn't complete because of alignment and object masks with thing segmentation even after increasing the mask region . Also the PhotogrammetrySample hangs with EXC_BREAKPOINTwhen I usePhotogrammetrySession(input: [PhotogrammetrySample])withobjectMask set and the Apple Developer Forums report crashes (“HelloPhotogrammetry” / sample+mask), and (EXC_BREAKPOINT` was being observed at ~4 GB) with freezing the execution and Xcode interface
4
0
682
2w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Xcode hangs when I call PhotogrammetrySession(input: [PhotogrammetrySample]) with objectMask set and traps on some devices, see the attached screenshot, it gets to the function and hangs. Even the folder reconstruction also doesn't complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. In this case it failed while object masking was ON, so RealityKit could not find enough consistent feature tracks inside the masked pixels across the image set. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding
4
0
1.7k
2w
PhotogrammetrySession(input: [PhotogrammetrySample])
Please I would really appreciate a timely response to this: Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. Please I would really appreciate a timely response to this
1
0
240
2w
Object Capture broken on iPhone 17 Pro (iOS 26.x): ObjectCaptureSession sensorFailed + PhotogrammetrySession cv3dapi 4004
The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)
1
0
364
3w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing clarity and response to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
0
0
746
3w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 I have a created a feedback/bug report with ID FB22823733 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
0
0
1.3k
May ’26
How to fix "Sample 0 missing LiDAR point cloud!" error?
I'm trying to run a PhotogrammetrySession based on photos taken in an AVCaptureSession and stored as .heic files. When I load the files I'm always seeing the error "Sample 0 missing LiDAR point cloud!" showing up for each individual sample. Debugging shows that sample.depthDataMap is populated, also the .heic contains depth data which can be extracted using e.g. heif-convert on my Mac. Comparing the .heic I created to one of the ObjectCaptureSession which doesn't show the LiDAR warning, I noticed the only difference being the HEIC information here: So my questions are: Are these the missing information in my manual capture causing this warning? Can I somehow add these information in an AVCaptureSession? Do these information allow better photogrammetry results?
2
0
526
Dec ’25
Best approach for high-quality textured room reconstruction using ARKit / RoomPlan / Object Capture?
I am developing an IOS App that allow users to scan rooms, view the scans on device, and add notes. I need to preserve actual geometry (odd angles, chamfers, fixtures), not simplified RoomPlan boxes. Are there any easy ways to incorporate high quality texture mapping or PBR? Where is the documentation for scene reconstruction?
1
0
1k
Nov ’25
PhotogrammetrySession fails with internal errors 4011 / 4012 when using iOS Object Capture (Area Mode) images
Hi all, I’m running into an issue when trying to reconstruct a 3D model using PhotogrammetrySession on macOS from a set of images captured via the iOS Object Capture sample app, specifically in Area mode. When I attempt to create the model from these images (using the raw Images/ folder exported directly from the capture session), I get the following errors: ERROR cv3dapi.pg: Internal error codes (2): 4011 4012 WARN cv3dapi.pg: Internal warning codes (1): 4507 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError I use the "Images" directory directly exported from Object Capture with my iphone 12 pro max (has lidar) set to "area mode" in the object capture app here is an example heic image metadata from the sequence. heif-info Images/00044.869568833.HEIC MIME type: image/heic main brand: heic compatible brands: mif1, MiHE, MiPr, miaf, MiHB, heic image: 3024x4032 (id=49), primary tiles: 6x8, tile size: 512x512 colorspace: YCbCr, 4:2:0 bit depth: 8 thumbnail: 240x320 color profile: nclx alpha channel: no depth channel: yes size: 192x256 bits per pixel: 8 z-near: 1.173828 z-far: 2.552734 d-min: undefined d-max: undefined representation: uniform Z metadata: Exif: 960 bytes uri /tag:apple.com,2023:ObjectCapture#CameraTrackingState: 4 bytes uri /tag:apple.com,2023:ObjectCapture#CameraCalibrationData: 1015 bytes uri /tag:apple.com,2023:ObjectCapture#ObjectTransform: 48 bytes uri /tag:apple.com,2023:ObjectCapture#ObjectBoundingBox: 48 bytes uri /tag:apple.com,2023:ObjectCapture#RawFeaturePoints: 832 bytes uri /tag:apple.com,2023:ObjectCapture#PointCloudData: 23984 bytes uri /tag:apple.com,2023:ObjectCapture#BundleVersion: 5 bytes uri /tag:apple.com,2023:ObjectCapture#SegmentID: 4 bytes uri /tag:apple.com,2024:ObjectCapture#SessionUUID: 36 bytes uri /tag:apple.com,2024:ObjectCapture#CaptureMode: 4 bytes uri /tag:apple.com,2023:ObjectCapture#Feedback: 4 bytes uri /tag:apple.com,2023:ObjectCapture#WideToDepthCameraTransform: 48 bytes uri /tag:apple.com,2023:ObjectCapture#TemporalDepthPointClouds: 864026 bytes transformations: angle (ccw): 270 region annotations: none properties: camera intrinsic matrix: focal length: 2813.695557; 2813.695557 principal point: 1522.338502; 2002.843018 skew: 0.000000 camera extrinsic matrix: rotation matrix: -0.695 0.344 -0.632 0.007 -0.875 -0.483 -0.719 -0.340 0.606 Questions: • What do internal error codes 4011 and 4012 refer to? • Is there something specific about Area mode captures that require preprocessing before they’re compatible with PhotogrammetrySession? • Has anyone successfully reconstructed a model from an Area mode session using the stock Apple tools? NOTE: I can provide the folder of images for debugging if that would help!
1
2
1.2k
Jul ’25
Photogrammetry Session - new model?
Hi Apple Team, We noticed the following exciting changelog in the latest macOS 26 beta: A new algorithm significantly improves PhotogrammetrySession reconstruction quality of low-texture objects not captured with the ObjectCaptureSession front end. It will be downloaded and cached once in the background when the PhotogrammetrySession is used at runtime. If network isn’t available at that time, the old low quality model will be used until the new one can be downloaded. There is no code change needed to get this improved model. (145220451) However after trying this on the latest beta and running some tests we do not see any differences on objects with low textures such as single coloured surfaces. Is there anything we are missing? the machine is definitely connected to the internet but we have no way of knowing from the logs if the new model is being used? thanks
5
1
809
Jul ’25
new algorithm significantly improves PhotogrammetrySession?
I noticed in the latest macOS beta 3 that there was this update: A new algorithm significantly improves PhotogrammetrySession reconstruction quality of low-texture objects not captured with the ObjectCaptureSession front end. It will be downloaded and cached once in the background when the PhotogrammetrySession is used at runtime. If network isn’t available at that time, the old low quality model will be used until the new one can be downloaded. There is no code change needed to get this improved model. (145220451) I am not noticing any difference to before with the reconstructions I tested so I am assuming it's reverting to the old model but in the logs there is no way to see if it succeeds or fails to download that new model. do you have any more information on what was improved here with some examples and what we should be looking for? also how can confirm the download of that new model has not failed?
0
0
441
Jul ’25
PhotogrammetrySession Polygon Count Limit – How Is It Determined by Hardware?
Hi Apple Team, I’m working on a human portrait scanning application using PhotogrammetrySession, and I’ve been very impressed by the results. Thank you for building such a powerful and accessible photogrammetry solution into macOS! I do, however, have a question regarding mesh detail limitations on different Mac hardware configurations. When using PhotogrammetrySession.Request.Detail.custom and trying to set maximumPolygonCount = 1000000, I see the following log message: Clamped max poly count: 1000000 to device limit. 250000 is used. This is on an M1 Max with 32 GB RAM. I’m aware that PhotogrammetrySession.limits can report values like maximumInputImageDimension and maximumNumberOfInputImages, but I haven’t found documentation on how the maximumPolygonCount is determined, and what hardware specs influence it. Is it tied more to: • GPU performance (e.g. neural/graphics cores)? • CPU architecture? • Memory size or bandwidth? • Or is it fixed per SoC generation? I’d love to understand what kind of hardware upgrades (e.g. moving to M4 Pro or increasing RAM) could allow me to increase mesh complexity and generate more detailed models. Any insights would be greatly appreciated—and if this is covered in upcoming WWDC sessions or documentation, I’d be happy to tune in. Thanks in advance! KitCheng
0
0
203
May ’25
Request: More Fine-Grained Control in Object Capture (PhotogrammetrySession)
Hi Apple Team and Developers, First of all, I’d like to express my appreciation for the incredible results achieved using PhotogrammetrySession. I’ve been developing a portrait scanning app using Object Capture, and in many tests—especially with human models—I’ve found the reconstructed body surfaces are remarkably smooth and clean, often outperforming tools like Metashape and RealityCapture in terms of aesthetic results. However, I’ve encountered some challenges when working with complex areas like long hair overlapping the face. For instance, with female models where strands of hair partially occlude the face, the resulting mesh tends to merge the hair and facial geometry. This leads to distorted or “melted” facial features, likely due to ambiguity in the geometry estimation phase. Feature Suggestion: Would it be possible to allow developers to supply two versions of the input images: • One version (original) for texture generation • A pre-processed version (e.g., contrast-enhanced or CLAHE filtered) to guide mesh reconstruction only This would give us the flexibility to enhance edge features or shadow detail without affecting the final texture appearance. In other photogrammetry pipelines, applying image enhancement selectively before dense reconstruction improves geometry quality in low-contrast areas. Question: Is there any plan to support this kind of two-path workflow in future versions of PhotogrammetrySession? Or perhaps expose more intermediate stages or tunable parameters to developers? Also, any hints on what we can expect from WWDC 2025 regarding improvements to Object Capture or related vision/3D technologies? Thanks again for this powerful API. Looking forward to hearing insights from the team and other developers. Warm regards, KitCheng
0
0
179
May ’25
Use native Swift API for HTTP request with auth proxy
I'm simply trying to use a proxy to route a http request in Swift. I've tried using a URLSession Delegate but that results in the same issue with the iOS menu. proxy format: host:port:username:password When I run the code below I am prompted with a menu to add credentials for the proxy. I closed this menu inside my app and tried the function below again and it worked without giving me the menu a second time. However even though the function works without throwing any errors, it does NOT use the proxies to route the request. I've spent days on this and the only solution I found was using a NWConnection but this is super low level and now I need a shared session to manage cookies. If you want to see the NWConnection solution I made its here func averageProxyGroupSpeed(proxies: [String], completion: @escaping (Int, String) -> Void) { let numProxies = proxies.count if numProxies == 0 { completion(0, "No proxies") return } var totalTime: Int64 = 0 var successCount = 0 let group = DispatchGroup() let queue = DispatchQueue(label: "proxyQueue", attributes: .concurrent) let lock = NSLock() let shuffledProxies = proxies.shuffled() let selectedProxies = Array(shuffledProxies.prefix(25)) for proxy in selectedProxies { group.enter() queue.async { let proxyDetails = proxy.split(separator: ":").map(String.init) guard proxyDetails.count == 4, let port = Int(proxyDetails[1]), let url = URL(string: "http://httpbin.org/get") else { completion(0, "Invalid proxy format") group.leave() return } var request = URLRequest(url: url) request.timeoutInterval = 15 let configuration = URLSessionConfiguration.default configuration.connectionProxyDictionary = [ AnyHashable("HTTPEnable"): true, AnyHashable("HTTPProxy"): proxyDetails[0], AnyHashable("HTTPPort"): port, AnyHashable("HTTPSEnable"): false, AnyHashable("HTTPUser"): proxyDetails[2], AnyHashable("HTTPPassword"): proxyDetails[3] ] let session = URLSession(configuration: configuration) let start = Date() let task = session.dataTask(with: request) { _, _, error in defer { group.leave() } if let error = error { print("Error: \(error.localizedDescription)") } else { let duration = Date().timeIntervalSince(start) * 1000 lock.lock() totalTime += Int64(duration) successCount += 1 lock.unlock() } } task.resume() } } group.notify(queue: DispatchQueue.main) { if successCount == 0 { completion(0, "Proxies Failed") } else { let averageTime = Int(Double(totalTime) / Double(successCount)) completion(averageTime, "") } } } Delegate example class ProxySessionDelegate: NSObject, URLSessionDelegate { let username: String let password: String init(username: String, password: String) { self.username = username self.password = password } func urlSession(_ session: URLSession, task: URLSessionTask, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void) { if challenge.protectionSpace.authenticationMethod == NSURLAuthenticationMethodHTTPBasic { let credential = URLCredential(user: self.username, password: self.password, persistence: .forSession) completionHandler(.useCredential, credential) } else { completionHandler(.performDefaultHandling, nil) } } }
2
0
493
Mar ’25
3D scanning and 3D model with texture mapping
I am new here and would appreciate help in coding or an explanation what to use in swift for an app which will be able to capture LiDAR scanning and RGB data from taken pictures, generate a 3D mesh, and create .OBJ, .MTL, and .JPEG file set for further manipulation of 3D model. I am able to create from LiDAR scanning 3D mesh and .OBJ file but can not generate .MTL and .JPEG for a texture of 3D model.
1
0
465
Feb ’25
Obect Capture App getting rejected for not supporting non Pro Models
Hi, I created an app using iOS Object Capture API which works only on Lidar enabled phones. It's a limitation of the Api provided by apple itself. I Submitted an app for Review , but It is getting rejected (Twice) saying it doesnt work on non pro models. Even though I explained that capturing Needs Lidar and supported only in PRO models, It still gets rejected after testing in Non Pro models. is there a way out?
1
0
378
Feb ’25
Reconstruction with Object Mask in PhotogrammetrySession Fails .
Reconstruction with Object Mask in PhotogrammetrySession Fails and doesn't complete because of alignment and object masks with thing segmentation even after increasing the mask region . Also the PhotogrammetrySample hangs with EXC_BREAKPOINTwhen I usePhotogrammetrySession(input: [PhotogrammetrySample])withobjectMask set and the Apple Developer Forums report crashes (“HelloPhotogrammetry” / sample+mask), and (EXC_BREAKPOINT` was being observed at ~4 GB) with freezing the execution and Xcode interface
Replies
4
Boosts
0
Views
682
Activity
2w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Xcode hangs when I call PhotogrammetrySession(input: [PhotogrammetrySample]) with objectMask set and traps on some devices, see the attached screenshot, it gets to the function and hangs. Even the folder reconstruction also doesn't complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. In this case it failed while object masking was ON, so RealityKit could not find enough consistent feature tracks inside the masked pixels across the image set. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding
Replies
4
Boosts
0
Views
1.7k
Activity
2w
PhotogrammetrySession(input: [PhotogrammetrySample])
Please I would really appreciate a timely response to this: Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. Please I would really appreciate a timely response to this
Replies
1
Boosts
0
Views
240
Activity
2w
Object Capture broken on iPhone 17 Pro (iOS 26.x): ObjectCaptureSession sensorFailed + PhotogrammetrySession cv3dapi 4004
The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)
Replies
1
Boosts
0
Views
364
Activity
3w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing clarity and response to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
Replies
0
Boosts
0
Views
746
Activity
3w
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 I have a created a feedback/bug report with ID FB22823733 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
Replies
0
Boosts
0
Views
1.3k
Activity
May ’26
FB22720952 (ObjectCaptureSession fails with ARError.Code.sensorFailed on iOS 26.4.2 while RoomPlan and Measure continue to work)
FB22720952 (ObjectCaptureSession fails with ARError.Code.sensorFailed on iOS 26.4.2 while RoomPlan and Measure continue to work). 17 pro Max
Replies
0
Boosts
0
Views
825
Activity
May ’26
Object Capture feature in visionOS
The object capture feature in Reality Composer App is only available in iOS and iPadOS at the moment, would this feature be available for visionOS in near future? Reality Composer App Store https://apps.apple.com/us/app/reality-composer/id1462358802
Replies
2
Boosts
0
Views
1.7k
Activity
May ’26
How to fix "Sample 0 missing LiDAR point cloud!" error?
I'm trying to run a PhotogrammetrySession based on photos taken in an AVCaptureSession and stored as .heic files. When I load the files I'm always seeing the error "Sample 0 missing LiDAR point cloud!" showing up for each individual sample. Debugging shows that sample.depthDataMap is populated, also the .heic contains depth data which can be extracted using e.g. heif-convert on my Mac. Comparing the .heic I created to one of the ObjectCaptureSession which doesn't show the LiDAR warning, I noticed the only difference being the HEIC information here: So my questions are: Are these the missing information in my manual capture causing this warning? Can I somehow add these information in an AVCaptureSession? Do these information allow better photogrammetry results?
Replies
2
Boosts
0
Views
526
Activity
Dec ’25
Best approach for high-quality textured room reconstruction using ARKit / RoomPlan / Object Capture?
I am developing an IOS App that allow users to scan rooms, view the scans on device, and add notes. I need to preserve actual geometry (odd angles, chamfers, fixtures), not simplified RoomPlan boxes. Are there any easy ways to incorporate high quality texture mapping or PBR? Where is the documentation for scene reconstruction?
Replies
1
Boosts
0
Views
1k
Activity
Nov ’25
PhotogrammetrySession fails with internal errors 4011 / 4012 when using iOS Object Capture (Area Mode) images
Hi all, I’m running into an issue when trying to reconstruct a 3D model using PhotogrammetrySession on macOS from a set of images captured via the iOS Object Capture sample app, specifically in Area mode. When I attempt to create the model from these images (using the raw Images/ folder exported directly from the capture session), I get the following errors: ERROR cv3dapi.pg: Internal error codes (2): 4011 4012 WARN cv3dapi.pg: Internal warning codes (1): 4507 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError I use the "Images" directory directly exported from Object Capture with my iphone 12 pro max (has lidar) set to "area mode" in the object capture app here is an example heic image metadata from the sequence. heif-info Images/00044.869568833.HEIC MIME type: image/heic main brand: heic compatible brands: mif1, MiHE, MiPr, miaf, MiHB, heic image: 3024x4032 (id=49), primary tiles: 6x8, tile size: 512x512 colorspace: YCbCr, 4:2:0 bit depth: 8 thumbnail: 240x320 color profile: nclx alpha channel: no depth channel: yes size: 192x256 bits per pixel: 8 z-near: 1.173828 z-far: 2.552734 d-min: undefined d-max: undefined representation: uniform Z metadata: Exif: 960 bytes uri /tag:apple.com,2023:ObjectCapture#CameraTrackingState: 4 bytes uri /tag:apple.com,2023:ObjectCapture#CameraCalibrationData: 1015 bytes uri /tag:apple.com,2023:ObjectCapture#ObjectTransform: 48 bytes uri /tag:apple.com,2023:ObjectCapture#ObjectBoundingBox: 48 bytes uri /tag:apple.com,2023:ObjectCapture#RawFeaturePoints: 832 bytes uri /tag:apple.com,2023:ObjectCapture#PointCloudData: 23984 bytes uri /tag:apple.com,2023:ObjectCapture#BundleVersion: 5 bytes uri /tag:apple.com,2023:ObjectCapture#SegmentID: 4 bytes uri /tag:apple.com,2024:ObjectCapture#SessionUUID: 36 bytes uri /tag:apple.com,2024:ObjectCapture#CaptureMode: 4 bytes uri /tag:apple.com,2023:ObjectCapture#Feedback: 4 bytes uri /tag:apple.com,2023:ObjectCapture#WideToDepthCameraTransform: 48 bytes uri /tag:apple.com,2023:ObjectCapture#TemporalDepthPointClouds: 864026 bytes transformations: angle (ccw): 270 region annotations: none properties: camera intrinsic matrix: focal length: 2813.695557; 2813.695557 principal point: 1522.338502; 2002.843018 skew: 0.000000 camera extrinsic matrix: rotation matrix: -0.695 0.344 -0.632 0.007 -0.875 -0.483 -0.719 -0.340 0.606 Questions: • What do internal error codes 4011 and 4012 refer to? • Is there something specific about Area mode captures that require preprocessing before they’re compatible with PhotogrammetrySession? • Has anyone successfully reconstructed a model from an Area mode session using the stock Apple tools? NOTE: I can provide the folder of images for debugging if that would help!
Replies
1
Boosts
2
Views
1.2k
Activity
Jul ’25
Photogrammetry Session - new model?
Hi Apple Team, We noticed the following exciting changelog in the latest macOS 26 beta: A new algorithm significantly improves PhotogrammetrySession reconstruction quality of low-texture objects not captured with the ObjectCaptureSession front end. It will be downloaded and cached once in the background when the PhotogrammetrySession is used at runtime. If network isn’t available at that time, the old low quality model will be used until the new one can be downloaded. There is no code change needed to get this improved model. (145220451) However after trying this on the latest beta and running some tests we do not see any differences on objects with low textures such as single coloured surfaces. Is there anything we are missing? the machine is definitely connected to the internet but we have no way of knowing from the logs if the new model is being used? thanks
Replies
5
Boosts
1
Views
809
Activity
Jul ’25
new algorithm significantly improves PhotogrammetrySession?
I noticed in the latest macOS beta 3 that there was this update: A new algorithm significantly improves PhotogrammetrySession reconstruction quality of low-texture objects not captured with the ObjectCaptureSession front end. It will be downloaded and cached once in the background when the PhotogrammetrySession is used at runtime. If network isn’t available at that time, the old low quality model will be used until the new one can be downloaded. There is no code change needed to get this improved model. (145220451) I am not noticing any difference to before with the reconstructions I tested so I am assuming it's reverting to the old model but in the logs there is no way to see if it succeeds or fails to download that new model. do you have any more information on what was improved here with some examples and what we should be looking for? also how can confirm the download of that new model has not failed?
Replies
0
Boosts
0
Views
441
Activity
Jul ’25
PhotogrammetrySession Polygon Count Limit – How Is It Determined by Hardware?
Hi Apple Team, I’m working on a human portrait scanning application using PhotogrammetrySession, and I’ve been very impressed by the results. Thank you for building such a powerful and accessible photogrammetry solution into macOS! I do, however, have a question regarding mesh detail limitations on different Mac hardware configurations. When using PhotogrammetrySession.Request.Detail.custom and trying to set maximumPolygonCount = 1000000, I see the following log message: Clamped max poly count: 1000000 to device limit. 250000 is used. This is on an M1 Max with 32 GB RAM. I’m aware that PhotogrammetrySession.limits can report values like maximumInputImageDimension and maximumNumberOfInputImages, but I haven’t found documentation on how the maximumPolygonCount is determined, and what hardware specs influence it. Is it tied more to: • GPU performance (e.g. neural/graphics cores)? • CPU architecture? • Memory size or bandwidth? • Or is it fixed per SoC generation? I’d love to understand what kind of hardware upgrades (e.g. moving to M4 Pro or increasing RAM) could allow me to increase mesh complexity and generate more detailed models. Any insights would be greatly appreciated—and if this is covered in upcoming WWDC sessions or documentation, I’d be happy to tune in. Thanks in advance! KitCheng
Replies
0
Boosts
0
Views
203
Activity
May ’25
Request: More Fine-Grained Control in Object Capture (PhotogrammetrySession)
Hi Apple Team and Developers, First of all, I’d like to express my appreciation for the incredible results achieved using PhotogrammetrySession. I’ve been developing a portrait scanning app using Object Capture, and in many tests—especially with human models—I’ve found the reconstructed body surfaces are remarkably smooth and clean, often outperforming tools like Metashape and RealityCapture in terms of aesthetic results. However, I’ve encountered some challenges when working with complex areas like long hair overlapping the face. For instance, with female models where strands of hair partially occlude the face, the resulting mesh tends to merge the hair and facial geometry. This leads to distorted or “melted” facial features, likely due to ambiguity in the geometry estimation phase. Feature Suggestion: Would it be possible to allow developers to supply two versions of the input images: • One version (original) for texture generation • A pre-processed version (e.g., contrast-enhanced or CLAHE filtered) to guide mesh reconstruction only This would give us the flexibility to enhance edge features or shadow detail without affecting the final texture appearance. In other photogrammetry pipelines, applying image enhancement selectively before dense reconstruction improves geometry quality in low-contrast areas. Question: Is there any plan to support this kind of two-path workflow in future versions of PhotogrammetrySession? Or perhaps expose more intermediate stages or tunable parameters to developers? Also, any hints on what we can expect from WWDC 2025 regarding improvements to Object Capture or related vision/3D technologies? Thanks again for this powerful API. Looking forward to hearing insights from the team and other developers. Warm regards, KitCheng
Replies
0
Boosts
0
Views
179
Activity
May ’25
Use native Swift API for HTTP request with auth proxy
I'm simply trying to use a proxy to route a http request in Swift. I've tried using a URLSession Delegate but that results in the same issue with the iOS menu. proxy format: host:port:username:password When I run the code below I am prompted with a menu to add credentials for the proxy. I closed this menu inside my app and tried the function below again and it worked without giving me the menu a second time. However even though the function works without throwing any errors, it does NOT use the proxies to route the request. I've spent days on this and the only solution I found was using a NWConnection but this is super low level and now I need a shared session to manage cookies. If you want to see the NWConnection solution I made its here func averageProxyGroupSpeed(proxies: [String], completion: @escaping (Int, String) -> Void) { let numProxies = proxies.count if numProxies == 0 { completion(0, "No proxies") return } var totalTime: Int64 = 0 var successCount = 0 let group = DispatchGroup() let queue = DispatchQueue(label: "proxyQueue", attributes: .concurrent) let lock = NSLock() let shuffledProxies = proxies.shuffled() let selectedProxies = Array(shuffledProxies.prefix(25)) for proxy in selectedProxies { group.enter() queue.async { let proxyDetails = proxy.split(separator: ":").map(String.init) guard proxyDetails.count == 4, let port = Int(proxyDetails[1]), let url = URL(string: "http://httpbin.org/get") else { completion(0, "Invalid proxy format") group.leave() return } var request = URLRequest(url: url) request.timeoutInterval = 15 let configuration = URLSessionConfiguration.default configuration.connectionProxyDictionary = [ AnyHashable("HTTPEnable"): true, AnyHashable("HTTPProxy"): proxyDetails[0], AnyHashable("HTTPPort"): port, AnyHashable("HTTPSEnable"): false, AnyHashable("HTTPUser"): proxyDetails[2], AnyHashable("HTTPPassword"): proxyDetails[3] ] let session = URLSession(configuration: configuration) let start = Date() let task = session.dataTask(with: request) { _, _, error in defer { group.leave() } if let error = error { print("Error: \(error.localizedDescription)") } else { let duration = Date().timeIntervalSince(start) * 1000 lock.lock() totalTime += Int64(duration) successCount += 1 lock.unlock() } } task.resume() } } group.notify(queue: DispatchQueue.main) { if successCount == 0 { completion(0, "Proxies Failed") } else { let averageTime = Int(Double(totalTime) / Double(successCount)) completion(averageTime, "") } } } Delegate example class ProxySessionDelegate: NSObject, URLSessionDelegate { let username: String let password: String init(username: String, password: String) { self.username = username self.password = password } func urlSession(_ session: URLSession, task: URLSessionTask, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void) { if challenge.protectionSpace.authenticationMethod == NSURLAuthenticationMethodHTTPBasic { let credential = URLCredential(user: self.username, password: self.password, persistence: .forSession) completionHandler(.useCredential, credential) } else { completionHandler(.performDefaultHandling, nil) } } }
Replies
2
Boosts
0
Views
493
Activity
Mar ’25
3D scanning and 3D model with texture mapping
I am new here and would appreciate help in coding or an explanation what to use in swift for an app which will be able to capture LiDAR scanning and RGB data from taken pictures, generate a 3D mesh, and create .OBJ, .MTL, and .JPEG file set for further manipulation of 3D model. I am able to create from LiDAR scanning 3D mesh and .OBJ file but can not generate .MTL and .JPEG for a texture of 3D model.
Replies
1
Boosts
0
Views
465
Activity
Feb ’25
3D scanning and 3D model with texture mapping
I would appreciate help in coding or an explanation what to use in swift for an app which will be able to capture LiDAR scanning and RGB data from taken pictures, generate a 3D mesh, and create .OBJ, .MTL, and .JPEG file set for further manipulation of 3D model.
Replies
2
Boosts
0
Views
410
Activity
Feb ’25
Obect Capture App getting rejected for not supporting non Pro Models
Hi, I created an app using iOS Object Capture API which works only on Lidar enabled phones. It's a limitation of the Api provided by apple itself. I Submitted an app for Review , but It is getting rejected (Twice) saying it doesnt work on non pro models. Even though I explained that capturing Needs Lidar and supported only in PRO models, It still gets rejected after testing in Non Pro models. is there a way out?
Replies
1
Boosts
0
Views
378
Activity
Feb ’25
[Object Capture API] Is there a way I can get the current frame ?
I would like to integrate the object capture API with a ML model for analysis. So, i will need to get the current frame into CG images for further process. Thanks in advance !
Replies
1
Boosts
0
Views
581
Activity
Jan ’25