WorldAnchor instantly removed when SpatialTrackingSession and ARKitSession run together

Bug: When SpatialTrackingSession and ARKitSession + WorldTrackingProvider are running concurrently, any WorldAnchor added via WorldTrackingProvider.addAnchor() triggers .added followed immediately by .removed—without any user call to removeAnchor(). The anchor never persists in allAnchors.

import SwiftUI
import RealityKit
import ARKit
struct ImmersiveView: View {
    @State private var worldTracking: WorldTrackingProvider?
    @State private var arSession: ARKitSession?
    @State private var processWorldTrackingUpdatesTask: Task<Void, Never>?
    
    var body: some View {
        RealityView { content in
            let configuration = SpatialTrackingSession.Configuration(tracking: [.world])
            if let unavailableCapabilities = await SpatialTrackingSession().run(configuration) {
                if unavailableCapabilities.anchor.contains(.world) {
                    fatalError("World tracking is not available on this device.")
                }
            }
            
            let worldTracking = WorldTrackingProvider()
            let arSession = ARKitSession()
            self.arSession = arSession
            try! await arSession.run([worldTracking])
            self.worldTracking = worldTracking
            
            processWorldTrackingUpdatesTask = Task { @MainActor [weak worldTracking] in
                guard let worldTracking else { return }
                for await update in worldTracking.anchorUpdates {
                    let worldAnchor = update.anchor
                    switch update.event {
                    case .added:
                        print("Anchor added: \(worldAnchor.id)")
                    case .updated:
                        print("Anchor updated: \(worldAnchor.id)")
                    case .removed:
                        fatalError("Anchor removed unexpectedly — this should not happen in this demo scenario.")
                    }
                }
            }
        }
        .task {
            try? await Task.sleep(for: .seconds(3))
            guard let worldTracking else { return }
            do {
                try await worldTracking.addAnchor(.init(originFromAnchorTransform: Transform.identity.matrix))
            } catch {
                print("Error adding anchor: \(error)")
            }
        }
    }
}

Expected: Anchors persist until explicitly removed or out of range.

Actual: SpatialTrackingSession interferes with WorldTrackingProvider's anchor lifecycle, causing immediate removal.

This was originally reported in 2025 (https://developer.apple.com/forums/thread/773351) , but remains unfixed in visionOS 27.0 beta. I've re-filed as FB23420195.

Answered by DTS Engineer in 895604022

I was wishing for an ARKit engineer to jump into this thread as I have not used that framework is a very long time. However I think you are answering your question as I don’t think those were ever intended to be used simultaneously for world tracking. Based on every sample and documentation I have seen.

If your app requires persistent world anchors, my suggestion is to completely drop SpatialTrackingSession and exclusively use ARKitSession with a WorldTrackingProvider.

If you were using SpatialTrackingSession for other things, you should migrate those to the ARKitSession as well by adding a any other provider?

Choose ARKitSession when you need persistence, when you call worldTracking.addAnchor(newAnchor). The system remembers this anchor on subsequent app launches in the same physical space.

I think you should describe to the ARKit engineers what you are trying to accomplish and maybe we can find a sample that will help you at least start on that path?

Albert  WWDR

Thank you for your post. I'm sure many engineers will help you with your question better than I can, I'm looking a the FB number you have provided.

You mentioned that you have re-filled a bug. Could you please provide me with the FB number for the first bug, excluding the one you have filed today? Regarding the file bug, instead of providing a Swift file, I recommend providing a zip file of a focused project that demonstrates the issue. This will allow engineering to use the project instead of making assumptions about the Xcode and settings you have used.

SpatialTrackingSession is designed as a high-level RealityKit abstraction that automatically manages an underlying ARKitSession on your behalf. When you run a SpatialTrackingSession configured for .world tracking, and run your own explicit ARKitSession with a WorldTrackingProvider, are both sessions competing for the underlying AR world-tracking resources?

Because RealityKit's internal session doesn't know about the WorldAnchor you have added to your explicit WorldTrackingProvider, it reconciles the tracking state by immediately invalidating/removing the unrecognized anchor, which triggers the .removed event?

Albert  WWDR

@DTS Engineer Thank you for the clarification.

The FB number mentioned in the previous forum thread was FB16424173. That report was filed by the original poster of that thread, not by me. My newly filed report is FB23420195.

I have also updated my Feedback Assistant report and attached a focused minimal Xcode project that demonstrates the issue, instead of only providing a standalone Swift file.

Regarding your explanation, yes, that matches my concern as well. If SpatialTrackingSession internally manages its own underlying ARKitSession, then running it together with an explicit ARKitSession + WorldTrackingProvider may cause both sessions to compete for the same world-tracking resources.

The main question I have is: if SpatialTrackingSession and ARKitSessionwith WorldTrackingProvider are not intended to be used together for persistent world tracking, is there any recommended way to add and manage persistent WorldAnchor s while keeping SpatialTrackingSession enabled?

In my real project, I rely heavily on APIs and features that require SpatialTrackingSession, so I would like to understand if there is a more appropriate way to handle this use case.

Accepted Answer

I was wishing for an ARKit engineer to jump into this thread as I have not used that framework is a very long time. However I think you are answering your question as I don’t think those were ever intended to be used simultaneously for world tracking. Based on every sample and documentation I have seen.

If your app requires persistent world anchors, my suggestion is to completely drop SpatialTrackingSession and exclusively use ARKitSession with a WorldTrackingProvider.

If you were using SpatialTrackingSession for other things, you should migrate those to the ARKitSession as well by adding a any other provider?

Choose ARKitSession when you need persistence, when you call worldTracking.addAnchor(newAnchor). The system remembers this anchor on subsequent app launches in the same physical space.

I think you should describe to the ARKit engineers what you are trying to accomplish and maybe we can find a sample that will help you at least start on that path?

Albert  WWDR

@DTS Engineer Thank you. That makes sense.

After thinking about this more, I realized that I may not need to completely drop SpatialTrackingSession for every tracking capability. In my case, the important conflict seems to be related to SpatialTrackingSession.Configuration(tracking: [.world]).

I did some additional testing, and it appears that if I do not enable .world in SpatialTrackingSession, I can still use other SpatialTrackingSession capabilities such as .hand and .accessory, while using my own ARKitSession with WorldTrackingProvider for persistent WorldAnchors.

The main reason I was using SpatialTrackingSession with .world was that it conveniently lets the system manage environment scanning and collision simulation for virtual object placement. If this could coexist with ARKitSession + WorldTrackingProvider, I think it would make development much more convenient. If I move persistent world anchors entirely to ARKitSession + WorldTrackingProvider, then I may need to implement the environment scanning and collision part separately through ARKit.

For context, I am developing a feature that scans the user’s environment, exports the spatial data so it can be used in Blender for character animation, and then imports the result back to Vision Pro for AR animation playback. Some virtual object placement in the app relies on environment collision, while placing the animated character back into the AR scene requires persistent world anchors through WorldTrackingProvider.

WorldAnchor instantly removed when SpatialTrackingSession and ARKitSession run together
 
 
Q