GeometricPins not updating on entities loaded from reality files

I'm attempting to create GeometricPins on an entity so I can attach objects to game characters. If I load a character from the USD file I can query the joint names on the ModelEntity and create GeometricPins as attachment points. These pins will track the location of the associated joint and I can position my objects at the pin locations as expected.

However, the pins do not update on characters loaded from a Reality File.

In the below image, the Skeleton Mage character in the middle is loaded as a USDC file and I attach the hat with a GeometricPin so that it tracks the movement of the head as it animates.

The animating Skeleton to the left is the same character (same .usdc file) but exported from RCP3 as a Reality File. Notice how the hat does not move with the character's head. The position and orientation of the GeometricPins attached to this character always return nil. I've also noticed that the jointNames array is empty for this entity as well.

Should I be able to access the jointNames and create Geometric pins on entities loaded from Reality files?

I'm adding the pin to the model entity with the following helper function. The helper function is called in the exact same way for the usd and reality entities.

func pinToJoint(_ entity: Entity, to target: Entity, at jointName: String, offset: SIMD3<Float> = .zero, orientation: simd_quatf = .init(angle: 0, axis: [0,1,0])) {
        let pinName = "pin_at_\(jointName)"
        if let rig:ModelEntity = target.findModelEntity() {
            for jointName in rig.jointNames {
                log.info("\(target.name): \(jointName)")
            }
            rig.pins.set(named: pinName, skeletalJointName: jointName, position: offset, orientation: simd_quatf(angle: -.pi/2.0, axis: .init(1,0,0)))
            
            entity.components.set(
                PinnedEntityComponent(
                    pinnedto: rig,
                    pinName: pinName,
                )
            )
        }
    }

The PinnedEntitySystem just assigns the location of the pin to the entity with the following

            guard let pinPosition = pin.position(relativeTo: nil) else { continue }
            guard let pinOrientation = pin.orientation(relativeTo: nil) else { continue }
            entity.setPosition(
                pinPosition
                , relativeTo: nil)
            entity.setOrientation(
                pinOrientation
                , relativeTo: nil)

Hi @PlatformGoblin,

You should be able to access the jointNames array and create geometric pins for entities loaded from Reality files. To confirm, I ran a quick test with the robot model from BOT-anist and was able to pin an entity to one of its animated joints (even after loading it from a Reality file exported from Reality Composer Pro 3) using exactly the approach you laid out above.

The fact that the jointNames array for your entity is empty sounds like the most likely culprit to me. Could you tell me more about how your entity is set up in Reality Composer Pro and how you're exporting it? Do you see Model and Skeleton components on the entity in the inspector?

If you'd like us to take a closer look at what's going on with your specific asset, please consider filing a report with Feedback Assistant with the asset attached. Don't forget to post the FB number here if you do so we can take a look right away!

Thanks!

Adding an additional note that eluded me in my previous response: consider using AttachedTransformComponent to pin your entity to a joint instead of writing a custom system! You should be able to replace your PinnedEntityComponent and PinnedEntitySystem with this component to much the same effect.

Let me know if you run into any issues!

@Vision Pro Engineer Thank you for the reply. I think I have this figured out now. To answer your questions.

The characters have both the Model and Skeleton components in RCP3.

The entity itself is just a root node with a reference to the prototype (SkeletonMage).

I'm exporting via the file menu File > Export Selected > Reality

I believe the problem that I'm running into is that my findModelEntity method is exiting out too soon for reality files. The Model and Skeleton are expected to be on the second Rig_Medium. It does this correctly for the entity loaded via USD. For the entity loaded via Reality File, it sees that SkeletonMage is a ModelEntity and returns early.

// output for usd file
Rig_Medium is a model entity
joint: head exists
tree for SkeletonMage USD
+ 
	+ root
		+ Rig_Medium
			+ Rig_Medium
			+ Skeleton_Mage_ArmRight
				+ Skeleton_Mage_ArmRight
			+ Skeleton_Mage_LegRight
				+ Skeleton_Mage_LegRight
			+ Skeleton_Mage_Eyes
				+ Skeleton_Mage_Eyes
			+ Skeleton_Mage_Hat
				+ Skeleton_Mage_Hat
			+ Skeleton_Mage_ArmLeft
				+ Skeleton_Mage_ArmLeft
			+ Skeleton_Mage_LegLeft
				+ Skeleton_Mage_LegLeft
			+ Skeleton_Mage_Body
				+ Skeleton_Mage_Body
			+ Skeleton_Mage_Skull
				+ Skeleton_Mage_Skull
			+ Skeleton_Mage_Jaw
				+ Skeleton_Mage_Jaw

// output for reality file
Skeleton_Mage is a model entity
target: Skeleton_Mage is a model entity but our joint is in another castle
tree for SkeletonMage Reality
+ SkeletonMage
	+ Skeleton_Mage
		+ root
			+ Rig_Medium
				+ Rig_Medium
				+ Skeleton_Mage_Hat
					+ Skeleton_Mage_Hat

I changed how I was searching for the rig and verified that the desired joint is present before creating the pin.

And thanks for the tip on AttachedTransformComponent. You're right that I should be using that instead of a custom system. I had other plans for the PinnedEntitySystem that I just don't need at the moment.

Hi @PlatformGoblin,

Thanks for sharing your setup—looks good to me—and I'm happy to hear you figured out the source of issue!

Let me know if you have further questions.

GeometricPins not updating on entities loaded from reality files
 
 
Q