Metal UIView to transform what's behind it

Question

Created Aug ’24

Replies 2

Boosts 0

Participants 3

I'm trying to create a custom Metal-based visual effect as a UIView to be used inside an existing UIKit-based interface. (An example might be a view that applies a blur effect to what's behind it.) I need to capture the MTLTexture of what's behind the view so that I can feed it to MTLRenderCommandEncoder.setFragmentTexture(_:index:). Can someone show me how or point me to an example? Thanks!

Answered by DTS Engineer in 807468022

For simpler effects like blur you may want to consider UIVisualEffect, and UIBlurEffect in particular.

Otherwise, the approach above is viable for compositing but costly. We recommend trying it and profiling the result to see if it's within your performance budget.

Answer 1

Graphics and Games Engineer OP

Apple

Aug ’24

Because of the way UIKit views are rendered, a view does not have access to the contents rendered behind it. Every view renders into its own layer, which are then composited together by Core Animation.

The only way to do what you want to do is to manually render the “below” portion of your view hierarchy into a bitmap using -[UIView drawViewHierarchyInRect:afterScreenUpdates:] and then use that bitmap as a texture input to your Metal shader. This will likely be too slow to be done in real-time.

Answer 2

DTS Engineer OP

Apple

Oct ’24

Recommended

For simpler effects like blur you may want to consider UIVisualEffect, and UIBlurEffect in particular.

Otherwise, the approach above is viable for compositing but costly. We recommend trying it and profiling the result to see if it's within your performance budget.