Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics
Posts under Media Technologies topic

Post

Replies

Boosts

Views

Activity

New download progress "banner" on iOS27
We are using AVAssetDownloadURLSession to download videos. On iOS27, when building with the iOS27 SDK, there is a new progress banner showing the file name and download progress. The user can also cancel the download from there. I have not seen this mentioned anywhere and I am wondering if there are any APIs related to this new feature? Reason: For example, we show a user notification when a download has finished but if there already was a banner displaying the status that notification is unnecessary.
2
0
473
Sep ’26
Camera direction and rotation in the half-folded pose on iPhone Duo
"Build a great camera experience for iPhone Duo" covers the device open and the device closed, including reconfiguring the session from AVCaptureDeviceDirectionCoordinator's change handler when the forward-facing camera changes. It does not cover the partially open pose, which "Leverage multiple displays and scenes on iPhone Duo" treats as a first-class hinge state with a continuous angle. Consider a camera app that keeps a live preview on screen and adopts AVCaptureDevice.RotationCoordinator — the documented remedy for cases like the square front sensor on iPhone 17, where a hard-coded 90° leaves the preview tilted — applying videoRotationAngleForHorizonLevelPreview to both the preview and the capture connection. In the partially open pose the two halves are at different physical attitudes: one stands up, the other lies flat. Which camera does AVCaptureDeviceDirectionCoordinator report as forward-facing while the hinge status is partially open? Does the hinge angle affect that decision? RotationCoordinator derives its angle from device attitude. With the two halves at different attitudes, which one does videoRotationAngleForHorizonLevelPreview reflect — the half carrying the active camera, the half hosting the preview layer, or a single device-level reference? If an app deliberately keeps the rear camera on the scene while its preview is drawn on the half facing the user, is that a supported configuration, or is the coordinator expected to override that choice? Is any of this modelled in the iPhone Duo simulator, or does the partially open pose require real hardware to test? Question 4 matters for planning: building against a rotation value that only appears on device would be worth knowing about before the hardware ships.
4
0
633
Sep ’26
Is the inner under-display camera going to be usable for real-time 2D body-pose analysis?
Hi! We run continuous 2D human body-pose estimation on a front-facing camera feed: one person, full body in frame, standing a meter or two away, 30-60fps. With iPhone Duo open, the camera that faces the person looking at the inner display is the under-display inner camera (builtInInnerUltraWideCamera). Published specs cap it at 1080p at 30 or 60 fps, and it is presented as the FaceTime camera. Two things are unclear from the camera tech talk and the specs: Is that 1080p format a native readout we can feed to Vision body pose, or a preview-oriented stream whose effective detail is too low for full-body pose at that distance? Is third-party use of this camera supported for a long-running analysis session, or is it intended only for video calls and live previews? Thanks
1
0
244
Sep ’26
About photo wallpaper on iPhone Duo
For a user-selected photo wallpaper on iPhone Duo, does iOS use one source image and automatically adapt or crop it for the outer and inner displays, or can users assign separate images to the outer and inner displays? If one image is used, is the crop or focal position shared between both displays, or can the user adjust them independently?
1
1
588
Sep ’26
Under-display camera behavior during FaceTime
When a persistent camera session such as FaceTime is active on the inner display, the under-display camera becomes visible. What is the behavior when such a session is active and the user switches to my app, where some content, such as Navigation titles, normally sits in the region of the camera? Will the camera become hidden again and show in FaceTime that the call participant's camera is unavailable, or is there some other behavior of which I as an app developer should be aware? Thank you!
1
0
188
Sep ’26
ARKit AR View on both screens
Hello everyone! Does the ARKit ARSession qualify for an active camera session? Reading through the documentation, I understand that showing content on both iPhone Duo screens at the same time is only possible by using CameraCaptureAccessory. ARSession drives the camera but doesn't vend an AVCaptureSession that the app owns, so it isn't clear whether the availability check keys off app-owned AVCaptureSession state specifically, or off the camera actively streaming for the app. If it is currently not possible, do you see any way or workaround to show the camera stream on the outside display while the inner display is displaying an AR view? Best regards, André N.
4
1
243
Sep ’26
Shutter button
Any tips on positioning a shutter button on camera screen (fullscreen preview) when device is in half-open state? As well as other centred controls (camera mode, etc)? How to handle hinge area updates? Thanks
2
0
158
Sep ’26
CameraCaptureAccessory: orientation of the accessory scene in the partially folded (laptop) pose
We present custom content on the outer display through CameraCaptureAccessory while a capture session runs (iOS 27.1). Our app is used in the partially folded "laptop" pose: the inner display is portrait with a horizontal fold, controls below the fold, content above it. The person in front of the outer camera sits opposite the device, so for them the outer display's long edge is horizontal. In Simulator (iPhone Duo, iOS 27.1) the accessory scene is always laid out as 466×678 pt — portrait — regardless of the pose, and regardless of the app's UISupportedInterfaceOrientations. Our landscape content is therefore letterboxed into a narrow band instead of filling the display. On device, what interface orientation does the accessory scene get in the partially folded pose? Does the system rotate it to match the physical orientation of the outer display, or is it always the outer display's portrait orientation? Is Simulator's behaviour here representative, or is accessory-scene orientation one of the things that only behaves correctly on device? If we have to compensate ourselves, is there a supported way to learn the outer display's physical orientation from inside the accessory scene — reserved regions, the scene's interfaceOrientation, something else? We would rather not hardcode a rotation. Does UIRequiresFullScreen (deprecated in iOS 26) or the app's supported interface orientations constrain the accessory scene in any way?
1
0
200
Sep ’26
ARKit behaviour across the fold
Hi, Can you talk about what happens with ARKit by default when the unfolding occurs in a normal ARWorldTrackingConfiguration - does the ARSession pause at all or is tracking completely uninterrupted? Does it swap cameras when using ARFaceTrackingConfiguration! Thanks.
1
1
184
Sep ’26
Is the inner under-display camera a supported source for real-time 2D body-pose analysis?
Hi! We run continuous 2D human body-pose estimation on a front-facing camera feed: one person, full body in frame, standing a few meters away, about 30 fps. With iPhone Duo open, the camera that faces the person looking at the inner display is the under-display inner camera (builtInInnerUltraWideCamera). Published specs cap it at 1080p at 30 or 60 fps, and it is presented as the FaceTime camera. Two things are unclear from the camera tech talk and the specs: Is that 1080p format a native readout we can feed to Vision body pose, or a preview-oriented stream whose effective detail is too low for full-body pose at that distance? Is third-party use of this camera supported for a long-running analysis session, or is it intended only for video calls and live previews? Thanks
1
0
193
Sep ’26
All cameras being used
Can all the cameras on iPhone Duo be active while an app tries to translate sign language using the front facing camera and back camera to monitor hand movements and show the translation with action in the expanded display with its camera on?
1
0
170
Sep ’26
Constituent Photo Delivery
When iPhone Duo is fully open, so the outer front camera and the rear cameras face the same direction, is there a virtual AVCaptureDevice whose constituentDevices include both the outer front camera and the rear 48 MP wide camera, so that virtual device constituent photo delivery that could produce one photo from each of those cameras simultaneously, as is possible on existing iPhones with two rear cameras, using isVirtualDeviceConstituentPhotoDeliveryEnabled / virtualDeviceConstituentPhotoDeliveryEnabledDevices?
1
0
165
Sep ’26
Camera positioning
In our app we assume that user positioned the smartphone the way when the front camera (120fps) is on the left and the screen position is landscape (for watching video). We use it in our ML model. Can we adopt it somehow on iPhone Duo?
4
0
255
Sep ’26
Fixed Interface Orientation
Camera apps commonly use portrait-only orientation specifically to prevent the interface from rotating. Otherwise, toolbars, the preview aspect ratio, and camera frames all need to be reorganized just to undo the rotation. On the iPhone Duo inner display, portrait-only no longer prevents rotation. Please provide a UIRequiresFixedInterfaceOrientation key! :-)
2
1
374
Sep ’26
New download progress "banner" on iOS27
We are using AVAssetDownloadURLSession to download videos. On iOS27, when building with the iOS27 SDK, there is a new progress banner showing the file name and download progress. The user can also cancel the download from there. I have not seen this mentioned anywhere and I am wondering if there are any APIs related to this new feature? Reason: For example, we show a user notification when a download has finished but if there already was a banner displaying the status that notification is unnecessary.
Replies
2
Boosts
0
Views
473
Activity
Sep ’26
Camera direction and rotation in the half-folded pose on iPhone Duo
"Build a great camera experience for iPhone Duo" covers the device open and the device closed, including reconfiguring the session from AVCaptureDeviceDirectionCoordinator's change handler when the forward-facing camera changes. It does not cover the partially open pose, which "Leverage multiple displays and scenes on iPhone Duo" treats as a first-class hinge state with a continuous angle. Consider a camera app that keeps a live preview on screen and adopts AVCaptureDevice.RotationCoordinator — the documented remedy for cases like the square front sensor on iPhone 17, where a hard-coded 90° leaves the preview tilted — applying videoRotationAngleForHorizonLevelPreview to both the preview and the capture connection. In the partially open pose the two halves are at different physical attitudes: one stands up, the other lies flat. Which camera does AVCaptureDeviceDirectionCoordinator report as forward-facing while the hinge status is partially open? Does the hinge angle affect that decision? RotationCoordinator derives its angle from device attitude. With the two halves at different attitudes, which one does videoRotationAngleForHorizonLevelPreview reflect — the half carrying the active camera, the half hosting the preview layer, or a single device-level reference? If an app deliberately keeps the rear camera on the scene while its preview is drawn on the half facing the user, is that a supported configuration, or is the coordinator expected to override that choice? Is any of this modelled in the iPhone Duo simulator, or does the partially open pose require real hardware to test? Question 4 matters for planning: building against a rotation value that only appears on device would be worth knowing about before the hardware ships.
Replies
4
Boosts
0
Views
633
Activity
Sep ’26
Is the inner under-display camera going to be usable for real-time 2D body-pose analysis?
Hi! We run continuous 2D human body-pose estimation on a front-facing camera feed: one person, full body in frame, standing a meter or two away, 30-60fps. With iPhone Duo open, the camera that faces the person looking at the inner display is the under-display inner camera (builtInInnerUltraWideCamera). Published specs cap it at 1080p at 30 or 60 fps, and it is presented as the FaceTime camera. Two things are unclear from the camera tech talk and the specs: Is that 1080p format a native readout we can feed to Vision body pose, or a preview-oriented stream whose effective detail is too low for full-body pose at that distance? Is third-party use of this camera supported for a long-running analysis session, or is it intended only for video calls and live previews? Thanks
Replies
1
Boosts
0
Views
244
Activity
Sep ’26
About photo wallpaper on iPhone Duo
For a user-selected photo wallpaper on iPhone Duo, does iOS use one source image and automatically adapt or crop it for the outer and inner displays, or can users assign separate images to the outer and inner displays? If one image is used, is the crop or focal position shared between both displays, or can the user adjust them independently?
Replies
1
Boosts
1
Views
588
Activity
Sep ’26
Shutter button
Hello. Any recommendations on approach to placing a camera shutter button in half-open state? Also, other controls, like capture modes, etc. Camera preview is fullscreen.
Replies
1
Boosts
1
Views
219
Activity
Sep ’26
Under-display camera behavior during FaceTime
When a persistent camera session such as FaceTime is active on the inner display, the under-display camera becomes visible. What is the behavior when such a session is active and the user switches to my app, where some content, such as Navigation titles, normally sits in the region of the camera? Will the camera become hidden again and show in FaceTime that the call participant's camera is unavailable, or is there some other behavior of which I as an app developer should be aware? Thank you!
Replies
1
Boosts
0
Views
188
Activity
Sep ’26
ARKit AR View on both screens
Hello everyone! Does the ARKit ARSession qualify for an active camera session? Reading through the documentation, I understand that showing content on both iPhone Duo screens at the same time is only possible by using CameraCaptureAccessory. ARSession drives the camera but doesn't vend an AVCaptureSession that the app owns, so it isn't clear whether the availability check keys off app-owned AVCaptureSession state specifically, or off the camera actively streaming for the app. If it is currently not possible, do you see any way or workaround to show the camera stream on the outside display while the inner display is displaying an AR view? Best regards, André N.
Replies
4
Boosts
1
Views
243
Activity
Sep ’26
Shutter button
Any tips on positioning a shutter button on camera screen (fullscreen preview) when device is in half-open state? As well as other centred controls (camera mode, etc)? How to handle hinge area updates? Thanks
Replies
2
Boosts
0
Views
158
Activity
Sep ’26
CameraCaptureAccessory: orientation of the accessory scene in the partially folded (laptop) pose
We present custom content on the outer display through CameraCaptureAccessory while a capture session runs (iOS 27.1). Our app is used in the partially folded "laptop" pose: the inner display is portrait with a horizontal fold, controls below the fold, content above it. The person in front of the outer camera sits opposite the device, so for them the outer display's long edge is horizontal. In Simulator (iPhone Duo, iOS 27.1) the accessory scene is always laid out as 466×678 pt — portrait — regardless of the pose, and regardless of the app's UISupportedInterfaceOrientations. Our landscape content is therefore letterboxed into a narrow band instead of filling the display. On device, what interface orientation does the accessory scene get in the partially folded pose? Does the system rotate it to match the physical orientation of the outer display, or is it always the outer display's portrait orientation? Is Simulator's behaviour here representative, or is accessory-scene orientation one of the things that only behaves correctly on device? If we have to compensate ourselves, is there a supported way to learn the outer display's physical orientation from inside the accessory scene — reserved regions, the scene's interfaceOrientation, something else? We would rather not hardcode a rotation. Does UIRequiresFullScreen (deprecated in iOS 26) or the app's supported interface orientations constrain the accessory scene in any way?
Replies
1
Boosts
0
Views
200
Activity
Sep ’26
ARKit behaviour across the fold
Hi, Can you talk about what happens with ARKit by default when the unfolding occurs in a normal ARWorldTrackingConfiguration - does the ARSession pause at all or is tracking completely uninterrupted? Does it swap cameras when using ARFaceTrackingConfiguration! Thanks.
Replies
1
Boosts
1
Views
184
Activity
Sep ’26
Is the inner under-display camera a supported source for real-time 2D body-pose analysis?
Hi! We run continuous 2D human body-pose estimation on a front-facing camera feed: one person, full body in frame, standing a few meters away, about 30 fps. With iPhone Duo open, the camera that faces the person looking at the inner display is the under-display inner camera (builtInInnerUltraWideCamera). Published specs cap it at 1080p at 30 or 60 fps, and it is presented as the FaceTime camera. Two things are unclear from the camera tech talk and the specs: Is that 1080p format a native readout we can feed to Vision body pose, or a preview-oriented stream whose effective detail is too low for full-body pose at that distance? Is third-party use of this camera supported for a long-running analysis session, or is it intended only for video calls and live previews? Thanks
Replies
1
Boosts
0
Views
193
Activity
Sep ’26
All cameras being used
Can all the cameras on iPhone Duo be active while an app tries to translate sign language using the front facing camera and back camera to monitor hand movements and show the translation with action in the expanded display with its camera on?
Replies
1
Boosts
0
Views
170
Activity
Sep ’26
Constituent Photo Delivery
When iPhone Duo is fully open, so the outer front camera and the rear cameras face the same direction, is there a virtual AVCaptureDevice whose constituentDevices include both the outer front camera and the rear 48 MP wide camera, so that virtual device constituent photo delivery that could produce one photo from each of those cameras simultaneously, as is possible on existing iPhones with two rear cameras, using isVirtualDeviceConstituentPhotoDeliveryEnabled / virtualDeviceConstituentPhotoDeliveryEnabledDevices?
Replies
1
Boosts
0
Views
165
Activity
Sep ’26
Camera positioning
In our app we assume that user positioned the smartphone the way when the front camera (120fps) is on the left and the screen position is landscape (for watching video). We use it in our ML model. Can we adopt it somehow on iPhone Duo?
Replies
4
Boosts
0
Views
255
Activity
Sep ’26
Fixed Interface Orientation
Camera apps commonly use portrait-only orientation specifically to prevent the interface from rotating. Otherwise, toolbars, the preview aspect ratio, and camera frames all need to be reorganized just to undo the rotation. On the iPhone Duo inner display, portrait-only no longer prevents rotation. Please provide a UIRequiresFixedInterfaceOrientation key! :-)
Replies
2
Boosts
1
Views
374
Activity
Sep ’26