PushToTalk: headset "play" never starts a transmission (goes to Music) — and is PTT appropriate for voice capture to an assistant?

Question

Created 3d

Replies 1

Boosts 0

Participants 2

Hi,

We're building a voice assistant app. The user speaks to the assistant through a Bluetooth earset; the app records the speech, transcribes it and the assistant acts on it. We want the user to start and stop that capture from the earset button while the iPhone stays locked, without the Phone call UI. We prototyped this with the PushToTalk framework and have two questions.

1. Headset play button does not reach our joined PTT channel

Setup (minimal test app, nothing else running):

iPhone 14 Pro Max, iOS 26.6.1 (23G83). Entitlements: com.apple.developer.push-to-talk, aps-environment. Background modes: push-to-talk, audio, bluetooth-central.
PTChannelManager.channelManager(delegate:restorationDelegate:) → requestJoinChannel in the foreground → didJoinChannel → setAccessoryButtonEventsEnabled(true, channelUUID:) completes without error.
Audio session configured once before joining: .playAndRecord, options .allowBluetoothHFP; tried mode .default and .voiceChat. We never call setActive; audio starts only in didActivate (AVAudioEngine).

What works:

The system PTT UI talk button, phone locked: didBeginTransmitting → didActivate (earset HFP 16 kHz) → audio recorded.
A Lock Screen / Control Center control running an intent that adopts PushToTalkTransmissionIntent, phone locked: one tap starts, one tap ends; 5+ minute transmissions; app cold-launched by the system when needed.

What doesn't: pressing the earset's play/pause button (single press) while joined and idle never produces didBeginTransmitting. Same result with AirPods and with a third-party A2DP/HFP earset, locked and unlocked, Music app closed or not. mediaremoted logs, for every press:

mediaremoted: Received command from client <... bundleIdentifier = com.apple.BTAvrcp ...>: <... command = Play ...>
mediaremoted: No context for context-sensitive command <... command = Play, SenderBundleIdentifier = <com.apple.BTAvrcp> ...>
mediaremoted: Destination app com.apple.Music not available for command <... command = Play ...>
mediaremoted: Sending previously queued command <... command = Play ...>      (Music is launched and plays)

In thread 804205 you described this path as pressing "play" on an A2DP/HFP accessory "that's currently in playback mode". Questions:

a) What exactly must be true for a headset AVRCP Play/Pause to be routed to the PTT system instead of the now-playing app? Is "playback mode" a state of our channel (e.g. only while setActiveRemoteParticipant is set / audio is playing), of the accessory, or of the route?
b) Is an idle joined channel (no remote participant, nothing playing) expected to receive accessory events at all?
c) Is a press-once start / press-once end mapping supported for accessory events, or is it press-and-hold only?
d) If headset buttons cannot start a transmission from idle, is the BLE path (requestBeginTransmitting from a CoreBluetooth event) the only supported option for an earset trigger? (Our earset exposes no BLE button characteristic, so that would require firmware changes.)

2. Is this an appropriate use of PushToTalk?

The audio goes from the user to our assistant service (speech-to-text, then the assistant responds in the app or by voice). There is no second human on the channel. In thread 816408 you noted the voice communication frameworks "can't be used for any other purpose". Would capturing a user's spoken requests/notes for an AI assistant through PTT be considered a supported use, or is PTT limited to person-to-person communication? If it isn't, is there any supported API that lets a user start and stop audio capture with an earset or a Lock Screen control while the device stays locked?

A sysdiagnose and the test project are available on request.

Thanks!

Answered by DTS Engineer in 915584022

a) What exactly must be true for a headset AVRCP Play/Pause to be routed to the PTT system instead of the now-playing app?

Nothing, the PushToTalk framework doesn't actually support this (at least not directly).

What makes it "look" like things like this work:

Is "playback mode" a state of our channel (e.g. only while setActiveRemoteParticipant is set / audio is playing), of the accessory, or of the route?

...is that the accessory switched to HFP, which the PTT framework DOES support. Now, a PTT app CAN choose to support this, but it requires becoming the now playing app and handling the events yourself. See this forum post for both the background on "why" and some general guidance on implementing the "Now Playing" route.

c) Is a press-once start / press-once end mapping supported for accessory events, or is it press-and-hold only?

This depends on exactly what the accessory is generating.

d) If headset buttons cannot start a transmission from idle, is the BLE path (requestBeginTransmitting from a CoreBluetooth event) the only supported option for an earset trigger?

Generally speaking, yes, though it's possible that HID over bluetooth might also work.

Is this an appropriate use of PushToTalk?

To be honest, I don't know if it would be approved or not. My general guidance has long been that "voip" is reserved for use cases that involve "people talking to people", however, that was primarily around CallKit and voip. The PushToTalk framework does generally fall under the same limitation, however, it's also FAR better designed to protect against abuse[1] which means our concerns aren't quite a significant. I'll let you know if I get a more definitive answer but if you want an "official" answer, you need to discuss the issue with App Review.

If it isn't, is there any supported API that lets a user start and stop audio capture with an earset or a Lock Screen control while the device stays locked?

No. This was possible up until late iOS 12, however, at that point we disabled all background recording activation outside of CallKit. Ironically, that actually broke all existing PTT apps until we were able to work out a solution that involved them reporting an outgoing (NOT incoming) call from the background. The combination of the iOS 13 CallKit reporting requirements and replacing the CallKit workaround are what directly lead to the creation of the PushToTalk framework.

In any case, the bottom line here is that apps are simply not "supposed" to be able to start recording form the background, so any route you found outside of the voip/PTT "paths" would be considered a bug.

[1] In particular, it's join channel architecture is designed so that PTT wakes will ONLY happen while the app is the active PTT app, which means it's always obvious when a PTT app could be active in the background.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Answer 1 · 1d

DTS Engineer OP

Apple

1d

Accepted Answer

a) What exactly must be true for a headset AVRCP Play/Pause to be routed to the PTT system instead of the now-playing app?

Nothing, the PushToTalk framework doesn't actually support this (at least not directly).

What makes it "look" like things like this work:

Is "playback mode" a state of our channel (e.g. only while setActiveRemoteParticipant is set / audio is playing), of the accessory, or of the route?

...is that the accessory switched to HFP, which the PTT framework DOES support. Now, a PTT app CAN choose to support this, but it requires becoming the now playing app and handling the events yourself. See this forum post for both the background on "why" and some general guidance on implementing the "Now Playing" route.

c) Is a press-once start / press-once end mapping supported for accessory events, or is it press-and-hold only?

This depends on exactly what the accessory is generating.

d) If headset buttons cannot start a transmission from idle, is the BLE path (requestBeginTransmitting from a CoreBluetooth event) the only supported option for an earset trigger?

Generally speaking, yes, though it's possible that HID over bluetooth might also work.

Is this an appropriate use of PushToTalk?

To be honest, I don't know if it would be approved or not. My general guidance has long been that "voip" is reserved for use cases that involve "people talking to people", however, that was primarily around CallKit and voip. The PushToTalk framework does generally fall under the same limitation, however, it's also FAR better designed to protect against abuse[1] which means our concerns aren't quite a significant. I'll let you know if I get a more definitive answer but if you want an "official" answer, you need to discuss the issue with App Review.

If it isn't, is there any supported API that lets a user start and stop audio capture with an earset or a Lock Screen control while the device stays locked?

No. This was possible up until late iOS 12, however, at that point we disabled all background recording activation outside of CallKit. Ironically, that actually broke all existing PTT apps until we were able to work out a solution that involved them reporting an outgoing (NOT incoming) call from the background. The combination of the iOS 13 CallKit reporting requirements and replacing the CallKit workaround are what directly lead to the creation of the PushToTalk framework.

In any case, the bottom line here is that apps are simply not "supposed" to be able to start recording form the background, so any route you found outside of the voip/PTT "paths" would be considered a bug.

[1] In particular, it's join channel architecture is designed so that PTT wakes will ONLY happen while the app is the active PTT app, which means it's always obvious when a PTT app could be active in the background.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware