Hi everyone,
Our team is encountering a reproducible crash when using VTLowLatencyFrameInterpolation on iOS 26.3 while processing a live LL-HLS input stream.
🤖 Environment
Device: iPhone 16
OS: iOS 26.3
Xcode: Xcode 26.3
Framework: VideoToolbox
💥 Crash Details
The application crashes with the following fatal error:
Fatal error: Swift/ContiguousArrayBuffer.swift:184: Array index out of range
The stack trace highlights the following:
VTLowLatencyFrameInterpolationImplementation processWithParameters:frameOutputHandler:
Called from VTFrameProcessor.process(parameters:)
Here is the simplified implementation block where the crash occurs. (Note: PrismSampleBuffer and PrismLLFIError are our internal custom wrapper types).
// Create `VTFrameProcessorFrame` for the source (previous) frame.
let sourcePTS = sourceSampleBuffer.presentationTimeStamp
var sourceFrame: VTFrameProcessorFrame?
if let pixelBuffer = sourceSampleBuffer.imageBuffer {
sourceFrame = VTFrameProcessorFrame(buffer: pixelBuffer, presentationTimeStamp: sourcePTS)
}
// Validate the source VTFrameProcessorFrame.
guard let sourceFrame else { throw PrismLLFIError.missingImageBuffer }
// Create `VTFrameProcessorFrame` for the next frame.
let nextPTS = nextSampleBuffer.presentationTimeStamp
var nextFrame: VTFrameProcessorFrame?
if let pixelBuffer = nextSampleBuffer.imageBuffer {
nextFrame = VTFrameProcessorFrame(buffer: pixelBuffer, presentationTimeStamp: nextPTS)
}
// Validate the next VTFrameProcessorFrame.
guard let nextFrame else { throw PrismLLFIError.missingImageBuffer }
// Calculate interpolation intervals and allocate destination frame buffers.
let intervals = interpolationIntervals()
let destinationFrames = try framesBetween(firstPTS: sourcePTS, lastPTS: nextPTS, interpolationIntervals: intervals)
let interpolationPhase: [Float] = intervals.map { Float($0) }
// Create VTLowLatencyFrameInterpolationParameters.
// This sets up the configuration required for temporal frame interpolation between the previous and current source frames.
guard let parameters = VTLowLatencyFrameInterpolationParameters(
sourceFrame: nextFrame,
previousFrame: sourceFrame,
interpolationPhase: interpolationPhase,
destinationFrames: destinationFrames
) else {
throw PrismLLFIError.failedToCreateParameters
}
try await send(sourceSampleBuffer)
// Process the frames.
// Using progressive callback here to get the next processed frame as soon as it's ready,
// preventing the system from waiting for the entire batch to finish.
for try await readOnlyFrame in self.frameProcessor.process(parameters: parameters) {
// Create an interpolated sample buffer based on the output frame.
let newSampleBuffer: PrismSampleBuffer = try readOnlyFrame.frame.withUnsafeBuffer { pixelBuffer in
try PrismLowLatencyFrameInterpolation.createSampleBuffer(from: pixelBuffer, readOnlyFrame.timeStamp)
}
// Pass the newly generated frame to the output stream.
try await send(newSampleBuffer)
}
🙋 Questions
Are there any known limitations or bugs regarding VTLowLatencyFrameInterpolation when handling live 60fps streams?
Are there any undocumented constraints we should be aware of regarding source/previous frame timing, pixel buffer attributes, or how destinationFrames and interpolationPhase arrays must be allocated?
Is a "warm-up" sequence recommended after startSession() before making the first process(parameters:) call?
1
0
444