EngineeringAVAssetWriter crash creating CMAF compliant segments

Since macOS 11 and iOS 14 AVAssetWriter is able to write CMAF compliant segments. The Common Media Application Format (CMAF) is a standard used in streaming video and audio, it describes how to encode and split media content for streaming. It’s a format that can be delivered over both Apple HLS and MPEG DASH without having to re-encode the content.

Often media is converted into segments ahead of time, but generating segments in realtime can also be useful. For example to streaming upload a recording to a server or to directly make an unfinished stream available for consumers in a replayable way.

Bezel · Mirror your iPhone on your Mac

Perfect for app demos & presentations; Simply plug in an iPhone and it automatically shows up on your Mac.

Learn more →

It crashes on Intel

When we created CMAF compliant fMP4 segments using AVAssetWriter from a screen recording we noticed it crashed consistently after a few seconds for a select group of users. They all had the following error:

Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-16364), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x6000007293b0 {Error Domain=NSOSStatusErrorDomain Code=-16364 "(null)"}}

We first incorrectly assumed this error was related to presentation time stamps being out of order in the CMSampleBuffers that were added. After some investigations it turned out this error occurs when the following four conditions are all met:

1. The application must run on an Intel mac, Apple Silicon macs are not affected.
2. AVAssetWriter.outputFileTypeProfile set to .mpeg4CMAFCompliant
3. The AVAssetWriterInput added to the writer must have expectsMediaDataInRealTime set to true.
4. CMSampleBuffers must be appended to the input either:
   • At a variable rate, for example because the source only generates samples when something changes (like ScreenCaptureKit does) or there is a discontinuity in samples from the capture device.
   • Frames are delivered significantly faster than 60 FPS, for example on ProMotion screens.

We’ve verified this issue on macOS 12 Monterey and 13 Ventura, it doesn’t seem to affect macOS 11 Big Sur.

Workaround

Breaking any of the above 4 conditions will prevent the crash. The most feasible workarounds probably are:

• Switch away from CMAF, if you’re using it for streaming upload there is a good chance your server can also handle Apple HLS. We went with this workaround to avoid this issue all together.
• If you must use CMAF, an alternative is to keep the frame rate stable at 60 FPS or lower if the source you’re recording supports this.
• If you can’t switch away from CMAF and have an unstable or high FPS input you can’t adjust, you could disable expectsMediaDataInRealTime. That seems to also work as long as the AVAssetWriter is keeping up with the frame rate, even if you do write media data in real time.

Feedback Assistant & Sample project

This issue was submitted to Apple as FB12057159. We also created a demonstration project for this issue on GitHub.

Bezel · Mirror your iPhone on your Mac

Perfect for app demos & presentations; Simply plug in an iPhone and it automatically shows up on your Mac.

Learn more →

References

• Nonstrict. (2023, March 13). Realtime CMAF crash sample. GitHub. https://github.com/nonstrict-hq/realtime-cmaf-crash-sample