[iOS] useVideo freezes after the first frame: AVPlayerItemVideoOutput self-suspends and RNSkAppleVideo never recovers (play and seek both go silent)

[fredabila]

Description

On iOS, useVideo renders the first frame and then freezes permanently: after the initial frame, neither playback (paused → false) nor seek ever produces another frame. The decoder looks alive (shaders over it keep animating) but the picture is a still image.

We instrumented the returned currentTime/frames from JS while issuing seeks: currentTime stays frozen while seeks keep being issued — i.e. even seek() (which calls expectFrame() natively) stops producing frames after the first one.

Root cause

RNSkAppleVideo pulls frames from an AVPlayerItemVideoOutput in onDisplayLink() via hasNewPixelBufferForItemTime: / copyPixelBufferForItemTime:.

Apple documents that AVPlayerItemVideoOutput suspends itself when the client stops pulling buffers for a while; once suspended, hasNewPixelBufferForItemTime: returns NO forever. The supported recovery is the pull-delegate dance (setDelegate: + requestNotificationOfMediaDataChangeWithAdvanceInterval: → outputMediaDataWillChange:), which RNSkAppleVideo doesn't implement. The display link is paused whenever the video is paused and not awaiting a frame — exactly the "client stopped pulling" condition — so the output suspends and the video is dead from then on: play() unpauses the display link but every hasNewPixelBufferForItemTime: check returns NO, and seek()'s expectFrame() can never deliver.

There is a second, independent hazard in the same file: the AVPlayer is constructed on the worklet runtime thread (Skia.Video() is invoked from video-metadata-runtime in useVideoLoading) and play()/pause() are then called from Reanimated's UI thread. AVPlayer is documented for main-thread use, and cross-thread play is the classic "silently does nothing" call.

Reproduction

• @shopify/react-native-skia 2.6.2, RN 0.86, Reanimated 4.5, new architecture, Expo SDK 57, iOS device build.
• useVideo(url, { paused, seek, volume }) with paused/seek shared values (the documented pattern).
• Mount with paused: true (a seek fires once and one frame renders), then set paused.value = false.
• Result: frame never changes; subsequent writes to seek.value do nothing. currentTime never advances.

Proposed fix (patch we're shipping via patch-package)

Short of implementing the full pull-delegate protocol, re-attaching a fresh AVPlayerItemVideoOutput recovers reliably — a newly attached output reports the current frame as new. We detect starvation in onDisplayLink() (~20 consecutive display-link ticks with no new buffer while _isPlaying || _waitingForFrame) and swap the output; we also dispatch play/pause to the main queue.

void RNSkAppleVideo::refreshVideoOutput() {
  if (!_playerItem) return;
  if (_videoOutput) [_playerItem removeOutput:_videoOutput];
  _videoOutput = [[AVPlayerItemVideoOutput alloc]
      initWithOutputSettings:getOutputSettings()];
  [_playerItem addOutput:_videoOutput];
}

void RNSkAppleVideo::onDisplayLink() {
  CMTime outputItemTime =
      [_videoOutput itemTimeForHostTime:CACurrentMediaTime()];

  if (![_videoOutput hasNewPixelBufferForItemTime:outputItemTime] &&
      (_isPlaying || _waitingForFrame)) {
    if (++_noFrameCount > 20) {   // ~1/3s starved while we SHOULD have frames
      refreshVideoOutput();
      _noFrameCount = 0;
    }
    return;
  }
  // ... existing copyPixelBufferForItemTime path, with _noFrameCount = 0 on success
}

void RNSkAppleVideo::play() {
  if (_player) {
    AVPlayer *player = _player;
    dispatch_async(dispatch_get_main_queue(), ^{ [player play]; });
    _isPlaying = true;
    _noFrameCount = 0;
    _displayLink.paused = NO;
  }
}

The complete diff is small (one new ivar + method in RNSkAppleVideo.h, the changes above in RNSkAppleVideo.mm). Happy to open a PR if this direction is acceptable — or the fuller fix is adopting the AVPlayerItemOutputPullDelegate protocol so the output never silently dies in the first place.

Caveat for transparency: the diagnosis is from observed device behaviour plus reading the implementation; we're verifying the patch on-device in our next build and will report back.