Skip to content

media-files.md

Latest commit

 

History

History
165 lines (116 loc) · 6.2 KB

media-files.md

File metadata and controls

165 lines (116 loc) · 6.2 KB

Media Files

Introduction

Several BIDS datatypes make use of media files — audio recordings, video recordings, combined audio-video recordings, and still images. This appendix defines the common file formats, metadata conventions, and codec identification schemes shared across all datatypes that use media files.

The following media suffixes are defined:

{{ MACROS___make_suffix_table(["audio", "video", "audiovideo", "image"]) }}

Datatypes that incorporate media files (for example, behavioral recordings or stimuli) define their own file-naming rules, directory placement, and datatype-specific metadata. The conventions described here apply uniformly to all such datatypes.

Relationship to the photo suffix

The media file definitions introduced here generalize the concept of all media in BIDS. The existing photo suffix (used for photographs of anatomical landmarks, head localization coils, and tissue samples) predates this framework and covers a narrower use case — still images in specific electrophysiology and microscopy datatypes.

The media suffixes (audio, video, audiovideo, image) are intended as the general-purpose mechanism for all media content in BIDS. In practice, a "photo" could equally be a video of an experimental setup with verbal narration, an audio recording describing electrode placement, or a drawing rather than a photograph. The media file framework should be generally adopted for new datatypes, and a future proposal may deprecate the photo suffix in favor of the broader image suffix with appropriate migration tooling (see bids-utils).

Supported Formats

Audio formats

{{ MACROS___make_extension_table(["wav", "mp3", "aac", "ogg"]) }}

Video container formats

{{ MACROS___make_extension_table(["mp4", "avi", "mkv", "webm"]) }}

Image formats

{{ MACROS___make_extension_table(["jpg", "png", "svg", "webp", "tif", "tiff"]) }}

When choosing a format, consider the trade-off between file size and data fidelity. Uncompressed or lossless formats (WAV, PNG, TIFF) preserve full quality but produce larger files. Lossy formats (MP3, AAC, JPEG) significantly reduce file size at the cost of some data loss.

Media Stream Metadata

Media files SHOULD be accompanied by a JSON sidecar file containing technical metadata about the media streams. The following metadata fields are defined for media files.

Duration

Applies to suffixes: audio, video, audiovideo.

{{ MACROS___make_sidecar_table("media.MediaDuration") }}

RecordingDuration reuses the existing BIDS metadata field already defined for electrophysiology recordings (EEG, iEEG, MEG, and others).

Audio stream properties

Applies to suffixes: audio, audiovideo.

{{ MACROS___make_sidecar_table("media.MediaAudioProperties") }}

Note: AudioSampleRate is used instead of the existing SamplingFrequency field because audio-video files require distinguishing the audio sampling rate from the video frame rate. The Audio prefix makes this unambiguous in multi-stream containers.

Visual properties

Applies to suffixes: video, audiovideo, image.

{{ MACROS___make_sidecar_table("media.MediaVisualProperties") }}

Video stream properties

Applies to suffixes: video, audiovideo.

{{ MACROS___make_sidecar_table("media.MediaVideoProperties") }}

Codec Identification

Codec identification uses two complementary naming systems:

FFmpeg codec names (RECOMMENDED)

The AudioCodec and VideoCodec fields use FFmpeg codec names as the RECOMMENDED convention. These names are the de facto standard in scientific computing and can be auto-extracted from media files using:

ffprobe -v quiet -print_format json -show_streams <file>

RFC 6381 codec strings (OPTIONAL)

The AudioCodecRFC6381 and VideoCodecRFC6381 fields use RFC 6381 codec strings. These provide precise codec profile and level information useful for web and broadcast interoperability.

Common codec reference

Codec FFmpeg Name RFC 6381 String Notes
H.264 / AVC h264 avc1.640028 Most widely supported
H.265 / HEVC hevc hev1.1.6.L93.B0 High efficiency
VP9 vp9 vp09.00.10.08 Open, royalty-free
AV1 av1 av01.0.01M.08 Next-gen open codec
AAC-LC aac mp4a.40.2 Default audio for MP4
MP3 mp3 mp4a.6B Legacy lossy audio
Opus opus Opus Open, low-latency audio
FLAC flac fLaC Open lossless audio
PCM 16-bit LE pcm_s16le — Uncompressed (WAV)

The FFmpeg name column shows the value to use for VideoCodec or AudioCodec. The RFC 6381 column shows the value for VideoCodecRFC6381 or AudioCodecRFC6381. RFC 6381 strings vary by profile and level; the values shown are representative examples.

Privacy Considerations

Media files — particularly audio and video recordings — may contain personally identifiable information (PII), including but not limited to:

  • Voices and speech content
  • Facial features and other physical characteristics
  • Background environments that could identify locations
  • Metadata embedded in file headers (for example, GPS coordinates, device identifiers)

Researchers MUST ensure that sharing of media files complies with the informed consent obtained from participants and with applicable privacy regulations. De-identification techniques (for example, voice distortion, face blurring, metadata stripping) SHOULD be applied where appropriate before data sharing.

Example

A complete sidecar JSON file for an audio-video recording:

{
    "RecordingDuration": 312.5,
    "VideoCodec": "h264",
    "VideoCodecRFC6381": "avc1.640028",
    "FrameRate": 30,
    "Width": 1920,
    "Height": 1080,
    "AudioCodec": "aac",
    "AudioCodecRFC6381": "mp4a.40.2",
    "AudioSampleRate": 48000,
    "AudioChannelCount": 2
}