1. Perceivable · 1.2 Time-based Media

1.2.5 — Audio Description (Prerecorded)

Level AA

Engineering guidance on this page is adapted from Appt.org (Appt Foundation), used under the CC BY-SA 4.0 license. In keeping with that license, the adapted content on this page is likewise available under CC BY-SA 4.0. "Appt" is a registered trademark of the Appt Foundation; this page is not affiliated with or endorsed by Appt.

Add audio description

As of Android 4.1, the MediaPlayer has support for multiple audio tracks. Use the selectTrack) method to select the correct audio track.

The code example belows shows a basic implementation of selecting an audio description track embedded inside a video.

kotlin

val player = MediaPlayer.create(this, R.raw.video)
try {
    player.trackInfo.forEachIndexed { index, trackInfo ->
        if (trackInfo.trackType == TrackInfo.MEDIA_TRACK_TYPE_AUDIO) {
            player.selectTrack(index)
            return@forEachIndexed
        }
    }
    player.start()
} catch (e: Exception) {
    e.printStackTrace()
}

In Jetpack Compose, the MediaPlayer has support for multiple audio tracks. Use the selectTrack) method to select the correct audio track.

The code example belows shows a basic implementation of selecting an audio description track embedded inside a video.

kotlin

var mediaPlayer: MediaPlayer? by remember { mutableStateOf(null) }
var error by remember { mutableStateOf<String?>(null) }
val currentContext = LocalContext.current

DisposableEffect(Unit) {
    val player = MediaPlayer.create(currentContext, R.raw.video)
    mediaPlayer = player

    try {
        player.trackInfo.forEachIndexed { index, trackInfo ->
            if (trackInfo.trackType == MediaPlayer.TrackInfo.MEDIA_TRACK_TYPE_AUDIO) {
                player.selectTrack(index)
                return@forEachIndexed
            }
        }
        player.start()
    } catch (e: Exception) {
        e.printStackTrace()
        error = e.message
    }

    onDispose {
        player.release()
        mediaPlayer = null
    }
}

On iOS, AVPlayer has support to add audio description. Users can enable audio description automatically through System Preferences. Turning on audio description works automatically if you add the public.accessibility.describes-video property to the audio description track.

The code example below shows a basic implementation of enabling audio description embedded inside a video.

swift

let composition = AVMutableComposition()

guard let videoUrl = Bundle.main.url(
    forResource: "Appt", 
    withExtension: "mp4"
) else { 
    return 
}

let videoAsset = AVURLAsset.init(url: videoUrl)

// Add video track to composition
if let videoTrack = try await videoAsset.loadTracks(withMediaType: .video).first,
   let videoCompositionTrack = composition.addMutableTrack(
        withMediaType: .video,
        preferredTrackID: kCMPersistentTrackID_Invalid
   ) {
    do {
        try await videoCompositionTrack.insertTimeRange(
            CMTimeRange(start: .zero, duration: videoAsset.load(.duration)),
            of: videoTrack,
            at: .zero
        )
    } catch { }
}

// Find and add the audio description track
for track in try await videoAsset.load(.tracks) {
    if try await track.load(.mediaCharacteristics).contains(.describesVideoForAccessibility) {
        if let audioCompositionTrack = composition.addMutableTrack(
                withMediaType: track.mediaType,
                preferredTrackID: kCMPersistentTrackID_Invalid
            ) {
            do {
                try await audioCompositionTrack.insertTimeRange(
                    CMTimeRange(start: .zero, duration: videoAsset.load(.duration)),
                    of: track,
                    at: .zero
                )
            } catch { }
            break
        }
    }
}

In SwiftUI, you can integrate AVPlayer which has support of adding audio description. Users can enable audio description automatically through System Preferences. Turning on audio description works automatically if you add the public.accessibility.describes-video property to the audio description track.

The code example below shows a basic implementation of enabling audio description embedded inside a video.

swift

let composition = AVMutableComposition()

guard let videoUrl = Bundle.main.url(
    forResource: "Appt", 
    withExtension: "mp4"
) else { 
    return 
}

let videoAsset = AVURLAsset.init(url: videoUrl)

// Add video track to composition
if let videoTrack = try await videoAsset.loadTracks(withMediaType: .video).first,
   let videoCompositionTrack = composition.addMutableTrack(
        withMediaType: .video,
        preferredTrackID: kCMPersistentTrackID_Invalid
   ) {
    do {
        try await videoCompositionTrack.insertTimeRange(
            CMTimeRange(start: .zero, duration: videoAsset.load(.duration)),
            of: videoTrack,
            at: .zero
        )
    } catch { }
}

// Find and add the audio description track
for track in try await videoAsset.load(.tracks) {
    if try await track.load(.mediaCharacteristics).contains(.describesVideoForAccessibility) {
        if let audioCompositionTrack = composition.addMutableTrack(
                withMediaType: track.mediaType,
                preferredTrackID: kCMPersistentTrackID_Invalid
            ) {
            do {
                try await audioCompositionTrack.insertTimeRange(
                    CMTimeRange(start: .zero, duration: videoAsset.load(.duration)),
                    of: track,
                    at: .zero
                )
            } catch { }
            break
        }
    }
}

With Flutter, you can use better_player to let users select different audio tracks.

The code example belows shows a basic implementation of changing audio tracks.

dart

BetterPlayerController controller = BetterPlayerController(
    const BetterPlayerConfiguration(
      controlsConfiguration: BetterPlayerControlsConfiguration(
        enableAudioTracks: true,
      ),
    ),
    betterPlayerDataSource: BetterPlayerDataSource.file(
      'assets/appt.mp4',
      useAsmsSubtitles: true,
    ),
  );

void changeAudioTrack(int track) {
  if (controller.betterPlayerAsmsAudioTracks?[track] != null) {
    controller.setAudioTrack(controller.betterPlayerAsmsAudioTracks![track]);
  }
}

@override
Widget build(BuildContext context) {
  return BetterPlayer(controller: controller);
}

On React Native, the React-Native-Video package has support for switching audio tracks. This allows you to offer users a way to switch to an audio description track.

Note: The audio tracks must be encoded in the file, this is not something you add programmatically.

jsx

import Video from 'react-native-video';

<Video
    selectedAudioTrack={{
        type: "audio-description",
        value: "en"
    }}
/>

In MAUI, you can use MediaElement to embed videos. Unfortunately, there is no built-in support to switch to an audio description track. In this case, a custom control will be required to support it. You can consider the following options:

Use a Custom Handler to create a control from scratch.
Use Platform Behavior to extend the MediaElement component.
Create a native binding to expose any native third-party library that supports this feature via Native Library Interop for .NET MAUI.

xml

Not available, contribute!

← Back to Engineering Support

1.2.5 — Audio Description (Prerecorded)

Add audio description

Android

Jetpack Compose

iOS

SwiftUI

Flutter

React Native

.NET MAUI

Xamarin