Add live captions
On Android, we recommend using a library such as Media3, also known as ExoPlayer, to support live captions. Media3 is developed by Google and is an open-source alternative to Android's MediaPlayer for audio and video playback. Many code examples can be found in the Camera & Media dev center. You can use DefaultTrackSelector in combination with DefaultHttpDataSource.Factory to show subtitles.
kotlin
val trackSelector = DefaultTrackSelector(baseContext).apply {
setParameters(buildUponParameters().setPreferredTextLanguage("nl"))
}
val player = ExoPlayer.Builder(context)
.setTrackSelector(trackSelector)
.build()
val dataSourceFactory = DefaultHttpDataSource.Factory()
.setUserAgent(Util.getUserAgent(context, "Appt"))
.setAllowCrossProtocolRedirects(true)
val mediaUri = Uri.parse("https://appt.org/live-video")
val mediaSource = HlsMediaSource.Factory(dataSourceFactory)
.createMediaSource(MediaItem.fromUri(mediaUri))
player.setMediaSource(mediaSource)
player.prepare()
player.playWhenReady = trueIn Jetpack Compose, we recommend using a library such as Media3, also known as ExoPlayer, to support live captions. Media3 is developed by Google and is an open-source alternative to Android's MediaPlayer for audio and video playback. Many code examples can be found in the Camera & Media dev center. You can use DefaultTrackSelector in combination with DefaultHttpDataSource.Factory to show subtitles.
kotlin
val context = LocalContext.current
var exoPlayer by remember { mutableStateOf<ExoPlayer?>(null) }
DisposableEffect(Unit) {
// Create a track selector with preferred text language
val trackSelector = DefaultTrackSelector(context).apply {
setParameters(buildUponParameters().setPreferredTextLanguage("nl"))
}
val player = ExoPlayer.Builder(context)
.setTrackSelector(trackSelector)
.build()
val dataSourceFactory = DefaultHttpDataSource.Factory()
.setUserAgent(Util.getUserAgent(context, "Appt"))
.setAllowCrossProtocolRedirects(true)
// Prepare the media source
val mediaUri = Uri.parse("https://appt.org/live-video")
val mediaSource = HlsMediaSource.Factory(dataSourceFactory)
.createMediaSource(MediaItem.fromUri(mediaUri))
player.setMediaSource(mediaSource)
player.prepare()
player.playWhenReady = true
exoPlayer = player
onDispose {
player.release()
exoPlayer = null
}
}On iOS, AVPlayer has built-in support for live video with captions. Users can automatically turn on captions via System Preferences. The easiest way to stream a live video is through AVPlayerViewController.
swift
guard let url = URL(string: "https://appt.org/live-video") else {
return
}
let player = AVPlayer(url: url)
let playerViewController = AVPlayerViewController()
playerViewController.player = player
present(playerViewController, animated: true) {
player.play()
}In SwiftUI, AVPlayer has built-in support for live video with captions. Users can automatically turn on captions via System Preferences. The easiest way to stream a live video is through VideoPlayer view.
swift
guard let url = URL(string: "https://appt.org/live-video") else {
return
}
let player = AVPlayer(url: url)
VideoPlayer(player: player)
// for a player on fullscreen
.edgesIgnoringSafeArea(.all)
.onAppear {
player.play()
}On Flutter, the video_player package does not have support for captions in live videos. Issue #50595 has been opened to request support.
Other packages, like better_player, do have support for using captions on live video. The better_player documentation contains detailed information on how to do so.
dart
BetterPlayerController controller = BetterPlayerController(
const BetterPlayerConfiguration(
controlsConfiguration: BetterPlayerControlsConfiguration(
enableAudioTracks: true,
enableSubtitles: true,
),
),
betterPlayerDataSource: BetterPlayerDataSource.network(
'https://appt.org/live-video',
liveStream: true,
useAsmsSubtitles: true,
),
);
@override
Widget build(BuildContext context) {
return BetterPlayer(controller: controller);
}React Native does not have any out-of-the box packages which enable you to provide real-time captions for live audio or video streams.
You can however embed a YouTube stream in your app and enable Live Stream Captioning.
jsx
import YoutubePlayer from "react-native-youtube-iframe";
<YoutubePlayer
height={300}
play={playing}
videoId="dQw4w9WgXcQ"
onChangeState={onStateChange}
/>In MAUI, you can use MediaElement to embed videos. Unfortunately, there is no built-in support for live captions. In this case, a custom control will be required to support it. You can consider the following options:
- Use a
Custom Handlerto create a control from scratch. - Use a
Platform Behaviorto extend theMediaElementcomponent. - Create a native binding to expose any native third-party library that supports this feature via Native Library Interop for .NET MAUI.
xml
Not available, contribute!On Xamarin, you can use MediaElement to embed videos. Unfortunately, there is no built-in support for live captions.
xml
Not available, contribute!