Section 508 requires that all multimedia (audio and video) provide synchronized captions for people who are deaf or hard of hearing and synchronized video description for people who are blind or visually impaired. However, there are many variables in caption format, video format, video encoding, and media players that are used for multimedia. There is no one caption solution that will work in all situations.

What are closed captions?

Captions are the text placed adjacent to multimedia content that provides a visual equivalent to the audio track. In addition to a transcription of the audio dialog, captions usually indicates the identity of the speaker and may describe other significant aspects of the audio such as laughter, applause, or music.

“Closed captions” or “soft captions” are those that can be turned on or off by the user, while “open captions” or “hard captions” always appear.

How do closed captions work?

Closed captions are created by first using software to create the text and synchronize it with the multimedia content, then exporting the captions in the desired format. Most captioning software can export the captions into several different types of files.

Usually the captions are in a separate file or even more than one file with special codes controlling the timing and format of the captions. Generally the user is able to turn them on and off and to change the size and appearance of the text.

Examples:

    • To play captions for a Windows Media Video file in Windows Media Player, you need the .wmv (video) file, the .smi (SAMI) caption file, and the .asx (Windows Media Playlist) file. Windows Media Player opens the .asx file, which tells it to play the video and the caption file together.
    • To play captions for a QuickTime Movie file in QuickTime player, you need the .mov (video) file, the .txt caption file, and the qt.smi (SMIL) caption file. QuickTime Player opens the qt.smi file, which tells it to play the video and the caption file together.
  • To play captions in YouTube, you need the video file (a number of different formats work) and the .srt (SubRip) caption file. YouTube will play the video file and, if there is a caption file available, will show the option to turn on captions.

Alternatively, some formats allow the captions to be embedded into the video file so that the process can be simplified. For example, in a Windows Media File, the SAMI formatted captions can be inserted into the header of the video file. Then the user only needs to be able to access that one file to play both the video and the captions.

What are the issues with closed captions?

There are a large number of incompatible caption file formats, multimedia formats, and multimedia players. For example, SMIL captions are used by both QuickTime and RealPlayer, but the .smi file is formatted a bit differently for each player, so QuickTime SMIL caption files are incompatible with RealPlayer and vice versa. In addition, the SAMI format used by Windows Media Player uses the same extension, .smi, as the SMIL format, so the user’s operating system may not distinguish between them. If the user has assigned a default media player such as QuickTime Player for all videos, the player will open a .wmv file but ignore the captions.

Why not just use open captions?

Many applications can encode the captions so that they are permanently burned into the video and are always on. This enables the media to be played on the widest variety of players, systems and mobile devices.

However, it is preferable to give the user the ability to turn the captions on and off as desired. With open captions the user has no control over the size or appearance of the captions. If the player window is maximized the captions may be too small for some users with visual impairments.

Open captions can be created directly in many professional editing software packages. Alternatively, they can be generated and exported from a captioning application and then encoded by a separate application depending on the compatibility of the caption and media file formats.

What are some of the closed caption format options?

There is no easy choice of caption format, media file format and media player that will work for all situations. Below is a small example of some of the variations that can be considered while researching the options. Keep in mind that even though players may play many types of media files, they may not display the closed captions for them.

  • Windows Media Player:
    • Displays captions for .wmv (Windows Media Video) files
    • Compatible with the SAMI caption format
    • When using separate caption files, these three files must be in the same directory:
      • .asx file – Windows Media Playlist file calls the other files
      • .smi file – SAMI caption file
      • .wmv file – Windows Media Video file
    • The captions can be embedded into the .wmv file
      • If the captioning application is capable of exporting a wmp.txt caption file, this file can be inserted into the header and encoded into the .wmv file using encoding software
  • QuickTime Player:
    • Displays captions for .mov (QuickTime Player) files
    • Compatible with the QuickTime SMIL caption format
    • When using separate caption files, these three files must be in the same directory:
      • qt.smi file – SMIL caption file calls the other files
      • bkgrd.txt file – xml file containing the formatting instructions for the captions
      • .mov file – QuickTime Movie File
    • The captions can be embedded into the .mov file
      • Many captioning applications are capable of embedding the captions into the .mov file
  • YouTube:
    • Displays captions for many media files
    • Compatible with the SubRip caption format
    • The .srt caption file must be uploaded with the media file

What else is out there?

Common media players include:

  • Windows Media Player
  • QuickTime
  • VLC
  • iTunes/iPhone/iPod/iPad (mobile devices)
  • RealPlayer
  • Web based players with Flash or HTML Video)
  • JW Player
  • FlowPlayer
  • Nomensa
  • YouTube
  • FlvPlayback component (Flash only)

Media file formats include:

  • .wmv (Windows Media Video file)
  • .mp4 or .m4v (Mpeg-4 level 4 video often used with HTML 5 video and iTunes/iPod/iPad/iPhone)
  • .mov (QuickTime video file format)
  • .avi (audio video interleave format defined by Microsoft)
  • .flv, f4v, or .swf (Flash video formats – swf contains embedded video in Flash object)
  • .mkv (open standard format supported by open source players)

Note: Different file formats may actually contain different video encoding, (“codecs”) are methods of compressing and decompressing the video to minimize size will keeping video quality. For example, the .mp4 format may contain video that is encoded using .h264 video standard, an Apple implementation of it, or DivX, FFMpeg MPEG-4 coding, or some other form of encoding.

Closed Caption file formats include:

  • .srt (subrip format)
  • .sub (Subviewer)
  • .ttxt (3gp for video of 3g devices)
  • .smil
  • .sami
  • .dfxp (format for Flash based on ttml)
  • .ttml
  • .vtt (WebVTT format for video over the web)
  • wmp.txt (text file for encoding into .wmv format)