FFmpeg Package

FFmpeg is a command line tool for manipulating or processing media data. 30Dec2019

FFmpeg Function

The basic function of FFmpeg is to process all input files according to specified options into the required output files.

Command

A FFmpeg command is a space-separated switch-based instruction. The instruction of ffmpeg command can be divided into four parts The first part is the command ffmpeg itself. The second part is the global options for the FFmpeg engine. The third part is the input area. And the four part is the output area.

Syntax

ffmpeg [global_options] {[input_file_options] -i input_url} ⋯ {[output_file_options] output_url} ⋯

Arguments

global_optionsoptional global options for ffmpeg input_file_optionsoptional input file options -iswitch for input file input_urllocation of input file output_file_optionsoptional output file options output_urllocation of output file

Remarks

The typical properties of FFmpeg instruction are.

• Global options for ffmpeg must be specified immediately after the command ffmpeg first. e.g. verbosity level
• A switch option is composed of a switch and a space-separated sub-option group. For example -vf scale=iw:ih:flags=neighbor where -vf is the switch option, scale, flags are ":"-separated switch-options, and iw:ih, neighbor are corresponding ":"-separated sub-options.
• For the global options of ffmpeg, all global switch options are specified just after command ffmpeg.
• However, for both input file options and output file options, all file switch options are placed immediately before the corresponding file option. In other words, these swith options are only used to specify the next immediately specified file option. Therefore individual switch options are needed to be specified for each specified file option which are needed .
• FFmpeg supports multiple input files of any supported files. Each input file location is specified by the -i switch. And all corresponding input file options of each file are immediately placed before the -i switch accordingly.
• FFmpeg also support multiple output files of any supported files. For each output file location, no ffmpeg switch is needed because ffmpeg always considers any space-separated strings that cannot be interpreted as a switched option to be an output file option. And all corresponding output file options of each file are immediately place before the output file location accordingly.
• The best practice for specifying option
  1. to specify all global options
  2. to specify all input file options for all input files one after one
  3. to specify all output file options for all output files one after one

Inputs

FFmpeg reads from an arbitrary number of input "files" (which can be regular files, pipes, network streams, grabbing devices, etc.), specified by the -i option, To refer to input files in options, you must use their indices (0-based). E.g. the first input file is 0, the second is 1, etc. Similarly, streams within a file are referred to by their indices. E.g. 2:3 refers to the fourth stream in the third input file.

Outputs

FFmpeg writes to an arbitrary number of output "files", which are specified by a plain output url. Anything found on the command line which cannot be interpreted as an option is considered to be an output url.

Processes

The transcoding process in ffmpeg for each output is similar under the control of parameters specified in options. ffmpeg calls the libavformat library (containing demuxers) to read input files and get packets containing encoded data from them. When there are multiple input files, ffmpeg tries to keep them synchronized by tracking lowest timestamp on any active input stream.
Encoded packets are then passed to the decoder except processing streamcopy. The decoder produces uncompressed frames (raw video/PCM audio/...) which can be processed further by filtering. After filtering, the frames are passed to the encoder, which encodes them and outputs encoded packets. Finally those are passed to the muxer, which writes the encoded packets to the output file.

Filtering

Before encoding, ffmpeg can process raw audio and video frames using filters from the libavfilter library. Several chained filters form a filter graph. ffmpeg distinguishes between two types of filtergraphs: simple and complex.

Simple filtergraphs

Simple filtergraphs are those that have exactly one input and output, both of the same type. they can be represented by simply inserting an additional step between decoding and encoding.
Simple filtergraphs are configured with the per-stream -filter option (with -vf and -af aliases for video and audio respectively).

Complex filtergraphs

Complex filtergraphs are those which cannot be described as simply a linear processing chain applied to one stream. This is the case, for example, when the graph has more than one input and/or output, or when output stream type is different from input.
Complex filtergraphs are configured with the -filter_complex option. Note that this option is global, since a complex filtergraph, by its nature, cannot be unambiguously associated with a single stream or file. The -lavfi option is equivalent to -filter_complex.
A trivial example of a complex filtergraph is the overlay filter, which has two video inputs and one video output, containing one video overlaid on top of the other. Its audio counterpart is the amix filter.

Stream copy

Stream copy is a mode selected by supplying the copy parameter to the -codec option. It makes ffmpeg omit the decoding and encoding step for the specified stream, so it does only demuxing and muxing. It is useful for changing the container format or modifying container-level metadata. The diagram above will, in this case, simplify to this:
Since there is no decoding or encoding, it is very fast and there is no quality loss. However, it might not work in some cases because of many factors. Applying filters is obviously also impossible, since filters work on uncompressed data.

Stream selection

ffmpeg provides the -map option for manual control of stream selection in each output file. Users can skip -map and let ffmpeg perform automatic stream selection as described below. The -vn / -an / -sn / -dn options can be used to skip inclusion of video, audio, subtitle and data streams respectively, whether manually mapped or automatically selected, except for those streams which are outputs of complex filtergraphs.
The sub-sections that follow describe the various rules that are involved in stream selection. The examples that follow next show how these rules are applied in practice. While every effort is made to accurately reflect the behavior of the program, FFmpeg is under continuous development and the code may have changed since the time of this writing.

Automatic stream selection

In the absence of any map options for a particular output file, ffmpeg inspects the output format to check which type of streams can be included in it, viz. video, audio and/or subtitles. For each acceptable stream type, ffmpeg will pick one stream, when available, from among all the inputs. It will select that stream based upon the following criteria: for video, it is the stream with the highest resolution, for audio, it is the stream with the most channels, for subtitles, it is the first subtitle stream found but there’s a caveat. The output format’s default subtitle encoder can be either text-based or image-based, and only a subtitle stream of the same type will be chosen. In the case where several streams of the same type rate equally, the stream with the lowest index is chosen. Data or attachment streams are not automatically selected and can only be included using -map.

Manual stream selection

When -map is used, only user-mapped streams are included in that output file, with one possible exception for filtergraph outputs described below.

Complex filtergraphs

If there are any complex filtergraph output streams with unlabeled pads, they will be added to the first output file. This will lead to a fatal error if the stream type is not supported by the output format. In the absence of the map option, the inclusion of these streams leads to the automatic stream selection of their types being skipped. If map options are present, these filtergraph streams are included in addition to the mapped streams. Complex filtergraph output streams with labeled pads must be mapped once and exactly once.

Stream handling

Stream handling is independent of stream selection, with an exception for subtitles described below. Stream handling is set via the -codec option addressed to streams within a specific output file. In particular, codec options are applied by ffmpeg after the stream selection process and thus do not influence the latter. If no -codec option is specified for a stream type, ffmpeg will select the default encoder registered by the output file muxer. An exception exists for subtitles. If a subtitle encoder is specified for an output file, the first subtitle stream found of any type, text or image, will be included. ffmpeg does not validate if the specified encoder can convert the selected stream or if the converted stream is acceptable within the output format. This applies generally as well: when the user sets an encoder manually, the stream selection process cannot check if the encoded stream can be muxed into the output file. If it cannot, ffmpeg will abort and all output files will fail to be processed.

Examples

The following examples illustrate the behavior, quirks and limitations of ffmpeg’s stream selection methods. They assume the following three input files. input file 'A.avi' stream 0: video 640x360 stream 1: audio 2 channels input file 'B.mp4' stream 0: video 1920x1080 stream 1: audio 2 channels stream 2: subtitles (text) stream 3: audio 5.1 channels stream 4: subtitles (text) input file 'C.mkv' stream 0: video 1280x720 stream 1: audio 2 channels stream 2: subtitles (image) Example: automatic stream selection ffmpeg -i A.avi -i B.mp4 out1.mkv out2.wav -map 1:a -c:a copy out3.mov There are three output files specified, and for the first two, no -map options are set, so ffmpeg will select streams for these two files automatically. out1.mkv is a Matroska container file and accepts video, audio and subtitle streams, so ffmpeg will try to select one of each type. For video, it will select stream 0 from B.mp4, which has the highest resolution among all the input video streams. For audio, it will select stream 3 from B.mp4, since it has the greatest number of channels. For subtitles, it will select stream 2 from B.mp4, which is the first subtitle stream from among A.avi and B.mp4. out2.wav accepts only audio streams, so only stream 3 from B.mp4 is selected. For out3.mov, since a -map option is set, no automatic stream selection will occur. The -map 1:a option will select all audio streams from the second input B.mp4. No other streams will be included in this output file. For the first two outputs, all included streams will be transcoded. The encoders chosen will be the default ones registered by each output format, which may not match the codec of the selected input streams. For the third output, codec option for audio streams has been set to copy, so no decoding-filtering-encoding operations will occur, or can occur. Packets of selected streams shall be conveyed from the input file and muxed within the output file. Example: automatic subtitles selection ffmpeg -i C.mkv out1.mkv -c:s dvdsub -an out2.mkv Although out1.mkv is a Matroska container file which accepts subtitle streams, only a video and audio stream shall be selected. The subtitle stream of C.mkv is image-based and the default subtitle encoder of the Matroska muxer is text-based, so a transcode operation for the subtitles is expected to fail and hence the stream isn’t selected. However, in out2.mkv, a subtitle encoder is specified in the command and so, the subtitle stream is selected, in addition to the video stream. The presence of -an disables audio stream selection for out2.mkv. Example: unlabeled filtergraph outputs ffmpeg -i A.avi -i C.mkv -i B.mp4 -filter_complex "overlay" out1.mp4 out2.srt A filtergraph is setup here using the -filter_complex option and consists of a single video filter. The overlay filter requires exactly two video inputs, but none are specified, so the first two available video streams are used, those of A.avi and C.mkv. The output pad of the filter has no label and so is sent to the first output file out1.mp4. Due to this, automatic selection of the video stream is skipped, which would have selected the stream in B.mp4. The audio stream with most channels viz. stream 3 in B.mp4, is chosen automatically. No subtitle stream is chosen however, since the MP4 format has no default subtitle encoder registered, and the user hasn’t specified a subtitle encoder. The 2nd output file, out2.srt, only accepts text-based subtitle streams. So, even though the first subtitle stream available belongs to C.mkv, it is image-based and hence skipped. The selected stream, stream 2 in B.mp4, is the first text-based subtitle stream. Example: labeled filtergraph outputs ffmpeg -i A.avi -i B.mp4 -i C.mkv -filter_complex "[1:v]hue=s=0[outv];overlay;aresample" \ -map '[outv]' -an out1.mp4 \ out2.mkv \ -map '[outv]' -map 1:a:0 out3.mkv The above command will fail, as the output pad labelled [outv] has been mapped twice. None of the output files shall be processed. ffmpeg -i A.avi -i B.mp4 -i C.mkv -filter_complex "[1:v]hue=s=0[outv];overlay;aresample" \ -an out1.mp4 \ out2.mkv \ -map 1:a:0 out3.mkv This command above will also fail as the hue filter output has a label, [outv], and hasn’t been mapped anywhere. The command should be modified as follows, ffmpeg -i A.avi -i B.mp4 -i C.mkv -filter_complex "[1:v]hue=s=0,split=2[outv1][outv2];overlay;aresample" \ -map '[outv1]' -an out1.mp4 \ out2.mkv \ -map '[outv2]' -map 1:a:0 out3.mkv The video stream from B.mp4 is sent to the hue filter, whose output is cloned once using the split filter, and both outputs labelled. Then a copy each is mapped to the first and third output files. The overlay filter, requiring two video inputs, uses the first two unused video streams. Those are the streams from A.avi and C.mkv. The overlay output isn’t labelled, so it is sent to the first output file out1.mp4, regardless of the presence of the -map option. The aresample filter is sent the first unused audio stream, that of A.avi. Since this filter output is also unlabelled, it too is mapped to the first output file. The presence of -an only suppresses automatic or manual stream selection of audio streams, not outputs sent from filtergraphs. Both these mapped streams shall be ordered before the mapped stream in out1.mp4. The video, audio and subtitle streams mapped to out2.mkv are entirely determined by automatic stream selection. out3.mkv consists of the cloned video output from the hue filter and the first audio stream from B.mp4.

Source and Reference

https://ffmpeg.org/ffmpeg.html
https://ffmpeg.org/ffmpeg-all.html