Stacking RTSP Streams with ffmpeg

Because ffmpeg is awesome!

Created: 2021-01-20

Updated: 2022-01-04

Background

I have 4 RTSP Streams available from my local NVR. I wanted to combine them into a single video.

Since I'm also using these as part of a CV Project with a Jetson Nano, I wanted to be able to grab frames every n seconds

• Recommended Flags as Streams start Synchronously

ffmpeg -fflags nobuffer -flags low_delay -strict experimental

• Multiple Inputs

-i input0 -i input1 -i input2 -i input3 ...

• Xstack see Stackoverflow

-filter_complex "[0:v][1:v][2:v][3:v]xstack=inputs=4:layout=0_0|w0_0|0_h0|w0_h0[v]"

• Map to output

-map "[v]" output.mp4

• All Together

ffmpeg -fflags nobuffer -flags low_delay -strict experimental \
    -i input0 -i input1 -i input2 -i input3 -filter_complex \
    "[0:v][1:v][2:v][3:v]xstack=inputs=4:layout=0_0|w0_0|0_h0|w0_h0[v]" -map "[v]" \
    output.mp4

Mosaic to Images

Since we're using a complex filtergraph we can't add -vf fps=x after -map

Make the following modification to the filtergraph:

"[0:v][1:v][2:v][3:v]xstack=inputs=4:layout=0_0|w0_0|0_h0|w0_h0[v];[v]fps=1[img]"

Basically, we're saying:

• take the output of xstack, 'v' ...
• take 1 frame per second ...
• call this operation 'img'

Then we just change the name map uses to to -map "[img]"

Lastly, since this won't go to a video file, modify the output to:

Image_%08d.jpg

Which is a template that names images sequentially:

• Image_00000001.jpg
• Image_00000002.jpg ...

All Together

Using variables for input

ffmpeg -fflags nobuffer -flags low_delay -strict experimental -i "$I0" -i "$I1" -i "$I2" -i "$I3" -filter_complex \
  "[0:v][1:v][2:v][3:v]xstack=inputs=4:layout=0_0|w0_0|0_h0|w0_h0[v];[v]fps=1[img]" -map "[img]" \
  /somepath/Image_%08d.jpg