148 lines
5.3 KiB
Markdown
148 lines
5.3 KiB
Markdown
# dve - the distributed video encoder
|
|
|
|
This is a small script to do distributed, high quality video encoding.
|
|
|
|
The script:
|
|
|
|
- breaks input video into chunks.
|
|
- distributes chunks to different servers via SSH.
|
|
- encodes those chunks in parallel.
|
|
- reassembles the chunks into final encoded video.
|
|
|
|
Why do this? So you can encode video using the best settings possible,
|
|
and use as many machines as you have available to ensure it doesn't
|
|
take forever. ☺
|
|
|
|
## Usage
|
|
|
|
By default, dve will just use your local host for encoding, which isn't likely to
|
|
improve performance. At a bare minimum, you should specify more than one host
|
|
to encode with:
|
|
|
|
dve.sh -l host1,host2,host3 media/test.mp4
|
|
|
|
If you're using a statically linked ffmpeg binary (recommended), then you'll also
|
|
want to specify the path to that binary:
|
|
|
|
dve.sh -e ~/bin/ffmpeg -l host1,host2,host3 media/test.mp4
|
|
|
|
After the encoding is completed and the chunks stitched back together, you
|
|
should end up with an output file named "test.mp4_new.mkv" in your current working
|
|
directory. You can adjust output naming, but note that the output container format
|
|
will currently always be mkv:
|
|
|
|
dve.sh -s .encoded.mkv -e ~/bin/ffmpeg -l host1,host2,host3 media/test.mp4
|
|
|
|
## Benchmarks
|
|
|
|
Hosts used for this benchmark were dual Xeon L5520 systems with 24GB of RAM,
|
|
16 HT cores per host. Input video file is a 4k resolution (4096x2304) test
|
|
clip, 3:47 in length.
|
|
|
|
### ffmpeg on a single host
|
|
|
|
$ time nice -n 10 ./ffmpeg -y -v error -stats -i test.mp4 -c:v libx264 -crf 20.0 -preset medium -c:a libvorbis -aq 5 -f matroska test.mkv
|
|
frame= 5459 fps=7.4 q=-1.0 Lsize= 530036kB time=00:03:47.43 bitrate=19091.2kbits/s
|
|
real 12m17.177s
|
|
user 182m57.340s
|
|
sys 0m36.240s
|
|
|
|
### dve with 3 hosts
|
|
|
|
$ time dve.sh -o "-c:v libx264 -crf 20.0 -preset medium -c:a libvorbis -aq 5" -l c1,c2,c3 test.mp4
|
|
Creating chunks to encode
|
|
|
|
Computers / CPU cores / Max jobs to run
|
|
1:local / 2 / 1
|
|
|
|
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
|
|
ETA: 1s 1left 1.57avg local:1/7/100%/1.6s
|
|
Running parallel encoding jobs
|
|
|
|
Computers / CPU cores / Max jobs to run
|
|
1:c1 / 16 / 1
|
|
2:c2 / 16 / 1
|
|
3:c3 / 16 / 1
|
|
|
|
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
|
|
ETA: 380s 6left 64.00avg c1:1/1/40%/132.0s c2:1/0/20%/0.0s c3:1/1/40%/132.0s
|
|
Computer:jobs running/jobs completed/%of started jobs
|
|
ETA: 90s 2left 45.33avg 1:1/2/37%/138.0s 2:0/2/25%/138.0s 3:1/2/37%/138.0s
|
|
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
|
|
ETA: 42s 1left 42.14avg c1:0/3/37%/99.7s c2:0/2/25%/149.5s c3:1/2/37%/149.5s
|
|
Computer:jobs running/jobs completed/%of started jobs
|
|
ETA: 50s 1left 50.29avg 1:0/3/37%/118.7s 2:0/2/25%/178.0s 3:1/2/37%/178.0s
|
|
Combining chunks into final video file
|
|
Cleaning up temporary working files
|
|
|
|
real 6m17.075s
|
|
user 1m29.630s
|
|
sys 0m22.697s
|
|
|
|
|
|
### Summary
|
|
|
|
dve has overhead, due to breaking the source file into chunks, transferring those chunks
|
|
across the network, retrieving the encoded chunks, and recombining into a new file.
|
|
|
|
Given these limitations, a ~2x speed increase by using 3 encoding machines is
|
|
a reasonable improvement over using a single system.
|
|
|
|
If you've got benchmarks using more hosts, please submit them!
|
|
|
|
## Installation
|
|
|
|
### SSH
|
|
|
|
SSH is used by GNU parallel to distribute the jobs to target systems.
|
|
It's recommended that you use "ssh-keygen" and "ssh-copy-id" to
|
|
setup key based authentication to all your remote hosts.
|
|
|
|
### Pre-reqs
|
|
|
|
The following need to be installed on the host running this script:
|
|
|
|
- [ffmpeg](http://johnvansickle.com/ffmpeg/)
|
|
- [GNU parallel](https://www.gnu.org/software/parallel/)
|
|
|
|
It will automatically copy the encoder binary to the target systems and run
|
|
it there, but you will need to make sure any required shared libraries are installed
|
|
on the target system.
|
|
|
|
It's recommended that you use the latest ffmpeg statically linked binaries (above),
|
|
which will also improve your chances of transcoding new / strange video formats.
|
|
|
|
## Restrictions
|
|
|
|
- currently only generates mkv containers on output.
|
|
|
|
## ⚠ Known Issues
|
|
|
|
Ubuntu ships the libav fork version ffmpeg wrapper instead of the actual ffmpeg
|
|
release. This version is missing many necessary features, including the
|
|
[concat demuxer](https://www.ffmpeg.org/faq.html#How-can-I-concatenate-video-files_003f)
|
|
used to stitch encoded chunks back together.
|
|
|
|
For now, you should download and use the statically linked ffmpeg
|
|
binaries listed above.
|
|
|
|
For more background about this fork, see:
|
|
[The FFmpeg/Libav situation](http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html)
|
|
|
|
## License
|
|
dve is copyright 2013 by Graeme Humphries <graeme@sudo.ca>.
|
|
|
|
This program is free software: you can redistribute it and/or modify
|
|
it under the terms of the GNU General Public License as published by
|
|
the Free Software Foundation, either version 3 of the License, or
|
|
(at your option) any later version.
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
GNU General Public License for more details.
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
along with this program. If not, see the
|
|
[GNU licenses page](http://www.gnu.org/licenses/).
|