dve/README.md

# dve - the distributed video encoder

This is a small script to do distributed, high quality video encoding.

The script:

- breaks input video into chunks.
- distributes chunks to different servers via SSH.
- encodes those chunks in parallel.
- reassembles the chunks into final encoded video.

Why do this? So you can encode video using the best settings possible,
and use as many machines as you have available to ensure it doesn't
take forever. ☺

## Usage

By default, dve will just use your local host for encoding, which isn't likely to
improve performance. At a bare minimum, you should specify more than one host
to encode with:

    dve -l host1,host2,host3 media/test.mp4

After the encoding is completed and the chunks stitched back together, you
should end up with an output file named something like "original_new.mkv" in
your current working directory. You can adjust output naming, but note that the
output container format will currently always be mkv:

    dve -s .encoded.mkv -e ~/bin/ffmpeg -l host1,host2,host3 media/test.mp4

Encoding currently breaks input videos into 1m (60s) chunks. This should give
reasonable parallelism across a reasonable number of hosts. If you have
many hosts you may need to adjust this down using -t. If you have a small number
of hosts and a long video, you may wish to bump this up to encode larger chunks
and get marginally better compression. Values larger than 300 (15m) are
probably a waste of time.

Since the ffmpeg situation in Ubuntu has been resolved, dve no longer
tries to copy over your local copy of ffmpeg for encoding, which greatly
simplifies the script logic. This means you need to have an ffmpeg binary on
every system used for encoding, and if you specify a custom path, that custom
path should be the same on every system.

## Benchmarks

Hosts used for this benchmark were dual Xeon L5520 systems with 24GB of RAM,
16 HT cores per host. Input video file is a 4k resolution (4096x2304) test
clip, 3:47 in length.

### ffmpeg on a single host

```bash
$ time nice -n 10 ./ffmpeg -y -v error -stats -i test.mp4 -c:v libx264 -crf 20.0 -preset medium -c:a libvorbis -aq 5 -f matroska test.mkv
frame= 5459 fps=7.4 q=-1.0 Lsize=  530036kB time=00:03:47.43 bitrate=19091.2kbits/s
real    12m17.177s
user    182m57.340s
sys     0m36.240s
```

### dve with 3 hosts

```bash
$ time dve -o "-c:v libx264 -crf 20.0 -preset medium -c:a libvorbis -aq 5" -l c1,c2,c3 test.mp4
Creating chunks to encode

Computers / CPU cores / Max jobs to run
1:local / 2 / 1

Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
ETA: 1s 1left 1.57avg  local:1/7/100%/1.6s
Running parallel encoding jobs

Computers / CPU cores / Max jobs to run
1:c1 / 16 / 1
2:c2 / 16 / 1
3:c3 / 16 / 1

Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
ETA: 380s 6left 64.00avg  c1:1/1/40%/132.0s  c2:1/0/20%/0.0s  c3:1/1/40%/132.0s
Computer:jobs running/jobs completed/%of started jobs
ETA: 90s 2left 45.33avg  1:1/2/37%/138.0s  2:0/2/25%/138.0s  3:1/2/37%/138.0s
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
ETA: 42s 1left 42.14avg  c1:0/3/37%/99.7s  c2:0/2/25%/149.5s  c3:1/2/37%/149.5s
Computer:jobs running/jobs completed/%of started jobs
ETA: 50s 1left 50.29avg  1:0/3/37%/118.7s  2:0/2/25%/178.0s  3:1/2/37%/178.0s
Combining chunks into final video file
Cleaning up temporary working files

real    6m17.075s
user    1m29.630s
sys     0m22.697s
```

### Summary

dve has overhead, due to breaking the source file into chunks, transferring those chunks
across the network, retrieving the encoded chunks, and recombining into a new file.

Given these limitations, a ~2x speed increase by using 3 encoding machines is
a reasonable improvement over using a single system.

If you've got benchmarks using more hosts, please submit them!

## Installation

### SSH

SSH is used by GNU parallel to distribute the jobs to target systems.
It's recommended that you use "ssh-keygen" and "ssh-copy-id" to
setup key based authentication to all your remote hosts.

### Pre-reqs

The following need to be installed on the host running this script:

- [ffmpeg](https://www.ffmpeg.org/download.html)
- [GNU parallel](https://www.gnu.org/software/parallel/)

It's recommended that you use recent (>= 2.5.x) versions of ffmpeg to ensure
they have all the required functionality for splitting and combining the
video chunks.

### Windows

dve can be run on Windows via [cygwin](http://www.cygwin.com/).

To do so, you'll need to:

- build GNU parallel manually from source (requires make).
- install a static build of [ffmpeg for Windows](http://ffmpeg.zeranoe.com/builds/).
- install (or symlink) above into your $PATH, usually ~/bin.

You'll also need to do the following if you want to use the host to render with:

- [configure sshd](http://www.noah.org/ssh/cygwin-sshd.html)
- alter ~/.bashrc as mentioned above.

## Restrictions

- currently only generates mkv containers on output.

## ⚠ Known Issues

See the [GitHub issues page](https://github.com/nergdron/dve/issues)

## License
dve is copyright 2013-2016 by Tessa Nordgren <tessa@sudo.ca>.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see the
[GNU licenses page](http://www.gnu.org/licenses/).