Attempting to update my sonarr ffmpeg encoder script to use cuda/h264-nvenc to see if it's any faster? I did stick a 2060 in this server, after all.
I did a test render (while my CPU was pegged no less) of a 4 gigabyte 28 minute video of Yakuza 0 and it crunched through it in just a few minutes, maybe like 5 or 6, tops. Got it down to 1.6 gigs.

