This tutorial aims at guiding advanced encoding on linux priviledging native tools. No virtual machine nor wine will be necessary here, except for specific needs. A few tools are still used with mono however, especially for OCRing subtitles.

This guide will mainly focus on the use of VapourSynth, the main alternative to Avisynth, and ffmpeg. If you are a windows user wanting to encode with VapourSynth, you can still have a look at this guide. The python scripts presented will work seamlessly. However, the bash shell script won't be usable on Windows.

I would like to warmly thank AHD for the Encoding Guide v3 by which this guide is inspired.


[1. Introduction to Vapoursynth]

[2. Demuxing]

[3. Encoding]

[4. Filtering]

[5. Remuxing]

[6. Appendix]


[1. Introduction to Vapoursynth]

[1.1 What is Vapoursynth?]

VapourSynth (VS) is an application for video manipulation, just like Avisynth. It has a core library written in C++ and a Python module to allow video scripts to be created.

VapourSynth Editor (VSE) is the script editor which is commonly used, with syntax completion and fast preview support.

Vapoursynth scripts are written in python. The main difference compared to Avisynth is the assignment of variables. You have to specify the variable names each time you’re doing something:

variable = doSomething()

You will have more things to write but it will be clearer to which variable you're assigning the operation.

[1.2 Installation of VapourSynth]

The prefered way to install VS is to get the needed packages from the repositories of your distribution, for its simplicity: python, x264, zimg, ffms2, l-smash, vapoursynth, vapoursynth-editor. However, I will here detail how to compile them if you want to have the latest versions which may not be available from your package manager. The initial instructions are taken from here. For Arch-based system users, you should be able to compile everything from AUR.

Compilation: Hide

In the following, we will make the installation in ~/apps. Make sure this directory exists or change it if you like.

1.2.1 Preparation

[Ubuntu]

Check to see if the "site-packages" directory exists:

if test -d /usr/local/lib/python3.6/site-packages; then echo "exists"; else echo "doesn't exist"; fi

If it does, moving all the files to dist-packages and deleting the directory is recommended.

Create a link to the "dist-packages" directory:

sudo ln -s /usr/local/lib/python3.6/dist-packages /usr/local/lib/python3.6/site-packages

[Fedora]

Create now a link for the plugins install directory to the VS one:

sudo ln -s /usr/lib64/vapoursynth /usr/local/lib/vapoursynth

For Fedora users, in the following when running ./configure for ffmpeg, l-smash, vapoursynth, x264 and zimg, add --prefix=/usr.

1.2.2 Dependencies for VS and its plugins

Required packages from the repositories of your distribution:

[Ubuntu]

sudo apt-get install build-essential git yasm libass-dev python3-pip python3-dev cython3 autoconf libtool libmagick++-dev qt5-default libfftw3-dev

[Fedora]

sudo dnf install automake gcc gcc-c++ yasm libass-devel python3-devel libtool ImageMagick-c++-devel qt-devel fftw-devel redhat-rpm-config qt5-qtbase-devel
sudo pip3 install cython

Compilation of the remaining dependencies:

L-SMASH:

1
2
3
4
5
6
cd ~/apps
git clone https://github.com/l-smash/l-smash.git
cd l-smash
./configure --enable-shared
make lib
sudo make install-lib

x264:

1
2
3
4
5
6
cd ~/apps
git clone https://code.videolan.org/videolan/x264/
cd x264
./configure --enable-shared
make
sudo make install

ffmpeg:

1
2
3
4
5
6
cd ~/apps
git clone https://github.com/ffmpeg/ffmpeg.git 
cd ffmpeg
./configure --enable-gpl --enable-libx264 --enable-avresample --enable-shared --enable-libfdk-aac --enable-nonfree
make
sudo make install

zimg:

1
2
3
4
5
6
7
8
cd ~/apps
git clone https://github.com/sekrit-twc/zimg.git
cd zimg
git submodule update --init --recursive
./autogen.sh
./configure
make
sudo make install

1.2.3 VapourSynth

Compilation of VapourSynth:

1
2
3
4
5
6
7
cd ~/apps
git clone https://github.com/vapoursynth/vapoursynth.git
cd vapoursynth
./autogen.sh
./configure
make
sudo make install

Now try it out:

vspipe --version

Possible errors: Hide

  • "vspipe: error while loading shared libraries: libvapoursynth-script.so.0: cannot open shared object file: No such file or directory”.

This is caused by the non-standard location of libvapoursynth-script.so.0. Your dynamic loader is not configured to look in /usr/local/lib. One way to work around this error is to set the LD_LIBRARY_PATH environment variable. Open ~/.bashrc and add:

export LD_LIBRARY_PATH=/usr/local/lib

Then reload ~/.bashrc with:

. ~/.bashrc

  • “Failed to initialize VapourSynth environment”.

This is caused by the non-standard location of the Python module, vapoursynth.so. Your Python is not configured to look in /usr/local/lib/python3.x/site-packages. One way to work around this error is to set the PYTHONPATH environment variable. Open ~/.bashrc and add (replace “x” with the correct number, you probably have python 3.6):

export PYTHONPATH=/usr/local/lib/python3.x/site-packages

Then reload ~/.bashrc with:

. ~/.bashrc

1.2.4 VapourSynth Editor

Compilation of VapourSynth Editor:

1
2
3
4
5
6
cd ~/apps
git clone https://bitbucket.org/mystery_keeper/vapoursynth-editor.git
cd vapoursynth-editor/pro
qmake pro.pro #NOTE: Fedora uses qmake-qt5.
make
sudo ln -s ~/apps/vapoursynth-editor/build/release-64bit-gcc/vsedit /usr/bin/vsedit

Add VapourSynth-Editor to application menu (Optional, for gnome):

sudo ln -s $HOME/apps/VapourSynth-Editor/vsedit.ico /usr/share/icons/vsedit.ico
sudo gedit ~/.local/share/applications/vsedit.desktop

Enter the following, save, then close gedit:

1
2
3
4
5
6
7
8
[Desktop Entry]
Type=Application
Name=VapourSynth-Editor
Comment=
Icon=/usr/share/icons/vsedit.ico
Exec=/usr/bin/vsedit
Terminal=false
Categories=VapourSynth;

[1.3 Installation of plugins]

Some plugin will be available in the repositories of your distribution. Others will have to be compiled. Github repositories usually include the installation instruction. Let's take ffms2, that you will need, as example.

Compilation: Hide

  • Download the source: cd ~/apps && git clone https://github.com/FFMS/ffms2.git
  • Browse to the source directory: cd ffms2
  • Next you need to find what files were included to assist with compiling the source: ls
  • You will be looking for one or more of the following files: autogen.sh, configure, CMakeLists.txt, or waf.
  • ffms2 has autogen.sh so you would compile and install by doing the following:
1
2
3
4
./autogen.sh
./configure
make
sudo make install
  • Pay attention to where the install places the file, most plugins will install to /usr/local/lib instead of the VS plugins directory. If this is the case you will need to create a symbolic link in the VS plugins folder by doing the following:

[Ubuntu] sudo ln -s /usr/local/lib/ffms2.so /usr/local/lib/vapoursynth/libffms2.so

[Fedora] sudo ln -s /usr/lib64/libffms2.so /usr/lib64/vapoursynth/libffms2.so

Follow the same procedure for each plugin you want to install.

[1.4 Scripts]

Most Avisynth functions have been ported by a couple of developers/contributers. Their scripts usually rely on each other’s. Here are the scripts you can have a look at and their respective functions:

havsfunc : Hide

Main functions:

daa
santiag
FixChromaBleedingMod
Deblock_QED
DeHalo_alpha
YAHR
HQDering mod
QTGMC
srestore
ivtc_txt60mc
logoNR
Vinverse
Vinverse2
LUTDeCrawl
LUTDeRainbow
Stab
GrainStabilizeMC
MCTemporalDenoise
SMDegrain
STPresso
SigmoidInverse, SigmoidDirect
GrainFactory3
InterFrame
SmoothLevels
FastLineDarken 1.4x MT MOD
Toon
LSFmod
TemporalDegrain
aaf

Utility functions:

AverageFrames
Bob
ChangeFPS
Clamp
KNLMeansCL
Overlay
Padding
Resize
SCDetect
Weave
ContraSharpening
MinBlur
sbr
DitherLumaRebuild
mt_expand_multi
mt_inpand_multi
mt_inflate_multi
mt_deflate_multi

muvsfunc : Hide

LDMerge
Compare (2)
ExInpand
InDeflate
MultiRemoveGrain
GradFun3
AnimeMask (2)
PolygonExInpand
Luma
ediaa
nnedi3aa
maa
SharpAAMcmod
TEdge
Sort
Soothe_mod
TemporalSoften
FixTelecinedFades
TCannyHelper
MergeChroma
firniture
BoxFilter
SmoothGrad
DeFilter
scale (using the old expression in havsfunc)
ColorBarsHD
SeeSaw
abcxyz
Sharpen
Blur
BlindDeHalo3
dfttestMC
TurnLeft
TurnRight
BalanceBorders
DisplayHistogram
GuidedFilter (Color)
GMSD
SSIM
SSIM_downsample
LocalStatisticsMatching
LocalStatistics
TextSub16
TMinBlur
mdering
BMAFilter

mvsfunc : Hide

Main functions:

Depth
ToRGB
ToYUV
BM3D
VFRSplice
Runtime functions:
PlaneStatistics
PlaneCompare
ShowAverage
FilterIf
FilterCombed

Utility functions:

Min
Max
Avg
MinFilter
MaxFilter
LimitFilter
PointPower
CheckMatrix
postfix2infix

Frame property functions:

SetColorSpace
AssumeFrame
AssumeTFF
AssumeBFF
AssumeField
AssumeCombed

Helper functions:

CheckVersion
GetMatrix
zDepth
PlaneAverage
GetPlane
GrayScale
Preview

fvsfunc : Hide

GradFun3mod
DescaleM (DebilinearM, DebicubicM etc.)
Downscale444
JIVTC
OverlayInter
AutoDeblock
ReplaceFrames (ReplaceFramesSimple)
maa
TemporalDegrain
DescaleAA
InsertSign

kagefunc : Hide

inverse_scale
inverse_scale_clip_array
clip_to_plane_array
plane_array_to_clip
generate_mask
apply_mask
apply_mask_to_area
getY
generate_keyframes
adaptive_grain
conditional_resize
retinex_edgemask
kirsch
fast_sobel
get_descale_filter
hardsubmask
hardsubmask_fades
crossfade
hybriddenoise
insert_clip
get_subsampling
iterate
is16bit
getw
shiftl
fit_subsampling
mask_detail

vshelpers of 4re : Hide

clamp
m4
build_mask
mt_lut
mt_lutxy
logic
get_luma
merge_chroma
rsoften
fit
move
get_decoder
subtract
starmask

• My own avisynth's ports and other utility functions [sgvsfunc]

Filtering functions:

FixRowBrightness
FixColumnBrightness
FixRowBrightnessProtect2
FixColumnBrightnessProtect2
FixBrightnessProtect2
bbmod
BlackBorders
CropResize
CropResizeReader
DebandReader
LumaMaskMerge
RGBMaskMerge

Utility functions:

SelectRangeEvery
FrameInfo
DelFrameProp
InterleaveDir
ExtractFramesReader
ReplaceFrames
Overlay
mt_lut
GetPlane
scale

To install a python script for VS, just download the .py file from github and copy it into your python packages directory:

[Ubuntu] /usr/local/lib/python3.6/dist-packages
[Fedora & Arch] /usr/lib64/python3.6/site-packages

You will also need to install Adjust which is a requirement for havsfunc.

[1.5 Initialisation of a script]

1.5.1 Basics

Open VSE. The first step when creating a script is to initialise VS. Write the following always in the first line:

import vapoursynth as vs
from vapoursynth import core

The second line can alternatively be core = vs.get_core()

And the following always in the last line of your script:

last.set_output()

where “last” should contain the clip you want to encode.

1.5.2 Loading plugins

Plugins will be automatically loaded once installed. You can rely on “core” to call them: variable = core.pluginname.function(clip)
For example: src = core.ffms2.Source('/path/to/source.mkv')

1.5.3 Loading scripts

Scripts are loaded manually. Import them at the beginning of your script. You may give them names. For example:

1
2
3
4
5
6
import fvsfunc as fvf
import mvsfunc as mvf
import kagefunc as kgf
import havsfunc as haf
import muvsfunc as muvf
import sgvsfunc as sgf

You can finally call them in your script: variable = scriptname.function(clip)
For example to call QTGMC: out = haf.QTGMC(src)


[2. Demuxing]

[2.1 Ripping with MakeMKV]

This section is dedicated to the ripping of BluRay discs, which consists of copying the disc to a hard drive while removing the encryption. If you are not planning to use a physical Blu-ray as a source, you can skip this part.

MakeMKV will be your best bet for decrypting discs on Linux. The instructions to compile it can be found here.

In order to create a backup, select the desired drive on main screen in "Source" box and press "backup" icon on a toolbar or select "Backup" from file menu. Be sure to select "decrypt video files" on the next dialog. You can alternatively backup a BluRay in command line with:

makemkvcon backup disc:/dev/sr0 /path/to/output/folder/ --decrypt

[2.2 Retrieving BluRay specifications]

BDInfo provides information about the different streams of the BluRay. I will here detail how to use this cli version of BDInfo.

Installation: Hide

Install mono if it's not already done and download the prebuilt version:

cd ~/apps
wget https://github.com/zoffline/BDInfoCLI-ng/blob/UHD_Support_CLI/prebuilt/BDInfoCLI-ng_0.7.3.zip?raw=true -O tmp.zip && unzip -d BDInfoCLI-ng -j tmp.zip && rm tmp.zip

Add an alias to use it easily:

echo "alias bdinfocli='mono ~/apps/BDInfoCLI-ng/BDInfo.exe'" >> ~/.bashrc && . ~/.bashrc

If your alias is bdinfocli, scan a disc with (report will be written in disc folder):

bdinfocli -w BD_PATH

[2.3 Demuxing]

Let's use ffmpeg to demux the disc. First identify the playlist and streams of your BluRay with:

ffmpeg -i bluray:"PATH/TO/BLURAY"

You should get such an output:

[bluray @ 0x55fcc39f7880] 2 usable playlists:
[bluray @ 0x55fcc39f7880] playlist 00000.mpls (1:20:51)
[bluray @ 0x55fcc39f7880] playlist 00005.mpls (0:25:58)
[bluray @ 0x55fcc39f7880] selected 00000.mpls

Input #0, mpegts, from 'bluray:LA POINTE COURTE':
Duration: 01:20:51.26, start: 600.000000, bitrate: 33023 kb/s
Program 1
Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
Stream #0:1[0x1100]: Audio: pcm_bluray (HDMV / 0x564D4448), 48000 Hz, stereo, s16, 1536 kb/s
Stream #0:2[0x1200]: Subtitle: hdmv_pgs_subtitle ([144][0][0][0] / 0x0090)

You can change the playlist with:

ffmpeg -playlist 5 -i bluray:"PATH/TO/BLURAY"

Additionally, if some subtitle tracks are missing, add -probesize 4G -analyzeduration 3600M in the command line, before -i. What's probesize and analyzereduction?: Hide

Without those options, subtitles may not appear because ffmpeg is just making a quick analysis. It is thus necessary to tell ffmpeg to scan deeper into the file so that it can find the them. The first is the number of bytes to read ahead into the file, and the second is the number of microseconds (of which there are 1,000,000 = 1M per second). These directives must come before the input file directive. In the example above, it will scan at least 4GB and 1 hour.

You can hide ffmpeg header with -hide_banner.

To extract specific streams, use the -map option. For instance, to extract the track identified as Stream #0:2 by ffmpeg, write -map 0:2 -c copy OUTPUT. Demuxing several streams from the example above will be done with:

ffmpeg -playlist 0 -i bluray:"PATH/TO/BLURAY" \
-map 0:0 -c copy video.h264 \
-map 0:1 -c flac audio.flac \
-map 0:2 -c copy subs.sup

Demuxing a DVD?: Hide

First identify the vob files storing the video you want to extract. They are usually the consecutive files with a 1.1Go size. For example if those are VTS_02_1.VOB, VTS_02_2.VOB and VTS_02_3.VOB, you will identify the whole video with:

ffmpeg -i concat:"VTS_02_1.VOB|VTS_02_2.VOB|VTS_02_3.VOB"

This way you can extract video, audio and subitle tracks.

Welp, I wanna use eac3to!: Hide

Alright! Let's use eac3to, which works fine with wine.

Installation: Hide

Get eac3to and place it in a folder like /home/user/apps. We will now setup handy aliases: Add these lines to .bashrc or .bash_aliases in your /home/user/ folder:

alias eac3to='wine ~/apps/eac3to/eac3to.exe 2>/dev/null'
alias eacout='wine ~/apps/eac3to/eac3to.exe 2>/dev/null | tr -cd "\11\12\15\40-\176"'

First, change the current directory to where you want your demuxed files in. To get a summary of the playlists on your BluRay:

eac3to "Z:/path/to/BluRay/disc/"

If you want to demux all streams of playlist 1 and show the process progression:

eac3to "Z:/path/to/BluRay/disc/" "1)" -progressnumbers -demux

You can alternatively get streams information with:

eac3to "Z:/path/to/BluRay/disc/" "1)"

and demux the identified streams with the following syntax:

eac3to "Z:/path/to/BluRay/disc/" "1)" 1:chapters.txt 2:video.* 4:main.thd \
4:main.ac3 -core 6:com.* 7:fr-forced.* 8:fr-sdh.* 9:eng.* 10:spa.* -progressnumbers

It will name them as stated and choose the correct file extension when ".*" is indicated. In this example, the TrueHD audio track is demuxed and a compatibility ac3 track is created by extracting its core.

[2.4 Extracting a remux]

You can directly encode from a remux, but you may need to extract the audio and subtitle tracks of the remux to encode and OCR them later on. Extracting tracks from a mkv file can be done with mkvtoolnix that you will find in your distribution repositories.

Identify the stream indexes:

mkvmerge remux.mkv --identify

Extract the streams you need:

mkvextract remux.mkv tracks 1:main-audio.dtshd 2:sub1.sup 3:sub2.sup

You can also extract the chapters with:

mkvextract remux.mkv chapters chapters.xml


[3. Encoding]

If you are releasing a remux, then jump to chapter 5. For an encode, if you demuxed a BluRay with eac3to, remux at least the video track with mkvtoolnix to encode from a mkv file.

[3.1 Vapoursynth script #1 : Filtering the source]

Let's create the first script which will be used for the test encodes. Open VSE and save a script containing:

import vapoursynth as vs
import sgvsfunc as sgf
core = vs.get_core()

#1 Import
src = core.ffms2.Source('/path/to/remux.mkv')

#2 Crop
crop = core.std.Crop(clip=src, left=0, right=0, top=0, bottom=0)

#3 Resize
h = round((1280*crop.height/crop.width) / 2) * 2
resized = core.resize.Spline36(clip=crop, width=1280, height=h)

#4 Select extract
extract = sgf.SelectRangeEvery(clip=resized, every=3000, length=50, offset=10000)

extract.set_output()

As you can see, we added 4 operations between the usual initialisation lines:

#1 imports the remux

The video will be indexed by ffms2 (recommanded for mkv files) and named "src".

If however you want to index a m2ts file, it is recommanded to import it with LWLibavSource this way:

src = core.lsmas.LWLibavSource('/path/to/source.m2ts')

For Mpeg1/2 files, import a d2v file. To do that, install D2VWitch and the d2vsource plugin. In D2VWitch add the Mpeg files. You can uncheck the demuxing of audio tracks. Click Engage and a d2v file is generated. In the VS script, replace the corresponding line by:

src = core.d2v.Source('/path/to/source.d2v')

#2 crops the video

Obviously it doesn't crop anything yet, so click on Script > Preview and on the "Crop assisstant" to crop the black lines. Report the values in the script.
You can alternatively use

CropAbs(clip, width, height, left, top)

It is possible to omit the input variable names "clip=", "left="... but in this case pay attention to use the correct order.

If you have an odd number to crop, please read part [4.1 Fixing borders]

#3 resizes the video

The width is calculated for a 720p resize by doing a simple cross-product and making it mod 2 for AR > 16:9 (1280x...). For AR < 16:9 (...x720), you should replace these lines by:

w = round((720*crop.width/crop.height) / 2) * 2
resized = core.resize.Spline36(clip=crop, width=w, height=720)

You should delete this part and replace "resize" by "crop" for a 1080p release.

#4 makes an extract

We are using SelectRangeEvery to trim the first and last 10,000 frames to omit intro and credits, and select 50 frames every 3000 frames. You can of course change these numbers.

Something faster?: Hide

You can alternatively replace steps 2 & 3 by using CropResize which can calculate the resolution by itself. You just have to indicate either the width or the height. For example:

crop = sgf.CropResize(clip=src, width=1280, top=138, bottom=138)

Avoid the "Unrecognized transfer characteristics" or "unrecognized color primaries" error with ffms2: Hide

This is only related to VSE and shouldn't appear when encoding. Just add

src = sgf.DelFrameProp(clip=src)

after importing the source
However if you notice a shift in the colors, indicate also the matrix with:

src = core.resize.Spline36(clip=src, width=src.width, height=src.height, matrix_in_s='709')

or just add the matrix_in_s argument in your resize line.

[3.2 Test encoding]

3.2.1 Basic utilization of vspipe

To encode from our script, we are going to evaluate it and ouput the frames to x264 using VSPipe. Here is an example of command line:

vspipe --y4m script.vpy - | x264 --demuxer y4m - --output encoded.mkv

You can add all the x264 settings you want to use in this command.

3.2.2 Using the testencodes bash script

To make test encoding easier, I wrote a bash script which enables to make mutiple encodes in a row changing a given x264 setting. You can find it in part [6.4 testencodes - A bash script for x264 test-encoding with VapourSynth (Linux)]. Follow the installation instructions. Provide the path of your VS script #1.

As an example, to make multiple encodes changing qcomp from 0.6 to 0.8 by steps of 0.02, with a bitrate of 7000 kbps and the default settings indicated in testencodes.sh, run the following command:

testencodes -t qcomp 0.6_0.8 0.02 --bitrate 7000

Once you have encoded a first set of test encodes for a given setting, you will need to compare them to choose which value lead to the best quality. For the comparison, please read part [3.3 Vapoursynth script #2: Comparing the test encodes]. Then come back to testencodes, specify the chosen value and test the next setting.

As an example, if you chose qcomp=0.68, testing aq-strength from 0.5 to 1.2 by 0.1 will be done with:

testencodes -t aq-strength 0.5_1.2 0.1 --qcomp 0.68 --bitrate 7000

Go back and forth until you have determined all the desired settings. The test method varies from an encoder to another, but the settings usually tested are crf, qcomp, aq-mode, aq-strength, psy-rd and sometimes ipratio and pbratio.

Usage example: Hide

Determining the bitrate to work with (for which there is a small loss of quality which will be compensated with optimized settings):

testencodes -t crf 15_20 1

Testing qcomp from 0.6 to 0.8 by 0.02 with a working bitrate of 7000kbps:

testencodes -t qcomp 0.6_0.8 0.02 --bitrate 7000

Testing aq-mode:

testencodes -t aq-mode --qcomp 0.68 --bitrate 7000

Testing aq-strength from 0.5 to 1.2 by 0.1:

testencodes -t aq-strength 0.5_1.2 0.1 --aq-mode 2 --qcomp 0.68 --bitrate 7000

Testing psy-rdo from 0.85 to 1.15 by 0.05:

testencodes -t psy-rd 0.85_1.15:0 0.05 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68 --bitrate 7000

Testing psy-trellis from 0 to 0.15 by 0.05:

testencodes -t psy-rd 1.05:0_0.15 0.05 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68 --bitrate 7000

Testing final crf from 15 to 20 by 1 after deleting previous crf tests:

testencodes -rm -t crf 15_20 1 --psy-rd 1.05:0.10 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68

Refining final crf by 0.1:

testencodes -t crf 17_18 0.1 --psy-rd 1.05:0.10 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68

[3.3 Vapoursynth script #2: Comparing the test encodes]

Let's imagine you just tested qcomp with testencodes. All the resulting tests are located in tests/qcomp/. This second VS script will automatically load and interleave them to enable you to make the comparison frame vs frame:

import vapoursynth as vs
import sgvsfunc as sgf
core = vs.get_core()

#1 Source script
src = core.ffms2.Source('/path/to/remux.mkv')
crop = core.std.Crop(clip=src, left=0, right=0, top=0, bottom=0)
h = round((1280*crop.height/crop.width) / 2) * 2
resized = core.resize.Spline36(clip=crop, width=1280, height=h)
extract = sgf.SelectRangeEvery(clip=resized, every=3000, length=50, offset=10000)

#2 Adding the frame properties
extract = sgf.FrameInfo(clip=extract, title='Source')

#3 Loading test encodes
folder = '/home/user/encoding/tests/qcomp'
comparison = sgf.InterleaveDir(folder, PrintInfo=True, first=extract, repeat=True)

comparison.set_output()

#1 should be the processing of your source copied from the VS script #1

#2 will print the frame number and picture type on the source and add the title "Source".

#3 will scan the chosen test folder for all mkv files, print the frame properties and the setting tested and interleave them with the source. repeat=True will display the source frame between all test frames (Source | test1 | Source | test2 | Source | test3...). repeat=False will display the source frame only once (Source | test1 | test2 | test3...). This is up to the encoder.

You then just have to preview, and you can compare P-B or B-B frames.

[3.4 Encoding the whole video]

Once you have all your settings, you can encode the entire video. To do so, come back to your script #1 and delete the part that is extracting a limited amount of frames. Use testencodes in output mode.

For example, to encode the final release with crf set to 17.3 and the optimized settings from the usage example:

testencodes -o /home/user/encoding/output.mkv --crf 17.3 --psy-rd 1.05:0.10 \
--aq-strength 0.8 --aq-mode 2 --qcomp 0.68

If your are encoding a DVD, you should do an anamorphic encode by specifying the correct SAR with --sar.

What's SAR?: Hide

The video, regardless of its width/height ratio, is stored into standard resolutions. For example, PAL DVD stores it into 720x576. So if you want your video to be displayed in correct proportions, you have to resize it. SAR is here to indicate "the width should be stretch that much to have the correct ratio". You determine it using this formula:

DAR = FAR × SAR

where:

  • DAR is the Display Aspect Ratio = (width of displayed video) / (height of displayed video), also given by MediaInfo
  • FAR (or PAR) is the Frame (Picture) Aspect Ratio = (width of video as it is stored) / (height of video as it is stored)
  • SAR is the Sample Aspect Ratio, the width stretch to apply to a pixel.

For example, let's imagine you have a NTSC DVD with a DAR of 16/9, this formula becomes:

16/9 = (720/480) × SAR
→ SAR = 32/27

You may be confused by what you can find on internet about SAR. This is because this standard is for MPEG4/AVC/x264 streams, and for MPEG2 names were switched: SAR was called PAR (for Pixel Aspect Ratio) and FAR/PAR was called SAR (for Storage Aspect Ratio)...


[4. Filtering]

In this section, we are going further into encoding by developping the main filters which can be applied to fix some artefacts present on a BluRay. Filtering has usually to be applied after cropping and before resizing (except debanding done after resizing).

[4.1 Fixing borders]

4.1.1 Black lines

Filling black lines

Sometimes you would like to crop an odd number of lines but you can't. Or the crop would vary along the movie. You thus will have to fill those lines (with a copy of the adjacent one) using FillMargins. See this example of filling 1 pixel on right and top: Source, Filtered: [Show comparison]

Install the fillborbers plugin and use the following code:

filtered = core.fb.FillBorders(crop, left, right, top, bottom, mode="fillmargins")

Resizing

If you are resizing to 720p and the black line is present on the whole movie, omit it when resizing with the following syntax. You still need to fill the black line first to avoid a dirty border.

fill = core.fb.FillBorders(crop, left, right, top, bottom, mode="fillmargins")
resized = core.resize.Spline36(fill, width, height, src_left, src_top, src_width, src_height)

where src_left/top is the number of omitted lines at left/top and src_width/height is the width/height of the non-omitted area.

For example, if you have a 1920x1080 video with 1 line at top and bottom to omit:

fill = core.fb.FillBorders(crop, left=0, right=0, top=1, bottom=1, mode="fillmargins")
h = round((1280*1078/1920) / 2) * 2
resized = core.resize.Spline36(fill, width=1280, height=h, src_left=0, src_top=1, src_width=1920, src_height=1078)

You can also use CropResize from sgvsfunc which will do this automatically.:

crop = sgf.CropResize(clip=src, width=1280, top=1, bottom=1)

4.1.2 Dirty lines

If some lines at the border are "dirty", they can usually be fixed. See this example (fixing 1 pixel on top and bottom): Source, Filtered: [Show comparison]

First you should check if the lines contain valuable information: Are they coherent with their neighboring lines? Are they not just poor copies? Zoom in to find out. If they are garbage data, just crop them.

Uniform brightness fix

If the dirty lines are uniformly too dark or bright, the proper way to fix them is to use FixBrightnessProtect2. In row and column, indicate the indexes of the row/column to fix, 1 being the first starting from the left/top. Negative numbers are relative to right/bottom. In adj_row/adj_column, specify the fix value (<100), positive to brighten, negative to darken:

filtered = sgf.FixBrightnessProtect2(clip, row, adj_row, column, adj_column)

For example:

filtered = sgf.FixBrightnessProtect2(clip=src, row=[1,2], adj_row=[18,-2], column=[3,-3], adj_column=[5,5])

will:

  • brighten the 1st row by +18
  • darken the 2nd row by -2
  • brighten the 3rd column from the left and the 3rd column from the right by +5

Uniform color fix

If the lines have a dirty color, you can use ContinuityFixer or bbmod.

fix = core.edgefixer.ContinuityFixer(src, left, top, right, bottom, radius)
filtered = sgf.bbmod(c, cTop, cBottom, cLeft, cRight, thresh, blur)

Here left/cLeft... are the number of pixels on which the fix is performed. With their default values of radius (=height of video) and blur (=999), they apply a uniform fix, based on the whole row/column near the dirty line. This is the less destructive scenario.

Note that ContinuityFixer will allow you to fix separately luma and chroma planes. So if the brightness is fine, you can only fix the colors. Lists of 3 values should be given to left/top/right/bottom, that apply to [y, u, v], where y is the luma plane (brightness) and u and v the chroma planes (color). For example:

fix = core.edgefixer.ContinuityFixer(src, left=[0,3,3], top=[0,0,0], right=[0,1,1], bottom=[0,0,0])

will repair the colors of the first 3 columns and the last column, while keeping their brightness untouched. bbmod is unable to separate luma and chroma fix, this is why ContinuityFixer is prefered.

Local matching

If the brightness or color can't be fixed with a uniform filter, you should use ContinuityFixer or bbmod while decreasing the radius/blur parameter. In practice, to determine the new brightness/color of a given pixel, the filters will take an average of the neighboring pixels located at a distance < radius/blur. This is why with their default high values, they perform a uniform fix. On the contrary, a radius/blur of 1 will lead to a copy of the nearest line (~FillMargins), in that case, all data is lost. Decreasing to 5-20 usually enables to have a good compromise between local brightness/color matching and the conservation of some data to have a different line. In any case, don't use ContinuityFixer/bbmod on more than 3 pixels, since it will deform the details near the border, especially with lower radius/blur.

You can also give BalanceBorders a try in the most damaged lines, but be careful with it as it is a very agressive filter. It is available in the muvsfunc script:

fix = muvs.BalanceBorders(c, cTop, cBottom, cLeft, cRight, thresh, blur)

It works similarly to bbmod, decrease blur if you need more local matching.

[4.2 Debanding]

Banding is discretization of gradients. See this example: Source, Filtered: [Show comparison]

4.2.1 f3kdb filter

The debanding filter usually used is f3kdb. Install the f3kdb plugin for VS and use it with:

db = core.f3kdb.Deband(resized,y=60,cb=60,cr=60,grainy=48,grainc=48,dynamic_grain=True,range=16)

  • y, cb and cr are the strengths of the filter on the different planes, which should be the lowest value leading to the disapearence of the banding.
  • grainy and grainc are factors responsible for the introduction of grain. It should be high enough to keep the scene unbanded, but low enough not to deform the detail of the image.
  • you can play with the range for better results

You should apply it after resizing. Preview the movie to check what it does on the banded scenes and you can even temporarily make the source-filtered comparison with:

import vapoursynth as vs
import sgvsfunc as sgf

#1 Source script
src = core.ffms2.Source('/path/to/remux.mkv')
crop = core.std.Crop(clip=src, left=0, right=0, top=0, bottom=0)
h = round((1280*crop.height/crop.width) / 2) * 2
resized = core.resize.Spline36(clip=crop, width=1280, height=h)

#2 Filtering
db = core.f3kdb.Deband(resized,y=60,cb=60,cr=60,grainy=48,grainc=48,dynamic_grain=True,range=16)

#3 Temporary comparison
comparison = core.std.Interleave([resized,db])

comparison.set_output()

to optimize the parameters and ensure that the filter is not destructive.

4.2.2 Applying the filter on given ranges

Debanding can be a destructive filter, this is why it should be applied on the smallest amount of frames. The next step is to check which frames it has to be applied to. Preview the source in VSE. Now you will have to go through the entire movie and write down the ranges (which should usually be < 1000 frames) where you see banding. Look for areas with smooth color gradation (shadows, sky...). Yes, this can be a painstaking job.

Once you have noted the ranges, you can use ReplaceFrames to only apply debanding on them. As an example:

filtered = sgf.ReplaceFrames(resized, db, mappings="[514 605][8959 9125][103670 104077]")

In "resized", it will replace the given ranges by the ones of "db", so that everything else is untouched.

You can define several debanding filters and apply them to their corresponding ranges if different strengths have to be applied from some ranges to others.

Load ranges and strengths from a csv file: Hide

If you have a high number of ranges, or various strengths to apply, it may be more efficient to create a csv file with the defined zones in the following format for each zone (row):

<startframe> <endframe> <strength>

and use my DebandReader function:

filtered = sgf.DebandReader(resized, csvfile=/path/to/file.csv, grain=64, range=30)

4.2.3 Applying the filter on given areas

You can additionally apply f3kdb to given areas in the picture by defining a mask based on:

  • brightness, to only deband dark or bright areas, what you can do with LumaMaskMerge

filtered = sgf.LumaMaskMerge(resized, db, threshold=128, invert=False)
Choose the right brightness level "threshold". Debanding will be applied where the brightness is below threshold, above if invert=True

  • RGB colors, to only deband a given range of colors, what you can do with RGBMaskMerge

filtered = sgf.RGBMaskMerge(resized, db, Rmin, Rmax, Gmin, Gmax, Bmin, Bmax)

Determine the extreme R G B values at boundaries of the area you want to deband to fill in Rmin, Rmax, Gmin, Gmax, Bmin, Bmax. Debanding will be applied where Rmin < R < Rmax and Gmin < G < Gmax and Bmin < B < Bmax.

4.2.4 x264 zoning

As it is often difficult to keep debanded scenes unbanded without introducing too much grain, the encode is usually zoned to lower crf in the debanded parts.

Selecting the ranges and testing crf

First you will have to determine the crf on the debanded scenes. You can select them using python code as the following example:

selec = filtered[514:606] + filtered[8959:9126] + filtered[103670:104078]

which is equivalent to:

selec = core.std.Trim(filtered, first=514, last=605) + core.std.Trim(filtered, first=8959, last=9125) + core.std.Trim(filtered, first=103670, last=104077)

As you can see, slicing operations in Python do not include the last frame.

If you wrote down the ranges in a csv file, you can directly extract them with ExtractFramesReader:

selec = sgf.ExtractFramesReader(filtered, csvfile=/path/to/file.csv)

Then perform a crf test encoding on those ranges.

Zoning the encode

You can finally zone the corresponding ranges in the encode by adding to the x264 or testencodes command:

--zones <startframe1>,<endframe1>,crf=<crf1>/<startframe2>,<endframe2>,crf=<crf2>...

With testencodes, you can load the zones from a csv file containing

<startframe> <endframe> <crf>

by adding

--zones f:/path/to/file.csv

[4.3 Fixing gamma bug]

On some BluRay discs, the brightness level is not correct. See this example: Source, Filtered: [Show comparison]. You can particularly notice it when comparing different BluRay editions of the movie.

Applying a 0.88 gamma shift will usually fix the brightness. It can be done with std.Levels, it is recommended to do this in at least 16 bits of precision.

filtered = core.std.Levels(clip, gamma=0.88, min_in=16, max_in=235, min_out=16, max_out=235, planes=0)

For 16 bit:

filtered = core.std.Levels(clip, gamma=0.88, min_in=16 << 8, max_in=235 << 8, min_out=16 << 8, max_out=235 << 8, planes=0)

[4.4 Anti-aliasing]

For a very slight aliasing, I had good results with this port of nnedi3_rpow2

import edi_rpow2 as edi
aa = edi.nnedi3_rpow2(clip=src,rfactor=2,nns=1,correct_shift="Spline36")

You can also look at vsTAAmbk

For heavier aliasing, daa is recommended:

aa = haf.daa(src)

Anti-aliasing can also be a destructive filter, so be sure to choose the appropriate one by comparing source/filtered like it is done for debanding.

[4.5 Deinterlacing]

If your source is fully interlaced, you can use QTGMC from havsfunc:

filtered = haf.QTGMC(src,TFF=True, FPSDivisor=2, Preset='Slow', TR2=1, SourceMatch=1)

[4.6 Detelecine]

If your source underwent a 2:3 pulldown, VFM will deinterlace the correct frames, and VDecimate will remove the duplicates. What's a 2:3 pulldown?: Hide

In the United States and other countries where television uses the 59.94 Hz vertical scanning frequency, video is broadcast at 29.97 frame/s. To that extend, the movie is converted from 24 to 29.97 frame/s, by interlacing the fields according to this scheme:

https://ptpimg.me/exzp73.png

A 2:3 pulldown can thus be noticed by 2 combed frames over 5.

dtlc = core.vivtc.VFM(clip=src, order=0, mode=0)
dcmt = core.vivtc.VDecimate(dtlc)

Please note that:

  • mode 0 is the safest of all the options in the sense that it won’t risk creating jerkiness due to duplicate frames when possible, but if there are bad edits or blended fields it will end up outputting combed frames when a good match might actually exist. My suggestion is thus to start with mode 0. If you still notice combed frames when it's applied, increase it up to mode 5 but mind the artifacts it could create.
  • VDecimate will drop one frame in every cycle frames – the one that is most likely to be a duplicate. By default, cycle=5, wich is the case in a 2:3 pulldown. If you have a different pulldown, change the cycle argument.

[4.7 Blended fields]

DVD wizardry can go pretty far. If you notice a "ghosting" effect after deinterlacing, that's probably because the movie is field blended. In this case you might benefit from using srestore after QTGMC.

ditl = haf.QTGMC(src,TFF=True, Preset='Slow', TR2=1, Sharpness=0.5, SourceMatch=1)
dblnd = haf.srestore(ditl)

[5. Remuxing]

[5.1 Audio encoding]

Audio encoding will be achieved with ffmpeg here. For best results with aac, compile ffmpeg with libfdk_aac support as it is done in part [1.2 Installation of VapourSynth]. Keep in mind that you may get lesser compatibility than with a certified encoder.

The basic command line to encode audio tracks is:

ffmpeg -i INPUT -c:a CODEC -b:a BITRATE -ac NB_CHANNELS OUTPUT

Examples:

  • Extracting DTS core:

ffmpeg -i bluray:"PATH/TO/BLURAY" -map 0:1 -bsf:a dca_core -c:a copy OUTPUT.dts

  • Encoding AC3 640kbps:

ffmpeg -i bluray:"PATH/TO/BLURAY" -map 0:1 -c:a ac3 -b:a 640k -ac 6 OUTPUT.ac3

  • Encoding FLAC:

ffmpeg -i bluray:"PATH/TO/BLURAY" -map 0:1 -c:a flac OUTPUT.flac

  • Encoding AAC:

ffmpeg -i bluray:"PATH/TO/BLURAY" -map 0:1 -c:a libfdk_aac -vbr 5 OUTPUT.aac

Use -vbr 5 for main audio (best quality). Decrease it for commentary tracks (to have around 80kbps)

Using a certified encoder for AC3: Hide

If you are encoding DTS or Dolby type streams, you will need to get individual WAV files for each channel, using the .wavs extension with eac3to. Each wav will be labeled with the proper channel. As an example:

eac3to input.thd output.wavs

if you need to downmix to 5.1 channels, append -down6 to eac3to command.

In this example, we are using SurCode for Dolby Digital 5.1. Installation of SurCode for Dolby Digital 5.1: Hide

Download Minnetonka Audio SurCode for Dolby Digital 5.1

Unzip it. From the unzipped directory, run Setup.exe via wine:

wine Setup.exe

Proceed to the installation

Copy lservrc file from the crack folder to SurCode for Dolby Digital program directory (default "/home/user/.wine/drive_c/Program Files (x86)/Minnetonka Audio Software/SurCode for Dolby Digital"), overwriting original file.

Run Surcode with usual wine command

Go to Options > Encode Options and set as below:

https://ptpimg.me/50bmmg.png

The only real thing you need to do is choosing the destination file, load up the wavs generated by eac3to respective to their label. Make sure start is 0 and end is the length of the movie, and click on Encode.

Using a certified encoder for DTS-HD MA and DTS: Hide

Export the individual wav files per channel as explained in the previous part about certified AC3 encoding.

Installation of DTS-HD Master Audio Suite: Hide

You may need to install lib32-mpg123 and gnutls28 from the repositories of your distribution.

Create a 32bits prefix for wine:

WINEARCH=win32 WINEPREFIX=$HOME/.wine32 winecfg

Choose Windows XP as Windows version.

Download Java Runtime Environment 7

Install Java in the 32-bits prefix of wine:

WINEPREFIX=$HOME/.wine32 wine start /unix jre-7u80-windows-i586.exe

Download DTS-HD Master Audio Suite

Unzip it and install DTS-HD Master Audio Suite in the 32-bits prefix of wine, from the unzipped directory:

WINEPREFIX=$HOME/.wine32 wine start /unix setup.exe

Go to the DTS-HD Master Audio Suite installation directory and run the license checker to introduce the key:

cd $HOME/.wine32/drive_c/Program Files/DTS/MAS-SAS
WINEPREFIX=$HOME/.wine32 wine start /unix MAS-SAS_Authorizer.exe

Always run DTS-HD Master Audio Suite with the 32-bits prefix:

WINEPREFIX=$HOME/.wine32 wine start /unix dtshd.exe

Load up the wav tracks created by eac3to. Set the correct Stream Type and change channel count as necessary (as well as bitrate). Start by clicking on the ENCODE button.

[5.2 Subtitles]

5.2.1 OCR subtitles

To OCR subtitles, I don't think there's a better tool than SubExtractor, which runs with mono. You will find some guides on how to use SubExtractor around, the utilisation is pretty straightforward.

Installation: Hide

Just execute the following command in a terminal:

mono DvdSubExtractor.exe

from the SubExtractor folder to run the software.

You can create a shortcut if you want the software to appear in your application menu. If you are using Gnome, this is done by creating a SubExtractor.desktop file in ~/.local/share/applications/ wich contains:

1
2
3
4
5
6
7
[Desktop Entry]
Type=Application
Name=SubExtractor
Categories=AudioVideo;AudioVideoEditing;
Exec=mono /home/user/apps/SubExtractor/DvdSubExtractor.exe %f
StartupNotify=true
Terminal=false

if your SubExtractor folder is located in /home/user/apps/

As native OCR tools, there is VobSup2srt which uses tesseracts (install the necessary tesseract language packs, for example tesseract-data-eng for english). I don't recommand this software as it is skipping some subtitle lines and won't format italic characters. I want to mention it however as it can prove useful when other OCR software like Subextractor fail and it is very convenient (you don't need to specify any character and it does relatively few mistakes). If you use it check for the aforementioned issues. You will need to convert sup subtitles to sub/idx using BDSup2Sub (run the jar file with 'java -jar BDSup2Sub.jar'). To OCR with vobsub2srt:

vobsub2srt subtitles_name --tesseract-lang eng

--tesseract-lang eng is optional if the langage is specified in the file

5.2.2 Checking subtitles

Once the subtitles OCR'd, or if you downloaded the subtitles online, we need to check them for errors. This can be done with SubtitleEdit by running a spell check (Spell Check > Spell Check) and fix them for common errors (Tools > Fix common errors).

Installation: Hide

Download the portable version of SubtitleEdit. Just execute the following command in a terminal:

mono SubtitleEdit.exe

from the SubtitleEdit folder to run the software.

You can create a shortcut if you want the software to appear in your application menu. If you are using Gnome, this is done by creating a SubtitleEdit.desktop file in ~/.local/share/applications/ wich contains:

1
2
3
4
5
6
7
[Desktop Entry]
Type=Application
Name=SubtitleEdit
Categories=AudioVideo;AudioVideoEditing;
Exec=mono /home/user/apps/SubtitleEdit/SubtitleEdit.exe %f
StartupNotify=true
Terminal=false

if your SubtitleEdit folder is located in /home/user/apps/

[5.3 Chapters]

Either named or sequentially numbered chapters are really appreciated. You can extract and edit chapters' disc with chaptorEditor.

[5.4 Muxing]

5.4.1 GUI

MKVToolNix will enable you to mux all the streams you prepared together:

  • Drag and drop your video track into Source files. Then add your main audio track, followed by any commentaries. Finally, add all your subtitle tracks. The tracks, chapters, and tags section will show all the tracks to be muxed and in what order
  • For each of those tracks, there is a General options tab. For track name, you can put something descriptive, like "Commentary with director Steven Spielberg". There is also the language field, obviously select the correct language for audio and subtitle tracks. For default track flag, this really only needs to be set for the main audio, and possibly the english subtitles. Don't force any track.
  • In Timestamps and default duration, you can set a delay, for example if you are including a track from an other source with a relative delay. Stretch can fix progressive delays and it can be set as the fps difference, for example 25000/23976. However, don't stretch an audio file as it leads to a pitch shift, unless the stretch is small (23.976 -> 24.000 for instance), or for commentary tracks.
  • In the Output tab, for the file title, you can put the release name or the movie name + year. For chapters, select the chapter file you have. You can get away with not setting language/charset if they are the same as your system's default.

Set the destination file and start multiplexing. You are done, congrats!

5.4.2 CLI

This is primarily aimed at server users. The cli version works as follows:

mkvmerge -o OUTPUT.mkv --title TITLE --chapters CHAPTERS.xml \
--track-name 0:TRACK_TITLE --language 0:ISO VIDEO.h264 \
--track-name 0:TRACK_TITLE --language 0:ISO AUDIO.dtsma \
--language 0:ISO --default-track 0/1 SUBTITLE.sup

  • You should put things in quotes if there are spaces or other metacharacters involved.
  • OUTPUT.mkv is your output file name, e.g. Mystic.River.2003.BluRay.Remux.1080p.VC-1.DTS-HD.MA.5.1.mkv
  • TITLE is the general movie title or release name (Mystic River [2003] or Mystic.River.2003.BluRay.Remux.1080p.VC-1.DTS-HD.MA.5.1), you can omit this option if you want. Note that if you try to take a specific track from another matroska file and that file as the title set, you will inherit that title, so you might want to just set the title to the empty string.
  • VIDEO is the video file you want to include, same idea for AUDIO and SUBTITLE
  • ISO is the iso code for the language, e.g. eng for English. See here for more. You want the ISO 639-2 Code.
  • TRACK_TITLE is the title of the track, you can put something descriptive, like "Commentary with director Steven Spielberg". For remuxes, you will have to set this area to the relevant portion of the BDInfo, e.g. for the video track, I would have something like "VC-1 Video / 25534 kbps / 1080p / 23.976 fps / 16:9 / Advanced Profile 3". This is only for the video track and the main audio. Most tracks will probably have this field blank.
  • --default-track should be set to 0 or 1

An example:

mkvmerge -o "Mystic.River.2003.BluRay.Remux.1080p.VC-1.DTS-HD.MA.5.1.mkv" \
--title "Mystic.River.2003.BluRay.Remux.1080p.VC-1.DTS-HD.MA.5.1.mkv" \
--chapters chapters.xml \
--track-name 0:"VC-1 Video / 25534 kbps / 1080p / 23.976 fps / 16:9 / Advanced Profile 3" --language 0:eng video.vc1 \
--track-name 0:"DTS-HD Master Audio / English / 5.1 / 48 kHz / 3507 kbps / 24-bit" --language 0:eng audio.dtsma \
--language 0:eng --default-track 0 eng.sup


[6. Appendix]

[6.1 Generating screenshot comparisons]

Making a screenshot comparison for an encode is similar to the comparison we made when testing settings. Here we will compare the source, the filtered video and the encode in a VS script #3:

import vapoursynth as vs
import sgvsfunc as sgf

core = vs.get_core()

#1 Source without filtering
src = core.ffms2.Source('/path/to/remux.mkv')
crop = core.std.Crop(clip=src, left=0, right=0, top=138, bottom=138)
h = round((1280*crop.height/crop.width) / 2) * 2
resized_src = core.resize.Spline36(clip=crop, width=1280, height=h)

#2 Filtered video
fix = sgf.FixBrightnessProtect2(clip=crop, row=[1, 2, -1, -2], adj_row=[18, -2, -2, 18])
h = round((1280*crop.height/crop.width) / 2) * 2
resized_fix = core.resize.Spline36(clip=fix, width=1280, height=h)

#3 Encode
encode = core.ffms2.Source('/path/to/encode.mkv')

#4 Printing info
resized_src = sgf.FrameInfo(resized_src, 'Source')
resized_fix = sgf.FrameInfo(resized_fix, 'Filtered')
encode = sgf.FrameInfo(encode, 'Encode')

#5 Interleaving
comparison = core.std.Interleave([resized_src, resized_fix, encode])

comparison.set_output()

#1 should be the processing of your source without any filtering (debanding, correction of dirty lines...). It should only include the cropping and the resizing.

#2 should be the processing of your source copied from the VS script #1, with all filters used (debanding, correction of dirty lines...).

#3 loads your encode

#4 prints the frame info on source, filtered and encode.

#5 interleaves the 3 aforementioned clips to compare them frame vs frame

Preview this script in VSE and look for P-B or B-B comparisons. Right-click on an image and select Save snapshot to save the png image.

Encoding a DVD? Make sure to resize the screenshots to respect DAR: Hide

As explained in part [3.4 Encoding the whole video], DVD video is stored into a standard resolution and is resized during playback to have the right AR. VSE will display the frames in their storage resolution. This means that you have to resize them to get them to their display resolution. Here is the script including this resizing:

import vapoursynth as vs
import sgvsfunc as sgf

core = vs.get_core()

#1 Source without filtering
src = core.d2v.Source('/path/to/source.d2v')
crop = core.std.Crop(clip=src, left=0, right=0, top=138, bottom=138)
sar=32/27
anamorphic_src = core.resize.Spline36(clip=crop, width=sar*src.width, height=src.height)

#2 Filtered video
fix = sgf.FixBrightnessProtect2(clip=crop, row=[1, 2, -1, -2], adj_row=[18, -2, -2, 18])
anamorphic_fix = core.resize.Spline36(clip=fix, width=sar*fix.width, height=fix.height)

#3 Encode
encode = core.ffms2.Source('/path/to/encode.mkv')
anamorphic_encode = core.resize.Spline36(clip=encode, width=sar*encode.width, height=encode.height)

#4 Printing info
anamorphic_src = sgf.FrameInfo(anamorphic_src, 'Source')
anamorphic_fix = sgf.FrameInfo(anamorphic_fix, 'Filtered')
anamorphic_encode = sgf.FrameInfo(anamorphic_encode, 'Encode')

#5 Interleaving
comparison = core.std.Interleave([anamorphic_src, anamorphic_fix, anamorphic_encode])

comparison.set_output()

[6.2 sgvsfunc - Filtering and utility functions for VapourSynth and VapourSynth Editor]

DOWNLOAD

Installation: Hide

Download sgvsfunc.py and place it in the python packages directory:

[Fedora and Arch-based systems] /usr/lib64/python3.6/site-packages
[macOS] /usr/local/lib/python3.6/site-packages
[Ubuntu] /usr/local/lib/python3.6/dist-packages
[Windows] C:\Users\<user>\AppData\Local\Programs\Python\Python36\Lib\site-packages

Functions: Hide

Filtering functions: Hide

FixRowBrightness/FixColumnBrightness: Hide

> Usage: FixRowBrighnessProtect2(clip, row, input_low=16, input_high=255, output_low=16, output_high=255, scale_inputs)

  • row is the row you want to work on.
  • input_low, input_high, output_low, output_high are the arguments passed to levels. input_high is more or less the more interesting value to tweak, lowering it will brighten the line and raising it will dim the line.
  • scale_inputs = True scales input_low, input_high, output_low, output_high from 8bits to current bit depth.

FixRowBrightnessProtect2/FixColumnBrightnessProtect2: Hide

> Usage: FixRowBrighnessProtect2(clip, row, adj_val, prot_val, scale_inputs)
Thanks zorbit and nonshatter for the original FixBrightnessProtect2

  • prot_val can be left out and the default of 16 will be used.
  • row is the row you want to work on.
  • adj_val should be a number x where -100 < x > 100. This parameter decides how much the brightness should be affected. Numbers below 0 will make it darker and number above 0 will make it brighter.
  • prot_val is the protect value. This is what makes it behave differently than the normal FixBrightness. Any luma above (255-prot_val) will not be affected which is the basic idea of the protect script.
  • scale_inputs = True scales prot_val from 8bits to current bit depth.

FixBrightnessProtect2: Hide

> Usage: FixBrightnessProtect2(clip, row, adj_row, column, adj_column, prot_val)

  • row and column are lists [<rowA> <rowB>...] where negative numbers are relative to right/bottom. Indexes are different from FixRow/ColumnBrightnessProtect2: First row is defined by index 1 (and last by -1), instead of 0.
  • adj_row and adj_column are lists [<adj_rowA> <adj_rowB>...] where values respectively applies to <rowA> <rowB>...

bbmod: Hide

The function changes the extreme pixels of the clip, to fix or attenuate dirty borders
Thanks eupho for your first version of bbmod
Inspired from BalanceBorders from https://github.com/WolframRhodium/muvsfunc/ and https://github.com/fdar0536/Vapoursynth-BalanceBorders/
> Usage: bbmod(c, cTop, cBottom, cLeft, cRight, thresh, blur)

  • c: Input clip. The image area "in the middle" does not change during processing. The clip can be any format, which differs from Avisynth's equivalent.
  • cTop, cBottom, cLeft, cRight (int, 0-inf): The number of variable pixels on each side.
  • thresh (int, 0~128, default 128): Threshold of acceptable changes for local color matching in 8 bit scale. Recommended: 0~16 or 128
  • blur (int, 1~inf, default 999): Degree of blur for local color matching. Smaller values give a more accurate color match, larger values give a more accurate picture transfer. Recommended: 1~20 or 999

Notes:

  • 1. At default values of thresh = 128 blur = 999: You will get a series of pixels that have been changed only by selecting the color for each row in its entirety, without local The colors of neighboring pixels may be very different in some places, but there will be no change in the nature of the picture. With thresh = 128 and blur = 1 you get almost the same rows of pixels, i.e. The colors between them will coincide completely, but the original pattern will be lost.
  • 2. Beware of using a large number of pixels to change in combination with a high level of "thresh", and a small "blur" that can lead to unwanted artifacts "in a clean place". For each function call, try to set as few pixels as possible to change and as low a threshold as possible "thresh" (when using blur 0..16).

BlackBorders: Hide

Avoids dirty lines introduced by AddBorders
Actually avoids dirty lines most of the time, but borders may stay slightly dirty where there are vivid colors
> Usage: BlackBorders(clip, left, right, top, bottom)

  • left, right, top, bottom are the thicknesses of black borders in pixels

CropResize: Hide

Crop borders mod 1 while resizing a clip
> Usage: CropResize(clip, width, height, left, right, top, bottom, fill, resizer)

  • width and height are the dimensions of the resized cropped clip. If none of them is indicated, no resizing is performed. If only one of them is indicated, the other is deduced.
  • left, right, top, bottom are the crop values, odd numbers are possible
  • bb is a list containing bbmod values [cLeft, cRight, cTop, cBottom, thresh, blur] where thresh and blur are optional.
  • fill is a list containning the number of pixels to fill to extend the borders [left, right, top, bottom]. Functionnality required for CropResizeReader, shouldn't be used otherwise.
  • resizer should be Bilinear, Bicubic, Point, Lanczos, Spline16 or Spline36 (default)

CropResizeReader: Hide

CropResize for variable borders by loading crop values from a csv file
Also fill small borders and fix brightness/apply bbmod relatively to the variable border
> Usage: CropResizeReader(clip, csvfile, width, height, row, adj_row, column, adj_column, fill_max, bb, FixUncrop, resizer)

  • csvfile is the path to a csv file containing in each row: <startframe> <endframe> <left> <right> <top> <bottom> where left, right, top, bottom are the number of pixels to crop Optionally, the number of pixels to fill can be appended to each line <left> <right> <top> <bottom> in order to reduce the black borders. Filling can be useful in case of small borders to equilibriate the number of pixels between right/top and left/bottom after resizing.
  • width and height are the dimensions of the resized clip If none of them is indicated, no resizing is performed. If only one of them is indicated, the other is deduced.
  • row, adj_row, column, adj_column are lists of values to use FixBrightnessProtect2, where row/column is relative to the border defined by the cropping
  • Borders <=fill_max will be filled instead of creating a black border
  • bb is a list containing bbmod values [cLeft, cRight, cTop, cBottom, thresh, blur] where thresh and blur are optional. Mind the order: it is different from usual cTop, cBottom, cLeft, cRight
  • FixUncrop is a list of 4 booleans [left right top bottom] False means that FixBrightness/bbmod is only apply where crop>0, True means it is applied on the whole clip
  • resizer should be Bilinear, Bicubic, Point, Lanczos, Spline16 or Spline36 (default)

DebandReader: Hide

Read a csv file to apply a f3kdb filter for given strengths and frames
> Usage: DebandReader(clip, csvfile, grain, range)

  • csvfile is the path to a csv file containing in each row: <startframe> <endframe> <strength>
  • grain is passed as grainy and grainc in the f3kdb filter
  • range is passed as range in the f3kdb filter

LumaMaskMerge: Hide

Merges clips using a binary mask defined by a brightness level
> Usage: LumaMaskMerge(clipa, clipb, threshold, invert, scale_inputs)

  • threshold is the brightness level. clipb is applied where the brightness is below threshold
  • If invert = True, clipb is applied where the brightness is above threshold
  • scale_inputs = True scales threshold from 8bits to current bit depth.

RGBMaskMerge: Hide

Merges clips using a binary mask defined by a RGB range
> Usage: RGBMaskMerge(clipa, clipb, Rmin, Rmax, Gmin, Gmax, Bmin, Bmax, scale_inputs)

  • clipb is applied where Rmin < R < Rmax and Gmin < G < Gmax and Bmin < B < Bmax
  • scale_inputs = True scales Rmin, Rmax, Gmin, Gmax, Bmin, Bmax from 8bits to current bit depth (8, 10 or 16).

Utility functions: Hide

SelectRangeEvery: Hide

> Usage: SelectRangeEvery(clip, every, length, offset)

  • select <length> frames every <every> frames, starting at frame <offset>

FrameInfo: Hide

> Usage: FrameInfo(clip, title)

  • Print the frame number, the picture type and a title on each frame

DelFrameProp: Hide

Deletes primaries, matrix or transfer frame properties
Avoids "Unrecognized transfer characteristics" or "unrecognized color primaries" associated with Vapoursynth Editor
> Usage: DelFrameProp(clip, primaries, matrix, transfer)

  • primaries, matrix, transfer are boolean, True meaning that the property is deleted (default)

InterleaveDir: Hide

Load all mkv files located in a directory and interleave them
> Usage: InterleaveDir(folder, PrintInfo, DelProp, first, repeat)

  • folder is the folder path
  • PrintInfo = True prints the frame number, picture type and file name on each frame
  • DelProp = True means deleting primaries, matrix and transfer characteristics
  • first is an optional clip to append in first position of the interleaving list
  • repeat = True means that the appended clip is repeated between each loaded clip from the folder

ExtractFramesReader: Hide

Read a csv file to extract ranges of frames
> Usage: ExtractFramesReader(clip, csvfile)

  • csvfile is the path to a csv file containing in each row: <startframe> <endframe> the csv file may contain other columns, which will not be read

ReplaceFrames: Hide

Basically a wrapper for std.Trim and std.Splice that recreates the functionality of
AviSynth's ReplaceFramesSimple (http://avisynth.nl/index.php/RemapFrames)
that was part of the plugin RemapFrames by James D. Lin
> Usage: ReplaceFrames(clipa, clipb, mappings="[200 300] [1100 1150] 400 1234")

  • This will replace frames 200..300, 1100..1150, 400 and 1234 from clipa with the corresponding frames from clipb.

Overlay: Hide

From havsfunc: https://github.com/HomeOfVapourSynthEvolution/havsfunc
Added plane selection
> Usage: Overlay(clipa, clipb, x, y, mask, planes)

mt_lut: Hide

> Usage: mt_lut(clip, expr, planes)

  • expr is an infix expression, not like avisynth's mt_lut which takes a postfix one

GetPlane: Hide

> Usage: GetPlane(clip, plane)

  • Extracts a plane of a clip

scale: Hide

Scales 8-bits values for different bit depths, from havsfunc

Changelog: Hide

v2.0.1 (20.09.2018)
Fixed:

  • InterleaveDir: repeat=False lead to 'Non boolean' error

v2.0 (09.07.2018)
Added:

  • FixBrightnessProtect2 to fix all dirty lines at once. Example: FixBrightnessProtect2(clip=source, row=[1, 2], adj_row=[18,-2], column=[3, -3], adj_column=[5,5])
  • CropResize to make the whole process of cropping, filling, resizing mod 1 automatically.
  • Reader functions to load data from csv files: CropResizeReader for variable borders, DebandReader to apply different strengths for each defined range.
  • Some of utility functions for Vapoursynth Editor to simplify screen comparison

Improved:

  • BlackBorders, to avoid "drips" of vivid colors
  • Support of high bit depths for all functions, even if it hasn't been tested thoroughly. When applicable, functions have a "scale_inputs" argument to scale 8bits values to current bit depth.

v1.0 (23.05.2018)
Initial functions:

  • FixColumnBrightness/FixRowBrightness
  • FixColumnBrightnessProtect2/FixRowBrightnessProtect2
  • bbmod: should work with any bit depth
  • BlackBorders, to avoid dirty lines introduced by AddBorders (uses FixBrightness)
  • LumaMaskMerge, merges clips using a binary mask defined by a brightness level (to apply debanding on dark areas for example)
  • RGBMaskMerge, merges clips using a binary mask defined by a RGB range (to apply debanding on a given range of colors for example)
  • Some utility functions required by the previous ones: Overlay, mt_lut, GetPlane (extracts a plane of a clip) and scale (used to scale values for bit depth >< 8)

[6.3 awsmfunc - Successor of sgvsfunc]

DOWNLOAD

Dependencies: Hide

vsutil.py
edi_rpow2.py
rekt.py (NOTE: NOT TO BE CONFUSED WITH THE REKT ON PYPI)

Installation: Hide

Download awsmfunc.py and its dependencies. Place all files in the python packages directory:

[Fedora and Arch-based systems] /usr/lib64/python3.6/site-packages
[macOS] /usr/local/lib/python3.6/site-packages
[Ubuntu] /usr/local/lib/python3.6/dist-packages
[Windows] C:\Users\<user>\AppData\Local\Programs\Python\Python36\Lib\site-packages

[6.4 testencodes - A bash script for x264 test-encoding with VapourSynth (Linux)]

This bash function makes various encodes of a clip, from a vapoursynth script, changing the value of a given x264 setting defined by a range and a step. It basically takes the same arguments as x264.

  • All test encodes will be sorted in subfolders according to the tested settings and named <setting>_<value>.mkv.
  • Two modes are available: 1. testing mode with -t usually followed by the setting, the range (min and max values separated by an underscore "_") and the step. 2. output mode with -o followed by the filepath to output a single file.
  • You can test crf, bitrate, deblock, qcomp, aq-mode, aq-strength, psy-rd, ipratio, pbratio, mbtree.
  • aq-mode and mbtree don't need a range nor a step, deblock doesn't need a step.
  • I added an ipbratio testing parameter which will test ipratio and keep a distance of 0.1 between ipratio and pbratio.
  • No need to manually encode 2 passes. Indicating --bitrate will automatically encode the first and the second pass.
  • A log is created for each encode.
  • testencodes will detect if a test is already present and skip the re-encoding. This is useful if you refine a setting around a given value. Write -ow to force re-encoding.
  • Zones can be imported from a csv file containing <startframe> <endframe> <crf> with --zones f:/path/to/file.csv
  • Indicate the source disc with -d to specify the corresponding color matrix and primaries (recommanded for DVD encoding)

DOWNLOAD

Installation: Hide

  • Vapoursynth and x264 should be installed.
  • Download testencodes.sh, for example in /home/user/apps/
  • Define the sh file as executable:

chmod +x testencodes.sh

  • In ~/.bashrc add:

alias testencodes='sh /home/user/apps/testencodes.sh'

  • Open testencodes.sh and fill in the filepath of the vapoursynth script, the output test folder and change the default x264 settings if needed. The default settings will be used if you don't specify them in command line.
  • Test it by executing in a terminal

testencodes

Usage: Hide

Syntax: testencodes [options]

Options:

-t, --test              Choose the x264 setting to test:
                             - crf &lt;min&gt;_&lt;max&gt; &lt;interval&gt;
                             - bitrate &lt;min&gt;_&lt;max&gt; &lt;interval&gt;
                             - deblock &lt;min&gt;_&lt;max&gt;:&lt;deblock-threshold&gt; &lt;interval&gt; or &lt;deblock-strength&gt;:&lt;min&gt;_&lt;max&gt; &lt;interval&gt;
                             - qcomp &lt;min&gt;_&lt;max&gt; &lt;interval&gt;
                             - aq-mode
                             - aq-strength &lt;min&gt;_&lt;max&gt; &lt;interval&gt;
                             - psy-rd &lt;min&gt;_&lt;max&gt;:&lt;psy-trellis&gt; &lt;interval&gt; or &lt;psy-rdo&gt;:&lt;min&gt;_&lt;max&gt; &lt;interval&gt;
                             - ipratio &lt;min&gt;_&lt;max&gt; &lt;interval&gt;
                             - pbratio &lt;min&gt;_&lt;max&gt; &lt;interval&gt;
                             - ipbratio &lt;min&gt;_&lt;max&gt; &lt;interval&gt; where &lt;min&gt;_&lt;max&gt; &lt;interval&gt; apply to ipratio and pbratio = ipratio - 0.1
                             - mbtree

-ow, --overwrite        Overwrite test encodes instead of skipping re-encoding.

-rm, --remove           Remove the test encodes for the current tested setting before starting encoding.

-c, --clean             Delete all the test encodes before starting encoding.

-o, --output &lt;string&gt;   If no test is to be performed, output of a single file

-d, --disc &lt;string&gt;     Choose the source disc:
                             - BR
                               colormatrix and colorprim bt709 are applied
                             - PAL
                               colormatrix and colorprim bt470bg are applied
                             - NTSC
                               colormatrix and colorprim smpte170m are applied

-h, --help              Display this help

     --preset
     --profile
     --threads
     --level
     --b-adapt
     --min-keyint
     --vbv-bufsize
     --vbv-maxrate
     --rc-lookahead
     --me
     --direct
     --subme
     --trellis
     --crf
     --bitrate
     --deblock
     --qcomp
     --ipratio
     --pbratio
     --aq-mode
     --aq-strength
     --merange
     --psy-rd
     --bframes
     --ref
     --mbtree
     --colormatrix
     --colorprim
     --sar
     --zones &lt;string&gt;    Zones can be loaded from a file with '--zones f:/path/to/file' which each row is:
                         &lt;startframe&gt; &lt;endframe&gt; &lt;crf&gt;
                         Each value is separated with a space

Example: Hide

Determining the bitrate to work with (for which there is a small loss of quality which will be compensated with optimized settings):

testencodes -t crf 15_20 1

Testing qcomp from 0.6 to 0.8 by 0.02 with a working bitrate of 7000kbps:

testencodes -t qcomp 0.6_0.8 0.02 --bitrate 7000

Testing aq-mode:

testencodes -t aq-mode --qcomp 0.68 --bitrate 7000

Testing aq-strength from 0.5 to 1.2 by 0.1:

testencodes -t aq-strength 0.5_1.2 0.1 --aq-mode 2 --qcomp 0.68 --bitrate 7000

Testing psy-rdo from 0.85 to 1.15 by 0.05:

testencodes -t psy-rd 0.85_1.15:0 0.05 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68 --bitrate 7000

Testing psy-trellis from 0 to 0.15 by 0.05:

testencodes -t psy-rd 1.05:0_0.15 0.05 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68 --bitrate 7000

Testing final crf from 15 to 20 by 1 after deleting previous crf tests:

testencodes -rm -t crf 15_20 1 --psy-rd 1.05:0.10 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68

Refining final crf by 0.1:

testencodes -t crf 17_18 0.1 --psy-rd 1.05:0.10 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68

Changelog: Hide

v1.0 (09.07.2018)


Changelog

1
2
3
4
07.08.20   HyerrDoktyer: Replace hvf.SmoothLevels with std.Levels
09.05.20   stargaze:  Added parts 4.6 Detelecine &amp; 4.7 Blended fields
           Miraculix: Various adjustments, added awsmfunc (thanks)
28.04.19   stargaze:  Released
Edit Report
Pub: 31 Dec 2024 15:14 UTC
Edit: 31 Dec 2024 15:20 UTC
Views: 26