This tutorial aims at guiding advanced encoding on linux priviledging native tools. No virtual machine nor wine will be necessary here, except for specific needs. A few tools are still used with mono however, especially for OCRing subtitles.
This guide will mainly focus on the use of VapourSynth, the main alternative to Avisynth, and ffmpeg. If you are a windows user wanting to encode with VapourSynth, you can still have a look at this guide. The python scripts presented will work seamlessly. However, the bash shell script won't be usable on Windows.
I would like to warmly thank AHD for the Encoding Guide v3 by which this guide is inspired.
[1. Introduction to Vapoursynth]
[2. Demuxing]
[3. Encoding]
[4. Filtering]
[5. Remuxing]
[6. Appendix]
[1. Introduction to Vapoursynth]
[1.1 What is Vapoursynth?]
VapourSynth (VS) is an application for video manipulation, just like Avisynth. It has a core library written in C++ and a Python module to allow video scripts to be created.
VapourSynth Editor (VSE) is the script editor which is commonly used, with syntax completion and fast preview support.
Vapoursynth scripts are written in python. The main difference compared to Avisynth is the assignment of variables. You have to specify the variable names each time you’re doing something:
variable = doSomething()
You will have more things to write but it will be clearer to which variable you're assigning the operation.
[1.2 Installation of VapourSynth]
The prefered way to install VS is to get the needed packages from the repositories of your distribution, for its simplicity: python, x264, zimg, ffms2, l-smash, vapoursynth, vapoursynth-editor. However, I will here detail how to compile them if you want to have the latest versions which may not be available from your package manager. The initial instructions are taken from here. For Arch-based system users, you should be able to compile everything from AUR.
Compilation: Hide
In the following, we will make the installation in ~/apps. Make sure this directory exists or change it if you like.
1.2.1 Preparation
[Ubuntu]
Check to see if the "site-packages" directory exists:
if test -d /usr/local/lib/python3.6/site-packages; then echo "exists"; else echo "doesn't exist"; fi
If it does, moving all the files to dist-packages and deleting the directory is recommended.
Create a link to the "dist-packages" directory:
sudo ln -s /usr/local/lib/python3.6/dist-packages /usr/local/lib/python3.6/site-packages
[Fedora]
Create now a link for the plugins install directory to the VS one:
sudo ln -s /usr/lib64/vapoursynth /usr/local/lib/vapoursynth
For Fedora users, in the following when running ./configure for ffmpeg, l-smash, vapoursynth, x264 and zimg, add --prefix=/usr.
1.2.2 Dependencies for VS and its plugins
Required packages from the repositories of your distribution:
[Ubuntu]
sudo apt-get install build-essential git yasm libass-dev python3-pip python3-dev cython3 autoconf libtool libmagick++-dev qt5-default libfftw3-dev
[Fedora]
sudo dnf install automake gcc gcc-c++ yasm libass-devel python3-devel libtool ImageMagick-c++-devel qt-devel fftw-devel redhat-rpm-config qt5-qtbase-devel
sudo pip3 install cython
Compilation of the remaining dependencies:
L-SMASH:
x264:
ffmpeg:
zimg:
1.2.3 VapourSynth
Compilation of VapourSynth:
Now try it out:
vspipe --version
Possible errors: Hide
- "vspipe: error while loading shared libraries: libvapoursynth-script.so.0: cannot open shared object file: No such file or directory”.
This is caused by the non-standard location of libvapoursynth-script.so.0. Your dynamic loader is not configured to look in /usr/local/lib. One way to work around this error is to set the LD_LIBRARY_PATH environment variable. Open ~/.bashrc and add:
export LD_LIBRARY_PATH=/usr/local/lib
Then reload ~/.bashrc with:
. ~/.bashrc
- “Failed to initialize VapourSynth environment”.
This is caused by the non-standard location of the Python module, vapoursynth.so. Your Python is not configured to look in /usr/local/lib/python3.x/site-packages. One way to work around this error is to set the PYTHONPATH environment variable. Open ~/.bashrc and add (replace “x” with the correct number, you probably have python 3.6):
export PYTHONPATH=/usr/local/lib/python3.x/site-packages
Then reload ~/.bashrc with:
. ~/.bashrc
1.2.4 VapourSynth Editor
Compilation of VapourSynth Editor:
Add VapourSynth-Editor to application menu (Optional, for gnome):
Enter the following, save, then close gedit:
[1.3 Installation of plugins]
Some plugin will be available in the repositories of your distribution. Others will have to be compiled. Github repositories usually include the installation instruction. Let's take ffms2, that you will need, as example.
Compilation: Hide
- Download the source:
cd ~/apps && git clone https://github.com/FFMS/ffms2.git
- Browse to the source directory:
cd ffms2
- Next you need to find what files were included to assist with compiling the source:
ls
- You will be looking for one or more of the following files: autogen.sh, configure, CMakeLists.txt, or waf.
- ffms2 has autogen.sh so you would compile and install by doing the following:
- Pay attention to where the install places the file, most plugins will install to
/usr/local/lib
instead of the VS plugins directory. If this is the case you will need to create a symbolic link in the VS plugins folder by doing the following:[Ubuntu]
sudo ln -s /usr/local/lib/ffms2.so /usr/local/lib/vapoursynth/libffms2.so
[Fedora]
sudo ln -s /usr/lib64/libffms2.so /usr/lib64/vapoursynth/libffms2.so
Follow the same procedure for each plugin you want to install.
[1.4 Scripts]
Most Avisynth functions have been ported by a couple of developers/contributers. Their scripts usually rely on each other’s. Here are the scripts you can have a look at and their respective functions:
• havsfunc : Hide
Main functions:
daa
santiag
FixChromaBleedingMod
Deblock_QED
DeHalo_alpha
YAHR
HQDering mod
QTGMC
srestore
ivtc_txt60mc
logoNR
Vinverse
Vinverse2
LUTDeCrawl
LUTDeRainbow
Stab
GrainStabilizeMC
MCTemporalDenoise
SMDegrain
STPresso
SigmoidInverse, SigmoidDirect
GrainFactory3
InterFrame
SmoothLevels
FastLineDarken 1.4x MT MOD
Toon
LSFmod
TemporalDegrain
aafUtility functions:
AverageFrames
Bob
ChangeFPS
Clamp
KNLMeansCL
Overlay
Padding
Resize
SCDetect
Weave
ContraSharpening
MinBlur
sbr
DitherLumaRebuild
mt_expand_multi
mt_inpand_multi
mt_inflate_multi
mt_deflate_multi
• muvsfunc : Hide
LDMerge
Compare (2)
ExInpand
InDeflate
MultiRemoveGrain
GradFun3
AnimeMask (2)
PolygonExInpand
Luma
ediaa
nnedi3aa
maa
SharpAAMcmod
TEdge
Sort
Soothe_mod
TemporalSoften
FixTelecinedFades
TCannyHelper
MergeChroma
firniture
BoxFilter
SmoothGrad
DeFilter
scale (using the old expression in havsfunc)
ColorBarsHD
SeeSaw
abcxyz
Sharpen
Blur
BlindDeHalo3
dfttestMC
TurnLeft
TurnRight
BalanceBorders
DisplayHistogram
GuidedFilter (Color)
GMSD
SSIM
SSIM_downsample
LocalStatisticsMatching
LocalStatistics
TextSub16
TMinBlur
mdering
BMAFilter
• mvsfunc : Hide
Main functions:
Depth
ToRGB
ToYUV
BM3D
VFRSplice
Runtime functions:
PlaneStatistics
PlaneCompare
ShowAverage
FilterIf
FilterCombedUtility functions:
Min
Max
Avg
MinFilter
MaxFilter
LimitFilter
PointPower
CheckMatrix
postfix2infixFrame property functions:
SetColorSpace
AssumeFrame
AssumeTFF
AssumeBFF
AssumeField
AssumeCombedHelper functions:
CheckVersion
GetMatrix
zDepth
PlaneAverage
GetPlane
GrayScale
Preview
• fvsfunc : Hide
GradFun3mod
DescaleM (DebilinearM, DebicubicM etc.)
Downscale444
JIVTC
OverlayInter
AutoDeblock
ReplaceFrames (ReplaceFramesSimple)
maa
TemporalDegrain
DescaleAA
InsertSign
• kagefunc : Hide
inverse_scale
inverse_scale_clip_array
clip_to_plane_array
plane_array_to_clip
generate_mask
apply_mask
apply_mask_to_area
getY
generate_keyframes
adaptive_grain
conditional_resize
retinex_edgemask
kirsch
fast_sobel
get_descale_filter
hardsubmask
hardsubmask_fades
crossfade
hybriddenoise
insert_clip
get_subsampling
iterate
is16bit
getw
shiftl
fit_subsampling
mask_detail
• vshelpers of 4re : Hide
clamp
m4
build_mask
mt_lut
mt_lutxy
logic
get_luma
merge_chroma
rsoften
fit
move
get_decoder
subtract
starmask
• My own avisynth's ports and other utility functions [sgvsfunc]
Filtering functions:
FixRowBrightness
FixColumnBrightness
FixRowBrightnessProtect2
FixColumnBrightnessProtect2
FixBrightnessProtect2
bbmod
BlackBorders
CropResize
CropResizeReader
DebandReader
LumaMaskMerge
RGBMaskMergeUtility functions:
SelectRangeEvery
FrameInfo
DelFrameProp
InterleaveDir
ExtractFramesReader
ReplaceFrames
Overlay
mt_lut
GetPlane
scale
To install a python script for VS, just download the .py file from github and copy it into your python packages directory:
[Ubuntu] /usr/local/lib/python3.6/dist-packages
[Fedora & Arch] /usr/lib64/python3.6/site-packages
You will also need to install Adjust which is a requirement for havsfunc.
[1.5 Initialisation of a script]
1.5.1 Basics
Open VSE. The first step when creating a script is to initialise VS. Write the following always in the first line:
The second line can alternatively be core = vs.get_core()
And the following always in the last line of your script:
where “last” should contain the clip you want to encode.
1.5.2 Loading plugins
Plugins will be automatically loaded once installed. You can rely on “core” to call them: variable = core.pluginname.function(clip)
For example: src = core.ffms2.Source('/path/to/source.mkv')
1.5.3 Loading scripts
Scripts are loaded manually. Import them at the beginning of your script. You may give them names. For example:
You can finally call them in your script: variable = scriptname.function(clip)
For example to call QTGMC: out = haf.QTGMC(src)
[2. Demuxing]
[2.1 Ripping with MakeMKV]
This section is dedicated to the ripping of BluRay discs, which consists of copying the disc to a hard drive while removing the encryption. If you are not planning to use a physical Blu-ray as a source, you can skip this part.
MakeMKV will be your best bet for decrypting discs on Linux. The instructions to compile it can be found here.
In order to create a backup, select the desired drive on main screen in "Source" box and press "backup" icon on a toolbar or select "Backup" from file menu. Be sure to select "decrypt video files" on the next dialog. You can alternatively backup a BluRay in command line with:
makemkvcon backup disc:/dev/sr0 /path/to/output/folder/ --decrypt
[2.2 Retrieving BluRay specifications]
BDInfo provides information about the different streams of the BluRay. I will here detail how to use this cli version of BDInfo.
Installation: Hide
Install mono if it's not already done and download the prebuilt version:
cd ~/apps
wget https://github.com/zoffline/BDInfoCLI-ng/blob/UHD_Support_CLI/prebuilt/BDInfoCLI-ng_0.7.3.zip?raw=true -O tmp.zip && unzip -d BDInfoCLI-ng -j tmp.zip && rm tmp.zip
Add an alias to use it easily:
echo "alias bdinfocli='mono ~/apps/BDInfoCLI-ng/BDInfo.exe'" >> ~/.bashrc && . ~/.bashrc
If your alias is bdinfocli
, scan a disc with (report will be written in disc folder):
bdinfocli -w BD_PATH
[2.3 Demuxing]
Let's use ffmpeg to demux the disc. First identify the playlist and streams of your BluRay with:
ffmpeg -i bluray:"PATH/TO/BLURAY"
You should get such an output:
[bluray @ 0x55fcc39f7880] 2 usable playlists:
[bluray @ 0x55fcc39f7880] playlist 00000.mpls (1:20:51)
[bluray @ 0x55fcc39f7880] playlist 00005.mpls (0:25:58)
[bluray @ 0x55fcc39f7880] selected 00000.mpls
Input #0, mpegts, from 'bluray:LA POINTE COURTE':
Duration: 01:20:51.26, start: 600.000000, bitrate: 33023 kb/s
Program 1
Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
Stream #0:1[0x1100]: Audio: pcm_bluray (HDMV / 0x564D4448), 48000 Hz, stereo, s16, 1536 kb/s
Stream #0:2[0x1200]: Subtitle: hdmv_pgs_subtitle ([144][0][0][0] / 0x0090)
You can change the playlist with:
ffmpeg -playlist 5 -i bluray:"PATH/TO/BLURAY"
Additionally, if some subtitle tracks are missing, add -probesize 4G -analyzeduration 3600M
in the command line, before -i
. What's probesize and analyzereduction?: Hide
Without those options, subtitles may not appear because ffmpeg is just making a quick analysis. It is thus necessary to tell ffmpeg to scan deeper into the file so that it can find the them. The first is the number of bytes to read ahead into the file, and the second is the number of microseconds (of which there are 1,000,000 = 1M per second). These directives must come before the input file directive. In the example above, it will scan at least 4GB and 1 hour.
You can hide ffmpeg header with -hide_banner
.
To extract specific streams, use the -map
option. For instance, to extract the track identified as Stream #0:2
by ffmpeg, write -map 0:2 -c copy OUTPUT
. Demuxing several streams from the example above will be done with:
ffmpeg -playlist 0 -i bluray:"PATH/TO/BLURAY" \
-map 0:0 -c copy video.h264 \
-map 0:1 -c flac audio.flac \
-map 0:2 -c copy subs.sup
Demuxing a DVD?: Hide
First identify the vob files storing the video you want to extract. They are usually the consecutive files with a 1.1Go size. For example if those are VTS_02_1.VOB, VTS_02_2.VOB and VTS_02_3.VOB, you will identify the whole video with:
ffmpeg -i concat:"VTS_02_1.VOB|VTS_02_2.VOB|VTS_02_3.VOB"
This way you can extract video, audio and subitle tracks.
Welp, I wanna use eac3to!: Hide
Alright! Let's use eac3to, which works fine with wine.
Installation: Hide
Get eac3to and place it in a folder like /home/user/apps. We will now setup handy aliases: Add these lines to .bashrc or .bash_aliases in your /home/user/ folder:
First, change the current directory to where you want your demuxed files in. To get a summary of the playlists on your BluRay:
eac3to "Z:/path/to/BluRay/disc/"
If you want to demux all streams of playlist 1 and show the process progression:
eac3to "Z:/path/to/BluRay/disc/" "1)" -progressnumbers -demux
You can alternatively get streams information with:
eac3to "Z:/path/to/BluRay/disc/" "1)"
and demux the identified streams with the following syntax:
eac3to "Z:/path/to/BluRay/disc/" "1)" 1:chapters.txt 2:video.* 4:main.thd \
4:main.ac3 -core 6:com.* 7:fr-forced.* 8:fr-sdh.* 9:eng.* 10:spa.* -progressnumbers
It will name them as stated and choose the correct file extension when ".*" is indicated. In this example, the TrueHD audio track is demuxed and a compatibility ac3 track is created by extracting its core.
[2.4 Extracting a remux]
You can directly encode from a remux, but you may need to extract the audio and subtitle tracks of the remux to encode and OCR them later on. Extracting tracks from a mkv file can be done with mkvtoolnix that you will find in your distribution repositories.
Identify the stream indexes:
mkvmerge remux.mkv --identify
Extract the streams you need:
mkvextract remux.mkv tracks 1:main-audio.dtshd 2:sub1.sup 3:sub2.sup
You can also extract the chapters with:
mkvextract remux.mkv chapters chapters.xml
[3. Encoding]
If you are releasing a remux, then jump to chapter 5. For an encode, if you demuxed a BluRay with eac3to, remux at least the video track with mkvtoolnix to encode from a mkv file.
[3.1 Vapoursynth script #1 : Filtering the source]
Let's create the first script which will be used for the test encodes. Open VSE and save a script containing:
As you can see, we added 4 operations between the usual initialisation lines:
#1
imports the remux
The video will be indexed by ffms2 (recommanded for mkv files) and named "src".
If however you want to index a m2ts file, it is recommanded to import it with LWLibavSource this way:
src = core.lsmas.LWLibavSource('/path/to/source.m2ts')
For Mpeg1/2 files, import a d2v file. To do that, install D2VWitch and the d2vsource plugin. In D2VWitch add the Mpeg files. You can uncheck the demuxing of audio tracks. Click Engage and a d2v file is generated. In the VS script, replace the corresponding line by:
src = core.d2v.Source('/path/to/source.d2v')
#2
crops the video
Obviously it doesn't crop anything yet, so click on Script > Preview and on the "Crop assisstant" to crop the black lines. Report the values in the script.
You can alternatively use
CropAbs(clip, width, height, left, top)
It is possible to omit the input variable names "clip=", "left="... but in this case pay attention to use the correct order.
If you have an odd number to crop, please read part [4.1 Fixing borders]
#3
resizes the video
The width is calculated for a 720p resize by doing a simple cross-product and making it mod 2 for AR > 16:9 (1280x...). For AR < 16:9 (...x720), you should replace these lines by:
w = round((720*crop.width/crop.height) / 2) * 2
resized = core.resize.Spline36(clip=crop, width=w, height=720)
You should delete this part and replace "resize" by "crop" for a 1080p release.
#4
makes an extract
We are using SelectRangeEvery to trim the first and last 10,000 frames to omit intro and credits, and select 50 frames every 3000 frames. You can of course change these numbers.
Something faster?: Hide
You can alternatively replace steps 2 & 3 by using CropResize which can calculate the resolution by itself. You just have to indicate either the width or the height. For example:
crop = sgf.CropResize(clip=src, width=1280, top=138, bottom=138)
Avoid the "Unrecognized transfer characteristics" or "unrecognized color primaries" error with ffms2: Hide
This is only related to VSE and shouldn't appear when encoding. Just add
src = sgf.DelFrameProp(clip=src)
after importing the source
However if you notice a shift in the colors, indicate also the matrix with:
src = core.resize.Spline36(clip=src, width=src.width, height=src.height, matrix_in_s='709')
or just add the matrix_in_s argument in your resize line.
[3.2 Test encoding]
3.2.1 Basic utilization of vspipe
To encode from our script, we are going to evaluate it and ouput the frames to x264 using VSPipe. Here is an example of command line:
vspipe --y4m script.vpy - | x264 --demuxer y4m - --output encoded.mkv
You can add all the x264 settings you want to use in this command.
3.2.2 Using the testencodes bash script
To make test encoding easier, I wrote a bash script which enables to make mutiple encodes in a row changing a given x264 setting. You can find it in part [6.4 testencodes - A bash script for x264 test-encoding with VapourSynth (Linux)]. Follow the installation instructions. Provide the path of your VS script #1.
As an example, to make multiple encodes changing qcomp from 0.6 to 0.8 by steps of 0.02, with a bitrate of 7000 kbps and the default settings indicated in testencodes.sh, run the following command:
testencodes -t qcomp 0.6_0.8 0.02 --bitrate 7000
Once you have encoded a first set of test encodes for a given setting, you will need to compare them to choose which value lead to the best quality. For the comparison, please read part [3.3 Vapoursynth script #2: Comparing the test encodes]. Then come back to testencodes, specify the chosen value and test the next setting.
As an example, if you chose qcomp=0.68, testing aq-strength from 0.5 to 1.2 by 0.1 will be done with:
testencodes -t aq-strength 0.5_1.2 0.1 --qcomp 0.68 --bitrate 7000
Go back and forth until you have determined all the desired settings. The test method varies from an encoder to another, but the settings usually tested are crf, qcomp, aq-mode, aq-strength, psy-rd and sometimes ipratio and pbratio.
Usage example: Hide
Determining the bitrate to work with (for which there is a small loss of quality which will be compensated with optimized settings):
testencodes -t crf 15_20 1
Testing qcomp from 0.6 to 0.8 by 0.02 with a working bitrate of 7000kbps:
testencodes -t qcomp 0.6_0.8 0.02 --bitrate 7000
Testing aq-mode:
testencodes -t aq-mode --qcomp 0.68 --bitrate 7000
Testing aq-strength from 0.5 to 1.2 by 0.1:
testencodes -t aq-strength 0.5_1.2 0.1 --aq-mode 2 --qcomp 0.68 --bitrate 7000
Testing psy-rdo from 0.85 to 1.15 by 0.05:
testencodes -t psy-rd 0.85_1.15:0 0.05 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68 --bitrate 7000
Testing psy-trellis from 0 to 0.15 by 0.05:
testencodes -t psy-rd 1.05:0_0.15 0.05 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68 --bitrate 7000
Testing final crf from 15 to 20 by 1 after deleting previous crf tests:
testencodes -rm -t crf 15_20 1 --psy-rd 1.05:0.10 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68
Refining final crf by 0.1:
testencodes -t crf 17_18 0.1 --psy-rd 1.05:0.10 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68
[3.3 Vapoursynth script #2: Comparing the test encodes]
Let's imagine you just tested qcomp with testencodes. All the resulting tests are located in tests/qcomp/. This second VS script will automatically load and interleave them to enable you to make the comparison frame vs frame:
#1
should be the processing of your source copied from the VS script #1
#2
will print the frame number and picture type on the source and add the title "Source".
#3
will scan the chosen test folder for all mkv files, print the frame properties and the setting tested and interleave them with the source. repeat=True will display the source frame between all test frames (Source | test1 | Source | test2 | Source | test3...). repeat=False will display the source frame only once (Source | test1 | test2 | test3...). This is up to the encoder.
You then just have to preview, and you can compare P-B or B-B frames.
[3.4 Encoding the whole video]
Once you have all your settings, you can encode the entire video. To do so, come back to your script #1
and delete the part that is extracting a limited amount of frames. Use testencodes in output mode.
For example, to encode the final release with crf set to 17.3 and the optimized settings from the usage example:
testencodes -o /home/user/encoding/output.mkv --crf 17.3 --psy-rd 1.05:0.10 \
--aq-strength 0.8 --aq-mode 2 --qcomp 0.68
If your are encoding a DVD, you should do an anamorphic encode by specifying the correct SAR with --sar
.
What's SAR?: Hide
The video, regardless of its width/height ratio, is stored into standard resolutions. For example, PAL DVD stores it into 720x576. So if you want your video to be displayed in correct proportions, you have to resize it. SAR is here to indicate "the width should be stretch that much to have the correct ratio". You determine it using this formula:
DAR = FAR × SAR
where:
- DAR is the Display Aspect Ratio = (width of displayed video) / (height of displayed video), also given by MediaInfo
- FAR (or PAR) is the Frame (Picture) Aspect Ratio = (width of video as it is stored) / (height of video as it is stored)
- SAR is the Sample Aspect Ratio, the width stretch to apply to a pixel.
For example, let's imagine you have a NTSC DVD with a DAR of 16/9, this formula becomes:
16/9 = (720/480) × SAR
→ SAR = 32/27You may be confused by what you can find on internet about SAR. This is because this standard is for MPEG4/AVC/x264 streams, and for MPEG2 names were switched: SAR was called PAR (for Pixel Aspect Ratio) and FAR/PAR was called SAR (for Storage Aspect Ratio)...
[4. Filtering]
In this section, we are going further into encoding by developping the main filters which can be applied to fix some artefacts present on a BluRay. Filtering has usually to be applied after cropping and before resizing (except debanding done after resizing).
[4.1 Fixing borders]
4.1.1 Black lines
Filling black lines
Sometimes you would like to crop an odd number of lines but you can't. Or the crop would vary along the movie. You thus will have to fill those lines (with a copy of the adjacent one) using FillMargins. See this example of filling 1 pixel on right and top: Source, Filtered: [Show comparison]
Install the fillborbers plugin and use the following code:
filtered = core.fb.FillBorders(crop, left, right, top, bottom, mode="fillmargins")
Resizing
If you are resizing to 720p and the black line is present on the whole movie, omit it when resizing with the following syntax. You still need to fill the black line first to avoid a dirty border.
fill = core.fb.FillBorders(crop, left, right, top, bottom, mode="fillmargins")
resized = core.resize.Spline36(fill, width, height, src_left, src_top, src_width, src_height)
where src_left/top is the number of omitted lines at left/top and src_width/height is the width/height of the non-omitted area.
For example, if you have a 1920x1080 video with 1 line at top and bottom to omit:
fill = core.fb.FillBorders(crop, left=0, right=0, top=1, bottom=1, mode="fillmargins")
h = round((1280*1078/1920) / 2) * 2
resized = core.resize.Spline36(fill, width=1280, height=h, src_left=0, src_top=1, src_width=1920, src_height=1078)
You can also use CropResize from sgvsfunc which will do this automatically.:
crop = sgf.CropResize(clip=src, width=1280, top=1, bottom=1)
4.1.2 Dirty lines
If some lines at the border are "dirty", they can usually be fixed. See this example (fixing 1 pixel on top and bottom): Source, Filtered: [Show comparison]
First you should check if the lines contain valuable information: Are they coherent with their neighboring lines? Are they not just poor copies? Zoom in to find out. If they are garbage data, just crop them.
Uniform brightness fix
If the dirty lines are uniformly too dark or bright, the proper way to fix them is to use FixBrightnessProtect2. In row
and column
, indicate the indexes of the row/column to fix, 1 being the first starting from the left/top. Negative numbers are relative to right/bottom. In adj_row
/adj_column
, specify the fix value (<100), positive to brighten, negative to darken:
filtered = sgf.FixBrightnessProtect2(clip, row, adj_row, column, adj_column)
For example:
filtered = sgf.FixBrightnessProtect2(clip=src, row=[1,2], adj_row=[18,-2], column=[3,-3], adj_column=[5,5])
will:
- brighten the 1st row by +18
- darken the 2nd row by -2
- brighten the 3rd column from the left and the 3rd column from the right by +5
Uniform color fix
If the lines have a dirty color, you can use ContinuityFixer or bbmod.
fix = core.edgefixer.ContinuityFixer(src, left, top, right, bottom, radius)
filtered = sgf.bbmod(c, cTop, cBottom, cLeft, cRight, thresh, blur)
Here left/cLeft... are the number of pixels on which the fix is performed. With their default values of radius (=height of video) and blur (=999), they apply a uniform fix, based on the whole row/column near the dirty line. This is the less destructive scenario.
Note that ContinuityFixer will allow you to fix separately luma and chroma planes. So if the brightness is fine, you can only fix the colors. Lists of 3 values should be given to left/top/right/bottom
, that apply to [y, u, v], where y is the luma plane (brightness) and u and v the chroma planes (color). For example:
fix = core.edgefixer.ContinuityFixer(src, left=[0,3,3], top=[0,0,0], right=[0,1,1], bottom=[0,0,0])
will repair the colors of the first 3 columns and the last column, while keeping their brightness untouched. bbmod is unable to separate luma and chroma fix, this is why ContinuityFixer is prefered.
Local matching
If the brightness or color can't be fixed with a uniform filter, you should use ContinuityFixer or bbmod while decreasing the radius/blur parameter. In practice, to determine the new brightness/color of a given pixel, the filters will take an average of the neighboring pixels located at a distance < radius/blur. This is why with their default high values, they perform a uniform fix. On the contrary, a radius/blur of 1 will lead to a copy of the nearest line (~FillMargins), in that case, all data is lost. Decreasing to 5-20 usually enables to have a good compromise between local brightness/color matching and the conservation of some data to have a different line. In any case, don't use ContinuityFixer/bbmod on more than 3 pixels, since it will deform the details near the border, especially with lower radius/blur.
You can also give BalanceBorders a try in the most damaged lines, but be careful with it as it is a very agressive filter. It is available in the muvsfunc script:
fix = muvs.BalanceBorders(c, cTop, cBottom, cLeft, cRight, thresh, blur)
It works similarly to bbmod, decrease blur if you need more local matching.
[4.2 Debanding]
Banding is discretization of gradients. See this example: Source, Filtered: [Show comparison]
4.2.1 f3kdb filter
The debanding filter usually used is f3kdb. Install the f3kdb plugin for VS and use it with:
db = core.f3kdb.Deband(resized,y=60,cb=60,cr=60,grainy=48,grainc=48,dynamic_grain=True,range=16)
- y, cb and cr are the strengths of the filter on the different planes, which should be the lowest value leading to the disapearence of the banding.
- grainy and grainc are factors responsible for the introduction of grain. It should be high enough to keep the scene unbanded, but low enough not to deform the detail of the image.
- you can play with the range for better results
You should apply it after resizing. Preview the movie to check what it does on the banded scenes and you can even temporarily make the source-filtered comparison with:
to optimize the parameters and ensure that the filter is not destructive.
4.2.2 Applying the filter on given ranges
Debanding can be a destructive filter, this is why it should be applied on the smallest amount of frames. The next step is to check which frames it has to be applied to. Preview the source in VSE. Now you will have to go through the entire movie and write down the ranges (which should usually be < 1000 frames) where you see banding. Look for areas with smooth color gradation (shadows, sky...). Yes, this can be a painstaking job.
Once you have noted the ranges, you can use ReplaceFrames to only apply debanding on them. As an example:
filtered = sgf.ReplaceFrames(resized, db, mappings="[514 605][8959 9125][103670 104077]")
In "resized", it will replace the given ranges by the ones of "db", so that everything else is untouched.
You can define several debanding filters and apply them to their corresponding ranges if different strengths have to be applied from some ranges to others.
Load ranges and strengths from a csv file: Hide
If you have a high number of ranges, or various strengths to apply, it may be more efficient to create a csv file with the defined zones in the following format for each zone (row):
<startframe> <endframe> <strength>
and use my DebandReader function:
filtered = sgf.DebandReader(resized, csvfile=/path/to/file.csv, grain=64, range=30)
4.2.3 Applying the filter on given areas
You can additionally apply f3kdb to given areas in the picture by defining a mask based on:
- brightness, to only deband dark or bright areas, what you can do with LumaMaskMerge
filtered = sgf.LumaMaskMerge(resized, db, threshold=128, invert=False)
Choose the right brightness level "threshold". Debanding will be applied where the brightness is below threshold, above if invert=True
- RGB colors, to only deband a given range of colors, what you can do with RGBMaskMerge
filtered = sgf.RGBMaskMerge(resized, db, Rmin, Rmax, Gmin, Gmax, Bmin, Bmax)
Determine the extreme R G B values at boundaries of the area you want to deband to fill in Rmin, Rmax, Gmin, Gmax, Bmin, Bmax. Debanding will be applied where Rmin < R < Rmax and Gmin < G < Gmax and Bmin < B < Bmax.
4.2.4 x264 zoning
As it is often difficult to keep debanded scenes unbanded without introducing too much grain, the encode is usually zoned to lower crf in the debanded parts.
Selecting the ranges and testing crf
First you will have to determine the crf on the debanded scenes. You can select them using python code as the following example:
selec = filtered[514:606] + filtered[8959:9126] + filtered[103670:104078]
which is equivalent to:
selec = core.std.Trim(filtered, first=514, last=605) + core.std.Trim(filtered, first=8959, last=9125) + core.std.Trim(filtered, first=103670, last=104077)
As you can see, slicing operations in Python do not include the last frame.
If you wrote down the ranges in a csv file, you can directly extract them with ExtractFramesReader:
selec = sgf.ExtractFramesReader(filtered, csvfile=/path/to/file.csv)
Then perform a crf test encoding on those ranges.
Zoning the encode
You can finally zone the corresponding ranges in the encode by adding to the x264 or testencodes command:
--zones <startframe1>,<endframe1>,crf=<crf1>/<startframe2>,<endframe2>,crf=<crf2>...
With testencodes, you can load the zones from a csv file containing
<startframe> <endframe> <crf>
by adding
--zones f:/path/to/file.csv
[4.3 Fixing gamma bug]
On some BluRay discs, the brightness level is not correct. See this example: Source, Filtered: [Show comparison]. You can particularly notice it when comparing different BluRay editions of the movie.
Applying a 0.88 gamma shift will usually fix the brightness. It can be done with std.Levels, it is recommended to do this in at least 16 bits of precision.
filtered = core.std.Levels(clip, gamma=0.88, min_in=16, max_in=235, min_out=16, max_out=235, planes=0)
For 16 bit:
filtered = core.std.Levels(clip, gamma=0.88, min_in=16 << 8, max_in=235 << 8, min_out=16 << 8, max_out=235 << 8, planes=0)
[4.4 Anti-aliasing]
For a very slight aliasing, I had good results with this port of nnedi3_rpow2
import edi_rpow2 as edi
aa = edi.nnedi3_rpow2(clip=src,rfactor=2,nns=1,correct_shift="Spline36")
You can also look at vsTAAmbk
For heavier aliasing, daa is recommended:
aa = haf.daa(src)
Anti-aliasing can also be a destructive filter, so be sure to choose the appropriate one by comparing source/filtered like it is done for debanding.
[4.5 Deinterlacing]
If your source is fully interlaced, you can use QTGMC from havsfunc:
filtered = haf.QTGMC(src,TFF=True, FPSDivisor=2, Preset='Slow', TR2=1, SourceMatch=1)
[4.6 Detelecine]
If your source underwent a 2:3 pulldown, VFM will deinterlace the correct frames, and VDecimate will remove the duplicates. What's a 2:3 pulldown?: Hide
In the United States and other countries where television uses the 59.94 Hz vertical scanning frequency, video is broadcast at 29.97 frame/s. To that extend, the movie is converted from 24 to 29.97 frame/s, by interlacing the fields according to this scheme:
A 2:3 pulldown can thus be noticed by 2 combed frames over 5.
Please note that:
- mode 0 is the safest of all the options in the sense that it won’t risk creating jerkiness due to duplicate frames when possible, but if there are bad edits or blended fields it will end up outputting combed frames when a good match might actually exist. My suggestion is thus to start with mode 0. If you still notice combed frames when it's applied, increase it up to mode 5 but mind the artifacts it could create.
- VDecimate will drop one frame in every cycle frames – the one that is most likely to be a duplicate. By default, cycle=5, wich is the case in a 2:3 pulldown. If you have a different pulldown, change the cycle argument.
[4.7 Blended fields]
DVD wizardry can go pretty far. If you notice a "ghosting" effect after deinterlacing, that's probably because the movie is field blended. In this case you might benefit from using srestore after QTGMC.
[5. Remuxing]
[5.1 Audio encoding]
Audio encoding will be achieved with ffmpeg here. For best results with aac, compile ffmpeg with libfdk_aac support as it is done in part [1.2 Installation of VapourSynth]. Keep in mind that you may get lesser compatibility than with a certified encoder.
The basic command line to encode audio tracks is:
ffmpeg -i INPUT -c:a CODEC -b:a BITRATE -ac NB_CHANNELS OUTPUT
Examples:
- Extracting DTS core:
ffmpeg -i bluray:"PATH/TO/BLURAY" -map 0:1 -bsf:a dca_core -c:a copy OUTPUT.dts
- Encoding AC3 640kbps:
ffmpeg -i bluray:"PATH/TO/BLURAY" -map 0:1 -c:a ac3 -b:a 640k -ac 6 OUTPUT.ac3
- Encoding FLAC:
ffmpeg -i bluray:"PATH/TO/BLURAY" -map 0:1 -c:a flac OUTPUT.flac
- Encoding AAC:
ffmpeg -i bluray:"PATH/TO/BLURAY" -map 0:1 -c:a libfdk_aac -vbr 5 OUTPUT.aac
Use -vbr 5
for main audio (best quality). Decrease it for commentary tracks (to have around 80kbps)
Using a certified encoder for AC3: Hide
If you are encoding DTS or Dolby type streams, you will need to get individual WAV files for each channel, using the .wavs extension with eac3to. Each wav will be labeled with the proper channel. As an example:
eac3to input.thd output.wavs
if you need to downmix to 5.1 channels, append
-down6
to eac3to command.In this example, we are using SurCode for Dolby Digital 5.1. Installation of SurCode for Dolby Digital 5.1: Hide
Download Minnetonka Audio SurCode for Dolby Digital 5.1
Unzip it. From the unzipped directory, run Setup.exe via wine:
wine Setup.exe
Proceed to the installation
Copy lservrc file from the crack folder to SurCode for Dolby Digital program directory (default "/home/user/.wine/drive_c/Program Files (x86)/Minnetonka Audio Software/SurCode for Dolby Digital"), overwriting original file.
Run Surcode with usual wine command
Go to Options > Encode Options and set as below:
The only real thing you need to do is choosing the destination file, load up the wavs generated by eac3to respective to their label. Make sure start is 0 and end is the length of the movie, and click on Encode.
Using a certified encoder for DTS-HD MA and DTS: Hide
Export the individual wav files per channel as explained in the previous part about certified AC3 encoding.
Installation of DTS-HD Master Audio Suite: Hide
You may need to install lib32-mpg123 and gnutls28 from the repositories of your distribution.
Create a 32bits prefix for wine:
WINEARCH=win32 WINEPREFIX=$HOME/.wine32 winecfg
Choose Windows XP as Windows version.
Download Java Runtime Environment 7
Install Java in the 32-bits prefix of wine:
WINEPREFIX=$HOME/.wine32 wine start /unix jre-7u80-windows-i586.exe
Download DTS-HD Master Audio Suite
Unzip it and install DTS-HD Master Audio Suite in the 32-bits prefix of wine, from the unzipped directory:
WINEPREFIX=$HOME/.wine32 wine start /unix setup.exe
Go to the DTS-HD Master Audio Suite installation directory and run the license checker to introduce the key:
cd $HOME/.wine32/drive_c/Program Files/DTS/MAS-SAS
WINEPREFIX=$HOME/.wine32 wine start /unix MAS-SAS_Authorizer.exe
Always run DTS-HD Master Audio Suite with the 32-bits prefix:
WINEPREFIX=$HOME/.wine32 wine start /unix dtshd.exe
Load up the wav tracks created by eac3to. Set the correct Stream Type and change channel count as necessary (as well as bitrate). Start by clicking on the ENCODE button.
[5.2 Subtitles]
5.2.1 OCR subtitles
To OCR subtitles, I don't think there's a better tool than SubExtractor, which runs with mono. You will find some guides on how to use SubExtractor around, the utilisation is pretty straightforward.
Installation: Hide
Just execute the following command in a terminal:
mono DvdSubExtractor.exe
from the SubExtractor folder to run the software.
You can create a shortcut if you want the software to appear in your application menu. If you are using Gnome, this is done by creating a SubExtractor.desktop file in ~/.local/share/applications/ wich contains:
if your SubExtractor folder is located in /home/user/apps/
As native OCR tools, there is VobSup2srt which uses tesseracts (install the necessary tesseract language packs, for example tesseract-data-eng for english). I don't recommand this software as it is skipping some subtitle lines and won't format italic characters. I want to mention it however as it can prove useful when other OCR software like Subextractor fail and it is very convenient (you don't need to specify any character and it does relatively few mistakes). If you use it check for the aforementioned issues. You will need to convert sup subtitles to sub/idx using BDSup2Sub (run the jar file with 'java -jar BDSup2Sub.jar'). To OCR with vobsub2srt:
vobsub2srt subtitles_name --tesseract-lang eng
--tesseract-lang eng
is optional if the langage is specified in the file
5.2.2 Checking subtitles
Once the subtitles OCR'd, or if you downloaded the subtitles online, we need to check them for errors. This can be done with SubtitleEdit by running a spell check (Spell Check > Spell Check) and fix them for common errors (Tools > Fix common errors).
Installation: Hide
Download the portable version of SubtitleEdit. Just execute the following command in a terminal:
mono SubtitleEdit.exe
from the SubtitleEdit folder to run the software.
You can create a shortcut if you want the software to appear in your application menu. If you are using Gnome, this is done by creating a SubtitleEdit.desktop file in ~/.local/share/applications/ wich contains:
if your SubtitleEdit folder is located in /home/user/apps/
[5.3 Chapters]
Either named or sequentially numbered chapters are really appreciated. You can extract and edit chapters' disc with chaptorEditor.
[5.4 Muxing]
5.4.1 GUI
MKVToolNix will enable you to mux all the streams you prepared together:
- Drag and drop your video track into Source files. Then add your main audio track, followed by any commentaries. Finally, add all your subtitle tracks. The tracks, chapters, and tags section will show all the tracks to be muxed and in what order
- For each of those tracks, there is a General options tab. For track name, you can put something descriptive, like "Commentary with director Steven Spielberg". There is also the language field, obviously select the correct language for audio and subtitle tracks. For default track flag, this really only needs to be set for the main audio, and possibly the english subtitles. Don't force any track.
- In Timestamps and default duration, you can set a delay, for example if you are including a track from an other source with a relative delay. Stretch can fix progressive delays and it can be set as the fps difference, for example 25000/23976. However, don't stretch an audio file as it leads to a pitch shift, unless the stretch is small (23.976 -> 24.000 for instance), or for commentary tracks.
- In the Output tab, for the file title, you can put the release name or the movie name + year. For chapters, select the chapter file you have. You can get away with not setting language/charset if they are the same as your system's default.
Set the destination file and start multiplexing. You are done, congrats!
5.4.2 CLI
This is primarily aimed at server users. The cli version works as follows:
mkvmerge -o OUTPUT.mkv --title TITLE --chapters CHAPTERS.xml \
--track-name 0:TRACK_TITLE --language 0:ISO VIDEO.h264 \
--track-name 0:TRACK_TITLE --language 0:ISO AUDIO.dtsma \
--language 0:ISO --default-track 0/1 SUBTITLE.sup
- You should put things in quotes if there are spaces or other metacharacters involved.
- OUTPUT.mkv is your output file name, e.g. Mystic.River.2003.BluRay.Remux.1080p.VC-1.DTS-HD.MA.5.1.mkv
- TITLE is the general movie title or release name (Mystic River [2003] or Mystic.River.2003.BluRay.Remux.1080p.VC-1.DTS-HD.MA.5.1), you can omit this option if you want. Note that if you try to take a specific track from another matroska file and that file as the title set, you will inherit that title, so you might want to just set the title to the empty string.
- VIDEO is the video file you want to include, same idea for AUDIO and SUBTITLE
- ISO is the iso code for the language, e.g. eng for English. See here for more. You want the ISO 639-2 Code.
- TRACK_TITLE is the title of the track, you can put something descriptive, like "Commentary with director Steven Spielberg". For remuxes, you will have to set this area to the relevant portion of the BDInfo, e.g. for the video track, I would have something like "VC-1 Video / 25534 kbps / 1080p / 23.976 fps / 16:9 / Advanced Profile 3". This is only for the video track and the main audio. Most tracks will probably have this field blank.
- --default-track should be set to 0 or 1
An example:
mkvmerge -o "Mystic.River.2003.BluRay.Remux.1080p.VC-1.DTS-HD.MA.5.1.mkv" \
--title "Mystic.River.2003.BluRay.Remux.1080p.VC-1.DTS-HD.MA.5.1.mkv" \
--chapters chapters.xml \
--track-name 0:"VC-1 Video / 25534 kbps / 1080p / 23.976 fps / 16:9 / Advanced Profile 3" --language 0:eng video.vc1 \
--track-name 0:"DTS-HD Master Audio / English / 5.1 / 48 kHz / 3507 kbps / 24-bit" --language 0:eng audio.dtsma \
--language 0:eng --default-track 0 eng.sup
[6. Appendix]
[6.1 Generating screenshot comparisons]
Making a screenshot comparison for an encode is similar to the comparison we made when testing settings. Here we will compare the source, the filtered video and the encode in a VS script #3:
#1
should be the processing of your source without any filtering (debanding, correction of dirty lines...). It should only include the cropping and the resizing.
#2
should be the processing of your source copied from the VS script #1, with all filters used (debanding, correction of dirty lines...).
#3
loads your encode
#4
prints the frame info on source, filtered and encode.
#5
interleaves the 3 aforementioned clips to compare them frame vs frame
Preview this script in VSE and look for P-B or B-B comparisons. Right-click on an image and select Save snapshot to save the png image.
Encoding a DVD? Make sure to resize the screenshots to respect DAR: Hide
As explained in part [3.4 Encoding the whole video], DVD video is stored into a standard resolution and is resized during playback to have the right AR. VSE will display the frames in their storage resolution. This means that you have to resize them to get them to their display resolution. Here is the script including this resizing:
[6.2 sgvsfunc - Filtering and utility functions for VapourSynth and VapourSynth Editor]
Installation: Hide
Download sgvsfunc.py and place it in the python packages directory:
[Fedora and Arch-based systems] /usr/lib64/python3.6/site-packages
[macOS] /usr/local/lib/python3.6/site-packages
[Ubuntu] /usr/local/lib/python3.6/dist-packages
[Windows] C:\Users\<user>\AppData\Local\Programs\Python\Python36\Lib\site-packages
Functions: Hide
Filtering functions: Hide
FixRowBrightness/FixColumnBrightness: Hide
> Usage: FixRowBrighnessProtect2(clip, row, input_low=16, input_high=255, output_low=16, output_high=255, scale_inputs)
- row is the row you want to work on.
- input_low, input_high, output_low, output_high are the arguments passed to levels. input_high is more or less the more interesting value to tweak, lowering it will brighten the line and raising it will dim the line.
- scale_inputs = True scales input_low, input_high, output_low, output_high from 8bits to current bit depth.
FixRowBrightnessProtect2/FixColumnBrightnessProtect2: Hide
> Usage: FixRowBrighnessProtect2(clip, row, adj_val, prot_val, scale_inputs)
Thanks zorbit and nonshatter for the original FixBrightnessProtect2
- prot_val can be left out and the default of 16 will be used.
- row is the row you want to work on.
- adj_val should be a number x where -100 < x > 100. This parameter decides how much the brightness should be affected. Numbers below 0 will make it darker and number above 0 will make it brighter.
- prot_val is the protect value. This is what makes it behave differently than the normal FixBrightness. Any luma above (255-prot_val) will not be affected which is the basic idea of the protect script.
- scale_inputs = True scales prot_val from 8bits to current bit depth.
FixBrightnessProtect2: Hide
> Usage: FixBrightnessProtect2(clip, row, adj_row, column, adj_column, prot_val)
- row and column are lists [<rowA> <rowB>...] where negative numbers are relative to right/bottom. Indexes are different from FixRow/ColumnBrightnessProtect2: First row is defined by index 1 (and last by -1), instead of 0.
- adj_row and adj_column are lists [<adj_rowA> <adj_rowB>...] where values respectively applies to <rowA> <rowB>...
bbmod: Hide
The function changes the extreme pixels of the clip, to fix or attenuate dirty borders
Thanks eupho for your first version of bbmod
Inspired from BalanceBorders from https://github.com/WolframRhodium/muvsfunc/ and https://github.com/fdar0536/Vapoursynth-BalanceBorders/
> Usage: bbmod(c, cTop, cBottom, cLeft, cRight, thresh, blur)
- c: Input clip. The image area "in the middle" does not change during processing. The clip can be any format, which differs from Avisynth's equivalent.
- cTop, cBottom, cLeft, cRight (int, 0-inf): The number of variable pixels on each side.
- thresh (int, 0~128, default 128): Threshold of acceptable changes for local color matching in 8 bit scale. Recommended: 0~16 or 128
- blur (int, 1~inf, default 999): Degree of blur for local color matching. Smaller values give a more accurate color match, larger values give a more accurate picture transfer. Recommended: 1~20 or 999
Notes:
- 1. At default values of thresh = 128 blur = 999: You will get a series of pixels that have been changed only by selecting the color for each row in its entirety, without local The colors of neighboring pixels may be very different in some places, but there will be no change in the nature of the picture. With thresh = 128 and blur = 1 you get almost the same rows of pixels, i.e. The colors between them will coincide completely, but the original pattern will be lost.
- 2. Beware of using a large number of pixels to change in combination with a high level of "thresh", and a small "blur" that can lead to unwanted artifacts "in a clean place". For each function call, try to set as few pixels as possible to change and as low a threshold as possible "thresh" (when using blur 0..16).
BlackBorders: Hide
Avoids dirty lines introduced by AddBorders
Actually avoids dirty lines most of the time, but borders may stay slightly dirty where there are vivid colors
> Usage: BlackBorders(clip, left, right, top, bottom)
- left, right, top, bottom are the thicknesses of black borders in pixels
CropResize: Hide
Crop borders mod 1 while resizing a clip
> Usage: CropResize(clip, width, height, left, right, top, bottom, fill, resizer)
- width and height are the dimensions of the resized cropped clip. If none of them is indicated, no resizing is performed. If only one of them is indicated, the other is deduced.
- left, right, top, bottom are the crop values, odd numbers are possible
- bb is a list containing bbmod values [cLeft, cRight, cTop, cBottom, thresh, blur] where thresh and blur are optional.
- fill is a list containning the number of pixels to fill to extend the borders [left, right, top, bottom]. Functionnality required for CropResizeReader, shouldn't be used otherwise.
- resizer should be Bilinear, Bicubic, Point, Lanczos, Spline16 or Spline36 (default)
CropResizeReader: Hide
CropResize for variable borders by loading crop values from a csv file
Also fill small borders and fix brightness/apply bbmod relatively to the variable border
> Usage: CropResizeReader(clip, csvfile, width, height, row, adj_row, column, adj_column, fill_max, bb, FixUncrop, resizer)
- csvfile is the path to a csv file containing in each row: <startframe> <endframe> <left> <right> <top> <bottom> where left, right, top, bottom are the number of pixels to crop Optionally, the number of pixels to fill can be appended to each line <left> <right> <top> <bottom> in order to reduce the black borders. Filling can be useful in case of small borders to equilibriate the number of pixels between right/top and left/bottom after resizing.
- width and height are the dimensions of the resized clip If none of them is indicated, no resizing is performed. If only one of them is indicated, the other is deduced.
- row, adj_row, column, adj_column are lists of values to use FixBrightnessProtect2, where row/column is relative to the border defined by the cropping
- Borders <=fill_max will be filled instead of creating a black border
- bb is a list containing bbmod values [cLeft, cRight, cTop, cBottom, thresh, blur] where thresh and blur are optional. Mind the order: it is different from usual cTop, cBottom, cLeft, cRight
- FixUncrop is a list of 4 booleans [left right top bottom] False means that FixBrightness/bbmod is only apply where crop>0, True means it is applied on the whole clip
- resizer should be Bilinear, Bicubic, Point, Lanczos, Spline16 or Spline36 (default)
DebandReader: Hide
Read a csv file to apply a f3kdb filter for given strengths and frames
> Usage: DebandReader(clip, csvfile, grain, range)
- csvfile is the path to a csv file containing in each row: <startframe> <endframe> <strength>
- grain is passed as grainy and grainc in the f3kdb filter
- range is passed as range in the f3kdb filter
LumaMaskMerge: Hide
Merges clips using a binary mask defined by a brightness level
> Usage: LumaMaskMerge(clipa, clipb, threshold, invert, scale_inputs)
- threshold is the brightness level. clipb is applied where the brightness is below threshold
- If invert = True, clipb is applied where the brightness is above threshold
- scale_inputs = True scales threshold from 8bits to current bit depth.
RGBMaskMerge: Hide
Merges clips using a binary mask defined by a RGB range
> Usage: RGBMaskMerge(clipa, clipb, Rmin, Rmax, Gmin, Gmax, Bmin, Bmax, scale_inputs)
- clipb is applied where Rmin < R < Rmax and Gmin < G < Gmax and Bmin < B < Bmax
- scale_inputs = True scales Rmin, Rmax, Gmin, Gmax, Bmin, Bmax from 8bits to current bit depth (8, 10 or 16).
Utility functions: Hide
SelectRangeEvery: Hide
> Usage: SelectRangeEvery(clip, every, length, offset)
- select <length> frames every <every> frames, starting at frame <offset>
FrameInfo: Hide
> Usage: FrameInfo(clip, title)
- Print the frame number, the picture type and a title on each frame
DelFrameProp: Hide
Deletes primaries, matrix or transfer frame properties
Avoids "Unrecognized transfer characteristics" or "unrecognized color primaries" associated with Vapoursynth Editor
> Usage: DelFrameProp(clip, primaries, matrix, transfer)
- primaries, matrix, transfer are boolean, True meaning that the property is deleted (default)
InterleaveDir: Hide
Load all mkv files located in a directory and interleave them
> Usage: InterleaveDir(folder, PrintInfo, DelProp, first, repeat)
- folder is the folder path
- PrintInfo = True prints the frame number, picture type and file name on each frame
- DelProp = True means deleting primaries, matrix and transfer characteristics
- first is an optional clip to append in first position of the interleaving list
- repeat = True means that the appended clip is repeated between each loaded clip from the folder
ExtractFramesReader: Hide
Read a csv file to extract ranges of frames
> Usage: ExtractFramesReader(clip, csvfile)
- csvfile is the path to a csv file containing in each row: <startframe> <endframe> the csv file may contain other columns, which will not be read
ReplaceFrames: Hide
Basically a wrapper for std.Trim and std.Splice that recreates the functionality of
AviSynth's ReplaceFramesSimple (http://avisynth.nl/index.php/RemapFrames)
that was part of the plugin RemapFrames by James D. Lin
> Usage: ReplaceFrames(clipa, clipb, mappings="[200 300] [1100 1150] 400 1234")
- This will replace frames 200..300, 1100..1150, 400 and 1234 from clipa with the corresponding frames from clipb.
Overlay: Hide
From havsfunc: https://github.com/HomeOfVapourSynthEvolution/havsfunc
Added plane selection
> Usage: Overlay(clipa, clipb, x, y, mask, planes)mt_lut: Hide
> Usage: mt_lut(clip, expr, planes)
- expr is an infix expression, not like avisynth's mt_lut which takes a postfix one
GetPlane: Hide
> Usage: GetPlane(clip, plane)
- Extracts a plane of a clip
scale: Hide
Scales 8-bits values for different bit depths, from havsfunc
Changelog: Hide
v2.0.1 (20.09.2018)
Fixed:
- InterleaveDir: repeat=False lead to 'Non boolean' error
v2.0 (09.07.2018)
Added:
- FixBrightnessProtect2 to fix all dirty lines at once. Example: FixBrightnessProtect2(clip=source, row=[1, 2], adj_row=[18,-2], column=[3, -3], adj_column=[5,5])
- CropResize to make the whole process of cropping, filling, resizing mod 1 automatically.
- Reader functions to load data from csv files: CropResizeReader for variable borders, DebandReader to apply different strengths for each defined range.
- Some of utility functions for Vapoursynth Editor to simplify screen comparison
Improved:
- BlackBorders, to avoid "drips" of vivid colors
- Support of high bit depths for all functions, even if it hasn't been tested thoroughly. When applicable, functions have a "scale_inputs" argument to scale 8bits values to current bit depth.
v1.0 (23.05.2018)
Initial functions:
- FixColumnBrightness/FixRowBrightness
- FixColumnBrightnessProtect2/FixRowBrightnessProtect2
- bbmod: should work with any bit depth
- BlackBorders, to avoid dirty lines introduced by AddBorders (uses FixBrightness)
- LumaMaskMerge, merges clips using a binary mask defined by a brightness level (to apply debanding on dark areas for example)
- RGBMaskMerge, merges clips using a binary mask defined by a RGB range (to apply debanding on a given range of colors for example)
- Some utility functions required by the previous ones: Overlay, mt_lut, GetPlane (extracts a plane of a clip) and scale (used to scale values for bit depth >< 8)
[6.3 awsmfunc - Successor of sgvsfunc]
Dependencies: Hide
vsutil.py
edi_rpow2.py
rekt.py (NOTE: NOT TO BE CONFUSED WITH THE REKT ON PYPI)
Installation: Hide
Download awsmfunc.py and its dependencies. Place all files in the python packages directory:
[Fedora and Arch-based systems] /usr/lib64/python3.6/site-packages
[macOS] /usr/local/lib/python3.6/site-packages
[Ubuntu] /usr/local/lib/python3.6/dist-packages
[Windows] C:\Users\<user>\AppData\Local\Programs\Python\Python36\Lib\site-packages
[6.4 testencodes - A bash script for x264 test-encoding with VapourSynth (Linux)]
This bash function makes various encodes of a clip, from a vapoursynth script, changing the value of a given x264 setting defined by a range and a step. It basically takes the same arguments as x264.
- All test encodes will be sorted in subfolders according to the tested settings and named <setting>_<value>.mkv.
- Two modes are available: 1. testing mode with -t usually followed by the setting, the range (min and max values separated by an underscore "_") and the step. 2. output mode with -o followed by the filepath to output a single file.
- You can test crf, bitrate, deblock, qcomp, aq-mode, aq-strength, psy-rd, ipratio, pbratio, mbtree.
- aq-mode and mbtree don't need a range nor a step, deblock doesn't need a step.
- I added an ipbratio testing parameter which will test ipratio and keep a distance of 0.1 between ipratio and pbratio.
- No need to manually encode 2 passes. Indicating --bitrate will automatically encode the first and the second pass.
- A log is created for each encode.
- testencodes will detect if a test is already present and skip the re-encoding. This is useful if you refine a setting around a given value. Write -ow to force re-encoding.
- Zones can be imported from a csv file containing <startframe> <endframe> <crf> with --zones f:/path/to/file.csv
- Indicate the source disc with -d to specify the corresponding color matrix and primaries (recommanded for DVD encoding)
Installation: Hide
- Vapoursynth and x264 should be installed.
- Download testencodes.sh, for example in /home/user/apps/
- Define the sh file as executable:
chmod +x testencodes.sh
- In ~/.bashrc add:
alias testencodes='sh /home/user/apps/testencodes.sh'
- Open testencodes.sh and fill in the filepath of the vapoursynth script, the output test folder and change the default x264 settings if needed. The default settings will be used if you don't specify them in command line.
- Test it by executing in a terminal
testencodes
Usage: Hide
Example: Hide
Determining the bitrate to work with (for which there is a small loss of quality which will be compensated with optimized settings):
testencodes -t crf 15_20 1
Testing qcomp from 0.6 to 0.8 by 0.02 with a working bitrate of 7000kbps:
testencodes -t qcomp 0.6_0.8 0.02 --bitrate 7000
Testing aq-mode:
testencodes -t aq-mode --qcomp 0.68 --bitrate 7000
Testing aq-strength from 0.5 to 1.2 by 0.1:
testencodes -t aq-strength 0.5_1.2 0.1 --aq-mode 2 --qcomp 0.68 --bitrate 7000
Testing psy-rdo from 0.85 to 1.15 by 0.05:
testencodes -t psy-rd 0.85_1.15:0 0.05 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68 --bitrate 7000
Testing psy-trellis from 0 to 0.15 by 0.05:
testencodes -t psy-rd 1.05:0_0.15 0.05 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68 --bitrate 7000
Testing final crf from 15 to 20 by 1 after deleting previous crf tests:
testencodes -rm -t crf 15_20 1 --psy-rd 1.05:0.10 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68
Refining final crf by 0.1:
testencodes -t crf 17_18 0.1 --psy-rd 1.05:0.10 --aq-strength 0.8 --aq-mode 2 \
--qcomp 0.68
Changelog: Hide
v1.0 (09.07.2018)
Changelog