nAV!gator logo

The Hitchhiker's Guide to becoming an AV1 Encoder

Hello, I am nAV1gator. You can find my work here. I encode AV1 videos for fun on 1337x, & I'd like to help others do the same. It seems like the current strategy to get into this niche hobby is just to talk to the right people, but I believe it can be made easier by setting the stage before you become involved. Before I get started, I want to outline a couple things:

  • I'm newer to the scene myself, & much of this advice is based on my subjective experience getting started; the technical side is more objective. It is important to keep this in mind, as it is perfectly fine to do things your own way sometimes.
  • Sometimes you make mistakes. I've certainly made a few; I have an uncropped release & a release with two scenes containing grey frames. It is important to know how to receive constructive criticism & learn to move forward.
  • I'll be writing from the perspective of a Linux user. Getting started on your own is (in my opinion) much easier on Linux, and I use Arch Linux where most things are very close to their newest version (bleeding edge). If you use Windows for other things, it is easy to dual boot & you can learn how to do that in many other places that aren't here
  • I assume the fact that you are reading this means you already understand that AV1 is generally more efficient at low to medium-high fidelity than h265 (HEVC) & h264 (AVC).

Torrenting

The best way to distribute material you've encoded is through torrenting. Torrenting is peer to peer file sharing that utilizes P2P traffic in order to allow many people who have a file to share it with others. In order to familiarize yourself with torrenting, I'd recommend trying to understand how it works as a consumer of torrented content. You can do this safely & legally by torrenting some Linux ISOs from places like the Fedora Project's torrent server. Here are some key terms to remember with torrenting:

  • Public Trackers are servers that connect people to allow them to torrent. Sites like 1337x.to, The Pirate Bay, & Torrent Galaxy are public trackers.
  • Private Trackers are servers that require registration or an invite to access their content via connecting with others on these private trackers. Examples include TorrentLeech, AnimeZ, & Reelflix. Private trackers often have rules surrounding how torrenting must be performed on their sites.
  • Peers are people connected to each other in a "swarm," sharing a file.
  • Seeders are people who have most or all of a file & are able to distribute it to others. They are uploading the content they torrented (or created), not downloading. It is highly recommended to continue to seed your torrents once they've completed (more on this later).
  • Leechers are people who are downloading the file from seeders in a swarm. If you start a torrent & you don't have any of the file, you are going to be a leecher while you are downloading this file.

Considering you're reading this, I assume you know this all already but it's always good to have a refresher. You can torrent with a client like Transmission or qBittorrent, which should be available in your package manager on your distro of choice. If not, you can get them as Flatpak packages. See the Quick Setup guide on how to get started with Flatpak on your distro.

As an encoder, you'll be expected to seed the files you create for a very long time to keep your torrents from dying. It is recommended to get a seedbox (a remote server that is used for torrenting) or to make sure you have a device that can seed for extended periods of time on a preferably high bandwidth internet connection.

Before accessing torrents (especially of copyrighted material) it is paramount to get a VPN. AirVPN is a newer option that is recommended for torrenting & looks like it is priced competitively. ProtonVPN is also great, although more manual setup is required to get port forwarding working on Linux. It is also highly important to use a browser with adblocking & tracking protection (I recommend Firefox, Mercury, or Waterfox - all with the uBlock Origin extension) to allow these sites to be usable.

Uploading Torrents

Where to Upload

The most popular place to upload AV1 live action is 1337x, with Torrent Galaxy at a close second. For anime, see nyaa.si.

Uploading a torrent is easier said than done. For a while 1337x moderators were rejecting people from the role of trial uploader or not responding to applications, and many fall victim to not reading the rules. 1337x has rules posted here for those who wish to apply, with an application submission form here that must be used when requesting trial uploader status. If you are able to upload 100 high quality torrents & the mods are satisfied with your work, you may be promoted to full uploader status which may grant you some extra benefits. As AV1 encoding is usually a tedious process, we cannot crank out encodes at a quick enough rate to rapidly achieve this individually, but slow & steady wins the race.

Torrent Galaxy (TGx) applications work slightly differently. There is a forum on the site where you are able to to make your case to staff who can then accept you. It is also important to read their rules for uploading. I don't upload there, so I'm not going to provide more details until I know more (stay tuned) but from what I've heard they are staffed by nice people who are receptive to feedback.

I believe Nyaa lets you upload without an account, although I've never tried. They have rules as well. Account registration is temporarily unavailable at the time of writing.

Formatting Titles

It is important to have consistent formatting when uploading torrents. I format my titles as shown below:
Movie Name (Year) Source Resolution AV1 AudioCodec [nAV1gator]

See my uploads for more examples. I try to keep it consistent. Filenames are formatted in a similar manner:
Movie.Name.Year.Source.Resolution.AV1.AudioCodec-nAV1gator

This is my formula because I like it. On anime sites, it is common to put your tag at the beginning. I think if you are consistent with your formatting & always provide all relevant information (eg HDR, MiNi, etc) you should be all set.

Formatting Descriptions

Here is, generally, how I format descriptions:

[b]Title[/b]: Movie Name (Year)
[b]Genres[/b]: Genre, Genre
[b]Description[/b]: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Screenshots previews are recompressed & are not representative of the video's full quality; actual video quality is higher. Please click on a screenshot to view a full quality lossless version.

If you cannot view the first image, consider using a browser that supports the JPEG XL image format.

[url=https://lossless.webp][img]https://lossy.jxl[/img][/url]

[url=https://lossless.webp][img]https://lossy.avif[/img][/url]

[url=https://lossless.webp][img]https://lossy.avif[/img][/url]

[url=https://lossless.webp][img]https://lossy.avif[/img][/url]

[url=https://lossless.webp][img]https://lossy.avif[/img][/url]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[b]Filesize[/b]: X.Xgb (from my disk, usually differs from size reported by 1337x because of the way storage is calculated)

[b]Tools[/b]: videoencoder, av1an, FFmpeg, audioencoder, MKVToolNix

[b]Encoding parameters[/b]: 
[code]full av1an parameters, needs to be able to be copied, pasted, & run by a third party with relevant dependencies without issue. This is to encourage experimentation & make my parameters as accessible as possible so they may be scrutinized & improved[/code]

[b]Video[/b] (Disclose if downscaled from higher res source): Codec | widthxheight - bitrate kb/s [bit depth]bit - fps ration (approximation) fps
Subtitle(s): Language, Language (SDH, if applicable), Language 2, Language 3

[b]Audio[/b]: Codec bitrate kb/s 5.1/7.1 Surround or Stereo
Language: Language

[i]Encoder's Notes[/i]: Something something video codecs probably

Some of my older encodes were formatted differently, but this is what I'm using at the time of writing. You may notice I host a compressed image as a preview & a lossless WebP as a fallback; this was suggested by fellow encoder Maxoverpower to allow images to load quickly & not waste bandwidth on sensitive or slow connections with the option to click them to view large lossless images for scrutinizing quality. A couple of my HDR encodes have real HDR PNG images with HDR metadata as the lossless alternates to the lossy previews, which may require a Chromium-based browser like Thorium to view properly.

Check out this script I wrote if you'd like to have an easy way to encode to every relevant image format. Given you're on Linux, simply run ./image_batch.sh input.png & you'll have output images encoded with jpegli, cjxl, avifenc, cwebp (lossy), & mozjpeg in an output directory. If your system doesn't support a particular format, the script will still work fine & the unsupported format will just be skipped.

I take my AVIF screenshots with -j 2 & -s 0, but here's an AVIF script that should encode with reasonable efficiency if you have a CPU with 8 threads or more:
avifenc -j 8 -d 10 -y 444 -s 4 --min 1 --max 63 --minalpha 0 --maxalpha 0 -a end-usage=q -a cq-level=13 -a tune=ssim input.png out.avif

Here's how I take my lossless WebP screenshots. The higher the -z level, the longer it will take in exchange for a small size reduction. The maximum is 9:
cwebp -mt -lossless -z 9 -alpha_filter best -metadata icc input.png -o output.webp

If you're interested in fighting the good fight for JPEG-XL & you have libjxl, here's how to encode one of those images:
cjxl input.png out.jxl -d 1.0 -e 9 -p
They won't be viewable on most browsers, but JXL images are really the only way to get small, lossy, high-fidelity images, which are perfect for our purposes. -d indicates target butteraugli distance (visual quality) with -d 0.0 being mathematically lossless, -d 1.0 attempting to be visually lossless, & higher values losing more quality in exchange for smaller filesize. All of my JPEG-XL images are -d 1.0. Raising the -e (effort) level does the same thing as raising -z in cwebp; more efficient compression in exchange for time. Maximum cjxl effort is 9.

It is important to check which image hosts are whitelisted where you are uploading. 1337x has a helpful image host list that allows you to look at their approved image hosts, but they don't necessarily disprove of image hosts that aren't on the whitelist or blacklist in my experience.

Scene Terms

It is important to know some scene terms unique to the torrenting community when uploading. These are also relevent when choosing a source, which I'll cover later:

  • WEBrip: Reencoded from a WEB release, like something from Netflix or HBO Max. The video has been lossily reencoded.
  • WEB-DL: A WEB release where the video is directly from the WEB platform. The video hasn't been reencoded from the source. People often mislabel WEBrip releases as WEB-DL. This term is allegedly being replaced by plain old WEB, which is a bit confusing.
  • x265: The video has been reencoded with the x265 video encoder. Don't confuse this with h265 which is the HEVC codec itself, not an encoder.
  • x264: The video has been reencoded with the x264 video encoder. Don't confuse this with h264 which is the AVC codec itself, not an encoder.
  • BluRay: The source medium was a BluRay.
  • REMUX: Most commonly refers to a media ripped directly from source without reencoding.
  • HYBRiD: Contains media from a variety of sources. For example, may have BluRay video with WEB audio tracks or subtitles. Useful when looking for a source.
  • BCORE: The source came from the BRAVIA CORE streaming service, which can offer higher quality video than BluRay.
  • NF: Netflix. An NF WEB-DL is web video coming from Netflix.
  • CAM: Someone recorded it with a camera. These usually look like crap.
  • REPACK: A prior release had errors that were fixed. Can also be denoted by PROPER or V2.
  • HDTS: High Definition Telesync. Can be a direct capture of a projector output via a camera, or have an external audio track synced up with a video track. Also not great looking, & not really up to par with my standards for source quality.
  • DV: Dolby Vision HDR. Not very fun to work with.
    For our purposes, the only thing we really need to worry about is what the source material is. As you can see, most of the title's other variables are just given to you without much thinking:
    Movie Name (Year) ????? 1080p AV1 Opus [nAV1gator] (given we encoded our audio with Opus, which is a standard pairing with AV1 video)

It is highly recommended to reencode source material that is in its highest quality form, like a BluRay that hasn't been reencoded by a third party.

Source Material

Picking a Source

Now that you know how format the encodes you release, it's time to find good source material. Here's a quick quiz: which one of the following would be best to use as a source?

a. Harry Potter And The Order Of The Phoenix 2007 Hybrid 1080p BluRay DDP 7 1 DV x265
b. Harry Potter and the Order of the Phoenix 2007 1080p WEB-DL H 264 DD5 1
c. Harry Potter and the Order of the Phoenix 2007 2160p UHD BluRay DV HDR10 DDP 7 1 x265
d. Harry Potter and the Order of the Phoenix (2007) [2160p REMUX] [HEVC DV HYBRID HDR10 DTS-X MA 7 1 24-bit Audio English

This is a bit of a trick question, & we don't know the sizes for the sake of this comparison, but that's intentional - this is more an exercise of decoding the meaning of the names here.

  • Lets start with (a); it is a hybrid release meaning it contains media from multiple sources, so it'd be important to check the description (NFO) in this instance. It is also Dolby Vision HDR video. However, it has been reencoded with x265 - regardless of how big this release is, the fact that it has been reencoded means there could potentially be artifacts present. We'd be reencoding a reencode, and even if we plan for our release to be significantly smaller, it would be wise to correspond with other encoders to see if we could access a release that hasn't been reencoded.
  • Next, with (b), we have a WEB release that hasn't been reencoded & is 1080p h264. While it is good that this hasn't been reencoded, WEB releases aren't known for their quality relative to BluRays so we'd potentially want to aim higher unless we intentionally want to do a WEBrip release (eg when there are no other sources available).
  • With (c) we have a BluRay release that has been reencoded, & looks to have HDR10-compatible DV HDR video.
  • (d) hasn't been reencoded, & it a HYBRiD release, meaning it'd be important to check the NFO. It contains very high quality DTS-X audio as opposed to the Dolby Digital Plus (DDP) audio present in other sources. Once again, it looks to contain DV HDR10-compatible video.

This is a tough scenario, but generally you want to stick with the highest quality source that preferably hasn't been reencoded. In this case, I'd go with (d). If I wanted to make a 1080p SDR release, I'd spend a bit more time looking for a 1080p BluRay REMUX that hasn't been reencoded, as it makes encoding easier.

Finding a Source

The best places to find high quality sources are private trackers. Some of the best options now are Torrent Leech (TL) & ReelFlix. TL usually requires receiving an invite, but it is relatively easy to get started using ReelFlix as registration opens relatively frequently. It is the home of the reputable group LEGi0N & is great for finding high-quality source material that is often not reencoded. I would recommend joining the AZwaitlist Discord server & keeping an eye on the Open Signups subreddit. If you're looking for an invite to TL, try to make a name for yourself as a reputable & dedicated encoder before looking for an invite.

It is important to pay attention to the rules on private trackers to avoid being kicked off. TL has a seeding requirement of 10 days for every torrent that you download from them, & you can keep track of your seeding progress on TL. ReelFlix requires a seed ratio of 1.0 right now, but that is subject to change in the future. Torrents labelled "Freeleech" don't count against your upload/download ratio, but still need to be seeded for a while. It is important to thoroughly read private tracker rules surrounding required overall ratio, seed time, etc once you join.

Lossless Intermediate File

Certain BluRay sources will output grey frame errors when you encode them directly through av1an. Instead of simply allowing this to happen, it is usually wise to create a lossless intermediary file with libx264 -qp 0 in ffmpeg. U can also discard the audio here with -an & add it back later with mkvtoolnix.

Here's how I usually do that:
ffmpeg -i input.mkv -c:v libx264 -preset ultrafast -qp 0 -an lossless.mkv

Preset ultrafast is more important than one might realize, as it disables costly H.264 features that make lossless H.264 more difficult to decode. With other presets, you are likely going to be wasting CPU cycles while you are decoding the complex lossless input stream as you are encoding AV1.

Encoding

Basically everything is covered in this blog post by community member Simulping on the Codec Wiki.

Here's the TL;DR on the various AV1 encoders:

  • SVT-AV1 is fast & parallel, but its benefits are largely negated by using av1an to parallelize instances of other workers. It isn't recommended to use SVT-AV1.
  • rav1e is Rust-based, & won't ever segfault. It is on you if it crashes. I love rav1e because its development is more community-oriented, & it is really hard to screw up using rav1e; everything is tuned for visual quality by default. Use speed 3 or 2.
  • aomenc is the reference implementation for an AV1 encoder by the Alliance for Open Media. The community maintains a fork called:
  • aom-av1-lavish, derived from av1-aom-psy. both are tuned for optimal visual quality, although further tuning (parameter spam) is required to reach optimal coding efficiency.
  • SVT-AV1-PSY is a community fork of SVT-AV1. It retains SVT-AV1's speed while introducing valuable psychovisual benefits. While it isn't as efficient as aom-av1-lavish, it is much faster for very little efficiency loss, and the defaults are very sane so you do not need to cargo cult parameter spam.

My Recommendation: SVT-AV1-PSY

It is important to always encode 10 bit video, even if you have an 8 bit source. It improves coding efficiency slightly, and it is worth it. You can check my 1337x account for the full scripts I use to encode my own videos, which are always disclosed unless I'm releasing for someone else. I still highly recommend reading Simulping's article to properly understand how to get these tools working properly, as the parameter meta can change daily based on community research & testing (this is more relevant for aom-av1-lavish, SVT-AV1-PSY is relatively stable and defaults seem to be a big consideration).

I know it is tempting, but please don't use hardware encoders to encode & release AV1 content. QSV & NVENC both support AV1 output for Nvidia's 40 series GPUs & Intel Arc GPUs, & AMF kind of does with RDNA3 GPUs. They produce poor quality video that is more on par with SVT-AV1 speed 9 or 10. For reference, we usually use speed 2-4 with more efficient encoders. Hardware accelerated encoding is lazy, try to make your releases high effort to give yourself a good name amongst those who care.

Community

The best thing about AV1 is community. You can check out some community links here:

There are of course other servers related to compression, but the AV1 Community & AV1 Weeb servers are the friendliest toward ... uhh ... our hobby, lol. Anyhow, happy encoding! I'll be sure to add to this as time goes on with more helpful information!

Last Edit: 14 Mar 2024 (SVT-AV1-PSY + updated some links)

Edit
Pub: 12 Apr 2023 06:58 UTC
Edit: 14 Mar 2024 23:25 UTC
Views: 490