Knots' Notes: Checkpoints, VAEs, Adaptions, and Negative Embeddings to use with Stable Diffusion Automatic1111 WebUI
I post my Stable Diffusion gens on Instagram over here: https://www.instagram.com/artificialsweetener.ai/
You are more than welcome to approach me with questions over there! I love helping people get started with SD, which is why I wrote out these notes to share.

  1. General Notes / Preface

This is mostly a list of downloads for tools I use to get good looking anime gens out of Stable Diffusion. I'm assuming you already have Stable Diffusion set up and all that. If you don't, here's a few resources you can look at to help...

Voldy's Guide (4chan's preferred): https://rentry.org/voldy
Aitrepreneur's video: https://www.youtube.com/watch?v=VXEyhM3Djqg
My own guide I posted to Insta: https://www.instagram.com/p/Cq6C1hju01i/

I'm still writing these notes assuming you're a beginner so I'll do my best to to explain things I think you might not know.

When using checkpoints and adaptions (loras/lycoris/locons etc, I will call these all "adaptions" going forward), it's a good idea to check the examples on CivitAI to see what the authors used for settings. Take note of which sampler they used, how many steps they have set, what CFG scale they are using, and what their negative prompt looks like. Checkpoint and adaption devs will often develop around specific negative embeddings, too, so pay close attention to which ones they are using in their negative prompt. I will link the most common ones in this little guide. And, hey, read what the author wrote on their download page, they will often tell you exactly what you need to know.

  1. CHECKPOINTS

FIRST, a word on terminology:
When you see an option to get a "pruned" version of a checkpoint, get that version. The difference between pruned and not pruned should be extremely minimal, it's not going to be "higher quality" just because you're using the bigger unpruned checkpoint.
You will sometimes see an option for a "BakedVAE" version of a checkpoint. I talk about VAEs in the next section. To get the results intended by the checkpoint author, using the "BakedVAE" option when available is probably what you want.

AbyssOrangeMix3 - Pretty good 2.5d/illustrated style Anime Checkpoint. Definitely has NovelAI Anime merged in, and it's likely that many of the anime checkpoints I list do although it's not always possible to tell. You'll find a lot of adaption devs use it in examples so it's a good one to have just for that reason. It's a "classic".
DL https://civitai.com/models/9942/abyssorangemix3-aom3

Anything v4.5 - Another "classic" for anime style. It's even older than AbyssOrangeMix3 (January!). This one is more tuned to a flatter shading style compared to AOM3.
DL https://civitai.com/models/4855/anytroll-v4

AnyOrangeMix - This is a 50/50 merge of the above 2 checkpoints. This is handy because adaptions that are developed with one of those two in mind will probably work with this one, too. I actually tend to prefer AnyOrangeMix to the regular "Anything" checkpoint to get a flatter shading style.
DL https://civitai.com/models/21503/anyorangemix-anything-abyssorangemix

Counterfeit - I think this might be the first anime checkpoint that does NOT have NovelAI Anime merged in (but again, it's kind of impossible to know). The main reason I think so is that it's a lot easier to work with than the 3 I listed above because it does not require "HiresFix" to work well. A lot of my earlier gens use this exclusively and I still like working with it.
DL https://civitai.com/models/4468/counterfeit-v30

SaluteMix - EXTREMELY reliable 2.5d "semi-real" checkpoint. It is VERY good at anatomy compared to other anime style checkpoints and pretty flexible overall. I use this one a lot, maybe even the most out of the ones I use. The examples speak for themselves, it's very easy to get results like in the examples. It also does not require "HiresFix" similar to Counterfeit which is a nice bonus.
DL https://civitai.com/models/19238/salutemix

NotSoXJBMix - This is the one Cubby.ai uses the most. She has had a lot of really great luck with it although I haven't haha. She is able to get very consistently good hands and anatomy with it, maybe you will have similar luck. The color style is nice and vibrant.
DL https://civitai.com/models/28200/notsoxjbmix-1

Aniflat - This one does a very consistently flat shading style. I've found it a little difficult to work with but I still think it's worth trying.
DL https://civitai.com/models/24387/aniflatmix-anime-flat-color-style-mix

xRikishi's Unstable Diffusion - I just recently started using this. Seems fairly easy to work with
DL https://civitai.com/models/9572/xrikishis-unstable-diffusion-alpha-version

  1. VAEs

FIRST, some tips on using VAEs:
Some Checkpoints do not have a VAE baked in. This can cause issues with colors and cause other strange things to happen when you don't have a VAE loaded. If you're new to Stable Diffusion, maybe you have been avoiding learning what a VAE is because it seems like "just one more thing" and you're just happy to be getting pretty pictures at all. Stop that! Your gens are obviously broken, you just don't know enough to see it yet!

While I do provide a little snippet about what each VAE does in my own experience, please understand that VAEs do more than just change the colors of your gen, though this is the most noticeable effect. You may get different details in the final result, for example! Do your own expirimentation when investigating which VAE is best for the gen you're trying to do.

Install VAEs into this folder: "...\stable-diffusion-webui\models\VAE\". Then, for your own sanity, add the VAE drop-down to your quick settings in WebUI by going to Settings > User Interface > Quicksettings List and pasting in "sd_vae," so that the dialog box says "sd_model_checkpoint, sd_vae," if you haven't added anything else already. Hit apply and reload the UI - now you have a VAE drop down right next to the one you use to select your checkpoint. Nice.

OrangeMix VAE - Decent baseline VAE. Colors look good. No complaints. I often just leave this set and forget about it. This one is often recommended by Checkpoint devs.
DL https://huggingface.co/WarriorMama777/OrangeMixs/blob/main/VAEs/orangemix.vae.pt

kl-f8-anime2 VAE - Very widely used. Works just fine.
DL https://civitai.com/models/23906/kl-f8-anime2-vae

MSE-840000 Ema Pruned - Very widely used VAE for anime gens. This is one seems to get recommended a lot by Checkpoint devs.
DL https://huggingface.co/stabilityai/sd-vae-ft-mse-original/blob/main/vae-ft-mse-840000-ema-pruned.ckpt

ClearVAE - This is the VAE I often switch to first when I'm not getting what I want out of OrangeMix VAE. Colors are usually a little brighter in my experience.
DL https://civitai.com/models/22354/clearvae

Pastel Waifu Diffusion VAE - In my experience, colors are (not surprisingly) shifted more into pastels.
DL https://huggingface.co/andite/pastel-mix/resolve/main/pastel-waifu-diffusion.vae.pt

In general, read the description on the Checkpoint download page to see what VAE the author recommends, if any. Like I mentioned before, sometimes the checkpoint will come in a "BakedVAE" format. In that case, best to set your VAE drop-down to "automatic" if you want to make sure you're using only the VAE that's baked into the checkpoint.

  1. ADAPTIONS (loras, lycoris, locon, etc)

To use "advanced" adaptions (the ones that are not called "lora"), you need this extension for Automatic1111 WebUI: https://github.com/KohakuBlueleaf/a1111-sd-webui-lycoris

LowRA - Because of the way that Stable Diffusion works when generating noise patterns, it's impossible to get "low light" scenes normally. LowRA can be used to fix this shortcoming so you can generate dark/night time scenes. Read the author's notes to get started with using it.
DL https://civitai.com/models/48139/lowra

Anime Extract - This is a way to get a 2.5d style checkpoint to skew more towards flat shading. It is especially effective when used in conjunction with SaluteMix, which is great because I love working with SaluteMix.
DL https://civitai.com/models/24796/anime-extract

Esthetic 90s style - Really love the results I get with this. Some of my favorite gens are using it. You need to pay very close attention to the recommendations given by the author on the CivitAI page to get good results! It can produce a fairly wide range of 1990s styles, from a more flat "animation cell" style to a more detailed "promotional"/"tankobon cover" illustration style.
DL https://civitai.com/models/28600/esthetic-90s-style

Make it Chibi - Great adaption that produces chibi characters. Easy to use. Quality results.
DL https://civitai.com/models/47953/make-it-chibi

CocoPPa Play Style - I've never played the game but the art style is adorable. It's a kind of vibrant chibi style. I've gotten lots of interesting results with it.
DL https://civitai.com/models/49786/cocoppa-playstar-girl-fashion-style-lora

Soejima Shigenori Style - Style based on the artist who does character artwork for Atlus video games like Persona, Shin Megami Tensei, and Catherine. I've always been a big fan of the artist.
DL https://civitai.com/models/57936/soejima-shigenori-atlus-artor-personaor-catherine

Angel's Egg Style - Reproduces an oldschool 1980s art house anime style. The movie it's based on is considered one of the most beautiful examples of retro anime and it's not hard to see why.
DL https://civitai.com/models/53348/angels-egg-lora

Author: TNP119 - This guy does amazing anime character style adaptions. He takes requests, so there is a lot of variety in the characters he's created adaptions for. Considering the amount of work he's done and how consistently high quality it is, I consider his work to be the gold standard. People should look at what he's doing and try to emulate it when making adaptions for characters.
Profile: https://civitai.com/user/TNP119/models

Author: Rindo192 - This person consistently creates very high quality adaptions. As of writing, they only have Naruto characters, but the training they've done for them is extremely good.
Profile: https://civitai.com/user/Rindo192/models

Author: cubby_ai - Bit of shameless plug. Cubby trained these using my computer and I watched her put a lot of work into getting things "just right". She's still working on making them better but her collection of Madoka characters is already quite good.
Profile: https://civitai.com/user/cubby_ai/models

  1. NEGATIVE EMBEDDINGS ("textual inversions" for the negative prompt)

FIRST, a word on Negative Embeddings, negative prompts, and prompting in general...
A lot of people go way crazy overboard on using negative embeddings. It might seem like a good idea to load all of these up at once and use all of them for every single gen. That is not how this works. Please don't do that, for your own sake. Always start by using the negative embeddings that the author of the checkpoint uses in their examples when possible.

The best negative embedding is NO negative embedding and the best negative prompt is NO negative prompt! If you're not sure where to start and you have no solid examples to go off of, start with a simple negative prompt that looks like this:

(worst quality, low quality, extra digits)

If that's not working well, before trying an embedding, up the weight like this...

(worst quality, low quality, extra digits:1.2)

and so on, up to even 1.4. You will be surprised how much this will impact your gens using anime style checkpoints without any other negative prompting. For the sake of completeness, I will also add that a positive prompt of "(masterpiece, best quality)," will often go really well with this, and you can try putting the weight of that positive prompt up to 1.2. No need to add silly things like "ArtStation award winner" or whatever to your prompt.

EasyNegative - This is the most common negative embedding. It needs a CFG scale of 11 or higher to work well. I consider this a drawback, as higher CFG scales are often undesirable when working with anime styles. Still, the results can be quite good, and a lot of checkpoint and lora authors use it in their examples so you'll be loading it in a lot.
DL https://civitai.com/models/7808/easynegative

BadHandv4 - This is another very commonly used one. Like EasyNegative, it requires a CFG scale of 11 or higher to work well. Try bouncing between this and Bad-Hands-5 if you're having trouble getting good looking hands.
DL https://civitai.com/models/16993/badhandv4-animeillustdiffusion

Bad-Hands-5 - This is another very commonly used one. Like EasyNegative, it requires a CFG scale of 11 or higher to work well. Try bouncing between this and BadHandsv4 if you're having trouble getting good looking hands.
DL https://huggingface.co/yesyeahvh/bad-hands-5/tree/main

bad-artist & bad-artist-anime - These ones work at lower CFGs. I often see people loading these together so you can try that. You should try weighting these lower than 1 to start. Try loading them in like this: "by (bad-artist:0.8), by (bad-artist-anime:0.8),".
DL https://huggingface.co/nick-x-hacker/bad-artist/tree/main

bad_prompt & bad_prompt_version2 - I see these used a lot so I'm including them but I don't tend to use them myself when left to my own devices. Load them seperately, I don't think it's adivsable to use them together. I'm fairly certain these work OK at lower CFGs.
DL https://huggingface.co/datasets/Nerfgun3/bad_prompt/tree/main

bad-image-v2-39000 - This is a less common one, but I actually like it fine. It works at CFG 7 from my expiriments. I often see it in examples loaded along side bad-artist and bad-artist-anime. Results seem to be good when doing this, though again I would caution against loading up on lots of negative embeddings if you can avoid it!
DL https://huggingface.co/Xynon/models/blob/main/experimentals/TI/bad-image-v2-39000.pt

ng_deepnegative_v1_75t - Another less common one. Seems to require CFG 11 like EasyNegative. I still see it pop up in examples often enough that I feel it should be an automatic download so you have it when you need it.
DL https://civitai.com/models/4629/deep-negative-v1x

  1. FURTHER READING

You probably didn't read the Automatic1111 features list/explainer yet, but you should. You really, seriously should. I see really crazy bad prompts all the time that blatantly ignore the rules around syntax explained in the docs.
Read it: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features

This reddit thread also has some great tips on prompting syntax. Please no more of (((((this))))) in your prompts.
Thread: https://www.reddit.com/r/StableDiffusion/comments/10ollq7/how_does_prompt_grammar_work/

If you are interested in creating an adaption yourself, I recommend holostrawberry's guide on the topic. This is also the guide TNP119 recommends.
Guide: https://civitai.com/models/22530

You might also appreciate UnavailableUsername's guide on training loras; it's done in a comic style which some people may find easier to read.
Guide: https://imgur.com/a/mrTteIt

It should go without saying since I link it a ton in this guide, but the best place to find checkpoints and adaptions is CivitAI.
Link: https://civitai.com/

I hope all this information was helpful! Good luck with your gens!!

Edit Report
Pub: 05 May 2023 17:21 UTC
Edit: 25 May 2023 01:27 UTC
Views: 320