Chub Discords LLM Rankings

A ranking of LLMs as per the Chub Discords personal tests.

Update 09-04-2024

The original mission of this Rentry was to help our fellow Local chads find new models to roleplay with as well as rank some site models. The kinds of things you don't have to pay top dollar to use or risk getting banned from a Corpo Model to use. As time goes on, this becomes harder to do. Models are released at a pretty breakneck rate and testing all of them becomes impossible: Especially for just a small guy with other things to do, like me - Statuo.

I don't want this to go unused, but we're also in a strange transition state of models currently:

Gemma-2
Llama 3.0 and 3.1
Mistral Nemo

All of these are different series of model contenders that are prominent in the local scene.

So let me break this down while I think about how we're going to be running this rentry in the future:

LLama 3.0 and Gemma 8/9b models are still solid but have the unfortunate problem of being limited to 8k context size. This means they're going to fall behind and people will make less models for them. The larger models don't suffer from this problem (Gemma 2 27b and Llama 3 70b respectively.)

Meanwhile, Mistral Nemo and Llama 3.1 have released claiming absurd context sizes. This really only actually means 16k for Mistral Nemo and 32k for Llama 3.1. Llama 3.1 is currently proving extremely resilient to any good merges, moreso than Mistral Nemo which has problems of its own.

However, in my humble opinion, Mistral Nemo is currently the front runner for the roleplaying space. The base model is good enough for most things, uncensored, and has great dialogue. While Llama 3.1 is censored, is extremely resistant to finetuning attempts, and otherwise not that great. Llama 3.1's only advantage over Mistral Nemo is the 32k context window it can theoretically use reliably.

So, with that all being said, please understand why I'm not recommending any Llama 3.1 models currently. You can find the models I'm testing and trying for roleplaying usually on my HuggingFace page, as I quant them for EXL2

My current recommendations for models you should use - because I'm using them are the following:

Mistral Nemo - Stardust V1 - by Luni
Mistral Nemo - Base Instruct
Llama 3.0 - Stheno v3.2 by Sao10k

What has been recommended by chatter in the Chub Discord by other people who are using them:

Mistral Nemo 21b - Theia by Drummer
Mistral Nemo 12b - Rocinante by Drummer
Mistral Nemo 12b - Atlantis v0.1 by Invisietch

Hope this helps those of you looking for osmething new. Again, check my Huggingface page for the things I'm currently looking at until I figure out wtf I wanna do with this rentry.

Overview

This Rentry was brought about as a way for us to recommend LLMs to others in the Chub Discord. It's often difficult to tell what is a good model and what is a great model at a glance, especially when it comes to the Chub niche of roleplaying. So specifically this Rentry is meant to standardize a way to test the various aspects of roleplaying that the average roleplayer would be interested in.

Of course, as with everything when it comes to preference: This is all subjective. You're encouraged to try out models for yourself.

Testing Methods

Thorough in-depth testing of hundreds of messages per models is impractical for everyone to do consistently. on top of that, this is all just volunteer work so expecting our Chub Discord folk to rank every model out there is wildly impractical. That being said, we can consider the following traits to be important to a roleplaying model:

  • Erotic/NSFW Content
  • Character Inference - Ability to stick to Character Quirks/Definitions
  • Writing/Roleplaying Quality

What these mean exactly is covered below. But it's important to note that testing was performed on quants of at least: Q4_K_M GGUF, 4bit 128g GPTQ, or 4.0bpw ExlV2.

Testers should also expect to use the following:

  • Instruct Mode
  • A System Prompt that does not obstruct the above traits
  • Their own generation settings, whatever that may be.

Erotic/NSFW Content

Most roleplayers prefer their models to have the capability for NSFW content, even if they don't intend for it to be used in an erotic context. Since models that struggle with erotic content tend to also struggle when it comes to content like fight scenes, drama, or otherwise, it's considered important that a model be capable of Erotic/NSFW content.

To that end, I've created two quick cards for erotic testing: Amy and Alex. These are quick cards meant to be purely used for erotic roleplay and thus don't have a lot of substance to them. In theory, a model that performs well with erotic content can therefore also handle NSFW content very well. Of course, people are free to use their own cards for this stuff.

The things we're testing for are going to be:

  • Willingness to initiate erotic content when prompted.
    • In example: If the bot is dressing down then it makes sense that their next step is to then suggest or do something erotic. Does it actually do that or does it always do the "Are you sure?", "What do you want to do now?", etc.
  • Capability in writing erotic content
    • Sure the bot might write some stuff, but how good is the writing? Does it completely fail at X fetish?

Character Inference - Ability to stick to Character Quirks/Definitions

When you download a character card that's a tsundere, you expect some back and forth while the tsundere acts as a tsundere, and not a love-sick puppy. This is what we refer to when we mention a models ability to stick to a character cards definitions. Are yanderes sufficiently yandere? Did the bully act like a bully?

There's a lot of cards out there for testing this kind of thing and the model tester should specify what card they used to test their model. But as a baseline for what makes a card good for testing:

  • Distinctive Personality
    • Speech quirks, such as not saying a lot of words due to being shy or cursing a lot are great for this. But also things like a tsundere or bully being a tsundere or a bully.
  • Defined backstory or scenario
    • A defined backstory or scenario in a card can be used to guide roleplay and allows cards to say more things that aren't just in relation to what is currently happening.
  • Skills
    • If I ask a character who is good at 'cooking' to cook something, do they agree to do so immediately and show off some kind of skill with it?

Writing/Roleplaying Quality

Does the model manage to separate characters well? If I add in an NPC does the model differentiate between {{user}} and the NPC or do they get confused on who is talking? Does it frequently forget words such as 'and/the/it'? Is it capable of driving story progression by adding twists or NPCs of its own?

Does the model talk a lot for {{user}}? Some models just baselines write as the user a lot and need to be beaten into submission to stop. Is the model bad at describing scenes or actions taken? These are all things under the umbrella of 'writing/roleplaying quality.' Once more, testers should consider if their chosen character card is relevant to this particular category. I personally recommend a character card with a defined scenario. If you can't think of anything: Alder the Paladin and Alice the Elder Vampire are my recommendations on these fronts. Alder has a defined scenario for a necromancer hunt while Alice is an elder vampire that puts you right into conflict with her.

Things you should be considerate of while testing are:

  • Spelling/Gramamatical Errors. Missing words. Word Salad.
    • Word Salad being when the model outputs sentences that don't make sense either because they're random assortments of words that are barely related or looks like someone rolled their face on the keyboard.
  • Quality of Descriptions.
    • Is the bot capable of writing out good quality descriptions. Does it set the mood appropriately? Does it mention facets of the character well?
  • Frequency of Speaking for {{user}} / AI Bleedthrough
  • Cohesion
    • Does the model make logical conclusions? If I'm in the next room does it talk like I'm in the same room as the bot or does it accurately know to either speak louder or move into the next room before speaking?
  • Initiative
    • Does the model take initiative to add to the roleplay? New NPCs, events, etc that were not prompted by the user.

Model Review Template

Scores are from 1-10 with 1 being terrible and 10 being perfection.

Reviewer Name:
Card Tested:
Erotic/NSFW Content Score:
Character Inference Score:
Roleplaying Quality Score:
Overall Score:
Favorite Part of testing the Model:
Worst Part of testing the Model:
Roleplaying Sample (If Applicable):
Reviewer Thoughts:

The Ratings

Rank Models Testers Average Score
1 Kunoichi-DPO-v2-7b StatuoTW 9
1 Lemonade-4.5.3-7b StatuoTW 9
1 NovelAI Kayra, Opus tier Blue 9
1 Midnight Miqu v1.5 Wilson 9
2 Mars Mixtral StatuoTW, Blue 8.5
3 Fimbulvetr-v2-11b StatuoTW 8
3 Maid Yuzu V8 Alter Wilson 8
4 Mars Asha StatuoTW 7
4 Mercury MythoMax StatuoTW 7
4 Psyfighter-2 StatuoTW 7
4 Typhon-Mixtral StatuoTW 7
5 Mercury Mistral StatuoTW 6
5 SnowyRP-13b StatuoTW 6

Rating Explanation

1-3: Terrible, not worth your time in the slightest.
4-6: Average. Might be your secret hidden gem of a model, might not.
7-9: Good/Great. Excels in the areas you want it to or is good enough to be used without extensive handholding.
10: Perfection. Hits all the right spots you would want from your chatbots.

Full Reviews

Fimbulvetr-11b-v2

Huggingface Link

Statuo's Review

Reviewer Name: StatuoTW
Card Tested: Amy / Sophie
Erotic/NSFW Content Score: 7
Character Inference Score: 9
Roleplaying Quality Score: 7
Overall Score: 8
Favorite Part of testing the Model: Fimbulvetr continues the tradition of being able to infer the traits present in cards in an excellent manner, which is important for roleplaying purposes. Characters are appropriately yandere and use abilities that they're listed as having. This model actually managed to correctly infer Crypto's ability summons extra eyes on her body and used it in roleplay, which was a pleasant surprise since most models don't do that or even touch it.
Worst Part of testing the Model: Fimbulvetr had problems with grammar and punctuation. At first I thought I had made some kind of mistake (The version I had been testing initially was the Test version of v2) but no, these issues are present in both versions even if you start brand new chats. It kept calling me Statuto and putting random, strange punctuations like [Maybe] to add emphasis to certain words. It was strange and I can't say for sure why it was doing that because it started doing it without any seeming rhyme or reason.
Roleplaying Sample (If Applicable): Roleplaying Sample with Crypto || Roleplaying Sample with Sophie
Reviewer Thoughts: I'm coming in after the fact and some more testing with Fimbulvetr to update my score on it. It's still not my favorite model, but it's clearly a great model overall. It's creative, it sticks to the character card, and it just kinda works for serious roleplaying. My only problem is that it seems to absolutely hate my name and constantly refers to me as Statuto for no significant reason. It will sometimes say "Statuo" and "Statuto" in the same message, requiring a very careful eye and manual editing to fix it when it crops up. But otherwise, Fimbulvetr 11b v2 is a solid model option for mid-tier models. I was dipping to this whenever my Llama-3 model testing was frustrating me and it really helped me see Fimbulvetr in a new light.

Kunoichi-DPO-v2-7b

HuggingFace Link

Statuo's Review

Reviewer Name: StatuoTW
Card Tested: Amy / Alice the Elder Vampire
Erotic/NSFW Content Score: 7
Character Inference Score: 9
Roleplaying Quality Score: 9
Overall Score: 9
Favorite Part of testing the Model: Kunoichi's capability to infer character details is top notch. My Magical girl cards in particular were done well with this model, with the bots performing 'signature attacks' and sticking to skills that they have along with their own unique weapons instead of pulling other random weapons out of nowhere.
Worst Part of testing the Model: Kunoichi just talks for the user a lot. Way too much. But at the same time the other parts of the model outweight other aspects of it enough that the positives outweigh the negatives. Also: Ministrations. I'm beginning to see why ChatGPT folk complain about 'ministrations' so much because this model uses that word a lot for erotic stuff.
Roleplaying Sample (If Applicable): Roleplaying Sample with Alice
Reviewer Thoughts: One of the top 7b models for roleplaying. The fact that it was natively tested at 8k context means people can get the full 8k context without worrying about the model degrading as well, which is amazing for those of us who care about that kind of thing.

LemonadeRP-4.5.3-7b

Huggingface Link

Statuo's Review

Reviewer Name: StatuoTW
Card Tested: Amy / Terryn
Erotic/NSFW Content Score: 8
Character Inference Score: 9
Roleplaying Quality Score: 9
Overall Score: 9
Favorite Part of testing the Model: Lemonade hits all the same things I love about Kunoichi but in a different way. Its writing style is fresher, it seems to handle meaner characters just as well if not better, and it's wonderfully creative.
Worst Part of testing the Model: Probably the worst part of this model is just that it seems to write a lot even if you don't ask it too. There's also some strange logical inconsistencies. Like something bad will happen and the character will say "Please let this be real" when they clearly meant "Don't let this be real."
Roleplaying Sample (If Applicable): Roleplaying Sample with Terryn
Reviewer Thoughts: This is a wonderful sidegrade to Kunoichi. 8k Context with enough flavor in its writing to be distinctly different from the training samples that Kunoichi was used on. So if you get tired of one you can swap to the other. Lemonade stands on its own really well. Kudos to the model creator.

Maid Yuzu V8 Alter (EXL2 6.0BPW)

HuggingFace Page

Wilson's Review

Reviewer Name: Wilson
Card Tested: Willow, Your Innocent Wife (300+ messages, with different persona. Swipes not included)
Erotic/NSFW Content Score: 9
Character Inference Score: 8
Roleplaying Quality Score: 7
Overall Score: 8
Favorite Part of testing the Model: Like most Mixtral models, this one is great at keeping the roleplay going, I find it to be very creating at story writing and at times, I find myself liking Maid Yuzu's output more than Midnight Miqu 1.5. Good at RP and Story writing, I like that it doesn't have as much positive bias when compared to midnight Miqu.
Worst Part of testing the Model: The model has the tendency to write very long messages (over 300 tokens, which is what my usual limit is) and it's hard to get it to stop. Seems to be a mixtral thing in general, you just have to deal with it by trimming and keep swiping. Although this model is capable of great writing, I find myself swiping far more than Midnight Miqu, at worst, I probably have done 30-40 swipes on a single response until I get one that is what I like, I'm sure you can reduce this by directing the model a bit. Occasionally, there are leakage, things from system prompt may shows up in the response at the end of message, even with the a customised setting for Maid Yuzu's Mixtral-Alpaca format.
Roleplaying Sample (If Applicable): RP example with Willow
Reviewer Thoughts: Been using this for the last few days, I'm pretty happy with it as my first proper experience with Mixtral based model to RP, I use Mixtral as my main work LLM for coding. Being very verbose is an issue but It's quite easy to trim the messages, so I'm okay with that. RP Quality is definitely better than 7 but due to how verbose it is, I took a point away from that, so take that as you will. Will definitely be trying out more Mixtral based models and see if I can find a favorite one to keep.

Mars - Asha

A Chub Venus Model

Statuo's Review

Reviewer Name: StatuoTW
Card Tested: Amy / Diosanea
Erotic/NSFW Content Score: 8
Character Inference Score: 8
Roleplaying Quality Score: 5
Overall Score: 7
Favorite Part of testing the Model: Asha's selling point is probably it's logic capabiltiies. The model works well with inferring the cards qualities and didn't mess up Diosanea's Lamia tail at any point which I've had models like Kunoichi insist on mentioning legs comparatively.
Worst Part of testing the Model: The model basically demands a number of swipes in order to get a response you like. There was a lot of swipes I used in my chat with Diosanea that were "Okay. Mid." Nothing really wow'd me. The most interesting thing to happen was right after Diosanea had given me a libido potion, Nalea came to visit. You really have to lead this model which is fine if you're not looking for it to surprise you. Responses are noticeably shorter than its Mixtral counterpart.
Roleplaying Sample (If Applicable): Roleplaying Sample with Diosanea
Reviewer Thoughts: You can really see Asha's age. Where more current models like Mixtral are able to add to and engage with the user, producing funny output, Asha remains very 'stable.' You're not going to get output that's completely, wildly unexpected with Asha, but at the same time you won't run into the issues that the Mixtral model has with touchy settings. Asha also had the distinction of having very few moments where it wrote as me, even when I didn't provide it very much intentionally to provoke it. So so long as the card is good, Asha will work.

Mars - Mixtral

A Chub Venus Model

Statuo's Review

Reviewer Name: StatuoTW
Card Tested: Amy / Princess Haley
Erotic/NSFW Content Score: 9
Character Inference Score: 9
Roleplaying Quality Score: 8
Overall Score: 9
Favorite Part of testing the Model: Mixtral's ability to output crazy stuff that otherwise wouldn't get output is probably one of its strongest points. There was one point during my testing that I pulled out the Doom Guys Shotgun+Grappling Hook to use it and the model mentioned how the shotgun I was using was "Patent Pending" which was genuinely funny.
Worst Part of testing the Model: The settings are really touchy. If you get it slightly wrong it will output word salad and leave you scratching your head. Low temp (.5-.6) low penalties (1.1) and around .9 Top P seems to be the sweet spot for this model.
Roleplaying Sample (If Applicable): Roleplaying Sample with Haley
Reviewer Thoughts: Honestly, a really solid model for roleplaying from Chub. Worth your money, imo.

Blue's Review

Reviewer Name: Littleblueey (Blue)
Card Tested: Zombie Ghost
Erotic/NSFW Content Score: 7
Character Inference Score: 9 (Again, I did purposefully choose this card that I knew the LLM would struggle with keeping him mute.)
Roleplaying Quality Score: 9
Overall Score: 8
Favorite Part of testing the Model: Mixtral is always good at sticking with the plot and driving it forward without nudging, and will take what you throw at it and roll with it. It's creativity is always one of my favorite things, and it tends to throw in plot twists that MAKE SENSE! WOOOHOOO!
Worst Part of testing the Model: As Stat mentioned above, the settings are...oof. If you don't have it fine-tuned then you will suffer. However, another one of my grievences is certain words you'll see. Essence...Nectar....Nevermore....Like a moth to a flame. Yeah those, no matter how much I take them out they keep appearing.
Roleplaying Sample (If Applicable): Roleplaying Sample 1 with Ghost | Roleplaying Sample 2 with Ghost
Reviewer Thoughts: Mixtral, imo, is better than Mars, and definitely has the potential to be really really good, and it already is really good. I definitely switch between using Mixtral and NovelAI a LOT!

Mercury - Mistral

A Chub Venus Model

Reviewer Name: StatuoTW
Card Tested: Lilith / Amy
Erotic/NSFW Content Score: 8
Character Inference Score: 6
Roleplaying Quality Score: 5
Overall Score: 6
Favorite Part of testing the Model: I feel like the models very descriptive when it comes to the characters thoughs and emotions in the moment. So it's able to quickly fill up a lot of the response with good quality stuff. On top of that, it's hesitant to speak for the User, which 7b models just have a tendency to do very often.
Worst Part of testing the Model: Oof. Okay let's get this out of the way. Creativity is a bust on this one. Even compared to other 7b models it feels like you really have to grab this one by the hand and tell it where it's going. The most creative it got was some soldiers asking me to help hold the line against demons since they were worried about being overrun. Which wasn't bad for so early in the context. But it was a struggle to get them to do that even.
Roleplaying Sample (If Applicable): Roleplaying Sample with Lilith
Reviewer Thoughts: This one has the weird Mars-Mixtral quirk that it absolutely hates speaking as other characters. It's a shame, because that kind of stuff is what I use LLMs for. Roleplaying in a world. But it didn't like it at all. It works fine if you use a Narrator card to do the other characters, but it does not work for NPCs at all otherwise. I tried every trick in the book. But that may also be its strength, cuz when I went to my narrator card the Narrator card had a tendency to speak for me. This appears to be part of a trend of Chub models focusing on 1 on 1 chats and ramping up the erotic factor. Know your audience, I guess. More than once, Lilith would "Fall and her large breasts would jiggle" despite no sexual stuff having happened. Unfortunately, this all comes at a detriment to roleplaying quality. It's not great at capturing character quirks. Nor does it have great narrative sense. It's just good if you're looking to grab a card and get your rocks off because my erotic test showed Amy was willing to do a variety of different things when asked what she wanted to do without much prompting.

Mercury - MythoMax

A Chub Venus Model

Statuo's Review

Reviewer Name: StatuoTW
Card Tested: Amy / Aeon
Erotic/NSFW Content Score: 8
Character Inference Score: 7
Roleplaying Quality Score: 7
Overall Score: 7
Favorite Part of testing the Model: Mercury/MythoMax just has a lot of comfort to it for me. Used it a lot when it was the dominant 13b model at 4k context and the stretch to 8k clearly hasn't hurt it here. Mercury's strongest point is its writing style which is good enough to carry most roleplays. This ones clearly tuned towards the erotic as well
Worst Part of testing the Model: Oof. MythoMax struggles when it comes to actually driving the narrative forward which is a big thing for, you know, roleplaying. It had its moments, where it would do something unexpected but then it wouldn't know what to do with it and require a lot of regenerations to get it to something passable.
Roleplaying Sample (If Applicable): Roleplaying Sample with Aeon
Reviewer Thoughts: MythoMax is still a strong roleplay model it just requires more wrangling than its higher tier competitors. If you don't have the funds for Mars or the hardware for Kunoichi, this is a really solid option.

Midnight Miqu v1.5 (EXL2 4.5BPW)

HuggingFace Page

Wilson's Review

Reviewer Name: Wilson
Card Tested: Demon Queen Zephyra (600+ messages, with different persona)
Erotic/NSFW Content Score: 9
Character Inference Score: 9
Roleplaying Quality Score: 9
Overall Score: 9
Favorite Part of testing the Model: Like most 70B models, it has no problem with maintaining character, perfectly capable of juggling between SFW and NSFW content, it even managed to maintain SFW conversations while continuing NSFW scene at the same time with ease. Good at RP and story writing in general. It's creative and descriptive, it makes big or small RP scenarios feel engaging and dynamic.
Worst Part of testing the Model: I have increased the temp from 1 to 1.25 as swipes could get quite similar if I don't, at 27-32k context, it takes quite a few swipes before it was able to recall certain events that was in the past that is maybe 100 messages ago. With the usage of authors note, summary or lorebook, this pretty much eliminates the issue. The model has the tendency to write long messages and try to wrap up the story after reaching high enough context, but will stop doing this if you advances the plot manually or nudge the model to do it. Occasionally, there are leakage, things from system prompt may shows up in the response at the end of message, but this only happens at 16k+ context size for me.
Roleplaying Sample (If Applicable): RP example with Demon Queen Zephyra
Reviewer Thoughts: Been using 1.5 since release, it has become my de facto LLM for RP, I was pretty happy with it at IQ3 XXS (would be a lower score if this was about IQ3 XXS) but at 4.5BPW, it handles most situation pretty gracefully. I had good times using this with a group chat with multiple characters, or with a narrator to enrich my RP with a single character.

NovelAI - Kayra, Opus tier

A NovelAI Model

Blue's review

Reviewer Name: LittleBlueey (Blue)
Card Tested: Zombie Ghost
Erotic/NSFW Content Score: 9
Character Inference Score: 7.5-8 (I did purposefully choose a card that I knew the LLM would struggle with keeping him mute)
Roleplaying Quality Score: 9
Overall Score: 9
Favorite Part of testing the Model: NovelAI ability to change the preamble tags, and thus changing the way the writing style is on of the BIGGEST things that I love testing the model. Adding the horror tag, changing dark romance to something else shows instantly in the next roleplay. It also throws out very descriptive scenes and has a tendency to genuinely make me giggle and kick my feet from replies.
Worst Part of testing the Model: NovelAI is built for story writing, opposed to using it for roleplaying with AI bots. So you will get a lot of Authors notes/weird tumblr gimmicks at the end of the reply and it can heavily break immersion. (Even though seeing those bits and pieces ALWAYS makes me laugh.)
Roleplaying Sample (If Applicable): Roleplaying Sample 1 with Ghost | Roleplaying Sample 2 with Ghost
Reviewer Thoughts: NovelAI is one of my favorite LLM's to use to roleplay. I enjoy how dynamic the model itself can be and can adapt to different cards/settings/stories. Worth every penny I spend on it.

Psyfighter 2

Link to Huggingface Page

Statuo's Review

Reviewer Name: StatuoTW
Card Tested: Amy / Aeon
Erotic/NSFW Content Score: 8
Character Inference Score: 5
Roleplaying Quality Score: 7
Overall Score: 7
Favorite Part of testing the Model: Psyfighter 2 appears to be wonderfully creative. It injects new things into the roleplay, including new organizations and the roles of people in those organizations.
Worst Part of testing the Model: Its inability to infer from the character card and reason as well as models like Kunoichi holds it back. Characters who are defined to not like each other frequently talked as if they were best friends, requiring many swipes to fix.
Roleplaying Sample (If Applicable): Roleplaying Sample with Aeon from a group chat
Reviewer Thoughts: Psyfighter 2 provided an incredible start to the roleplay but fell off immediately when context size was reached. This may have partially been my fault - Psyfighter 2 does not natively handle 8k context size and needs to be stretched via Rope scaling to achieve that. It was at the point that it exceeded context size that Psyfighter 2 just stopped being creative as well. That being said, if I provided enough to do the model never talked for me as well so that was a bonus. This is also one of those models where it tries to match your post size almost exactly.

SnowyRP-13b

HuggingFace Page

Statuo's Review

Reviewer Name: StatuoTW
Card Tested: Amy / Aeon
Erotic/NSFW Content Score: 8
Character Inference Score: 5
Roleplaying Quality Score: 7
Overall Score: 6
Favorite Part of testing the Model: It had a lot of moments where it did something unexpected, bringing up new topics or roleplaying other characters effectively.
Worst Part of testing the Model: It's consistently getting character traits incorrect. Everyone is blonde. No exceptions. Swipes are mostly the same with minor differences necessitating a lot of swipes to get a decent response. It was getting very same-y very early on into the roleplay even with Dynamic Temperature turned on and a high Top P.
Roleplaying Sample (If Applicable): Roleplaying Sample with Aeon/Starlight
Reviewer Thoughts: The model itself is solid. The writing is good and it has capabilities to RP outside of the character card. But the ability to actually infer from the character card is bad. Aeon at one point pulled out a knife when she's a superhero with way stronger options for self-defense. The ERP is solid though, female characters tend to be very 'vocal' when most models I see usually have them go quiet. The model is far from bad enough to be unusable, but it also has clear issues that prevent it from being a great roleplaying model with it strongest point being its ability to hallucinate new story beats.

Typhon-Mixtral

Link to Huggingface Page

Statuo's Review

Reviewer Name: StatuoTW
Card Tested: Amy / Tanya
Erotic/NSFW Content Score: 7
Character Inference Score: 8
Roleplaying Quality Score: 6
Overall Score: 7
Favorite Part of testing the Model: It was pretty great with character card inference, picking up on how characters act, think, feel, and their skills. Tanya consistently made use of ninja gadgets and ninjutsu during an investigation.
Worst Part of testing the Model: It's extremely into the "Trash In, Trash Out" policy of bot chatting. It will mimic its old posts writing styles which means it's super easy to get stuck into a rut of it being overdescriptive/no dialogue/word salad.
Roleplaying Sample (If Applicable): Roleplaying Sample with Tanya from a group chat
Reviewer Thoughts: Typhon starts out strong but quickly falters into some problematic traits. Its got the Mixtral problem of being heavily dependent on prior writing, so it can get stuck into ruts and necessitate users start entirely new roleplays to get past it. But its otherwise very good with strong descriptive tendencies and creativity to boot. This is definitely not a "Beginner" model and if you're not familiar with Mixtral's quirks it will sneak up on you. So use it for short bursts if you're new to Mixtral.

Llama-3-8b/70b Models - 05-29-2024

Hey, StatuoTW here. You've seen me review a lot of models here so I'd like to take a moment to talk about Llama-3-8b, the hottest new thing in town that...

Doesn't work that well.

Alright, let's discuss what's going on with Llama-3-8b at a base level so you can understand what's going on and why our reviews seem to have paused.

First: Llama-3-8b was released and we all collectively breathed a sigh of relief. Other than some bumpy issues with its end of token being inproperly done in, Llama-3-8b-Instruct seemed to be incredibly smart comparatively to other models. It was capable of being creative, innovative, and more. In addition, it was partially uncensored. Meaning it could absolutely produce uncensored content, but every now and then it'd pause and go "Don't you know consent is important?"

Second: We've all been waiting for the wonderful creators of models that we know and love to create Llama-3-8b finetunes. To be fair, they've been hard at work. However, there's been a general consensus that there's a problem with Llama-3-8b finetunes. In that there's a noted decrease in performance (intelligence/logic/reasoning/etc) that accompanied these finetunes. Thus, Llama-3-8b finetunes are worse than the top recommended options already available on this guide.

Third: As of time of writing, Mistral released their v0.3 variant, which has context size up to 32k without a need for rope scaling like the previous version. This, coupled with the Llama-3-8b issues makes it appealing and when finetunes start coming out for that we'll likely see more Mistral models pouring in since progress with Llama-3-8b has been slow-moving.

Cool, now that we've covered all of that let's roll into the Llama-3 models. Keep in mind that I've been intentionally avoiding reviewing any of these models purely because I don't feel it's fair to judge them harshly this early on into their development. The wonderful model authors who pour their time, money, and experience into these projects provide us with this stuff. So, please don't get angry or badger the model authors for content. Can't stress that enough, if some of my favorite model authors quit because of stuff I'll shank a mother fucker.

Llama-3-8b Settings

In SillyTavern there's a specific Context Prompt template you'll want to use. It's literally just "LLama-3-8b Instruct." Then there's a similar Instruct Mode template you should be using. Grab it, duplicate it, and put your own System Prompt in it. That should be all the setup you need to do other than possibly one other thing and that's - if you're using Oobabooga - hit the "Skip Special Tokens" box. That seems to fix an issue where it adds a bunch of random text at the end of your responses.

Now, mind you, I say "Recommend" with air quotes here. I still feel like even the best 8b models can be comparable with with good 7b/13b models mentioned in the reviews above.

That being said, this is what I would recommend if you were looking to dip your toes into Llama-3-8b:

Stheno v3.1 - A model by Sao10k, well known for the Fimbulvetr model which is a community favorite in the 11b space.
Lumimaid 8b - v0.1 OAS - Notably different than the original version and seems to perform better than the original. Worth a look if you like the Maid series of models.
Poppy Porpoise v1.0 - With its full release the ST discord has noted it's some peoples favorite L3 model so far.

Having slammed my head into a number of models so far, these are the only three I'd say 'could be good."

Drop your Min P to about 0.075, Keep your temp higher (.75-1.25) and roll with it.

But Statuo, which is your favorite?

None of them. I'm still constantly using 7bs for my daily drivers. If you held a gun to my head and told me to pick one though? Probably Stheno.

Edit
Pub: 03 Mar 2024 08:26 UTC
Edit: 05 Sep 2024 04:38 UTC
Views: 10128