You know the drill, not on discord log of phrase bias files.
- But What ARE Phrase Biases?
The NovelAI implementation of token biasing. Right now, a majority of what we know comes from the words of the team themselves.
NovelAI's unique Phrase Biasing, is far more powerful than the logit biasing we implemented, tested, and open-sourced earlier but found unsatisfactory.
For example, we can bias positively for
ourand make sure that Sigurd sticks to a first-person narrative.
Another example would be to bias for various classes in an RPG such as a
Priest, or against fantasy races and creatures. Alpha test users have reported being able to get a very distinct experience without the use of AI Modules or Lorebooks with Phrase Biasing alone!
Possible bias values are decimal numbers ranging from -2.0 (for negative bias) to 0 (for no bias) to 2.0 (for positive bias). Please note that the strength of biasing is logarithmic, not linear. For example, -2.0 is much much stronger than -1.0. We've observed that bias values in the range of -0.4 to 0.4 are often enough to have a good effect.
The phrase bias menu looks like this at default. Placed underneath the Banned Tokens section of the right-sided window.
A "phrase" is a token or sequence of tokens you can modify the bias value of to modify its chance of showing up.
Entering words / sentences works like banned tokens, you type however long you want it to be, and pressing enter confirms it as a "phrase" which can later be edited and removed by clicking on them. Similar to banned tokens, encasing your output in square brackets allows you to input token values, but encasing them in curly brackets allows you to enter them as is without any modification by the frontend. Unlike banned tokens however, phrase bias will allow you to have duplicates of the same exact phrase.
What you enter by default is case sensitive and contains a space.
robert will be treated as two unique phrases, and the tokens they point to aren't ones used to start a new paragraph. If you don't want the space, use curly brackets.
Increasing and decreasing the Bias value understandably increases or decreases the chance of the collection of phrases showing up in your outputs. As said by the devs, the value works logarithmically, which effectively means that the most drastic changes can be felt by only slightly nudging values.
If you're too dumb to know what means exactly, here's a helpful graph I found on the internet comparing expected curves.
Whatever value you think you need to set your phrase biases to in order to get more words to show up is probably too high. Instead, try adjusting the value by 0.1 increments and then getting more granular as you find the sweet spot. When compared to HoloAI's logit bias implementation, phrase biasing is also extremely context-aware. So far, I haven't needed to nudge things past 0.3 positive or negative, but this is all down to personal preference, and it's braindead easy to change if the AI isn't bringing up enough reptilian tails and scales.
There really isn't a good rule of thumb here about what value to set what kind of thing, and this is entirely up to your own personal preference. Some tokens you want to boost only need a little nudge while others need a bigger push, and it's the same when trying to cut back.
Experiment. The dial is there, so fiddle with it. You'll figure out what you like soon enough.
Phrases are categorized into individual sets which you control the bias value applied to. Because I'm too lazy to ask, we'll refer to them as bias sets.
With the plus button, you can create a new bias set ; all enabled bias sets are active at the same time as all other enabled sets. Similarly, if you have two sets that both contain the same word, the values assigned to the word stack on top of each other.
Like a majority of other options NAI has to offer, you can export your entire collection of bias sets in a story to import into another story, appending those sets at the end of your list.
The last two options, "Ensure Completion After Start" and "Unbias When Generated" are self-explanatory, but I have a few thoughts on them based on preliminary testing.
"Ensure Completion After Start" means that whenever the AI generates the first token of a phrase, the subsequent tokens that comprise the rest of the phrase are high enough to ensure the phrase is completed. This is a very extreme option based on my experiments, as it can cause the AI to bring up the phrase at incongruous times, so why would you ever enable it?
I see two possibilities: either normal bias operation isn't good enough for you (perhaps you're attempting to use an extremely uncommon name or word), or you don't want any other tokens to appear.
Consider my earlier bias set; increasing the bias to 0.5 makes my desired tokens appear often, but also has unintended consequences:
So keep that in mind; increasing bias can increase the likelihood of a lot more tokens than just the ones you've entered. Clever utilization of stacking values can help you overcome this.
This option disables the bias applied to a token if it appears only once during generation. This is certainly more useful in smaller generation sizes where what little output you get is more useful when it isn't the same couple tokens being repeated ad nauseam, but it can also serve to curtail more dramatic biasing efforts where you definitely want words to show up each generation but don't want them to show up more than once.
It's now enabled by default for good reason; without it you'd be opening yourself up to egregious repetition that makes figuring out the right positive bias value quite annoying. Only disable it if that's what you want, like if you want to decrease the chance of full stops and paragraph breaks, you'd turn this option off then.
- If you're scared of a particularly high set swarming over a majority of your outputs One thing you could do is try and increase your Output Length while Unbias is enabled, to space out guaranteed words in the AI's text.
- A cool thing you can do is make a functional 'whitelist', enabling Ensure Completion on a set with 0.0 Bias, and entering in phrases to make the AI only choose from those completions when it generates the first tokens.
- Biases are handled in a first in first out order, so keep that in mind if you have similar starting tokens; the AI will most likely prefer the oldest one to generate—unless the situation really wouldn't work with it.
Alright, so this might count as "I may have gone too far in a few places", but here's the second version of my Comfy Bias experiment
This is basically a "throw everything at the wall with low bias rate and see what happens" approach, and I'm appreciative of feedback on how to do better, since I'm just experimenting at this point. Feel free to try fiddling with values and such, I have no idea what I'm doing.
This bias's goal as it is now is to bring additional focus to slow, loving foreplay scenes and drawing out sex scenes with caresses, touches, kisses and all that other good stuff.
Note: I made this with the intent to use on my own prompts, which are pretty much exclusively first person present tense. As such, verbs used in the bias are set up for that kind of stuff. Third person past tense and such might have problems.
i know some people in the thread were having trouble getting TF shit to work so i made a logit bias list module probably really helps but i couldn't find a TF related module
ok i made the bias set into a card and it works significantly better now that its based on context
now you can enjoy autistic furturdery the way god intended
a card with biases to make the AI understand clothing more
except when it wants to come up with really fucking dumb ass ways to break it, but haha its better than nothing
Alright, here's the finished version with just a few slight adjustments! Though this is purely a bias lorebook entry, I notice none of the other phrase biases on the rentry are set up as a card format so uhhh... Yeah, you guys can decide where this shit should go.
As always, let me know if you see room for improvement.
This updated version covers more edge cases and now works for any perspective and tense!
Add this card to your story's lorebook to automatically detect and enhance any character's actions involving observation.
This lorebook provides additional support and or focus for specific notes/directions to the AI. It's intended to be used either stand alone or in combination with @lion's Pocket Notes and / or @Dichotomy's Editorial Control. It is heavily inspired by both works, and has certain biases taken directly from Editorial Control.
Scenes (use with
[ Scene: ... ] or
[ Theme: ... ])
Scenes are intended to be used when you want to move the story in a specific sort of direction. The keyword Scene tends to be stronger, but also tends to start new scenes (even inserting ***). Theme tends to not be as strong, but is more likely to keep the current context.
Supported Scenes: Combat, Slice of Life, Sex, Romantic, Action, Philosophical, Exposition, Mystery, Medical.
Moods (use with
[ Sentiment: ... ] or
[ Mood: ... ])
Moods guide the story to have a particular sort of emotional weight. The Sentiment keyword is if you want to combine the specific mood with Pocket Notes. If you don't want to combine it use Mood on its own.
Supported Moods: Anger, Joy, Sorrow, Fear, Suspense
Descriptors (use with
[ Describe: ... ] or
[ Description: ... ])
Descriptors are if you want to have a detailed description of something. The Describe keyword should be used to combine the effect with Pocket Notes. The Description keyword should be used if you want to use it without activating Pocket Notes.
Supported Descriptors: Feeling, Location, Clothing Person
These can be formed into sentences, for example,
[ Describe: The person standing in front of me. ] or
[ Mood: I'm angry at John. ] so long as it has the Annotation: and the keyword is in it somewhere.
Recommended to load after Editorial Control and Pocket Notes.
This lorebook card will greatly reduce instances of he said, she asked, etc, instead replacing them with more descriptive words such as he contended or she inquired.
Comes with two versions: Past Tense and Present Tense. By default, Past Tense is the only entry enabled. You can change to Present Tense (common in adventure/2nd person stories) by simply swapping which entry is enabled. More info in the included Read Me.
Hello, friends. Since it's being brought up I figured I'd share a personal experiment that I've been working on. It's built upon the Pocket Notes phrase bias, particularly the Prose Rephraser, which I've been having a lot of fun with lately, as well as ideas from the Den of Sin. After some time with it I realized you could theoretically make on-demand text converters akin to the Boring to Sexy text generator someone made for AI Dungeon.
With a simple command you can turn your boring/incomplete sentence into something the AI will parse on its own. Might be helpful for those who are more interested in offering suggestions instead of typing out absolutes, or as an ad hoc CYOA substitute. ESLs might also find it useful since it doesn't require perfect grammar to operate. It'll understand an entry as simple and concise as “she dance horny”.
The difference between this and typical phrase bias with keywords is you're giving the AI more information to work with, as opposed to the AI jamming in keywords that are related to your interest and hoping it understands the correlation in its outputs. Since it requires a command with special characters to operate it should also mean that the AI won't accidentally trip it on its own. That said, it could work quite well in tandem with phrase bias sets. Such as negatively biasing “He, She, Them, You” sentence starters.
[This is a] Slightly updated version to reflect changes made to Pocket Notes v4. If you are already using Pocket Notes or plan to in the future this should slip in without conflict. If you want to create your own command/text generator you can duplicate either the Boring to Sexy entry or, if you're already using Pocket Notes, one of the Pocket Directors or Pocket Writers and build from there.
Note that the original Boring to Sexy text generator wasn't really designed for something like this, so it attempts to cover all perspectives with broad strokes. This can lead to leakage of the incorrect perspective (particularly 2nd person) in scenarios where it might not be appropriate. To try and address this I created 3 entries for each perspective (Default is also available if you'd like to use or tinker with it). All entries are disabled by default. So be sure to ENABLE whichever is most relevant to your scenario. If you're willing to go the extra mile you can also create perspective based command keys for each entry so you can leave everything enabled (ex. sexy1, sexy2, sexy3, etc.). You can also fiddle with Phrase bias to get similar results by negatively biasing perspective keywords, but I figure most people want a set-it-and-forget-it sorta thing. Whatever floats your boat.
A big thank you to the creator of Pocket Notes for such a useful tool.
"Glue words" are words that help convey relationship between other words, but are ambiguous or contain little meaning in and of themselves. Although they are essential, too many glue words can make sentences "sticky", or overly difficult to parse for the reader without conveying much meaning. A "glue index" represents what percentage of words in a sentence are "glue words" vs "working words". Euterpe out of the box has a relatively high glue index at ~50% (likely because these are the most common words in typical text, so heavily represented in a typical English corpus) where a typical guideline for good quality writing is to aim for an average glue index around 40%.
Version 2.0 of this bias set has been constructed from scratch with a completely revamped set of glue words and testing and regression done on the Pro Writer 1.0 generation preset. The default when imported targets the ideal 40% glue index at a bias of -0.18, but you can adjust this up or down to your liking (even with a positive bias, if you like rambling, meaningless sentences). May require adjustment for different presets (or models, when such become available).
The end result is that generated text is punchier and more meaningful, although it may result in shorter sentence lengths on average (feel free to offset with tags or additional biasing to get the best of both worlds if you like long AND meaningful sentences - this is only doing half the job).
Further Reading on Glue Words:
- What are glue words and how do they affect readability?: https://readable.com/blog/what-are-glue-words-and-how-do-they-affect-readability/
- Delete glue words to unstick your sentences: https://write.co.nz/delete-glue-words-to-unstick-your-sentences/
so, refined the catgirl de-fuzzer bias list (two lists, one at -0.4 and one at +0.1), tested on a human biased story and a blank prompt, if anyone wants to test on a furry prompt to see if it keeps them from becoming furry (mostly just for judging how effective it is in a fully hostile scenario) I'd appreciate.
Learned that blocking the leading phrase/word is much more important than trying to nail down every possible permutation (and way more practical too - covered apparently is the key word that leads to furball descriptions, and biasing toward humanoid/human-like appearance allowed keeping the desired traits while allowing an alternative to the undesired ones)
This set adds a subtle set of biases to steer NovelAI towards better writing.
- Avoiding Using Expletives At The Beginning of Sentences
- Avoid Circumlocutions In Favor of Direct Expressions
- Avoid Overusing Noun Forms Of Verbs
This lorebook adds a Game Master Emulator (also known as an Oracle) to your story. Often used in solo roleplaying games, oracles can be consulted in place of a human game master in order to answer questions, generate content, and provide direction about limitations and possibilities of the game world.
To ask the oracle a question, start your query with the phrase Question: or /Question. For example:
Question: Do I notice any traps? or
/Question Where am I currently headed?
This oracle lorebook can also be found within my entry for the 2021 NovelAI Halloween Contest, called The Forbidden Land so be sure to try that out if interested.
For more information about Oracles/GM Emulators, check out https://rpgsolo.com/
Here's a Euterpe v2 version of Pocket Directors, the first (and most used) component for Pocket Notes.
The other components (editors, guides, & writers) are coming soon, just wanted to get this one out same burger hours day as Eute v2 launch.
>Wtf is this autistic shit?
Lets you use AID style DO/SAY/STORY type commands outside of the text adventure module.<br>
Use it like
Do: she sucks you off
Dialogue: "I wanna suck your cock"
Dialogue: she offers to suck me off
etc etc, it comes with a read me
✎ Portable Tools For Any Story ✐
This lorebook contains a suite of useful command-based tools including new and enhanced versions of almost every utility I released previously.
- Pocket Directors - Force the AI to proceed as you please!
/doEnforce a specific action or event to happen.
/do backflip into the water
/sayEncourage a character to speak a specific line or expression an intention more fluently.
- Example: /say Triss: This is delicious
/storyDescribe the scene, an action, or set the tone in more vivid detail.
/story describe her appearance
- Pocket Guides - Ask the AI for answers for clarification!
/interpretHave the AI decipher keywords or a vague message.
/interpret Trying to stop you is nightmare skilled in juggling
/questionAsk the AI a specific question about your current story.
/question Why does he want the sword?
- Pocket Writers - Can't think of anything? Have the AI write for you!
/plotFabricate plot hooks and twists.
/plot cosmic horror
/startGenerate new scenario prompts in 1st, 2nd, or 3rd person.
/start2 lion, urban fantasy(the number after start corresponds with the 1st/2nd/3rd person perspective)
/setupBuild lore exposition or backstory for a character or setting.
/setup cyberpunk, crime
/writeRephrase any sentence with more eloquent prose.
/write I knew something wasn't right.
Please Note: To utilize this lorebook, you need to type specific commands. You can use upper/lowercase and alternate phrasing for every command, such as do: or even DO: instead of /do. Some commands even have alternate keywords such as /ask or /tone. Check the associated lorebook entries for more examples.
Results may vary. If you're having trouble, try changing up your phrasing since the AI might not like the tokens you are using.
After using a command, it is intended that you either delete or comment out the command (by putting two hashtags like ## at the start of the line) so that the AI doesn't start repeating input commands itself. You can also surround your input in brackets.
Prompt generation with /start requires numbers 1-3 such as /start1, /start2, or /start3 to work. Also it is encouraged to include a character name as that will be included in the prompt.
Newlines are not necessary after your command. Simply type your command with your input and hit send.
While some commands can be used on their own without any additional input, most work best when given an input following the command on the same line. There are usually multiple ways to initiate a command. When in doubt, refer to the examples in the corresponding lorebook entry.
Pocket Notes utilizes/includes improved (but non-togglable) versions of existing tools within some of the commands:
- Prose Enhancer - (NAI default scenario browser) & https://discord.com/channels/836774308772446268/845669718735716362/855558266188595270
- Input Enhancer - (NAI default scenario browser)
- Better Said Words - https://discord.com/channels/836774308772446268/868472982471843870/905232262839287830
- Sensory Enhancer - https://discord.com/channels/836774308772446268/868472982471843870/904930078104907776
- Oracle - https://discord.com/channels/836774308772446268/868472982471843870/903496215059107872
- Tell, Don't Show - https://discord.com/channels/836774308772446268/895192255818792970/898107583011909652
- DIY Prompt Starter Kit - https://discord.com/channels/836774308772446268/845669718735716362/885045907173810207
- Plot Hook Generator - https://discord.com/channels/836774308772446268/845669718735716362/880378288915218472
- ...as well as some minor examples from the community (Thank you everyone for all your help!)
The expanded and gamified pocket notes, now with a new name. To fit the goal of my Generators idea, I had to add slight alterations to Pocket Directors, Pocket Editors and Pocket Guides. Hence RPG now being less compatible with lion's original pocket notes vision and the name being changed to reflect this.
RPG is a bunch of commands, mostly alterations of pocket notes, to temporarily order the AI storyteller (Sigurd) to output better guided exposition. These commands, mostly in the pocket directors category and the Judge command, take at least your recent paragraph of context into account, making the output relevant to the story. It is not a replacement for pocket notes, as pocket notes is better oriented for zero-shot generator scenarios, or using commands that rely less on what is inside your context. Pick the tools that best suit your purposes.
List of added command words: Ambience, Creatures, Equipment, Exposition, History, List, Plants, Population, Quest, Recall, Wildlife.
In testing with lion we found that Judge: is surprisingly good at working with instructions you give it. Judge: danger: dragon or Judge: dragon, danger level usually work fine for estimating how dangerous something in the story is, but your mileage may vary. If you use this LB and a command is broken take the complaints to me because I probably broke something during editing lol. The LB comes with a readme and a link to lion's github/wiki. If the command doesn't immediately give you a decent response, retry it a couple of times then test if editing the prompt helps. Happy Holidays
This is the phrase bias I use to get longer sentences, it works great. Since you want the opposite, shorter sentences, just reverse the direction of the slider after you import it (i.e. change it from -0.2 to +0.2)
By biasing for and against a set of tokens:
You can make your story terse, with snappy interchange of dialogue by sliding to
+0.1, or you can make your story verbose, with details and action by sliding to:
Following feedback on the first release, I've done some research into finetuning. The changes are based on recommended guidelines from https://www.thepunctuationguide.com/ as well as The Chicago Manual of Style, the primary style guide used in American book publishing for both fiction and non-fiction (which I totally did not scrape from the website for offline referencing after my free trial ends).
After importing, you'll see 12 groups of presets, of which you only need to adjust the first 6. All 6 values are set to 0 on fresh import. Going from top to bottom:
- Group 1: . ? ! — Adjust this if you want to increase/decrease sentence length.
- Group 2: Reduce this if you want to decrease sentence length. Do not set to positive.
- Group 3: " ?" !" ." ," Adjust this if you want to increase/decrease dialogue frequency.
- Group 4: Reduce this if you want to decrease dialogue frequency. Do not set to positive.
- Group 5:  The standard newline. Adjust this if you want to increase/decrease paragraph length.
- Group 6: The other vertical whitespace tokens. Reduce this if you want to increase paragraph length. Positive bias not recommended.
Sentence length, paragraph length & dialogue frequency have each been broken up into two groups. The first group is made up of the most normally commonly used punctuation, so increase/decrease the bias as you wish. The second group is made up of "the rest" which are more uncommon in writing. Excessive occurrences will may your sentences look weird, which is why you don't give these tokens positive bias. A possible guideline is to maintain the second group's bias at a non-positive value lower than the bias of your first group.
Groups you don't need to touch unless you really know what you're doing:
- Group 7: The rest of LYlyLYLY lyLYlyly's Dialogue Killer Banned Tokens which are not covered in Groups 3-4. Converted outright bans into maximum negative bias so that you can "unban" them more easily.
- Group 8: Other punctuation combinations disallowed by the referenced guides. Note that single quotes and brackets are "banned" via Groups 7-8. These are permitted in the style guides, however, but only when nested inside double quotes or parentheses respectively. For readability and due to their general rareness in fiction, I've taken the liberty to ban them. Feel free to extract them out if you wish to customise for your own purposes.
- Group 9-12: Better Writing Biases, unchanged from v1. You shouldn't need to touch these.
Unlock various toggles and sliders to give yourself a finer level of control over how your story feels, while increasing Sigurd's baseline level of quality. Version three optimizes for the new Sigurd, while implementing a few fixes to some issues that were pointed out to me.
Special thanks to OccultSage for his phrase bias guide and Fluffy for making |Fear| ||!
A bias for when you want the narrator or AI to speak themselves instead of through a character. Utilizes a tweaked variant of OccultSage's two Story Flow Control and Better Writing biases along with some additional keywords and other catches to prevent dialogue and progression of the story. Load the bias for the full breakdown.
- Exposition (empty prompt)
Nanashi's mother was a beautiful woman, but she could be stubborn and selfish sometimes. Her husband had been killed in the war two years before Nanashis birth; since then he'd left her to bring up their son alone—and when they were short of cash or food, it fell on him because his wife would not work ... which caused problems for them both later down the line as well. But that wasn't what made things difficult right now ... there simply weren't enough coins about this time of year...
- Scenario Generation
You are Lion, a member of the most powerful family in all Feltrim. Your father is one of our wisest and strongest men; your mother was once an Acolyte to Amethyst's High Priestess ... You have been trained since birth for this role, but you must be ready when it comes time: on some day soon there will arrive at court here...
- Solo Roleplaying
Question: Where am I right now?
Answer: On a small boat, sailing through the dark waters of Lake Champlain in an area called Bemis Heights in Vermont... or so they tell you when your compass doesn't work anymore.
- and anything else that requires speaking directly with the AI/Narrator.
by nobody important
Individual bias sets containing the most common pronoun sentence starters NAI can frequently devolve to if you're not careful. I've tried to keep redundancy to a minimum and make each as modular as possible so that you can easily mix and match.
Yourself (preceding space variation) and
Contains the most:
Surprised no one had made this already, but here it is: A simple bias you can use to reinforce a certain narrative perspective.
By default this is set to second person, but you can easily change it by altering the values of the two bias groups (not just reversing positive to negative). For example:
- For Second Person, just leave it alone. It's plug and play.
- For First Person, change the
0.05(note the decimal placement).
- For Third Person, change both groups to
-0.5. That's it.
Not that this bias takes a light touch approach favoring a heavier discouragement of unwanted terms and a very light encouragement of desired terms. However, as a result, this bias may not be strong enough on its own to firmly force a perspective on a new blank story, thus this works best within existing stories with non-blank prompts.
This is intended as it is least destructive to your stories (as it doesn't try to add you/I every chance it gets, only changing when it would normally).
Similar to the above, here is a bias to reinforce a specific tense.
By default this is sent to present tense. Here's how to change it:
- For Present Tense, just leave it alone. It's plug and play.
- For Past Tense, change the
0.01(note the decimal placement).
Like with Perspective Reinforcement, this bias takes a light touch approach favoring a heavier discouragement of unwanted terms and a very light encouragement of desired terms. However, as a result, this bias may not be strong enough on its own to firmly force a given tense on a new blank story, thus this works best within existing stories with non-blank prompts.
This is intended as it is least destructive to your stories (as it doesn't try to add is/was every chance it gets, only changing when it would normally).
It's the entire Pokedex (without forms). I fixed the bias, 0.1 seems to be too high for such a long list so I kept it at 0.05, which works fine for me. (also fixed a mistake where I had a whitespace token in front of all entries, so Sigurd mostly just outputted whitespaces)
On a side note, I used Nidoranm and Nidoranf since Sigurd apparently can use the correct symbol (screenshot below)