Home⠀●⠀@buybread on Discord

NovelAI on SillyTavern

Last updated using SillyTavern 1.12.6 'staging' (24e558599).

This page receives updates from time to time, usually nothing major though, just style/grammar/clarification.

Table of Contents


Important

Erato, and all NovelAI models in general, are strictly text completion, not chat completion. There's no predetermined "AI assistant" personality beneath it all. The model is simply continuing the train of thought you provide it with. You are very much expected to write proper, intelligible plot alongside Erato, guiding it into your desired direction through your own writing. Putting little to no effort will result in weaker responses, as Erato will conform to your writing style to an extent.

TL;DR: garbage in = garbage out, keep this in mind.

This page is written from my amateur experience in playing out my silly ideas. I'm not an expert on how every single option and value affects Erato's output. If you're interested in tweaking various values on your own, I will include reference material from NovelAI's website in the appropriate locations.

As of writing, modules are not a thing for Erato. While the dropdown menu for them is still there in SillyTavern, the feature itself is not functional.

Special Symbols

NovelAI's training data tunes their models to adhere to certain sequences of tokens that influence what the model is more likely to write about. Even with Llama 3 as a base, this has not changed and you should not attempt to utilize any of the pre-packaged Llama 3 templates in Context Template and Instruct Template. Erato simply treats all that extra formatting as part of the story, instead of anything special.

This section will go over the special symbols that are relevant to influencing roleplay-related scenarios. If you're interested in the full list, click here.

Arrowing (>)

Used to denote quotes, this includes digital outputs like messages, emails, and so on. In addition, it can also be used to narrate your actions. The documentation recommends putting a space after the arrow.

Example

> Looks over shoulder in his direction.

Dinkusing (***)

Putting *** at any point will essentially be interpreted as a "fade to black" scene change. In most cases, it will pick up the plot in the logical next step, so, as an example, if you put *** at the start of a lecture scene, Erato will pick up the story after the lecture ends. Obviously, this can also be influenced by how you word the last few sentences, possibly skipping along the entire day at the campus in the lecture example.

Horizontal Lining (----)

Denotes informational text. Basically, sectioning. Helps the model hallucinate less, as it will not treat things like character sheets and lorebook entries as part of the main plot if they're written underneath ----.

Spaced Bracketing ([ ])

Used to denote a myriad of things that heavily influence the model's output and understanding of the context. Check out the table below for more information.

Usage Explanation Example
ATTG (author, title, tags, genre) Gives Erato a clue on which themes are present. Since Erato is trained on many popular books, you may note down the author as one of your favorites (the model will (maybe) write in their style). If you don't want to bother with titling, attaching an author, etc., just don't include that field. [ Author: y/n; Title: some story; Tags: Genshin Impact, Kamisato Ayaka; Genre: high fantasy, slice of life ]
Rating Specifies the writing quality level, ranging from 1 to 5. "4" is the default value that's always appended in the NovelAI front-end editor. [ S: 4 ]
Location An effective way to set the scene without needing to explain anything. [ Inazuma, Yae Publishing house ]
Viewpoint Clearly denotes the current point of view. Erato picks up on the "POV style" rather quickly (e.g. [ Keqing ] uses first person, [ Ganyu ] uses third person). [ Yae Miko ]
Styling While ATTG clues in Erato on the general tone, Styling clues in Erato on how to write in general. [ Style: chat, complex, sensory, visceral ] (default ST preamble)
Knowledge If you had not already tagged the context with some franchise/topic, you can put in the more "appropriate" place. [ Knowledge: Genshin Impact ]

Each instance of bracketed text is supposed to be put on its own line of text, not inserted at the end, middle or beginning of a line on which you'll be writing something else.

Utilizing "Continue"

There are several use cases for "Continue", but the most prominet ones would be:

  • Unless the starting message and your own response to it already establish a multi-paragraph response format, this behavior will have to be manually wrangled into the context. If it refuses to, or does gymnastics to avoid talking as the character you intend to, you can always add a few beginning words to make it clear that it's supposed to be writing for a specific character.
  • Correct any lore mistakes that Erato makes before they begin to significantly carry over into the rest of the context. Usually, this shouldn't have to be done more than a handful of times before it "gets the point".
  • Co-author a message or two to enforce a specific mood/style to the narration or dialogue.

Instructions

While modules don't exist, Erato does (seem to?) have (undocumented?) instruct capabilities with the same old syntax (e.g. { Write out five cities in Japan. }). They're marginally better than Kayra's instruct module, though because there's no "Assistant" personality, initial answers range from semi-helpful to nonsense. Whether it adheres to an instruction mid-roleplay is also a 50/50 coin flip.

Additionally, there are clear signs that it largely disregards the formatting of instructions as "part of the plot", since it sometimes writes them out itself. It basically feels like it's still in the same experimental state as it was over a year ago, just slightly more coherent. Try it, don't try it, just don't attempt to use it like a chat completion prompt ("pls write 2-3 paragraph response as {{char}}").

Presets

Creating your own presets is unnecessary, as the default presets included in SillyTavern are straight from NovelAI's editor, meaning they're already well-refined by people who know the model inside out. The purpose of each preset (the ones prefixed by Erato-) is as follows:

  • Golden Arrow - default, all-rounder.
  • Wilder - as the name suggests, it's more random, though also more prone to mistakes.
  • Zany Scribe - avoids mistakes and repetition, prioritizes more complex words.
  • Dragonfruit - varied and complex language with little repetition, though makes more mistakes and contradictions.
  • Shosetsu (小説家) - designed for Japanese writing, though works in English too.

Feel free to try out all of them to find your favorite. Otherwise, if you wish to experiment yourself, refer to these:

Additionally, there are a few community-made presets that some users claim are superior to the default ones. You can find them by joining the Anlatan Discord server and navigating to the novelai-content-sharing channel. The most prominent preset I've seen is the "ProWriter Series". There are also many other helpful threads posted there.

Context Size

Make sure to adjust the Context Size of all your Erato presets to the max (8192). Don't forget to save afterwards!

Preamble

This block of text is inserted after the Story String (character sheets, scenario, lorebook). As seen in the table, the default SillyTavern premable is sufficient in setting the tone, though there can be a few improvements to it, as well as adding more bracketed text (such as tags and rating).

Example

1
2
3
4
***
[ Tags: explicit, plotty; Genre: slice of life ]
[ S: 4 ]
[ Style: chat, complex, vivid ]

Dinkusing the beginning separates it from character sheets, lorebook entries and so on. Not sure if it's effective, but placebo tells me to "do it".

Banned Tokens

Generally speaking, it's a good idea to ban the Special Symbols that you don't want the AI to be using. Dinkusing, especially, is a favorite of Erato, as the model likes to switch scenes rather often at "conclusions" (which I personally don't enjoy).

Example

1
2
3
4
----
***
[
]

Phrase Repetition Penalty

If you dislike a bunch of "she said"/"he said", then turn this up. It'll lean towards more creative ways to write those, like "the kitsune said".

Advanced Formatting

In this panel, you can configure how the entire context is presented to Erato, meaning you can employ tricks to enforce roles (AI writes this, you write that), separate the character sheets and lorebook from the conversational context, and so on. There are several ways to go about this, as per everything explained so far, but I'll leave my own relatively simple template below for copy pasting.

One thing of note before continuing, turn off the System Prompt, as it will be completely useless. Additionally, make sure Tokenizer is set to Best match (or Llama 3 if you're paranoid).

If I'm not mistaken, a character card's Main Prompt override will be {{system}}. I've not tested if SillyTavern still bothers adding it if the System Prompt is disabled. If it does, it could be used to tag your character cards.

Story String

{{#if system}}----
{{system}}{{/if}}
{{#if wiBefore}}----
{{wiBefore}}{{/if}}
{{#if persona}}----
{{persona}}{{/if}}
{{#if description}}----
{{description}}{{/if}}
{{#if personality}}----
{{char}}'s personality: {{personality}}{{/if}}
{{#if scenario}}----
Scenario: {{scenario}}{{/if}}
{{#if wiAfter}}----
{{wiAfter}}{{/if}}

Example Separator

***

Chat Start

***

Methods for enforcing character roles

As there are no "role tokens" to denote User input and AI Assistant output, there are a few different ways of ensuring that Erato writes only as the desired character.

No roles
Of course, if you feel like it, you can simply set the Instruct Template to Story with no changes. Do note that it'll get a little messy.

Viewpoints

Set the Instruct Template to Story, as it's essentially a blank slate. From here, you only need to add [ {{user}} ] under User Message Prefix and [ {{char}} ] under Assistant Message Prefix. Make sure that Includes Names is set to Never. Now either just normally save the template, or save it as a new template if you don't want to override the Story template.

This method is better for multi-paragraph responses.

Character Names

Turn off Instruct Template and make sure that Always add character's name to prompt under Context Formatting is turned on. Save and add [ "{{char}}:", "{{user}}:" ] to Custom Stopping Strings.

If you had not changed any settings prior, this should already have been the default setup.

This method is better for single-paragraph responses.

Text Adventure (Arrowing)

Before doing anything, it's recommended that chat (if you're using Styling in the Preamble) is replaced by text adventure, as well as add > to the list of Banned Tokens.

You can either use the Story template and manually add Arrowing to your messages, or put > (and a single space afterwards) under User Message Prefix. Untick Wrap Sequences with Newline, otherwise there'll be a newline immediately after the >. Make sure the scenario you're doing this in makes it easy for Erato to understand that arrowed text is acted out by the selected {{user}} persona.

Writing

This is an extra section to remind you that NovelAI as a service aims not to provide an AI writing assistant, but an AI writer that mimics YOU. Unless you're content with short, snappy responses that have little to no interesting language, be prepared to perform rewrites (doing it in tandem with Continue alleviates this to a lot) until Erato begins outputting in your desired manner. Text formatting, pacing, perspective (firstperson, thirdperson) and tense are all inferred, not taught or instructed.

In a super duper simple example, if you use slang or acronyms like "lol", Erato will also use them where appropriate.

Bot & Lorebook making

Even if you don't intend to make your own, take a moment to read and understand what best works on Erato.

Preferred

  • Clear, concise, grammatically correct English.
    • Allows to denote the overall tone of the roleplay.
    • Ali:Chat is also an option to also denote speech patterns and quirks.

Works

  • XML
  • YAML
  • PList

Tiny tendency to hallucinate.

Avoid

  • Overly specific descriptions.
    • No, the exact, precise location of an accessory is not needed.
    • You also don't need to explain what every strand of hair is doing.
    • Please save your tokens.
  • Complex syntaxes that try to be "smart".


Edit
Pub: 17 Nov 2023 15:56 UTC
Edit: 12 Nov 2024 00:52 UTC
Views: 4971