Balaur of thought v5.10: The friendly manual
If you're looking for download links and/or installing instructions, they can be found HERE.
Table of contents
Overview
Balaur of thought (BoT) is an attempt to mittigate LLMs in-principle flaws, observed across multiple models and scales; namely, the tendency to hallucinate.
Observation
Empiric observation have demonstrated that modern LLMs are (somewhat) decent at basic logic (as long as it doesn't involve spatial reasoning) when directly prompted; just as it's very bad at the same thing when left on their own. Likewise, common-sense can be predicted by a LLM when directly asked but will be ignored when not.
Sure, one can ask a bunch of questions manually, add a bunch of guidelines, and then ask for a character reply, which is good-ish on some settings, but very bad for RP and inmersion. BoT is an attempt to streamline the process of mittigating hallucinations by putting situational awareness into the context
The BoT technique
The actual technique involves two parts:
- The body: A series of prompts force the LLM to shift attention to the kind of things that typically don't get it, by directly asking qbout tvem. LLM replies are injected into the context, putting those conclusions last to maximize power.
- The heads: The many heads of the mithological beast represent the branching of paths that happens afterwards. With the body of analyses in the context, the LLM is asked to generate a number of actions it's character/s could take, limmiting the possible plot to only a few feassible options.
This implementation
The full BoT technique as described above was fully implemented in version 4 of BoT, but since it was buggy and restrictive, the whole codebase ended in the trashbin. Current v5.10 lets the user to write his/her own analyses and group them into batteries in the order and length they see fit; furthermore, batteried can be crudely scripted.
Guidelines and RAG can be used to aid in thevBoT technique and on RP in general. Also the array of tools provided (which will grow withvtime) add uses, related to RP and creative writing.
Analysis
Analyses are the basic building blocks of the BoT technique. An analysis is a prompt containing questions about the context (typically on a RP scenario). The LLM answers it and the result is injected onto the context to aid on further responses.
Menu explainer
The analysis menu can be accessed from the brain icon.
- Create: User needs to input a prompt and a name. Both can include any number of macros, such as {{char}} or {{group}}, as they look here, no need for escaping.
- View: Shows the analysis prompt, macros are replaced in the shown version to reflect their current state.
- Perform: It generates a reply to the analysis prompt. All analyses results are ephemeral injected after the next user message arrives.
- Edit: The prompt is shown in an input box, macros are not replaced. The user can then edit the prompt. If not cancelled, the new version is saved and a new input box is shown, giving the option to change the analysis name, cancelling this will leave the name unchanged.
- Remove: Removes the analysis from the list. It is also removed from any analysis battery that may contain it. If a battery is left empty after this operation, it removed.
Practical knowledge
The name should be unique to avoid conflicts, but as of v5.10 BoT does not check for duplicate names.
As analyses are performed, the results are wrapped in a prefix and a suffix, this behavior an be changed in configs, under the wrappers item. Once wrapped, they are put in a chatwide array. As mentioned above they are ephemerally injected (single use) after a user message arrives in a single card chat, or, after the next soeaker is drafted in a group chat.
By default BoT comes with a number of analyses, all oriented to implement BoT technique on a limited universe of RP settings. However, the same questions that explain a romantic date does not explain a battlefield. So uders are strogly encouraged to add their own prompts for their specific needs. Default analyses can be deleted/modifued too, which can also help to keep the list tidy.
When an analysis if removed, BoT automatically removes it from any battery that contained it. If a battery is left empty by this action, it is also removed. If the dleleted analysis is set for automation, it is removed from there.
All analyses performed from the analysis menu are injected in the order they are performed, prior to the next user medsage in the context.
Battery of analyses
Batteries of analyses are two steps above analyses in terms of sophistication, as they allow you to group and order infividual analyses as well as defining what to do with each one's result.
Menu explainer
The batteries menu can be accessed from the literal battery icon.
- Create: A list of all stored analyses is displayed, they can be added one by one, the text above will display the names of the ones added so far. Every analysis must be asdociated to a behavior, which is basically what to do with the analysis,' result; These are: Pass to have the next analysis see the current result, Send to have it injected, and Both which does both things. This menu bas no save button, you might' e noticed, in order to do that, you need to cancel, which will open a menu where you can save or discard the current battery, also you can remove the last added analysis.
- View: A basic list with the names of the analyses and their behavior contained in the battery is displayed.
- Perform: Each individual analysis is performed in order. Depending on their behavior, one or more results will be injected after the next user message srrives.
- Rename: Lets you change the name of the selected battery. A battery's bame only use is for you to see it in the list, so feel free to use wvatever name you want.
- Edit: Displays a list with the names and behavior of each individual analyses contained in the battery. You can then switch tvem, reorder, remove and change their behavior. New ones can also be added.
- Remove: Eliminates the battery. If the deleted battery is set for automation, it will also be removed from there.
Practical knowledge
Batteries names must be unique, however, as of v5.10 it is your duty to keep it that way.
Each battery can run an arbitrary number of chains of thought, outputting the result of each one to the context. In order to tailor a battery that actually does what you want it is important to unferstand that there are two arrays at play:
- Chain of thought: This one gets filled with the results of the individual analyses. It is injected into the context before performing an analysis. This array can be emptied by an analysis with the Send behavior.
- Battery result: Contains the results of the analyses with the behaviors Send or Both. Once the battery has run the trdults in this array are the ones to be injected on the appropriate moment.
The precise action of the different behaviors is:
- Pass: Adds the result of the current analysis to the chain of thought, this chain is injected into the context before performing the next one. Pass also excludes the current result from the result array of the battery.
- Send: Clears the chain of thought and does not add the current result to it. The current analysis result is added to the battery result array.
- Both: Adds the current analysis result to the chain of thought. Sneds the current analysis result to the battery tesult array.
All batteries of analyses performed from this menu will be injected in the same order they are performed prior to the next user message in the context.
Theory&tips
A well-planned battery acts like a two-level chain of thought. With a series of chains of thought, each one sending a result to the battery result. This array of results, if each one is a component of a larger reasoning process, can act as a higher-level vhain of thought.
Setting more than one battery for automation with different frequencies, can help remind the LLM to keep certain elements in mind.
Guidelines
A guideline is a simple instruction you can type and get injected into the context.
Menu explainer
Guidelines can be accessed from the signpost icon.
- Quick guideline: Lets you type a guideline, then generate a character reply. You can decide between adding the reply as a new message or as a swipe on the last one.
- Create: An input box allows the you to type a new guideline, macros such as {{user}} or {{char}} can be used as shown, no need to escape them. Once created. Once the name is entered, the guideline is saved for later use.
- View: The actual text of the guideline will be shown, macros are replaced in order to reflect the current value.
- Inject: Sets the selected guideline to be injected after the next user message has arrived. Only one guideline can be active at a time.
- Edit: Displays the selected guideline in an input box. After editting the guideline can also be renamed in a separate input box.
- Remove: Eliminates the selected guideline from the list.
Practical knowledge
If a guideline is injected and then removed, that will not remove the injection.
Regardless of when a guideline is set for injection, it is always injected after all analyses/batteries results and after the next user message in the context, basically at depth 0.
Rethink
While having BoT does not keep you from regeneratkng/swiping, the rethink option adds a few different ways to do it.
Menu explainer
The rethink menu is an arrow pointing in a circle. It has threevbasic options, the third being far more nuanced than the other two.
- Same injects: It will reinject the same analyses/guidelines as when the original message was generated and then adds a new swipe to the last message.
- Current injects: It will inject the results of all analyses generayed after the last character message arrived.
- Rethink generator: First thing, you need to select how bad the last message was, then type why it was bad and finally select whether to simply geberate a new message or use a two steps approach. The simple mode will take the prior input and injext it as instruction and generate a new character reply. In the two steps mode a midle-step is generated, ib which the LLM is asked to reason about how the message should be, before actually generating it.
Practical knowledge
Given the non-deterministic nature of LLMs, reusing the same injections that produced a bad reply is not as dumb as it sounds, however, this largely depends on the tempatature parameter you are using.
Automation
Automation gives a flexible way to have analyses and batteries performed per message while at the same time adapting to your needs.
Menu explainer
Automation menu can be opened with the car icon.
- New: Use this to add a new analysis or battery to be automatically performed. Once one is selected you need to input every how many user messages you want it to run.
- View: Displays a list of the included analyses and the behaviors for batteries, or the actual text for single analysis.
- Change: It lets you change the current analysis/battery for a different one.
- Frequency: Lets you set every how many user messages to perform the current analysis/battery.
- Remove: The selected analysis/battery will not run automatically anymore.
Practical knowledge
All analyses/batteries performed automatically are injected in the same order they are performed, after the next user messsge, but before guideline (if any).
Data bank & RAG
The databank feature relies on ST's vector storage extension, and Data Bank extension, so you might as well want to have that up and running and correctly configjred if you plan to use it. This feature sill still be able to manage DB files even if verctor storage is disabled, however, in that case files will not have an effect on the conversation.
The data bank feature aids you in RP and/or creative writting by keeping entries organized by topic, with a consistent format, and even to be generated by the LLM itself on your demand..
Menu explainer
The vault icon gives access to the sata bank menu, here you can create and manage files from ST's data bank.
- New: New entries must beling to either the currebt character ir the currebt chat but can be transferred between them once created. Type a topic and whether you want to write it manually or let the LLM create it for you.
- View: All entries for the current topic are displayed in order.
- Rename: This allows you to change the name of the currebt topic, for DB files, the topic is implrtant because it is part of the actual file.
- Edit: Lets you efit each entry in the currebt topic individually.
- Transfer: Here you can take a file from the current character's DB and move/copy it to the chat DB ir the other way around. This option is not available for group chats due to a ST limitation.
- Remove: Deletes the current topic, and file from the data bank.
- ST Databank: Opens Silly Tavern's vanilla interface for the data bank.
Practical knowledge
The actual format of DB files is applied autonatically when creating/modifying topics and/or entries. The format itself can be modified from the config menu.
BoT allows only a single file per topic, each file/topic can have multiple entries
Tools
The tools menu (screwdriver & wrench icon) gives you access to sumple but useful features. These are not part of the BoT technique but can aid it, and also help in general RP.
Rephrase tool
Sometimes LLMs spit out good content in the wrong tense or person, rephrasin aims to fix that.
Menu explainer
- Weite now: Lets you manually input a criteria for the LLM to rephrase the last message
- Select now: Opens a series of menus where you can select the desired tense, person and POV.
Practical knowledge
Rephrased versions are generated using the genraw command, this means that the LLM only has access to the rephrasing promot and the message, not the shole conversation. In order to generate a different message use ST's swipe, regenerate, or BoT's rethink.
Reverse-prompt tool
Unlike analysis, shere the LLM answers user-generated questions, this asks the LLM to write the questions which you need to then answer.
Menu explainer
- Edit prompt generator: The prompt that asks the LLM to write auestions can be modified from here.
- Perform reverse-prompting: The LLM will generate a bunch of questions to better understand what's going on.
Practical knowledge
The actual questions are displayed on the same input box you use yo write the answers, whether you leave them or delete them, only leaving your answers is up to you.
Configuration
Global BoT settings can be modified here, as well as running some maintainance procedures.
Menu explainer
- Check generations (former mindread): When enabled BoT-generated text, such as analysis' results is displayed, giving the user the chance to edit or cancel it. If disabled, generation results are always assumed accepted.
- Delay: Some APIs reject too many continued requests. Delay is a workaround that can be enabled or disabled and the actual delay time set. The way this works is by opening an input box, cancelling disables delay, while setting a valid time (in milliseconds) will enable delay and set the time in a single step
- Wrappers: Analyses prompts, results, and guidelines are by default added a prefix and a suffix when generated/injected. Individual wrappers can be editted and/or disabled.
- Databank: The prompts used to auto generate entries can be edited here. Also the strings used to format DB files can be toggled on of off and edited; just keep in mind that unless you want topic and entries to a be a single, continuous line you need to at least put a few newlines; Finally the update option takes all BoT-managed DB files for the current character/chat and rewrites them with the current format, useful to keep a uniform style across all of them.
- Reset: Deletes all custom prompts, batteries, guidelines and settings, restoring original values. Also eliminates strings set for injection. Reverts all wrappers and DB file format strings to their default values but does not modify nor release the actual files.
Balaur of thought v5.10
The half-decent implementation of a peculiar prompting technique based on romanian mithology <hr>
Made possible by
ST dev team and contributors, AI Horde devs and volunteers.
With the kind help of these reddutors
Mamelukturbo, h666777, Grobuk, Sad-Flatworm-3240, Weird_Internet_2642, JoeySalmons, HowWasRoyadinTaken, Individual-Web-5391, Mimotive11, Wasted_Election_8361, sorosa, SnussyFoo, disposable66, Samueras, BlueEye1814, RedX07, og_mrrubberducky, Gr3y_Matter, HornyMonke1, jmsfindorff, ShiftShido, Cool-Hornet4434, Agreeable_apraline_15, This_Speaker_6767, Comprehensive-Joke13, amanpgmh, guchdog IZA_does_the_art, Geechan, Targren, SensitiveFlamingo12, Jarwen87, pixelnull, DoJo_Mast3r, BeneficialScheme6010, Alternative-Fox1982, CommonPurpose1969, National_cod9546.