Evening-Truth

LLM 101

Home

How AI Actually Works and what the hell are samplers???

Hello pumpkin,
welcome to class. Pick a blanket, a cup of tea and let's dive right into it.

So, you are digging deeper into the world of LLM and AI chat? And now you read about weird things like Tokens, Temperature, Top P, Repetition Penalty or Jailbreak?
Someone told you to not touch them.... someone else said you must change a sampler?

Don't let that overwhelm you!
Lemme explain...

Disclaimer: This is designed as a functional introduction to LLM lingo. I skipped the boring math on purpose.

I have simplified the complex backend logic into concepts that make sense for writers and roleplayers. I traded "technical perfection" for "understandability" (yes, that’s a word).
If you are a computer scientist looking for logit probability distributions, this isn't the guide for you, sweetcheeks.
If you want to dig deeper into the topic... baby... go forth and be a wonderful menace.


AI vs LLM

Why I avoid using the term Artificial Intelligence.

Because there is no intelligence behind it and by naming Large Language Models (short LLM) intelligent, people who use it have certain expectations.
You wouldn't call wikipedia intelligent, right? Right!

The facts:
LLMs generate their response based on math, probabilities, and training data. They are designed to be agreeable, attentive and reactive to you. They mirror our vibe, our desires and our style of communication. They will try to validate what you say at all costs.
An interesting example is that you can convince some LLMs that 2+2=5.
Let that sink in and remember it the next time you ask an LLM to check your paper for factual mistakes. πŸ˜‰


What the hell is a Token?

I know, you've seen that word floating around a lot.

Your AI companion, as cute as they are, decide what naughty/flirty messages they give you by doing tons of math. And since words and math don't mix so good, very smart brains translated words and sub-words into tokens.
For more on that topic, google "Byte-Pair Encoding" and "Philip Gage". Or stay here... I have cookies. πŸ˜‰

Now every word is one or more tokens with their own distinct ID like a dictionary.

Example:
The word "Apple" is Token #17060.
The word "Strawberry" is a combination of token #46356 "Straw", token #711 "ber", and token #432 "ry".

The ID makes the word understandable and the math for the next likely token calculateable for AI. (I might have just invented the word calculateable. Not sure.) Also, the ID changes depending on what dictionary the company uses. Is that important for you? No. Just nice to know.

So, whenever you see context sizes, response lengths or whatnot measured in tokens.... you now know they are talking about words and sub-words translated into numbers.

Interesting to know:
Shakespeare's Hamlet is approximately 40.000 tokens.
The Great Gatsby by F. Scott Fitzgerald is somewhere around 63.000 tokens.


What AI remembers

The mean truth is... AI remembers nothing about your chat.
I know what you're thinking.
"But it does remember what I said 12 messages earlier. Evening-Truth... don't lie to me!!!"

Fact is, the human concept of "remembering" is different to how AI remembers.

Your message is only one part of what is sent to the LLM in what is called a request. To keep your roleplay immersive and to not bother your with the technical stuff, you will only see your conversation with the bot.
The LLM receives a request that includes some technical stuff like Sampler Values, model ID, a set of rules that explains the AI what it has to do (my System Prompts for example), the description of the character you are talking to, the description of your persona, descriptions of the world and lore, the messages you have exchanged with the model so far, and the message you just wrote.

So, quite a lot of text... or tokens. πŸ˜‰

The LLM doesn't have a specific place on the server for your conversation with it, so it does something that seems strange from a human perspective but is actually very smart from the technical side.

Let's call the request with everything it contains a book.
Now, everytime you write a message you are writing on the first free page in that book and send it back to the LLM. Since the LLM is computing big amounts of requests every second of the day, it takes the book you just send, reads it from front to end and generates a message that fits to everything that it just read.

You receive that message but only have to read that one message to know that it fits with what happened before.

Your next message to the model will be processed exactly the same... the entire book is sent, the LLM reads it again from front to back and generates the reply.

If you leave the chat or switch to another character, the LLM gets a completely different book to read and reply to.


"What is context size?"

There are LLM's that have a context size of 2 million tokens, and others that have 30.000 tokens. To put that into perspective

  • Shakespear's Hamlet is approximately 40.000 tokens,
  • The Great Gatsby by F. Scott Fitzgerald is somewhere around 63.000 tokens.

So the context size tells you how big the book you send to the LLM can be. How much data/tokens/words it can process overall... but that's just technical.
For roleplay and creative writing, context coherency is more important.


"What is context coherency?"

This tells you how good the LLM is in "remembering" or focusing on details of what you talked about earlier in your roleplay and that number is drastically lower than the context size.

LLMs try to understand what parts of your conversation are important. Which is relatively easy in factual tasks like coding, statistic analysis, technical documentation and so on. In creative writing and roleplay, that's a tricky bit since there is a lot of prose, background info and rules to stay on top of.

In the olden days (Translate: a few months ago) context coherency started to drop around 9K tokens already. So the AI very very soon would start to forget what you talked about before, or to stay with the book metaphor, what it read 5 pages earlier.

With all the magic and mysteries and shenanigans happening in the AI world, we now have models that can handle up to 30K tokens and process them coherently in roleplay and creative writing.

That's almost the entire play of Hamlet and the AI still would know that Claudius is the bad guy!


"What happens when the context size is full? Do I lose the chat?"

No. Your chat will always go on. But the AI will start to forget things. To make sure it doesn't forget the important parts, smart brains made a difference between permanent tokens and temporary tokens.

Permanent tokens are information that will always be sent in your requests to AI. Things like the system prompt, character descriptions, designated memories, and other technical bits and pieces, depending on the platform you chat on.

Your chat history with the bot are temporary tokens or temporary information. That means when the context size of your chat is filled and to keep the chat running, the oldest messages of the conversation drop out of the book. AI nerds call it the rolling context window

Here's a little bit of math for you.

Let's take the saucepan model Hirosaki.
The bot you're talking to contains 3.000 permanent tokens. The initial message from that companion has 600 tokens.
You have an advanced prompt of 2.500 permanent tokens (which you don't because you are using my prompts that are highly token efficient, right? RIGHT???)
The conversation you had so far with the companion is roughly 3.000 tokens. (approximately three to five back and forth messages.)

That's
3.000 + 600 + 2.500 + 3.000
= 9.100 tokens in total

The context size of Hirosaki is 24.000 tokens.
So you have a decent amount of tokens free for shenanigans with your companion without even coming close to the context window. πŸ˜‰

Now go forth and traumatize some algorithms.


Samplers explained without tech speech 😘

The deeper you dive into AI and LLM the more technical bits and pieces you will find.
Samplers are one of them and googling them often ends up with a very technical explanation that leaves you with more questions than you had before.
Especially the question "Why did I bother googling that in the first place?"

You feel that, right?
Come in, love.
Pick a plushy, make yourself cozy, and let me explain it without messing up your synapses.

What are Samplers?

Samplers control rods that decide how the AI chooses its words/tokens while generating a reply for you.
They all work together but to keep it simple, I'll just explain what they do.

Today we'll be talking about the most important ones.

  • Temperature
  • Top P
  • Top K

Temperature

The metaphor:

You hire a writer (AI) to create a text for you. With Temperature you control how creative the writer is. How much they are allowed to stray from the next obvious word choice.
The lower you set the Temperature, the less creative the writer will be.

Low (0.1 - 0.5):
The overly careful writer that only uses the most obvious words.
This is mostly used for coding and factual tasks. Not useful for roleplay and creative writing.

  • Only the most likely and statistically safest words are picked.
  • You try to fix the deliciously broken Biker-bot but they wont let you? Set the Temperature higher.

Medium (0.7 - 1.1):
The creative writer with a script.
Allows for creative ideas without forgetting the initial task.

  • The sweet spot for most modern AIs to write creative and in character. But also highly depending on the model.
  • Deepseek, Qwen and Mistral can mostly handle up to 1.1. GLM, Kimi K2 and many other LLMs start to hallucinate wildly at 1.0

High (1.2 - 2.0):
The writer that's high on hallucinogens or the WTF Setting.
The AI will be highly creative and picks even the most unlikely/risky words

  • Ever seen random Kanji or Sanskrit letters in the replies? The sexy secretary was just abducted by a horde of squirrels in space suits? Yeah... lower the Temperature.

Top P

Controls the pool of words the AI can choose from. This also has an influence on the creativity of the AI.

The metaphor:

The writer you hired knows every word there is. They look at all the words that make sense in the given context (adding up to the value you set) and ignore the absolute nonsense. In clear situations, the list is short; in vague ones, it's long.

1.0 (or 100%):
No filter. The writer considers every possible word in its dictionary.

  • The model uses words that feel strangely scientific? Or the replies contain words that were common in the Victorian era, but you are currently in space with a leaking oxygen tank?
    Lower Top P!

0.99 to 0.95 (or 99 to 95%):
The standard for roleplay and creative writing. It cuts out the bottom 1% to 5% of "junk" words that make no sense, but keeps the variety.

0.8 and below (or 80% and below):
High filter.

  • The bot describes the eye color of the character as "blue like a storm over the ocean" or repeats the example dialogue verbatim, every damn time?
  • Your roleplay has the immersion of a tax slip?
    **Turn up Top P! **

Top K

A "Hard Limit" on word choices. **This Sampler is highly restrictive! **

We don't like it... say it with me... "WE DON'T LIKE TOP K."
Well done, Baby. That's a good boy/girl/pet right there. ;-)

The metaphor:

You tell your writer to ONLY pick from the top 10 most obvious words for the current sentence.

10 to 100:

The AI will ignore any word under that value.
The lower the setting the more restrictive. That perfect, unique word you'd want to read. Yeah... that's outside the Top K you set.

  • The AI turns your wholesome neighbour bot into a growling, territorial werewolf?
  • You get replies that read like the AO3 post with a low rating?
  • ClichΓ© phrases like "their words hit like a physical blow.", "You are mine. And I don't share what's mine.", "Shivers running down spines", "The scent of ozone.".
    That's Top K and why we do not like it. πŸ˜‰
    We want our Companions to write freely and creatively. So we disable Top K or at least choose the highest possible setting.

Interesting facts

  • Z.ai models default to Top P 0.6 when you give it any restrictive Samplers.
  • Older AI architectures needed this to not go berserk.

Other Samplers

There are a lot of them. Here's an overview just so you know they exist... forgetting them is highly encouraged. We don't need them as long as you're not in Computer Science.

  • Dynamic Temperature
  • XTC
  • DRY Sampling
  • Smoothing Factor and Curve
  • CFG Scale
  • Min P / Typical P
  • Top A
  • Mirostat Tau / Mirostat Learning Rate
  • ETA Cutoff
  • Epsilon Cutoff
  • Squirrel Enhancer

Now take a break you wonderful menace!
Drink some water or touch grass before we continue.


The Penalties (Repetition)

info: this section is not finished yet. Gimme a little more time. 😘

Slapping the writer for repeating itself

  • Repetition Penalty
  • Frequency Penalty
    Penalises tokens based on how often they have already been used. The more times a token has appeared so far, the more its probability score is reduced.
    Your writer has to pay a fee everytime they use a word they have used before. And everytime they use it again, the fee gets higher.
    Example:
    The word "shadow"
    The first use is free, the second use costs 0.10 credits, the third costs 0.20 credits, the fourth 0.40 credits ... and so on.
    With your writer being "frugal," they will eventually decide that "shadow" is too expensive and will look for a cheaper alternative, like "gloom", "silhouette", or "darkness". They will dig deeper into their vocabulary to avoid verbatim repetition.
  • Presence Penalty
    Penalises tokens that have already been used in the text but instead of increasing the penalty like Frequency Penalty, it only "charges" once.
    Your writer has a list of words they've already used in the text. If they use a word from that list it costs one flat fee to use it again. No matter if they used it once or fifty times. So to avoid the fee the writer will more likely steer towards new topics.
    This sampler doesn't care as much about "word echoes" as it cares about new ideas.

** The Instructions (Prompts)**

What you tell the AI to do.

  • System Prompt
  • Jailbreak / Post-History Instructions
  • Assistant Prefill
Edit

Pub: 30 Nov 2025 14:07 UTC

Edit: 22 Mar 2026 16:13 UTC

Views: 251