Local Models Related Links

/lmg/ Accelerate
Guides
Quick Start Guide Anon's tutorial for getting models running locally
SillyTavern Guide Instructions for roleplaying via koboldcpp. Additional GNBF grammar usage
LM Tuning Guide Training, fine-tuning, and LoRA/QLoRA information
LM Settings Guide Explanation of various settings and samplers with suggestions for specific models
LM GPU Guide Current as of the 40 series. Alternatively some Anons made a few different build guides
Models
HuggingFace Best source for current quants (filter by GGUF or EXL2)
OpenModelDB Specifically models for upscaling images and videos
Open TTS Tracker Open Text-To-Speech models with relevant use case information
Voice Models Easily searchable list for use mainly with RVC 1/2
Models Info Table Googlesheet of models, AI labs, datasets, and various other ML info by Alan Thompson
Chat Leaderboard Closed and local models ELO rated with additional MMLU/MT-bench scores
Papers
Local Models Papers Papers and articles I've found to be interesting with a way to search via abstracts
Arxiv ML Primary source of machine learning papers
PapersWithCode Indexer that allows sorting by GitHub stars
Semantic Scholar Scientific literature semantic search tool
Scholar Inbox ML focused paper recommendations based off personal preferences
News
AI Explained General AI news with well sourced links (Youtube)
AI News Blog Lesswrong cultist so "AI Bad" takes but does a good weekly AI news roundup (Blog)
ML Resources Broader sporadically updated list (not fully local)
Previous Threads Always good to search for previous questions before asking
Learn
LLM Course Collection of articles, videos, courses, and colabs for learning applied ML
Andre Karpathy YT In-depth videos of LLM construction from one of OpenAI's founding members
TF From Scratch Blogpost with Juypter notebook that goes step by step for coding and training a small GPT
LLM-Sampling Token Probability visualizer with support for current popular samplers
LLM Visualization Drag and pull 3D model of various LLMs with explanation for components
Intro to DNN Book format of a Neural Networks course that serves as in introduction to ML
Principles of DL Textbook that introduces the math behind Deep Learning
LLM Inferencing
Text Gen WebUI Frontend to most GPU/CPU model backends
WebUI Extensions Most notable XTTSv2 and Stable Diffusion
llama.cpp Main CPU inferencing development with GPU acceleration (GGUF models)
kobold.cpp llama.cpp fork with Kobold UI and additional features (with support for older GGML models)
exllama2 Inference library for local LLM with new quant style (70B llama2 on 24GB VRAM)
TabbyAPI FASTAPI application for exllama2 backend for use with SillyTavern
SillyTavern Frontend that is a heavily modified TavernAI fork
vllm Inference library with fast inferencing and PagedAttention for KV management
LLM Tools
Axolotl Fine-tuning tool for various architectures with integrated support for flash attention and rope scaling
QuaRot 4/6/8bit weight/activation/KV quantization scheme based on rotations to remove outliers
Mergekit Toolkit for merging LLMs including piecewise assembly of layers
promptfoo Tool for testing and evaluating LLM output quality also with side-by-side feature
Floneum Graph/node editor for AI workflows with a focus on community made plugins
FourierFT Fine-tune method using Fourier that uses significantly less added parameters than LoRA with improved performance
LLM Research
DropBP Randomly drops layers during backward propagation for quicker tuning with no reductions in accuracy
LASER Selectively remove higher-order components of LLM weight matrices for improved performance
SPIN Self-play fine-tuning method that generates its own training data from previous iterations
Temp LoRA Employs a temporary LoRA module during text generation to preserve contextual knowledge
HOMER Hierarchical context merging training-free method that works with conventional RoPE-scaling techniques
SKVQ KV cache quantization method using a sliding window technique that performs better than KIVI
LLM Guiding
Langchain Set of resources to maximize LLMs Chains/tool integrations/agents/etc.
llama_index Central interface to connect LLM's with external data
llama-hub Simple library of all the data loaders/readers for llama index/langchain
SGLang Structured generation language designed for LLM/VLMs
DSPy Composable and declarative modules for instructing LMs in a familiar Pythonic syntax
EasyEdit Knowledge editing framework for LLMs
Datasets
Huggingface Best source for datasets
Wiki Embeddings Predone embeddings for various language of Wikipedia
ERP Scrapes (1)(2) Raw RP/ERP/ELIT content
VN JP/EN Scrape 60 million tokens of dialogue and actions/narration
WN JP/EN Scrape 100k chapters of webnovels paired with fantranslations
janitorai-cards 190k character cards converted to v2 format and viewable as local webpage
chub.ai Archive of various character cards from chub as well as from some other sources
Dataset Tools
augmentoolkit Generates multi-turn instruct-tuning data from input documents
dswav Audio dataset preparation tool using whisper and ffmpeg to transcribe and split inputs
lilac Dataset curation tool for RAG or tuning with annotating/clustering/labeling support
Data-Juicer Dataset preparation tool with support for multimodal data
Non-LLM Models
Vision/Image
ComfyUI Node based stable diffusion GUI. User submitted workflows
Fabric ComfyUI Uses iterative feedback to personalize diffusion outputs
LDSR ComyUI Image super resolution upscaler with less artifacts than others but slower
Upscale Hub Set of resources and models for image and video upscaling (anime focused)
VAR Image Gen models based on next-resolution prediction rather than next-token that outperform DiT
DiffEditor Tuning-free method for fine-grained image editing using score-based diffusion
VideoMamba SSM to enable efficient memory usage for high resolution vision/video tasks
EfficientViT-SAM Faster and more accurate version of Segment Anything Model via EfficientViT
Depth-Anything Robust monocular depth estimation that works well with semantic segmentation
ProLab Semantic segmentation via property-level label space rather than just categories
LLaVa-NeXT Visual language models using qwen/llama3 with new video understanding capability
DeepSeek-VL VLM (1B and 7B) trained on OCR data that outperforms local 13B models and at OCR
ShareCaptioner Image captioning model with lower hallucinations than LLaVa
Spandrel Library for loading various upscaling models for use with chaiNNer or SD WebUI
SUPIR Image restoration and upscale method with semantic adjustment editing ability
DDColor Vivid and natural colorization for black and white photos (and possibly video)
lama-cleaner Local inpainting tool (remove or erase and replace)
GRM Gaussian Reconstruction Model that works with sparse view images or Text/Image-to-3D models
DreamReward Text-to-3D model fine-tuned with human preference data for superior generations
Ground-A-Video Video Editing via Text-To-Image diffusion models with groundings/motion/depth data
roop-cam Real time face swap with webcam and one click video support
MetaCLIP Improvement over the CLIP model with superior dataset quality pipeline
AniTalker Animate images to audio with face reenactment/cloning ability
loopvlm Real time multimodal PaliGemma usage (with a 4090) for video workloads like segmantic segmentation
Audio/Speech
Amphion Audio/Music/Speech toolset of various models with visualization capability
GPT-SoVITS Few-shot voice cloning and Text-to-Speech WebUI (ENG/JPN/CHN)
VoiceCraft Zero shot Text-to-Speech and speech editing model with voice cloning capability
StyleTTS2 English Text-to-Speech via style diffusion (can fine-tune with custom dataset)
Qwen-Audio Audio (speech and music) instruction tuned multimodal LLM
whisper.cpp Speech-to-Text inference library with CPU/GPU support for various whisper based models
AudioEditing Zero-shot unsupervised and text-based audio editing using DPPM inversion
RVC Retrieval based Voice Conversation model
Urhythmic Unsupervised rhythm modeling for voice conversion
Anticipation Text-to-Music based on anticipatory infilling (MIDI currently)
Descrpyt High-Fidelity audio compression with improved RVQGAN (can drop-in replace EnCodec)
DeepFilterNet Real time noise suppression using deep filtering
UVR Audio source separation GUI for various models with full Demucs and MDX23C support
AudioSR Audio super resolution (any -> 48kHz)
EAT Audio and speech classification
Other
AnythingLLM RAG and agent focused frontend with support for local and cloud models
T-Ragx Translation fine-tune method that works with RAG (glossaries) and preceding text
GenTranslate Fine-tune of SeemlessM4T from N-best hypotheses dataset for MT and Speech-to-Text
Dragon+ Dual-encoder based dense retriever for use with the RA-DIT FT approach with paired LLM
Magica File content type detector model
AutoACT Automatic agent learning framework using a division-of-labor strategy
LOCUST State-space model for long document abstractive summarization
LongEmbed Context window extention methods applied to embedding models for SOTA 4k+ context
ESPN GPUDirect Storage implementation for multi-vector embedding retrieval and bindings
FastFit Text few-shot classification fine-tuning method with high accuracy and fast training time
Edit
Pub: 20 Mar 2023 19:58 UTC
Edit: 18 May 2024 15:30 UTC
Views: 56757