!!! danger
[TOC2]
[05-31-2023] - Added Wizard Vicuna 30B Uncensored
[05-29-2023] - Added Chronos 13B
[05-26-2023] - Added BluemoonRP 30B
[05-24-2023] - Added SuperHOT Prototype
!!! note VRAM Required takes full context (2048) into account. You may be able to load the model on GPU's with slightly lower VRAM, but you will not be able to run at full context. If you do not have enough RAM to load model, it will load into swap. Groupsize models will increase VRAM usage, as will running a LoRA alongside the model.
Model Parameters | VRAM Required | GPU Examples | RAM to Load
--- | --- | --- | ---
7B | 8GB | RTX 1660, 2060, AMD 5700xt, RTX 3050, RTX 3060, RTX 3070 | 6 GB
13B | 12GB | AMD 6900xt, RTX 2060 12GB, 3060 12GB, 3080 12GB, A2000 | 12GB
30B | 24GB | RTX 3090, RTX 4090, A4500, A5000, 6000, Tesla V100 | 32GB
65B | 42GB | A100 80GB, NVIDIA Quadro RTX 8000, Quadro RTX A6000 | 64GB
!!! note 5bit to 8bit Quantized models are becoming more common, and will obviously require more RAM. Will update these with the numbers when I have them.
Model | 4-bit | 5-bit | 8-bit
--- | --- | --- | ---
7B | 3.9 GB
13B | 7.8 GB
30B | 19.5 GB
65B | 38.5 GB
!!! info
Model | Type | Download
--- | --- | ---
7B 16bit | HF Format | [HuggingFace](https://huggingface.co/Neko-Institute-of-Science/LLaMA-7B-HF)
13B 16bit | HF Format | [HuggingFace](https://huggingface.co/Neko-Institute-of-Science/LLaMA-13B-HF)
30B 16bit | HF Format | [HuggingFace](https://huggingface.co/Neko-Institute-of-Science/LLaMA-30B-HF)
65B 16bit | HF Format | [HuggingFace](https://huggingface.co/Neko-Institute-of-Science/LLaMA-13B-HF)
All the above | HF Format | [Torrent Magnet](magnet:?xt=urn:btih:8d634925911a03f787d9f68ac075a9b24281573a&dn=Safe-LLaMA-HF-v2%20(4-04-23)&tr=http%3a%2f%2fbt2.archive.org%3a6969%2fannounce&tr=http%3a%2f%2fbt1.archive.org%3a6969%2fannounce)
!!! info
Model | Type | Download
--- | --- | ---
7B, 13B, 30B, 65B | CPU (old format) | [Torrent Magnet](magnet:?xt=urn:btih:481dee5424b7024433504803a90efd32dae40fdf&dn=LLaMA-ggml-4bit_2023-03-31&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=udp%3a%2f%2ftracker.altrosky.nl%3a6969%2fannounce&tr=udp%3a%2f%2fopentracker.i2p.rocks%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=https%3a%2f%2fopentracker.i2p.rocks%3a443%2fannounce&tr=udp%3a%2f%2ftracker.theoks.net%3a6969%2fannounce&tr=udp%3a%2f%2ftracker-udp.gbitt.info%3a80%2fannounce&tr=udp%3a%2f%2ftracker.tiny-vps.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6969%2fannounce&tr=udp%3a%2f%2f9.rarbg.com%3a2810%2fannounce&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.monitorit4.me%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler.org%3a6969%2fannounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker2.dler.org%3a80%2fannounce)
7B, 13B, 30B, 65B | CPU (new format) | [7B](https://huggingface.co/TheBloke/LLaMa-7B-GGML), [13B](https://huggingface.co/TheBloke/LLaMa-13B-GGML), [30B](https://huggingface.co/TheBloke/LLaMa-30B-GGML)
7B, 13B, 30B, 65B | GPU CUDA (no groupsize) | [Torrent Magnet](magnet:?xt=urn:btih:e88abf1b84290b162f00d3a9d79fb4f8719c2053&dn=LLaMA-HF-4bit&tr=http%3a%2f%2fbt2.archive.org%3a6969%2fannounce&tr=http%3a%2f%2fbt1.archive.org%3a6969%2fannounce)
7B, 13B, 30B, 65B | GPU CUDA (128gs) | [Torrent Magnet](magnet:?xt=urn:btih:88f7d9d2460ffcaf78b21e83012de00939eacb65&dn=LLaMA-HF-4bit-128g&tr=http%3a%2f%2fbt2.archive.org%3a6969%2fannounce&tr=http%3a%2f%2fbt1.archive.org%3a6969%2fannounce)
7B, 13B, 30B, 65B | GPU Triton | [Neko Institute of Science HF page](https://huggingface.co/Neko-Institute-of-Science)
!!! info
Model | Type | Download
--- | --- | ---
30B GGML | CPU | [Q4_0, Q4_1, Q5_0, Q5_1, Q8](https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GGML)
30B | GPU | [Q4](https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ)
!!! info
Model | Type | Download
--- | --- | ---
13B GGML | CPU | [Q4_0, Q4_1, Q5_0, Q5_1, Q8](https://huggingface.co/TheBloke/chronos-13B-GGML)
13B | GPU | [Q4 CUDA 128g](https://huggingface.co/elinas/chronos-13b-4bit), [Q4 CUDA](https://huggingface.co/Yhyu13/chronos-13b-gptq-4bit), [Q4 Triton](https://huggingface.co/TheYuriLover/chronos-13b-GPTQ-Triton)
!!! info
Model | Type | Download
--- | --- | ---
30B GGML | CPU | [Q5_0](https://huggingface.co/reeducator/bluemoonrp-30b)
30B | GPU | [Q4 CUDA 128g](https://huggingface.co/reeducator/bluemoonrp-30b)
!!! info
Model | Type | Download
--- | --- | ---
30B LoRA | LoRA | [HF Link](https://huggingface.co/kaiokendev/SuperHOT-LoRA-prototype)
30B GGML | CPU | None yet.
30B | GPU | [Q4 CUDA 128g](https://huggingface.co/ausboss/llama-30b-SuperHOT-4bit-128g)
!!! info
Model | Type | Download
--- | --- | ---
13B 16-bit | Unquantized | [HF Link](https://huggingface.co/openaccess-ai-collective/manticore-13b)
13B GGML | CPU | [Q4_0, Q4_1, Q5_0, Q5_1, Q8](https://huggingface.co/TheBloke/Manticore-13B-GGML)
13B | GPU | [Q4 CUDA 128g](https://huggingface.co/TheBloke/Manticore-13B-GPTQ)
https://huggingface.co/kaiokendev/SuperHOT-LoRA-prototype
!!! info
Model | Type | Download
--- | --- | ---
30B 16-bit | Unquantized | [HF Link](https://huggingface.co/ehartford/WizardLM-30B-Uncensored)
30B GGML | CPU | [Q4_0, Q4_1, Q5_0, Q5_1, Q8](https://huggingface.co/TheBloke/WizardLM-30B-Uncensored-GGML)
30B | GPU | [Q4 Triton](https://huggingface.co/TheBloke/WizardLM-30B-Uncensored-GPTQ/tree/main)
!!! info
Model | Type | Download
--- | --- | ---
13B 16-bit | Unquantized | [HF Link](https://huggingface.co/openaccess-ai-collective/manticore-13b)
13B GGML | CPU | [Q4_0, Q4_1, Q5_0, Q5_1, Q8](https://huggingface.co/TheBloke/Manticore-13B-GGML)
13B | GPU | [Q4 CUDA 128g](https://huggingface.co/TheBloke/Manticore-13B-GPTQ)
!!! info
Model | Type | Download
--- | --- | ---
13B Pygmalion/Metharme XOR | XOR | https://huggingface.co/PygmalionAI/
13B Pygmalion GGML | CPU | [Q4_0, Q4_1, Q5_0, Q5_1, Q8](https://huggingface.co/notstoic/pygmalion-13b-ggml/tree/main)
13B Metharme GGML | CPU | [Q4_1, Q5_1, Q8](https://huggingface.co/TehVenom/Metharme-13b-GGML/tree/main)
13B Pygmalion | GPU | [Q4 CUDA 128g](https://huggingface.co/notstoic/pygmalion-13b-4bit-128g)
13B Metharme | GPU | [Q4 CUDA 128g](https://huggingface.co/TehVenom/Metharme-13b-4bit-GPTQ)
!!! info
Model | Type | Download
--- | --- | ---
LoRA | LoRA | [HF Link](https://huggingface.co/Neko-Institute-of-Science/VicUnLocked-30b-LoRA)
30B GGML | CPU | [Q4_0, Q4_1, Q5_0, Q5_1, Q8](https://huggingface.co/TheBloke/VicUnlocked-30B-LoRA-GGML)
30B | GPU | [Q4 Triton](https://huggingface.co/TheBloke/VicUnlocked-30B-LoRA-GPTQ), [Q4 CUDA](https://huggingface.co/QMB15/VicUnlocked-30B-gptq-cuda)
!!! info
Model | Type | Download
--- | --- | ---
13B GGML | CPU | [Q4_0, Q5_0, Q5_1, Q8](https://huggingface.co/TheBloke/wizard-mega-13B-GGML)
13B | GPU | [Q4 CUDA 128g](https://huggingface.co/TheBloke/wizard-mega-13B-GPTQ)
!!! info
Model | Type | Download
--- | --- | ---
13B GGML | CPU | [Q4, Q5, Q8](https://huggingface.co/TheBloke/WizardLM-13B-Uncensored-GGML)
13B | GPU | [Q4 CUDA 128g](https://huggingface.co/ausboss/WizardLM-13B-Uncensored-4bit-128g)
!!! info
Model | Type | Download
--- | --- | ---
13B GGML | CPU | [Q5](https://huggingface.co/reeducator/bluemoonrp-13b)
13B | GPU | [Q4 CUDA 128g](https://huggingface.co/reeducator/bluemoonrp-13b)
!!! info
Model | Type | Download
--- | --- | ---
13B GGML | CPU | [Q5, Q8](https://huggingface.co/reeducator/vicuna-13b-cocktail)
13B | GPU | [Q4 CUDA 128s, Q4 Triton](https://huggingface.co/reeducator/vicuna-13b-cocktail)
!!! info
Model | Type | Download
--- | --- | ---
30B GGML | CPU | Awaiting re-quantization
30B | GPU | [Q4 CUDA](https://huggingface.co/askmyteapot/GPT4-x-AlpacaDente2-30b-4bit)
https://huggingface.co/askmyteapot/GPT4-x-AlpacaDente2-30b-4bit
!!! info
Model | Type | Download
--- | --- | ---
13B GGML | CPU | [Q5, f16](https://huggingface.co/reeducator/vicuna-13b-free)
13B | GPU | [Q4 CUDA 128g](https://huggingface.co/reeducator/vicuna-13b-free)
!!! info
Model | Type | Download
--- | --- | ---
7B Pygmalion/Metharme XOR | XOR | https://huggingface.co/PygmalionAI/
7B Pygmalion GGML | CPU | [Q4_3](https://huggingface.co/xzuyn/Pygmalion-V3-6B-ggml-q4_3), [Q5_0](https://huggingface.co/xzuyn/Pygmalion-V3-6B-ggml-q5_0), [Q5_1](https://huggingface.co/xzuyn/Pygmalion-V3-6B-ggml-q5_1), [Q8](https://huggingface.co/xzuyn/Pygmalion-V3-6B-ggml-q8_0)
7B Metharme GGML | CPU | [Q4_1](https://huggingface.co/waifu-workshop/metharme-7b-ggml-q4_1), [Q5_1](https://huggingface.co/waifu-workshop/metharme-7b-ggml-q5_1), [Q8](https://huggingface.co/waifu-workshop/metharme-7b-ggml-q8_0), [f32](https://huggingface.co/waifu-workshop/metharme-7b-ggml-f32)
7B Pygmalion | GPU | [Q4 Triton](https://huggingface.co/TehVenom/Pygmalion-7b-4bit-GPTQ-Safetensors), [Q4 CUDA 128g](https://huggingface.co/gozfarb/pygmalion-7b-4bit-128g-cuda)
7B Metharme | GPU | [Q4 Triton](https://huggingface.co/TehVenom/Metharme-7b-4bit-GPTQ-Safetensors), [Q4 CUDA](https://huggingface.co/askmyteapot/metharme)
!!! info
Model | Type | Download
--- | --- | ---
30B GGML | CPU | [Q4_0](https://huggingface.co/spanielrassler/GPT4-X-Alpasta-30b-ggml)
30B | GPU CUDA | [Q4 CUDA, Q4 CUDA 128g](https://huggingface.co/MetaIX/GPT4-X-Alpasta-30b-4bit)
!!! info
Model | Type | Download
--- | --- | ---
30B XOR | XOR | https://huggingface.co/OpenAssistant/oasst-sft-7-llama-30b-xor
30B GGML | CPU | [Q4_0, Q5_0, Q5_1, Q8](https://huggingface.co/TheBloke/OpenAssistant-SFT-7-Llama-30B-GGML)
30B | GPU | [Q4 CUDA](https://huggingface.co/Peeepy/llama-33b-oasst-4bit), [Q4 CUDA 128g](https://huggingface.co/Peeepy/llama-30b-oasst-4bit-128g)
!!! info
Model | Type | Download
--- | --- | ---
Original LoRA | LoRA | https://huggingface.co/kaiokendev/SuperCOT-LoRA
13B GGML | CPU | [Q5_0](https://huggingface.co/xzuyn/Alpacino-SuperCOT-13B-ggml-q5_0), [f16](https://huggingface.co/camelids/llama-13b-supercot-ggml-f16)
30B GGML | CPU | [Q4_1](https://huggingface.co/camelids/llama-33b-supercot-ggml-q4_1), [Q5_1](https://huggingface.co/camelids/llama-33b-supercot-ggml-q5_1)
13B | GPU | [Q4 CUDA 128g](https://huggingface.co/ausboss/llama-13b-supercot-4bit-128g)
30B | GPU | [Q4 CUDA](https://huggingface.co/tsumeone/llama-30b-supercot-4bit-cuda), [Q4 CUDA 128g](https://huggingface.co/tsumeone/llama-30b-supercot-4bit-128g-cuda)
!!! info
!!! info
https://rentry.org/backupmdlist