Model support with https://github.com/0cc4m/KoboldAI latestgptq and https://github.com/0cc4m/GPTQ-for-LLaMa latestmerge

Chart with models: https://github.com/underlines/awesome-marketing-datascience/blob/master/awesome-ai.md#llama-models

Working 6/7b:

https://huggingface.co/ozcur/alpaca-native-4bit
https://huggingface.co/Monero/llama-metharme-7b-4bit
https://huggingface.co/mayaeary/pygmalion-6b-4bit-128g
https://huggingface.co/gozfarb/pygmalion-7b-4bit-128g-cuda
https://huggingface.co/Aitrepreneur/wizardLM-7B-GPTQ-4bit-128g https://huggingface.co/TheBloke/wizardLM-7B-GPTQ
https://huggingface.co/TheBloke/vicuna-7B-1.1-GPTQ-4bit-128g
https://huggingface.co/TehVenom/Pygmalion-7b-4bit-GPTQ-Safetensors

Working 12/13b:

https://huggingface.co/4bit/llama-13b-4bit-gr128
https://huggingface.co/gozfarb/llygmalion-13B-4bit-128g-cuda
https://huggingface.co/digitous/Alpacino13b
https://huggingface.co/mzedp/dolly-v2-12b-GPTQ-4bit-128g
https://huggingface.co/notstoic/OPT-13B-Erebus-4bit-128g
https://huggingface.co/tsumeone/gpt4-x-alpaca-13b-native-4bit-128g-cuda https://huggingface.co/anon8231489123/gpt4-x-alpaca-13b-native-4bit-128g
https://huggingface.co/wcde/llama-13b-4bit-gr128
https://huggingface.co/KoboldAI/OPT-13B-Nerybus-Mix
https://huggingface.co/gozfarb/oasst-llama13b-4bit-128g
https://huggingface.co/gozfarb/llama-13b-pretrained-sft-do2-4bit-128g
https://huggingface.co/verymuchawful/llama-13b-pretrained-dropout-hf-int4-128g
https://huggingface.co/tsumeone/stable-vicuna-13B-4bit-128g-cuda https://huggingface.co/TheBloke/stable-vicuna-13B-GPTQ
https://huggingface.co/anon8231489123/vicuna-13b-GPTQ-4bit-128g
https://huggingface.co/tsumeone/vicuna-13B-1.1-GPTQ-4bit-128g-cuda
https://huggingface.co/tsumeone/wizard-vicuna-13b-4bit-128g-cuda
https://huggingface.co/ausboss/WizardLM-13B-Uncensored-4bit-128g
https://huggingface.co/reeducator/vicuna-13b-free
https://huggingface.co/GamerUntouch/LLaMa-Storytelling-4Bit
https://huggingface.co/reeducator/vicuna-13b-cocktail
https://huggingface.co/reeducator/bluemoonrp-13b

Working 30b:

https://huggingface.co/tsumeone/llama-30b-supercot-4bit-cuda https://huggingface.co/tsumeone/llama-30b-supercot-4bit-128g-cuda https://huggingface.co/tsumeone/llama-30b-supercot-3bit-128g-cuda
https://huggingface.co/TheBloke/OpenAssistant-SFT-7-Llama-30B-GPTQ
https://huggingface.co/MetaIX/GPT4-X-Alpasta-30b-4bit
https://huggingface.co/MetaIX/Alpaca-30B-Int4-Safetensors https://huggingface.co/MetaIX/Alpaca-30B-Int4-128G-Safetensors https://huggingface.co/MetaIX/Alpaca-30B-Int4
https://huggingface.co/MetaIX/GPT4-X-Alpaca-30B-4bit
https://huggingface.co/MetaIX/OpenAssistant-Llama-30b-4bit

Working 65b:

https://huggingface.co/camelids/llama-65b-int4-gptq-groupsize128-safetensors
https://huggingface.co/Aeala/Alpaca-elina-65b-4bit

Not working, may not be up to date:

https://huggingface.co/TheBloke/koala-7B-GPTQ-4bit-128g
https://huggingface.co/TheBloke/koala-13B-GPTQ-4bit-128g
https://huggingface.co/Aeala/Alpaca-elina-65b-4bit
https://huggingface.co/wojtab/llava-13b-v0-4bit-128g

???:

https://huggingface.co/Bradarr/toolpaca-13b-native-4bit-128g-cuda
https://huggingface.co/4bit/stable-vicuna-13B-GPTQ
https://huggingface.co/TheBloke/gpt4-alpaca-lora-13B-GPTQ-4bit-128g
https://huggingface.co/catalpa/codecapybara-4bit-128g-gptq
https://huggingface.co/kajdun/llama-30b-4bit-128g
https://huggingface.co/TheBloke/medalpaca-13B-GPTQ-4bit
https://huggingface.co/ausboss/llama-13b-supercot-4bit-128g
https://huggingface.co/teknium/GPT4-x-Alpaca13b-RolePlayLora-4bit-v2
https://huggingface.co/Selyam/gpt4-x-alpaca-13b-native-4bit-128g
https://huggingface.co/TheBloke/vicuna-13B-1.1-GPTQ-4bit-128g
https://huggingface.co/4bit/vicuna-13b-GPTQ-4bit-128g
https://huggingface.co/TheBloke/vicuna-AlekseyKorshuk-7B-GPTQ-4bit-128g
https://huggingface.co/gozfarb/vicuba-7b-int4-128g
https://huggingface.co/gozfarb/instruct-13b-4bit-128g
https://huggingface.co/elinas/alpaca-30b-lora-int4
https://huggingface.co/tloen/alpaca-lora-7b/
https://huggingface.co/elinas/alpaca-13b-lora-int4
https://huggingface.co/Dogge/alpaca-13b
https://huggingface.co/TheBloke/alpaca-lora-65B-GPTQ-4bit
https://huggingface.co/TianXxx/llama-30b-int4
https://huggingface.co/kuleshov/llama-30b-4bit
https://huggingface.co/elinas/llama-30b-int4
https://huggingface.co/TehVenom/Metharme-7b-4bit-32g-GPTQ-Safetensors

Edit
Pub: 02 May 2023 15:24 UTC
Edit: 10 May 2023 18:00 UTC
Views: 998