 oobabooga
|
49c10c5570
Add support for the latest GPTQ models with group-size (#530)
|
2 years ago |
|
oobabooga
|
a6bf54739c
Revert models.py (accident)
|
2 years ago |
|
oobabooga
|
a80aa65986
Update models.py
|
2 years ago |
|
oobabooga
|
ddb62470e9
--no-cache and --gpu-memory in MiB for fine VRAM control
|
2 years ago |
|
oobabooga
|
e26763a510
Minor changes
|
2 years ago |
 Wojtek Kowaluk
|
7994b580d5
clean up duplicated code
|
2 years ago |
|
Wojtek Kowaluk
|
30939e2aee
add mps support on apple silicon
|
2 years ago |
|
oobabooga
|
ee164d1821
Don't split the layers in 8-bit mode by default
|
2 years ago |
|
oobabooga
|
e085cb4333
Small changes
|
2 years ago |
 awoo
|
83cb20aad8
Add support for --gpu-memory witn --load-in-8bit
|
2 years ago |
|
oobabooga
|
1c378965e1
Remove unused imports
|
2 years ago |
|
oobabooga
|
66256ac1dd
Make the "no GPU has been detected" message more descriptive
|
2 years ago |
|
oobabooga
|
265ba384b7
Rename a file, add deprecation warning for --load-in-4bit
|
2 years ago |
 Ayanami Rei
|
8778b756e6
use updated load_quantized
|
2 years ago |
|
Ayanami Rei
|
e1c952c41c
make argument non case-sensitive
|
2 years ago |
|
Ayanami Rei
|
3c9afd5ca3
rename method
|
2 years ago |
|
Ayanami Rei
|
edbc61139f
use new quant loader
|
2 years ago |
|
oobabooga
|
65dda28c9d
Rename --llama-bits to --gptq-bits
|
2 years ago |
|
oobabooga
|
fed3617f07
Move LLaMA 4-bit into a separate file
|
2 years ago |
 draff
|
001e638b47
Make it actually work
|
2 years ago |
|
draff
|
804486214b
Re-implement --load-in-4bit and update --llama-bits arg description
|
2 years ago |
 ItsLogic
|
9ba8156a70
remove unnecessary Path()
|
2 years ago |
|
draff
|
e6c631aea4
Replace --load-in-4bit with --llama-bits
|
2 years ago |
|
oobabooga
|
e9dbdafb14
Merge branch 'main' into pt-path-changes
|
2 years ago |
|
oobabooga
|
706a03b2cb
Minor changes
|
2 years ago |
|
oobabooga
|
de7dd8b6aa
Add comments
|
2 years ago |
|
oobabooga
|
e461c0b7a0
Move the import to the top
|
2 years ago |
 deepdiffuser
|
9fbd60bf22
add no_split_module_classes to prevent tensor split error
|
2 years ago |
|
deepdiffuser
|
ab47044459
add multi-gpu support for 4bit gptq LLaMA
|
2 years ago |
 rohvani
|
2ac2913747
fix reference issue
|
2 years ago |
