model
Bonsai {1, 1.58}-bit GPU
Experimental low-bit quantization model, currently only suitable for technical exploration
4.8Overall
Utility4
Onboarding5
Craft5
Niche fit7
Longevity3
Five dimensions scored independently; overall is a weighted average. Scores are only comparable within this same rubric.
Good for
Researchers in quantization or developers wanting to experiment with extreme LLM compression
Not for
Production use or users needing stable inference
Project description
Run {1, 1.58}-bit Bonsai LLMs on GPUs
Alternatives
GPTQAWQbitsandbytes