Some quick (non imatrix) quants I made for release.
I recommend using other quanters over me as they're better at it than I am.
- Downloads last month
- 1,287
Hardware compatibility
Log In to add your hardware
4-bit
6-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support