YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
This is a high quality quantization of Yi-VL-6B and of the visual transformer. Q5_K is almost equal to fp16 in inference and Q6_K is basically the same (not as reliably tested for visual transformers but I assume it's equal to language models) You currently need to apply this PR to make it work: https://github.com/ggerganov/llama.cpp/pull/5093 - this adds the additional normalization steps into the projection
I do not like this model, it's hallucinating more than anything else based on llava.
- Downloads last month
- 56
Hardware compatibility
Log In to add your hardware
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support