Shittiest Performing Model
The model has no known presets, other than the developer's (chatml) general declaration. This model is by far the shittiest performing model I have ever tested. It responds in complete different context, doesn't speak at all at times. it's just pure shit.
skill issue
default settings work fine. or you can read and learn.
https://rpwithai.com/understanding-sampler-settings-for-ai-roleplay/
The only issue with this version of Snowpiecer is the reduced context length from 65536 to 2048 although I generally use 8192 as my default. As for performance and quality its thedrummer, all of the models are superb !!
The only issue with this version of Snowpiecer is the reduced context length from 65536 to 2048 although I generally use 8192 as my default. As for performance and quality its thedrummer, all of the models are superb !!
Where do you see that? Based on the config.json of the model, it's set to 65k tokens. https://huggingface.co/TheDrummer/Snowpiercer-15B-v4/blob/main/config.json
I'm not super familiar with smaller mistral models so i can't say when things start to devolve with them, but i would assume 8-16k should be fine like their bigger 24B variants.
I'd also assume the model uses Mistral-V7-Tekken for Context/Instruct templates. That's what i generally use for the 24B's and it works well for me.
the context is not reduced to 2048. must be something on your end.
https://huggingface.co/TheDrummer/Snowpiercer-15B-v4/blob/main/config.json
it uses ChatML
For inference I am using LM Studio (0.3.34 todays updated version with Vulkan 1.62) and the max context length for this specific model version is 2048. snowpiercer-15b-v3c.gguf, Q8-0. I am not a geek or expert when it comes to Ai and I do not know of anyway that I can increase the context length beyond the default values available.
For inference I am using LM Studio (0.3.34 todays updated version with Vulkan 1.62) and the max context length for this specific model version is 2048. snowpiercer-15b-v3c.gguf, Q8-0. I am not a geek or expert when it comes to Ai and I do not know of anyway that I can increase the context length beyond the default values available.
You may want to try KoboldCpp. That's what i use, and it is pretty customizable and very easy to use. https://github.com/LostRuins/koboldcpp . It ties into Sillytavern just fine.
When you build your first config that works for you, make sure you also specify your admin dir where you keep your RP gguf models and save your config in that directory. That will allow you to go to the "admin" button in the KoboldCpp web ui and swap between other models and config files. I generally keep a few different default configs with kv sizes and context sizes so i can swap them as i move between larger and smaller models. Please note that it will not see subdirectories so you must have all your configs and gguf files together.
come visit us in the beaver discord. we are friendly and happy to help :)
You are rude, get help.
can confirm, beaver discord is fun