Anyone else feel like we're early adopters of the next big entertainment medium?
I've been messing with locally hosted LLMs for a while now - tried everything from 7B - 32B models on my own hardware to cloud-hosted 70B and 124B on RunPod. They were decent. But no matter how I tweaked the samplers, which checkpoint, finetune, or merge I used, there would always be those moments - hallucinations, repetitive phrases, etc... nothing that ruined the fun, but enough to remind me I was just interacting with an LLM.
Then I finally tried Claude 3.7 Sonnet.
Holy shit.
The difference absolutely floored me. Far fewer repetitive patterns, incredible recall of details woven organically throughout the story, better spatial awareness, and writing quality that blows everything else away. Felt like a completely different experience. I am now currently addicted in a way I've never been before.
Now, I (sadly) can't really see myself going back to locally hosted LLMs now, at least not for the complex story-focused stuff I use SillyTavern for. (Don't get me wrong! Small local models still definitely have their place and use cases!!)
I feel like our SillyTavern storytelling and world-building hobby thing is still pretty niche. Like most people on the street would have no clue what you're talking about if you mentioned it. Sure, they might know about AI chatbots, but creating worlds with lore and complex characters and living in them? Very unlikely...
So here's my question: If models like 3.7 were dirt cheap tomorrow, would SillyTavern-esque AI storytelling & world building become much more mainstream? Or do you think what we do here with SillyTavern will always remain a bit of a niche hobby? Or are we early adopters of the next big entertainment medium?
TLDR: Tried Claude 3.7 after using local LLMs for a while. Feels like a completely different experience for story-rich/complex RP. Mind blown, addicted, feels different. Can't go back to local LLMs now (for complex-story/characters tasks). Will SillyTavern-type AI storytelling & world building be a mainstream thing once the good models (like 3.7) are way cheaper? Or will this always remain a sort of niche hobby (at least for the next half-decade or so).