ollama is a gem
Having trying to setup and run models and was pretty painful. Recently tried ollama, love it. The installation is so easy and such a relief to have a micro service setup with pipeline and make it light weighted.
Btw you can run Gemma3 https://ollama.com/library/gemma3 already with single GPU. I'm trying it today.