Does anyone here regret leaving a good company for a pay raise/progression?

3YOE SRE completely happy at current place. The pay is only just slightly below average for my role but it's offset by the fact I'm fully remote.

I understand the way to consistently make money is to job hop, but even though I am curious what other companies are like, I ask peers at other companies and they complain about things I can't at my place. The other reason people move is for progression/a new challenge. I understand this, maybe it's a lack of ambition on my part but if I wanted to learn something new I could always do it on the side?

TL;DR I'm at a company that I'm happy with in all aspects. Would I regret leaving? Would like to hear people's experiences if anyone's been in a similar situation.