BJJ for big boys

So I train no-gi BJJ, I'm 6'4 and 300lbs. Anytime I roll with people smaller (majority of the time) I go more gentle, I don't want to hurt any of my teammates, but its irritating that some people specifically coaches or higher belts will think I'm not as good because I don't exsessively use my weight when rolling with them but in reality I'm just not being a dick, I know for a fact I could demolish these guys if I turned up the heat. Iv heard the same hazing over the years like "you're still a white belt/blue belt" etc but what does your internal belt hierarchy matter if I could still smash through guard?