Is the altars of lilith mandatory?

I just can't do them it's tedious for me . Is there a big gap between someone who did them and someone who didn't ? In terms of stats and etc.