Problems when you don’t have full custody, what are your stories?

Background: (M35) currently with joint custody of out of state son (3) who is in Michigan with ex wife. Changed jobs to a lower pay thus necessitating decrease in child support per Michigan law. Ex wife and her lawyer are offering waiving of full child support (that I pay) in exchange for giving up joint custody. No mention of what visitation would be like. Me and ex wife agree on education (homeschool), faith (Muslim), and future extracurricular activities. Am told this is usually the extent of legal custody coverage. Something tells me this is a trap, just on a gut feeling. What are your thoughts? Single dads who did not get any legal custody/chose to give up legal custody, what has been your experience?