Is this worth fighting for?

Husband and I have had issues over the in law topic for the last 2 years. I find his parents very overbearing and stressful and just a general mismatch with my personality so I tend to not spend too much time with them. Typically only once a week. My husband sees this as me trying to drive them apart and he says he is not going to be with someone who is making him feel like he is not living life how I want to live it and is constantly stressing him out. Living life how he wants = chilling with his parents frequently with our son and me.

He imagined this life where we’d have dinner/ chill together frequently throughout the week but I have a demanding job and I am done work at 6. With the baby’s bed time starting at 8/8:30 I don’t see it as worth my time to get ready to go see my inlaws. (I work from home so I would need to get ready after work).

He also always makes me feel bad and says things like. “You are exactly the type of woman I didn’t want to be with” and “I wanted someone who will be good to my family and treat them well”.

To be fair, because his family rubs me the wrong way I am not overly warm with them but I’m not mean or rude by any means.

I’m starting to lose hope. We fought about this topic 3 times in the last two weeks alone and he has hung out with them 4 or 5 times in the last two weeks plus daily for the quick pick up and drop off of our child since his parents are helping us out with daycare while we work.

Is this even worth fighting for at this point?