I feel demonized because I have low empathy

I know about the stereotype that autistic people have no feelings or empathy or remorse, like ASD is the same thing as ASPD. I know that stereotype is not true, and autism is a spectrum, some autistic people have abnormally high empathy. I do appreciate people who correct the stereotype, but my overly literal mind thinks they're saying "all autistic people have hyper empathy" and it makes me feel left out. I have very low cognitive empathy, often I feel like other people are just like characters in my life and they don't have thoughts or emotions. I cannot comfort people to save my life, and when I think I've done a great job at comforting someone, I actually ended up making it worse. When I hurt someone, it is very hard for me to apologize appropriately. I often apologize when someone points out something about me, because I don't understand what they think. Also, just because I have low empathy, doesn't mean I have no sense of justice or no feelings or remorse. Sociopaths manipulate people, I CANNOT manipulate people at all. Unless I thought I did nothing but someone confronts me and is like "you manipulated me!!! You monster!!!" And I get confused. I sometimes get told I'm being mean or teasing too much, but I thought it was funny, but apparently I was supposed to read their mind or something. When I get comfortable around someone, I get very excited and tell them all my jokes, but then I annoy them too much and they hate me and don't want to be around me anymore, even though i had no idea i was annoying them. I do have a high sense of justice, I don't understand why people kill civilians, hate certain groups, take advantage of people, or talk behind someone's back. I also get emotionally attached easily, especially to objects and animals, but not so much to people.