Frustration...

So I just don't get this... it's totally confusing me... so let's see if I can try to work this out in my head...
My mom... she's great and all but My grandfather, her dad, isn't. Wayne... he's a despicable man. He's done unmentionable things to people in my family and attempted such an act with me... He's not to be trusted, at least not in my eyes. So why would anyone expect me to be happy that he just showed up at my moms house this week. My husband and I made an adult decision here... we chose not to go to visit this man. We don't want to be around him and definitely don't want our beautiful children around him. My job as a mother is to do everything I can to prevent my children from being harmed. Red flags flying all over... So no... I didn't want to visit this man.
My mom, being the woman she is, got very upset with me for my choice. She also attempted to justify it by telling me that he's there with his girlfriend and that no one will be alone with him. Like, Why?? If we all know the kind of person he is then why would we even want to see him. Am I wrong for feeling this way? I really don't feel like I am, but my mom hasn't spoken to me in 2 days and I wish she knew that I made this choice for my family and not to hurt her... I'm so frustrated by this that I had to write this... I just needed to get it out.