I have many expectations of how I am to be treated. They're based on a collection of information I received interpersonally, that is to say I have expectations of my society because I've learned a language which helps me describe and predict their actions and responses. So this expectation comes from our language. For example, we describe one type of behavior as making someone a bully, therefor being able to predict that bully's actions we're able to avoid conflict with said bully.
There is no right and wrong. People who think in those terms are unequivocally less capable of actually being a helpful person. Instead of living life with simple tenants, help those in need, the righteous tend to create a belief structure (like what you've done) that put them and their kind into a frame of moral superiority. There are no morals, so these people are standing on a fake pedestal. It happens to people like myself all the time:
I'll make an opinion not based on lofty ideals but on rational executable pragmatic ideals. As a result someone, usually a woman or a pussy, will accuse me of being immoral. For example, once while on a trip with some friends, one friend began to become burdensome to the rest. Very quickly in our trip, I told him how people were feeling and told him what, specifically, he had to do to change that opinion. This girl on the trip said I was being a dick, that I shouldn't have made things "awkward". I told her that her moral high ground on this issue wasnt going to save her, she was feeling as nauseated by this person as the rest of us and just as complacent in influencing me to call this person out. Yet once the calling out commenced, she was denying her involvement calling me immoral, telling me I was wrong and mean and other useless shit.
So you want to know how rights are maintained? With willpower. Not God, not fairy dust, not "social contract theory" or anything else. Rights are ONLY for those who insist upon them.