Now you are a celebrity
That means that… There are people who don't know you… and who don't like you.
Specifically, there are people who don't know your work, who haven't taken the time to understand your point of view, who nonetheless have had to draw a conclusion about who you are and what you do.
"I don't like Angelina Jolie."
"Which movie didn't you like?"
"Oh, I've never seen any of her movies. I just don't like her."
More positively, celebrity, particularly social media celebrity (which more and more of us have every day) earns you trust and access and an audience. Your twitter followers or friends of friends on Facebook are more likely to cut you slack because you're not a stranger.
But it's unreasonable to expect only the upside. There are now people in the world who don't know you and who don't like you. Sorry.