What other people think of you is called "other-people esteem." That's a different thing altogether.
If it's better to think well of yourself, then by all means, go ahead. No need to wait for us to do it for you. Without a doubt, uncaring people can tear you down and make this more difficult, but at some point, you can make a choice.
Self esteem is something that is done to childen, but for adults, it becomes a choice.
It's up to you.