Do you care what people think about you?

This arises from something that happened in my theater class today. I was being myself, reading a new book I got for my b-day, when two annoying preppy jerks commented on how "retarded" my book was. So, I told them, "I don't matter what you think." Then, the preppy chick with no self-confidence said, "It's 'it doesn't matter." I mean, seriously, why? What's the point in trying to degrade me when it don't work? Anyway, answer please.
Do you care what people think about you? (Woah, deja vu...^_^)