Question about "chicken skin."?

I have ugly little pimple looking things all over my upper arms and thighs. I'm kind of self conscious about them… I exfoliate, I've tried acne cleanser and cream.. all sorts of things. I read that it's a build up of keratin on the hair follicle which makes sense cause they're all at the base of hairs. But i kind of don't care what they are i just want them gone.
any help would be much appreciated, thanks.