I think it's because sex has always been considered as something, what's the right word, immoral? At least if we take it to the status of religion. As a result, many people don't feel comfortable about their body. I have some friends, for instance, who can't stand watching sex scenes, they'll begin turning around or giggling and that kind of stuff. Well, they can't actually talk about it either.
I've only read one book describing sex, Atonement, and the scene was important to the plot. Otherwise, I'm not sure what descriptions would make me feel uncomfortable.
I don't generally mind it in films either, but usually when it's decipted more emotionally, otherwise it makes me a little uncomfortable, maybe due to the fact that I don't have any sexual experiences.