What do you think about the safety of polymer clay?

I just picked up my old stash from college and found that I LOVE to sculpt, paint and varish! But, I have found a few websites that talk about the dangers of the chemicals in the clay that are the same as those in plastic, and I worry about my hands touching it and then my kids, food, etc. (Sometimes I eat while crafting, what can I say?)

I haven't got a chance to do it a lot, but would like to. My IDEAL, perfect scenario is my own ceramic kiln and clay with glosses, but I don't have a budget anywhere in that galaxy, so polymer seems to be the next best thing.

What do you guys think? Do you think that almost anything causes cancer, so why not? Do you think I should be extremely cautious? Or somewhere in between?

I realize it's just opionions, but I thought I'd ask.