My grandfather worked with asbestos for decades as a carpenter and he lived to be over 100 years old with no lung issues ever. When they first started talking about the dangers in the 1980's he thought it was bullshit. 
Think of your life as a Baccarat shoe...This shoe is filled with cards...Some have Influenza written on them...Some have various cancers and diseases written on them..."Lung Cancer"..."Skin Cancer"..."Emphysema"...The rest of them say "No Disease Today"...
These are all shuffled and loaded in the shoe, and each day you draw a card.
If you have bad habits like smoking, or spending too much time in the Sun...you are loading your "Life Shoe" with extra "Disease Cards" in relation to the "No Disease" cards, increasing your chances of getting something...But not guaranteeing that you will contract something.