I've been training since I was 15. After a few years, I knew I didn't have the genetics to be a competitive bodybuilder. I built a decent physique compared to the average populace walking around, but definitely not on the level to be a successful competitor. One of the main reasons I never did gear.
I'm small boned and not built for strength. My legs are the only body part that grew easily. Pushing too hard over the years is probably the main reason my rotators are fucked up. So yes, I'm convinced genetics are the main factor.