Of the three factors being discussed: Nutrition, training, and hormones; I'm not sure it matters as to what is more important because if even one is neglected then everything falls apart.
Look at, say, an offseason prime Jay Cutler. He claimed to eat 8-10 meals a day to maintain and grow muscle mass. If he dropped to three meals even with the drugs and hard training within six months he'd look like an entirely different person. If maintained the diet, took hormones but stopped training entirely then again it would be a huge difference. Same with training and eating but off hormones entirely.
I do think that if you go off drugs but still keep an intense training routine and follow a good diet that difference, the lost of pro-level physique, wouldn't be as dramatic then if you continued to take drugs and eat well but zero training.
People like to point out how much a bber has shrunk in retirement and use that as proof that it's "all drugs". But don't you think that the fact they are no longer training and eating as much also play a huge factor.