Totally. I owned a gym for a few years and I'd say it was mostly guys trying to get bigger and woman trying to get smaller LOL. But those guys who started from a meso/endo base with a good natural strength and muscle mass base, wide shoulders and good basic muscle mass seemed to lose less between cycles
So I guess I'm saying the further the fall may be, the stronger the urge to stay on 'forever'
I was 6' 1" and 120 lbs when I started lifting weights so I agree with you that most men do this to gain muscular size and strength. In 9th grade I was 5' 9" and 90 lbs. Always been skinny. At 10 years of age I was 4' 11" and 45 lbs. I could run like the wind, especially if it was at my back!

My family went to the Pali Lookout in Hawaii when I was in the 9th grade. I'm certain Pellius has been there, but it is quite windy. My dad, being my dad, bet me that I couldn't jump off because the wind would blow my skinny ass back.
I being kinda slow took him up on it and to the terror of my entire family including my dad, did a long run toward the edge and leaped for all I was worth.
Fortunately the wind came up and blew me back like a potato chip. Way back. I hit the ground HARD but since I was often doing dumb stuff like that I was okay. When I hit the whole family and several people that had seen it just started laughing their collective butts off. Ahhhhh...Stupid times that made good memories.