When people first start to work out, and are still girly men waiting for their nuts to drop from their stomach, they can workout for hours on end because there is only so much muscle to be damaged; therefore the body has less work to do to repair itself. Things change when you start putting on a lot of size, and the body has a lot more repairing to do.
Years ago, I would work out for 2 hours, 5-6 days a week. Now, I only need about a 45 min heavy duty workout 3-4 times a week to achieve the same results. Any more than that, and my recovery takes forever and I won't make the same gains.
Naturally, age has a lot to do with this equation, as well as what kind of “supplements” you take, but overall, the bigger you are, the less you need.