I honesty think  a bit of this comes down to age. When I was in my teens-early 20's I could hang for  90 min -2 hours, 4 times a week  and it wasn't a big deal. Now 10 yeas later with real life commitments, that workload just wears me down. 
I've found that 3x wk/1hr. seems to to a nice happy compromise. I've also found that exercise selection plays a big part.  I've dumped or cut back a lot of the small stuff.