I've worked out with plenty of guys that often said this;
" I'd hate to work behind a desk and glad I get to work outside/ outdoors?"
Ok, I understand wanting to be outside on a nice day, but what about all the cold, hot , rainy days?
Plus, what makes working at a desk so terrible and manual labor so enjoyable? 
For centuries working outside or manual labor was considered undesirable because it often exposed one to injury, illness, and premature death; the pay was also correspondingly low. That is less true today, but the legacy of that association is all around us. Different strokes for different folks obviously, but I work in a corner office with two widows and love it. If I were parked in a cubicle that might not be so much fun, but my office is about the size of a small two car garage. I get outside plenty after work and on the weekends: working on my yard, riding my bike, walking the dog, etc. When I want to be outside during the work day, I can walk across the way to a cafeteria, a restaurant, the library or any other building on campus. As you get older, you may conclude that working outside is overrated. Sun exposure, precipitation, and temperature extremes can get old real fast!

They say "sitting is the new cancer." Workday sitting should be balanced with a lot of physical activity.