Obama's original statement was silly. Nobody can work without some downtime, especially in jobs where there is high pressure and mental acuity is important. In fact, counterintuitive as it may seem, mentally taxing work requires a lot of downtime because the brain doesn't "spin down" instantly and a good night's sleep won't necessarily help someone who has been doing taxing mental work as much as it would help someone doing taxing physical work.
I find the notion that a President shouldn't be able to go play a round of golf or a game of basketball, to run or ride his bicycle, and to take some time to unwind to be stupid. The President is a human being, like the rest of us. He's not invested with superpowers making him immune to stress and fatigue.
Add to that that the President is never really on vacation anyways, since the office basically travels with and surrounds him at all times, and I don't think much about whether Obama is in the Oval Office, on Air Force 1, in Chicago, Hawaii, or Martha's Vineyard. Same with Bush II before him, and Clinton before that, etc.
But it can certainly look bad, and Obama (like Bush) is looking bad because he does these things at inopportune times and without regard for appearances. I find that weird from an otherwise talented and polished politician.
Beyond that, I see little to outrage me here, except the expense to taxpayers.