Dood, my boss is always on my case, I just had my yearly performance review and he blasted me for no reasons. Well, not entirely as I suspect he is singling me out for being fit and muscular. I don't dislike the guy personally but his lack of leadership is sickening and definitely has a toll on productivity and team spirit.
First, thank you for your question!
Not everyone is born a leader; only few are born with what I refer to as the alpha aura. Also consider that most are threaten by physical size as our brains instinctively perceive it as a treat or at the very least as a potential "takeover".
Your boss is simply threaten by you; unless there is a quantitative measurement of your lack of performance, I would not make a big deal about it but go the extra mile to put yourself in position that gives you more external exposure; to other managers, customers, executives and even your peers.
Would you go to war with your boss? Would you stand tall next to him and take a couple of punches for him?
To give you a point of reference, at work I directly answer to the VP of Engineering. I have total confidence in him, in his decision making and thought process,. I would take a fall for him without even blinking; He is a true leader and sees my size and brain as a complementary asset, not a treat.
He also passed down the torch to me; I have a team of 22 engineers that would do the same.
Not everyone is suited to be a leader. my advice is don't stay under someone with blatant lack of leadership and confidence and slowly build your case to get outside recognition, bypass him... and STAY POSITIVE!!!!!!!!!!!!!!!!!!!!!!!
On a related note, size does matter; in every stratification of corporate America / europe, in fact globally (I was in Japan recently and they were in awe). Use it to your advantage; physicality and brains rarely go hand in hand, individuals often make a choice to fork out. If you have both, it's agreat asset to attract attention and get your message across.
bodybuilding is awesome.