i'm the same way, but at the same time i believe that if you've got it and you worked hard for it, you've got the right to flaunt it and show it off, especially in situations like music festivals or similar events.
While in law school, I worked at the local gym, an awesome facility.
The only real sign of me being an avid strength trainee (while wearing clothes), were my traps. We played pick up basketball quite a bit, and one day we played outside shirts/skins. A couple of law school girls that I didn't really associate wiht came to watch. Two girls came up to me afterwards and were like: "I didn't know you were buff", "Where do you work out?"
Their reaction was more than likely due to the very short guy at our school who was juiced, and constantly talked about his "bodybuilding", yet never trained his bird legs because "I always run bro."