It is simply a matter of confidence. If you don't care and have a good, muscular physique, it can look quite masculine. With bodybuilders it actually draws more attention to the body and , obviously, doesn't obscure traps and other muscles. It is when a man is a big girl about it, wearing silly combovers or rugs, trying to hide it, then that is when it looks silly.