I weight train and also bicycle race. I have found that when my weight goes over 190 (I am 5'9"), its pretty much impossible to be competitive on even moderately hilly courses on a bike. So yes, you can be too big, and not just by appearance only.
As a compromise, I keep my weight about 185 now and still do both. The positive side of this is I have watched my diet and gotten very lean. I lift moderately heavy (6-10 RM) with smooth, controlled reps and am not too much concerned about getting bigger. Lifting is pretty much a rush since i just go for "the pump" . No joint pains now either.
Funny thing is that now that i am in no hurry to hoist big numbers, my shoulders and biceps have gotten a lot fuller and more pronounced. Never would've guessed that!