Yeah, I think a pro bodybuilder could beat down the lighter weight boxer in a street fight.  Of course, not in every case.  It would honestly come down to if the boxer could get a punch in or not.  If a pro boxer gets a clean bare knuckled shot in your face, 140 or not, you're probably going down.  I used to train for boxing under a world light weight champion, Luis Resto, who unfortunately was made famous for having cheated in a fight.  The guys hands were like bricks!  
If the bodybuilder could grab the guy or maybe even take the fight to the ground, the smaller boxer is going to have a tough time.