Simply because an overly powerful guy can very easily make up for fighting ability. Being able to overpower an opponent either by striking him harder, grabbing and controlling him, or manipulation of limbs isn't something that has to be "trained" and can devastate another human faster than a lot of skills can make up for.
Don't get me wrong, seals are tough.. but they're not these great fighters like you see in the movies. H2H is a very small part in thier training. You're over estimating what they bring to the table as a far as a fight goes.