I think in almost all cases the standard criterion for evaluating physical appearance is based upon what one believes is sexually appealing.
It is good that you didn't state this as an absolute because I know from personal experience that I don't always evaluate someone's physical appearance based on what I find sexually appealing.
Do you believe one can be sexually attracted to someone who does not live up to what they find physically attractive? At least for me, I sometimes find people who, although not ugly or gross looking, have something about their personality that makes them sexually attractive to me. Likewise, I have met physically attractive people that I don't find sexually appealing at all.