I agree. I have some buddies who are doing it now and the women there are insanely hot and most the people who go are overall in very good shape. If you get into the intense classes they are not easy classes to do.
I think the bad name comes from idiots doing dangerous and stupid workouts and calling if "crossfit".
Most REAL crossfit I've seen looks extremely taxing. It's a different kind of workout, with more of military style strength/endurance goal.
In other words - actually in good overall shape, rather than just being strong, or just being lean, or just being fast in the short term.