There have been way too many people who have done this in the past 2 years or so. Each state and bank which owns your loan has their own way of dealing with this.
One way is if you do walk away, and they do finally sell the place, they will send the IRS a 1099 for the difference, because this can be considered 'income' since you didnt pay it. They will get out at the end someway.
Second is take you to court, but you can claim financial hardship and more, and it is very hard for them to get anything back out of it. Still, they can sue you for breach of contract, and if you do have money somewhere, they can get a judgement against you.
Third, there is something caleld the Homestead Act which provides protection so they cannot take your new home, in case something goes 'bad' on the old mortgage. Not sure of exactly the terms of this, but someone, it has protected some people.
There are others issues too, such as having your credit being in the tank for 5-7 years too. Which wont help if you want to get a new credit card, get a car or more.
And yes, of course it hurts everyone. When banks fail because of nonpayment of mortgages, people LOSE their jobs. Communities lose income and spending from the value of the jobs, and taxes.