There is nothing fair about our current system when you have a small percentage of the population paying the majority of income taxes, and nearly half the population paying no income taxes.
A fair system would be something like a sales tax (in lieu of an income tax), where people could actually control how much or how little they pay, and everyone pays the same rate.
You're right. A fair system would close the tax breaks many families get when they have children or file as a married couple. Fairness for all in taxation would help cut down on the deficit.
The problem with a VAT tax only is that it moves the tax burden from the current system of those who can pay, a progressive taxation system, to those who can't, a regressive taxation system. Lower income earners are the loser in the VAT only equation, and would require tax breaks or no VAT on essentials in order to get by.