I don't think the government should be engaging in religious expression, moreso when it starts making only certain religious expression or affiliation acceptable. The problem is when several bodies of government, down to the local level, only permit prayers and invocations from certain cults or religions and when others request to do the same they are either not granted permission or the rules and legislation are suddenly changed to stop the invocations.
It has never been our custom, practice, or part of the Constitution to prohibit any form of religious expression by government. From the Founders to president day, we are largely comprised of people of faith. That's why we put a chaplain on the payroll in the 1700s.
I do agree that allowing prayers by some sects and not others could be a problem. Not sure of specific instances? And I'm not talking about people like Michael Newdow (we have our version of him in Hawaii). I mean legit sects that were denied the opportunity.
That said, you and Agnostic are making a much broader point than prayers before legislative bodies. You're talking about censoring all forms of religious expression on public property. Way too broad IMO.