You can go too low...there is a threshold as to where this limit is...if you go past it you will lose muscle mass. the body is going to get what it needs one way or the other, so when your calories get "too low" major muscle and strength loss will occur. However, this threshold is going to vary from person to person...weight, LBM, and genetics make up a large part of this.
I've made this mistake in the past, taking my calories much lower then I should have. I'd get to a point and hit a wall with my weight loss and lower my calories even more. Doing this I'd always experience a bit more weight loss, but more muscle loss then anything else. When you go into severe starvation mode, your body eats the muscle like there is no tomorrow and it seems to make losing fat even more difficult...I think it does this because the body is trying to save the fat because it's worried about not getting an adequate supply of nutrients.
Some things you might try to help your process:
*Don't rely on a strictly protein based diet. It's true, your muscle do require protein for growth and maintenance, but the body also requires some level of an energy source. Obviously fats and carbs are these two sources, without them your body will eat muscle for this energy purpose and hang on to the fat.
*I like the idea of high protein moderate fat diets, with little to no carbs. Now there are dieting methods that do include a limited amount of carbs and they can work, but the protein/fat diets seem to be more efficient.
*When following a diet like this your body becomes accustomed to not having a supply of glucose but it has an ample supply of fat to pull from for energy. When your body runs out of fat to feed on from the foods you ate, it will then go after stored bodyfat since that is the mode it's already in...this will spare your muscles from being attacked by the body in search of glucose.
*I'd follow a plan like the fat/protein guideline, but I'd also incorporate a heavy carb meal 1x per wk in order to trick the body...this will continue to aid in the entire process.
*Last off, and it's what no one wants to hear, do more cardio. However, I am not a propionate of high-intensity cardio. This requires large amounts of glucose to attempt and on this type of diet you don't have that to spare. Moderate speed long duration cardio is best, because it will only burn fat.
A lot of what I believe to be the most efficient way to do things, as many of you will be easily able to tell, comes straight from Dave Palumbo's dieting philosophies. To me, Dave's ideas make more sense then any other way one could diet, being that they are the most conducive to losing fat while maintaining as much muscle mass as humanly possible. Love him or hate him, when it comes to dieting and BB in general, Dave is the king.
I hope some of this helps and answerers your questions. If not, keep asking whatever you want.