There are some nutritional and other issues you should be examining
1) Do you get at least 20% of your calories from fats, low fat intake can depress testosterone levels
2) Are at least some of your fats high cholesterol type fats (i.e., red meat)
3) How is your zinc and magnesium inrtake levels, consider supplementation and also oysters
4) Are you consuming cruciferous vegetables (known anti-estrogenic properties)
5) Sleep. Are you getting a solid 7.5 plus (TV off, all lights off)