Also, if you're not already doing so now, make sure you stretch out your quads/hip muscles/hams too.
I used to NEVER stretch. Then I think it was about 3yrs ago, I started a diet phase, walked on the treadmill backwards (noticing a bit of my hips tightening up, didn't think anything of it), lots of Hanging Leg Raises and still no stretching.
One day I'm warming up as usual for back squats. Empty bar for 5, 135/5, 185 on the third rep I felt an excruciating pain radiated down my back. Ever since then my back has seemed pretty fucked.
After trying to work around it as much as possible-much like yourself- I finally started to ask around realizing it's not getting any better. I went to a get a massage and when I was asked if I had any problem areas, I said my back.
When they tested my back with a finger on my spine, nothing. But when my hip muscles were to be worked- I shot through the roof!
Since then I've made sure to stretch really well, especially my psoas muscles- which were the culprit in my case.
I'm near squatting my old weight again, but I still have to be extra careful.