What is "good acting"? Sometimes an actor has a part that is only seconds long, maybe without any speaking, and people praise the acting. Sometimes people have such interessting faces that they don't have to act to be good lol.
I mostly go by feel. For instance if he's playing a villain, does he make me feel legitimate menace? Would I feel it if I met the guy on the street? Also Do they feel right for the time and dialogue they're portraying? I mentioned McShane and Sanderson, I've seen both be comfortable working across many different genres, and they never once took me out of the moment with their acting.
Granted, this also falls on the director and screen writer also.
----------------
And then as you mentioned, every once and a while, there is just someone you can't keep your eye off of. Timothy Carey, who I mention a page back, was like that. Not sure I'd class him as a great, great actor, but I can't take my eyes off the screen during his scenes. Al Lettieri, a 70's character actor was like that for me too.