It’s not justified; he shouldn’t be throwing punches.
But it is to be expected; something can be both unjustifiable and wholly predictable. Open disrespect may not be, in and of itself, an invitation to an ass-whooping, but it’s definitely working on the seating chart and auditioning DJs.
So you didn’t deserve the blow, but you damned sure deserve to feel shitty about the circumstances; If it were me and he sought it, I’d offer instantaneous forgiveness and pray he returned it. Even if she were okay with me saying degrading shit to her, that doesn’t give me a license to freak the family and insult her old man. And if she weren’t okay with it, well, there’s someone else probably wondering why you’re worrying so much about her daddy right now.
Those little girl-knuckles can hurt when they’re properly motivated.