He is angry and missing his OW. He is picking fights to leave so he doesn't have to be around you and feel guilty and/or see his OW again.
Sounds awful and sounds like he isn't really sorry.
As far as him accusing you of cheating, that's just liek a cheater. They think EVERY ONE can do what they did. Their brain just isn't right.
