Nanook

I agree, you need to let him go, very hard to do. But for your sanity. You can not keep living in his messed up mind, it will mess you up. Let him go, which does not mean you will stop caring.

What is that saying, "if you love something set if free........"
Dont think of it as 'us' needs to be you and the kids, focus on that, and that is all.
When you have a moment of weakness, think 'is this good for us and the kids', for once follow your head, not your heart.
Good luck