My ex also tried to badmouth me, so I have some insight.
The most important thing is for you to be involved in their lives. Be consistently good (does not mean coddle or spoil) them. Encourage them to judge you based on what they experience rather than what they've been told. Kids are smart and figure out what's what from a fairly early age.
Another alternative is simply to get a court order for counseling. Tell your attorney you are seeing your kids act strangely around you (and you have an idea why) and you want a qualified therapist to figure it out. If your lawyer cannot settle it with a letter, he or she can go to court to get an order.
I'm not sure exactly what concerns you have regarding your ex. In my case, I had many serious concerns so I requested (and the court granted) a full custody evaluation. Essentially, it's a process where a psychologist digs deep into the living situation to make recommendations on behalf of the children. I've found the following resource accurately details the process:
Parenting After Divorce - Anatomy of a Child Custody Evaluation
I won't lie - the process is invasive and expensive. You will have to pay the therapist fees for the evaluation, fees for your lawyer to address the issue, and maybe additional fees to the therapist if he or she needs to testify at trial. But, I feel mine was worth every penny in protection for my relationship with my daughter.