It sounds to me like you judge them as harshly as they judge you. So maybe y'all are even steven.
People judge each other all the time. And if you ever dare do something outside the norms, whatever those are, the judgements are always harsher. That leaves all of us with a choice. Do what others want or expect us to do, or stop caring about what they think. I opt for the latter.
I don't think I judge them at all. I THINK of them how I described, but I keep my thoughts and facial expressions to myself when it comes to disagreeing/disapproving with our lifestyles.