Are you getting anything out of this relationship that is worth sticking around for? I'm guessing there isn't, because otherwise, why would you be posting about this right now? I mean, when she has one weekend where she's "off" isn't really enough to break off what is otherwise a really great relationship. And you don't see this ever moving towards marriage. It doesn't sound like you're happy with this woman. So it might be time to break up with her.
(And, FWIW, I think she was weird... she wants alone time, but expects you to just chill out until she decides she's ready to hang? I think it was completely reasonable for you to leave. She said she wanted to be alone. You have better things to do than sit around and wait for her. I spend every weekend at my partner's place--as soon as I get home from work, I pack a bag, go to his place, and I'm there until about 10 pm on Sunday night. But sometimes, he has to do work stuff on the weekend, and when that happens, I'm welcome to chill at his place--which I sometimes do, I just chill and read a book--but most of the time, I head back to my place and take care of chores, run errands, or just do whatever needs to be done. Or if he needs a nap, which he sometimes does--lack of sleep makes him cranky--I usually leave for a little bit, unless I want a nap, too. And he texts me when he's done with his work or his nap, and I'll finish up what I'm doing and come back. He doesn't expect me to sit around and wait for him. That's weird that she was like, "where did you go?" It could have been medication or something, but I don't know.)
Don't worry what other people think. If mutual friends are going to slag you off, then they aren't really your friends. And it's not their business, anyway. I agree with @Cooper
, you just say that the relationship wasn't working or it wasn't good. Don't say anything about her, just that it didn't work. You can have two great people, but they just aren't compatible and the relationship doesn't work.
And that thing about you having been wrong about breaking up before? Just because you think you were wrong in breaking up with other people before doesn't mean that you're wrong in breaking up now. And if you regret those breakups NOW, it doesn't mean that you were wrong ending those relationships before. If you ended something that was good, you likely weren't ready for it, and if you weren't ready for it, you would have screwed it up later anyway if you didn't end it when you did. Does that make sense? What I'm saying is don't let your past define your present and your future. Everyone makes mistakes and wrong decisions. Don't let it define you.