Well, in my humble opinion, the law is not what makes something "right" or "wrong" (it's wrong before there's a law written), nor should the law attempt to legislate morality. I think we would all agree that adultery is "wrong" and that no one "should" do it, and yet, because I believe in freedom, I also believe that people can choose things with which I disagree. In other words, people can choose to be idiots if that's what they choose!
Where we fall apart, I think, is in two ways. Let people be free to make their own choices, but then allow them to experience the cost and the benefit of the choice they made (#1), and treat marriage like you would any other business contract (#2).
For #1 if a person chooses to be an idiot--hey I might disagree but they can do it! But let then experience the BENEFIT of being an idiot (the fun at the moment or the instant gratification) and the COST of being an idiot (jail time, child support...whatever cost we assess). That's where the law comes in: we write down, "If someone does ____, the cost will be ___"
For #2 if a marriage was like any other business contract, it would be that Party A agrees to partner with Party B. They agree to build a household and family together. Party A will bring A, B, and C to the partnership, and Party B will bring X, Y, and Z...and they agree together to build 1, 2, and 3 and own it jointly. Both parties agree that this partnership will be exclusive. If either party breaches the contract, they lose their rights to the partnership and lose their interest in their partner's property and the jointly owned property...and their partner can sue for damages.
The end.
Thus, it's not "adultery" that's allowed or not allowed legally--it's breaking the contract. And the cost of breaking the contract is the same as any other business deal: you breach the contract (and the partner can prove it), you lose. Period.