What can you do, nowadays? You can't really get into a fight with someone without reprecussions. If some jerk is trying to get a rise out of you, he's probably pitifully insecure and looking to compare penis size or whatever. It's probably not worth getting into it with someone... be the bigger person and ignore it. If they won't let you ignore it, be a jerk back- laugh at him or say something derogatory in turn and keep on walking.
The old lady and I were waiting for a bus one time... some (apparent) transient (my city seems to have a variety of very aggressive panhandlers, tranisents and other unfortunates down on their luck) came up and was angry that he "saw us standing for the bus all of the time, being quiet." We ignored him as best we could. Then he was "offended" that we were wearing dark clothes (!!) and that we looked like a... he groped for a name... and the best he could do was "an Ozzy Osbourne cover." We laughed and said "Good" which confounded him, and made him madder... he yelled some more at us; told my girlfriend she was "ugly as sin," to which she replied "piss off," and I replied "coming from the man that hasn't got a home or a woman to go to," which really enraged him... he started ranting about Hurricane Katrina and some other junk... got up in my face (I'm a pretty big guy- 6'1 and close to 300 lbs)... I told him he was utterly pathetic to go around screaming at random people and to get the hell out of our faces before I did something about it. Luckily, the bus pulled up, and we calmly got on the bus as he had an utter meltdown. He wasn't prepared for anyone to stand up for themselves. If he wasn't so overtly hostile and clearly going out of his way to needle people, I would have felt very bad for him. Sometimes, ignoring this king of thing (my preferred choice) doesn't work- this guy was angry and had set his sights on killing time this way... but, in this case, returning his belligerence took the wind out of his sails. He was expecting whomever he came up to to just take it. I guess it's largely about reading the situation- the bottom line is that you can't go around with your chest puffed out looking for a fight, but you certainly don't have to take abuse if you don't want to.
Come to think of it, nearly every jerky encounter I've had has been waiting for the bus or getting on the bus. Something about the downtown Seattle bus system that really brings out aggression in some people.