That's a fair point. And if every altercation was started by the Nazis, then ya, there'd be no blame on the left's side. I just have a hard time believing that, but it's not like I was there.
Again, it is truly terrifying that there are people who think that commies/Jews have their hands on the levers of power and are resorting to arming themselves with guns and Nazi ideology. Absolutely 100% ing stupid. But, that is entirely their right. And frankly I think it's sad that we're in a socio-political climate where we "aren't surprised" that a person's response to an idiot brandishing a swastika is to throw a punch. I get it, but that's not how our system was set up. The idea behind the marketplace of ideas was that cooler heads would prevail and these people would be outed as idiots and not taken seriously. That's why I think counter-protests are idiotic -- they lend credibility to stupid ideas.
As an aside, I'm pretty sure the Skokie Supreme Court case held that publicly marching with Nazi flags wasn't incitement under the First Amendment.