I keep getting told that this is a day to take care on. Is it really that bad?

With it being a public holiday I was speculating about what I would do and talked about spending the day wandering around Zurich. Only to then hear all the "take care" warnings!

I understand that there are political elements who "celebrate" the fact it is Mayday and that this doesn't involve men wearing skirts with bells on their socks dancing around a maypole

But how bad do things actually get? And if it's bad enough, is it a good idea to take off somewhere else on that day?