I'd think that it is more grounds to reject people they don't like the looks of, than a hard and fast rule without exception. If you have followed the news, there is an increase in beatings and stabbings around Switzerland, including Bern, and it is mostly non-permit holders behind it, though Swiss sometimes too.