Difficult to say for sure. I found this:
"The European Economic Area (EEA)
The EEA includes EU countries and also Iceland, Liechtenstein and Norway. It allows them to be part of the EU’s single market.
Switzerland is neither an EU or EEA member but is part of the single market - this means Swiss nationals have the same rights to live and work in the UK as other EEA nationals." https://www.gov.uk/eu-eea
BUT, under point 1 in your link is this phrase.
"Documents issued on any other basis, for example (biometric) residence permits issued under the national law of another Member State are not acceptable and do not exempt the holder from the requirement to obtain an EEA family permit."
So unless the Swiss permit conforms to Articles 10 or 20 as mentioned in point 1 you will probably still need to apply for a visa.
Checking my C permit I find neither of the phrases mentioned.
Article 10 residence card: “Residence Card of a Family Member of a Union Citizen”.
Article 20 residence card: “Permanent Residence Card of a Family Member of a Union Citizen”