Unfortunately I am a passport holder of a country which need a visa to enter Gibraltar. On some websites it says it's allowed to enter Gibraltar with a multi-entry Schengen visa, but I don't know if it applies to a Swiss resident permit.

Similar things are in Balkan countries like Serbia, which allows multi-entry Schengen visa, but is unclear for a resident permit of a Schengen country.

Strictly speaking, Gibraltar is UK territory and not part of the Schengen agreement.

This from the Visit Gibraltar website:

The following persons do not need a visa for Gibraltar under the following circumstances:

Those nationals or citizens of Morocco, the People's Republic of China, Mongolia, India or Russia who are holders of multiple entry Schengen visas with a minimum remaining validity of 7 days on the date of departure. Such persons are only allowed to enter Gibraltar for a maximum period of 21 days.

The following website gives more information:

https://www.gibraltar.gov.gi/civil-s...tration-office

Yes that's why I am asking. It only mentioned "multiple entry Schengen visas ", but I don't know if a Swiss resident permit counts.

Under normal circumstances, your residence permit serves as a Shengen visa as long as it's valid. In this case because the wording is so specifc, I'd recommend contacting the authorities in Gibraltar directly to be sure.

I go to Gib twice a year. You'll need a passport for sure. If you go across the border from Spain often you can flash your passport and the border control wont check it but you'll be taking a chance. I wouldn't recommend it. Good luck in any case

The Swiss residence permit definitely counts as a multiple entry Schengen visa.