There should be a double taxation agreement in place between the UK and Switzerland. See paragraph 9 of
this document. Which means either: a) you won't be taxed a second time in the UK as I assume you already had Quellensteuer deducted here in Switzerland, or b) the UK tax authorities will want to tax you the difference of what you would have paid in the UK and what you have already paid in Switzerland.
I think the system is as fair as possible given that you are not outside the UK for the full tax year.
You're best bet is to speak to your local inland revenue office in the UK.