I guess it will also depend on how long you intend to stay in Greece over the summer. if less than, say, 3 months, there is no way they can tax you or make you import your car (especially as you do not live there!).

As long as you can prove residency in Switzerland (you can, you have your B permit)and the car is swiss registered (also true) then you should have no problem. On entering Greece I would imagine you would say you are there on holiday visiting friends and family (or whatever). I think you are worrying a bit too much!