If the police finds out about, you drove after 12 month you should have changed to the swiss residency driving permit and now you are driving wihout a valid drivinglicence, you probably get a hefty fine and 1-3 month ban of driving.

You also need to consider, if you drive an in Switzerland registered car you need to chanche the licence, otherwise in case of an accident the Incurance will not cover any claim.
Make sure you are listed as the main driver in the insurance policy while you take the car with you.