Paulie, just to encourage you - I haven't got to Switzerland yet but will be coming in June. Alone. Minimal German. Less French. Job hunting. Flat hunting. Not a 'pretty young thing'!
I reckon if I can do it under those circumstances then you can probably do it much easier as you've already been here!
Go with the gut feeling - a change could be good, and lots about France would be fascinating. You could always return to Switzerland. If you really love being in Zurich, stay with it and just see how it all pans out!
Good luck whichever way you go.
Pam