Hi Lob, I don't actually agree on that one. This happened to me my first month or so, but as I improved my pronunciation and fluency it stopped happening. In fact the opposite happened - many Swiss would then be embarassed to speak English around me, as if I would judge them. I found this most curious. Nowadays I only get the switch to english thing once in a while, usually from someone who is overjoyed at the chance to practice.

One thing to bear in mind that many expats, even though they work really hard on their german seem to do very little work on their pronunciation. The English seem to have a habit of putting umlauts on all their vowel sounds and refusing to do the deeper sounds such as u and o. Most insist on pronouncing Zug as "Zoooog" (as if it were spelt süüg in German). Many Americans seem totally unaware of the rolling "R" in German, flattening it and dragging it into a nasal sound which is like fingernails on a blackboard to a native speaker. These dead giveaways simply trigger a response in a native that "this person is very uncomfortable with German" and the response would come in English simply out of politeness.

To think about it another way - imagine you are in the UK and someone comes up to you and says "vitch vay to bahnhof". You'd hear the heavy accent and probably answer in German. But if you just heard a light German accent, you'd most likely answer in English.