Try having a look at Cyclophile Lausannois. They organize group rides, I have no idea of the atmosphere or speed, but they should have something for everyone because they are involved in road cycling, track, mountain biking, and cyclocross.

I agree, Hood Cycles is a very nice bike shop. I've bought a nice lightweight trekking frame from them and have recommended them to several friends. The fact that they speak English is a big bonus around here. Other local bike shops with English-speaking staff include The Bike in Pully (dealers for Cannondale, Specialized, and others), and BBR Cycles in Le-Mont-Sur-Lausanne (dealers in Trek and others) have one Swiss-German assistant who speaks English.