Welcome to Basel !
Music: Kindermusic (http://www.kidsmusic.ch/), my 5.5 year old son loves it !

For some fun activites when bad weather or for birthday parties: http://www.okidoki-loerrach.de/

Gymboree too (near SBB) but with 4 year old they are already almost too old. English courses otherwise at Open door or Ahead with English.

Cheers