You could also rent a bike from Bern rollt and ride around the city for the day
www.bernrollt.ch . I would also suggest getting a map from the tourism place at the railway station as there are some great sites you don't want to miss especially in the old town from Zytglogge downwards. Don't miss the Münster cathedral!
www.berninfo.com also has good tips and up to date information about what is going on around Bern.
If you have time, go up to the Gurten, on a clear day, the view of the mountains and of the city is great.