Make sure you get some warmer gloves too. Most cycling gloves are designed to keep your hands as cool as possible. But riding in winter your hands can get painfully cold.

As already mentioned, cold feet can be uncomfortable, a cold head can give you an unpleasant headache and cold dry air blowing on your eyes in unpleasant. I find that generally I don't have to worry as much about keeping my body warm as the physical activity helps a lot there. It is extremities I have to concentrate on.