You can usually take your mortgage with you to a new property - with the approval of your lender.

Beware of capital gains tax if downsizing. You can avoid this by buying at the same price or higher you are selling at - but that isn't the idea of downsizing...