four years ago I changed. It cost me 1k to change the mortgage and my interest went up 0.5%. My landlord insurance was about the same though (even renting furnished).
It never occurred to me to breath my contract by renting it out on residential, because I wanted to keep my house. Plus, in the HMRC ever do check deeper into my tax returns I guess they might let the bank know. And my letting agency were quite strict on the right paperwork.

I would say that as long as the bank continue to receive their money then this never hits their radar.

If i had to switch my mortgage to a buy to let that would make a real difference to my monthly finances. Just make sure you use a decent letting agent in the UK to manage the process to help avoid as much risk as possible.