Who pays the rest of your debt when you switch mortgage lender?

Say for example there are two banks. One called ABC and another called XYZ. I have £100,000 left, my bank is ABC. I pay £500.00 per month, however I wish to chnge contract to save money. XYZ are offering £400.00 per month. If I want to change, bviously rhere is a charge fee of 5 percent (£5000.00) What im confused about is essentially who do I pay the rest of the money to? Is it the new bank im with, or the old bank? (ABC)

Read more: Who pays the rest of your debt when you switch mortgage lender?