Whilst there is an element of picking who you want to be PM in selecting who to vote for, most people vote for party or policies, and the PM ends up being the person who can most likely command a majority in the HoC.
There is after all no guarantee at a GE that a party leader going into the election will keep their seat, ( eg Jo Swinson ).
Would we be having the same "no mandate" demands if the PM left office due to ill health, like Eden and MacMillan ?