It isn't the actors, per se, it's the revisionist history that comes with it. Yes a non-white actress can play Jane Seymour or Anne Boleyn, but the accompanying claims that the real historical characters were "people of colour" is what offends.
For instance, in the recent drama about the Norman invasion, the Earl of Northumbria was played by a non white actor as part of a "colour blind" approach to casting. This is all well and good, but historical accuracy is thrown out the window.
Do you think you could get away with castng Laurence Olivier as Othello these days ?