Vaughan,
We are all a bit myopic, we see clearly small differences in what's familiar to us and lump together what's less familiar.
Having once been a professional theoretical physicists I see clear limits for what's physics research (but I couldn't exclude modern chemistry in any other way than saying that by definition, chemistry is not a subfield of physics). The line between physics and mathematics is drawn in water and moving around that line should be made easy. Both sciences have benefited greatly from work done in this bordering area. The great polymaths of the past, like Newton, were both mathematicians and physicists.
In my case the problem was that an expression formed as <i>epithet + physicists</i> means intuitively that we are discussing about a physicists, not a mathematician. Perhaps that expression is used only because <i>physical mathematicians</i> doesn't sound good and because it's not easy to find a better one.
Irrespectively of the above, I'm not happy, when I notice that a person calling her or himself a physicist with or without the epithet <i>mathematical</i> shows severe lack of understanding of physics. On several occasions that has happened with people claiming to be mathematical physicists. Thus there must be a significant population of bad mathematical physicists who fail on this point. A number of them have written ridiculous papers as climate skeptics.