Family myths and stories suggesting posh or royal connections where the more recent family is not well-connected are almost always just that!
My aunt was certain that we were descended from "Robert the Bruce" despite every evidence to the contrary, as research into family tree lines showed. She was also convinced that every "Irish" surname that occurred was actually Scottish, as she simply couldn't imagine being related, no matter how distantly, to any Irish families!! Shades of Hyacinth Bucket!!
Oddly, another relative, when we unearthed a distant connection to a quite prominent former Scottish Politician, was so dismayed that she wanted me to "forget it" immediately!!"
-Nowt as funny as folk, is there?
As suggested, work backwards from what you know, and can trace and prove. Much more satisfying in the long run.