I get confused about X matches too. I was told an X match to a male meant you were connected down the female line but I have masses of X match males who do not share my direct maternal line and have a very varied selection of Haplogroup subclades. As they only carry MtDNA from their mother, they don't pass it on, I don't know whether you assume all those matches haplogroups form part of your DNA through unspecified generations of females?
?
I can only assume that when you match to a female and don't have an X match to her, it's down the paternal lines but which one? There are learning tools on FtDNA, maybe it's best to bite the bullet and take notes. I will have to do that too.
I did Y700 via my brother after finding out my paternal line had a break in it at 1801 due to a widow having an illegitimate child, we had no connection to the name I was brought up with. I have a female non X match who is 66 cm and her mother had the maiden name of my brothers two closest Y700 matches, different clade to him but in the same one as each other. I still can't figure out the connection though, partly because the surname is of all things 'Smith', 4th generation once removed (she is 20 years older) may get us to 1801??? Perhaps you should look to see what generation she would have to be at, there are charts on line for guidance. If she has a family tree and can tell you all the names in it there could be a common one to you both.
My Heritage, group people into clusters of shared ancestry. That could be useful in this case because any male matches may have a name you recognise.
I think people who are good at this must watch a lot of video's and take notes, probably well worth the time and effort.