I agree, everything points to him really thinking that he had been born in Ovingham (and perhaps he was).
I thought perhaps his mother’s family might live in Ovingham. She seems to be Amelia Hannah Watts (the marriage was in Newcastle) but there is a strong candidate for this AHW being baptised in London in 1797. I haven’t followed this up any further however.