Thanks all : this is definitely a strange one but think I've solved it with your help.
Firstly, how did you find the info on Arthur Perry's birth certificate, Harriett's name at the Baptism & Joseph & Harriet''s Marriage record, I'm on Ancestry & looked on the Family Search website but can't find that sort of info out ? perhaps I'm not looking in the right place.
So I think it looks like this now with your help, so it's epic
On the 1851 Census George Perry ( 1811 the father ) & Jane ( Fisher mother ) there is a George Perry listed aged 4 but on the 1861 Census & all those afterwards it says Joseph aged 14 so I'm assuming that this is the same person & something went wrong when writing the census.
From your info Harriett married John Batty & had Thomas, this would explain the ages differences, then became a widow.
It looks like she then married Joseph Perry & had George Perry then on the 1891 census Harriet is Head of the household with Benjamin down as a lodger. Then on the 1901 Census is shows them as husband & wife. The probate I think proves they weren't married but what a set up.
George in 1881 was living as an adopted son to Charles & Eliza Wetherall : Eliza was a Perry and I think would make her his Aunt
Thanks so much for your help