The 1851 and 1861 census gives his place of birth merely as Rugby, though he is living in Bilton. (Noting his children have Bilton as place of birth.) I can't find a baptism, however there is this item which may be of interest:
"RUGBY ...PETTY SESSIONS, Tuesday ... Sarah Jarvis, Bilton, applied for a warrant against her husband for threatening to murder her. The warrant was granted."
Saturday, Oct. 7, 1854
Publication: Leicestershire Mercury