Have you followed his children, are John and Frederick actually grand children?
David 1807 married Mary Ann Smith 1827 St Benedict's, she was buried as Marianne 3 August 1829 age 21, he then married Eliza Dowson 1831 at St Margaret's.
Robert 1809 married Sarah Nixon 1828
John 1812 possible marriage to Maria French 1835 Lakenham St John. Witness David Houghton. This requires further research as there is another Houghton family the area.
William 1814 possible marriage to Joanna Shaw Cornish 1836 Lakenham, 2nd Maria Trott
Caroline 1817 married Elijah Gray 1835 St Margaret's
Rebecca 1820 married William Drury 1840 at All Saints, 2nd John Sapey 1867 at St Peter Mancroft
Henry 1825 (with David 55 a Baker 1841 census) St Giles. Henry was 17 a weaver.
(Sister Caroline Gray is next door)
ADDED
1851 Hnery aged 26 is living with his sister Caroline and husband Elijah Gray on Distillery Street, Norwich listed as a lodger - Hawker