Hello, I have just picked this up. I don't have any more of the family in the 1841 census. In 1851, Charles and Margueretta are in Ramsgate with their son James. I have the following information about the family:
1. Marguiritta Mill (c.1788-Montrose,Angus,Scotland;d.1859)
sp: Major Charles Walker (b.1788;m.1815;d.1854)
2. John Charles Walker (c.1816-St Mary le Tower,Ipswich,Suffolk,England;d.1854)
2. Magdalen Elizabeth Marion Walker (c.1818-St Mary le Tower,Ipswich,Suffolk,England)
sp: Robert Knox (m.1849;d.1876)
2. Helen Mary Ann Walker (c.1820-St Mary le Tower,Ipswich,Suffolk,England;d.1900)
sp: Reverend Charles Green (b.1775-Cambridge,Cambridgeshire,England;m.1849;d.1852)
sp: Reverend Henry Calthrop (b.1803-Shottesham,Norfolk,England;m.1862;d.1887)
2. Jane Isabella Donaldson Walker (b.1821-Ipswich,Suffolk,England;d.1899)
sp: Cornelius Jago Borrow (b.1823-Chelsea,Middlesex,England;m.1849;d.1886)
2. Major James Mill Walker (b.1823-Montrose,Angus,Scotland;d.1890)
sp: Caroline Mary Tylden-Pattenson (b.1828-Biddenden,Kent,England;m.1858;d.1902)
2. Colonel Hercules Walker (b.1822;d.1895)
sp: Julia Decima Lethbridge (b.1843;m.1860;d.1900)
2. George Henry William Walker (b.1826-Levington,Suffolk,England;d.1893)
sp: Julia Elizabeth Twyford (b.1841-Trotton,Sussex,England;m.1858;d.1926)
2. William Walker (c.1828-St Mary le Tower,Ipswich,Suffolk,England;b.1828)
2. Thomas Walker (c.1828-St Mary le Tower,Ipswich,Suffolk,England;d.1854)
2. Captain William Francis Walker (b.1832;d.1913)
sp: Jane Gorringe (b.1841-Southwick,Sussex,England;m.1858)
2. Ronald Octavius Walker (b.1833)
I hope that this information is useful. If you have further details I would welcome sharing additional notes.