I think its a great find from Annette7, and I have been trying to follow it up.
Margaret Carpenter born Ashurst 1830 appears as MARGATE on my index, and in the same location is a Thomas Carpenter born Rotherfield, 1851. There are NO other Carpenters in Rotherfield so it suggests that Thomas could be another son of Margate and therefore James Carpenter's elder brother.
The SFHG index has James Carpenter baptised on 3 July 1853 to Margaret, but more tellingly in 1850, Thomas son of Margaret Carpenter of
Slab Castle was baptised in Rotherfield as Thomas
LATTER Carpenter. That suggests the couple only got married after they had achieved a couple of kids. So, even though it negates some my own previous findings, I offer sincere congratulations to Annette7 for finally cracking the problem.
The reason that Margaret does not appear on the 1861 census is that SFHG has her buried in Rotherfield as Margaretta Latter in 1856 (listed as Margretta on FreeBMD) and that was followed by John's death in 1858. So by 1861, Thomas and James can only be under the care of a relative, or in an institution for orphans. Perhaps finding John's origins and what surnames the two kids were under in 1861 is your next step. Hopefully the two were together making them marginally easier to find.
I do know that at zero years of age, Thomas Carpenter was listed as a lodger in the 1851 census, and he could have been with the same family that may have taken him back again with his younger brother. But there is another possibility. Page 58 of the History of Crowborough (
http://theweald.org/B10.asp?bookid=Crowbo058) states that Slab Castle, where the unmarried servant girl Margaret gave birth to Thomas Latter Carpenter in 1850, was not a prestigious residence as it suggests, it was another name for the Crowborough-Uckfield extension of the Rotherfield parish Workhouse. Your James may have entered there at 3 years old when his mother died or aged 5 when his father died.
Roy G