Good Morning
As he's on the 1861 census as a 13 year old midshipman lodging 19 Park St,that does raise some questions.
Have you tried looking for his enlistment in the Royal Navy or merchant navy mates certificate etc?
There is a second mates certificate 26.1.1866 for a Charles William Sam Palmer age 18 from London-number 81571.To be honest I don't think this is him but if it is supposedly you can use other records to track down the boats he served on and his personal details using the certificate number at the Royal Maritime Museum/records Greenwich.
That young he may have been an orphan enrolled by Stepney Poor Law Union ,you could try and check that out.I don't want to step on any toes ,and this may not be the case,but have you considered trying Lee as a surname,could perhaps his parents have been unmarried,hence the 2 surnames?
Ciderdrinker