Who did he marry? You should be able to find his marriage on freebmd.rootsweb.com - there is one in 1842 in Shoreditch, a possibility?
Paris is difficult; partly because many records pre-1860 were destroyed in the fire of 1871 (about 30% of civil registrations have been recovered from copies housed in other areas or reconstituted from other sources), partly because it's such a large place. It's like searching in London - only there's no IGI. The LDS hold civil registrations/church records, including some forty-odd churches with records that cover around 1820 and reconstructed records from 20 Arrondissements (areas of Paris).
In addition often 'Paris' might mean 'a town/village NEAR Paris', and 'Paris' is just what they say since everyone knows where it is. My family insisted one of my relatives was born at St-Gilles Croix-La-Vie, and only one family paper mentioned the fact that he was actually born at Givrand, a village nearby (with records held seperately). Without that, I might never have found him (his immigration records only gave his last residence, which was quite far from his birthplace).
It's not completely hopeless, though. The records you want may be at the PRO -
13. Aliens Act, 1836: Certificates of Aliens (HO 2), 1836-1852/
Aliens Act, 1836: Returns and Papers (HO 3), 1836-1869 records might hold him, if he came at around that time - I'm not sure how much info these records give, though. They're unlikely to list names of family, possibly will list place of origin.
http://www.geopatronyme.com/cgi-bin/carte/nomcarte.cgi?nom=Gomme&submit=Valider&client=cdip'Gomm' was most probably 'Gomme', if he was French. Otherwise, he may have been the son of a British family born overseas for some reason, as the name _GOMM_ does appear in English records fairly far back. Does he claim to be a British Subject in every census? (I can see he does in the 1881). Does it ever say 'naturalized British Subject?', or just 'B S' every time?