I think we may have more than one (unrelated) families of Livermore her. Although it appears to be a rare name, it is surprisingly common in Essex, Cambridgeshire, Suffolk and in London (particularly Enfield / Edmonton area) and some of the London ones were certainly were migrants from Essex.
Have we actually established where all the Joseph Livermores were actually born and when? And which of them are definitely connected to the OP.
We seem to have
Joseph Livermore 1799-1838 (born Cambridge died Yorkshire according to OP) - I have this one baptised in Barley in 1799 s/o of John Livermore and Elizabeth nee Ridgell. Barley is a village 4 miles SE of Royston. He married his first wife in Barley in 1820, she died in 1824 and is buried in Barley. He married his second wife Ann Freshwater in Melbourn on 31 Dec 1826. Melbourn is a village just over the Cambridgeshire border 3.5 NE of Royston. (I have spelt it correctly - there is no "e" on Melbourn, Cambs). This Joseph did not die in Yorkshire - His death is registered in Royston / Buntingford district in 1838 Q1 age at death 39. I have some information on his children - Joseph jnr (from 1st marriage) and Sarah Ann (from 2nd marriage)
Joseph Livermore b 1841 in London, married Matilda Peel 1870 and had children born in Hampshire - not researched this one - but he is possibly the one born in Marylebone s/o Daniel. As his wife was born in Corfu, and oldest child born Aldershot it is likely he was in the army, the is a Chelsea pensioner record for a Joseph of correct age born Marylebone and there is a Joseph b abt 1841 in Marylebone. His father Daniel is a cabman. There is a baptism of a Joseph to daniel Livermore and Isabella in the Workhouse chapel Marylebone on 14 May 1840 (dob 21 Jan 1840) which might fit. Can't find his birth reg to check mother's maiden name.
Joseph Livermore who married Matilda Stone in 1897. - Probably descendant of Sarah Ann (see above)
Are Joseph #1 and #3 connected to OP and Joseph #2 a red herring?