I've found a few more things that seem to be connected.
There was a marriage between Charles and Eliza but later than I was expecting. Charles Livermore married Eliza Charlotte Hilleary on 8 Nov 1874 at parish church St Marylebone, He was a tailor and widower. She was a spinster. His father: John Livermore - upholsterer and hers was Robert Hilleary - farmer. Winesses George Hawkins and L? Hawkins.
On 24 Feb 1822, a Charles Livermore married Charlotte Scott at St Martins in the Fields. witnesses Martha Aldridge and W? Taylor.
Baptism for children of Charles (a tailor) and Charlotte Livermore
Charlotte Emma 11 Dec 1822 Westminster Lying in hospital
Louisa 3 Dec 1826 Holborn St Giles in the Fields
Louisa 7 Oct 1827 (born 9 July 1827) St James Clerkenwell
(Entry below on same date is Henry Livermore s/o George - mattress maker and Mary)
Eliza, Julia and George all on 21 July 1839 Christ Church St Pancras. no dobs recorded for these three.
1841 census has Charlotte Livermore 40 Ind, Charles 20 Tailor J(ourneyman?), Eliza 12, Julia 10, George 9 and Isabella 3 at Clarence Garden, St Pancras. I can't find Charles jnr's baptism, but given his age he might be baptised under Scott. Isabella is a mystery. Her birth seems to be registered Q1 1838 St martins but was not included in the 1839 baptism.
A Charles Livermore was baptised at St Ann Blackfriars on 8 Sep 1803 to John and Mary. No occupation given but there are several children who are his possible siblings where some records say that John is an upholsterer. One possible brother is called George and his second wife is Mary Bennett. George is upholsterer.
I haven't found Charlotte and family on later censuses or found her death, which if it is all the famly would be just before Charles's second marriage in 1874.