Hello Gillian, and welcome to RootsChat.
First of all, are you using
www.scotlandspeople.gov.uk? If not, see
https://www.rootschat.com/forum/index.php?topic=714261.0SP has a baptism record of Eliza Jarvie, daughter of George Jarvie and Margaret Clark, in 1823 not 1831. They also had James 1818, Philip 1821, and Margaret Paterson in 1827. Philip is an unusual given name, so it seems to imply that George Jervie could be the son of Philip Jervie and Jane Carnie, baptised in 1787.
In the 1841 census Philip Jervie, 19, is a seaman on the 'Tigris' of Leith, and Elizth Jervie, 16, is a servant in the household of Robert Carnie, fisherman. George, 50; Margaret, 45 and William Jervies, 13, are listed at Citadel Street, Leith.
In 1851 George Jarvie, 62, wife Margaret, 54 and daughter Jean, 38, are in Maitland Place, Leith and James Jervie, 32, fisherman, is in Parliament Square, Leith wife wife Isabella, daughter Isabella and two visitors, Margaret and Helen Goldie.
Neither George nor Margaret Sr is listed in Midlothian in the 1861 census (transcription). However unless they moved away from Midlothian I don't see a death of either after 1855.
I see that you already mention all these surnames, so it may be that you already have this information. If so, please tell us everything else you know about them to avoid anyone duplicating what you have already done.