Just an update on this....
If I am right about Elizabeth Mountain being the one born in Lincolnshire, c.1860, then she married Joe Athey in 1879 and had a daughter, Edith Jane Athey. After they appear together on the 1881 Census, they are never together again. Edith is with her maternal grandparents in 1891, and then died in 1893, at the age of 13. Joe appears on the Census as a lodger with various families in 1891, 1901 and 1911, describing himself as "Married", and even filling in the 1911 bit, saying his present marriage has lasted 33 years. Unfortunately, he died in 1925, after falling over a wall above a river in Hooton Street, Kilnhurst, Yorkshire.
So, I may have had it right about bigamy - but David was the innocent party and it was his first marriage that was possibly bigamous, but on his wife's part, not his! Sorry, David!
If he married and later found all this out, no wonder he went off to join the Army.
I still can't find what happened to Elizabeth, or even find her on the 1891 Census, when she had possibly already left Joe Athey and gone to London.
I shall keep trying.....