rayaco - I think Jeremiah emigrated to the USA.
As far as I can see there was only one Jeremiah Atkinson born in England between 1850 and 1860 - the Jeremiah you are looking for, born in the October/November/December quarter of 1852.
In the 1900 USA census for Oregon there is a Jeremiah Atkinson, who states he was born in England in October 1852. He states that he emigrated in 1880. He has a 'wife' Ann, also born in England, a son Earl aged 9 born in the USA and two stepsons, both born in the USA. He states that he and Ann have been married for 19 years (i.e. since 1881)
In the 1910 USA census they are still in Oregon: he states that they have been married for 24 years (i.e. since 1886) and that he emigrated to the USA in 1880.
The date of emigration would tie with the fact that Mary Atkinson is alone but says she is married in the 1881 English census.
If I have the right person and he did indeed actually marry Ann then you have a bigamist on your hands.