Trying to trace when relation came back to Scotland

[Isobel]

I have a relation Richard Hogg b25/10/1819 Eskdalemuir Dumfriesshire.  He lived with his sister in Crawford Lanark and is on the census for 1851  then in 1855 him and his sister and daughter emigrated to Canada.  Richard came back some time because he is on the 1881 census for Selkirk.  How do I find out when he returned.