I would think that the one who became the shepherd was the elder one. His age changes a lot but never to the extent that he could be mistaken age wise, for his own nephew.
There are 3 of them that I can find on the 1871 census. The one that I sent you yesterday, another the son of William and Sarah, born 1854 in Hertford Hunts, and then one born 1837 in Histon Cambs lodging in Kent.