It looks as if he might have fallen foul of 'the law', but I've not found a record of imprisonment:
Western Gazette - Friday 24 May 1872 TEMPLE CLOUD. Petty Sessions.— Jas. WYATT, of West Harptree, [inter alia] fined £5 for keeping dogs without a license. The last four cases were recommended to be reduced to £1 each.
— James WYATT, a carrier, of West Harptree, was sent to prison for 14 days, and afterwards to find sureties to keep the peace for six months, himself in £20 and two others in £10 each, or in default to be further imprisoned for six months, for assaulting his wife, Martha WYATT.
Western Gazette - Friday 06 June 1873 TEMPLE CLOUD Petty Sessions
Bevis COLES and Mark LOVELL, of East Harptree, and James WYATT, of West Harptree, were convicted of killing a hare on land at Chewton Mendip, on the 28th ult., and fined - Coles 30s, Lovell 14s 6d, and Wyatt 20s, and costs.
Whilst it's not currently possible to link all the petty crimes, it's worthy of note that in 1867 a James Wyatt, of West Harptree, was fined 15s and costs for obstructing the police in the execution of their duty, at West Harptree, on the 20th July.
Association with lead mining did not augur well for a man's mental health or longevity, so your man might have suffered an early demise in an institution or out of area if following the carrier occupation. A death was required to be recorded in the district where it occurred.