There are two James Grants in Oldham on the 1861 census:
James Grant age 8, b 1853 in Oldham, son of John and Bridget, living at 190 Barrack Street. John Grant is a Tailor, born Ireland.
James Grant age 69, born Ireland. Sadly an inmate at Oldham workhouse (on Union St? I think), occupation Chair Mender.
Any good?

Barbara