the James you found in 1881 is unmarried -as are his sisters!
this is a guess - but I wonder if james could be this james- as he lived in Kimberworth -and John James gregory was born Kimberworth
1851
Kimberwrth Common
Yorks
James Gregory head m 43 Engine tenter Hilltop Yorks
Sarah wife 42 Kimberworth
Thomas son 18 Iron Roller Kimberworth
Joseph son 16 Errand boy Kimberworth
Catherine daughter 14 Hill Top
Samuel sn 13 iron Roller Hill Top
JAMES son 10 Riddler at Coal pit Hill Top
Emma daughter 8 Hill Top
henry son 5 Hill Top
William son 2 Kimberworth
HO107 2344 191 19
problem is I cant find james after this

can find parents in 1871 -only son William is at home?

Suz