1881: 34 Hill Street, lodging with Moses Thompson, age 31, born Glasgow
1871: 12 Weaver Street, age 20, born Glasgow, with widowed mother Mary and brother John, 35
1861: Weaver Street, age 10, born Glasgow, with parents, brother John, 15, and younger siblings Peter and Catherine.
1851: St Nicholas Place, aged 3 months, with parents and siblings Sarah, 8; John, 5; and Ann, 4, all born Islay.
John seems to have aged 20 years in the 10 years between the 1861 and 1871 census - might be a good idea to view the originals just to check.
For elimination purposes:
1851 Census, Islay: James Morrison, 26, with wife Mary, 25, and son William, 0, with Mary's parents Thomas Morrison and Isabella Gilchrist
1861 census, Kildalton, Islay: James Morrison, 35, with wife Ann, 30 and children William, 10; Janet, 8; Isabella, 6 and Mary, 4.
In 1871: James, 44; wife Ann, 39; children William, 20; Mary, 15; Ann, 9; Eliza, 7; Margaret, 5; Catherine, 3; Duncan, 1. The births index confirms that maiden surname the mother of the younger children was MacCuaig. James Morrison and Nancy McCuaig were married in Kildalton in 1861.
Can't find a death of James' son Archibald by his first wife Mary Morrison.
James Morrison died in Kildalton in 1871. His death certificate should confirm the name of his first wife.