I had a look for William s/o Mary b 1789 to see if he could shed any light on brother James .
I think this is him , (wife Alice nee Pighills/Pickles)
(Listed males in household first)
1841 Census
North St , Keighley
William Butterfield 1791 - Tinner
Pickles -(male) 1821
Alice 1796 (wife of William)
Martha 1826
Mary 1829
Lydia 1831
Pickles Butterfield was bapt 1816 of William , Tinner, , and Alice. Abode Clubhouses
1851 Census
Temple St , Keighley
William b 1789 Keighley- tin plate worker
Alice - 1793 Keighley - wife
Lydia 1831 Keighley - dressmaker
1861
Temple Row
Keighley
William 1789 - tinner and brazier
Lydia Redman - 1831 - dau
Frederick Redman age 1 grandson
Also looked at burials . Do you have James burial ?
I think this is burial of James son of Mary
(I was in a rush and scribbled down numbers but might have got one of numbers bit wrong)
Burial at Christchurch , Oakworth (this ,if i remember right, is where Elijah his cousin is buried)
1865
James Butterfield of Lane Ends
Age 70 ? (Need to look at this again)
Unless you already have a burial , this fits with your James (did he come home ?) because we know your James is on the census and alive in at least 1851. But a problem is on the bmd index there are two James Butterfield deaths for that year in Keighley dist. If you do have a burial then this discounts James s/o Mary as being your James .
Something else I need to look into is a detail about one of Elijah's grandsons Joseph (son of my William 1816, my line) . In one of my old notes I have him giving "Kildwick" as his place of birth on a census . There is a further connection going toward that area at Addingham . William (Elijah's son) and family are on census there in 1861 probably working at New Mill. Also wonder if this is connected in some way with the Mary Removal Order 1767
Anyway, i am enjoying getting all the old Butterfield notes out and reading them
Will look into it again at online records tomorrow