Could this be a possibility for James
1841 census Piece: 972 Folio: 45 Page 32
Silver Street Taunton Somerset
Joseph POOLE 66 Cordwainer
JAMES 11William 9
Ruth 15
Ann 12
Maria 6
All noted as born in this county
Mary Poole Birth 11 Mar 1822
Father: Joseph Poole, shoemaker Mother: Mary
Baptism 30 Apr 1822 St Mary Magdalen, Somerset
Denomination: Presbyterian
Ruth Poole Birth 8 Sep 1824
Father: Joseph Poole, shoemaker Mother: Mary
Baptism 12 Nov 1824 St Mary Magdalen, Taunton Somerset
Denomination: Presbyterian
Joseph Poole Birth 27 Jan 1826
Father: Joseph Poole, shoemaker Mother: Mary
Baptism 13 Feb 1827 St Mary Magdalen, Taunton Somerset
Denomination: Presbyterian
Anne Poole Birth 28 Jul 1828
Father: Joseph Poole, shoemaker Mother: Mary
Baptism 14 Jan 1830 St Mary Magdalen Taunton, Somerset
Denomination: Presbyterian
1851 census Piece: 1922 Folio: 624 Page 8
Silver Street Taunton Somerset
Joseph POOLE 76 Shoemaker
Ruth 24
William 17 errand boy
Maria 15 dress maker
All born Taunton Somerset
Haven't found baptisms for James, William or Maria

from
England & Wales, Non-Conformist and Non-Parochial Registers, 1567-1970